記事

From (server) Farm to (model) Fork: the Importance of Trusted Data

A healthy AI requires quality data, transparency, and human oversight, similar to how nutritious food benefits our health. Trust and governance are essential.

Simon Axon
Simon Axon
2024年8月30日 4 分で読める

Nutritionists have been advising that ‘you are what you eat’ for decades. A healthy diet full of nutritious products promotes good physical and mental health which leads to better outcomes for individuals, communities, and societies. The same could be said for Artificial Intelligence. AI must be fed good quality data both whilst being trained and in inference phase in deployment. Today we talk about farm to fork provenance for the food that we eat. It's relatively easy to do for simple products like fruit and vegetables, but harder for complex products with many ingredients. The same will be true for AI. Quite straightforward for basic reports, but much harder for AI that has ingested potentially millions of data points. It’s important to create the right data environment in which you have the governance, quality, and management controls to precisely define the provenance of every data item consumed by your AI now.

A healthy data diet

It is not always easy to know why an AI model delivered a particular decision or outcome, but knowing what data it used to make it can help. Effective deployment of AI at scale requires trust. Seeing consistent connections between data input and AI output, and experience that demonstrates that the data relied upon does deliver correct, accurate and explainable AI-driven outcomes, builds that trust. So, just as knowing the provenance of your food helps build the confidence that it is good for you, transparency, auditability, and good governance around data are essential prerequisites for trusted AI.  Data is data to an AI; it will train on, use, and provide outputs with whatever data it is fed. Ensuring that the data it consumes is fresh, accurate and relevant is as important as a healthy diet is to the development of a child. Making sure that happens requires people, data, and AI that work together transparently to deliver value.

Humans at the heart

Keeping control at the heart of AI technology and processes is essential. Afterall, today’s AI is the equivalent of a toddler. You wouldn’t leave a small child to eat what it liked and make its own decisions! Engaged and accountable individuals have an essential role in carefully selecting, preparing, and monitoring what data is used to train and then deploy AI. Only humans can ensure that security, privacy, compliance, ethical, and sustainability requirements are met and upheld. People in the loop prevent bias and other potentially harmful effects from creeping into AI training and decision-making. 
Seamless, integrated and harmonized data access across an organisation ensures that these critical human roles are not a ‘heavy-lift’. Automatable, repeatable and reusable components can be developed and used to empower people to perform these essential tasks efficiently without compromising quality. In regulated industries like financial services the technical and governance systems that support people in maintaining high quality data are essential for the auditability of AI-based decisions.  

Transparency

The ability to use trustable and reliably accurate data as the foundation of governance and reporting has always been important for financial services firms. The AI revolution currently underway only amplifies the importance of these capabilities. Transparency is fundamental to trusted AI – for regulators, enterprises, and their customers. Organizations must be able to provide visibility of the data behind AI decisions. 
Regulators, faced with the ‘black box’ problem of AI models, will demand to know what data was used in training, and to make specific decisions. They will want to see not only how specific models used specific data to make a decision, but that both model and data complied with all relevant regulations. For any user of AI, transparency into the data behind the model should make it clear that a decision was fair and equitable, even if unexpected. For all these reasons it is essential that every data source used for training or inference is validated before any AI implementation takes place.

An open and connected data ecosystem

The open and connected data ecosystem which provides the foundation for the human-centric deployment of AI and the necessary transparency, will also provide the flexibility to support experimentation and innovation. To be accepted, and to capitalize on the promise of AI to deliver what McKinsey estimates to be a $25.6 trillion opportunity, businesses need to be confident in the performance and outcomes of AI. Financial services, more so than many sectors, need to demonstrate to regulators that AI-based decisions are fair and non-exclusionary.
Experimentation and innovation are vital for financial services competing not only with each other but with new entrants from other sectors including the big tech players currently driving the AI wave. But, to create value, successful AI experiments need to scale. That means banks potentially deploying thousands of AI models to service millions of customers using billions of data points from thousands of sources. The complexities of managing many models and huge volumes of data with confidence demand that the fundamental building blocks of data harmonization, scalability, and security across an enterprise-wide data platform are all in place. 

Experience in delivering quality data

Not only has Teradata been providing exactly this to a large number of businesses for decades, but it is now engaging with regulators to explore ways in which they can use data platforms and AI to better regulate AI. They will utilize the ‘farm to fork’ ethos and seek deep visibility into the data supply chains that ultimately define whether an AI is healthy, and makes good, trustable and auditable decisions or not. Speak to us to find out how we can help improve your AI’s diet so you can scale to value and meet the demands of regulators with confidence. 

Tags

Simon Axon について

Simon Axon leads the Financial Services Industry Strategy & Business Value Engineering practices across EMEA and APJ. His role is to help our customers drive more commercial value from their data by understanding the impact of integrated data and advanced analytics. Prior to his current role, Simon led the Data Science, Business Analysis, and Industry Consultancy practices in the UK and Ireland, applying his diverse experience across multiple industries to understand customers' needs and identify opportunities to leverage data and analytics to achieve high-impact business outcomes. Before joining Teradata in 2015, Simon worked for Sainsbury's and CACI Limited.

Simon Axonの投稿一覧はこちら

最新情報をお受け取りください

メールアドレスをご登録ください。ブログの最新情報をお届けします。



テラデータはソリューションやセミナーに関する最新情報をメールにてご案内する場合があります。 なお、お送りするメールにあるリンクからいつでも配信停止できます。 以上をご理解・ご同意いただける場合には「はい」を選択ください。

テラデータはお客様の個人情報を、Teradata Global Privacy Statementに従って適切に管理します。