In my last article, I reflected on what “AI at scale” really means for B2C industries in general and in financial services in particular. Let’s double-click on some of those ideas.
Just about all the real-world AI use cases being deployed in large enterprises require multiple structured datasets to be integrated and then combined with at least one vectorized representation of an unstructured dataset. (Here is a recipe for practitioners that describes a design pattern that I recommend.) Because of that, my hunch is that five years from now, every major bank is going to need to vectorize every customer interaction (email, chat, call recording, etc.) and all of its internal policy documents, potentially with several different embedding models and chunking strategies optimized for different requirements. If I’m right about that, there are billions of vector embeddings in your institution’s near future, and you better have a robust data management strategy to deal with them. And because AI doesn’t change the laws of physics, there will be a very good case in large organizations for co-locating what we might call the Enterprise Vector Store with the enterprise data warehouse, or data lakehouse. More on that in a moment.
Now let’s think through another implication of deploying AI agents widely in large enterprises.
Teradata is working with healthcare companies that envision a not-too-distant future in which there will be an agent for every registered patient. This agent will work quietly in the background, monitoring the patient’s health and fitness data and predicting likely medical issues based on the incidence of disease in similar groups. It will nudge the patient to make lifestyle changes and alert physicians when health indicators move in the wrong direction or new treatments become available.
My personal healthcare agent is going to have questions. Lots of them. It will need to know about my current health and fitness and what the data say about how they are changing over time. It will ask how that compares with my cohort, and what that says about my likely future. And it will want to consider new research, treatments, and interventions and evaluate their success rates for similar groups.
In a large healthcare system, this will require tens of millions of agents running thousands of queries against back-end data platforms—continuously. Some of them might be relatively nonurgent but time-critical (“When is Martin’s next routine checkup, and when was the last time we talked to him about prostate cancer risks among patients over 50?”). Other queries might be more complicated (“Update Martin’s risk score for myocardial infarction and predict how that risk score changes if he is prescribed statins”). That’s why AI experts are so excited about Model Context Protocol (MCP): it enables us to create tools that intelligent agents can discover and use to query back-end platforms to get answers.
In these hyper-personalization scenarios, it’s easy to imagine a tenfold—or greater—increase in query volumes to back-end systems. I can envision a similar scenario in every B2C industry vertical. For example, why wouldn’t my bank want to offer me personalized, continuously updated savings and investment advice aligned with my retirement goals? Enterprise data warehouses and data lakehouses are going to become “enterprise context stores” or “intelligent agent frameworks,” if you will. Most users aren’t going to be carbon-based lifeforms; they’re going to be AI agents searching for the context they need to make intelligent, personalized recommendations—and ultimately, decisions. And since agents don’t sleep, we should not only expect that query workloads will increase sharply, but also that those workloads will be running continuously.
So now let’s ask ourselves: What happens to IT costs if organizations must support a 100x increase in query volumes within the next five years? These aren’t outrageous projections for large enterprises that want to use agentic AI to provide hyper-personalization to millions of customers and transform complex business processes in the AI era.
Clearly, enterprises will need to be very intentional about the architectures they deploy and the technologies they use if they hope to successfully move these visions from the innovation lab into full production. Given that 80% of the data in large enterprises is unstructured, vectorizing essentially all of that data is already a significant undertaking. But volunteering to do that over and over, which is the situation enterprises are likely to find themselves in if their technology leaders allow the current generation of AI prototypes to become “accidental architecture,” is likely to be ruinously expensive.
As other commentators have pointed out, the widespread deployment of these sorts of solutions has important implications for data security and privacy, which will mean that companies operating in the regulated industries will need to place a high premium on AI governance, but that is another discussion for another time. In fact, deploying fully automated solutions requires us to integrate and orchestrate multiple components, which is why we are developing our “intelligence framework” – and more on that soon.
I hope you've found this blog post helpful. If you have any questions, feel free to send me a message on LinkedIn.