The LLM Is Not Your Analytics Engine
Anwer Sadath
AUTHORLast week, I was discussing an AI solution with a financial institution that had accumulated years of data across operational systems, investment applications, reporting platforms, and databases. They already believed AI could help them make better use of that information. The real discussion was about how the solution should work.
At one point, someone asked, “Does this mean we have to give all of our data to the AI?” My first thought was that this was mainly a privacy concern. That would have been understandable for a financial institution handling sensitive information. But as the discussion continued, I realized the concern was also architectural. They assumed that an LLM would need access to most of their enterprise data before it could answer intelligent questions.
Even with the model running on an on-premises GPU server inside their own data centre, they imagined large volumes of data continuously moving from multiple systems into the LLM. We started discussing the practical impact of that approach, including context-window limitations, hallucinations, processing overhead, infrastructure costs, and the complexity of constantly transferring data between systems. More importantly, an LLM reasoning directly over large volumes of raw data could produce answers that sounded convincing without consistently applying the organization’s business rules, calculations, or data relationships.
The more we talked, the more uncomfortable the architecture became. Then one question completely changed the direction of the conversation:
Why does the LLM need all that data in the first place?
Enterprise data already lives where it belongs. It sits inside secured databases and business applications, protected by access controls, permissions, governance, and years of carefully implemented business rules. Moving large volumes of that data into an LLM, even within the organization’s own environment, felt like we were solving the wrong problem.
So we turned the architecture around. Instead of moving the data towards the AI, we started thinking about how to move the intelligence closer to the data.
The enterprise systems would continue to own and protect the information. An analytics engine would work close to those systems, retrieve only the data required for a specific question, apply the relevant business rules, perform calculations, and produce a meaningful analytical result. Only after that work was completed would the LLM become involved.
The LLM would not be responsible for analysing millions of records or understanding every schema, relationship, and calculation inside the organization. Its role would be to understand what the user was asking and present the analytical result in a natural, conversational form.
That distinction became the most important architectural decision in the solution.
Once we separated those responsibilities, many of the original concerns became easier to address. The sensitive data remained inside the systems designed to protect it. The LLM did not need direct access to the databases. Context-window limitations became less important, inference costs were reduced, and the risk of hallucinated analytical conclusions was lowered because the model received verified results rather than being asked to calculate directly from raw enterprise data.
What made the discussion particularly interesting for me was that this was not a new idea we had arrived at during the meeting. It was the same architectural principle we had used when building DataLens Engine last year.
DataLens Engine was designed to sit between enterprise systems and the LLM. It retrieves only the information required, performs the analytical work, applies the relevant rules and calculations, and passes only the necessary context to a locally hosted language model. In this design, the LLM becomes the language layer, not the analytics layer.
Last week’s conversation reminded me why that separation matters so much. As organizations rush to add AI to existing systems, it is easy to assume that the model should sit at the centre of everything. But an LLM does not need to become the database, the business-rules engine, and the analytics platform at the same time.
The Analytics Engine Understands the Data. The LLM Understands the User.
Once we separated those responsibilities, many of the original concerns became easier to address. The sensitive data remained inside the systems designed to protect it. The LLM did not need direct access to the databases. Context-window limitations became less important, inference costs were reduced, and the risk of hallucinated analytical conclusions was lowered because the model received verified results rather than being asked to calculate directly from raw enterprise data.
What made the discussion particularly interesting for me was that this was not a new idea we had arrived at during the meeting. It was the same architectural principle we had used when building DataLens Engine last year.
DataLens Engine was designed to sit between enterprise systems and the LLM. It retrieves only the information required, performs the analytical work, applies the relevant rules and calculations, and passes only the necessary context to a locally hosted language model. In this design, the LLM becomes the language layer, not the analytics layer.
Last week’s conversation reminded me why that separation matters so much. As organizations rush to add AI to existing systems, it is easy to assume that the model should sit at the centre of everything. But an LLM does not need to become the database, the business-rules engine, and the analytics platform at the same time.
Instead of asking, “How do we connect our data to AI?"
Perhaps we should first ask a more fundamental question: “Why does AI need direct access to all of our data in the first place?”
The answer may be simple: It does not.
We should not move data to AI. We should move AI closer to the data.
Ready to transform your business?
Join hundreds of companies already growing with TA. Start your journey today.