Why Solid Data Foundations Matter More Than Flashy AI

[noravonthenen]

From all the conversations I have been having the past years since the release of ChatGPT, AI is totally underhyped and overhyped at the same time. From doomsday speculations to AI being useless, I have heard it all. We all have heard it all. But what I am missing in all these discussions is differentiation. It is the simultaneity of these things.

While AI might possibly turn on us one day, we should already now be looking at the risks and some of which have been there way before the release of ChatGPT. AI algorithms steering public opinion through social media, AI already in the hands of people with questionable morals and in the case of business AI we need to watch out for hallucinations, AI systems with unreliable data foundations, and biases in AI that can cause harm especially when human data is involved.

While AI might not solve all our problems today, it is far from useless. We just need to understand how it works, adjust our expectations accordingly and give it the necessary tools and information to get the results we need.

Unlocking AI’s Potential Starts with Rethinking Your Data

One of the biggest problems that companies have today is finding good use cases that lead to business value and getting their company ready for the AI age. We can build the best AI agents and retrieval augmented generation (RAG) workflows, if we do not have our data available with the logic, key metrics and relationships intact, then it is extremely difficult for AI agents to make good use of it.

We also must start thinking about our data differently. AI agents might need data differently than the data we already store in our systems. We might need to think about additional fields in our datasets to help AI make decisions the way we do. We need to capture more sentiment, give more explanations as to why we make certain decisions.

We expect magic from systems built on disorganized data, yet we’re often unwilling to invest the time and resources needed to structure and clean that data properly. This disconnect leads to disappointment when AI fails to deliver, not because the technology lacks potential, but because its foundation is shaky.

Imagine how powerful an AI agent could be when it has access to your relational data, understand your business data and has access to powerful tools such as SAP-RPT-1, data that is spread over documents and internal wikis, models such as perplexity or custom machine learning models. Such an agent could seamlessly analyze complex datasets, draw actionable insights, automate decision-making processes, and deliver tailored recommendations in real time, all while continuously learning from new information across your organization.

Do we need AI to be boring again?

The truth is: many AI agents being built today are reaching too high. While everyone loves flashy AI use cases right now and stages are full of autonomously moving browser windows, voice calls to AI agents and robots sorting laundry, the real power of business AI lies in the processes that work quietly behind the scenes.

I still hear of countless processes where people map data from A to B and manually copy cells between Excel sheets. These are the quiet inefficiencies that drain thousands of hours where AI can create immediate, meaningful impact.

But none of this works without the right data foundation. Newer models might hallucinate less, but “less” is often not enough for businesses. What we need is often not more data but the right data. Context‑rich, structured, connected, and reflective of how decisions are actually made within the company.

Another mistake I often see and why I believe AI needs to be a little more boring again is that, everyone wants to build AI agents now but we should stop firing cannons at sparrows. Not every problem needs an AI agent. Some require classical ML or simple RAG workflows and some require no AI at all.

To achieve this companies must upskill alongside their employees, not only technically but also ethically. As developers working with AI today we get to shape the future and how AI will be used going forward. Let’s make sure, we do that wisely.

To summarize: agents only work when they have reliable, consolidated information. AI use cases are only as successful as the data fabric that feeds them. This is exactly why we at SAP believe in the power of SAP Business Data Cloud.

So what does it take to build a data fabric that is robust, reliable, and rich with business context?

For developers tasked with shaping the future of AI, this is the critical question. To find answers, join us for the Fabric of Data & AI event. You’ll learn directly from industry leaders how a business data fabric connects different data sources, metadata, and processes to keep AI grounded in real business context. This is your opportunity to learn how to build the solid foundations that turn AI’s potential into actual business outcomes.

The Fabric of Data & AI virtual event on March 24: https://url.sap/vlm80u