- SAP Managed Tags
Hi All
I had the pleasure of sharing some insights on AI journey for customers and partners at SAPTeched 2025 Berlin.
Let me share (the best I can in a BLOG) the content of my session.
I started by laying out that the presentation was based on a previous blog: The AI Stack Is Ready — Are You? The Challenge No Longer Is Technology . It is worth reading for context but I diggeed deeper in my TechEd session.
Let's dive in.
From a technology perspective, a GenAI (agent building) stack needs to provide the following capabilities:
- Flexibility of choice of LLMs
- Secure AI
- RAG scenarios
- Agent Building
- Tool supporting
- Orchestration
And we are there. Most platforms support that, at least in some way. That is true for SAP BTP and many others.
So we can consider that, the basic tools needed are provided. We can and should be able to start building.
Particularly around SAP, the latest TechEd announcements highlight the fast pace of innovation. More and more is coming.
MCP Support + ABAP LLM
RPT-1 and many other nice improvements
From a LLM perspective, the models keep getting better both in raw performance, Speed and cost.
As they keep getting better, a wrong approach would to say, the current models are not good enough or that we will decide to wait for next version. Believe me, you can get GPT6 performance today..... Let me explain.
Welcome TEST TIME COMPUTE
By applying some techniques that basically force the model to work harder at the question, will increase the performance.
An example of this is to ask for a brief answer vs compare that to an application that first determines a plan, proof it, execute it, revise and summarize the response. This is the principle of reasoning that are now more and more embedded in the SOTA (state of the art) LLMs, but you can keep pushing further....
SO, WHAT IS THE POINT MR. LEO???
Lets focus the discussion elsewhere. THE PROBLEM IS NO LONGER A TECHNICAL ONE.
Reality check
There are multiple reasons why that is the case. FIRST ONE is that the title of the article is misleading: It should be written 95% of the companies fail to demonstrate value. Sure a good chunk of that is due to use cases that delivered no or little value, true, but there is more to that story.
In short:
- Expectations are often not properly set;
- It is natural that we get a high number if vailures - We are all experimenting and in this space, fail fast is the norm
- There often a suboptimal approach (more on that later)
- AND YES, it is hard to demonstrate value (methodology and tooling)
Lets set expectations right on very common misconceptions:
In ABAP you write 3 * 3 and the result will ALWAYS be 9 - This is deterministic
In GenAI (like with humans) it is probabilistic. The answer is 9 with 99.9999% prob. In other words, ask 5 billion people the same question and you won't get 100% either.
Garbage in = garbage out! This could not be more true in GenAI. If you data, assumptions or instructions are poor, expect S**t
And how many times I've heard:
Bus:"Lets put AI to solve the problem"
Tech:"describe me the problem?"
Bus: "AI should"figure it out"
If you can't even explain the problem you are trying to solve it will be impossible to solve it.
Also, AI is not band-aid
Putting AI on a messy process will get you, well, a messy process with AI...
NOW, let focus on the IDEATION problem....
There is a reason why EVERY SINGLE VENDOR out there is after the "business cases". Good, valid, appealing and above all universal use cases are HARD to come by.
The nature of the game is, try often, fail fast, big return will come over time
In that process of getting your feet wet, you will try and mature your AI budgeting process.
Also, from a development perspective, as it is highly volatile, we recommend and approach from IDEA to POC to MVP to eventually PRODUT. Stepping stones.
In the ideation process, you need to evaluate the ideas in FEASIBILITY and VALUE. Ideally the Easiest and that will drive most business value will come to the top.
And how do you get there?
Representation = You need tech folks who know the tech capabilities WITH the business folks who know the current business challenges.
Approach = BOTTOM UP approach is one where the ideation is around ideas to improve current processes while a BOTTOM UP approach is one where we re-think the process entirely with the new possibilities this technology brings
Also, effective enterprises adopt AI in 3 strategic levels
While company sets corporate wide transformational initiatives, departments should have their own plans and internal investments complemented by individuals trained and empowered to drive innovation in their own day to day.
The importance of OCM - Organizational Change Management
Lets talk beyond the BUILD phase
Cost reporting
You have 75 agents, each calling several LLMs cycles called by 2500 users. How much is it costing? who should pay for it? What is the ROI?
Helping value realization
You can only sucessfully demonstrate value if you determine a KPI, set a target and prove progress.
First determine a goal or a measurable outcome, establish a target and put in place what is required to measure it.
Only then, you will be able to demonstrate return on your investment (or not).
Understand now why 95% AI pilots fail ? 😉
Bluesky| User | Count |
|---|---|
| 33 | |
| 28 | |
| 24 | |
| 14 | |
| 13 | |
| 12 | |
| 11 | |
| 11 | |
| 9 | |
| 8 |
