Anyone using AI coding tools for ABAP?

[henryleeflow]

I've been experimenting with a small harness for AI coding agents to see if I can make them better at ABAP.

On my current eval tests i got these accuracy gains:

Claude Fable 5    92.2% -> 97.7%
MiniMax M3        88.5% -> 93.5%

I'm curious how people are actually using AI with ABAP today.

• Is anyone using Claude Code, Codex, Copilot, etc. for ABAP?
• In real SAP environments, can you normally use cloud models, or does everything need to stay on-prem?
• Would something like this be more useful as a skill/plugin for an existing coding agent, or as its own ABAP agent?
• Is there a natural place in the ABAP workflow/tooling where something like this should hook in?

I'm thinking about putting the ABAP-specific parts on GitHub if they're useful. Mostly looking for advice from people who work with ABAP regularly.

[Yrfo]

@henryleeflow I use (or still experiment would be more correct 😄 ) Claude (via CLI or VS code extension) + https://github.com/arc-mcp/arc-1 MCP to connect to SAP system. Looks promising and interesting, but definitely takes some time to step by step progress in that direction, to be careful and make some safeguards.

[AlexandrePassarelli · Accepted solution]

Hi,

I use Claude Code every day for ABAP, connected to the SAP systems through an MCP server that talks to ADT (the same APIs Eclipse uses). Some things I learned on the way, answering your points:

• What makes the difference is not the model, it's the loop with the real system. The agent writes, runs the syntax check, activates, runs ABAP Unit and ATC, reads the result and fixes. My rule for "done" is the object active and the tests read, not the code looking right. Without that loop even a good model invents a field or a method signature now and then.
• Read the result, not the status. A test run or an ATC run can finish "successfully" with failures inside. The agent has to read the outcome, otherwise it reports green on a red result.
• Cloud or on-prem depends on the customer. What made it acceptable where I work: read-only by default, every write only with explicit approval, and treating everything that comes back from the system (code, texts, table content) as data, never as instructions to the agent.
• Where to hook in: I'd go with tools (MCP) plus skills, instead of a separate ABAP agent. The tools give any agent the system access and the checks; the skills carry the method (naming, Clean ABAP, how to test). That way you keep the agent people already use.

If you put the ABAP-specific parts on GitHub, the eval set alone would already be useful. Most of us only have impressions, not numbers.

[henryleeflow]

this makes sense to me, and just out of curiosity how is your experience building ABAP solutions using Claude Code have been? My eval shows 92% accuracy on Anthropic only when you use the flagship models, so I would imagine you'd be seeing some feature bugs at least every 3~4 prompts you do. My eval for Fable on Javascript was close to 97% accuracy, and so Javascript engineers don't have to worry about hallucinations as much

[AlexandrePassarelli]

Hi Henry,

In my daily work the result is much better than the raw number suggests. A single-shot eval measures the model alone. In real work the model never delivers its first attempt: the harness around it (the MCP against the real system, syntax check, activation, ABAP Unit, ATC, plus project context and conventions) catches most of those 8% before anyone sees the code.

In practice our accuracy is very high. Quite often the code is tested and approved by the whole team with no further adjustments, starting only from the specification and some conversations with the functional consultant. I don’t write any ABAP myself anymore. My part is review, architecture and making sure the requirement was understood correctly.