o1 coding in ABAP

[L_Skorwider]

Dear Community,

I tested o1-preview this weekend. I tested how it handles ABAP programming. It is presented as great at coding, but we all know that ABAP is specific. Although GPT 4o was already doing quite well. Will it do better this time? Is the chain of thought a revolution or not?

By the way, I'm curious to know how many of you are using LLM as a programming assistant, or if it's not that level yet. I honestly have my thoughts, but I wouldn't want to perhaps get ahead of the comments.

I invite you to watch.

[MarioDeFelipe]

I had fun with the video, looks promising

[L_Skorwider]

I think we'll be using AI for programming more and more. At Google, already 25% of the code is being created with the help of artificial intelligence. However, let's not kid ourselves; for now, programming in ABAP is only going so-so for the models. Will it get better? We'll see. Maybe not in the immediate future, but sooner or later, they'll master this skill perfectly.

[Cocquerel]

Can we expect the new SAP ABAP LLM that was announced in techEd to be better ?

[L_Skorwider]

Hard to say without touching this in practice. 150M lines of codes for training sounds promising, but effectiveness depends also on training techniques. I would like to test this one day.

For code explanation we probably don't need more than we have in current LLMs. They are great in this task.

[Devraj]

SAP has fine tuned a model but it has taken far too long. In between we have had 3 new generations of LLMs that have been released. I do hope it is better than what we are getting but SAP will have to constantly keep updating/ changing models to keep up with the fast changing landscape.

[L_Skorwider]

And it will remain this way until a more effective form of learning than fine-tuning is developed... 

[MarioDeFelipe]

that said, the true value of a coding LLM in a complex scenario like SAP is the support for an specific landscape.

If one landscape is on version ABAP Netweaver 750 SPXX + other SAP propietary versions (Java or others) it can't recommend the latest and greatest advancements in ABAP, RAP or whatever if that particular landscape does not use it, also if there are millions of code in Z, it must provide answers with those recommendations. if not, , its a partial tool.

Once SAP releases an LLM trained on its own languages, it must be able to fine tune and pre train.

[DJ_ISU]

To answer your question from the initial post. We trained an LLM Model with our development guidelines, naming convention and some further information as an ABAP Assistant. It's in the moment not more, than an Assistant but it's already helpful in the daily tasks. Especially for the functional teammember, so they can together with the ABAP assistant do some development tasks already independently.