topic Re: May Developer Challenge - SAP AI Services in Application Development and Automation Discussions

May Developer Challenge - SAP AI Services

noravonthenen — Mon, 24 Jun 2024 11:58:51 GMT

!!! THIS CHALLENGE IS CLOSED !!!

Welcome to week 1 of the May Developer Challenge on AI at SAP! The topic of this month’s challenge are the SAP AI Services; Document Information Extraction and Data Attribute Recommendation. To participate in the challenge you just have to post a screenshot of your solution as a reply in this discussion of the corresponding week.

SAP AI Services help you implement custom use cases by providing powerful algorithms specifically tailored to business problems.

Document Information Extraction:

The Document Information Extraction service is available in two editions, the original Base Edition and the new genAI-based Premium Edition. The genAI-based Premium edition is using a large language model via generative AI hub on SAP AI Core to extract information from all kinds of documents.
With Document Information Extraction you can extract information from the file types PDF or single page JPEG, PNG and TIFF.
Supported document types are: invoice, paymentAdvice, purchaseOrder, businessCard, deliveryNote, resume and birthCertificate. You can also create your own schema to process other document types.
You can also extract OCR results directly to process the raw text from you document files as well as use the classification capabilities to classify your documents into the three classes: invoice, purchase order and payment advice.
You can also enrich your extracted data with your metadata.
You can access the Document Information Extraction service via the UI, via swagger/client calls and the Python SDK.

Data Attribute Recommendation:

With Data Attribute Recommendation you can train your own model to classify data records, you can also tackle more complex classification problems such as hierarchical classification of products and predict missing data records
Data Attribute Recommendation can be used via swagger/client calls as well as the AI API Python SDK and SAP AI Launchpad
If you want to access Data Attribute Recommendations via Postman you can download this Postman Collection

Weekly Challenges

Week 1 Challenge – DOX UI

This week you will use the UI of the Document Information Extraction service to extract information from your favorite recipe. The UI is great to try out your use case and get a feeling of the capabilities of the service. For productive use cases you would call the APIs or implement a workflow using the Python SDK. Productively, you could then for example implement a workflow that processes documents right out of your mailbox, saves the extracted information in the system and structure you need as well as triggers other necessary workflows.

For this week’s challenge, use the UI to extract the header fields “recipe name”, “portions” and the line items “quantity” and “ingredient” from your chosen recipe. Therefore, you need to create a custom schema. Make sure the recipe is in one of the supported languages.

When creating a custom Schema chose the Setup Type auto to use the llm/genAI-based Premium Edition. In the description field provide information for the large language model to understand what you are referring to e.g. “the name of the recipe”.

Get a free trial account and run DOX booster: https://developers.sap.com/tutorials/cp-aibus-dox-booster-key.html
Get the Document Information Extraction UI: https://developers.sap.com/tutorials/cp-aibus-dox-ui-sub.html
Create a custom schema: https://developers.sap.com/tutorials/cp-aibus-dox-ui-gen-ai.html
OPTIONAL: Create a template and add your document to the template (improves performance for future recipes)
Upload your favorite recipe to extract the name, portions, quantity and ingredients. Make sure your recipe pdf is only 1 or 2 pages long, otherwise you will quickly reach the limit (50 pages) of the trial plan. And try not to use the entire 50 page quota because we will need it next week as well!
Submission: share a screenshot of the extraction results and the document and write a comment to share your experience using the UI in the discussion below.

Example Screenshot:

Additional information:

Processing a ©Pokémon Card in 90 seconds with Document Information Extraction powered by generative AI: https://community.sap.com/t5/technology-blogs-by-sap/processing-a-pok%C3%A9mon-card-in-90-seconds-with-document-information/ba-p/13571759

Be aware of limits that apply in free tier and trial accounts: https://help.sap.com/docs/document-information-extraction/document-information-extraction/free-tier-option-and-trial-account-technical-constraints

How to improve your results: https://help.sap.com/docs/document-information-extraction/document-information-extraction/best-practices-298a9a0936d5436494c644ec51bbdcea

In this “2-min of” video I am describing the technical aspects of the BASE service (without use of LLM) behind the scenes.

Re: May Developer Challenge - SAP AI Services

geek61 — Wed, 01 May 2024 15:58:27 GMT

Some positive results:

Some less so:

Re: May Developer Challenge - SAP AI Services

PieterB — Wed, 01 May 2024 16:45:05 GMT

Here my result

Not yet a 100% correct result, but looking forward to the next challenges to learn more about the AI services

Re: May Developer Challenge - SAP AI Services

satya-dev — Thu, 02 May 2024 06:57:27 GMT

Read restaurant name and address from image

Re: May Developer Challenge - SAP AI Services

M-K — Thu, 02 May 2024 15:10:47 GMT

Here is my result:

Interestingly some of the ingredients were highlighted in the instructional text and not in the list, however they were all correct.

Re: May Developer Challenge - SAP AI Services

Alpesa1990 — Thu, 02 May 2024 16:18:33 GMT

My submission.

In my case (Spanish language), the IA doesn´t could separate the quantity and the ingredients... But it´s so close...

Re: May Developer Challenge - SAP AI Services

IanStubbings — Thu, 02 May 2024 20:16:50 GMT

My recipe. All good.

Re: May Developer Challenge - SAP AI Services

jasperdebie — Fri, 03 May 2024 09:55:43 GMT

Interesting to see how it extracts data with the minimum of information:

Some small mistakes like the quantity not separately placed in the quantity field but merged in the ingredients field, even after changing the type of Quantity. Highlights almost fully correct.

Re: May Developer Challenge - SAP AI Services

Ruthiel — Fri, 03 May 2024 15:02:26 GMT

Hello @noravonthenen!

Thanks for this wonderful content!

I am mesmerised by this tool and the results of it!

The unit on the time-related field surprised me since I had minutes and hours in the recipe however, all the units were correctly inserted in minutes.
I could distinguish the main ingredients and the quantities independently of the unit of measure for each line item!

Re: May Developer Challenge - SAP AI Services

kishore_kumar_g — Fri, 03 May 2024 17:14:19 GMT

Hi @noravonthenen,

Thank you for the excellent content, it's truly impressive!

Re: May Developer Challenge - SAP AI Services

johna69 — Fri, 03 May 2024 20:03:00 GMT

Is it May, Mai or M-AI challenge 😉

Nearly right:

Re: May Developer Challenge - SAP AI Services

narendran_nv — Sat, 04 May 2024 02:48:13 GMT

Not bad though, in the first attempt it wasn't able to identify any of the ingredients from the document. But I tried to mark them explicitly (only for the first 5), then on my second run of the same document it tried to map those exact same lines.

Re: May Developer Challenge - SAP AI Services

gphadnis2000 — Sat, 04 May 2024 06:25:41 GMT

Interesting how Document Information extraction reads data with minimum efforts.

Re: May Developer Challenge - SAP AI Services

thomas_mller13 — Sat, 04 May 2024 08:00:19 GMT

Is this AI service using LayoutLM algorithms? - In the context of a specific business application as e.g. incoming invoices or delivery notes for a single company a large language model is maybe a sort of overkill, since there is so much more specific information available about these documents and these documents are contained in a very small subset of all documents? A lot of specific informatin is not used. What AI model would you suggest in such a case?

Re: May Developer Challenge - SAP AI Services

Sabarinathan_m — Sat, 04 May 2024 12:54:46 GMT

Hi,

If the document / image has text in it, then the data is extracting. That is working fine.
Whether the ingredient or quantity wont be determined from the picture which doesn't have any text?

Thanks

Re: May Developer Challenge - SAP AI Services

moh_ali_square — Mon, 06 May 2024 07:00:59 GMT

Hi,

I got nice results. title of the recipe and the ingredients.

Re: May Developer Challenge - SAP AI Services

noravonthenen — Mon, 06 May 2024 07:05:02 GMT

Hi @Sabarinathan_m, Yes the service we are using is for extracting text from documents (pdf or images) and identifying what the test is. So title and ingredients in our example. In business context that could be order number or customer or phone number, email and address, line items, total amount or currency and so on. Therefore, feeding only an image without text does not work with this service. What you are suggesting would be an image recognition and object detection task.

Re: May Developer Challenge - SAP AI Services

noravonthenen — Mon, 06 May 2024 07:09:34 GMT

Hi @thomas_mller13, no this service does not use LayoutLM but there are other algorithms based on layout that are being used. Here is a description of the underlying algorithms of the base edition: this “2-min of” video. The premium edition uses GPT in the background to determine all kinds of other values.

Re: May Developer Challenge - SAP AI Services

noravonthenen — Mon, 06 May 2024 07:10:34 GMT

@johna69 LOVE the M-AI challenge comment 😄

Re: May Developer Challenge - SAP AI Services

Venkat_Vyza — Mon, 06 May 2024 20:40:30 GMT

Thank you for starting this AI Challenge @noravonthenen