OCR for Grounding management

[Animatron56]

We want to implement a basic RAG use case with AI Core and grounding management. In our files we have a lot of image and chart. 

At the moment we can only generate a data repository with the Text Embedding 3 Large model without any OCR. 

Because of that all our images and charts are embedded very poor and the results of the model are not that great. 

Is there a way in document grounding to apply ocr for specifc files or is there another solution for this issue? 

[Animatron56]

Preprocess Files with OCR Before Grounding.

1.  Run OCR on images and scanned PDFs using:
   • SAP AI Core custom OCR pipeline
   • SAP Document Information Extraction
   • External OCR service (Azure OCR, AWS, Textract, Tesseract, etc.)
2. Extract the recognized text
3. Append or replace the original file content with the extracted text
4. Then generate embeddings using Text Embedding 3 Large
5. Store in your vector repository.

This ensures the embedding model works with real textual context.

[adilmd98]

The issue is arising from the capabilities of the Embedding model as they do work well with text tokens but not great with pixels which is the case for charts and images. Exploring more in depth related to Document Grounding would be less beneficial in your case.

What would help is to  preprocess your data before using an OCR which would then give you text. This text is what you want for your Embedding Models to produce great results. 

Hope that helps !

[Animatron56]

ok so I need to setup a pipeline before hand and parse all the images with their meaning into text and then put them into document grounding, there isnt currently a way to do all of that in the document grounding service? 

[meenakshignanakannan]

Preprocess Files with OCR Before Grounding.

1.  Run OCR on images and scanned PDFs using:
   • SAP AI Core custom OCR pipeline
   • SAP Document Information Extraction
   • External OCR service (Azure OCR, AWS, Textract, Tesseract, etc.)
2. Extract the recognized text
3. Append or replace the original file content with the extracted text
4. Then generate embeddings using Text Embedding 3 Large
5. Store in your vector repository.

This ensures the embedding model works with real textual context.