SAP AI Services - DOX - Gen AI with SAP CAP Plugin - Part 3

[sabarna17]

After Part1 & Part2, What's in Part 3? Create Custom Plugin Call DOX Services from your Plugin Fill your entity with GenAI extract

without any further ado, let's read this wonderful blog on how to create a custom plugin in SAP CAP.

Reusable components for CAP with cds-plugin by @daniel_schlachter 

Now I hope you understand the way I am thinking. Let's crack fingers and start code again...

Code CAP-DOX Plugin

1. Create a baseline with this - 

mkdir cap_dox && cd cap_dox && npm init -y && cd ..

2. Modify your package.json with the below section 

3. Now try to replicate the below screenshot.

4. Now, take no time to annotate my.Recipe like this - 

5. Now have some magical code inside the cds-plugin.js.

6. Now let's add the secret JSON from default_key.json. It's the secret-key of your DOX service.

cds.cap_dox_key ->

API time for SAP AI Services - DOX

Let's try to understand some of the APIs from SAP BTP AI services - Document information extraction service.

GET /document-information-extraction/v1/schemas?clientId=default 

List of your all schemas in SAP-AI-DOX

GET /document-information-extraction/v1/schemas/{schemaId}?clientId=default 

Details of SAP-AI-DOX schema fields and parameters

Spoiler

Another spoiler.
This is just 2 API executions. This service is there for quite a long, I think it released at around 2019 and since then it's one of my favorite. It evolved a lot. 
- This one was from 2020 - Getting Started with Document Information Extraction Trial Service
- This one was from 2023 - Introducing Document Information Extraction Premium Edition - Unleashing the Power of Large Language...

Anyways, mostly the APIs remained kindof same.

If you want to play with all the swagger APIs, please go through - 
https://aiservices-trial-dox.cfapps.us10.hana.ondemand.com/document-information-extraction/v1/

Open this url for token - https://{subaccount}.authentication.us10.hana.ondemand.com/oauth/token?grant_type=client_credentials
then give the clientid as username and clientsecret as password. 
Again, this above information you would receive from default_key.json.

>>>>> You can use CF CLI for these details  <<<<<<<

1. Check your service lists - cf8 services

2. Details of your service - cf8 service default_document-information-extraction-trial

3. List of your service-keys for your service instance - cf8 service-keys default_document-information-extraction-trial

4. Get your service-key - cf8 service-key default_document-information-extraction-trial {service-key-name}

Bonus - 
There is an interesting thing, that I found here. If you create the SAP-AI-DOX service from booster, it creates the service_key as this pattern - 
defaultKey_{service-guid}

where this service-guid is - cf8 service default_document-information-extraction-trial --guid

I might have written a small blog in this spoiler... 😉😉

Next is coding time. Which is boring. Hence, get a clone from this git repo - cap_plugin_genai_dox_demo.

DOX Service Calls

Create a library lib_cap_dox.js where you can export the functions as you want - 

module.exports = {  
  auth_token: async function() { 
    return await get_token();
  },
  schema_id: async function(){
    return await get_schema();
  },
  post_job: async function(pdf, fileName, auth_token){
    return await post_job(pdf, fileName, auth_token)
  },
  get_job_status: async function(job_id, auth_token){
    return await get_job_status(job_id, auth_token);
  }, 
  entity_mapping_head: async function(dox_output, entity) {
    await entity_mapping_head_def(dox_output, entity)
  },
  entity_mapping_item: async function(dox_output, entity) {
    await entity_mapping_item_def(dox_output, entity)
  }  
}

Then also consume these functions file - cds-plugin.js then use this function to trigger your extractions.

Dynamic entity filling with the GenAI extract

Two things we should discuss here - 

Why?

Custom schema based extraction in SAP AI Document Information extraction service can provide you the possible entity mapping automatically. If not, then anyway you can manage to change it via the Document Information extraction UI, or from the Custom CAP Application.

How?

It's just some dynamic codes like below - 

async function entity_mapping_head_def(headerFields, entity){
  for(let item of headerFields){
    entity[item.name] = item.rawValue
  }
  return entity
}

async function entity_mapping_item_def(lineItems, entity){
  let ingredients = [];
  let ingredient = {};
  for(let item of lineItems){
    for(let item_properties of item){
      ingredient[item_properties.name] = item_properties.rawValue
    }
    ingredients.push(ingredient)
  }
  entity.ingredients = ingredients
  // log(entity)
  return entity
}

You can also make the entity names dynamic and read it from the package.json cds-environments configurations.

That's it then:

• It's an unofficial CAP plugin for SAP AI-DOX
• This CAP application can be bound with the Document-Information-Extraction Service Instance/Key, which can be used for authentications
• CAP-Plugin will change the way we code in CAP, it's like function modules in ABAP-Odata service. Also gives us wonderful flexibilities to code more. 
• SAP Joule is awesome, but I think it's a bit template driven. It can be a more reluctant in place we change and what we change. 
• I use this URL to start/stop my BAS - https://{subaccount}.us10cf.trial.applicationstudio.cloud.sap/index.html?externalRedirect=true
• I tried to upload some AI Generated pictures of robot, but somehow this community portal is blocking.
• Writing in the new UI of Community page is a struggle. Lot's of functionalities are there now. Tried lots of things, don't know what it will do. '# tags gives glitch while writing', 'extra space gets added near your code section every-time you edit the blog'. I also spent a lot to time to identify the issues caused in the HTML formatting.

Please comment on what could go wrong and what could have been better.

[MioYasutake]

Hi @sabarna17,

Thanks for sharing this series of blogs! I have a question regarding the plugin code.

https://github.com/sabarna17/cap_plugin_genai_dox_demo/blob/48a6840088e4939e7b4461d3318d7334b33ab806...

srv.on('ai_dox_extract', entity, async data => {...}

It seems that 'ai_dox_extract' is a bound action. Could you please clarify where this action is defined and how it is triggered?

[sabarna17]

Thank you, @MioYasutake  for reading the blog. You pointed this out correctly. It was a standard event 'CREATE', which I was using. Updated the git repo.