BTP CAP : Custom plugin to separate generic change log capture logic

[sreehari_vpillai]

Introduction

I have a business requirement to capture data change logs for (almost) all the entities in an API. We are using CAP framework with NodeJS to implement our OData services . A CAP plugin lets you separate a generic logic in a separate codebase , which later can be used in CDS files using annotation framework .

Please refer here for the standard audit log feature provided by the framework .

What's the plan 

In the service CDS files , I shall annotate any entity with @audit  . When an "UPDATE" is triggered against this entity , changed data fields must be captured and logged .

CDS Service

Plugin development 

Refer this blog for understanding how you must setup a plugin .

A plugin is loaded when the service is started. We must , programmatically identify the entities / properties or services which are annotated by our custom annotations ( @audit in our case) . We do this after once the CDS framework has served our model .

I capture the before UPDATE event of the service entities which are annotated with '@audit' . Then read the existing record from the database and compare the attributes to track the changed records alone.  This shall then be saved in a DB or log file as you please.

const cds = require('@sap/cds')

cds.once('served', () => {

    for(let srv of cds.services){

        //for each served services

        var entitiesToAudit = [];

        for(let entity of srv.entities){

            if(entity['@audit']){

                console.log('Audit needed for ' , entity.name );

                entitiesToAudit.push(entity);

            }

        }

        srv.before("UPDATE" , entitiesToAudit , async (req)=>{

            //incoming payload

            let data = req.data; 

            let dataKeys = Object.keys(data);

            //keys of the entity

            let entityKeys = Object.keys(req.target.keys);

            let keyData = {};

            entityKeys.forEach((k)=>{

                keyData[k] = data[k];

            });

            //entity itself 

            let target = req.target; 

            //read the exisitng record from DB via service 

            var originalData = await srv.read(target).byKey(keyData);



            // do comparison field by field

            var changeLog = {};

            dataKeys.forEach((dk)=>{

                if(originalData[dk]!= data[dk]){

                    changeLog[dk] = {

                        oldValue : originalData[dk] ,

                        newValue : data[dk] 

                    };

                }

            });

            var changeLogWithSignature = {

                user : req.user.id ,

                at : new Date() , 

                changeLog : changeLog

            } ;

            //log the data on console, Kibana , or write it to DB / File

            return data ; 



        });

    }

});

 

Now my CDS codebase looks clean. I don't need to write this logic in the service handlers for each services.

Test it 

Run cds watch on the terminal . You can see

[cds] - loaded plugin: { impl: 'audit-plugin/cds-plugin' }

[cds] - loaded model from 2 file(s):



  srv/Service.cds

  db/Database.cds

Create an entry 

 

Entry has been created ( in the in-memory DB )

Update this entry 

 

It has tracked the changed records' old and new values along with the timestamp. I would store this in a DB or save it to a file / console.log() it.

 

Now lets Build & Deploy the application 

Right click the mta.yaml and choose build . And it failed with the below messages

[2023-05-10 12:07:21] INFO the build results of the "simple-cds-srv" module will be packaged and saved in the "/home/user/projects/CAP-Apps/simple-cds/.simple-cds_mta_build_tmp/simple-cds-srv" folder
[2023-05-10 12:07:21] ERROR could not package the "simple-cds-srv" module when archiving: could not read the "/home/user/projects/CAP-Apps/simple-cds/gen/srv/node_modules/audit-plugin" symbolic link: stat /home/user/projects/CAP-Apps/simple-cds/gen/srv/audit-plugin: no such file or directory
make: *** [Makefile_20230510120706.mta:37: simple-cds-srv] Error 1
Error: could not build the MTA project: could not execute the "make -f Makefile_20230510120706.mta p=cf mtar= strict=true mode=" command: exit status 2

 

Framework couldn't understand the new plugin directory we created ( audit-plugin ) . To fix this , I am adding this directory in the build-parameter of mta.yaml

to do this

add

"copyfiles": "2.4.1"

as devDependencies in the root project's package.json

then in mta.yaml , add this statement as build-parameter , before-all block

build-parameters:

  before-all:

    - builder: custom

      commands:

        - npx -p @sap/cds-dk cds build --production

        - npx copyfiles -f audit-plugin/*.* gen/srv/audit-plugin/ -a 

Note : There may be a smoother way to copy the plugin to the gen folder while 'cds build' . If I come across any such , I shall update the blog with it.