SAP Successfactors Incentive Management- What I wi...

yogananda · ‎2023 May 04

Dear All,

As you may be aware, Smart Data Integration (SDI) is seldom utilized in projects due to the availability of various ETL tools (SAP DataSphere, SAP Data Intelligence Cloud, CPI, etc.) that may be combined to move data from the source to the target system. In this manner, you may not be aware of what has to be considered, what limitations exist, how to solve your performance issues or errors or to debug, or how to go deeper for better design.

During my initial days, was not aware of whole SDI Implementation, the information was not much available where to start, how to debug/trace the errors... so started collecting all the necessary information during the projects..

Still I am by no means an expert, but over time I have compiled a list of Known things to be considered in the design that have proven extremely useful to me. If you are just starting to learn how to design and implement, this will hopefully offer you some guidance. If you are a seasoned developer, hopefully you will still learn something new.

### Note: Data Provisioning Agent version 2.3.5.2 or higher.

Available List of commands for agentcli

--createFullSystemDump   Gathers all realted log files and ini settings and creates an archive
--replicationSetup       Replication Setup Tool (Interactive operation)
--setSecureProperty      Secure Storage Utility
--configAgent            DPAgent Configuration Tool
--configAdapters         DPAgent Adapters Configuration Tool
                         Prerequisite: DPAgent should have already been started.
--configAgentIniFile     DPAgent Runtime Options (dpagent.ini editor)
--configAgentKeystore    DPAgent Keystore Configuration Utility
--LostDataTracker        Lost Data Tracker (Interactive operation)
--help                   To print this help menu
--help --all             To print detail help for all commands
<operation> --help       To print help for the given operation

How to create DP agent FullSystemDump - SDI ?

The DPAgent provides a simple utility to export all configuration, logs, and runtime environment information in a convenient ZIP format that is easy to attach in a support case.

Steps

Using the command line, go to the DPAgent installation's bin folder (Defaults are "C:\usr\sap\dataprovagent\bin" for Windows, or "/usr/sap/dataprovagent/bin" for Linux)

Find agentcli.bat or agentcli.sh file, run it with the following command:

On Windows:

agentcli.bat --createFullSystemDump

On Linux:

agentcli.sh --createFullSystemDump

After a few seconds of processing, the script will output the name of the ZIP file to your screen; gather and attach that file to your case.

How to change "Logging Level" (Trace level) of a Data Provisioning Agent (DPAgent)?

In your DPAgent, you can enable/change the "Logging level" to trace or debug your issue.. which would be easy to detect your problems quickly.

Steps

Go to DP agent installation bin folder <DPAgent_Install>\bin
Run agentcli.bat --configAgent for Windows, or agentcli.sh --configAgent for Linux
Choose option "Agent Preferences"
Choose option "Set Agent Preferences"
Choose option "Logging Level"
Change level to ALL/DEBUG/TRACE/INFO according to your requirement, usually for troubleshooting problems, we need ALL level for trace
No need to restart DP agent, after troubleshooting, you can follow the same steps to change trace level back to INFO

You can also manually change dpagentconfig.ini file with "framework.log.level=<NEW Value>", this needs DP agent restart

Encryption supports for FileAdapter

You must use a version older than GnuPGP 2.3 or a version compatible with -rfc4880. For example if using Kleopatra you can use the version 3.3.16 or older: (this is specifically tested with 3.1.15 from https://gpg4win.org/change-history.html)
For 3.3.15 version, it had the following details:

GnuPGP: 2.2.27
Kleopatra: 3.1.15
GPA: 0.10.0
GpgOL: 2.4.10
GpgEX: 1.0.6
Kompendium DE: 4.0.1

How to increase DP Agent memory?

If you want to improve the performance of SDI

Steps

Stop DP agent.
Open the dpagent install folder.
Open dpagent.ini file(e.g. /usr/sap/dataprovagent/dpagent.ini)
Locate the line starting with -Xmx, to edit the value to desired size.
Increase parameter -xmx to a higher value (ex: -Xmx4096m to -Xmx8192m).
Save dpagent.ini
Start DP agent and test again.

How to disable streaming for SDI?

SAP Support asked you to disable streaming on DP Agent - DP Server communication channel

In target HANA execute: ( SAP Commissions Support team will perform this task)

ALTER SYSTEM ALTER CONFIGURATION ('dpserver.ini', 'SYSTEM') SET ('framework', 'useStreaming') = 'false' WITH RECONFIGURE;

How to update HANA XS user password for DP agent connection

Symptom

The HANA user used for DP agent connection with password change recently

User lock may happen in HANA system because of DP agent keeps retry the connection through old password

com.sap.hana.dp.adapter.sdk.AdapterException: Error 401 - Not authorized

Authentication Error

Steps

At the command line, navigate to <DPAgent_root>/bin
Execute one of the following commands using the --setSecureProperty parameter:
On Windows, agentcli.bat --setSecureProperty
On Linux, ./agentcli.sh --setSecureProperty
If you want to use a different HANA User for Agent Messaging, choose Set HANA XS Username and enter the new username as prompted
Choose Set HANA XS Password and enter the new password as prompted
Restart the Data Provisioning Agent

How to get Remote Source Statements Status ?

It displays detailed information about executed remote queries. This information includes the query status and the number of fetched rows. This helps for monitoring the data load.

 SELECT * 
    FROM "M_REMOTE_STATEMENTS"
    ORDER BY "START_TIME";

Single command to perform individual configuration tasks

e.g. ./agentcli.sh --configAgent --function agentStatus   --- ShellScript

e.g. agentcli.bat --configAgent --function agentStatus    --- Command Prompt

Hana XSA Web IDE build fails because of timeout

[ERROR] Your build has been shut down because it took longer than the currently configured timeout."

Since the resolution requires access to XSA via the XSA_ADMIN user, and the customer does not have access to these users, it is suggested that the Database Administrator (DBA) perform the steps outlined in the KBA.

https://me.sap.com/notes/3423910/E

Maintaining Services in XS Advanced ( SAP Incentive Management Support team Guidance)

Use the xs command-line interface to maintain so-called “Backing Services” for applications running in the XS advanced run-time platform.

https://help.sap.com/docs/SAP_HANA_PLATFORM/6b94445c94ae495c83a19646e7c3fd56/7800cd3cb0714b95829698c...

Error: "Error executing query Request timed out" while inserting to virtual table through CamelJDBCAdapter - SAP HANA Smart Data Integration

Insert to Virtual table through CamelJDBCAdapter failed

Error: "SAP DBTech JDBC: [403]: internal error: Error executing query Request timed out. for query " INSERT INTO """<Schema_Name>"".""<Virtual Table>""" ( "**", "**") VALUES (?, ?)" "

Slow performance for insert, in the DP agent framework.trc file, insert logic is record by record, no batch insert(bulk loading)

Solution :
Increase the following timeout value in target HANA, default is 600

ALTER SYSTEM ALTER CONFIGURATION ('dpserver.ini', 'SYSTEM') SET ('framework', 'prefetchTimeout') = '7200' WITH RECONFIGURE;

ALTER SYSTEM ALTER CONFIGURATION ('dpserver.ini', 'SYSTEM') SET ('framework', 'messageTimeout') = '7300' WITH RECONFIGURE;

SAP HANA Smart Data Integration REST API

Use the SAP HANA smart data integration REST API to programmatically run and monitor flowgraphs, to process data for interactive data transformation within your application, and to create, modify, and delete virtual tables.

https://help.sap.com/docs/HANA_SMART_DATA_INTEGRATION/bd4ae8b11fd749cabb248aff53837eb8/5ffc497e8d4d4...

2447413 - Interested in new features for SAP HANA Smart Data Integration?

https://me.sap.com/notes/2447413

DP agent status switch between connected and disconnect in SAP DataSphere - SAP HANA Smart Data Integration

https://me.sap.com/notes/3420691

SAP HANA returns a result that seems to be wrong.

https://me.sap.com/notes/2222121/E

Central Note for Behavioral Changes in SAP HANA

https://me.sap.com/notes/2679360/E

SAP HANA 2 SPS07 Revision 072.00

https://me.sap.com/notes/0003348635

I will keep adding more and more information for you which may help in your project implementation.