Case Study: Host Agent using high CPU when connecting to multiple Focused Run and Solution Manager

[Vera_Jiang88]

In this blog post I would like to share a case of High CPU usage by the SAP Host Agent on Linux when connecting to multiple SAP Focused Run and SAP Solution Manager. After a thorough investigation lasting three months, the problem was successfully resolved.

Background
Outside Discovery serves as an important data supplier to the System Landscape Directory (SLD) and Landscape Management Database (LMDB). It reports critical information such as Database details, Host specifics, Simple Diagnostics Agent data, Microsoft IIS, and other XML-formatted payload files.

For SAP Solution Manager systems, Outside Discovery Diagnostics Agent (OD DAA) is supported by default. However, switching to the newer Outside Discovery SAP Host Agent (OD SHA) is recommended due to limitation in the older OD DAA mechanism, achievable by following SAP Note 2556432. 

In contrast, SAP Focused Run systems exclusively support the Outside Discovery SAP Host Agent, as outlined in the FRUN Preparing Outside Discovery Guide.

Problem Statement:
A high CPU usage issue was observed with SAP Host Agent on the server, seriously impacting server performance. The top command revealed that users sapadm and daaadm were consuming substantial resources.

Solution

1. Clear the data flow from the managed server to the Focused Run or Solution Manager. Make sure there is no configuration loop.
   
   A configuration loop as the following example increases the SAP Host Agent workload and it has no meaning. A clear indication is seeing error message "[SLD Data Supplier Forwarding] ...429 Too Many Requests" in the /usr/sap/hostctrl/work/ousidediscovery.log. Note 0003082196 explains this error further. The 429 http response code is only returned by LMDB if we are getting two payloads from the same data supplier within a short time range. 
   
   The solution for the following example is to remove the "FRUN on Cloud" from the SLDR1 outbound connection. Alternatively, remove the SLDR2 destination file from Host Agent /usr/sap/hostctrl/exe/config.d. See the Preparing System Landscape Data Router for more information.
   
   
2. Enable only the OD SHA for SAP Host Agent.
   
   Avoiding using the OD DAA to supply the Solution Manager, while using OD SHA to supply the Focused Run. Switching to the OD SHA in the Solution Manager Agent Administration as steps in SAP Note 2556432. It's suggested to integrate the Solution Manager link into the SLDR outbound connection to further reduce SAP Host Agent workload.

By following these steps, the high CPU usage issue was resolved, optimizing server performance and enhancing data reporting reliability.