Performance Optimization in SAP APO

[Former Member]

Abstract

Performance optimization is an integral part of any APO implementation. Unlike transaction intensive processes, advanced planning in APO utilizes Live Cache to process the planning calculations and therefore subject to multiple performance constraints. To address the performance issues, SAP recommends multiple OSS notes along with few key configurations and adherence to best practices to execute the interactive as well as background jobs.

This whitepaper documents key APO processes and related background jobs for demand planning and supply network planning which substantially impact the performance of APO landscape unless otherwise rectified during the realization phase. Along with different phases of test life cycle, typical issues encountered during the execution of performance jobs have been discussed and recommendations to improve performance of the system with respect to each issue identified have been emphasized. Content in this document is focused upon the approach to be adopted towards performance optimization, preparation, requirement gathering and execution of test life cycles rather than describing each and every transaction impacting the system.

1.0 Overview of Performance Optimization

Performance optimization is a thorough assessment whether an IT solution satisfies its performance, capacity and scalability requirements. Performance tests are designed to analyze system attributes such as responsiveness, throughput, peak workload capacity and stability under load.

Performance testing is intended to ensure the production infrastructure’s ability to support a threshold that is acceptable to the Client’s processing requirements. Performance testing involves the following objectives:

• Simulate an expected business operational load.
• Measure the system response time as it relates to CPU and memory usage of the production like environment.
• Emulate the number of users required to produce the defined frequencies and volumes for a targeted 8 hour period.

Performance testing provides the following benefits to the customer:

• Identifies performance defects so that they can be addressed before deployment
• Aids in planning for and potentially reducing the IT capacity required for deployment
• Improves productivity, stability, user / customer satisfaction and ROI related to mission-critical systems

Performance defects resolved during testing are realized once the tested system is deployed in production. Capital savings in IT infrastructure spending are realized over time with the deployment of more resource-efficient, scalable systems.

Typical Performance Issues in an APO Landscape:

• Poor performance of mission critical applications – demand forecasting, supply planning etc.
• Uncertainty of application performance e.g. forecast not available by 8AM on Monday morning as an SLA for the Demand Planners.
• Difficulty predicting performance of complex architectures e.g. fluctuation in time consumed to create forecast or supply plan.
• Significant and costly rework required to resolve production performance problems e.g. changing hardware sizing parameters in production environment.
• Limited client experience dealing with complex performance testing – client considers APO similar to other ECC modules and is unaware of the nuisance of performance parameters of an APO landscape.
• Inadequate performance test environments – performance test environment should be exactly like production, otherwise the performance testing does not reveal the real-life results.
• Adverse impact of poor performance on user experience, brand image and customer relationships e.g. frustrated demand planners with forecast data on Monday morning as expected from APO.

  Benefits of Performance Optimization: 

• Performance testing based upon S.M.A.R.T. (specific, measurable, achievable, repeatable, testable) performance requirements (derived from non-functional requirements) will identify stress points in the infrastructure and a need for additional resources and testing.
• Performance testing under load (Volume testing) explores the limits of the implementation and prepares the business to better respond to expected and unexpected demands
• Performance testing and performance monitoring are key to managing complex systems
• Performance testing enables the tuning of systems to optimize performance.

Performance Optimization Testing Life Cycle:

Performance optimization assessment follows the following phases during its life cycle:

• Test Planning - Planning phase deals with development of a performance test strategy followed by a detailed test plan.
• Test Design – This phase deals with definition of test scripts and preparation of test data in a test environment created for performance testing. A detailed test execution plan must be in place before the start of actual testing.
• Test Execution – In this phase, the actual testing is performed. Issues/defects encountered and results are logged followed by a detailed analysis of results. Output of this phase is a detailed performance test report including risks.

2.0 Performance Optimization Test Planning

Typical scope of performance testing involves the following planning modules in APO and ECC:

• Demand Planning & S&OP –
  • Interactive demand planning
  • Demand Planning batch jobs
• Supply Network Planning & Deployment –
  • Interactive planning for DRP and Deployment
  • DRP and Deployment through batch jobs
• Master Production Schedule and MRP –
  • Interactive planning for MRP and MPS
  • MRP and MPS through batch jobs
• Supply Network Collaboration –
  • Interactive planning and replenishment of VMI
  • VMI planning and replenishment through batch jobs
• ECC Execution & Interfaces -
  • Any volume intensive interfaces designed to support the solution

Test planning involves two-pronged approach towards performance optimization:

• Performance Tuning – Performance fine tuning is realized by following SAP best practices (please refer appendix – 1 for details). This deals with application of recommended reports, parallel processes, control of master and transaction data, application of suitable OSS notes etc.
• Performance Improvement – Performance improvement is executed by following optimization of system parameters e.g. block size, no of parallel processes, server group etc. for certain volume of data for different process areas. During execution, multiple runs will be executed with different set of parameters to optimize the run time (please refer appendix - 2 for details).

3.0 Performance Optimization Test Design

Output of this phase is a set of test scripts to be used for different process areas as planned. Performance test scripts should be drafted for all process areas as planned and loaded to HP Quality Center / HP ALM.

Test scripts focus on the performance rather than on the system functionality and hence integration test scripts can not be used directly for performance testing.

Test data is an integral part of the Design phase and needs to be built into the system created for performance testing. A pre-production system or a dedicated performance test system should be created with full master and transaction data transferred from ECC to APO. Performance test system should resemble actual production and hence should contain full load of transaction and master data as expected in production environment after go-live. Data can be loaded either manually or through conversion process.

Before performance test scripts, it has to be ensured if all the system functionalities are up and working. This requires execution of few select integration test scripts preceding performance testing, to ensure the smooth functioning of the system. This activity can be carried out as a part of pre-performance testing readiness or as a part of cut-over simulation.

Key input to design of performance optimization test script is the availability of non-functional requirement. Along with functional requirement for solution design, gathering of non-functional requirement (e.g. SLA, timing of batch jobs etc.) should be emphasized during the blue-print phase of the project. Non-functional requirement

4.0 Performance Optimization Test Execution

Test execution involves running the test scripts designed in the pre-production system (or any system designated for volume testing) and results were recorded in HPQC.

For example, the following job has been described to explain the approach towards performance test execution.

Name of the Job – APO DP Forecasting Job

Business requirement: This is extracted from non-functional requirement for a certain proportion of actual/expected production volume of data.

For example - 30 minutes with 10000 CVCs

In the production, it is expected to be 40000 CVCs in NA and LA markets with total 250000 CVCs in the global model.

Test Volume:

Test1: 10000 CVCs,

Test2: 10000 CVCs,

Test3: 47000 CVCs,

Test4: n CVCs etc.

Job name: RUN MONTHLY FORECAST

Program: /SAPAPO/TS_BATCH_RUN

Response Time Monitoring:

SAP Basis team helps in recording this time as mentioned in the table above.

Analysis of Issues encountered:

- Only 1 AP server is assigned to the sever group for parallel processing

- Optimal setting may not be used in the parallel processing of DP job

- Possible locking issues causes failure of DP forecasting job

- Excessive use of forecast errors in DP forecasting

Similarly, other jobs as identified for volume testing needs to be executed.

Other typical jobs subjected to volume testing are as summarized below:

•      DP Forecast Release Job (Job/program name: /SAPAPO/RTSOUTPUT_FCST)

(/SAPAPO/MC90 should not be used for mass-processing of DP forecast release)

• SNP Heuristic Job
  • Volume criteria – no of location products
• SNP Deployment Job
  • Volume criteria – no of products, no of demands and no of stock transfers
• SNP Capacity Leveling Job
  • Volume criteria – no of resources
• SNP Interactive Planning
  • Volume criteria – no of location products, no of users planning at one time etc.

5.0 Analysis of Issues and Recommendations

This section describes the important issues encountered during execution of the performance test scripts and recommendations to improve performance.

Issue log is a very crucial record of errors/defects noticed during the execution of the performance test and constitutes an important part of the final Test Report.

Few commonly observed issues have been summarized below for the reference of readers.

5.1 Only 1 AP server is assigned to the sever group for parallel processing

Issue Description

• Only one server is assigned to server group ‘parallel_generators’ which is used for parallel processing in SCM.
• There are multiple application servers provided in the production environment.
• If you can assign more servers, it becomes possible to increase the number of parallel processing much more.

Business Impact

• If only one server is assigned, the load balancing among servers does not work and all child processes are executed in 1 application server in the parallel processing. This can cause performance issue in SCM background job and CIF processing jobs.

Recommendation

• Assign other more servers to server group ‘parallel_generators’.

5.2 Optimal setting may not be used in the parallel processing of DP job

Issue Description

• Currently the number of parallel processing is less, but if more servers can be used, it can be increased much more.
• Block size (currently 1.000) is also can be optimized by decreasing the value.

Business Impact

• It takes more time to execute the background jobs if the optimized setting is not used.
• In worst case, it causes the delay of the daily operation in the PTS process.

Recommendation

• Increase the number of parallel processing if further performance improvement necessary (i.e. from 4 to 8, 12).
• Find the optimal block size by decreasing the value. (i.e. 500, 250, 125, 50, 25…).

5.3 Possible locking issue causes failure of DP forecasting job

Issue Description

• Lock conflicts occur between forecasting in /SAPAPO/TS_BATCH_RUN and other applications which use time-series live Cache.
• This can be avoided by implementing SAP Note 1386453.

Business Impact

• It can cause the error of DP job. As a result, DP process delays.

Recommendation

  Apply SAP Note 1386453 - /SAPAPO/TS_BATCH_RUN - Locking issue in forecast.

5.4 Excessive use of forecast errors in DP forecasting

Issue Description

• Selection of all six forecast errors in forecast run is performance intensive and the usage of these forecast errors should be minimum.
• Each forecast error is stored in a key figure, which needs more liveCache memory and calculation time.

Business Impact

• More memory is consumed and the performance issue causes the delay of DP process.

Recommendation

• Consider minimizing the usage of forecast errors in the forecast profile.
• The recommendation is calculating a maximum of two forecast errors.

5.5 SAPAPO/MC90 should not be used for mass-processing of DP forecast Release

Issue Description

• Typically, forecast release from DP to SNP is executed through transaction /SAPAPO/MC90, in which parallel processing cannot be used, so it needs more time to finish it.
• SAP program /SAPAPO/TS_BATCH_RUN should be used for the mass processing of forecast release.

Business Impact

• The sequential processing of /SAPAPO/MC90 needs more time to finish.
• As the result, it consumes more time in the weekly job.

Recommendation

• Use SAP program /SAPAPO/TS_BATCH_RUN to release forecasts.
• Refer to SAP Note 546079 - FAQ: Background jobs in Demand Planning, which provides the information to setup the job.

5.6 Planning version in APO DP should not be used as the active version

Issue Description

• Active planning version 000 can be used in APO DP without impacting any core functionality.

Business Impact

• If both DP as well as SNP/PPDS/CTM are also being implemented, then active planning version 000 gets overloaded with data. SNP/PPDS/CTM must use active planning version 000 for integration with ECC.
• As the result, overall efficiency of the APO landscape becomes sub-optimized.

Recommendation

• Use any non-active planning version (non 000) to design APO DP solution.
• Refer to SAP Consulting Note 429400 (recommendations about planning versions for SAP APO DP).

Similarly all other performance intensive jobs in APO needs to be analyzed and optimized. Few OSS notes mentioned below can be reviewed and applied to enhance system performance. Check the following SAP consulting notes and apply all recommendations which can be applied:

• Note 398726 - DP 3.0: performance planning book/data view
• Note 608985 - Performance in interactive planning
• Note 707789 - Planning book/data view performance
• Note 821044 - Drill down in SNP planning book too slow etc.

6.0 Lessons Learned

• Plan the performance test scripts during preparation for Integration Testing

Performance test scripts should be planned simultaneously while planning for integration test scripts. This will help in identifying most of the scripts which can be re-used in both the testing process and will prevent duplication of effort for re-writing the scripts solely for performance testing. It helps to pre-plan if the performance testing is to be recorded using HPQC as integration testing to prevent any last minute surge in consulting effort for the process and testing team

• Apply performance relevant OSS Notes before the start of Performance Testing

Relevant OSS notes which aids in improving performance in APO, should be proactively implemented during or before integration testing. This ensures system readiness for performance testing and negative effect of the OSS notes automatically gets tested during integration testing. Server sizing is the most important activity in predicting the performance of an APO system and we need to follow the SAP guidelines and recommendations to decide the appropriate size.

• Performance Testing is as important as Integration Testing for APO modules

APO modules are highly performance sensitive due to involvement of complex calculations in live Cache and needs appropriate sizing of servers to make the solvers and algorithms deliver the output in required time frame. Even if the system functionalities are in place, client may not experience the benefit of the APO system unless the performance issues have been addressed in time appropriately, unlike ECC modules.

7.0 Conclusion:

Final Test Report is prepared based on the defects logged during the execution of performance test scripts which analyses the defects, possible recommendations and their implementation plans. Once the recommendations get implemented, defects can be tested again and closed after satisfactory performance of the system.

Performance testing is an integral part of any APO implementation and constitutes an inevitable part of any APO project plan. While gathering the functional requirements, consultant should not lose sight of the non-functional requirements and associated planning required to implement them. While performance is not a glaring issue in most of the SAP ECC modules, client might under-estimate the importance of this in SAP APO. Therefore the consultant needs to be abreast with the latest developments in the field of performance testing to be able to recommend appropriate solutions during the realization phase of an APO project.

8.0 About the Author

Pravat Dash, CPIM is a Consulting Manager in the Business Consulting Services Group of IBM Global Services. He has over 16 years of SAP SCM/ERP implementation and industry experience in the area of Supply Chain Management and Logistics. He has worked as Lead Consultant implementing several SAP SCM solutions for clients in various industry verticals. He has authored multiple papers in Supply Chain Planning space. You may reach him via email prabhatdash2003@yahoo.co.in.