Support package stack updates for SAP ABAP systems: Software Update Manager

[Mani_GMK_2822]

Hello, BASIS Team.
In this article, I'd like to discuss the SPS update procedure from the scratch for ABAP systems.

To obtain approval to carry out the activity, we must first prepare the technical plan document.

Pre-requisites:
1. Request access for the ddic user.
Validate user by logging into the system and collect the following information.

• Installed Softwares and their SPS levels and components.
• License Information.
• SPAM/SAINT Version.
• Kernel version and their patch level.

2.For Windows applications, use sidadm and sapserviceSID as user credentials.
Linux requires sidadm and root privileges.

Validate the user credentials using RDP for Windows & Putty/Moba for Linux operating systems and capture the following information.

• SAP-Host agent version and their patch levels.
• Kernel version and their patch levels.

3. For database types of ASE: sapsa, sapsso, and sa users' credentials.
HANA: system user credentials.

Process:

1.Create a stack XML file and add software components to the download basket.

• Follow the below article for Maintenance planner transaction ID generation.

Generation of Maintenance Planner Transaction ID f... - SAP Community

• If there is a problem with synchronizing data in maintenance planner and Solman is unavailable, you must manually sync the system data.

Generating the System-info file for ABAP and JAVA ... - SAP Community

• Generation of stack xml for sub-sequent systems in the landscape (i.e. Quality and Production) 

2419417 - Maintenance Planner - Copy a stack XML for an identical system - SAP for Me

SAP Support Portal Home –>Maintenance –> Maintenance Planner –> Access Maintenance Planner–>Explore Transactions –>

Choose the system from the center of the maintenance planning cycle

Choose Software tab and choose Generate Stack XML Copy

In the Generate Stack XML Copy dialog box, enter or choose the technical system (SID) for which you want to generate a stack XML copy.

Note:

• No new transaction is created when you generate a copy of the stack XML.
• The feature is only available for update or upgrade transactions and for transactions generated for technical systems with product maintenance dependency.
• The feature is not available for SAP S/4HANA conversions and does not include transactions for system tracks.

2.The media files can be downloaded using the SAP download manager.

3. Transfer the media files to the server, ensuring suitable permissions and ownership.

Disk space in the file system for the SUM directory, the download directory, and directory DIR_TRANS.
The space required depends on the product you are updating.

SAP suggested at least 40 GB of free space for both the SUM and download directories, as well as 20 GB for the DIR_TRANS directory.

The Software Update Manager calculates the database's space requirements. The required amount of free space ranges from 50 to 200 GB.

In our case we placed at <Drive>:usr\sap\SID\Downloads

Move the sum.sar file from downloads to <Drive>:usr\sap\SID and extract the sum.sar file with sapcar

After successful extraction, a new directory will be created under the name SUM.

4. Launch the SUM tool with the following command.

Paste the above URL in any web browser.

https://<fully qualified hostname>:1129/lmsl/sumabap/<SID>/slui

Enter the sidadm user credentials and click on the sign in button.

If there is an issue with launching the SUM URL.

It might be attributed primarily to the SAP host agent. (Faulty installation, incorrect permissions, etc.). Dev_saphostexec and sapstartsrv.log (found in the /work directory) are crucial logs for resolving common UI issues on the SAP host agent side.

If no apparent problem is observed in these logs, try increasing their trace level by following.

Open the SAP Host Agent profile ( host_profile ) which is located in the exe directory of the SAP Host Agent (/usr/sap/hostctrl/exe/ or C:\Program Files\SAP\hostctrl\exe\).
Add or modify the required values, for example:
hostexec/trace = 3
service/trace = 3
Restart SAP Host Agent by executing saphostexec -restart.

Resolution can be found from the below KBA's

2475146 - SUM : logon via SAP Host Agent is failing "Unable to locate the requested resource" or "<h...

The Software Update Manager internally calls the SAPup executable. Technically, SAPup is started twice,
which is visible in the process list. One entry with gt=httpchannel represents the SAPup handling the
requests coming from the SAP Host Agent. The second SAPup with parameter guiconnect triggers tools
such as R3trans, tp, or R3load.

For windows: Task manager will list ou

###############################Roadmap-Phase###################################

Specify the path of stack XML file i.e. Drive<Name>:\usr\sap\SID\Downloads and click the next button.

Click the next button after selecting the Standard Configuration scenario for the strategy.

Check Archive Authenticity: Select the check box if the authenticity of all archives in the download directory shall be checked.
Switch expert mode on: Select the checkbox if you want to decide on additional options to adapt the tool configuration. If you want to enter a specific instance number and a password manually, choose the option Switch expert mode on in the initial dialogs. Make sure that you enter an instance number that is not used in your system landscape.

Customer Transport Integration: Include customer transport requests to reduce business downtime.
This option is only available for strategy modes Standard or Downtime-optimized. It is not displayed if
Single System has been selected.
Select the checkbox Customer Buffer File, if a customer buffer file shall be considered that contains
customer transports for the target release. A subsequent dialog prompts you to enter the path and the
name of the buffer file.

Select the radio buttons Check archive authenticity & Switch expert mode.

Download the file from this link: https://tcs.mysap.com/crl/crlbag.p7s.
Move the downloaded file to <Drive>:\usr\sap\SID\Downloads.
Then click the next button.

###################################Extraction-Phase###############################

These passwords are stored in the memory file MEMSAPup.dat and it is located in /usr/sap/SID/SUM/abap/mem
Although the passwords are encrypted, you have to deny the access to the mem subdirectory for other
operating system users. Only the <sapsid>adm user needs access to this subdirectory. In addition, you should not share this subdirectory in the network. If you save the SUM directory for later analysis After the update has finished, delete the file MEMSAPup.dat.
ddic credentials are needed for the RFC connection to the SAP system. To be able to perform all necessary actions, the user DDIC needs the authorization SAP_ALL.

Usually, it is not necessary to change the password of the user DDIC during the upgrade. If you do change
the password, you have to change it in the original system, the shadow system, and in the Software Update Manager.

Enter the credentials for ddic, sapserviceSID, and system.

Click the Next button.

Scans the entire update directory i.e. /usr/sap/SID/SUM/Downloads

Extracts all sar files into /usr/sap/trans/EPS/in, excluding the files indicated below.

• SAP hostagent
• SUM
• Kernel files

If a newer version of spam/saint is found in the update directory, sum will update the spam tool.

Extracting the kernel executables from either the update directory or the global directory (depending on your system kernel version) to /usr/sap/SID/SUM/abap/exe (such as sapexe, sapexedb, R3trans, igs, igshelper, and vc_redist.

If installed kernel is the latest, then kernel extraction will be done from global directory to <Drive>:usr\sap\SID\abap\exe.
For example, if the system kernel is the most recent version, the SUM will appear as shown below.

Drop down each step to get more information, then click the next button.

#################################Configuration- Phase##############################

Select the radio option for "No disabling of the database archive mode."

ABAP Processes (Uptime & Downtime):

This parameter determines the number of processes used in job phases. It affects all job-run phases, including long runtime ones such as DBCLONE (in update situations), ACT_UPG and PARDIST in uptime, and XPRAs in downtime.

The DBCLONE phase has a technical restriction of 10 parallel processes at most, therefore increasing the number would not assist.

SQL Processes (Uptime & Downtime):

This option specifies the number of parallel SQL statements to be executed on the database at the same time. This setting is especially beneficial for the DDL (data definition language) phases, such as MVNTAB and PARCONV, where commands like "create table," "alter table," "create index," and others are executed. This parameter is also used by the phases that construct aliases/synonyms for the shadow system (SCEXEC_GRANT + SCEXEC_ALIAS), albeit to a lesser extent. The recommended value is the total number of physical CPUs.

R3trans Processes (Uptime & Downtime):

This parameter is used for tp and R3trans transport tools in parallel during the phases where support packages are imported (DDIC_UPG, SHADOW_IMPORT*, and TABIM_UPG).

Recommended value is 1 to 2 times the CPU, with a maximum of 50 CPUs.

If your CPU is multi-core, the SAP system will treat it as a physical, available CPU.
For example, if you have four quad-core CPUs, the SAP system will recognize them as sixteen CPUs in ST06. In this situation, you can put '16' for both the UPTIME and DOWNTIME R3TRANS PROCESSES fields (note that standard transport requests import may still occur during uptime, and they share the same tp/R3trans resources). So, avoid importing requests during upgrades.

R3load Processes (Uptime & Downtime):

This parameter guides the upgrade tool how many R3load processes will be executed during phases EU_IMPORT* (in system releases upgrade) and DMO (data migration option) scenarios to transfer data from the old database to the new HANA one. Because R3load uses modest CPU processing, you can safely specify a large value for this parameter: 3 to 5 times the number of CPUs.

Select the radio button Automated batch job distribution (If you have multiple instances and batch job load scheduling is scheduled)

In our case, select the radio button HOSTNAME_SID_<nn>

Uncheck the Start ABAP Load Generation During Update checkbox and then click the Next button.

SAP Load Generator (SGEN) is a transaction that compiles ABAP repository objects that are newly supplied into the system, either through an installation or an upgrade/support package import. The term "Load" refers to an ABAP runtime object used in ABAP settings.
The ABAP objects are stored on their source code in table REPOSRC and on their runtime state in table REPOLOAD.

If you selected the Advanced pre-configuration mode in SUM, you would get the following prompt

Choose an execution strategy for transaction SGEN.

• Do not start ABAP load generation during the update

If you plan to apply a high number of relevant changes to your system in the form of SPS updates or upgrades. We recommend that you generate the ABAP loads manually at end of post-processing phase.

• Generate ABAP loads on shadow system during uptime.

With this option, the Software Update Manager uses the maximum number of processes entered below to generate the loads in the shadow instance during the uptime in the update phase SUBMOD_SHD2_RUN/RUN_RSGEN.

Usually, the shadow SGEN prolongs the uptime part of the update procedure or several hours. If you run into unexpected time issues with your update schedule and you want to save the SGEN time, you can abort a running shadow SGEN by executing the report RSUPG_SGEN_ABORT_SHADOW in the shadow system. Use the DDIC user for the logon. Note that as soon as the SUM run is completed, you have to perform a manual SGEN run to generate the missing loads.

• Generate ABAP loads on shadow system during uptime and start asynchronously in post downtime.

With this option, the Software Update Manager uses the maximum number of processes entered below to generate the ABAP loads in the shadow instance during the uptime in the update phase SUBMOD_SHD2_RUN/RUN_RSGEN

The transaction SGEN automatically starts again after the downtime during the Postprocessing roadmap step to regenerate the ABAP loads that were invalidated during the downtime in phase MAIN_POSTP/RUN_RSREGENLOD

In the development system, we'll select our own SPDD transport request radio button.
Don't fill anything in SPDD and SPAU transport
then click the next button.

For sub-sequent systems in the landscape SPDD request will be displayed as radio button.
Fill the SPDD and SPAU transports.
Check the radio button Yes, I want to check the modification requests and click on the next button.

Click on the next button.

In our case, we will not change the version of the add-on product.
Click the "Next" button.

If the SUM identifies more support packages in the download directory or TRANS folder that are acceptable for upgrading the target system, the notification "Deviations to calculated level found" appears.

You can mark the checkbox "Take over calculated level" to take over the calculated support package level.
For Example:

If you do not click the box, the SUM process will continue with the support package level given in the column "Current Level".

Click on the next button.

Assign the instance number to the shadow instance and then click the Next button.

Assign the password for the Database Shadow user and click the next button.

Choose the radio button "no" and click the next button.

Drop-down each step for more information, then click the next button.

################################Checks- Phase##################################

No manual input is required.
DB related parameters and DB space verified here.

Click on the next button.

###############################Preprocessing- Phase##############################

After getting approval from the Application Development Manager, click the radio button to lock the development environment now before proceeding to the next step.

Approval from several teams is required because the landscape has been locked, and no changes will be allowed.

If there are any modifiable state requests in the system, it will display a list of open repairs.
Inform the corresponding team, who will include all changes into the request and release it.
Click on the next button.

DB Clone Phase:

During this phase, the target system (shadow system) is copied from the current system. The Software Update Manager initiates background processes to copy the necessary tables for the shadow system's operation.

Depending on your system and your hardware, this operation can take several hours. These background jobs are executed on the background server that is configured with the BATCH HOST parameter. The number of jobs is determined by the parameter MAX BATCH PROCESSES. It cannot be higher than nine or the number of background processes configured on the background server.
Each table is copied with a single INSERT statement. Therefore, the administrator has to take care that the undo-logs can grow accordingly.

Each cloning job writes a log file called DBCLONE<NUMBER>.<SID>.
At the end of the phase, all errors found in these log files are collected in the DBCLONE.ELG file.
The information in the DBCLONE.ELG file is rather sparse. Therefore, you also have to check the corresponding log files to find detailed information about the error.

SAPup automatically starts the shadow instance in the START_SHDI_* phases. You can manually start or
stop the shadow instance using the commands SAPup startshd or SAPup stopshd. 
To lock or unlock the shadow instance, use the commands SAPup lockshd or SAPup unlockshd.

To perform SPDD modifications, proceed as follows:

1. The Software Update Manager prompts you to confirm that you want to perform a modification adjustment.
2. Add an entry for the shadow instance to the SAP Logon. For the server and system ID, use the values from the original system
For the instance number, use the value you specified in the preparation units for the shadow instance. The default value is the instance number of the original system plus one.
Since the original system is still running if you use the scenario strategy Standard or Downtime-optimized, you can also log on to the shadow instance in transaction SM59 with the RFC connection SAP_UPGRADE_SHADOW_SYSTEM.

3. Log into the shadow instance as user DDIC with the DDIC password of the original system.
Only the users DDIC and SAP* exist in the shadow instance.
4. To set the system change option, call transaction SE06. Perform the following actions
Set the global setting to Modifiable.
Set the change option for the software components to Modifiable or Restricted modifiable.
Set the SAP namespace to Modifiable.
5. Call transaction SU01 to create one or more users to perform the modification adjustment. The new users exist only on the shadow instance and are not copied to the original system.
Release upgrade only: Copy the user DDIC of client 000. SPS update: Since all users of client 000 are copied in the DBCLONE phase to the shadow instance, any of them can be used for the modification adjustment.
6. Log into the shadow instance with one of the new users. Modification adjustment of ABAP Dictionary objects has to be performed in client 000.
7. To determine the ABAP Dictionary objects that must be adjusted, call transaction SPDD.

 Error related to SPDD.

Applied the resolution (cause-B) available in the 1735879 - Error occurs in phase MAIN_POSTP/UEXP_SPAU or MAIN_POSTP/UEXP_SPDD or MAIN_SHDRUN/UEXP_SPD...

Ensure that the following actions are done.

• Stop all production activities.
• Ensure that no scheduled batch jobs can start anymore, and await regular completion of currently running jobs
• Make sure that all users have logged out.
• Isolate the application server instance (AS instance) as described in the SUM guide, section “isolating the instance’’

Check the radio button Actions completed, then click the next button.

Perform the full DB backup and take backup of SUM directory
Select the radio button Backup completed

Click the next button.

#############################Execution- Phase##################################

Possible Errors in this phase:

1. There is a mismatch in the kernel versions and their patch levels in the ASCS and D00 instances.
During the KX_switch phase, D00 upgraded to the latest kernel from the /usr/sap/SID/SUM/abap/exe directory, but ASCS still had the old kernel executables in the instance specific instance.

Please check the dev_disp log in Drive<Name>: \usr\sap\SID\D<nn>\work
Error: Kernel is not compatible with the system

Please check the kernel version and patch level by executing disp+work in instances like ASCS and PAS by going to path
Drive<Name>:\usr\sap\SID\ASCS<nn>\exe

Drive<Name>: \usr\sap\SID\D<nn>\exe

If any mismatch in Kernel version, then execute the below command in the global profile

Drive<Name>: \usr\sap\SID\SYS\profile: sapcpe pf=<Instance Profile>

Check kernel versions now and restart the application

Resolution: Using the sapcpe command, copy the global kernel executable to the directory appropriate to the ASCS instance.

2. Follow the resolution in the below KBA's

2963518 - FAIL: process igswd.EXE IGS Watchdog not running - SAP for Me

3. Manually move the executables to another location

################################Postprocessing- Phase################################

Downtime:

The time during which you cannot use the system productively.
The upgrade strategy determines when downtime begins during an upgrade.

The duration of the downtime depends on several factors, including:
• Hardware
• Modifications and enhancements of SAP Objects
• Customer developments
• Add-on software
• Integration of Support Packages

Business downtime:  The total time during which the SAP system is not available to end users.

Technical downtime:  The time during which the SAP system is not available due to processing routines in the system launched by the SUM tool.

SUM technical downtime: The time during which the system is not available due to processing during the
SUM runs.

UPGANA File: The report generated upon successful execution additionally contains the time statistics of the performed update, for example, the calculated SUM technical downtime value.

This file will be very useful for taking downtime approval and addressing issues in each phase for the sub-sequent systems in the landscape.

For SUM 1.1: <Drive>:\usr\sap\SID\SUM\abap\doc

For SUM 2.0: <Drive>:\usr\sap\SID\SUM\abap\htdoc\analysis

If you encounter any problems when opening the UPGANA file, please refer to the KBA below.

3412465 - UPGANA.XML not visible (blank page) - SAP for Me

Post-Steps:
1.To release the suspended jobs, execute the report BTCTRNS2 in sa38.

2. Triggering SGEN

Select the check box Regenerate after SAP system upgrade option and click on the components

Select all the components and click on the continue.

Click on the continue button.

Select all the application servers and click on the continue button.

In the event that the system has many applications server's setup, you will get the following screen.

A time out error may occur when the generation set is calculated. You must increase the value of the profile option max_wprun_time, which determines the maximum runtime of a dialog procedure.

In our case Generation set contains 432,668 objects.
Select the job and click the flag button (Execute immediately), which will run the RSPARAGENER8M background job.

You can check the status of the job by using the refresh button.

You need to analyze SM37(RSPARAGENER8M), SM21, ST22 and ST11.

Following the execution of SGEN, you can verify table GENSETM.

• If the GENSTATUS flag is X, it indicates that the object's generation has been completed.
• If the GENSTATUS flag is set to S, the object's generation failed due to a syntax errors.
• If the GENSTATUS flag is E, this indicates that the object's generation failed due to a system error.
• If you do not see an entry for the object in GENSETC or GENSETM, it signifies that it was not selected during SGEN execution or that a new SGEN execution was initiated.

432,575 Objects generation has been completed.

91 objects generation failed due to syntax error.

Involve the ABAP and functional teams for better understanding.

There are several reasons that cause syntax error. In any case, you need to identify the component of the objects and query the component about whether the syntax error can be ignored.

• Program does not have a main program defined.
• Old programs that are not used anymore.
• Switch linked to the object is off, and program will not be executed until it is turn on.
• A real syntax error issue.

2 object's generation failed due to a system error.

You need to analyze SM37(Job name: RSPARAGENER8M), SM21, ST22 and ST11.

Possible Errors during the launch of SUM URL:

1. 3475662 - SUM Installing runtime vc_redist.x64.exe - Installation of vcredist finished with errors -...
2. 2913760 - SUM hangs on screen "Process execution in progress" - SAP for Me
3. 2322467 - SUM hangs and cannot continue - SAP for Me
4. 2708098 - SUM UI error: "Forbidden: The user root is not authorized to process the operation Execute...
5. 2475146 - SUM : logon via SAP Host Agent is failing "Unable to locate the requested resource" or "<h...

Possible errors during Extraction Phase:

1. 3154296 - [ASE Error SQL4601] or DBSL error 99 (db code 4002) in SUM - SAP for Me
2. If there is a conflict between the Stack.xml content and the download directory

Possible errors during Configuration Phase:

1. 2594386 - Message "Deviations to calculated level found" in the SUM phase BIND_PATCH - SAP for Me
2. 2509742 - SUM: PREP_EXTENSION/BIND_PATCH: *WARNING* For some components you have not selected enough...
3. 1978965 - How to find the equivalent support package for your release - SAP for Me
4. 2053132 - How to find the import conditions of a Support package or ADD-ON - SAP for Me
5. 2386898 - Message "Equivalence SP level not reached" received while upgrading/installing an Addon in...
6. 1616401 - Parallelism in the Upgrades, EhPs and Support Packages implementations - SAP for Me

Possible errors during Checks Phase:

1. All DB related errors like

• DB free Space
• DB specific parameters
• DB Patch Update
• Implementation of Snotes (Depends on the SUM version and patch level. Sometimes it occurs in the extraction and configuration phases.

Possible errors during Pre-processing Phase:

1. 1901463 - How to unlock the SAP system to perform correction(s) during an upgrade. - SAP for Me
2. 1790486 - SAP_ABA is in an undefined state that is not safe to be upgraded - SAP for Me
3. 1972214 - Error during Upgrade: ABAP batch job fails with error "Log writer aborted with RC 1" - SAP...
4. 2530440 - How to configure DIR_PUT in AL11 for the upgrade directory - SAP for Me
5. Starting and stopping issues with shadow instances.

Possible errors during Execution Phase:

1. 2963518 - FAIL: process igswd.EXE IGS Watchdog not running - SAP for Me
2. Startup issues with system.

For any kind of error during update, please refer to the KBA below

2920107 - How to analyze detailed error log files of SUM during the update/upgrade of a NetWeaver AB...

Thank you for reading. If you find this informative, do not forget to hit a like or leave a comment.

Thanks,

Mani