Downtime Minimizing Techniques for BOBJ 4.2 update on Complex deployed systems

[dpr3776]

Introduction

This document will provide the technics to Minimize the down time during the update/upgrade of a BOBJ system specially on Complex deployments.

Summary

Update/Upgrading to BIPS 4.2 from lower versions is a time consuming process especially when the BOBJ system is configured with complex deployment options and when there are add-on components like Lumira.

Adding to down time There are many misconceptions on how to update existing system with lower version to reach BIPS 4.2, This document will explain the misconceptions, update rules and will provide the down time minimizing technics.

A. Misconceptions explained:

There are misconceptions which leading customers to go for Full installation to update their BOBJ system version instead of using update procedures.

Below picture will explain the root map to update BO system to 4.2 based on the current version of the BO system. If the current version is BIPS 4.1 with any SP Level, we can directly update to 4.2. if the current version is lower than 4.1 we need to choose either Upgrade or Migrations as shown below.



Misconception 1:

Full install is needed to save disk space which is not entirely true.  As with update option also we can easily uninstall Older Support Packs and Patches. However, older ‘add-ons’, cannot be uninstalled.

Misconception 2:

Full install is needed for new features (not true)

• New features can be added by ‘modify’ installation


• Applicable for Languages as well as product components

 

Misconception 3:

Full install is somehow ‘cleaner’

– Against an existing repository (not true)

– Alternative is a new repository and migrate content at huge cost

Misconception 4:

Full install is needed when changing Operating Systems or CMS database vendor (not true)

– Repository and Cluster management alternatives are far more suitable

 

B. Update Rules

1. Update Server software before Client software

All software on a machine must be the same version, no mismatched client-server versions.

Below Picture explains the supported work flow

2. Update to same Support Pack level before updating to same Patch Level

Notice server and client software updated to same level before starting next update

3. Each node must go through the update

• You must update at least 1 node within the cluster


• You must update all(*) other nodes within the cluster


• You don’t have to update nodes that will be deleted

4. Must never mismatch node and repository version

• Can’t just install new software on a new machine and

point it to an old repository

5. Update servers either in series or in Parallel.

In both the cases CMS servers should be update first and then non CMS Servers. after that Add-ons must be installed separately.

• In series

– One server after another

– CMS servers first

• In parallel

– CMS servers first

– Then non-CMS servers

                                                                  C. Reducing downtime

As on today, in conventional procedure to update BOBJ version we need to run installer 2 times per node.  First time to update SP level and second time to update Patch Level.

When we are working on complex deployed systems where we have multiple Application servers in cluster and add-ons Like Lumira installed as shown below.

There are 4 BOBJ (BIPS) CMS application servers in cluster and 2 Lumira (BIPS + Lumira) Non CMS servers connected to that CMS Cluster.



 

To update the BOBJ system as described above, For BIPS in the 6 servers we need to run Installer for 12 times to update BIPS SP level along with Patch level. And for 2 Lumira servers we need to run installer for 4 times to update Lumira Application SP level along with Patch level. In total we need to run installer for 16 times, which is time consuming operation and getting down time is almost impossible for critical Production systems.

 

To overcome the time consuming updates in complex deployments, SAP come up with below 4 technics to minimize the down time as much as possible.

1. Combined Installer


2. Parallel Installation


3. Phase wise Installation


4. Deploy Web Apps Later

   1. Combined Installer

 

Combined Installer will do Combining the Support Pack and Patch installer in a single installer package, means One Step Patch Update Merging Support Pack and Patch.

Combined Installer software package Will be shipped on the discretion of SAP depending on the complexity of the customer deployment. Currently available on-demand and Customer needs to request via raising OSS incident to the following component "BI-BIP-INS".

Note:- In Future SAP planning to provide an “ALL-in-ONE” package.

For the System Explained Above, with Combined Installer Number of installations will reduced to 10 (6 BIPS + 4 Lumira) from 16 times as Explained below.

Without combined installer



 

For BIPS on 6 servers -> 6 Times for BIPS Step1 + 6 times for BIPS steps 2

For Lumira on 2 servers -> 2 Times for Lumira Step1 + 2 Times for Lumira steps 2

In total we need to run Installer for 16 Times.

With Combined installer :



For BIPS on 6 servers -> 6 Times for BIPS in one Step (SP + Patch at a time)

For Lumira on 2 servers -> 2 Times for Lumira Step1 + 2 Times for Lumira steps 2

In total we need to run Installer for 10 Times.

2. Parallel Installation

PARALLEL PATCHING INSTRUCTIONS:

Patch or update the host machines in the distributed or clustered deployments in the following order:

1. Update, in parallel, all CMS host machines. Wait for all machines updating in parallel to finish.


2. Update, in parallel, all non-CMS host machines. Wait for all machines updating in parallel to finish.


3. Once all the updates have finished running, restart all CMS servers in your cluster.


4. The process needs to be repeated for every product that is installed on the machines in the cluster (e.g. BI platform, Explorer, BI platform Client Tools).
   If a product is only installed on non-CMS machines then step 1 can be skipped.

The following rules apply while conducting parallel updates :

• You must wait for all machines updating in parallel to finish before proceeding to the next step.


• Do not restart a host machine until all machines updating in parallel have finished updating, even if a restart is requested by the installation program.


• There must be at least one CMS machine available to the non-CMS host machines that are updating.


• All CMS hosts that are running when you begin updating, and any additional CMS hosts that start during the update, must be available for the entire duration of the update.


• You should not be running any additional installation, maintenance, or server administration workflows that could cause the CMS machines to restart while the update is taking place.

 

As explained in above Instructions with Parallel Installation we can Update all servers in Parallel at time, once for BIPS and second time for Lumira. But as we discussed in Update rules we cannot run installer at a time in all CMS and NON CMS servers.  To over cone this, Create New CMS service at CMC and start it. After update we can remove the newly created CMS from CMC.

For the servers that do not have a CMS service

– Create new CMS and start it (within CMC) – takes just a few moments

– Run the update in parallel on all servers

– Stop CMS and delete it (within CMC)

 

With Out Creating CMS.



 

With Creating CMS



IMPORTANT: READ THIS SECTION ONLY IF YOU ARE UPDATING  FROM  4.0/4.1/4.2  TO  4.2 SP02+ or above :
Starting BI 4.2 SP02 there has been a New License Key Requirement when updating from BI4.0/BI4.1 , which invalidates your license on first CMS update.
This is known to cause issues with Parallel update on CLUSTERED environments due to older license key getting invalidated (as stated in the above blog link).
As such it is very important to follow below additional steps when updating in this scenario:

1. First identify and update a PRIMARY CMS on One Server Node with rest of Server Nodes Stopped & all non-essential services stopped.


2. Request for new 4.2 SP02+ license key, Delete the Old license key and Update the new license key from CMC. Enable all services.


3. Then proceed with rest of parallel update procedures (with rest of nodes stopped) using to the running primary updated CMS.

for more details, please check following SAP KBAs - 
2308674 - "License Key Error : Invalid Key" when adding BI 4.2 keycode after upgrade from BI 4.0 / 4.1
2367507 - After updating SAP BI 4.x to BI 4.2 SP2 or afterwards, one of CMS servers cannot be started in a clustered environment.

2013616 - Patching multiple BI 4.x servers in parallel in a distributed deployment simultaneously? [How-To / Parallel Patching]

For the System we are taking as reference if we go for Combined Installer + Parallel Patching we can reduce the number of installations from 8 to 2

1. 6 servers at a time First time for BIPS (even though we are running installer in 6 servers, the time taken would be of one server as we have opted for parallel)


2. 2 servers at a time second time for Lumira

Note:  If there is version change from “4.0/4.1/4.2  TO  4.2 SP02+ or above” another time of installation will add as discussed earlier for applying License.(total 3 times)

 

 3. Phase-wise Update

Phase wise installation splits the install into 2 steps

Setp1- Uptime Caching

Step2- Actual Installation with Minimized downtime



For Windows Servers after Launching Installer GUI we can select Phase wise Installation as shown below.

• Cache files (no downtime)


• Actual file installation (requires reduced amount of downtime)


• Requires BI 4.2 Support Pack 3 onwards


• Command Line install also supports this feature

For Command line:



For the Complex Deployed System we are taking as reference, with Combined Installer + Parallel Installation + Phase Wise Installation we can further reduce down time required.

4. Deploy Web Apps Later

Web apps will be uninstalled and deployed for every installer run as explained below, which is time consuming operation. When we have add-ons like Lumira, Deploy Web Apps just once at Final run of installer to save downtime.

After each and every install

Web Applications are:

– Uninstalled

– Re-packaged for new content

– Re-deployed

From BI 4.2 Support Pack 3

• Web Applications can be updated just once, on the last install


• If you forget to deploy on the last, just run wdeploy

We can select “Deploy Web Application Later” option as shown below for initial runs of the installer.



Installer Operations without skipping Web Applications deployment.



Installer Operations with Skipping Web Applications deployment for initial runs.

Conclusion

By following the above 4 Technics along with Update rules, we can minimize down time at measurable extent.