BiWhy – Web Intelligence Processing Server (WEBI) Memory

[ilya_ulyanov]

Disclaimer:
BiWhy is available again for FREE to everyone on biwhy.net

BOBJ Sizing Guide:
‘Enable Memory Analysis’ is now unchecked by default when you perform a new installation from BI 4.2 onwards... We now recommend this is disabled as your default setting, enabling the process to use as much memory as it needs and without the need to reclaim memory or generate any related error due to memory limits becoming breached.

But I strongly recommend always keeping it enabled to protect the host memory from being exhausted by WEBI.

The exhaustion can happen for two reasons: 

Memory leak.

I once saw WEBI consume 100 GB during an Excel export. I am sure this specific issue was fixed long ago, but nobody can guarantee that the code is fully leak free.

Real load:

• Multiple concurrent users
• A few users running very large reports
• A combination of both

Think about Memory Thresholds the same way you think about the Java Xmx parameter.

If thresholds are reached, the system will handle the situation gracefully. In the worst case, WEBI will be restarted, which impacts only the users assigned to this specific WEBI.
If thresholds are not set, the same scenario may cause the entire host to run out of memory, which can lead to system-down situations or worse. What do you prefer?

Here is a conservative formula for calculating memory allocation for WEBI:
WEBI_MAX = Available_RAM – Sum of Xmx (APS and others) – 2 GB for OS – 2 to 4 GB for CMS (or more, depending on actual usage) – Xmx for Tomcats if they run on the same host – other major memory consumers on the server.

My recommendation: set thresholds high enough that they are never reached during normal operation, but still low enough to avoid endangering the host memory.

Why does the Sizing Guide recommend switching off Memory Analysis?

I do not know for sure, but the most likely reason is related to thresholds configuration.
Thresholds must be customized for each system. They cannot be set correctly by default, because default values would be either too high or too low.

BOBJ administrators need to understand what Lower, Upper, and Maximum Thresholds mean and how to connect the warning messages seen by end users to these configured parameters, so they can adjust thresholds or upscale the system instead of opening incidents.

A possible improvement on the SAP side would be to explicitly state in warning messages that they originate from the Memory Analysis configuration.

Threshold meaning from Admin Guide:
If this property is enabled then the following properties will be active and recognized by the server:

• Memory Maximum Threshold
• Memory Upper Threshold
• Memory Lower Threshold

If the Memory Lower Threshold limit is reached, the server swaps out inactive documents onto the hard disk allocating additional memory for the documents which are active. Each user is allowed to have up to one active document instead of Maximum Documents per User.

If this Memory Upper Threshold is reached, the following server actions will take place in order to free resources and protect the server:

• The server will reject new connections and new client calls. Only the option to Save Web Intelligence documents will be allowed. Users that request an action will receive a Server Busy message, and they will be notified that they should save any pending changes.
• The server will turn on system cleanup to free enough resources so that the amount of allocated memory is below the limit set by the Memory Upper Threshold property.
• The server tries to close read-only documents.
• If not enough memory was freed during system cleanup then the server will begin to close documents that are in Editing mode. The server will begin to close documents based on the LIFO protocol; the most recent active document will be purged from memory first. The server will continue to close documents until a safe level is reached; this level is based on the following calculation: Memory Upper Threshold - (20%*(Memory Upper Threshold)). For example, if the Memory Upper Threshold property is set to 4.5GB then the safe level would be: 4.5GB - 20%*4.5GB = 3.6GB
• The server cannot close documents when a client call is running. Any document that is refreshed or exported to another format or any other time consuming operation will not be closed when the server reaches this threshold. If the server cannot recover enough memory and is still above the Memory Upper Threshold, it restarts.

If the Memory Maximum Threshold limit is reached, all current operations abort. All client calls will be terminated. Once the a call terminates, the corresponding document will be closed.

Or shortly:

When the server's process memory is above the Memory Upper Threshold, the only operation that is allowed is saving documents.

When the process memory is above the Memory Maximum Threshold, all operations stop and fail.

Screenshots:

Content:

• BOBJ Metrics: cumulative WEBI usage of 430 GB, with recommendations on WEBIs per host
• BOBJ Metrics: individual WEBI usage reaching 74 GB
• OS-level monitoring: cumulative WEBI allocation of 450 GB

Here is a production system with 800 sessions. Cumulative WEBI memory usage reaches 430 GB, with individual WEBI processes using up to 52 GB.
You can see multiple WEBI servers per host. SAP recommends one WEBI per host, or two for failover if the system has only a few WEBI nodes. Although in the past we observed during VTOs that in some cases multiple WEBIs perform better, for simplicity and to reduce communication overhead I recommend fewer WEBIs unless there is a clear reason to do otherwise.

Here is a VTO run with 1,400 sessions, with a WEBI process consuming up to 74 GB

OS-level monitoring shows that on a system with only 46 active sessions, WEBI on two nodes cumulatively consumes about 450 GB.
However, I do not believe this reflects real memory usage – it is rather allocated memory. This is mainly due to the customer running 10 WEBI servers per host, with Memory Analysis switched off and 1 TB of RAM available per processing node.