
Dear SAP BW enthusiasts!
Recently, one of my customers asked me for possible approaches to implement real-time reporting. During preparation my research also covered “Streaming Process Chains” with surprising results. For this reason I thought it would be a good idea to share “one of the best kept secrets of BW functionality” here in the SAP community. This blog post will focus on the streaming process chain run modes and provides URLs to additional sources in the end. I use the general term “SAP BW” which refers to both SAP BW 7.5 and SAP BW/4HANA (any release).
In general, acquiring data as fast as possible should not be regarded as first choice. Instead, SAP recommends leveraging virtualization and data federation wherever possible. In combination with SAP HANA, BW provides multiple suitable modeling options, mainly based on SAP HANA integration services (SDA, SDI), SAP HANA modeling options (Calculation Views), and Open ODS Views or CompositeProviders as virtual InfoProviders on the BW application side.
Only if these approaches do not meet your business requirements, loading data in "real-time" into SAP BW should be evaluated as a serious alternative. Introduced long ago in 2006 with NetWeaver 7.0, “Real-Time Data Acquisition (RDA)” was the related concept for InfoCubes and DSOs (classic). However, with the introduction of ADSOs, SAP decided to retire RDA and to enhance the existing framework of process chains instead.
Streaming is an operation mode of BW process chains which can be set in the properties of each process chain. When a process chain is running, each step takes a certain amount of time; the interesting question is what actually happens, when the chain is started again -maybe even several times- during this period, and the execution reaches the process which is already active from a previous execution:
In a nutshell, the main value-added the Streaming mode is that it facilitates the frequent data transfer from a source system to SAP BW without scheduling errors and without unreasonable waste of resources. As a result, data can be acquired in SAP BW in almost real-time. It is not 100% real-time, because the realistic latency is measured in minutes in the best case. So do not expect latency times of milliseconds to seconds which are common in SAP SLT replication scenarios for example.
In general, there are different run modes to be distinguished: A Push and two Pull run modes. The main difference between them is whether the consuming system or the source system is triggering the data transfer.
A) The PUSH Run ModeMain Characteristics:
SAP BW receives a trigger from a so-called ODP Daemon in the source system if a new record is generated there. This event starts the immediate execution of the process chain which transfers the new data to the SAP BW InfoProvider.
This is the preferred mode for ODP sources, because the process chain runs only when new data is available in the source. The process chain is scheduled once in streaming mode, the definition of periodic repetitions is not required.
The daemon has to be scheduled as regular job ODQ_DAEMON_CLIENT_xxx in the SAP source system (tr. ODQMON, GoTo/Real-Time Daemon...). By default scheduling, It checks all 15 sec for updates and sends an event to BW if there are any.
Preconditions:
B) The Intelligent PULL Run Mode
Main Characteristics:
SAP BW checks very frequently (each 60 seconds) if a new record is generated in the source. If this check is successful, the process chain is started immediately to transfer the new data to the target InfoProvider. The process chain is scheduled once in streaming mode, the definition of periodic repetitions is not required.
This is the preferred mode for HANA sources, because the process chain runs only when new data is available in the source system.
Preconditions:
In a nutshell, the SDI adapter replicates each change to a generated BW table (/BIC/CMT*) in real-time. You can identify this table in the BW DataSource definition, there is a button called SAP HANA objects. This is the location the streaming process chain checks for new data regularly.
Note: For BW DataSources related to source system HANA_* the Streaming property is always off ("Streaming cannot be triggered by this DataSource"). Do not be confused - I was confused myself quite a long time...it seems like this property only refers to the PUSH mode and thus to ODP sources.
C) The Scheduled PULL Run ModeMain Characteristics:
SAP BW executes a process chain which leverages the streaming framework, no matter if there are new data in the source or not. The Streaming process chain is scheduled as a periodic job and it can be executed very frequently if required (up to each minute). If there are updates available, they will be loaded immediately to SAP BW. On the other hand, if there is no new data, then an empty request is created in the target InfoProvider.
This is the preferred mode, if the source is receiving streaming data, e.g. a record each second. In this case the PUSH or intelligent PULL would not be able to process the new data in the same speed toward SAP BW. So this simple frequent scheduling could be most efficient approach.
Preconditions:
Note: The fact that the preconditions the PUSH mode (A) or the Intelligent PULL mode (B) are not met, does not mean, that you cannot run process chains in streaming mode. You can still leverage the queued task manager framework to run them with high frequency for Delta-capable DataSources. In this case, the process chain needs to be scheduled as a periodic job to pull data from the source in up to 1 minute frequency, instead of the process chain being triggered only when new source data is available.
D) Appendix: Special Mode for Write Interface-enabled ADSOs
Main Characteristics:
As described in my separate blog, there is a new SAP BW/4HANA write interface which enables SAP and non-SAP tools to provision data directly into an ADSO comparable the web service source system in BW 7.x . Technically, the data is pushed into the ADSO inbound table and the data can be processed from there by BW/4HANA standard means.
If this ADSO requires an activation process, this activity can be triggered automatically by means of a Process Chain in ‘Streaming’ mode, because this process chain can be triggered (started) as soon as a request in the inbound queue is closed.
Preconditions:
SUMMARY
The Streaming property is a really cool enhancement of the process chain concept in SAP BW. It provides a flexible toolset, if you really have to load data as fast as possible into a BW InfoProvider. The preferred run modes are the PUSH mode and the Intelligent PULL mode, as they only trigger the DTP execution if new source data is generated.
I recommend following SAP Education offerings if you are eager to learn more about this topic and much much more. We have prepared a sophisticated scenario based on SDI HanaAdapter where you can explore streaming process chains hands-on. Both trainings are currently upgraded to SAP BW/4HANA 2.0 SP07 with more details about streaming process chains. Availability of this latest version expected by mid-2021.
REFERENCES
There are some interesting blog posts in this community which provide use cases for the run modes introduced above:
For additional details refer to the SAP help portal or following SAP notes:
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
User | Count |
---|---|
14 | |
12 | |
12 | |
9 | |
9 | |
7 | |
6 | |
6 | |
6 | |
5 |