The holy trinity of Integration

[svenhuberti]

Imagine you have a good old diesel car. It somehow let's you travel from A to B. But it is missing air conditioning, the radio only memorizes 5 stations and the most digital feature is the quartz clock on the dashboard. When arriving, you will be pretty tired, have listened to unwanted country songs and have hit multiple traffic jams. But that is still far better than a horse carriage.

On your way to your way to work it suddenly breaks down. Fixing it would cost more than a new car.

What do you do?

Do you buy another 80s station wagon?
or...
Do you go to the dealership-street in your area and look for a future-proof, fast and easy to use car?

Well - believe it or not - some people seem to favor the first option - also when it comes to an Integration Platform: they would rather continue doing text-file transfer, run scheduled jobs and install local agents everywhere.

Fortunately some customers decide to embrace the future and redesign the communication of their systems from the ground up. This is one of these stories.

The context

Let's take the example of the company Gebr. Heinemann.

They are operating as a retailer and wholesaler with a global and end-to-end supply chain, and need to get integration right to serve their customers in the best way.

Gebr. Heinemann is represented at airports and border crossings, on cruise ships and ferries, in planes and downtown shops, as well as in diplomatic missions and special military zones.

All these locations require different payment options (stationary, mobile, per app...) and specific services (product service, travel service, sales service, ...) and will connect to many different backends (SAP, Visma, ...).

Having all the systems smoothly interact with each other means that there is a need for different communication patterns, that can be combined as needed:

• data massage: data is sent from one system to another and the payload is transformed (converted, enriched, filtered, mapped, ...).
• asynchronous communication: the data is sent out in a "fire and forget" mode, allowing for strong decoupling and real-time communication.
• synchronous communication: the calling system expects a response to his query in order to continue processing.

Instead of using their former "SAP Process Orchestration" to connect all these applications, they turned towards SAP BTP Integration Suite and SAP Integration Suite, Advanced Event Mesh. These SAP BTP services, as well as the Kyma runtime, are the cornerstones of the GH Checkout Platform.

In the next chapters, we will use the example of the price journey to explain the implementation in detail.

Prices and promotional offerings ("Data Massage")

The product prices are stored in an SAP Customer Activity Repository (CAR) system, located on-premises at GH.

These prices are influenced by promotional pricing like bulk discount ("3 for the price of 2"), discounts due to coupons or discounts for card holders. The complete configuration is stored in SAP Omnichannel Promotion Pricing Service (OPPS).

You already guessed it: prices need to be synchronized from SAP CAR to SAP OPPS. This needs data massage and processing, and has been implemented in SAP Integration Suite, Cloud Integration.

Indeed, Cloud Integration provides connectivity, data transformation and orchestration capabilities. It will:

• extract the price data from SAP CAR (or any other backend system),
• perform data transformation (like filtering) using predefined flow steps,
• load the data into SAP OPPS.

Also, using SAP Integration Suite, Cloud Integration, allows GH to stick to their architectural principles: on-premises systems should always be connected through a middleware. This eventually decouples the various backend systems from their shops, simplifying overall operations.

Price distribution to the shops ("Asynchronous communication")

Once the sales prices are available in SAP OPPS, they need to be distributed to both cash registers and electronic shelf labels, physically located in the shops.

Distributing (initial load and updates) more than 300.000 product prices means updating more than 600.000 electronic shelf labels (ESLs) and cash registers on a regular basis. The ESLs and cash registers are managed by specific providers, so GH doesn't actually take care of the last mile communication.

In order to cope with this magnitude of updates, Gebr. Heinemann has chosen to use SAP Integration Suite, Advanced Event Mesh (AEM) which receives price data in dedicated queues and topics, and makes them available to any consumer, like the ESL or cash register SaaS.

Because OPPS is not yet event-enabled (on the roadmap), a microservice has been implemented that generates price changing events, based on deltas in SAP OPPS. These events are then published to AEM. Note that this microservice runs in an SAP BTP Kyma runtime environment.

When publishing the price changes, the topics may look something like this:

price/update/eu/mondelez/chocolate/toblerone/white

Consumers can subscribe to all specific price update events using this dynamic subscription:

price/update/eu/*/chocolate/>

Eventually, both ESL and Cash Register infrastructures listen to these events through the Solace Message Format (SMF) protocol, which offers very comprehensive features like dead-letter queue, dynamic topics, rejection of events, optimized latency, ....

The managed services of the cash registers and ESLs then distribute the new prices to the hardware, without GH having to intervene. 

 

The payment ("Synchronous communication")

Once a customer has decided to buy the special offer, he makes his way to the cash register.

During the payment process, the cash register requests the final price based on promotional offers in real-time from OPPS.

Indeed, a customer could use one or more promotions (like a coupon, his loyalty card and/or a "multibuy offer".) But OPPS makes sure that only the right price is paid by the customer and that impossible combination are not used. This cannot be done by the cash register or the ESL, hence OPPS provides that final price - in real-time - during the payment.

In order to simplify and safeguard the access to the OPPS backend system, SAP Integration Suite, API Management (APIM) comes into play. It sits between the cash register and OPPS and enforces central governance for both runtime and design-time:

• Runtime: Security, monitoring, internal monetisation,
• Design-time: documentation, versioning and standardization of APIs.

This central API governance helps GH to run efficiently by enforcing standards centrally and enforce re-use or APIs.

Summary

At the end of the day, GH has implemented an agile, real-time and resilient Checkout Platform based on SAP Business Technology Platform based on communication patterns specific to hybrid landscapes.

Through the pricing and payment example we dove into the details of the platform. However, there are many more use cases implemented on it, and many more are yet to come. But they all share one thing in common: thanks to the standardization, modular approach and re-usable patterns, requirements can be implemented in a cost-effective and fast way.

Last but not least, SAP Process Integration has no relevance anymore to Gebr. Heinemann: the old platform has been replaced by a modern and future-proof IPaaS providing all necessary communication patterns. Obviously, introducing these new concepts came at the cost of organizational changes, education and new mindset but the result was definitely worth the journey.

Special thanks

A big thank you goes out to Benedikt Althaus and Stefan Franke who prepared and delivered an awesome Integration Roundtable presentation in September 2024, and who spent a lot of time discussing the content of the blog with me!