This blog post describes how you can connect to an Amazon MQ service which is a managed message broker service for Apache ActiveMQ for configuring asynchronous message processing using the AMQP (Advanced Message Queuing Protocol) adapter. The AMQP adapter is available for SAP Cloud Integration customers with the 08-December-2019 release. Kindly read the blog from mandy.krimmel to know more about the configuration, prerequisites and limits of this new AMQP adapter.
Note: Please note that this blog talks about the Non-SAP integration and the screenshots and configuration options given below might differ in visual appearance and technical capabilities due to the future upgrades of the Amazon MQ service.
Prerequisite: Amazon MQ Message Broker Setup
Before you can use Amazon MQ, you must complete the following steps:
Click on the Get started button which is visible inside the Create brokers tile.
Select deployment and storage type: Based on your requirement, choose the appropriate options. For this blog, I have choosen the options as shown in the given screenshot and then click on the Next button.
Configure Settings: Based on your requirement, do the required configurations. For this blog, I have configured the minimum options like Broker Name, Broker instance type, Username and Password for ActiveMQ Web Console access as shown in the given screenshot and then click on the Create broker button.
This will then start creating the broker which takes about 15 minutes.Refresh the screen to check the status change.
Once the status changes to Running, click on the newly created broker.
Scroll down to the Connections, this will lists the ActiveMQ Web Console URL and wire-level protocol endpoints including AMQP. By default all the inbound traffic is resricted. To be able to access your broker's ActiveMQ Web Console URL or wire-level protocol endpoints, you must configure security groups to allow inbound traffic.
Enable connections to your broker
In the broker Details section, under Security and network, choose the name of your security group.
The Security Groups page of the EC2 Dashboard is displayed. From the security group list, choose your security group.
At the bottom of the page, choose Inbound rules tab, and then click on the Edit inbound rules button.
In the Edit inbound rules dialog box, we need to add the following two rules:
A rule for an Active MQ Web Console access from your system IP.
Choose Add Rule.
For Type, leave Custom TCP selected.
For Port Range, type the ActiveMQ Web Console port i.e. 8162.
For Source, select anything from the three options i.e. Custom, Anywhere and My IP based on from where you want to be able to access the ActiveMQ Web Console.
A rule for an AMQP endpoint acess from your SAP Cloud Integration tenant. Based on the Cloud Integration tenant region, we need to add all the IP range of that region as per the given help documentation.
Choose Add Rule.
For Type, leave Custom TCP selected.
For Port Range, type the AMQP endpoint port i.e. 5671.
For Source, leave Custom selected and then type the IP ranges of your SAP Cloud Integration tenant region. For neo-eu2, add the following IP ranges: 22.214.171.124/24, 126.96.36.199/24 and 188.8.131.52/24
Save the changes by clicking on the Save rules button.
Your broker can now accept inbound connections. Click on the ActiveMQ Web Console URL to access it.
Also do the connectivity test from your SAP Cloud Integration tenant to the AMQP server.
Open your SAP Cloud Integration tenant web tooling and navigate to the Monitor tab.
In the Manage Security section, click on the Connectivity Tests tile.
Open the AMQP tab, provide the AMQP details as given in the screenshot and then click on the Send button.
If you get javax.jms.JMSException: connection timed outexception, this mean the inbound rules to the AMQP endpoint has not been setup properly. Kindly again follow the Step 4 carefully.
If you get javax.jms.JMSException: General SSLEngine problem exception, this mean the AMQP endpoint is accessable but the SAP Cloud Integration tenant is not able to validate Amazon MQ Server certificate.
In order to validate the Amazon MQ Server certificate, uncheck the Validate Server Certificate checkbox and click on the Send button. This will display the Amazon MQ Server certificates chain, download it and upload the root certificate of the chain in your SAP Cloud Integration tenant keystore.
Once you upload the root certificate of the chain in SAP Cloud Integration keystore successfully, test the AMQP connectivity again and check the Validate Server Certificate checkbox. This time you should get the successfull response.
Create a queue in Amazon ActiveMQ message broker
To be able to connect to queues or topics in the message broker, you have to create queues and/or topics in the message broker. Follow the following steps to create a queue in Amazon ActiveMQ:
Click on the Manage ActiveMQ broker link which is available on the landing page of ActiveMQ Web Console. Provide Username and Password which you have set earlier while creating the message broker and then click on the Sign in button.
Click on the Queues tab and create a queue with a name Success_Queue. Kindly note, as per the default broker configuration, all inactive queues gets deleted automatically after 10 minutes. An ‘inactive’ queue is one that has had no messages pending and no consumers connected for some configured period of time.
Configure Asynchronous Messaging Using AMQP Adapter
In many cases integration scenarios have to be decoupled asynchronously between sender and receiver message processing to ensure that a retry is done from the integration system/message broker rather than the sender system.
Follow the steps described below to setup the sample scenario using Amazon ActiveMQ message broker and AMQP adapter in SAP Cloud Integration.
Setup Scenario With Asynchronous Decoupling
To configure the decoupling of inbound and outbound message processing you need to configure two processes:, one process to receive the inbound message and store it in the Amazon ActiveMQ queue and a second process to trigger the message from the Amazon ActiveMQ queue to the receiver backend. The blog describes the configuration using two separate integration flows.
Configure the Integration Flow Receiving the Message
The first integration flow will be configured to receive the message via any inbound adapter. In this sample setup we use the HTTP adapter to receive the student records in xml format, process it with Iterating Splitter and then move individual student record to Amazon ActiveMQ queue.
Configure the AMQP Receiver Channel
Create the integration flow with the inbound channel required by your scenario, and use the AMQP adapter with TCP protocol as the outbound adapter. You have to configure the AMQP endpoint details as given in the screenshot. To learn more about each of its configuration options, kindly read the blog
Deploy the Integration Flow
Now you can deploy the integration flow. In this case, the queue with name Success_Queue has been already created in the message broker but whatever queue name you provide in the AMQP adapter that will be created automatically in the message broker.
Configure the Integration Flow doing the Retry
To consume the messages from the Amazon ActiveMQ queue, you configure a second integration flow with a AMQP sender channel and the outbound adapter needed for your scenario. In this sample configuration we use the HTTP adapter.
Configure the AMQP Sender Channel
Create the integration flow with the outbound channel required by your scenario, and use the AMQP adapter with TCP protocol as the inbound adapter. You have to configure the AMQP endpoint details as given in the screenshot. Use the same queue name used in the receiving integration flow.To learn more about each of its configuration options, kindly read the blog
If an error occurs during the processing of the consumed message in Cloud Integration, the message is not removed from the messaging system, but is retried again immediately. There is no option to configure a delay in retry processing in the AMQP adapter because this is not supported by the AMQP protocol. To learn more about this, kindly read the blog
Deploy the Integration Flow
Now you can deploy the integration flow.
Execute the scenario
From Postman, make a POST call to SAP Cloud Integration HTTPS endpoint with student records:
It will enqueue 5 messages in Amazon ActiveMQ Success_Queue.
Other integration flow will automatically then start polling the messages and POST it to the REST mock service in beeceptor
With this blog post, you can now easily setup event-driven architecture and configure asynchronous message processing using external Amazon MQ message broker and SAP Cloud Integration.This way you can ensure that a retry is done from the integration system/message broker rather than the sender system.
Event-driven architectures are ideal for improving agility and moving quickly.You can easily move from any message broker that uses these standards to Amazon MQ because you don’t have to rewrite any messaging code in your applications.
Thank you for reading this blog post, happy learning!