How to Implement Load Balancing in SAP CAP Node.js on BTP?

[fabi2295]

What are the steps to implement load balancing for a Node.js application using SAP Cloud Application Programming (CAP) on SAP BTP to ensure high availability and avoid service downtime during high traffic flows, so that no service becomes unavailable or overloaded?

[Beyhan · Accepted solution]

yes, the Cloud Foundry platform has a routing component which distributes the requests to an app instances by default in round robin algorithm. Additionally, I would recommend you to check the High Availability at Platform and Application Level section in the SAP BTP help portal. 

[fabi2295 · Accepted solution]

Incredible, I got it. I ran some tests, and it's indeed balancing automatically. I also enabled the autoscaler with the CPU configuration so that when it reaches 5%, it adds another instance.

Test: I triggered a script that makes 4 * 10,000 calls to the service, and it automatically added or removed instances based on CPU usage.

Config:

{
	"instance_min_count": 1,
	"instance_max_count": 5,
	"scaling_rules": [
		{
			"metric_type": "cpuutil",
			"breach_duration_secs": 60,
			"threshold": 5,
			"operator": ">",
			"adjustment": "+1"
		},
		{
			"metric_type": "cpuutil",
			"breach_duration_secs": 60,
			"threshold": 5,
			"operator": "<",
			"adjustment": "-1"
		}
	]
}

3 CPU's

[gregorw]

Hi,

during normal operation you can use:

Application Autoscaler 

to add additional instances when needed or shut down instances when not needed anymore. But be aware that in the Cloud Foundry enviornment the costs are calculated based on the maximum memory used at one time in the month.

During deployment you should check out:

Blue-Green Deployment of Multitarget Applications 

Best regards,
Gregor

[Beyhan]

Hi,

please check this cost calculation example for the Cloud Foundry Environment to get an understanding about how cost are calculated. Additionally, Cloud Foundry has now support for the so called readiness health check, which is helpful in situation when an application instance is overloaded and cant process any new requests. A readiness health check can deregistered an overloaded app instance from request processing until it can take new requests again.

[fabi2295]

Hi, gregorw, thanks for responding! So, if I just add new instances to my SRV, will the load balancing happen automatically?

Example:

[fabi2295]

Solution