- SAP Managed Tags:
In my documentation I’ll explain how to setup and configure SAP Hana system replication (HSR) with 3 system for two different case.
• In the first scenario I will use configure the replication with 2 Hana system in the same datacenter by using HanaSR (SLES12) and connect another Hana system from an outside datacenter.
• In second scenario add a QA instance on my secondary node in order to run for a HSR Cost Optimized solution.
For my setup I will SAP Hana revision 122 on SLES 12 virtual machine running on Vmware 6.0
Disclaimer: My deployment is only for test purpose, I make the security simple from a network perspective in order to realize this configuration and use open source software.
High Availability for SAP Hana
SLES Release note for SAP Application 12.1
Novell SAPHanaSR 10162 - SAPHanaSR provides resource agents to control the SAP HANA database in high...
For my deployment I will use 3 virtual server running SLES 12.1 image with the following specification for each
Ram : 64
CPU : 2 virtual quad-core
Disk : 1 local disk of 30gb for /usr/sap
NFS : 1 nfs mount point of 70gb for /hana
Detail Architecture
From a detail point of view, on my primary site the SAP Hana Revision 122 will be installed on two SLES 12.1 server respectively and will be setup for HSR in SYCH mode.
The replication will use a dedicated network, both server will be installed will operate on cluster mode with the SAPHanaSR deployed in order to automated failed over of Hana in case of server failure or else.
On my secondary site another Hana instance will be deploy and will be setup for HSR with the primary site on a dedicated network as well.
Install SAP Hana and setup SLES cluster in the primary datacenter
My Two Hana instance are installed and running, I can start the HSR configuration.
Because I want to use a specific IP for my replication, the following “system_replication_hostname_resolution” needs to be maintain in the global.ini on all site in order to mapped the dedicated IP to the current hostname
Once done check if the log mode is set to normal and take a backup the primary node
I can now enable the primary site
And register my secondary, we can see that I’m using the dedicated ip for replication
And run some check
The replication setup completed I will now take care of the cluster setup/configuration of the SLES 12.1 server.
Before to run the cluster setup, i install the SAPHanaSR add-on in order to be able to use it for the phase. By default the package is already in place from the server repository but is not the latest version
From the SLES portal the patch 17 is available at the time I’m doing the documentation
Download and install the package, once done I setup the SLES cluster.
To have a valid cluster an SBD device needs to be configured for STONITH mechanism, to do so I have add an additional shared iscsi drive on both server but no mounted it (/dev/sdd)
I initiate the device by the following command
I install the necessary SLES HA package on both server
Before to initiate the cluster configuration on both node make sure to have the “watchdog” device in place as well as the process running
And join the secondary node to the cluster
Note: before to add the second node copy the “corosync.conf” from the primary node
Once done from HAWK web interface, check the cluster health
You can also run the command “crm_mon –r” to check the cluster status
My Hana HSR in place and my SLES cluster green, I can now start the SAPHanaSR configuration.
I’ll use the HAWK web interface to make the setup, this configuration can also be done from command line if you would like to script the process
Review the parameter and hit ‘Apply”
Note: I’ll leave the parameter highlighted to the value “no”, what does it mean?
Defines whether a former primary should be automatically registered to be secondary of the new primary.
With this parameter you can adapt the level of system replication automation.
If set to false the former primary must be manually registered. The cluster will not start this SAP HANA RDBMS till it is registered to avoid double primary up situations.
Now if I come back on the Status of my cluster I can see the resources added for Hana
And from the dashboard the current state of my Hana node
Failed over testing
I add my cluster Hana IP in my hana studio to check on which node I’m working on
For a test purpose I’ll kill my primary node in order my trigger the automatic failed over on my secondary node
I can see the error message from HAWK that my primary resource is not up
Note: make sure to clean up the error by invoking “crm resource cleanup <resource name> <host>”
Once the failed over completed we can see my secondary node as the primary
And from the dashboard I can confirm it as well
I choose earlier to not set the automated_register option the “true”, so in order to put the node back in the HSR I need to register it as secondary and start the node
Once my former primary node registered as secondary and started, I do a refresh on my studio and I can see that I’m operate now on my secondary node which replicate my first node
My test completed I can start the work on my secondary datacenter
Install SAP Hana secondary datacenter and setup SAP Hana replication with secondary datacenter
My cluster and HSR ready I install my Hana system in my second datacenter
Once installed I set the parameter for system_replication_hostname
Once completed I have to make my secondary node on primary site as the source site for replication
And finally register my secondary site against my primary site with the second node
Once the third node started we can check from the studio that my secondary is replicating to the secondary site on third node
From the cockpit view we can see it also
Failed over testing
My setup is completed I can perform and final test which will consist of:
1Bring down the primary node
2) Set the former third node in secondary site as source site for replication
3) Register former primary node as third node against secondary site node
I kill the primary node first and wait until the takeover take place on the secondary node
Now I set my third node in secondary site as source site for replication
And finally register the primary node as third node against secondary site node
The result should look like this (vhana02 -> vhana03 -> vhana01)
Note that because the former primary node is now registered as the third node against the secondary site, the HAWK error will always show up for the particular resources until the instance is added back to the cluster
As of today this mechanism is only supported for two node cluster with Hana.
Scenario 2 – using 2 datacenter with Cost Optimized Option
Install SAP Hana QA instance on my secondary node
For this particular scenario I will use the automatic failed over mechanism between 2 datacenter but will install a QA instance on my secondary node in order to use the cost optimize option.
Several condition needs to be meet before to make the installation:
- Table pre-load is turned off in the secondary system
- The secondary system disk infrastructure needs to be doubled
- The non-production systems are stopped with the takeover to the production secondary
- The memory sizing must be adapted and the global allocation limit should be set on the QA and PRD instance
Before to run the installation I’ll shutdown my secondary node so it will avoid memory bottleneck
Once installed i need to create a SAPDBCONTROL user
Provide a key to the user
And finally install the hana client
Setup HAWK for Cost Optimize scenario
My QA instance installed and configured, from HAWK interface I need to add the QA instance as a new resource as part of my cluster and create new constraint such as “location”, “colocation” and “order”
From HAWK start the add a new resource and select “Primitive”
And set the following parameters
Once done I can see my new resource in the cluster resource
Now I select “Add constraint” from the resource panel and start by the location
And add the parameters
Note: I had error during the process while clicking on the create button such as
If you experience the same error run the command manually by creating and loading the config file with all the constraint
And load it
My configuration completed I can make my failed over testing to check the behavior
Failed over testing
In order to check the behavior of the QA instance during the failed over, I will kill my primary and make some check
I do run a “tail –f” command on the /var/log/message and I can see my QAS system stopped correctly
And I check from HAWK, it’s down too
My configuration is completed for my both case scenario.
• In the first scenario I will use configure the replication with 2 Hana system in the same datacenter by using HanaSR (SLES12) and connect another Hana system from an outside datacenter.
• In second scenario add a QA instance on my secondary node in order to run for a HSR Cost Optimized solution.
For my setup I will SAP Hana revision 122 on SLES 12 virtual machine running on Vmware 6.0
Disclaimer: My deployment is only for test purpose, I make the security simple from a network perspective in order to realize this configuration and use open source software.
In order execution
Scenario 1 – using 2 datacenter
- Install SAP Hana and setup SLES cluster in the primary datacenter
- Setup SAP Hana automate failed over (SAP HanaSR) in the primary datacenter
- Failed over testing
- Install SAP Hana secondary datacenter and setup SAP Hana replication with secondary datacenter
- Failed over testing
Scenario 2 – using 2 datacenter with Cost Optimized Option
- Install SAP Hana QA instance on secondary node
- Setup HAWK for Cost Optimize scenario
- Failed over testing
Guide used
- SAP Hana Administration Guide SP10
- SAP HANA virtualization and TDI_V2
- SUSE Linux Enterprise High Availability ExtensionSLEHA
Note used
- 1999880 - FAQ: SAP HANA System Replication
- 2303243 - SAP HANA Multitier System Replication – supported replication modes between sites
- 1944799 - SAP HANA Guidelines for SLES Operating System Installation
- 2070786 - SAP systems on VMware vSphere: minimum SAP and OS requirements
- 1788665 - SAP HANA Support for Virtualized Environments
- 1625203 - saphostagent / sapdbctrl for NewDB
Link used
High Availability for SAP Hana
SLES Release note for SAP Application 12.1
Novell SAPHanaSR 10162 - SAPHanaSR provides resource agents to control the SAP HANA database in high...
Overview Architecture
For my deployment I will use 3 virtual server running SLES 12.1 image with the following specification for each
Ram : 64
CPU : 2 virtual quad-core
Disk : 1 local disk of 30gb for /usr/sap
NFS : 1 nfs mount point of 70gb for /hana
Detail Architecture
From a detail point of view, on my primary site the SAP Hana Revision 122 will be installed on two SLES 12.1 server respectively and will be setup for HSR in SYCH mode.
The replication will use a dedicated network, both server will be installed will operate on cluster mode with the SAPHanaSR deployed in order to automated failed over of Hana in case of server failure or else.
On my secondary site another Hana instance will be deploy and will be setup for HSR with the primary site on a dedicated network as well.
Scenario 1 – Use 2 different datacenter
Install SAP Hana and setup SLES cluster in the primary datacenter
My Two Hana instance are installed and running, I can start the HSR configuration.
Because I want to use a specific IP for my replication, the following “system_replication_hostname_resolution” needs to be maintain in the global.ini on all site in order to mapped the dedicated IP to the current hostname
Once done check if the log mode is set to normal and take a backup the primary node
I can now enable the primary site
And register my secondary, we can see that I’m using the dedicated ip for replication
And run some check
The replication setup completed I will now take care of the cluster setup/configuration of the SLES 12.1 server.
Before to run the cluster setup, i install the SAPHanaSR add-on in order to be able to use it for the phase. By default the package is already in place from the server repository but is not the latest version
From the SLES portal the patch 17 is available at the time I’m doing the documentation
Download and install the package, once done I setup the SLES cluster.
To have a valid cluster an SBD device needs to be configured for STONITH mechanism, to do so I have add an additional shared iscsi drive on both server but no mounted it (/dev/sdd)
I initiate the device by the following command
I install the necessary SLES HA package on both server
Before to initiate the cluster configuration on both node make sure to have the “watchdog” device in place as well as the process running
And join the secondary node to the cluster
Note: before to add the second node copy the “corosync.conf” from the primary node
Once done from HAWK web interface, check the cluster health
You can also run the command “crm_mon –r” to check the cluster status
Setup SAP Hana automated failed over (SAP HanaSR) in the primary datacenter
My Hana HSR in place and my SLES cluster green, I can now start the SAPHanaSR configuration.
I’ll use the HAWK web interface to make the setup, this configuration can also be done from command line if you would like to script the process
Review the parameter and hit ‘Apply”
Note: I’ll leave the parameter highlighted to the value “no”, what does it mean?
Defines whether a former primary should be automatically registered to be secondary of the new primary.
With this parameter you can adapt the level of system replication automation.
If set to false the former primary must be manually registered. The cluster will not start this SAP HANA RDBMS till it is registered to avoid double primary up situations.
Now if I come back on the Status of my cluster I can see the resources added for Hana
And from the dashboard the current state of my Hana node
Failed over testing
I add my cluster Hana IP in my hana studio to check on which node I’m working on
For a test purpose I’ll kill my primary node in order my trigger the automatic failed over on my secondary node
I can see the error message from HAWK that my primary resource is not up
Note: make sure to clean up the error by invoking “crm resource cleanup <resource name> <host>”
Once the failed over completed we can see my secondary node as the primary
And from the dashboard I can confirm it as well
I choose earlier to not set the automated_register option the “true”, so in order to put the node back in the HSR I need to register it as secondary and start the node
Once my former primary node registered as secondary and started, I do a refresh on my studio and I can see that I’m operate now on my secondary node which replicate my first node
My test completed I can start the work on my secondary datacenter
Install SAP Hana secondary datacenter and setup SAP Hana replication with secondary datacenter
My cluster and HSR ready I install my Hana system in my second datacenter
Once installed I set the parameter for system_replication_hostname
Once completed I have to make my secondary node on primary site as the source site for replication
And finally register my secondary site against my primary site with the second node
Once the third node started we can check from the studio that my secondary is replicating to the secondary site on third node
From the cockpit view we can see it also
Failed over testing
My setup is completed I can perform and final test which will consist of:
1Bring down the primary node
2) Set the former third node in secondary site as source site for replication
3) Register former primary node as third node against secondary site node
I kill the primary node first and wait until the takeover take place on the secondary node
Now I set my third node in secondary site as source site for replication
And finally register the primary node as third node against secondary site node
The result should look like this (vhana02 -> vhana03 -> vhana01)
Note that because the former primary node is now registered as the third node against the secondary site, the HAWK error will always show up for the particular resources until the instance is added back to the cluster
As of today this mechanism is only supported for two node cluster with Hana.
Scenario 2 – using 2 datacenter with Cost Optimized Option
Install SAP Hana QA instance on my secondary node
For this particular scenario I will use the automatic failed over mechanism between 2 datacenter but will install a QA instance on my secondary node in order to use the cost optimize option.
Several condition needs to be meet before to make the installation:
- Table pre-load is turned off in the secondary system
- The secondary system disk infrastructure needs to be doubled
- The non-production systems are stopped with the takeover to the production secondary
- The memory sizing must be adapted and the global allocation limit should be set on the QA and PRD instance
Before to run the installation I’ll shutdown my secondary node so it will avoid memory bottleneck
Once installed i need to create a SAPDBCONTROL user
Provide a key to the user
And finally install the hana client
Setup HAWK for Cost Optimize scenario
My QA instance installed and configured, from HAWK interface I need to add the QA instance as a new resource as part of my cluster and create new constraint such as “location”, “colocation” and “order”
From HAWK start the add a new resource and select “Primitive”
And set the following parameters
Once done I can see my new resource in the cluster resource
Now I select “Add constraint” from the resource panel and start by the location
And add the parameters
Note: I had error during the process while clicking on the create button such as
If you experience the same error run the command manually by creating and loading the config file with all the constraint
And load it
My configuration completed I can make my failed over testing to check the behavior
Failed over testing
In order to check the behavior of the QA instance during the failed over, I will kill my primary and make some check
I do run a “tail –f” command on the /var/log/message and I can see my QAS system stopped correctly
And I check from HAWK, it’s down too
My configuration is completed for my both case scenario.
1 Comment
