cancel
Showing results for 
Search instead for 
Did you mean: 

HXE Server + Apps on Docker not able to start container

Phillip_P
Product and Topic Expert
Product and Topic Expert
0 Kudos

Hi,

Trying to get HXE server + apps working on docker as per the following

https://www.sap.com/uk/developer/tutorials/hxe-ua-install-using-docker-xsa.html

The container doesnt start, the following error:

FAIL: process hdbxsexecagent HDB XS Execution Agent not running

The first possible error I notice is:

nameserver hxehost:39001 not responding

Anyone managed to get this container working on Docker? the Server only version works but not the Server + Apps version.

Former Member
0 Kudos

Hi everybody,

trying to start SAP HANA Express with Docker I also encounter the reported bug and hope somebody can help me:

  • nameserver hxehost:39001 not responding
  • FAIL: process hdbnameserver HDB Nameserver not running.

Any tipps are much appreciated! 🙂

Besh wishes,

Tobi

My Environment:

System: Windows 10

Docker: Community Edition Version 18.06.0-ce-win72 (19098)

Procedure:

docker pull store/saplabs/hanaexpressxsa:2.00.031.00.20180712.1

docker run -ti --privileged centos sysctl fs.file-max=20000000

docker run -ti --privileged centos sysctl fs.aio-max-nr=262144

docker run -ti --privileged centos sysctl kernel.shmmax=1073741824

docker run -ti --privileged centos sysctl kernel.shmmni=524288

docker run -ti --privileged centos sysctl kernel.shmall=8388608

docker run -ti --privileged centos sysctl vm.max_map_count=135217728

all above respond fine.

docker run

-p 39013:39013

-p 39015:39015

-p 39041-39045:39041-39045

-p 1128-1129:1128-1129

-p 59013-59014:59013-59014

-p 39030-39033:39030-39033

-p 51000-51060:51000-51060

-p 53075:53075

-h hxehost

-v c:/Users/braeut_t:/hana/mounts

--ulimit nofile=1048576:1048576

--sysctl kernel.shmmax=1073741824

--sysctl net.ipv4.ip_local_port_range="60000 65535"

--sysctl kernel.shmmni=524288

--sysctl kernel.shmall=8388608

--name express_edition3

store/saplabs/hanaexpressxsa:2.00.031.00.20180712.1

--agree-to-sap-license

--passwords-url file:///hana/mounts/hanapw.json

--proxy-host <MY_HOST>

--proxy-port <MY_PORT>

Result

======== Starting HANA container run script ========

Started at: Fri Aug 10 11:38:38 UTC 2018

Script parameters: --agree-to-sap-license --passwords-url file:///hana/mounts/hanapw.json --proxy-host

HANA version: 2.00.031.00.1528768600

Linux kernel version: 4.9

Processing hooks in folder /hana/hooks/system_check ...

Hook candidates:

/hana/hooks/system_check/100_hxe_check_system_prerequisites

Executing hook /hana/hooks/system_check/100_hxe_check_system_prerequisites ...

Checking /proc/sys prerequisites ...

ok: /proc/sys/kernel/shmmax: 1073741824>=1073741824

ok: /proc/sys/kernel/shmmni: 524288>=4096

ok: /proc/sys/kernel/shmall: 8388608>=8388608

ok: /proc/sys/fs/file-max: 20000000>=20000000

ok: /proc/sys/fs/aio-max-nr: 262144>=131072

WARNING: Unable to access /proc/sys/vm/memory_failure_early_kill (skipping check)

ok: /proc/sys/vm/max_map_count: 135217728>=2048576

ok: /proc/sys/net/ipv4/ip_local_port_range: 60000>=40000

Check succeeded: /proc/sys prerequisites

Checking limits ...

ok: max number of open file descriptors (ulimit -n): 1048576>=1048576

Check succeeded: limits

Execution of hook /hana/hooks/system_check/100_hxe_check_system_prerequisites finished (exit code 0)

Finished execution of hooks in folder /hana/hooks/system_check

Checking version compatibility ...

Downloading password file from file:///hana/mounts/hanapw.json ...

% Total% Received % XferdAverage SpeedTimeTimeTimeCurrent

DloadUploadTotalSpentLeftSpeed

100401004000199900 --:--:-- --:--:-- --:--:-- 19990

Deleting local password file: /hana/mounts/hanapw.json

Entering pre start initialization phase ...

Creating consistency check files ...

Processing hooks in folder /hana/hooks/pre_start_init ...

Hook candidates:

/hana/hooks/pre_start_init/010_license_agreement

/hana/hooks/pre_start_init/010_license_agreement.json (ignored, not an executable regular file)

Hook /hana/hooks/pre_start_init/010_license_agreement requires parameters: AGREE_TO_SAP_LICENSE

Executing hook /hana/hooks/pre_start_init/010_license_agreement ...

License agreement check succeeded (AGREE_TO_SAP_LICENSE=true)

Execution of hook /hana/hooks/pre_start_init/010_license_agreement finished (exit code 0)

Finished execution of hooks in folder /hana/hooks/pre_start_init

Entering pre start phase ...

Processing hooks in folder /hana/hooks/pre_start ...

Hook candidates:

(no executable files detected)

Finished execution of hooks in folder /hana/hooks/pre_start

Change hardware key...

nameserver hxehost:39001 not responding.

Opening persistence ...

run as transaction master

converting topology from cloned instance...

- keeping instance 90

- keeping host hxehost

done.

Finished pre start sequence

Starting up HANA server...

Impromptu CCC initialization by 'rscpCInit'.

See SAP note 1266393.

StartService

OK

OK

Starting instance using: /usr/sap/HXE/SYS/exe/hdb/sapcontrol -prot NI_HTTP -nr 90 -function StartWait 2700 2

StartWait
FAIL: process hdbdaemon HDB Daemon not running

Accepted Solutions (1)

Accepted Solutions (1)

lcasavola
Explorer

Hi guys,I worked out the issue,

I removed the container and restarted it with the added option "--dont-exit-on-error".

In this way I have had time to inspect the system and looked around the step process and finally discovered that the stager command ( actually a java program) wrote other log informations under /hana/shared/HXE/xs/controller_data/controller/tracing/log

In the file controller_0.log I found the useful indications ( a java exception):

[2018-06-26 20:44:39:467]-[Controller.Admin]-[error]-[controller]-[startup]: Could not startup the controller components: /hana/shared/HXE/xs/app_working/stager

java.nio.file.DirectoryNotEmptyException: /hana/shared/HXE/xs/app_working/stager

at sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:500)

at sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:262)

at java.nio.file.Files.move(Files.java:1395)

at com.sap.xs2rt.server.impl.fileaccess.FileAccessBasic.move(FileAccessBasic.java:238)

at com.sap.xs2rt.controller.impl.staging.StagerImpl.reliablyMove(StagerImpl.java:1841)

at com.sap.xs2rt.controller.impl.staging.StagerImpl.<init>(StagerImpl.java:542)

at com.sap.xs2rt.controller.staging.StagerFactory.createSynchronized(StagerFactory.java:59)

at com.sap.xs2rt.controller.impl.ControllerImpl.startUpInternal(ControllerImpl.java:1288)

at com.sap.xs2rt.controller.impl.ControllerImpl.startUp(ControllerImpl.java:789)

at com.sap.xs2rt.controller.impl.ControllerConsole.main(ControllerConsole.java:723)

Hence, the folder /hana/shared/HXE/xs/app_working/stager is not empty. Probably it has created in docker image and the setup process fails because he wants it empty.

In my case I solved the problem just removing such folder and restart all

I don't know exaclty how to restart hana express, in my case I used the command

sapcontrol RestartSystem ALL

Now by accessig the web page https://hxehost:39030/ I get the system is up and running.

Unfortunately I have now another problem, I cannot autenticate to the system, the password provided with the json file seems not to be imported and I don't know which other password to use:

hxeadm@hxehost:/usr/sap/HXE/HDB90/hxehost/trace> hdbsql -i 90 -u SYSTEM -d SYSTEMDB
Password:
* 10: authentication failed SQLSTATE: 28000

Is there a way to reset the password or just provide a default one ?

For example I found onhanaexpress docker 's entrypoint shell ( /run_hana ) a reference to HXEHana1 ( declare system_password=HXEHana1 )

but unfortunateli it doesn't work.

Please Help!

lcasavola
Explorer
0 Kudos

Hi again, I have un update,

acctually HXEHana1 works, after connecting the system asks to enter a new password.

Phillip_P
Product and Topic Expert
Product and Topic Expert
0 Kudos

Thanks for the update Luca, this worked for me too on Fedora Server 28.

Answers (2)

Answers (2)

Octav
Participant
0 Kudos

Hi Luca,

thank you for your input.

Indeed, after deleting the content of stager folder (/hana/shared/HXE/xs/app_working/stager> rm -R ./* ) i had the same issue with login.

I have worked it out doing following steps:

1 - logon to xsa_cockpit using SYSTEM / HXEHana1

The system will prompt you for changing the password-> set up the new one

2 - with the new SYSTEM's pswd logon to your SYSTEMDB:39017 using HANA STUDIO or Eclipse and change the pswd for XSA_ADMIN / XSA_DEV

Best regards

BJarkowski
Active Contributor
0 Kudos

It will be difficult to find the root cause without log files, but what I recommend to check is the available memory. The Server+Apps requires more memory than just Server.

Phillip_P
Product and Topic Expert
Product and Topic Expert
0 Kudos

Hi Bartosz

Most recent attempt at docker run...

----

sudo docker run -p 39013:39013 -p 39015:39015 -p 39041-39045:39041-39045 -p 1128-1129:1128-1129 -p 59013-59014:59013-59014 -p 39030-39033:39030-39033 -p 51000-51060:51000-51060 -p 53075:53075 \ > --ulimit nofile=1048576:1048576 \ > --sysctl kernel.shmmax=1073741824 \ > --sysctl net.ipv4.ip_local_port_range='40000 60999' \ > --sysctl kernel.shmmni=524288 \ > --sysctl kernel.shmall=8388608 \ > --name express_edition \ > store/saplabs/hanaexpressxsa:2.00.030.00.20180403.2 \ > --passwords-url <....> \ > --agree-to-sap-license ========

Starting HANA container run script ======== Started at: Fri Jun 8 11:15:43 UTC 2018 Script parameters: --passwords-url <...> --agree-to-sap-license HANA version: 2.00.030.00.1522210459 Linux kernel version: 4.15 Processing hooks in folder /hana/hooks/system_check ... Hook candidates: /hana/hooks/system_check/100_hxe_check_system_prerequisites Executing hook /hana/hooks/system_check/100_hxe_check_system_prerequisites ... Checking /proc/sys prerequisites ... ok: /proc/sys/kernel/shmmax: 1073741824>=1073741824 ok: /proc/sys/kernel/shmmni: 524288>=4096 ok: /proc/sys/kernel/shmall: 8388608>=8388608 ok: /proc/sys/fs/file-max: 20000000>=20000000 ok: /proc/sys/fs/aio-max-nr: 262144>=131072 ok: /proc/sys/vm/memory_failure_early_kill: 1 ok: /proc/sys/vm/max_map_count: 135217728>=2048576 ok: /proc/sys/net/ipv4/ip_local_port_range: 40000>=40000 Check succeeded: /proc/sys prerequisites Checking limits ... ok: max number of open file descriptors (ulimit -n): 1048576>=1048576 Check succeeded: limits Execution of hook /hana/hooks/system_check/100_hxe_check_system_prerequisites finished (exit code 0) Finished execution of hooks in folder /hana/hooks/system_check Checking version compatibility ... Downloading password file from https:<...> ... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 30 100 30 0 0 42 0 --:--:-- --:--:-- --:--:-- 42 Entering pre start initialization phase ... Creating consistency check files ... Processing hooks in folder /hana/hooks/pre_start_init ... Hook candidates: /hana/hooks/pre_start_init/010_license_agreement /hana/hooks/pre_start_init/010_license_agreement.json (ignored, not an executable regular file) Hook /hana/hooks/pre_start_init/010_license_agreement requires parameters: AGREE_TO_SAP_LICENSE Executing hook /hana/hooks/pre_start_init/010_license_agreement ... License agreement check succeeded (AGREE_TO_SAP_LICENSE=true) Execution of hook /hana/hooks/pre_start_init/010_license_agreement finished (exit code 0) Finished execution of hooks in folder /hana/hooks/pre_start_init Entering pre start phase ... Processing hooks in folder /hana/hooks/pre_start ... Hook candidates: (no executable files detected) Finished execution of hooks in folder /hana/hooks/pre_start Change hardware key...

nameserver hxehost:39001 not responding. Opening persistence ... run as transaction master converting topology from cloned instance... - keeping instance 90 - keeping host hxehost done. Finished pre start sequence Starting up HANA server... Impromptu CCC initialization by 'rscpCInit'. See SAP note 1266393. StartService OK OK Starting instance using: /usr/sap/HXE/SYS/exe/hdb/sapcontrol -prot NI_HTTP -nr 90 -function StartWait 2700 2 08.06.2018 11:17:02 Start OK 08.06.2018 11:18:40

StartWait

FAIL: process hdbxsexecagent HDB XS Execution Agent not running

----

I have 32GB assigned for RAM.

Phillip_P
Product and Topic Expert
Product and Topic Expert
0 Kudos

Started from scratch, created a new Fedora Server image, installed docker and proceeded with the HXE required steps. This time a different service cannot start..FAIL: process hdbxsuaaserver HDB XS UAA Server not running. I also notice the tutorial was updated on Thursday and ports were changed.

Wondering if anyone has successfully run this scenario?

sudo docker run -p 39013:39013 -p 39015:39015 -p 39041-39045:39041-39045 -p 1128-1129:1128-1129 -p 59013-59014:59013-59014 -p 39030-39033:39030-39033 -p 51000-51060:51000-51060 -p 53075:53075 \ > --ulimit nofile=1048576:1048576 \ > --sysctl kernel.shmmax=1073741824 \ > --sysctl net.ipv4.ip_local_port_range='60000 65535' \ > --sysctl kernel.shmmni=524288 \ > --sysctl kernel.shmall=8388608 \ > --name express_edition \ > store/saplabs/hanaexpressxsa:2.00.030.00.20180403.2 \ > --agree-to-sap-license \ > --passwords-url <...> ======== Starting HANA container run script ======== Started at: Sat Jun 9 00:07:17 UTC 2018 Script parameters: --agree-to-sap-license --passwords-url <....> HANA version: 2.00.030.00.1522210459 Linux kernel version: 4.16 Processing hooks in folder /hana/hooks/system_check ... Hook candidates: /hana/hooks/system_check/100_hxe_check_system_prerequisites Executing hook /hana/hooks/system_check/100_hxe_check_system_prerequisites ... Checking /proc/sys prerequisites ... ok: /proc/sys/kernel/shmmax: 1073741824>=1073741824 ok: /proc/sys/kernel/shmmni: 524288>=4096 ok: /proc/sys/kernel/shmall: 8388608>=8388608 ok: /proc/sys/fs/file-max: 20000000>=20000000 ok: /proc/sys/fs/aio-max-nr: 262144>=131072 ok: /proc/sys/vm/memory_failure_early_kill: 1 ok: /proc/sys/vm/max_map_count: 135217728>=2048576 ok: /proc/sys/net/ipv4/ip_local_port_range: 60000>=40000 Check succeeded: /proc/sys prerequisites Checking limits ... ok: max number of open file descriptors (ulimit -n): 1048576>=1048576 Check succeeded: limits Execution of hook /hana/hooks/system_check/100_hxe_check_system_prerequisites finished (exit code 0) Finished execution of hooks in folder /hana/hooks/system_check Checking version compatibility ... Downloading password file from <...> ... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 30 0 30 0 0 41 0 --:--:-- --:--:-- --:--:-- 41

Entering pre start initialization phase ... Creating consistency check files ... Processing hooks in folder /hana/hooks/pre_start_init ... Hook candidates: /hana/hooks/pre_start_init/010_license_agreement /hana/hooks/pre_start_init/010_license_agreement.json (ignored, not an executable regular file) Hook /hana/hooks/pre_start_init/010_license_agreement requires parameters: AGREE_TO_SAP_LICENSE Executing hook /hana/hooks/pre_start_init/010_license_agreement ... License agreement check succeeded (AGREE_TO_SAP_LICENSE=true) Execution of hook /hana/hooks/pre_start_init/010_license_agreement finished (exit code 0) Finished execution of hooks in folder /hana/hooks/pre_start_init Entering pre start phase ... Processing hooks in folder /hana/hooks/pre_start ... Hook candidates: (no executable files detected) Finished execution of hooks in folder /hana/hooks/pre_start Change hardware key... nameserver hxehost:39001 not responding. Opening persistence ... run as transaction master converting topology from cloned instance... - keeping instance 90 - keeping host hxehost done. Finished pre start sequence Starting up HANA server... Impromptu CCC initialization by 'rscpCInit'. See SAP note 1266393. StartService OK OK Starting instance using: /usr/sap/HXE/SYS/exe/hdb/sapcontrol -prot NI_HTTP -nr 90 -function StartWait 2700 2 09.06.2018 00:08:34 Start OK

09.06.2018 00:11:27 StartWait

FAIL: process hdbxsuaaserver HDB XS UAA Server not running

Former Member
0 Kudos

Can you add "-h hxehost" option to the "docker run" command?

sudo docker run -h hxehost ...

jens_seifert2
Explorer
0 Kudos

hello,
i try it with "-h hxehost" same result as before...

BJarkowski
Active Contributor
0 Kudos

Just a hint: log in to the deployed docker and check the log files (you can also attach them here).

Phillip_P
Product and Topic Expert
Product and Topic Expert
0 Kudos

Hi Tai,

My docker run command looks like this:

sudo docker run -h hxehost -p 39013:39013 -p 39015:39015 -p 39041-39045:39041-39045 -p 1128-1129:1128-1129 -p 59013-59014:59013-59014 -p 39030-39033:39030-39033 -p 51000-51060:51000-51060 -p 53075:53075 \

--ulimit nofile=1048576:1048576 \

--sysctl kernel.shmmax=1073741824 \

--sysctl net.ipv4.ip_local_port_range='60000 65535' \

--sysctl kernel.shmmni=524288 \

--sysctl kernel.shmall=8388608 \

--name express_edition \

store/saplabs/hanaexpressxsa:2.00.030.00.20180403.2 \

--agree-to-sap-license \

--passwords-url https://<url>;

and the final error is still:

13.06.2018 19:03:51 StartWait FAIL: process hdbxscontroller HDB XS Controller not running

Also still seeing...

nameserver hxehost:39001 not responding... soon after docker run command is used


I will try look at the xsexeagent.* log files as mentioend in the other thread but does anyone have this working (Server+Apps) successfully in docker? I have tried Ubuntu and Fedora now and the issue is the same. Also seeing a few others with the same error.

Cheers

Phill

jens_seifert2
Explorer
0 Kudos

when the massage "FAIL: progress hdbxcontroller HDB XS Controller not running" appears. The docker container goes in to status "Exited".

lcasavola
Explorer
0 Kudos

Hi Bartosz,

I join the discussion because I am facing the same issu.

Before the error ( which occurred in my case after 3 - 4 minute since the container start) I accessed the container trhough bash and inspect the trace file ( xscontroller.out) and found these messages:

Node version check as user '<sidadm>' returned version: [v8.9.3]
Node version check as user '<sidadm>' returned version: [v8.9.3]
Node version check as user '<sidadm>' returned version: [v8.9.3]
Initializing ConfigStore... OK
Initializing Platform Router..... OK
Initializing BlobStore. OK
Initializing MonitoringStore. OK
Initializing Stager................................................................................................... FAILED!
        Could not startup the controller components: /hana/shared/HXE/xs/app_working/stager
FAILED to start: Could not startup the controller components: /hana/shared/HXE/xs/app_working/stager

so , in my case the culprit is the stager.

Any clues ?

Former Member
0 Kudos

Can you confirm the Docker container size setting? Can you run?

sudo docker info | grep -i "Base Device Size"

Phillip_P
Product and Topic Expert
Product and Topic Expert
0 Kudos

I've set dm.basesize=50G. sudo docker info doesnt return the basesize though but docker-storage.rpmsave in /etc/sysconfig has DOCKER_OPTS="-s devicemapper --storage-opt dm.basesize=50G" set.

After this docker run still gives this error

StartWait

FAIL: process hdbxscontroller HDB XS Controller not running

And

xscontroller.out has this:

Node version check as user '<sidadm>' returned version: [v8.9.3]

Node version check as user '<sidadm>' returned version: [v8.9.3]

Node version check as user '<sidadm>' returned version: [v8.9.3]

Initializing ConfigStore..... OK

Initializing Platform Router... OK

Initializing BlobStore.. OK

Initializing MonitoringStore. OK

Initializing Stager................................................................................................... FAILED!

Could not startup the controller components: /hana/shared/HXE/xs/app_working/stager

FAILED to start: Could not startup the controller components: /hana/shared/HXE/xs/app_working/stager

Former Member
0 Kudos

Default Docker container size is 10GB. This is not enough for server+apps container. The "Base Device Size" from "sudo docker info" must have at least 30GB (50GB recommended). We need to find out why it is not returning 50GB as set in /etc/sysconfig/docker file.

Can you check if the file system of the Docker storage base directory has enough free disk space?

Run this to get Docker storage base directory:

sudo docker info | grep -i root

Run this to get disk usage:

df -h <Docker storage base directory>

Former Member
0 Kudos

By the way, what Docker version are you using? Run:

sudo docker version

Phillip_P
Product and Topic Expert
Product and Topic Expert

Using Fedora 25 I have Base Device Size now at 53GB confirmed in docker info. Had to use the daemon.json file to get it to confirm. Docker version is: 1.12.2. Installed with Fedora 25 and uses Storage Driver: devicemapper.

Other newer docker versions on Fedora dont register the --storage-opt dm.basesize=50G option correctly.

As per the tutorial instructions this should work now but the error at docker run is now:

FAIL: process hdbdaemon HDB Daemon not running

lcasavola
Explorer
0 Kudos

I run docker on a centos7 edition and docker version is :

Engine:
Version: 18.05.0-ce
API version: 1.37 (minimum version 1.12)
Go version: go1.9.5

I also set the suggested configuration under /etc/default/docker:

DOCKER_OPTS="-s devicemapper --storage-opt dm.basesize=50G"

Any way, df for me seems to report the correct size:

[root@hxehost sysconfig]# df -h /var/lib/docker/
File system Dim. Usati Dispon. Uso% Montato su
/dev/mapper/centos-root 50G 33G 18G 66% /

Former Member
0 Kudos

I have found the Docker version 1.12.2 came with Fedora Server 25 doesn't work (FAIL: process hdbdaemon HDB Daemon not running).

The Docker Community Edition (CE) works. Please follow the instruction at https://docs.docker.com/install/linux/docker-ce/fedora/ to uninstall and install the Docker Community Edition (CE).