Java Stack Does not start

[Former Member]

Hello Guys ,

I have got a serious issue with my Java stack. I am unable to start Java stack sucessfully. when i check trace files, I find that it was started but eventually stopped due to some error.

Here I am posting bootstrap trace file to investigate issue and to help me out.

-

---

trc file: "/usr/sap/N4S/DVEBMGS01/work/dev_bootstrap_ID14704950", trc level: 1, release: "700"

-

---

node name : bootstrap_ID14704950

pid : 12621

system name : N4S

system nr. : 01

started at : Tue Jun 15 13:56:20 2010

arguments :

arg[00] : /usr/sap/N4S/DVEBMGS01/exe/jlaunch

arg[01] : pf=/usr/sap/N4S/SYS/profile/N4S_DVEBMGS01_n4shost

arg[02] : -DSAPINFO=N4S_01_bootstrap

arg[03] : pf=/usr/sap/N4S/SYS/profile/N4S_DVEBMGS01_n4shost

arg[04] : -DSAPSTART=1

arg[05] : -DCONNECT_PORT=32910

arg[06] : -DSAPSYSTEM=01

arg[07] : -DSAPSYSTEMNAME=N4S

arg[08] : -DSAPMYNAME=n4shost_N4S_01

arg[09] : -DSAPPROFILE=/usr/sap/N4S/SYS/profile/N4S_DVEBMGS01_n4shost

arg[10] : -DFRFC_FALLBACK=ON

arg[11] : -DFRFC_FALLBACK_HOST=localhost

[Thr 182897318208] Tue Jun 15 13:56:20 2010

[Thr 182897318208] *** WARNING => INFO: Unknown property [instance.box.number=N4SDVEBMGS01n4shost] [jstartxx_mt. 841]

[Thr 182897318208] *** WARNING => INFO: Unknown property [instance.en.host=n4shost] [jstartxx_mt. 841]

[Thr 182897318208] *** WARNING => INFO: Unknown property [instance.en.port=3200] [jstartxx_mt. 841]

[Thr 182897318208] *** WARNING => INFO: Unknown property [instance.system.id=1] [jstartxx_mt. 841]

**********************************************************************

JStartupReadInstanceProperties: read instance properties [/usr/sap/N4S/DVEBMGS01/j2ee/cluster/instance.properties]

-> ms host : n4shost

-> ms port : 3900

-> OS libs : /usr/sap/N4S/DVEBMGS01/j2ee/os_libs

-> Admin URL :

-> run mode : NORMAL

-> run action : NONE

-> enabled : yes

**********************************************************************

**********************************************************************

Used property files

-> files [00] : /usr/sap/N4S/DVEBMGS01/j2ee/cluster/instance.properties

**********************************************************************

**********************************************************************

Instance properties

-> ms host : n4shost

-> ms port : 3900

-> os libs : /usr/sap/N4S/DVEBMGS01/j2ee/os_libs

-> admin URL :

-> run mode : NORMAL

-> run action : NONE

-> enabled : yes

**********************************************************************

**********************************************************************

Bootstrap nodes

-> [00] bootstrap : /usr/sap/N4S/DVEBMGS01/j2ee/cluster/instance.properties

-> [01] bootstrap_ID14704900 : /usr/sap/N4S/DVEBMGS01/j2ee/cluster/instance.properties

-> [02] bootstrap_ID14704950 : /usr/sap/N4S/DVEBMGS01/j2ee/cluster/instance.properties

**********************************************************************

**********************************************************************

Worker nodes

-> [00] ID14704900 : /usr/sap/N4S/DVEBMGS01/j2ee/cluster/instance.properties

-> [01] ID14704950 : /usr/sap/N4S/DVEBMGS01/j2ee/cluster/instance.properties

**********************************************************************

[Thr 182897318208] JLaunchRequestQueueInit: create named pipe for ipc

[Thr 182897318208] JLaunchRequestQueueInit: create pipe listener thread

[Thr 1074792800] JLaunchRequestFunc: Thread 1074792800 started as listener thread for np messages.

[Thr 1085282656] WaitSyncSemThread: Thread 1085282656 started as semaphore monitor thread.

[Thr 182897318208] Tue Jun 15 13:56:21 2010

[Thr 182897318208] SigISetDefaultAction : default handling for signal 17

[Thr 182897318208] NiInit3: NI already initialized; param 'maxHandles' ignored (1;202)

[Thr 182897318208] CPIC (version=700.2005.12.02)

[Thr 182897318208] [Node: server0 bootstrap] java home is set by profile parameter

Java Home: /usr/lib/java

[Thr 182897318208] JStartupICheckFrameworkPackage: can't find framework package /usr/sap/N4S/DVEBMGS01/exe/jvmx.jar

**********************************************************************

JStartupIReadSection: read node properties [bootstrap_ID14704950]

-> node name : server0 bootstrap

-> node type : bootstrap

-> node execute : yes

-> jlaunch parameters :

-> java path : /usr/lib/java

-> java parameters : -Djco.jarm=1

-> java vm version : J2RE 1.4.2 IBM J9 2.2 Linux amd64-64 j9xa64142ifx-20051125 (JIT enabled)

-> java vm vendor : IBM J9SE VM (IBM Corporation)

-> java vm type : <default>

-> java vm cpu : amd64

-> heap size : 203M

-> root path : /usr/sap/N4S/DVEBMGS01/j2ee/cluster

-> class path : ./bootstrap/launcher.jar

-> OS libs path : /usr/sap/N4S/DVEBMGS01/j2ee/os_libs

-> main class : com.sap.engine.offline.OfflineToolStart

-> framework class : com.sap.bc.proj.jstartup.JStartupFramework

-> registr. class : com.sap.bc.proj.jstartup.JStartupNatives

-> framework path : /usr/sap/N4S/DVEBMGS01/exe/jstartup.jar:/usr/sap/N4S/DVEBMGS01/exe/jvmx.jar

-> parameters : com.sap.engine.bootstrap.Bootstrap ./bootstrap ID014704950

-> debuggable : yes

-> debug mode : no

-> debug port : 60000

-> shutdown timeout : 120000

**********************************************************************

[Thr 1086335328] JLaunchIStartFunc: Thread 1086335328 started as Java VM thread.

**********************************************************************

JHVM_LoadJavaVM: VM Arguments of node [server0 bootstrap]

-> stack : 1048576 Bytes

-> arg[ 0]: exit

-> arg[ 1]: abort

-> arg[ 2]: vfprintf

-> arg[ 3]: -Djco.jarm=1

-> arg[ 4]: -Dsys.global.dir=/usr/sap/N4S/SYS/global

-> arg[ 5]: -Dapplication.home=/usr/sap/N4S/DVEBMGS01/exe

-> arg[ 6]: -Djava.class.path=/usr/sap/N4S/DVEBMGS01/exe/jstartup.jar:/usr/sap/N4S/DVEBMGS01/exe/jvmx.jar:./bootstrap/launcher.jar

-> arg[ 7]: -Djava.library.path=/usr/lib/java/jre/bin:/usr/lib/java/bin/../jre/bin:/usr/lib/java/jre/bin/j9vm:/usr/lib/java/jre/bin:/usr/sap/N4S/DVEBMGS01/exe:/usr/sap/N4S/DVEBMGS01/exe:/usr/sap/N4S/DVEBMGS01/exe:/tmp/sapinst_exe.11133.1149865079:/usr/sap/N4S/SYS/exe/run:/sapdb/programs/lib:/usr/lib:/usr/sap/N4S/DVEBMGS01/j2ee/os_libs:/usr/sap/N4S/DVEBMGS01/exe:/usr/sap/N4S/DVEBMGS01/exe:/usr/sap/N4S/DVEBMGS01/exe:/usr/sap/N4S/SYS/exe/run:/sapdb/programs/lib

-> arg[ 8]: -Dmemory.manager=203M

-> arg[ 9]: -Xmx203M

-> arg[ 10]: -DLoadBalanceRestricted=no

-> arg[ 11]: -Djstartup.mode=BOOTSTRAP

-> arg[ 12]: -Djstartup.ownProcessId=12621

-> arg[ 13]: -Djstartup.ownHardwareId=A0340344147

-> arg[ 14]: -Djstartup.whoami=bootstrap

-> arg[ 15]: -Djstartup.debuggable=yes

-> arg[ 16]: -DSAPINFO=N4S_01_bootstrap

-> arg[ 17]: -DSAPSTART=1

-> arg[ 18]: -DCONNECT_PORT=32910

-> arg[ 19]: -DSAPSYSTEM=01

-> arg[ 20]: -DSAPSYSTEMNAME=N4S

-> arg[ 21]: -DSAPMYNAME=n4shost_N4S_01

-> arg[ 22]: -DSAPPROFILE=/usr/sap/N4S/SYS/profile/N4S_DVEBMGS01_n4shost

-> arg[ 23]: -DFRFC_FALLBACK=ON

-> arg[ 24]: -DFRFC_FALLBACK_HOST=localhost

-> arg[ 25]: -DSAPSTARTUP=1

-> arg[ 26]: -DSAPSYSTEM=01

-> arg[ 27]: -DSAPSYSTEMNAME=N4S

-> arg[ 28]: -DSAPMYNAME=n4shost_N4S_01

-> arg[ 29]: -DSAPDBHOST=n4shost

-> arg[ 30]: -Dj2ee.dbhost=n4shost

**********************************************************************

[Thr 1086335328] *** ERROR => JHVM_LoadJavaVM: Cannot create Java VM (rc=-4) [jhvmxx_mt.c 542]

[Thr 1086335328] *** ERROR => Cannot load Java VM (<default>) (rc=-1) [jlnchxxi_mt. 739]

[Thr 1086335328] SigISetIgnoreAction : SIG_IGN for signal 17

[Thr 1086335328] JLaunchCloseProgram: good bye (exitcode = -1)

[Former Member]

instance properties

ID14704900.ClassPath=./bin/boot/boot.jar:./bin/system/bytecode.jar:.

ID14704900.Debuggable=no

ID14704900.JLaunchParameters=

ID14704900.JavaParameters=-Djava.security.policy=./java.policy -Djava.security.egd=file:/dev/urandom -Djco.jarm=1 -Xms203M

ID14704900.JavaPath=/usr/lib/java

ID14704900.LogName=dispatcher

ID14704900.MainClass=com.sap.engine.boot.Start

ID14704900.MaxHeapSize=203

ID14704900.Name=dispatcher

ID14704900.Parameters=

ID14704900.RootPath=/usr/sap/N4S/DVEBMGS01/j2ee/cluster/dispatcher

ID14704900.Type=dispatcher

ID14704950.ClassPath=./bin/boot/boot.jar:./bin/boot/jaas.jar:./bin/system/bytecode.jar:.

ID14704950.DebugMode=no

ID14704950.DebugPort=50121

ID14704950.Debuggable=no

ID14704950.JLaunchParameters=

ID14704950.JavaParameters=-Djava.security.policy=./java.policy -Djava.security.egd=file:/dev/urandom -Dorg.omg.CORBA.ORBClass=com.sap.engine.system.ORBProxy -Dorg.omg.CORBA.ORBSingletonClass=com.sap.engine.system.ORBSingletonProxy -Djavax.rmi.CORBA.PortableRemoteObjectClass=com.sap.engine.system.PortableRemoteObjectProxy -Djco.jarm=1 -verbose:gc -Xms1024M -Dorg.omg.PortableInterceptor.ORBInitializerClass.com.sap.engine.services.ts.jts.ots.PortableInterceptor.JTSInitializer

ID14704950.JavaPath=/usr/lib/java

ID14704950.LogName=server0

ID14704950.MainClass=com.sap.engine.boot.Start

ID14704950.MaxHeapSize=1024

ID14704950.Name=server0

ID14704950.Parameters=

ID14704950.RootPath=/usr/sap/N4S/DVEBMGS01/j2ee/cluster/server0

ID14704950.Type=server

bootstrap.ClassPath=./bootstrap/launcher.jar

bootstrap.JavaParameters=-Djco.jarm=1

bootstrap.JavaPath=/usr/lib/java

bootstrap.MainClass=com.sap.engine.offline.OfflineToolStart

bootstrap.MaxHeapSize=256

bootstrap.Name=bootstrap

bootstrap.Parameters=com.sap.engine.bootstrap.Bootstrap ./bootstrap ID0147049

bootstrap.RootPath=/usr/sap/N4S/DVEBMGS01/j2ee/cluster

bootstrap.Type=bootstrap

bootstrap_ID14704900.ClassPath=./bootstrap/launcher.jar

bootstrap_ID14704900.JLaunchParameters=

bootstrap_ID14704900.JavaParameters=-Djco.jarm=1

bootstrap_ID14704900.JavaPath=/usr/lib/java

bootstrap_ID14704900.MainClass=com.sap.engine.offline.OfflineToolStart

bootstrap_ID14704900.MaxHeapSize=203

bootstrap_ID14704900.Name=dispatcher bootstrap

bootstrap_ID14704900.Parameters=com.sap.engine.bootstrap.Bootstrap ./bootstrap ID014704900

bootstrap_ID14704900.RootPath=/usr/sap/N4S/DVEBMGS01/j2ee/cluster

bootstrap_ID14704900.Type=bootstrap

bootstrap_ID14704950.ClassPath=./bootstrap/launcher.jar

bootstrap_ID14704950.JLaunchParameters=

bootstrap_ID14704950.JavaParameters= -Djco.jarm=1

bootstrap_ID14704950.JavaPath=/usr/lib/java

bootstrap_ID14704950.MainClass=com.sap.engine.offline.OfflineToolStart

bootstrap_ID14704950.MaxHeapSize=1024

bootstrap_ID14704950.Name=server0 bootstrap

bootstrap_ID14704950.Parameters=com.sap.engine.bootstrap.Bootstrap ./bootstrap ID014704950

bootstrap_ID14704950.RootPath=/usr/sap/N4S/DVEBMGS01/j2ee/cluster

bootstrap_ID14704950.Type=bootstrap

instance.box.number=N4SDVEBMGS01n4shost

instance.en.host=n4shost

instance.en.port=3200

instance.enabled=yes

instance.install.dir=/usr/sap/N4S/DVEBMGS01/j2ee

instance.ms.host=n4shost

instance.ms.port=3900

instance.osLibsPath=/usr/sap/N4S/DVEBMGS01/j2ee/os_libs

instance.runAction=NONE

instance.runMode=NORMAL

instance.system.id=1

[Former Member]

Hi,

Please check your default and instance profiles and make sure that all parameters for hostname and port numbers are correct.

Then, if everything is ok there, try and reboot the server and then start the system.

By the way, do you know what was being done on the system before restart? I mean someone could have changed some parameters. Can you find out?

Regards,

Shitij

[former_member432219]

Hi Shiv K Sharma

The AS Java is not starting since there is a problem in bootstrapping for server0. This is evident from the dev_jcontrol file you pasted into the thread here.

You should focus on the analysis of the log files in the work directory (/usr/sap/N4S/DVEBMGS01/work) that have bootstrap and ID14704950 in their name to determine why this element bootstrapping phase fails.

Use note 997510 "Troubleshooting J2EE Engine 6.40/7.0 Bootstrap Issues" as a guide when analyzing these logs

[TomCenens]

Dear

Can you please perform the following command as user <sid>adm:

java -fullversion

Please post the output on the forum.

It might be a problem with a bad JVM version or that your environment variables are pointing to the wrong JVM.

Can you tell us what recent action(s) took place ?

Is this a fresh install, a system copy or have one or more components been patched (either SAP or OS) ?

Kind regards

Tom