J2EE database not available via test

[Nibu]

Hi Experts,

I have installed NW 7.0 portal Ehp2 system on Solaris cluster with logical name  . Now when I try to start SAP with the startsap script . it is not starting the database (Oracle) . It just checks whether DB is running and giving message J2EE database not available via test , and asking to check JdbCon.log .

This is Bootstrap log

*************************************************************************************************

com.sap.engine.frame.core.configuration.ConfigurationException: Error while connecting to DB.

        at com.sap.engine.core.configuration.impl.persistence.rdbms.DBConnectionPool.createConnection(DBConnectionPool.java:384)

        at com.sap.engine.core.configuration.impl.persistence.rdbms.DBConnectionPool.<init>(DBConnectionPool.java:132)

        at com.sap.engine.core.configuration.impl.persistence.rdbms.PersistenceHandler.<init>(PersistenceHandler.java:38)

        at com.sap.engine.core.configuration.impl.cache.ConfigurationCache.<init>(ConfigurationCache.java:150)

        at com.sap.engine.core.configuration.bootstrap.ConfigurationManagerBootstrapImpl.init(ConfigurationManagerBootstrapImpl.java:214)

        at com.sap.engine.core.configuration.bootstrap.ConfigurationManagerBootstrapImpl.<init>(ConfigurationManagerBootstrapImpl.java:46)

        at com.sap.engine.bootstrap.Synchronizer.<init>(Synchronizer.java:76)

        at com.sap.engine.bootstrap.Bootstrap.initDatabaseConnection(Bootstrap.java:474)

        at com.sap.engine.bootstrap.Bootstrap.<init>(Bootstrap.java:156)

        at com.sap.engine.bootstrap.Bootstrap.main(Bootstrap.java:946)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

        at java.lang.reflect.Method.invoke(Method.java:331)

        at com.sap.engine.offline.OfflineToolStart.main(OfflineToolStart.java:81)

Caused by: java.sql.SQLException: ORA-01034: ORACLE not available

ORA-27101: shared memory realm does not exist

SVR4 Error: 2: No such file or directory

************************************************************************************************

If I start db manually , then runnig startsap , it runs fine . And also stopsap script , wont stop database .

I checked listener.ora,tnsnames.ora, user profiles, environment variables, permissions etc , nothing found wrong .

When I execute startj2eedb as sidadm , it starts db .

Can you please throw some light on this issue

Regards,

Nibu Antony

[Nibu · Accepted solution]

Hi All,

Thanks for the valuable  suggestions .

Resolved the issue , we tried to  switched to node 2 and there startsap starts the DB .

Then compared everything between node 1 and 2 . Found some difference in /etc/hosts file , where in node 1 , the database hostname was wrongly added by OS team . We did not check host file before , since hostname,ping -a all  were returning the correct DB host logical name , but it seems like startsap first checks the hostfile ,not the DNS .

Changed  /etc/hosts with the logical host name of DB corresponding to IP . Now all works fine , fail over, fail back are fine .

Thanks again for your time.

Regards,

Nibu Antony

[Nibu]

Thanks Reagan for the note .

SAPDBHOST pointing to my DB logical host name . It looks fine .

startsap script is the latest one with the latest kernel .

I just looked at the script and manually added /sapmnt/<SID>/exe/startj2eedb in the beginning of the script and then it is starting the DB with the startsap .  But this is not a work around, I don't wanna edit the standard script that works in all other systems .

Regards,

Nibu Antony

[former_member188883]

Hi Nibu,

Refer SAP note 809477 - startsap/stopsap for SAP WebAs 640, 700, 701, 710, 711, 720

Suggestion here is to download new sapstart and stopsap scripts from marketplace and replace existing ones with the new ones. Few bugs have been fixed here.

Hope this helps.

Regards,

Deepak Kori

[Reagan]

Hello Nibu

Check the default profile, make sure SAPDBHOST was set same as the output of hostname command.

What is the output of the command hostname ? Is the output = SAPDBHOST ?

I just looked at the script and manually added /sapmnt/<SID>/exe/startj2eedb in the beginning of the script and then it is starting the DB with the startsap .

If this works then the problem sits somewhere in the script that starts the DB.

Like you said there are other system that are working fine (hoping that they are on the similar setup) then try to use the script used on the other systems.

Having the latest version of kernel on the system doesn't mean that it is bug free. All kernels patches will have a bug. Try using a different version of start and stop scripts and see if that helps.

The other thing I would try is to remove this : /sapmnt/<SID>/exe/startj2eedb and then set the hostname condition with the virtual hostname under the setHostname() in the start and stop scripts.

Restart the system once this is done and see if that helps.

Regards

RB

[Reagan]

Check this SAP note

1837499 - startsap/stopsap script not able to start or stop DB

If that doesn't help then update the stop and start scripts.

Regards

RB

[Nibu]

Listener is working fine .

And when I execute startj2eedb manually with SIDADM , it starts the database fine .

Regards,

Nibu Antony

[former_member188883]

Hi Nibu,

Manually stop the application and database.

clear all the shared memory segments with sidadm and orasid user.

Then with sidadm execute the command startsap and check the results.

Regards,

Deepak Kori

[Nibu]

Thanks Deepak for the reply.

I tried this already .  The issue here is startsap script is not even trying to start the database. We are not getting any error message during startsap . It just checks whether the database available, and gives the message Database not available , then it simply goes to start SC00 , JC01 etc . But DB is still not started . And so , in bootstrap I get above error and server0 getting stoped .

Regards,

Nibu Antony

[Former Member]

Your Oracle instance was not started up correctly. Check/attach startdb.log file and also check alert_<SID>.log for errors.

[former_member188883]

Hi Nibu,

Please check sidadm user environment whether correct DBNAME and DBHOST are specified.

Similary check in the Default and instance profile whether correct entries for database exists.

You may check your database connectivity using config tool

Hope this helps.

Regards,

Deepak Kori

[Nibu]

Deepak,

Yes, I checked environment varaiables,below are fine

dbs_ora_tnsname

dbms_type

ORACLE_SID

ORACLE_BASE

TNS_ADMIN

DB_SID

ORACLE_HOME

Config tools is connected once I start database manually . Just wondering why startsap not forcing DB to startup or not giving any error as its not started .

Entries for DB exist in DEFAULT and INSTANCE profile same as other systems .

Roman: I am not seeing any error in alert log , it just starting and stopping when I do it manually and those alerts are appearing . Since startsap is not forcing, aler log not giving any sort of error .

Regards,

Nibu Antony

[former_member188883]

Hi Nibu,

Just to confirm again. all the configuration files have logical hostname maintained & not physical hostname. Also check entries in /sapmnt/SID/profile/oracle folder as well.

Regards,

Deepak Kori

[sumit_goel553]

check whether oracle  account is locked or not .

[Nibu]

Yes, it is all logical hostname .

For more clarification on the issue

bash-3.2$ startsap <logical hostname>

Checking <SID> Database

------------------------------

J2EE Database is not available via  test

See logfile /usr/sap/<SID>//JdbcCon.log

Starting Startup Agent sapstartsrv

-----------------------------

Instance Service on host hostname<SID>p1 started

Starting SAP Instance SCS00

------------------------------

Startup-Log is written to /usr/sap/<SID>//startsap_SCS00_hostname<SID>p1.log

Instance on host hostname<SID>p1 started

Starting Startup Agent sapstartsrv

-----------------------------

Instance Service on host hostname<SID>p1 started

Starting SAP Instance JC01

------------------------------

Startup-Log is written to /usr/sap/<SID>//startsap_JC01_hostname<SID>p1.log

Instance on host hostname<SID>p1 started

Starting SAP-Collector Daemon

------------------------------

12:38:20 13.03.2014     LOG: Using PerfDir (DIR_PERF) = /usr/sap/tmp

******************************************************************************

* This is Saposcol Version COLL 21.04 720 - V3.83 64Bit

* Usage:  saposcol -l: Start OS Collector

*         saposcol -k: Stop  OS Collector

*         saposcol -d: OS Collector Dialog Mode

*         saposcol -s: OS Collector Status

* Starting collector (create new process)

******************************************************************************

saposcol on host hostname<SID>p1 started

bash-3.2$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.3.0 Production on Thu Mar 13 12:38:34 2014

Copyright (c) 1982, 2011, Oracle.  All rights reserved.

Connected to an idle instance.

here is the logs

******************************************************************************

JdbcCon.log

Start: Thu Mar 13 12:36:48 AST 2014

Error: Couldn't connect to DB

java.sql.SQLException: ORA-01034: ORACLE not available

ORA-27101: shared memory realm does not exist

SVR4 Error: 2: No such file or directory

*******************************************************************************

[sumit_goel553]

can you check whether listener is working properly or not

[Former Member]

could you check which hostname (logical & physical) maintained in bootstrap config. server name should be as same as mentioned in listener net config