Index Server failed to start after Hana SPS 04 data snapshot recovery

[former_member631426]

Hi,

I am recovering the HANA SPS 04 ( single tenant) database from snapshot backup. The recovery is successful but the index server failed to start up with the following error. This is intermittent error.

26073]{-1}[-1/-1] 2019-09-21 01:45:45.699382 e Logger RecoveryHandlerImpl.cpp(02620) : Missing log, cannot recover starting from 0x1a79002, restart pos 0x20e61c0

[25963]{-1}[-1/-1] 2019-09-21 01:45:47.926319 f PersistenceLayer PersistenceController.cpp(00735) : startup failed exception 1: no.3000280 (DataAccess/impl/PersistenceManagerImpl.cpp:3190) TID: 25963

Setup : Hana SPS 04 single tenant both /hana/data and /hana/log are in separate VMDKs.

Note : Before running the recovery the /hana/data contains the data snapshot files. The /hana/log was not modified. I am running this outside hana studio as command line script using the command recover data using snapshot . Also the clear log is set. Here is the error I see indexserver_alert_*.trc
26073]{-1}[-1/-1] 2019-09-21 01:45:45.699382 e Logger RecoveryHandlerImpl.cpp(02620) : Missing log, cannot recover starting from 0x1a79002, restart pos 0x20e61c0
[25963]{-1}[-1/-1] 2019-09-21 01:45:47.926319 f PersistenceLayer PersistenceController.cpp(00735) : startup failed exception 1: no.3000280 (DataAccess/impl/PersistenceManagerImpl.cpp:3190) TID: 25963
Missing log, recovery stopped at log pos 0x1a79002
exception throw location:
1: 0x00007f89d98626be in DataAccess::PersistenceManagerImpl::replayOrClearLog(bool, bool, bool, unsigned long&, unsigned long&)+0x65a at PersistenceManagerImpl.cpp:3190 (libhdbdataaccess.so)
2: 0x00007f89d985af1c in DataAccess::PersistenceManagerImpl::replayImpl(bool)+0x178 at PersistenceManagerImpl.cpp:4941 (libhdbdataaccess.so)
3: 0x00007f89dbbd48ae in PersistenceLayer::PersistenceSystem::start(NameServer::ServiceStartInfo const&, bool, bool)+0xa8a at PersistenceSystem.cpp:561 (libhdbpersistence.so)
4: 0x00007f89dbb8b00e in PersistenceLayer::PersistenceFactory::startPersistence(NameServer::ServiceStartInfo&, bool, bool, DataAccess::TablePreloadWriteCallback*, ltt::smartptr_handle<DataAccess::DisasterRecoveryPrimaryCallback>, DataAccess::DisasterRecoveryESCallback*)+0xba at PersistenceFactory.cpp:538 (libhdbpersistence.so)
5: 0x00007f89f8d1886e in PersistenceController::startup(PersistenceLayer::PERSISTENCE_MODE, NameServer::ServiceStartInfo*, bool, DataAccess::TablePreloadWriteCallback*, DataAccess::TablePreloadReadCallback*, Backup::RecoverCbc_Federation*)+0x5da at PersistenceController.cpp:678 (libhdblogger.so)


Please help how to debug further.

thank you,
VJ

[mohamedamin]

Usually you see "startup failed exception 1: no.3000280" after os crash , The log area is inconsistent. try to Recover the system

2956393 - SAP HANA service does not start with error "Missing log, recovery stopped at log pos"

[HakanHaslaman]

Please return the missing/moved logs to the original location (See note 2352487, which outlines a similar scenario) or alternatively recover to a point in time before the issue.