DBSQL_SQL_ERROR CX_SY_OPEN_SQL_DB SQL code: -10807

[avijish]

Hi Experts,

Lately I have been facing below short dumps every now and then

when I check the HANA side, I see the indexserver has a restart time in line with these dumps, maybe because of memory exhaustion

Indexserver logs has below messages around start time but not sure if its related

revalidateObjectsInternal: Running with non-serializable transaction mode, this might result in unexpected behavior for runtime plugins! MULTIDB Transaction Error - Database Mismatch: Got token for invalid database [5340]{-1}[-1/-1] 2020-09-10 20:22:37.520627 e trans_token TransToken.cpp(02097) : Watermark mismatch

the dump details as below

I tried to find a note or solution for this behaviour but no help, have you faced this before and can possibly suggest how to troubleshoot this.

Thanks in advance!

[deepak-kalra]

Hi avijish,

Looks like one of the services of SAP HANA was unavailable or restarted.

My suggestion is, when you do not understand any kind of Hana database traces - always look for SAP note "2380176 - FAQ: SAP HANA Database Trace"

For error msg: """"trans_token TransToken.cpp(02097) : Watermark mismatch""""
You can see same error and its explanation in above mentioned SAP note 🙂 For more details on this error - you can also review and follow SAP note "2689722 - Error message stating "Watermark mismatch [...] one of nodes may have restarted" in index...

If you are using SAP BW on HANA then also review below mentioned notes.
1) 2621464 - SAP BW and HANA connection hanging
2) 2749387 - SQL error "SQL code: -10807" occurred while accessing table "TABLE_NAME"

I hope it helps

Regards
Deepak

[avijish]

Hi Deepak,

Thanks for providing the SAP notes, I will analyse these and update the outcome. Strangely I didnt find these notes when I searched with the keyword "Watermark Mismatch".

[avijish]

Hi Deepak,

Ths suggested SAP Notes havent helped to progress unfotunately. The sai dump starts occurring and at the HANA DB level I could see the index server being restarted, tried to analyse the indexserver logs but all there is the said watemark mismatch error and other infos and warnings. Have raised this with SAP Support. waiting for their response.

[deepak-kalra]

Hi Vijish An,

It will be great if you can share the indexserver log file when it was restarted.

Regards
Deepak

[avijish]

Hi Deepak, Attaching the indexserver file for info. Tme frame should be between 14 Sept 00:00 - 10:00 AMindexserver-hostname30240420.txt

I tried to analyse the same but there isnt much info.

Regards,

Vijish

indexserver-hostname30240420.txt

[avijish]

Hi,

I tried to find top memory consumers in HANA DB using SAP Note 1999997 - FAQ: SAP HANA Memory and did some housekeeping on the largest tables. Havent faced the dump since then however will update the solution here if this helps anyone facing similar issue.

The concern is I wasnt unable to find direct root cause of the problem which was causing the indexserver service to restart and fix the same. The SAP note is quite comprehensive to traverse and find the exact cause.

Regards,

Vijish