Timeouts on connections that have been inactive for a while (Connection terminated abnormally; client socket shut down)

[Former Member]

We have a system running on SQL Anywhere 12 which was working fine for long time, but recently we started to get random timeouts on connections that have been inactive for a while. I excluded possibility that client application somehow messes up the connection, and the problem is now reproducible with a simple test program that does this:

• Open Connection to Sybase server (CT library)
• Run the loop: wait N minutes, confirm that connection is alive by running simple SQL, double the value of N

When N reaches 64 (minutes), SQL starts to time out on the call to ct_results(). Message callback gets called with the following message:

Client Library error: severity(0) number(63) origin(2) layer(1)
ct_results(): user api layer: internal Client Library error: Read from the server has timed out.

This message gets repeated every 15 seconds for up to several minutes, while ct_results() call is in progress. This is also reflected on the database server, which logs the following message in its own log:

... Connection terminated abnormally; client socket shut down
... Disconnected TCPIP client's AppInfo: PID=NNN;EXE=process[NNN].host

That message is repeated several times. At that point, the connection is still visible to sa_conn_info, but attempts to execute any SQL on it result in the timeout. We have no idle timeout enabled. Server starts with -ti0 option, and liveness timeout doesn't apply to TDS connections, from what I understand.

Server version is SQL Anywhere Network Server Version 12.0.1.3457.

I've seen this bug description: link:http://search.sybase.com/kbx/changerequests?bug_id=743662 which mentions similar problem, but the error message is not exactly the same.

Has anyone run into similar problem ? Is this a bug in the DB engine or client library ?

[VolkerBarth]

As 12.0.1.3457 is a rather old build: May it be possible for you to try a newer EBF - cf. the (rather) current readme overview...?

(I don't claim that a newer EBF will fix that issue.)

[Breck_Carter]

FWIW none the timeout-related fixes described in the readme for 12.0.1.4231 seem to apply to this symptom... they talk mostly about the relay server, MobiLink, UltraLite and so on.

Perhaps the Open Client software changed, or something on the client side, or perhaps the Sybase Open Client configuration needs to be changed.

[VolkerBarth]

Server starts with -ti0 option, and liveness timeout doesn't apply to TDS connections, from what I understand.

Just to add: I was asking myself whether the connection itself may have set the IDLE connection parameter (which would "overwrite" the server's -ti 0 setting), but alas:

TDS connections (including jConnect) ignore the SQL Anywhere Idle connection parameter.

So that cannot be a cause either...

[Former Member]

I upgraded to 12.0.1.4127, and the problem is still happening. This particular system is not a production one, and runs Developer edition of SQL Anywhere. I am not sure if it's functionally different from non-developer version.

[Former Member]

You may have to open up a support issue or start a new thread supplying more details about your test program.

From what little I know about CT-Library the error

ct_results(): user api layer: internal Client Library error: Read from the server has timed out.

can only come about because you are waiting on a result set to come back from your query AND you have exceed your CS_TIMEOUT setting. If your SQL is a batch that loops and waits without returning that could be the perfectly normal and expected result.

The repeat of that same exception every 15 seconds is likely has more to do with the CS_TIMEOUT being set to 15 seconds than anything else. The fact that the same error is coming back, could be because it is the same error for the same reason ... and most likely will be related back to the way your code loops and make repeated calls to ct_results().

As to how that correlates to the server abnormal disconnect, I would need to get exact clock timings on when these get logged (both sides) but that too could be after the application gave up; and thus perfectly normal.

If the two, possibly, normal interpretations apply to your situation you could be just looking at a problem that is slowing the server down. That could be contention, excessive concurrency, just loading up the system enough to exceed the hardware and platform capabilities or something else that is contributing here.

And finally, if all you are using is the CT-Library applications to diagnose this then you may be missing the server-busy scenarios that could be leading up to this. Whenever this is occurring can you verify if the server is responsive to connections and requests from dbisql or scjview; and is not busy when monitored by Performance Monitor or other admin tools.

Of course if it is not server and it is not client then you could be looking at a glitch at the network level.

Best of luck.