How do I debug "Invalid response from the HTTP server" in 12.0.1.4231?

[Breck_Carter]

How do I diagnose and debug SQLCODE=-988 "Invalid response from the HTTP server" from a SQL Anywhere 12.0.1.4231 procedure defined as TYPE 'HTTP:GET'?

The code works with versions 10 and 11, and fails with 12, 16 and 17.

The version 16 and 17 messages may be red herrings due to the changeover to OpenSSL; I am ONLY INTERESTED in version 12.0.1.

Here is the code, edited after testing to obfuscate all names:

CREATE PROCEDURE ppp (
  in aaa integer,
  in bbb long varchar default '',
  in ccc long varchar default '' )
result( Attribute long varchar,
  Value long varchar )
 url 'https://xxx.yyy.com/Push/status'
type 'HTTP:GET'
CERTIFICATE 'file=C:\\\\xxx\\\\.yyy.cer';

SELECT * FROM ppp ( 20664587, '', 'Test456' );

-- @@VERSION '12.0.1.4231' fails...

There was an error reading the results of the SQL statement.
The displayed results may be incorrect or incomplete.
Invalid response from the HTTP server
SQLCODE=-988, ODBC 3 State="HY000"

-- @@VERSION '11.0.1.3158' works ok...

Attribute,Value
Status,HTTP/1.1 200 OK
Body,{"status":true,"error":null}
Content-Type,application/json
Date,Sun, 06 Mar 2016 15:26:36 GMT
Connection,close
Content-Length,28
Server,Microsoft-IIS/8.0

Summary of tests results...
SQL Anywhere 10.0.1.4310 - works OK
SQL Anywhere 11.0.1.3158 - works OK
SQL Anywhere 12.0.1.4231 - fails - Invalid response from the HTTP server
SQL Anywhere 16.0.0.2127 - fails - The secure connection to the remote host failed: The TLS handshake failed, error code 20
SQL Anywhere 17.0.0.1062 - fails - The TLS handshake failed, error code 0x14090086

[VolkerBarth]

Anything in the WebClientLog (i.e. dbsrv12 -zoc)?

FWIW, v12.0.1.4231 also incorporates the OpenSSL changeover, so that may be of importance here, as well.

[Breck_Carter]

> also incorporates the OpenSSL changeover

oooo I did not know (or remember?) that.

and thanks for the -zoc tip

[Vlad]

What, what was shown in logs, when you enabled this parameter? 🙂 Please lift the veil of mystery.

[Breck_Carter]

The "dbsrv12 -zoc weblog.txt" produced the following unhelpful output:

[connid = 6, 03/07 08:21:03.642]
[connid = 6, 03/07 08:21:03.642, REQUEST]
GET /Push/status?InspID=20664587&RepID=&Status=Test456 HTTP/1.0
Connection: close
ASA-Id: b31452fd615b4775be4f0cf7b8cfa95b
Accept-Charset: windows-1252, UTF-8, *
Date: Mon, 07 Mar 2016 13:21:02 GMT
Host: rest.xxx.com
User-Agent: SQLAnywhere/12.0.1.4231
[connid = 6, Error: socket closed by peer]
[connid = 6, socket closed]

[Breck_Carter]

Volker's comment about OpenSSL was helpful because it led to this:

    ================(Build #4086  - Engineering Case #761751)================

    The OpenSSL vulnerability known as Heartbleed impacted some components of 
    SQL Anywhere software as follows:
     - SQL Anywhere Server when using TLS (Transport Layer Security) communications 
    and/or HTTPS web services, though only to the networks that can access the 
    server. Calling external web services over HTTPS from the database server 
    were also affected.

     - MobiLink Server when using TLS and/or HTTPS communications, though only 
    to the networks that can access the MobiLink server.

     - Relay Server Outbound Enabler

    Affected Versions (note that all platforms were impacted by the vulnerability):
     - SQL Anywhere 12.0.1 builds 3994-4098
     - SQL Anywhere 16.0 builds 1690-1880

    This vulnerability has been resolved by replacing the OpenSSL libraries 
    with corrected versions. Once this SP has been applied, regenerate any certificates 
    that were being used, and then change any passwords/keys associated with 
    SQLA web service calls or TLS authentication.

[Vlad]

So, am I correct that the issue you had was related to "weak" certificates? After you regenerated them, the error disappeared.

Ok, so it seems that -zoc wasn't helpful at all. The remote server rest.xxx.com simply closed the connection with no reason.

[VolkerBarth]

that -zoc wasn't helpful at all

Hm, I would think it at least showed no response at all in contrast to showing an invalid response (however that may have looked, I dunno...). It might have been helpful to compare whether v16/v17 would have the same log entries here, particularly as they seem to give a more helpful error message.

But these are rather post mortem thoughts, as Breck has hopefully solved the case now.

[Breck_Carter]

compare whether v16/v17 would have the same log entries

Excellent idea!

[Former Member · Accepted solution]

This is unfortunately an all too common problem.

Given you are getting a TLS handshake error with 16 and 17, combined with the fact that your build of 12.0.1 also uses OpenSSL I would strongly suggest the error with 12.0.1 is going to be due to the same issue; as v16 and V17. {ie. a TLS-handshake error .... more on this later)

Given any build of 11.0.x and 10.0.x still used the Certicom libraries you are likely just looking at a certificate usage that simply (but correctly) no longer passes scrutiny. As such you'll need to correct the trusted certificate somehow.

Since the Certicom libraries were notorious for passing server supplied certificates that did not always have a 'trusted' root certificate, combined with a historical practice/tendancy of supplying a certificate chain with a match only on the leaf/child portion, then, just getting the corrected public root or public certificate and supplying that in your CERTIFICATE= parameter should be all there is to this. Of course obtaining that root certificate will require you identifying the correct one somehow ... which can be as simple as investigating this via your browser (IE / Chrome/ Mozilla/FireFox ...) but may require a deeper dive into the protocol failure using network sniffing/tracing with Wireshark (for example).

Most times technical support spends hours with the full URL and certificate (or certificate file) and after pooring over a few wireshark frames just to find the certificate supplied was always incorrect and all the was needed to supply just the correct root cert.

[Breck_Carter · Accepted solution]

From the client...

That gave me the info I needed.

I exported the root certificate of the certificate my website uses, just the top of the chain, and used that as the certificate. Calls go through just like that now.

In detail: On the website server, in IIS, go to the bindings page. Edit the SSL Binding View SSL Certificate Go to Certification Path page Select the root certificate in that tree and click “View Certificate” On the details page, click copy to file Export as base-64 X.509

That’s the certificate that worked. (I wrote it out because I thought you may want to post this as a solution on the thread you created)

Problem solved!