cancel
Showing results for 
Search instead for 
Did you mean: 

NetWeaver HTTP/2 request duplicating ":authority" pseudo-header?

SeanKilleen
Explorer
0 Kudos

Hi all,

I'm trying to troubleshoot an issue where I'm unsure if it originates on the Kyma side or the calling client side (NetWeaver in this case); this is the companion question to an SAP forum question in the Kyma area and the question I raised on the Kyma repo (which has more technical information including logs and examples.)

Background

  • I have an API running within SAP BTP Kyma, using a Kyma API Rule to expose it.
  • I can make calls to this fine from many applications. HTTP/1.1 calls via Postman, HTTP/2 calls via Insomnia or Curl.
  • For SAP calls to this API, NetWeaver is upgrading our HTTP/1.x calls to HTTP/2, which seems like it should be fine.

The Problem

  • Calls from SAP NetWeaver are receiving a 404 back from Kyma when HTTP/2 is enabled. When we disable HTTP/2 (an option for our environment but not our customers), the problem is resolved.
  • When I look in the Kyma logs, I see something strange -- all the failed requests have an ":authority" pseudo-header where the hostname is duplicated. If our hostname is "xyz.com", the ":authority" header contains "xyz.com,xyz.com"
  • This causes Kyma's Istio instance to return a 404 during ALPN filtering, I think because it's looking for an exact match on the authority.

The Question

I'm unsure how to proceed. This seems to be an issue only with NetWeaver calls (though I was able to reproduce it using another Windows client app, Apidog, but that app's HTTP/2 support is only experimental). So the questions I have are:

  • Is there any way to confirm whether NetWeaver is sending this heading in this way? Or better, any way to stop it?
  • Is there any way to ask NetWeaver not to upgrade a given request to HTTP/2 short of disabling HTTP/2 entirely? (I'm assuming disabling HTTP/2 won't be an option our customers' systems)

Accepted Solutions (1)

Accepted Solutions (1)

SeanKilleen
Explorer
0 Kudos

I now have an answer that includes a workaround.

Cause #1: A Bug Within Envoy

I confirmed that this appears to be an issue in the Envoy proxy used by Istio, which is then used by BTP Kyma:

https://github.com/envoyproxy/envoy/issues/31118#issuecomment-1834164342

Any client that sends both the ":authority" pseudo-header and the "Host" header will run into this problem with Envoy it seems, as Envoy will incorrectly concatenate into the ":authority" field which winds up with an authority that Istio then doesn't find any routes for, because it's looking for a single instance of the name.

Cause #2: NetWeaver Sending Both ":authority" and "Host" headers

NetWeaver sends both ":authority" and "Host" headers even when those values are the same. While allowed by the HTTP/2 spec, this is still slightly unusual behavior for a client (curl, Insomnia, and others appear to interpret the HTTP/2 spec the same way that Envoy did initially).

The only workaround until Envoy releases a fix, it seems, is to get clients (such as NetWeaver) to only send one of the ":authority" or "Host" headers as part of the request. This will result in a request that does not encounter the Envoy bug with Kyma, and thus the request can complete successfully.

(Slightly Hacky) Workaround: Remove hostname from request_url

In our SAP code that utilizes the NetWeaver client to make the HTTP request, we we modify the request_url field to remove the hostname prior to sending the request to Kyma. It's missing the Host header when it goes out, but it has the :authority header already set. Because of that, the header isn't duplicated, and therefore it doesn't hit the Envoy bug.

Answers (0)