cancel
Showing results for 
Search instead for 
Did you mean: 

Issue with SAP BTP CredentialStore w/Basic Auth in US10 region?

SeanKilleen
Explorer
643

Hi all,


I am writing this in addition to the support ticket I submitted because I want to understand if others are seeing this issue.

Background:

  • We have been using Credential Store successfully in multiple environments for some time.
  • We typically use the credential store via a Kyma secret binding.
  • We currently use basic authentication to access the credential store. The username and password are supplied to us via the bindings.

Issue:

  • Last night, we began seeing credential store failures in our dev environment (which uses the us10 region of credentialstore, based on its BTP binding).
  • The failure response contains a header, "sapcp-credstore-error-code", with the value "invalid_or_missing_credentials".

What I checked:

  • I first saw this in my local environment, so I assumed my bindings were out of date. I went to update them, only to realize they're the same as what I saw in the BTP cockpit.
  • I confirmed the bindings matched in BTP Kyma and in my local environment, and matched what I saw in the cockpit.
  • I reverted to a known good version of the code (based on release tags). The problem persisted.
  • I verified that the logic for authentication hadn't changed, that it was being base64 encoded correctly, etc.
  • I realized that requests in an another environment, which uses us21, are succeeding.
  • Confirmed that the issue is happening for multiple devs in their local environment (e.g. it's not a typo in the bindings I'm using locally somehow).
  • Restarted containers in the dev environment where we saw this show up too.

Minimal Reproduction:

  • In Postman, I set up a request to the "get all keys" endpoint.
  • In the us21 region, with the values copied from that binding, this request succeeds -- I receive back an (encrypted, as expected) response.
  • The same call in the us10 region, using username/password values copied from the existing bindings for that environment (which triple-checked to be correct), the request fails with this error.

It appears to be there is an issue with the SAP Credential Store environment and basic auth in the us10 regions.

View Entire Topic
SeanKilleen
Explorer

I was able to discover the true cause -- thankfully, it was totally my fault! 😄

I thought I had ruled out all of the things that could be caused by our environment, but I missed a very important one, which I'll explain here in case some other poor soul finds themselves in the same predicament. :)

-----

In our Kyma environment, we use the SAP BTP Operator to facilitate our service bindings.

By default, the SAP BTP Operator does not rotate service bindings. It instead allows them to expire and fail. I did not expect this default behavior, and months ago I missed the documentation about enabling credentials rotation.

Once I added the logic to enable the credential rotation (see https://github.com/SAP/sap-btp-service-operator#credentials-rotation), the bindings were immediately rotated and the app began functioning again.