Kyma runtime stories. mTLS routes made easy (-ier) with APIRule v2

quovadis

Kyma runtime stories. mTLS routes made easy (-ier) with APIRule v2

Eventually, following the official announcement, the lastest istio-based APIRule in the stable version v2 is out.

Kyma Runtime - API Gateway module: Update to version 3.0.2

This version of the API Gateway module introduces APIRule CR in the stable version v2.

The new CRD allows you to expose your workloads using one of the three supported access strategies: jwt, noAuth, and extAuth. The noAuth access strategy provides a simple configuration for exposing workloads over the specified HTTP methods. The jwt access strategy allows you to secure your workload by defining Istio JWT configuration and the extAuth access strategy allows for providing custom authentication and authorization logic. For more information, see APIRule Custom Resource

Additionally, version 3.0.2 contains the following fixes:

We've fixed the behaviour of in-cluster connectivity when using JWT handler in APIRule. Now, in-cluster connectivity is blocked.
Now, when you create an APIRule with the noAuth access strategy, it is validated if the target workload has an Istio sidecar proxy injected.
We've added a deprecation message, which is displayed when you use the k8s API to interact with APIRule v1beta1 CRs. Now, the following deprecation message is presented during each call: Warning: Version v1beta1 of APIRule is deprecated and will be removed in future releases. Use version v2 instead.

Kyma Runtime - API Gateway module: Update to version 3.0.2This version of the API Gateway module introduces APIRule CR in the stable version v2.The new CRD allows you to expose your workloads using one of the three supported access strategies: jwt, noAuth, and extAuth. The noAuth access strategy provides a simple configuration for exposing workloads over the specified HTTP methods. The jwt access strategy allows you to secure your workload by defining Istio JWT configuration and the extAuth access strategy allows for providing custom authentication and authorization logic. For more information, see APIRule Custom Resource Additionally, version 3.0.2 contains the following fixes:We've fixed the behaviour of in-cluster connectivity when using JWT handler in APIRule. Now, in-cluster connectivity is blocked.Now, when you create an APIRule with the noAuth access strategy, it is validated if the target workload has an Istio sidecar proxy injected.We've added a deprecation message, which is displayed when you use the k8s API to interact with APIRule v1beta1 CRs. Now, the following deprecation message is presented during each call: Warning: Version v1beta1 of APIRule is deprecated and will be removed in future releases. Use version v2 instead.

Until now, customers and developers were invited to test the istio-based APIRule in the version v2alpha1.

Good to know:

The promotion of APIRule manifests from version v2alpha1 to version v2 merely consists of manually changing apiVersion from v2alpha1 to the stable version v2 in APIRule manifests.
As depicted above, for convenience, Kyma dashboard will display the APIRules CRs grouped by the apiVersion field.

APIRule v2 and mTLS routes

The APIRule v2 is istio-based. What that means is that both authentication and authorisations (permissions) are handled natively through istio.

Let me try to demonstrate how and whether it can help to ease the implementation effort of the mTLS routes.

Good to know:

Albeit the mTLS routes are mainly used for system to system communications, they can also be used for the principal propagation (user context) with short-lived client certificates.
For reference, the basics of mTLS communications with kyma runtime environment are covered in the following blogpost, namely: Developing enterprise-grade applications with mutual TLS easy with SAP BTP, Kyma runtime and BTP des...

When implementing mTLS routes one needs to forward the client certificate to the server side workloads. This is achieved by requesting the istio SSL headers to be forwarded to the server side, for instance:

      request:
        headers:
          X-CLIENT-SSL-CN: '%DOWNSTREAM_PEER_SUBJECT%'
          X-CLIENT-SSL-ISSUER: '%DOWNSTREAM_PEER_ISSUER%'
          X-CLIENT-SSL-SAN: '%DOWNSTREAM_PEER_URI_SAN%'
          test: 'true'

Let's consider the following APIRule v2 manifest:

apiVersion: gateway.kyma-project.io/v2
kind: APIRule
metadata:
  labels:
    app.kubernetes.io/name: httpbin-mtls-${namespace}

  name: httpbin-mtls-${namespace}
  namespace: montypython
spec:
  gateway: ${namespace}/quovadis-${namespace}-gateway-mtls
  hosts:
    - httpbin-mtls-${namespace}.${mtls}.${base_domain}
  rules:
    - methods:
        - GET
      noAuth: true
      path: /*
      timeout: 300
      request:
        headers:
          X-CLIENT-SSL-CN: '%DOWNSTREAM_PEER_SUBJECT%'
          X-CLIENT-SSL-ISSUER: '%DOWNSTREAM_PEER_ISSUER%'
          X-CLIENT-SSL-SAN: '%DOWNSTREAM_PEER_URI_SAN%'
          test: 'true'
  service:
    name: httpbin
    port: 8000

Good to know:

Any custom headers can be included here and they will be forwarded to the server side as well.
The kyma dashboard view of the APIRule request headers is depicted below:

The above APIRule definition is translated into auto-generated istio VirtualService and the AuthorizationPolicy resource definitions, namely:

VirtualService

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  labels:
    apirule.gateway.kyma-project.io/v1beta1: httpbin-mtls-${namespace}.montypython

  name: httpbin-mtls-${namespace}-jw7tv
  namespace: montypython

spec:
  gateways:
    - ${namespace}/quovadis-${namespace}-gateway-mtls
  hosts:
    - httpbin-mtls-${namespace}.mtls-quovadis-<id>.quovadis.kyma.dev.sap
  http:
    - headers:
        request:
          set:
            X-CLIENT-SSL-CN: '%DOWNSTREAM_PEER_SUBJECT%'
            X-CLIENT-SSL-ISSUER: '%DOWNSTREAM_PEER_ISSUER%'
            X-CLIENT-SSL-SAN: '%DOWNSTREAM_PEER_URI_SAN%'
            test: 'true'
            x-forwarded-host: >-
              httpbin-mtls-${namespace}.mtls-quovadis-<id>.quovadis.kyma.dev.sap
        response:
          remove:
            - Access-Control-Allow-Origin
            - Access-Control-Expose-Headers
            - Access-Control-Allow-Headers
            - Access-Control-Allow-Credentials
            - Access-Control-Allow-Methods
            - Access-Control-Max-Age
      match:
        - method:
            regex: ^(GET)$
          uri:
            prefix: /
      route:
        - destination:
            host: httpbin.montypython.svc.cluster.local
            port:
              number: 8000
          weight: 100
      timeout: 300s

AuthorizationPolicy

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  labels:
    apirule.gateway.kyma-project.io/v1beta1: httpbin-mtls-${namespace}.montypython
    gateway.kyma-project.io/hash: montypython.47fnidc80q3pr.65q2a0rkmeuic
    gateway.kyma-project.io/index: '0'

  name: httpbin-mtls-${namespace}-g6kzx
  namespace: montypython
spec:
  rules:
    - from:
        - source:
            principals:
              - >-
                cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account
      to:
        - operation:
            methods:
              - GET
            paths:
              - /{**}
  selector:
    matchLabels:
      app: httpbin

Good to know:

The current design of the APIRule implies having multiple AuthorizationPolicy resources, one per path.
The above AuthorizationPolicy guarantees a given workload can only be accessed via an istio ingress gateway, with the advertised HTTP method(s) and from the advertised paths.

Smoke-testing of the mTLS route

Good to know:

The smoke test using curl command is described here

Assuming we have a CA-signed client certificate at hand, we can test the XFF headers being forwarded to the httpbin workload running on a kyma cluster, as follows:

curl https://httpbin-mtls-${namespace}.mtls-quovadis-<id>.quovadis.kyma.dev.sap/headers --cert poster.x509 --key poster.key | jq .

{
  "headers": {
    "Accept": "*/*",
    "Host": "httpbin-mtls-${namespace}.mtls-quovadis-<id>.quovadis.kyma.dev.sap",
    "Test": "true",
    "User-Agent": "curl/8.7.1",
    "X-Client-Ssl-Cn": "CN=poster-quovadis (P000000),L=poster.***.demo.sap,OU=37d98dec-719a-4137-9ed7-***,OU=SAP Cloud Platform Clients,O=SAP SE,C=DE",
    "X-Client-Ssl-Issuer": "CN=SAP Cloud Platform Client CA,OU=SAP Cloud Platform Clients,O=SAP SE,L=EU10,C=DE",
    "X-Envoy-Attempt-Count": "1",
    "X-Envoy-External-Address": "**.214.184.**",
    "X-Forwarded-Client-Cert": "Hash=***;Cert=\"-----BEGIN%20CERTIFICATE-----%0AMIIGxTCCBK2gAwIBAgIRAPPDecfD1O2rVgSbz5qRTw0wDQYJKoZIhvcNAQELBQAw%0AeTELMAkG
***************** truncated ********************
upVq0l1ImDE%0A-----END%20CERTIFICATE-----%0A\";Subject=\"CN=poster-quovadis (P000000),L=poster.***.demo.sap,OU=37d98dec-719a-4137-9ed7-***,OU=SAP Cloud Platform Clients,O=SAP SE,C=DE\";URI=,By=spiffe://cluster.local/ns/montypython/sa/httpbin;Hash=9fbcc6e4a30e6c24673dd6d822ddea353346d939d907b8cfd3cd3d558d91524b;Subject=\"\";URI=spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account",
    "X-Forwarded-Host": "httpbin-mtls-${namespace}.mtls-quovadis-<id>.quovadis.kyma.dev.sap"
  }
}

So far so good. But is that all?

Well, not necessarily, as it would imply having to validate the X-Forwarded-Client-Cert header in each workload that receives it.

When using istio, one would simply have all these checks done by a dedicated AuthorizationPolicy, for instance:

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:

  name: httpbin-mtls-policy-${namespace}
  namespace: montypython

spec:
  action: ALLOW
  rules:
    - to:
        - operation:
            hosts:
              - >-
                httpbin-mtls-${namespace}.mtls-quovadis-<id>.quovadis.kyma.dev.sap
      when:
        - key: request.headers[X-Client-Ssl-Cn]
          values:
            - >-
              CN=poster-quovadis (P000000),L=poster.***.demo.sap,OU=37d98dec-719a-4137-9ed7-***,OU=SAP Cloud Platform Clients,O=SAP SE,C=DE
  targetRef:
    group: gateway.networking.k8s.io
    kind: Gateway
    name: quovadis-${namespace}-gateway-mtls

That's it. By simply adding the above manifest, if the client certificate sent does not have the policy matching X-Client-Ssl-Cn header, then the workload access will be denied with the following error message:

RBAC: access denied

That's important, as otherwise any dully CA-signed client certificate (that assuming there is an existing cacert bundle secret for a given gateway) would be forwarded to the server side.

~quovadis

Appendix

Here goes a terraform provisioner script snippet to help deploy the httpbin workload into a kyma cluster.

resource "terraform_data" "httpbin" {

 provisioner "local-exec" {
   interpreter = ["/bin/bash", "-c"]
   on_failure = continue
   command = <<EOF
     (
    KUBECONFIG=kubeconfig-headless.yaml
    NAMESPACE=montypython

    set -e -o pipefail
    HTTPBIN=$(./kubectl --kubeconfig $KUBECONFIG -n $NAMESPACE get deployment httpbin --ignore-not-found)
    if [ "$HTTPBIN" = "" ]
    then
      ./kubectl create ns $NAMESPACE --kubeconfig $KUBECONFIG --dry-run=client -o yaml | ./kubectl apply --kubeconfig $KUBECONFIG -f -
      ./kubectl label namespace $NAMESPACE istio-injection=enabled --kubeconfig $KUBECONFIG
      ./kubectl -n $NAMESPACE create -f https://raw.githubusercontent.com/quovadis-btp/istio/refs/heads/master/samples/httpbin/httpbin.yaml --kubeconfig $KUBECONFIG

      while [ "$(./kubectl --kubeconfig $KUBECONFIG -n $NAMESPACE get deployment httpbin --ignore-not-found)" = "" ]
      do 
        echo "no deployment httpbin"
        sleep 1
      done      
    fi

    HTTPBIN=$(./kubectl --kubeconfig $KUBECONFIG -n $NAMESPACE rollout status deployment httpbin --timeout 5m)
    echo $HTTPBIN 

     )
   EOF
 }
}

For reference, the basics of mTLS communications with kyma runtime environment are covered in the following blogpost, namely: Developing enterprise-grade applications with mutual TLS easy with SAP BTP, Kyma runtime and BTP des...