Kubernetes service accounts are somewhat
similar to a technical user concept, and are very useful in managing service to service automation and communication. For instance, one would rely on service accounts to manage CI/CD pipelines but there are
other use-cases as well.
Putting it all together
My personal motivation behind this brief was to procure a service account based
kubeconfig as to eliminate an OIDC-user browser redirect and make it suitable for use in
headless and
unmanned contexts.
Here goes the agenda for this brief.
main course
|
coffee corner
|
|
|
|
|
|
|
Brief anatomy of a kyma cluster
Let's assume you have got access to your
SAP BTP, Kyma runtime cluster.
OIDC-user based kubeconfig
After you have provisioned your kyma cluster you can either download the OIDC-user based
kubeconfig or directly launch a kyma dashboard from a BTP sub-account.
Worker nodes
Typically, a kyma cluster will have a number of worker nodes, possibly in different availability
zones (in the same region/data center though), as depicted below:
SAP BTP, Kyma runtime cluster with AZURE with 4 nodes in 3 availability zones |
|
SAP BTP Free Tier, Kyma runtime cluster with AWS with a single node in one availability zone |
|
Each worker node features a certain CPU and Memory capacity that can be reserved by kyma workloads. Moreover, each node is mapped into specific hardware (machine type) from a cluster provider (AZURE, AWS, GCP).
Good to know:
- It is possible to dynamically change a worker node machine type without incurring any downtime.
Namespaces
On a logical level, each kyma cluster comes provisioned with two public namespaces, namely
default
and
kube-public
. The
default
namespace cannot be deleted and
kube-public
can be used to host kyma dashboard extensions.
Namespaces are very useful in organizing cluster resources.
Thus, indeed, most likely, you will end up creating several other
namespaces to cater for different tasks or users profiles as a means of exercising some level of control over cluster resources.
Namespaced isolation. Keep your clutter at bay.
If we were to manage a population of several thousand or so kyma cluster named users as well as a number of service accounts, we could simply divide the entire user population into logical teams or tasks and then assign each team or task a separate namespace.
Subsequently, we would grant each team member a type of an access -
a role binding - that is compatible with their duties. Furthermore, we'd create a service account for each team for a service access to a namespace.
To get you more insight about organizing with namespaces here goes a very good read:
Role bindings
Namespaced role bindings are created based on the existing roles. These roles can be either cluster wide or namespaced roles.
- Let us start with a full cluster-admin access for all team members and then eventually narrow down the level of access according with each team member profile.
- Let's see how to create a service account with a never expiring token for a team namespace.
Service Accounts
One of the
qualities of service accounts is that they are
namespaced.
Service Accounts tokens.
Tokens are required to grant access to Kubernetes API server.
Nowadays, with Kubernetes versions 1.24.8 onwards, the secret-based tokens are no longer auto-generated by default for new service accounts. But there are other ways to get service accounts tokens, as described here:
bound tokens
Bound, short-lived
tokens can be created with
kubectl create token
Alternatively, a kyma dashboard can be used to create bound tokens for a service account, as depicted below:
secret-based tokens
Albeit the short-lived bound token are recommended for workload automations, legacy secret-based tokens can still be useful if you want to gain access to a kyma namespace from a headless environment without incurring a browser redirect to localhost (or if the redirect port was already taken).
Let's create a Makefile and a helm chart to automate the entire process, namely:
- create a helm chart to create a service account, service account secret and a role binding (see appendix)
- create a Makefile with the following targets:
btp-easy-deploy
and btp-easy-kubeconfig
(see appendix)
Run the following make target:
$make btp-easy-deploy
$ make NAMESPACE=toto btp-easy-deploy
kubectl create ns toto --kubeconfig ~/.kube/kubeconfig--btp-easy--btp.yaml --dry-run=client -o yaml | kubectl apply --kubeconfig ~/.kube/kubeconfig--btp-easy--btp.yaml -f -
namespace/toto created
kubectl label namespace toto istio-injection=enabled --kubeconfig ~/.kube/kubeconfig--btp-easy--btp.yaml
namespace/toto labeled
helm upgrade --install --create-namespace btp-easy ./helm/btp-easy/ --namespace=toto --kubeconfig ~/.kube/kubeconfig--btp-easy--btp.yaml
Release "btp-easy" does not exist. Installing it now.
NAME: btp-easy
LAST DEPLOYED:
NAMESPACE: toto
STATUS: deployed
REVISION: 1
TEST SUITE: None
A new namespace, a service account, a service account secret and a role binding will be created in the namespace, as follows:
service account |
role binding |
|
|
View service account based kubeconfig
Run the following make target:
$ make NAMESPACE=toto view-btp-easy-kubeconfig
$ make NAMESPACE=toto view-btp-easy-kubeconfig
kubectl-view_serviceaccount_kubeconfig toto-sa -n toto --kubeconfig ~/.kube/kubeconfig--btp-easy--btp.yaml
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tBtS05SbGo4TENFL2ZVTi8vbDFsZwotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
server: https://api.***.btp-easy.shoot.canary.k8s-hana.ondemand.com
name: shoot--btp-easy--***
contexts:
- context:
cluster: shoot--btp-easy--***
namespace: toto
user: toto-sa
name: shoot--btp-easy--***
current-context: shoot--btp-easy--***
kind: Config
preferences: {}
users:
- name: toto-sa
user:
token: eyJhbGciOiJSUzI1NiIsImtpZCI6Im1yTGRIdW9EOFJLZ3RQaDh2ZGxqSEhpWnVwN3ZSQThxdmlWb1dlWnBid
This is using a third-party kubectl
plugin called
kubectl-view-serviceaccount-kubeconfig
which allows to view a service account based kubeconfig...
Run
$ make NAMESPACE=toto btp-easy-kubeconfig
to create a kubeconfig file, namely
~/.kube/kubeconfig--btp-easy-toto--btp.yaml
Good to know:
- For enhanced security the file has both group and world read attributes stripped as follows:
chmod 600 ~/.kube/kubeconfig--btp-easy-$(NAMESPACE)--btp.yaml
Conclusion
Please continue reading through to the coffee corner as it gathers all the resources to help you create a Makefile and a helm chart to automate the creation of a Service Account.
Coffee corner
helm chart structure
┣ 📂helm
┃ ┗ 📂btp-easy
┃ ┃ ┣ 📂templates
┃ ┃ ┃ ┣ 📜clusteradmin-role-binding.yaml
┃ ┃ ┃ ┣ 📜sa-secret.yaml
┃ ┃ ┃ ┗ 📜sa.yaml
┃ ┃ ┗ 📜Chart.yaml
┣ 📜Makefile
1. create
sa.yaml:
- create a service account and then a never expiring secret attached to the sa.
kind: ServiceAccount
apiVersion: v1
metadata:
name: {{ .Release.Namespace }}-sa
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ .Release.Namespace }}-sa
automountServiceAccountToken: false
secrets:
- name: {{ .Release.Namespace }}-sa-token
2. create
sa-secret.yaml
- create a service token secret
apiVersion: v1
kind: Secret
type: kubernetes.io/service-account-token
metadata:
annotations:
kubernetes.io/service-account.name: {{ .Release.Namespace }}-sa
name: {{ .Release.Namespace }}-sa-token
labels:
app.kubernetes.io/name: {{ .Release.Namespace }}-sa-token
data: {}
3. create
clusteradmin-role-binding.yaml
- The new role binding will be created based on cluster-admin cluster role.
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: clusteradmin
namespace: {{ .Release.Namespace }}
subjects:
- kind: User
apiGroup: rbac.authorization.k8s.io
name: foo.bar@acme.com
- kind: ServiceAccount
name: {{ .Release.Namespace }}-sa
namespace: {{ .Release.Namespace }}
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
4. create
Chart.yaml
apiVersion: v2
name: btp-easy
description: Helm chart for btp-easy
# A chart can be either an 'application' or a 'library' chart.
type: application
# This is the chart version.
version: 0.0.1
# This is the version number of the application being deployed.
appVersion: 0.0.1
Makefile
5. create
Makefile with a number of targets
.DEFAULT_GOAL := help
LOCAL_DIR = $(dir $(abspath $(lastword $(MAKEFILE_LIST))))
NAMESPACE=btp-easy
KUBECONFIG=~/.kube/kubeconfig--btp-easy--btp.yaml
help: ## Display this help.
@awk 'BEGIN {FS = ":.*##"; printf "\nUsage:\n make \033[36m<target>\033[0m\n"} /^[a-zA-Z_0-9-]+:.*?##/ { printf " \033[36m%-15s\033[0m %s\n", $$1, $$2 } /^##@/ { printf "\n\033[1m%s\033[0m\n", substr($$0, 5) } ' $(MAKEFILE_LIST)
echo $(KUBECONFIG) $(NAMESPACE)
.PHONY: btp-easy-deploy
btp-easy-deploy: ## btp-easy-deploy
kubectl create ns $(NAMESPACE) --kubeconfig $(KUBECONFIG) --dry-run=client -o yaml | kubectl apply --kubeconfig $(KUBECONFIG) -f -
kubectl label namespace $(NAMESPACE) istio-injection=enabled --kubeconfig $(KUBECONFIG)
helm upgrade --install --create-namespace btp-easy ./helm/btp-easy/ --namespace=$(NAMESPACE) --kubeconfig $(KUBECONFIG)
.PHONY: btp-easy-undeploy
btp-easy-undeploy: ## btp-easy-undeploy
helm uninstall btp-easy --namespace=$(NAMESPACE) --kubeconfig $(KUBECONFIG)
kubectl delete ns $(NAMESPACE) --kubeconfig $(KUBECONFIG)
.PHONY: btp-easy-template
btp-easy-template: ## btp-easy-template
helm template btp-easy ./helm/btp-easy/ --namespace=$(NAMESPACE) --kubeconfig $(KUBECONFIG)
.PHONY: btp-easy-kubeconfig
btp-easy-kubeconfig: ## btp-easy-kubeconfig
kubectl-view_serviceaccount_kubeconfig $(NAMESPACE)-sa -n $(NAMESPACE) --kubeconfig $(KUBECONFIG) > ~/.kube/kubeconfig--btp-easy-$(NAMESPACE)--btp.yaml
chmod go-r ~/.kube/kubeconfig--btp-easy-$(NAMESPACE)--btp.yaml
.PHONY: view-btp-easy-kubeconfig
view-btp-easy-kubeconfig: ## view-btp-easy-kubeconfig
kubectl-view_serviceaccount_kubeconfig $(NAMESPACE)-sa -n $(NAMESPACE) --kubeconfig $(KUBECONFIG)
helm template
helm template is very useful to dry run the chart without ever touching the cluster...
helm template btp-easy ./helm/btp-easy/ --namespace=toto > toto.yaml
---
# Source: btp-easy/templates/sa.yaml
kind: ServiceAccount
apiVersion: v1
metadata:
name: toto-sa
namespace: toto
labels:
app.kubernetes.io/name: toto-sa
automountServiceAccountToken: false
secrets:
- name: toto-sa-token
---
# Source: btp-easy/templates/sa-secret.yaml
apiVersion: v1
kind: Secret
type: kubernetes.io/service-account-token
metadata:
annotations:
kubernetes.io/service-account.name: toto-sa
name: toto-sa-token
labels:
app.kubernetes.io/name: toto-sa-token
data: {}
---
# Source: btp-easy/templates/clusteradmin-role-binding.yaml
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: clusteradmin
subjects:
- kind: User
apiGroup: rbac.authorization.k8s.io
name: foo.bar@acme.com
- kind: ServiceAccount
name: toto-sa
namespace: toto
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
Troubleshooting and un-deploy
$ make NAMESPACE=toto btp-easy-undeploy
helm uninstall btp-easy --namespace=toto --kubeconfig ~/.kube/kubeconfig--btp-easy--btp.yaml
release "btp-easy" uninstalled
kubectl delete ns toto --kubeconfig ~/.kube/kubeconfig--btp-easy--btp.yaml
namespace "toto" deleted
$ kubectl plugin list
The following compatible plugins are available:
~/.krew/bin/kubectl-krew
~/.krew/bin/kubectl-oidc_login
~/.krew/bin/kubectl-view_serviceaccount_kubeconfig
$ kubectl krew list
PLUGIN VERSION
krew v0.4.3
oidc-login v1.27.0
view-serviceaccount-kubeconfig v2.3.0
$ helm version
version.BuildInfo{Version:"v3.12.0", GitCommit:"c9f554d75773799f72ceef38c51210f1842a1dea", GitTreeState:"clean", GoVersion:"go1.20.3"}
SAP Kyma Community and SAP BTP, Kyma runtime Q&A Tags
Follow me in SAP Community:
piotr.tesny