GitOps – Kubernetes The Easy Way
Part I: Best practices with Kubernetes management techniques
The management of Kubernetes should be easy, simple and clear. Well-known DevOps practices and ultimately automation should be accessible by the management technique. The first GitOps principle, the declarative approach, is the essential Kubernetes management technique to leverage the desired feature set.
Introduction
This is the first article of a series on GitOps and its paradigms to make the configuration management of Kubernetes easy. The four GitOps principles are:
- declarative approach: the entire system is described declaratively,
- version control: the canonical desired system state is versioned in Git,
- deployment automation: approved changes are automatically applied to the system,
- continuous delivery: software agents ensure correctness and alert on divergence.
This article introduces different Kubernetes management techniques and will emphasize on the importance and the value of the declarative approach. The declarative approach is the first GitOps principle and it recommends to describe the entire system declaratively. In the following sections, three Kubernetes management techniques are explained. One technique is an approach without Kubernetes object configuration files:
- imperative commands.
The other two techniques are based on Kubernetes object configuration files:
- imperative object configuration,
- declarative object configuration.
Join the LIVE webinar, „Kubernetes the Easy Way“, on May 15th, 10:00 am!
Imperative Commands
Create objects
The simplest way of interacting with the Kubernetes API are imperative commands. This management technique works without Kubernetes object configuration files, but with imperative steps. Imperative commands can be classified in verb-driven and object-driven commands. For example, a Deployment object for the vote containers can be generated by an verb-driven, imperative command:
kubectl run vote --image=andywirtzk8s/voting-vote:v1.0 --port=8080 -l "app=vote" -r 3 -n voting
With this command, the vote Deployment object in the voting Namespace is created by Kubernetes. The object creation requires only a single step and no configuration file is needed. The image can be found at dockerhub. A Service object for the vote containers can be generated with:
kubectl expose deployment vote --name vote -n voting
With this command, the vote Service object in the voting Namespace is created by Kubernetes. An HorizontalPodAutoscaler object for the vote containers can be generated with:
kubectl autoscale deployment vote --min 2 --max 5 --cpu-percent 80 --name vote -n voting
With this command, the vote HorizontalPodAutoscaler object in the voting Namespace is created by Kubernetes. These three commands are verb-driven. The next command is object-driven. With the object-driven command, different Kubernetes objects can be created. For example, a ConfigMap object for the vote containers can be generated with:
kubectl create configmap options --from-literal A=Imperative --from-literal B=Declarative -n voting
With this command, the options ConfigMap object in the voting Namespace is created by Kubernetes. Figure 1 depicts the four imperative commands to create Kubernetes objects.
Figure 1: Create objects with imperative commands.
Update Objects
The management technique with imperative commands for interacting with the Kubernetes API can also be used to change Kubernetes objects. Three available imperative commands are verb-driven. For example, a Deployment object can be horizontally scaled with:
kubectl scale deployment vote --replicas 2 -n voting
With this command, the replicas field of the vote Deployment object in the voting Namespace is updated by Kubernetes. The change requires only a single step and no configuration file is needed. A Kubernetes object can be annotated with:
kubectl annotate deployment vote atix.de/technique='imperative commands' -n voting
With this command, the annotations field of the vote Deployment object in the voting Namespace is updated by Kubernetes. A Kubernetes object can be labeled with:
kubectl label deployment vote technique=imperative -n voting
With this command, the labels field of the vote Deployment object in the voting Namespace is updated by Kubernetes. Alternatively, an aspect-driven command can be used. With an aspect-driven command, different fields for different Kubernetes objects can be changed. For example, the field of a Kubernetes object can be changed with:
kubectl set image deployment vote vote=andywirtzk8s/voting-vote:v1.1 -n voting
With this command, the image field of the pod template of the vote Deployment object in the voting Namespace is updated by Kubernetes and a rolling update is triggered. In addition, there are two imperative commands with a more general approach to changing live Kubernetes objects. For example, a specific field of a Kubernetes object can be modified by a patch string with:
kubectl patch deployment vote -p '{"spec": {"minReadySeconds": 10}}' -n voting
With this command, the minReadySeconds field of the vote Deployment object in the voting Namespace is updated by Kubernetes. Here, understanding the JSON format of the Kubernetes API Resources is required. For bigger changes to a Kubernetes object, the live object can be opened in the default editor with:
kubectl edit deployment vote -n voting
With the editor, changes can be made and saved. For example:
...
- image: andywirtzk8s/voting-vote:v1.1
name: vote
envFrom:
- prefix: OPTION_
configMapRef:
name: options
livenessProbe:
httpGet:
path: /
port: http
readinessProbe:
httpGet:
path: /
port: http
ports:
- containerPort: 8080
name: http
...
In this example, the fields envFrom, livenessProbe, readinessProbe and ports[0].name are added to the live Deployment object. After making these changes, saving the file and exiting the editor, the vote Deployment object in the voting Namespace is updated by Kubernetes and a rolling update is triggered. Here, understanding the YAML format of the Kubernetes API Resources is required. Figure 2 depicts the six imperative commands to update Kubernetes objects.
Figure 2: Update objects with imperative commands.
Delete Objects
To delete a Kubernetes object, there exists one imperative object-driven command. For example, a Kubernetes object can be deleted with:
kubectl delete deployment vote -n voting
With this command, the vote Deployment object in the voting Namespace is deleted by Kubernetes. Figure 3 depicts the one imperative command to delete Kubernetes objects.
Figure 3: Delete objects with imperative commands.
Benefits and Challenges
The major advantage of the Kubernetes management with imperative commands is the simplicity. Imperative commands are simple, easy to learn and easy to remember. Configuration changes require only a single step. In addition, changes to Kubernetes objects can be made by multiple writers without conflicting.
The major disadvantage of the Kubernetes management with imperative commands is the lack of Kubernetes object configuration files. Without the Kubernetes object configuration files, automation, standardization and reproducibility is impossible. DevOps practices like source control, change review processes and audit trails associated with changes can not be accessed.
Table 1 lists the features of the imperative commands for managing Kubernetes objects. More reading on this management technique can be found under imperative commands.
| Management technique | Imperative commands |
|---|---|
| Simplicity | β |
| Multiple editors | β |
| Operate on directories | β |
| Source control | β |
| Change review processes | β |
| Audit trails | β |
| Standardization | β |
| Reproducibility | β |
| Idempotence | β |
| Automation | β |
Table 1: Features of the imperative commands.
Imperative Object Configuration
Create objects
A more sophisticated way of interacting with the Kubernetes API is the imperative object configuration. This management technique works with Kubernetes object configuration files. For example, a Deployment object for the vote containers can be defined in a file:
FILE=${HOME}/configs/voting/vote/deployment.yaml
The content of the file is a YAML descriptor of the Deployment object:
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
atix.de/technique: imperative object configuration
labels:
app: vote
technique: imperative
name: vote
namespace: voting
spec:
minReadySeconds: 10
replicas: 3
selector:
matchLabels:
app: vote
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 0
type: RollingUpdate
template:
metadata:
labels:
app: vote
spec:
containers:
- image: andywirtzk8s/voting-vote:v1.0
name: vote
envFrom:
- prefix: OPTION_
configMapRef:
name: options
livenessProbe:
httpGet:
path: /
port: http
readinessProbe:
httpGet:
path: /
port: http
ports:
- containerPort: 8080
name: http
This is a YAML manifest representing the desired state of the Deployment object. This Deployment object can be generated with:
kubectl create -f ${FILE}
With this command, the vote Deployment object in the voting Namespace is created by Kubernetes. For the command to work, it is required that the vote Deployment object in the voting Namespace does not exist beforehand. Figure 4 depicts the one imperative object configuration command to create Kubernetes objects.
Figure 4: Create objects with imperative object configuration.
Update objects
The management technique of imperative object configuration for interacting with the Kubernetes API can also be used to change Kubernetes objects. For example, the Deployment object configuration can be replaced by:
kubectl replace -f ${FILE}
Depending on the change made in the YAML manifest of the vote Deployment object, Kubernetes acts to ensure the new desired state. If the replicas field is changed, the live Deployment object is scaled horizontally. If the image field is changed in the pod template of the Deployment object, a rolling update is triggered. Kubernetes replaces the live object even if there is no configuration change. This results in imperative, non-idempotent configuration management. For the command to work, it is required that the vote Deployment object in the voting Namespace does exist beforehand. Figure 5 depicts the one imperative object configuration command to update Kubernetes objects.
Figure 5: Update objects with imperative object configuration.
Delete objects
To delete a Kubernetes object, there exists one command within the imperative object configuration technique. For example, a Kubernetes object can be deleted with:
kubectl delete -f ${FILE}
With this command, the vote Deployment object in the voting Namespace is deleted by Kubernetes. Figure 6 depicts the one imperative object configuration command to delete Kubernetes objects.
Figure 6: Delete objects with imperative object configuration.
Benefits and challenges
The major advantage of the Kubernetes management with imperative object configuration is the record of the desired state in form of Kubernetes object configuration files. With these Kubernetes object configuration files, standardization and reproducibility can be enabled. DevOps practices like the usage of a source control system, of change review processes and of audit trails associated with changes can be accessed.
The major disadvantage of the Kubernetes management with imperative object configuration is the missing feature of idempotent configuration management. Updates are done by replacing the live object, even if there is no configuration drift between the configuration file and the live object.
If multiple editors manage the same Kubernetes object, changes of one source might be lost by the replace command of the second source. Such updates to live objects must be reflected in the configuration files. Moreover, there are additional requirements for the create and replace commands to work. Imperative object configuration works best on files, not directories. All these issues make it difficult to perform or automate configuration changes.
Table 2 lists the features of the imperative object configuration for managing Kubernetes objects. More reading on this management technique can be found under imperative object configuration.
| Management technique | Imperative object configuration |
|---|---|
| Simplicity | πΆ |
| Multiple editors | β |
| Operate on directories | πΆ |
| Source control | β |
| Change review processes | β |
| Audit trails | β |
| Standardization | β |
| Reproducibility | β |
| Idempotence | β |
| Automation | β |
Table 2: Features of the imperative object configuration.
Declarative Object Configuration
Create objects
The most sophisticated way of interacting with the Kubernetes API is the declarative object configuration. This management technique works with Kubernetes object configuration files. It supports operating on directories with Kubernetes object configuration files. For example, a directory containing YAML descriptor files of Kubernetes objects can be defined:
DIR=${HOME}/configs/voting/vote
In this directory, the YAML manifests representing the desired state of the Deployment, the Ingress and the Service object for the vote containers are stored:
.
βββ deployment.yaml
βββ ingress.yaml
βββ service.yaml
The files can be found on github. The three Kubernetes objects can be printed with:
kubectl diff -f ${DIR}/
With this command, the vote Deployment, the vote Ingress and the vote Service objects are printed by Kubernetes and can be reviewed. The output does not only show the fields defined in the YAML manifests, but is enriched by the default field values of Kubernetes. The three Kubernetes objects can be generated with:
kubectl apply -f ${DIR}/
With this command, the vote Deployment, the vote Ingress and the vote Service objects are posted to Kubernetes and created by Kubernetes in the voting Namespace. There are no requirements on whether the objects exist beforehand or not. Figure 7 depicts the two relevant declarative object configuration commands to create Kubernetes objects.
Figure 7: Create objects with declarative object configuration.
Update objects
The management technique of declarative object configuration for interacting with the Kubernetes API can also be used to change Kubernetes objects. The difference between the locally stored YAML manifests and the live Kubernetes object counter-parts can be printed with:
kubectl diff -f ${DIR}/
With this command, the configuration drift of the vote Deployment, the vote Ingress and the vote Service objects is printed by Kubernetes and can be reviewed. The output shows only the fields that are not in synchronization between the locally stored Kubernetes object configuration files and the live Kubernetes objects. Independent of whether a difference is shown or not, the three Kubernetes objects can be updated with:
kubectl apply -f ${DIR}/
With this command, three different scenarios have to be considered. In case one, the Kubernetes object did not exist beforehand. Here, it is created by Kubernetes as described in the last subsection. In case two, there is no difference between the YAML manifest and the live Kubernetes objects. Here, no action is performed. In case three, one or more fields in one or more Kubernetes objects have been changed. Here, a merge patch is calculated and the Kubernetes objects are patched by Kubernetes.
With this merge patch calculation, the desired system state is not only described declaratively in the YAML manifests, but also communicated to Kubernetes in a declarative way. This results in idempotent configuration management. For example, if the replicas field and the image field are changed in the locally stored Kubernetes object configuration files, the live Deployment object is scaled horizontally and a rolling update is triggered, while the live Ingress and Service objects remain untouched.
On the other hand, if a field is not defined in the YAML manifest file, it is ignored by the command. If some imperative changes have been made on default field values not defined in a Kubernetes object configuration file, this change will not be overwritten. Figure 8 depicts the two relevant declarative object configuration commands to update Kubernetes objects.
Figure 8: Update objects with declarative object configuration.
Delete objects
To delete Kubernetes objects, there exists one command within the declarative object configuration technique. For example, the three Kubernetes objects can be deleted with:
kubectl delete -f ${DIR}/
With this command, the vote Deployment, the vote Ingress and the vote Service objects in the voting Namespace are deleted by Kubernetes. Figure 9 depicts the one declarative object configuration command to delete Kubernetes objects.
Figure 9: Delete objects with declarative object configuration.
Benefits and challenges
The major advantage of the Kubernetes management with declarative object configuration is the support for operating on directories containing Kubernetes object configuration files and automatically detecting the operation type per object. Kubernetes can automatically decide whether an object or field should be created, patched, deleted or remain untouched. This enables idempotent configuration management.
With the directories containing the Kubernetes object configuration files, automation, standardization and reproducibility can be enabled. DevOps practices like the usage of a source control system, of change review processes and of audit trails associated with changes can be accessed.
The Kubernetes management with declarative object configuration is the most powerful of the management techniques. It comes with the cost of a steeper learning curve, as the patch merge calculation can result in unexpected behavior.
Table 3 lists the features of the declarative object configuration for managing Kubernetes objects. More reading on this management technique can be found under declarative object configuration.
| Management technique | Declarative object configuration |
|---|---|
| Simplicity | β |
| Multiple editors | β |
| Operate on directories | β |
| Source control | β |
| Change review processes | β |
| Audit trails | β |
| Standardization | β |
| Reproducibility | β |
| Idempotence | β |
| Automation | β |
Table 3: Features of the declarative object configuration.
Summary
The goal of this article is to show the value of the declarative approach to manage Kubernetes. Table 4 compares the features of the three different Kubernetes management techniques. More reading on these options can be found under management techniques.
| Management technique | Imperative commands | Imperative object configuration | Declarative object configuration |
|---|---|---|---|
| Simplicity | β | πΆ | β |
| Multiple editors | β | β | β |
| Operate on directories | β | πΆ | β |
| Source control | β | β | β |
| Change review processes | β | β | β |
| Audit trails | β | β | β |
| Standardization | β | β | β |
| Reproducibility | β | β | β |
| Idempotence | β | β | β |
| Automation | β | β | β |
Table 4: Comparison of the features of the management techniques.
First, it can be differentiated between management via imperative commands and via Kubernetes object configuration files. In the imperative approach with imperative commands, actions are defined. In Kubernetes object configuration files, state is defined. In the configuration files, the desired system state is described declaratively.
Second, it can be differentiated between management via imperative object configuration and declarative object configuration. In the imperative approach with imperative object configuration, state is communicated with an imperative command. In the declarative approach with declarative object configuration, state is communicated with a declarative command. Based on the actual and the desired system state, necessary actions are derived in the declarative approach.
The management technique with imperative commands is good for first steps with Kubernetes in a development environment. But this imperative approach does not enable DevOps practices. Ultimately, automation is impossible.
The management technique with imperative object configuration is a hybrid method, as it describes the desired Kubernetes state declaratively, but it communicates the state to Kubernetes with an imperative command. This hybrid approach enables a lot of DevOps practices. But ultimately, the lack of idempotence makes automation impossible.
The management technique with declarative object configuration is the recommendation for production environments of Kubernetes. This declarative approach enables DevOps practices. And the ultimate goal of automation is possible.
The declarative object configuration is the most sophisticated management technique and the one with the steepest learning curve. To reduce its complexity, it is important to embed this declarative approach in a wider set of practices. This article series recommends the GitOps principles.
The first GitOps principle is the declarative approach. This paradigm advocates to describe the entire system declaratively. The declarative object configuration is the Kubernetes management technique enabling this first GitOps principle.
Andy Wirtz
Neueste Artikel von Andy Wirtz (alle ansehen)
- GitOps – Kubernetes The Easy Way: Part 1 - 16. April 2020
