In this new post, we are going to talk about another Kubernetes object, DaemonSet, where we will see how to use a DaemonSet in Kubernetes, what a DaemonSet is, and why and for what purpose we need to create them within our Kubernetes architectures.
What is a Daemonset?
A DaemonSet is a Kubernetes object, and like other Kubernetes objects, it manages a set of Pod replicas.
The main function of a DaemonSet is to ensure that one or a set of Pods are running on the specified node or nodes. When you delete a DaemonSet, all the Pods that were created will also be deleted.”
What do we use a DaemonSet for?
All Kubernetes objects usually have a specific and concrete purpose, and in the case of a DaemonSet, the most common uses are:
- Running a daemon to enable storage on each node, for example, Ceph, an open-source storage platform.
- Having a daemon for log collection, which is a very common use case. For example, tools such as Fluentd or Logstash.
- Running a daemon for continuous monitoring, such as Prometheus.
As we mentioned, a DaemonSet is responsible for managing and creating Pods. So, why not use a deployment or create Pods directly? The recommendation is to use a DaemonSet whenever you want and it’s important for a Pod to run on one or multiple hosts, although you can use a deployment or create Pods.
How are DaemonSets scheduled?
Usually, Kubernetes’ default scheduler schedules the Pod running on a node. However, in the case of DaemonSets, the DaemonSet controller can schedule the Pod.
If you choose to do scheduling with the DaemonSet controller, it has its own set of characteristics. Firstly, the ‘Pending’ state disappears because this scheduler doesn’t use it. Secondly, if Pod preemption is enabled (the preference), the DaemonSet controller will schedule without taking any priority or preference into consideration.
To use the default scheduler, you can make use of ‘ScheduleDaemonSetPods,’ which allows you to add NodeAffinity to bind it to a node by affinity
Example of how to use a DaemonSet in Kubernetes
Next, we are going to create a DaemonSet for Fluentd, which is a perfect example of using a DaemonSet as it allows us to collect and unify information from different services and consume it to understand and make better use of all that information.
Before creating and running our test DaemonSet, let’s create a new namespace where we will run it.
kubectl create namespace logging
apiVersion: apps/v1 kind: DaemonSet metadata: name: fluentd-elasticsearch namespace: logging labels: k8s-app: fluentd-logging spec: selector: matchLabels: name: fluentd-elasticsearch template: metadata: labels: name: fluentd-elasticsearch spec: tolerations: # para ejecutar en el nodo master, con minikube es el por defecto - key: node-role.kubernetes.io/master effect: NoSchedule containers: - name: fluentd-elasticsearch image: quay.io/fluentd_elasticsearch/fluentd:v2.5.2 resources: limits: memory: 200Mi requests: cpu: 100m memory: 200Mi volumeMounts: - name: varlog mountPath: /var/log - name: varlibdockercontainers mountPath: /var/lib/docker/containers readOnly: true terminationGracePeriodSeconds: 30 volumes: - name: varlog hostPath: path: /var/log - name: varlibdockercontainers hostPath: path: /var/lib/docker/containers
Save the previous DaemonSet in a file and name it ‘fluentd-daemonset.yaml,’ then we execute it as follows:
kubectl apply -f fluentd-daemonset.yaml -n logging
We will receive a creation message, and then we will check if it has been created successfully.
Run:
kubectl get daemonset -n logging
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE fluentd-elasticsearch 1 1 1 1 1 <none> 104s
We can see how we have our DaemonSet running in our namespace.
Next, let’s check the pods that have been created for this DaemonSet.
kubectl get pods -n logging -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES fluentd-elasticsearch-mjt5d 1/1 Running 0 99m 172.17.0.5 minikube <none> <none>
Keep in mind that you cannot control the number of Pods or scale in a DaemonSet because it runs Pods per Node.
Conclusion
In this article, we have seen how to use a DaemonSet in Kubernetes. We have discussed the reasons and its primary use cases.
These types of Kubernetes objects will be very useful in some of the scenarios we have discussed and/or in complex architectures that include a complete logging stack, for example, we could apply Fluentd with Kibana and Elasticsearch.