In this new post, we are going to talk about another Kubernetes object, DaemonSet, where we will see how to use a DaemonSet in Kubernetes, what a DaemonSet is, and why and for what purpose we need to create them within our Kubernetes architectures.

What is a Daemonset?

A DaemonSet is a Kubernetes object, and like other Kubernetes objects, it manages a set of Pod replicas.

The main function of a DaemonSet is to ensure that one or a set of Pods are running on the specified node or nodes. When you delete a DaemonSet, all the Pods that were created will also be deleted.”

What do we use a DaemonSet for?

All Kubernetes objects usually have a specific and concrete purpose, and in the case of a DaemonSet, the most common uses are:

1. Running a daemon to enable storage on each node, for example, Ceph, an open-source storage platform.
2. Having a daemon for log collection, which is a very common use case. For example, tools such as Fluentd or Logstash.
3. Running a daemon for continuous monitoring, such as Prometheus.

As we mentioned, a DaemonSet is responsible for managing and creating Pods. So, why not use a deployment or create Pods directly? The recommendation is to use a DaemonSet whenever you want and it’s important for a Pod to run on one or multiple hosts, although you can use a deployment or create Pods.

How are DaemonSets scheduled?

Usually, Kubernetes’ default scheduler schedules the Pod running on a node. However, in the case of DaemonSets, the DaemonSet controller can schedule the Pod.

If you choose to do scheduling with the DaemonSet controller, it has its own set of characteristics. Firstly, the ‘Pending’ state disappears because this scheduler doesn’t use it. Secondly, if Pod preemption is enabled (the preference), the DaemonSet controller will schedule without taking any priority or preference into consideration.

To use the default scheduler, you can make use of ‘ScheduleDaemonSetPods,’ which allows you to add NodeAffinity to bind it to a node by affinity

Example of how to use a DaemonSet in Kubernetes

Next, we are going to create a DaemonSet for Fluentd, which is a perfect example of using a DaemonSet as it allows us to collect and unify information from different services and consume it to understand and make better use of all that information.

Before creating and running our test DaemonSet, let’s create a new namespace where we will run it.

kubectl create namespace logging

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: fluentd-elasticsearch
  namespace: logging
  labels:
    k8s-app: fluentd-logging
spec:
  selector:
    matchLabels:
      name: fluentd-elasticsearch
  template:
    metadata:
      labels:
        name: fluentd-elasticsearch
    spec:
      tolerations:
      # para ejecutar en el nodo master, con minikube es el por defecto
      - key: node-role.kubernetes.io/master
        effect: NoSchedule
      containers:
      - name: fluentd-elasticsearch
        image: quay.io/fluentd_elasticsearch/fluentd:v2.5.2
        resources:
          limits:
            memory: 200Mi
          requests:
            cpu: 100m
            memory: 200Mi
        volumeMounts:
        - name: varlog
          mountPath: /var/log
        - name: varlibdockercontainers
          mountPath: /var/lib/docker/containers
          readOnly: true
      terminationGracePeriodSeconds: 30
      volumes:
      - name: varlog
        hostPath:
          path: /var/log
      - name: varlibdockercontainers
        hostPath:
          path: /var/lib/docker/containers

Save the previous DaemonSet in a file and name it ‘fluentd-daemonset.yaml,’ then we execute it as follows:

kubectl apply -f fluentd-daemonset.yaml -n logging

We will receive a creation message, and then we will check if it has been created successfully.

Run:

kubectl get daemonset -n logging

NAME                    DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
fluentd-elasticsearch   1         1         1       1            1           <none>                   104s

We can see how we have our DaemonSet running in our namespace.

Next, let’s check the pods that have been created for this DaemonSet.

kubectl get pods  -n logging -o wide

NAME                               READY   STATUS    RESTARTS   AGE   IP             NODE       NOMINATED NODE   READINESS GATES
fluentd-elasticsearch-mjt5d        1/1     Running   0          99m   172.17.0.5     minikube   <none>           <none>

Keep in mind that you cannot control the number of Pods or scale in a DaemonSet because it runs Pods per Node.

Conclusion

In this article, we have seen how to use a DaemonSet in Kubernetes. We have discussed the reasons and its primary use cases.

These types of Kubernetes objects will be very useful in some of the scenarios we have discussed and/or in complex architectures that include a complete logging stack, for example, we could apply Fluentd with Kibana and Elasticsearch.