Skip to content

Garbage Collection#

This chapter covers the two main forms of Kubernetes garbage collection and when + how to use them with controllers.

Owner References#

When your object owns another resource living inside Kubernetes, you can put an owner reference on the child object so that when you delete the parent object, Kubernetes will automatically initiate cleanup of the dependents.

This is explained in more detail in Kubernetes.io :: Owners and Dependents.

OwnerReferences are for children

You should use owner references on generated child objects that have a clear owner object within Kubernetes.

To successfully use owner references you need to:

  1. insert the reference on any object you are creating from your reconciler
  2. mark the children as watchable through owner relations

Owner Reference Example#

In the configmapgen_controller example, the controller creates a ConfigMap from a contrieved ConfigMapGenerator custom resource (cmg crd). The example's reconciler for the ConfigMapGenerator objects insert the owner_reference into the generated ConfigMap:

let oref = generator.controller_owner_ref(&()).unwrap();
let cm = ConfigMap {
    metadata: ObjectMeta {
        name: generator.metadata.name.clone(),
        owner_references: Some(vec![oref]),
        ..ObjectMeta::default()
    },
    data: Some(contents),
    ..Default::default()
};

using Resource::controller_owner_ref. It then marks ConfigMap as a dependent type to watch for via relations:

Controller::new(cmgs, watcher::Config::default())
    .owns(cms, watcher::Config::default())

Finalizers#

A finalizer is a marker on a root object that indicates that a controller will perform cleanup if the object is ever deleted. Kubernetes will block the object from being deleted until the controller completes the cleanup. The controller is supposed to remove this marker in the finalizer list when cleanup is done, so that Kubernetes is free to proceed with deletion.

This is explained in more detail in Kubernetes.io :: Finalizers.

Finalizers mark the need for controller cleanup

You should mark objects with a finalizer if it needs external cleanup to run in the event it is deleted.

The main way to use finalizers with controllers is to define a unique finalizer name (many controllers can finalize an object) and make the finalizer helper manage it. The finalizer helper is desingned to be used within a reconciler and split the work depending on the state we are in:

  • have a deletion happened, and we need to cleanup? we are in the Event::Cleanup arm
  • no deletion has been recorded? we are in the normal Event::Apply arm

Finalizers can prevent objects from being deleted

If your controller is down, deletes will be delayed until the controller is back.

Finalizer Example#

In the secret_syncer example, the controller manages an artificially external secret resource (in reality the example puts it in Kubernetes, but please ignore that) on changes to a ConfigMap.

Because we cannot normally watch external resources through Kubernetes watches, we have not setup any relations for the secret. Instead we use the finalizer helper in a reconciler (here as a lambda), and delegate to two more specific reconcilers:

|cm, _| {
    let ns = cm.meta().namespace.as_deref().ok_or(Error::NoNamespace).unwrap();
    let cms: Api<ConfigMap> = Api::namespaced(client.clone(), ns);
    let secrets: Api<Secret> = Api::namespaced(client.clone(), ns);
    async move {
        finalizer(
            &cms,
            "configmap-secret-syncer.nullable.se/cleanup",
            cm,
            |event| async {
                match event {
                    Event::Apply(cm) => apply(cm, &secrets).await,
                    Event::Cleanup(cm) => cleanup(cm, &secrets).await,
                }
            },
        )
        .await
    }
}

in this example, the cleanup fn is deleting the secret (which you should imagine as not living inside Kubernetes), and the apply fn looks like how your reconcile fn normally would look like.

If you run this example locally and apply the example configmap, you will notice you cannot kubectl delete it the object once it has been reconciled once without keeping the controller running; the cleanup is guaranteed to run.

Default Cleanup#

Not every controller needs extra cleanup in one of the two forms above.

If you are satisfied with your object being removed if someone runs kubectl delete, then that's all the cleanup you need.

You only need these extra forms of garbage collection when you are directly in charge of the lifecycle of other resources - inside or outside Kubernetes.

Summary#

In short, if you need to:

  1. Automatically garbage collect child objects? use ownerReferences
  2. Programmatically garbage collect dependent resources? use finalizers

If you are generating resources both inside and outside Kubernetes, you might need both kinds of cleanup (or make a bigger cleanup finalizer routine).