Garbage Collection of Custom Resources Available in Kubernetes 1.8

Today Kubernetes 1.8 was released and CRI-O graduated to general availability.

If you’ve been following along with Michael and Stefan’s Kubernetes Deep-Dive series, you’ve heard about how Kubernetes (and its associated APIs) have been designed with extensibility in mind. Kubernetes custom resources and custom controllers are two of the primary abstractions for extending the platform.

In Kubernetes, objects can "own" other objects for lifecycle management purposes; for example, a deployment can own a replication controller, which in turn owns a number of pods. If the deployment is deleted, Kubernetes contains a garbage collector which can clean up the corresponding replication controller and pods.

Kubernetes isn't limited to the object types provided out of the box. Custom resources allow the creation of new object types which integrate seamlessly with the Kubernetes API and client tooling and are supported by the garbage collector as of Kubernetes 1.8.

Try Custom Resource Garbage Collection

Here's a simple example of custom resource garbage collection in action.

First create a new type of resource, a Foo:

# foo.yaml

kind: CustomResourceDefinition

apiVersion: apiextensions.k8s.io/v1beta1

metadata:

  name: foos.stable.example.com

spec:

  group: stable.example.com

  version: v1

  scope: Namespaced

  names:

    plural: foos

    singular: foo

    kind: Foo

$ kubectl create -f foos.yaml

Create an instance of a Foo, which will be the owner of another object:

# owner.yaml

kind: Foo

apiVersion: "stable.example.com/v1"

metadata:

  name: owner

spec:

  trashDay: friday

$ kubectl create -f owner.yaml

You should now be able to get the YAML of owner using kubectl:

$ kubectl get foos owner -o yaml

Create a simple core Kubernetes resource (a config map) which depends on the Foo, replacing the uid field with the UID of owner from the previous step:

# dependent.yaml

kind: ConfigMap

apiVersion: v1

metadata:

  name: dependent

  ownerReferences:

  - kind: Foo

    apiVersion: "stable.example.com/v1"

    name: owner

    # Replace with UID of `owner`

    uid: "1d73cee9-56a4-11e7-bfd9-acbc32bcadbb"

$ kubectl create -f dependent.yaml

Finally, delete owner and notice that dependent will soon follow:

$ kubectl delete foos owner

Housekeeping for Your Custom Controllers

Imagine you're building a service on Kubernetes which allows users to create a complete application by creating a single instance of custom resource, a FancyApplication. You've written a custom controller which responds to the creation of a FancyApplication by creating many core Kubernetes resources on behalf of the user, such as deployments, secrets, and services.

The object graph your controller created may look something like this:

Let's say the user wants to delete their FancyApplication and everything associated with it.

Prior to Kubernetes 1.8, controllers are responsible for understanding how to track and delete all resources owned by a custom object. Since this varies from controller to controller, garbage collection is cumbersome and likely contains bugs.

Now, when the user deletes FancyApplication, Kubernetes takes care of the cleanup. You can focus on FancyApplication logic in your controller and leave the housekeeping to Kubernetes.