Operator SDK Reaches v1.0

On behalf of the Operator SDK maintainers and engineering team as well as the contributing members of the community, we are happy to announce that the Operator SDK project reached the v1.0 milestone.

First released beginning of 2018, the Operator SDK marked the beginning of the Operator Framework. In the same year, the Operator Lifecycle Manager was open-sourced. Both projects build on top of the Operator concept, first laid out by CoreOS almost two years earlier. The framework aims to bring the Operator concept into mainstream Kubernetes and caters to both, developers of Operators as well as cluster administrators and tenants, discovering and using Operators.

The Operator Framework Project

The SDK provides Operator developers with all the necessary means to get started writing a Kubernetes Operator, starting with code scaffolding to avoid some of the boiler-plate code that comes with Kubernetes client libraries when writing Golang Operators. The SDK also provides higher abstraction levels with Ansible-based Operators and converting Helm charts into Operators.

Over time the SDK added a lot of additional scope when it comes to developing Operators, such as continuous testing at various levels.

Operators are seen like cloud services that travel with the cluster and provide managed workloads, so they become responsible for managing important infrastructure like databases, message queues or entire application stacks. Testing is vital to create robust and feature-rich Operators and the SDK provides the means to create unit tests, integration tests as well as functional tests.

Releasing Operators can be as simple as publishing the source code and some example YAML manifests on a Git repository. The SDK supports that but can also automatically create metadata to run the Operator safely in multi-tenant clusters with OLM. This provides means to publish versions of your Operator in a controlled manner where developers get to define the update graph of their Operator and have the ability to release different versions in different update channels.

Finally, the SDK helps developers evolve their Operator over time, be it by adding new APIs and functionality or moving it towards a newer version of the Kubernetes libraries as part of updating the SDK.

What’s new with Operator SDK 1.0

Version 1.0 represents almost 2 years of work to continue to refine the Operator pattern, embed best practices and evolving Kubernetes capabilities when it comes to cluster extensions. In the process, there were many breaking changes between releases and with 1.0 we have reached a level of stability and maturity that provides a solid foundation to build your Operator. Especially, if you are coming from an earlier version, you’ll notice a couple of major changes and new additions.

Kubebuilder for Golang Operators

After several of the SDK features have been upstreamed to the Kubebuilder project and the controller-runtime library, Kubebuilder has been embedded into the SDK as the scaffolding solution for Golang Operators. That means existing Kubebuilder projects can be used as is with the SDK and continue to work.

Kubebuilder and Operator SDK have long been seen as competitors in the field of Operator tooling, when in reality there was collaboration going on for some time already. Now, with the Operator Framework being incubated into CNCF, we are coming together on a single solution that creates Golang Operators (Kubebuilder) while the SDK provides a holistic experience in creating, testing, bundling and releasing Operators.

New Project Layout and CLI switches

As a result of the above, the SDK adopted the Kubebuilder’s project layout and the Makefile-based approach to build and maintain it. The SDK maintainer contributed a plugin system to Kubebuilder which allows extending the functionality with custom behavior in this project's workflow. As a result, developers continue to leverage the SDK features and artifact format while Kubebuilder is leveraged under the hood to generate the scaffolding and retains the definition of the Golang project. With the new layout comes a new central PROJECT configuration file which can be used to influence several behaviors of scaffolding and building binaries. This reduces the overall amount of subcommands and command line switches.

Operator Bundle Format

Part of the new artifact output is an Operator Bundle, an OLM concept that developers can opt-in to, in order to release a version of their Operator into a catalog, so it can be installed by OLM. OLM allows developers to worry less about auxiliary tasks like pre-registering CRDs, converting CRDs, avoiding collision with existing Operators, orchestrating Operator updates, or configuring the cluster to use webhooks. This is all happening as part of OLM’s lifecycle definition of a bundle.

The bundle format uses standard container technology for shipping the metadata and allows developers to publish their own Operator update streams in catalogs. This is very similar to how OCI artifact spec plans to ship non-runnable image artifacts through registries. The SDK and OLM will support that, once that specification is finalized.

In particular, the native SDK support for Bundles alleviates the need for the developer to generate and customize a ClusterServiceVersion YAML manifest, required by OLM. This process is now fully automated using kustomize internally.

Bundles are agnostic of the type of Operator and not only contain all metadata to run it but also to display it on websites like OperatorHub.io. Furthermore, they can contain functional tests which makes this format well suited for continuous testing in a CI pipeline.

Custom Functional Tests

Next to Golang unit and end2end tests, the SDK offers a way to declaratively define black-box tests using the kuttl project. As part of the scaffolding, developers can simply define YAML manifests that specify the expected before-and-after state of a cluster when one of the Operator’s services is used. To implement application-specific tests the test harness of the SDK, scorecard, provides the ability to ship custom code in container images as well, which can be referenced in the test suite. Because this metadata travels with the Operator bundle it allows for functional testing of the Operator without the source code or the project layout being available.

New Integration Tests

Previously the SDK featured its own end2end test framework. Unlike the name implied it only operated at the level of Go structures and allowed developers to provide mocks for most of the system aspects to test larger parts of their Operator in isolation. However it did not see a lot of adoption. In order to align on an already established upstream technology envtest from controller-runtime was chosen as a vehicle to mimic parts of a Kubernetes environment to test components of your Operator in. Please see the migration documentation to the new integration tests.

OLM integration

The SDK provides an easy path to bundle your Operator to be managed by OLM in multi-tenant clusters.

Beyond the aforementioned support to create the bundle, the SDK also has shortcuts to run your Operator on a cluster where OLM is present. Note that, at this point, this does not yet leverage the Operator bundle format, but it allows developers to quickly test if the Operator deploys and behaves correctly when managed via OLM.

Furthermore, for Golang Operators the SDK supports webhooks to be created via Kubebuilder and managed by OLM.

Operator SDK Library

Many of the Operator SDK features have been upstreamed into either Kubebuilder or controller-runtime at this point but not everything fits either of those projects. To separate concerns and also help reduce some of the code changes that updates to either of those projects typically incur on an Operator project, the SDK introduced a separate library project with its own independent versioning.

Breaking changes there do not always cause breaking changes in the SDK and allow Golang Operator developers to opt-in to more helpers of the SDK, for example to help emit metrics or to provide meaningful status conditions for good UX.

These are the highlights of the SDK 1.0 release, be sure to check out the changelog to get all the details as well as our upgrade guide to move to 1.0 if you have an existing project using an older version.

Outlook

The 1.0 release of the SDK is a milestone that builds upon all the learnings of extending the Kubernetes control plane over the past couple of years. It provides the developers with a solid foundation to create mature Operators that leverage modern Kubernetes extensions features like webhooks, v1 CRDs, CRD versioning and OpenAPI schema and validation. It puts an emphasis on thorough and continuous testing while making it easy to use OLM.

For Red Hat, it is also the basis to create a supported version of the SDK that targets software vendors and other partners that want to productize their offering on OpenShift. Stay tuned for more information as we move towards this goal.