A First Look at the Helm 3 Plan

Earlier this month, Helm moved from a top-level Kubernetes project to the Cloud Native Computing Foundation (CNCF). The CNCF is becoming the parent foundation for best-of-breed open source, cloud-native tools. It is a huge honor for Helm to become part of this organization. And our first big project under the auspices of CNCF is no small undertaking—We are building Helm 3.

A Brief History of Helm

Helm was originally an open source project of Deis. Modeled after Homebrew, the focus of Helm 1 was to make it easy for users to rapidly install their first workloads on Kubernetes. We announced Helm officially at the inaugural KubeCon San Francisco in 2015.
A few months later, we joined forces with Google’s Kubernetes Deployment Manager team and began iterating on Helm 2. Our goal was to maintain the ease of use of Helm, but add the following:

1. Chart templates for customization
2. In-cluster management for teams
3. A first-class chart repository
4. A stable and signable package format
5. A strong commitment to semantic versioning and retaining backward compatibility version-to-version

To accomplish these goals, we added a second component to the Helm ecosystem. This in-cluster piece was called Tiller, and it handled installing and managing Helm charts.

Since the release of Helm 2 in 2016, Kubernetes has seen explosive growth and major feature additions. Role-Based Access Control (RBAC) was added. Many new resource types have been introduced. Custom Resource Definitions (CRDs) were invented. And most importantly, a set of best practices emerged. Throughout all of these changes, Helm continued to serve the needs of Kubernetes users. But it became evident to us that now was the time to introduce some major changes so that Helm can continue to meet the needs of this evolving ecosystem.

This brings us to Helm 3. In what follows, I’ll preview some of the new things on the roadmap.

Saying Hello to Lua

In Helm 2, we introduced templates. Early in the Helm 2 dev cycle, we supported Go templates, Jinja, raw Python code, and even a prototype of ksonnet support. But having multiple template engines simply caused more problems than it solved. So we decided to pick one.

Go templates had four advantages:

1. The library was built into Go
2. Templates were executed in a tightly sandboxed environment
3. We could inject custom functions and objects into the engine
4. They worked well with YAML

While we retained an interface in Helm for supporting other template engines, Go templates became our default. With a few years of experience, we have seen engineers from many industries build thousands of charts using Go templates. And we’ve learned their frustrations:

1. The syntax is hard to read and poorly documented.
2. Language issues, such as immutable variables, confusing data types, and restrictive scoping rules make simple things hard.
3. An absence of the ability to define functions inside of templates makes it even harder to build reusable libraries.

Most importantly, by using a template language, we were essentially reducing Kubernetes objects to their string representation. (In other words, template developers must manage Kubernetes resources as YAML text documents.)

Work on Objects, not YAML Chunks

We repeatedly hear our users asking for the ability to inspect and modify Kubernetes resources as objects, not as strings. But they are equally adamant that however we would choose to provide this, it must be easy to learn and well supported in the ecosystem.

After months of investigating, we decided to provide an embedded scripting language that could be sandboxed and customized. In the top 20 languages, there is only one candidate that fits that bill: Lua.

In 1993, a team of Brazilian computer scientists created a lightweight scripting language designed to be embedded in other tools. Lua has a simple syntax, is broadly supported, and has hovered in the top 20 languages for a long time. IDEs and text editors support it, and there is a wealth of tutorials and books teaching it. This is the type of existing ecosystem that we want to build upon.

Our work on Helm Lua is still very much in its proof-of-concept stage, but we’re looking at a syntax that would be at once familiar and flexible. A comparison of old and new approaches highlights where we are thinking of going.

Here’s what the example alpine Pod template looks like for Helm 2:

 
apiVersion: v1
kind: Pod
metadata:
  name: {{ template "alpine.fullname" . }}
  labels:
    heritage: {{ .Release.Service }}
    release: {{ .Release.Name }}
    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
    app: {{ template "alpine.name" . }}
spec:
  restartPolicy: {{ .Values.restartPolicy }}
  containers:
  - name: waiter
    image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
    imagePullPolicy: {{ .Values.image.pullPolicy }}
    command: ["/bin/sleep", "9000"]

This is an unsophisticated template, but it’s possible to see at a glance all of the template directives (like {{ .Chart.Name }}) that are embedded.

Here’s what that same Pod definition looks like in our draft Lua code:

 
function create_alpine_pod(_)
  local pod = {
    apiVersion = "v1",
    kind = "Pod",
    metadata = {
      name = alpine_fullname(_),
      labels = {
        heritage = _.Release.Service or "helm",
        release = _.Release.Name,
        chart = _.Chart.Name .. "-" .. _.Chart.Version,
        app = alpine_name(_)
      }
    },
    spec = {
      restartPolicy = _.Values.restartPolicy,
      containers = {
        {
          name = waiter,
          image = _.Values.image.repository .. ":" .. _.Values.image.tag,
          imagePullPolicy = _.Values.image.pullPolicy,
          command = {
            "/bin/sleep",
            "9000"
          }
        }
      }
    }
  }

  _.resources.add(pod)
end

We don’t need to go through the example line-by-line to understand what is happening. We can see at a glance that the code defines a Pod. But instead of using a YAML string with embedded template directives, we’re defining the Pod as a Lua object.

Let’s Make That Code Shorter

Because we’re working directly with objects (instead of manipulating a big glob of text), we can take full advantage of scripting facilities. This is most attractive in the way that it opens the possibility of building shareable libraries. We hope that by introducing (or enabling the community to produce) utility libraries, we could even reduce the above to something like this:

 
local pods = require("mylib.pods");

function create_alpine_pod(_)
  myPod = pods.new("alpine:3.7", _)
  myPod.spec.restartPolicy = "Always"
  -- set any other properties
  _.Manifests.add(myPod)
end

In this example, we are taking advantage of the fact that we can treat the resource definition as an object, setting properties with ease while keeping the code succinct and readable.

Templates… Lua… Why Not Both?

While templates are not great for all things, they do have advantages. Go templates represent a stable technology with an established user base and plenty of existing charts. Many chart developers report that they like writing templates. So we are not planning on removing template support.

Instead, we are going to allow templates and Lua to work together. Lua scripts will have access to the Helm templates both before and after they are rendered, giving advanced chart developers the opportunity to perform sophisticated transformations on existing charts, while still making it easy to build Helm charts with templates.

While we’re excited about introducing Lua scripting, we’re also removing a big piece of the Helm architecture.

Saying Goodbye to Tiller

During the Helm 2 development cycle, we introduced Tiller as part of our integration with Deployment Manager. Tiller played an important role for teams working on the same cluster—It made it possible for multiple different operators to interact with the same set of releases.

But Tiller acted like a giant sudo server, granting a broad range of permissions to anyone with access to Tiller. And our default installation pattern was permissive in its configuration. DevOps and SREs therefore had to learn additional steps when installing Tiller into a multi-tenant cluster.

Furthermore, it turned out that with the advent of CRDs, we no longer needed to rely upon Tiller to maintain state or act as a central hub for Helm release information. We could simply store this information as separate records in Kubernetes.

Tiller’s primary goal could be accomplished without Tiller. So one of the first decisions we made regarding Helm 3 was to completely remove Tiller.

A Security Improvement

With Tiller gone, the security model for Helm is radically reduced. User authentication is delegated to Kubernetes. Authorization is, too. Helm permissions are evaluated as Kubernetes permissions (using the RBAC system), and cluster administrators can restrict Helm permissions at whatever granularity they see fit.

Releases, ReleaseVersions, and State Storage

Without Tiller to track the state of various releases in-cluster, we need a way for all Helm clients to cooperate on managing releases.

To do this, we are introducing two new records:

1. Release: This will track a particular installation of a particular chart. So if we do a helm install my-wordpress stable/wordpress, this will create a release named my-wordpress and track the lifetime of this WordPress installation.
2. ReleaseVersion: Each time we upgrade a chart, Helm will need to track what changed, and whether the change was successful. A ReleaseVersion is tied to a release, but records just the details of the upgrade, rollback, or deletion. So when we do helm upgrade my-wordpress stable/wordpress, the original Release object will stay the same, but a new child object, ReleaseVersion will be created, packaging the details of this upgrade operation.

Releases and ReleaseVersions will be stored in the same namespace as the chart’s created objects.

With these features, teams of Helm users will still be able to track the records of Helm installs inside of the cluster, but without needing Tiller.

But Wait, There’s More!

In this article, I have tried to highlight some of the big changes coming to Helm 3. But this list is by no means exhaustive. The full plan for Helm 3 includes features like improvements to the chart format, performance-oriented changes for chart repositories, and a new event system that chart developers can tie into. We’re also taking a moment to perform what Eric Raymond calls code archeology, cleaning up the codebase and updating parts that have languished over the last three years.

With Helm’s entrance into CNCF, we are excited not just about Helm 3, but also about Chart Museum, the brilliant Chart Testing tool, the official chart repo, and other projects under the Helm CNCF umbrella. We feel that good package management for Kubernetes is just as essential to the cloud-native ecosystem as good package managers are for Linux.