- Part 1: Kustomize Introduction
- Part 2: Kustomize Advanced Features
- Part 3: Kustomize Enhancement with KRM functions (this article)
In the first two articles, we have seen that although simplicity is a key aspect of Kustomize, it is still a powerful tool to manage client-side Kubernetes resources. Nevertheless, there are still some features lacking that we would probably like to have:
- We may want to generate complex resource structures from a simple configuration containing all the best practices defined by a platform team
- We may want to run validations on generated resources on the client side to not have to wait for the real deployment
- We may want to inject cross-cutting functionality in to all resources like setting a sidecar.
Kustomize has always had some kind of means to add functionality by building plugins. From Kustomize’s point of view, the generators and transformers are already kind of plugins anyway. In the past, however, the plugins were proprietary and complicated to build in the form of executable scripts or go plugins. This way is becoming deprecated and will be replaced by a more KRM-driven approach. One of the first steps is to introduce KRM functions as an alpha feature.
What is KRM?
KRM stands for Kubernetes Resource Model and is a definition of the API-focus design that defines the Kubernetes core. This is probably one of the most important reasons for the success of Kubernetes as it defines more or less a common API to request all kinds of operational needs in the form of resources. Every requirement such as load-balancing, auto-scaling, routing, placement of pods, and gateway is defined as a resource. Additionally, Kubernetes supports the extension of known resources in the form of CRDs (Custom Resource Definition). So in short, KRM defines the way we interact with Kubernetes via resources.
What are KRM Functions?
The KRM function specification defines how to manipulate a list of KRM resources. Simplified, it describes how a containerized application (the function) shall receive a list of resources and additional, optional, configurations. It also describes how it provides a new list of resources and optionally a result:
The function uses the resources and results as follows:
- If the function generates new resources or manipulates resources given as an input, it will output the new list of resources
- If the function validates the given resources based on rules, it will output the result
You can roughly group functions into Transformers and Generators on the one side and Validators on the other.
Let’s have a look at an example input for a function described in the specification
The input is read from stdin as a yaml structure named
ResourceList with two parts.
functionConfig This describes the configuration for this function. We will see later how we build our configuration and function
items This is a list of items our function will work on.
After the function runs, an example output looks like this:
It has two parts, again.
items The new list of items is used for further processing. Deleted items will no longer be processed by the next step.
A list of structural validation results. If the function has an unstructured result, e.g. it can not parse the input, it would return with a status code other than
What is Kustomize in the KRM Function Context?
Besides the function definition, the specification defines an orchestrator for functions. In the end, this is a tool that would prepare the correct input for a function and will take the output to either stop further processing (in case of a failure) or to forward it to the next function in the chain.
Kustomize is an orchestrator from the perspective of a function but the specification is not limited to Kustomize. There are other implementations of an orchestrator available.
Build a KRM Function
Let’s create our function to see how it works with Kustomize. The goal is that we want to reduce the errors when creating typical resources defining an application. Our application consists of a deployment and a service. It exposes an endpoint. When we do this manually, we have to create a Deployment resource and a Service resource which are correctly configured so that the Service endpoints are mapped to the correct port in the Deployment.
Therefore, what we want to build is a generator that can generate these resources for us based on a simple configuration with a few fields. The configuration we are aiming for is as follows:
As you can see, a KRM function configuration is also expressed in the form of a Kubernetes resource. This configuration is, however, only used on the client side. Therefore, Kubernetes does not need to know this structure, only Kustomize. The 3 values we want to make configurable are the port, the image, and the size. The size is a T-shirt size value to define how much CPU and memory the service gets. The values are
large to make it easier for developers to define.
Whilst this is a very simple definition of an app that ignores many aspects such as auto-scaling or configurations and secrets, based on this setup we could extend the function later with everything we want to have.
So, first we have to create a new golang project containing the code for our function.
The next step is to create templates for the deployment and the service.
These are inline golang templates defined as variables we can use in the main function. We can already see the if-else block to map the T-shirt size to concrete CPU and memory limits.
Next, we need to install some golang packages which help to implement the function.
Kustomize provides some helper structs and functions to simplify the implementation.
The next step is to implement the function.
The three structs map to our configuration above. Kustomize provides some helper functions to build functions for some typical use cases, in our case a template-based processor (see
framework.TemplateProcessor). We provide some templates and the configuration. Then, the framework fills the configuration with the real values from the configuration before merging the configuration with the templates.
The filter function
filterAppFromResources is just a precaution in case the function configuration is also present in the resource list. This function filters out the App resource before delivering the result.
There are other helper functions for other use cases. For more examples have a look at the examples in the package documentation.
command.AddGenerateDockerfile, we can automatically generate a Dockerfile for our function.
The corresponding Dockerfile looks like this:
Now, we can generate our containerized function
Testing the New Function
We can provide a test file that simulates the input as it would come from an orchestrator. The test file looks like this:
If we run our function against this, we would see the following:
In the output, we see the two new resources (Deployment, Service) based on our templates. We also see the limits set based on the T-shirt size as we defined them in the function. If we would change the size to
large, other limits would be set. Additionally, the resource of kind App is removed from items (as said Kustomize would not provide this in the items list, but other orchestrators could).
Use your Function with Kustomize
So how can we run this function with Kustomize?
First, we create a new Kustomize project:
Then we define the configuration for the function:
This is the same configuration as above in the functionConfig but we have an additional annotation. The annotation is needed for Kustomize to know the link between this configuration and the function to be called.
Next, we have to link this configuration to our
You can add functions under the
validators block. Depending on the block, the function will work slightly differently. If we put our function configuration into the validators block, Kustomize will ignore the new items list and will only use the results. On the other hand, generator functions will always run before transformer functions.
That’s all. Now we can run Kustomize.
We have to run Kustomize with the
--enable-alpha-plugins flag as it is still quite new and under active development. The good thing is the functions can be used in parallel with all other known Kustomize features like, for example, the name prefix. Let’s add a name prefix:
Then we build the resources:
We see that the resources generated by our function are also correctly modified with the name prefix.
Other KRM Function Orchestrators
Kustomize mixes KRM functions with their traditional built-in features to provide further possibilities to adapt to our needs. Nevetheless, it is quite difficult to understand in which order the KRM functions and built-in features run and this can only be influenced to a certain extent. We are not completely free to run generator, transformer, or validator functions in any order.
Another tool supporting KRM functions is kpt. In fact, KRM functions in kpt are first-class citizens. Kpt provides a possibility to define a pipeline of KRM functions running on resources. For our example above, a corresponding kpt file could look like this.
Here, we see a pipeline definition with mutators. We have to rename the image as kpt expects a registry otherwise it uses
gcr.io/kpt-fn as default.
We also have to define a
.krmignore file to ignore the Kptfile itself in the final render output. It works similarly to .gitignore or .dockerignore.
Now we can run the same function with kpt.
We see the same resources in the output here as with Kustomize.
This is just a glimpse into the kpt features but we can see that we are able to use the same KRM function with Kustomize and kpt. Both are following the KRM function specification to prepare the input for the KRM function and to run it.
Verdict and Alternatives
KRM functions are a great extension for Kustomize for client-side validation and the generation of new resources. We can combine all the features we already know from Kustomize and introduce best practices by providing a simplified interface to the user/developer via client-side CRDs (here in the form of the App resource).
Another tool solving similar issues is Crossplane as it provides so-called XRDs (Composite Resource Definitions) as an interface for developers.
Both fulfill the same need for a simplified interface.
On the Kustomize side, KRM functions can do more than just generate resources. They can modify existing resources and validate them, all on the client side before applying them to the cluster. It lacks on the other hand the possibility to replace the best practice in the background without telling the developers that there is a new version of the KRM function.
Crossplane is more of a meta-framework for handling all kinds of resources. It also includes CRDs to generate external resources like databases directly on the provider platform like AWS or GCP. It focuses more on the platform team’s needs and limits the debugging possibilities for developers.
So which tool fits best depends on your current needs. Additionally, these tools are not mutually exclusive, so you can mix them as needed. But always be aware that every additional tool increases the complexity and makes testing and debugging more difficult.