Kubernetes is the most widely used orchestration platform in the world. This platform is becoming a standard in the DevOps world. His popularity has led to the emergence of many projects at different scales to facilitate and optimize the management of internal Kubernetes resources, Helm is a good example.
Helm is the package manager of the Kubernetes universe. It is used by most of the DevOps community for its structuring and ease of YAML file templatization needed to define the Kubernetes resources to create. Like any package manager, It also facilitates the lifecycle of artifacts created by the simplicity of managing the versioning of these.
Lifecycle obviously means supervision and automation of updates. With Kubernetes having a fast lifecycle based on a release every three months, it is expected to set up and follow a process of sustained updating of the platform and its resources to avoid depreciation and misconfiguration. Helm Charts, in particular, require special attention in this process.
Important Considerations
There is no single recipe for setting up a management procedure to update the platform or its ecosystem.
Nevertheless, several considerations must be taken into account:
- The process must be automated to ensure ongoing supervision of resources. The recurring nature and importance of this practice do not allow us to rely on a specific human action but rather a dedicated program.
- The process must also be configurable to facilitate its integration in different contexts (Cloud, on-premise, open or closed system, etc.).
- The process must be simple in its configuration and use to allow its adoption by any team. The aim is not to add complexity to an already complex system but to facilitate its management.
- The process must be documented, both in the tools used and in the process itself. Keep in mind that sharing knowledge is the key to successful adoption.
- Depending on the context, the process and tools must allow the use of public and private Helm registry. Ideally, public registries will be managed directly by an external trusted entity and only private registries will be configured.
- The process must be able to integrate with existing internal tools again to simplify and facilitate its adoption. The objective is not to add new components to the already existing technical debt.
- The process must determine whether a Helm chart is deprecated or not. This is absolutely vital to limit the increase in technical debt.
- Promoting its adoption also means facilitating the resolution of detected updates and their prioritization. For this, it is strongly recommended to integrate ticket creation in order to share information and prioritize the task in a future sprint.
This brief list of considerations obviously depends on the number of Kubernetes operators available, the time allocated to manage the clusters, and the automation ecosystem surrounding them. It is obviously not easy to keep such a dynamic system up to date but there are always little tricks to put in place to facilitate our daily life.
What Is Nova?
Nova is an open-source command-line interface developed by Fairwinds for cross-checking the Helm charts running in your cluster with the latest versions available. Basically, Nova will let you know if there is any out-of-date or deprecated Helm chart on a remote cluster, so you can make sure you are always aware of updates.
Fairwinds provides a binary to easily deploy Nova on multiple operating systems. Its portability makes it easy to run it in a container to facilitate its integration in a pipeline.
In the example above, Nova identified that some major core applications have updates available like MetalLB, Prometheus, Falco, Victoria Metrics, etc. Nova also identified a Helm chart version deprecated that deserves attention, Vector Agent.
Currently, Nova cannot check multiple clusters at once, so you must run and configure the command line for each cluster you want to check. Nova can take information from the KUBECONFIG environment variable or a specific file passed to the command line.
How to Configure Nova To Check Private Repositories?
Nova has the distinction of being configurable at the registry level to check the availability and status of a Helm Chart. By default, Nova uses a file that records a large part of the public Helm registries. This “hub” file allows you to scan a Kubernetes cluster to obtain a report on existing resources without any specific configuration. This file is publicly available on Github if you want to take a look.
In secure contexts, Github may not be accessible or the network is closed and therefore access to Helm’s public registries is not allowed. In this type of environment, a private Helm mirror is usually available for downloading Helm Charts. Nova can be used in this type of environment with a simple configuration file listing the internal registry URLs.
This is just an example to demonstrate the way to add Helm registries to Nova, but keep in mind that the main repositories are used by default thanks to the official Helm Hub file.
The internal address configuration does not prevent the use of the Helm hub available on Github, Nova can jump between both to get the information and report the status of the Helm Charts.
This feature is very interesting because it allows:
- Operationalization teams to automatically detect via the Helm hub any new updates on a core element of the Kubernetes ecosystem.
- Development teams to identify an update available for a dependency of their project, whether it is an internal micro-service or an application external to the company.
- DevOps engineers to detect and plan potential production deployment.
- SRE team to limit access to potentially dangerous public resources.
How To Get Notified?
Monitoring the Helm Chart version in an automated way is a good start, but being notified is even better. Indeed, very often the update of systems, software, or other components is deferred because not deemed a priority until the detection of critical severity, for example, the Log4J CVE identified in December 2021.
Due to the development velocity of the ecosystem around Kubernetes, the update process must move from an old/bad prioritization system to a continuous update system in order to limit the impacts and the time required for their deployment.
Notification is therefore essential in this process because it allows one to be warned of the necessary actions, their potential criticity, their impacts and to keep a history in order to better prioritize tasks. It is strongly recommended to keep track of actions taken in the form of a tutorial, documentation, or other formats in order to share knowledge, onboard easily a newcomer, and identify improvement points to gain efficiency in the actions to be carried out.
There are several notification formats:
- An email sent to a group of people in charge of updates.
- A notification in a common channel on a corporate chat tool like Slack, Discord, etc.
- A ticket created automatically in the task management tool, such as a JIRA ticket generated via the Atlassian API.
The combination of the three formats is obviously recommended in order to increase the visibility of the update and the identification of criticality. The objective is to follow up and process the ticket as soon as possible.
Regarding automation, there are also several formats:
- A scheduled task (cronjob on Linux or Kubernetes) running Nova command line and generating an alert in case of identification of an update.
- A step in a weekly maintenance pipeline (Jenkins, Gitlab, Airflow, etc.).
- A supervisory task performed by the monitoring tool with a certain frequency.
- A weekly alert in the team calendar to remember to perform the supervisory task, it is strongly discouraged.
The possibilities are obviously infinite but must nevertheless follow a pattern, the task must be performed at least once a month (recommendation of once a week) and keep things simple in order to promote the adoption of this system of continuous updating. Encourage the use of tools currently mastered by maintenance teams (internal ticketing tool, observability platform, maintenance process, etc.).
Nova’s information processing can be easily included in any monitoring system whether by processing output data, by a Shell script, or processing data in the form of logs. The choice is yours!
Additional Resources
For more information on topics mentioned in this article, please refer to this documentation:
