Software Engineering WikiSE Wiki

Helm

Render, install, upgrade and roll back Helm releases, and understand the template and values behaviour behind most chart surprises.

Reviewed MarkdownEdit

On this page

Cheatsheet#

Commands target Helm 4. Helm 3 differences are noted where they matter.

TaskCommand
See the YAML before applyinghelm template my-app ./chart -f values.yaml
Validate against the live APIhelm upgrade --install my-app ./chart -f values.yaml --dry-run=server
Diff against what is installedhelm diff upgrade my-app ./chart -f values.yaml (helm-diff plugin)
Install or upgrade idempotentlyhelm upgrade --install my-app ./chart -f values.yaml -n my-namespace
Wait for readinesshelm upgrade --install my-app ./chart --wait --timeout 5m
Roll back automatically on failurehelm upgrade --install my-app ./chart --rollback-on-failure
Values actually in effecthelm get values my-app --all
Manifests actually appliedhelm get manifest my-app
Release historyhelm history my-app
Roll back one revisionhelm rollback my-app
Lint a charthelm lint ./chart -f values.yaml
Resolve dependencieshelm dependency update ./chart
Show a chart’s default valueshelm show values oci://registry.example.com/charts/my-app --version 1.2.3
Uninstall but keep historyhelm uninstall my-app --keep-history

Render before installing#

Helm is a template engine plus a release ledger. helm template runs the engine locally and prints manifests. helm install and helm upgrade do the same, send the result to the API server, and record the release in a Secret named sh.helm.release.v1.<release>.v<revision> in the release namespace.

helm template my-app ./chart -f values.yaml | less
helm template my-app ./chart -f values.yaml | kubectl apply --dry-run=server -f -
helm upgrade --install my-app ./chart -f values.yaml --dry-run=server   # output includes Secrets unless --hide-secret
helm diff upgrade my-app ./chart -f values.yaml                          # plugin: databus23/helm-diff

--dry-run=server sends the manifests through admission, which catches schema errors, webhook rejections and quota problems that local rendering cannot see. It also lets lookup read the cluster; with helm template or --dry-run=client, lookup returns an empty map.

Installing and upgrading releases#

helm upgrade --install my-app ./chart \
  -n my-namespace --create-namespace \
  -f values.yaml -f values.prod.yaml \
  --set image.tag=1.4.2 \
  --wait --timeout 10m \
  --rollback-on-failure

Values merge in this order, later winning: chart values.yaml, then each -f file left to right, then --set flags left to right. Maps merge key by key. Lists replace whole, so --set ingress.hosts[0].host=example.com discards every other entry in that list. Use --set-string to stop 1.10 becoming a float and true becoming a boolean.

Helm 4 flagHelm 3 equivalentBehaviour
--wait (same as --wait=watcher)--waitBlocks until resources are ready, using kstatus in Helm 4
no --wait (hookOnly)no --waitWaits for hooks only, then marks the release deployed
--wait=legacy--waitThe Helm 3 readiness checks
--rollback-on-failure--atomicRolls back on failure and turns on --wait; --atomic still works with a deprecation warning
--force-replace--forceDeletes and recreates resources that fail to update; causes downtime
--server-side=auto (default)client-side apply onlyNew releases use server-side apply; upgrades keep the release’s previous method

--rollback-on-failure suits CI. Leave it off when you need the failed state in the cluster to debug. --timeout (default 5m) applies to each wait and hook, not the whole command.

With server-side apply, a field owned by another manager (an HPA on replicas, a controller on annotations) fails the upgrade with a conflict. Remove the field from the chart, or pass --force-conflicts to take ownership.

Upgrades do not reuse the previous release’s values unless asked. --reuse-values merges new overrides into the last release’s values and ignores new chart defaults. --reset-then-reuse-values starts from the new chart defaults, applies the last release’s values, then the overrides; prefer it when a chart version adds keys.

helm list -A                                        # releases and revisions
helm get values my-app --all                        # user values merged with chart defaults
helm get manifest my-app | kubectl diff -f -        # drift between the release record and the cluster
helm status my-app

Rolling back a release#

Every revision stores the rendered manifests, so rollback re-applies an earlier snapshot rather than re-rendering the chart. It creates a new revision number.

helm history my-app
helm rollback my-app 4 --wait

Rollback does not undo side effects. Schema migrations run by hooks, external resources and deleted PVCs stay as they are.

When a release is stuck#

A release in pending-install, pending-upgrade, pending-rollback or uninstalling means the Helm process exited before writing the final status, usually because it was killed or timed out. The cluster objects are not corrupted; the ledger never received its last write. The error for this state is another operation (install/upgrade/rollback) is in progress, even when no Helm process is running.

helm history my-app -n my-namespace                  # find the last revision with status deployed
helm rollback my-app <last-good> -n my-namespace     # usually clears the pending state
kubectl get secret -n my-namespace -l owner=helm,name=my-app   # the ledger itself

Deleting a release Secret

kubectl delete secret sh.helm.release.v1.my-app.v7 -n my-namespace removes revision 7 from the ledger. It is a last resort when rollback fails. Delete only the pending revision; deleting every revision makes Helm treat the next install as new and fail on the existing objects.

Writing a chart#

chart/
  Chart.yaml            # apiVersion: v2, name, version, appVersion, dependencies
  values.yaml           # defaults, every key documented
  values.schema.json    # optional JSON Schema, validated on install, upgrade, lint and template
  templates/
    _helpers.tpl        # named templates: fullname, labels, selectors
    deployment.yaml
    NOTES.txt           # printed after install
{{- define "chart.labels" -}}
app.kubernetes.io/name: {{ include "chart.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}
Template featureBehaviour
{{- and -}}Trim whitespace on the left or right; the cause of most YAML indentation failures
| nindent 4Newline, then indent the block by 4; use it instead of manual spacing
include vs templateinclude returns a string you can pipe to nindent; template cannot be piped
required "msg" .Values.xFail rendering with a readable message instead of producing invalid YAML
default{{ .Values.x | default "y" }}: empty string, 0, false and nil all count as unset
toYamlEmit a values subtree: {{- toYaml .Values.resources | nindent 12 }}
lookupRead live cluster state; empty map during helm template, client dry-run and in Argo CD
.Release.IsUpgradeBranch between install and upgrade
tplRender a string from values as a template, for user-supplied snippets
Hookshelm.sh/hook: pre-upgrade on a Job runs it before the release manifests are applied

Selector labels must be stable. spec.selector.matchLabels is immutable on Deployments and StatefulSets, so putting app.kubernetes.io/version in the selector makes every upgrade that changes the version fail.

Charts with apiVersion: v2 work unchanged in Helm 4. Chart API v3 is experimental and needs HELM_EXPERIMENTAL_CHART_V3.

Chart dependencies#

# Chart.yaml
dependencies:
  - name: postgresql
    version: "16.x.x"
    repository: oci://registry.example.com/charts
    condition: postgresql.enabled
helm dependency update ./chart     # resolve ranges, write Chart.lock, download charts/*.tgz
helm dependency build ./chart      # download exactly what Chart.lock pins

Set subchart values under the subchart’s name (postgresql.auth.database). global: is the only key that parent and subcharts all see. Commit Chart.lock and run dependency build in CI; otherwise builds pick up new upstream versions within the range.

For OCI registries, helm registry login takes the host name only in Helm 4 (helm registry login registry.example.com), not a URL with a path.

Named templates and helpers#

_helpers.tpl holds named templates. Files starting with _ are never rendered as manifests, only defined. Names are global across the chart and its subcharts, so prefix them with the chart name to avoid a subchart silently overriding labels.

{{/* templates/_helpers.tpl */}}
{{- define "my-app.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-%s" .Release.Name .Chart.Name | trunc 63 | trimSuffix "-" }}   # 63 is the DNS label limit
{{- end }}
{{- end }}

{{- define "my-app.selectorLabels" -}}
app.kubernetes.io/name: {{ .Chart.Name }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}
# templates/deployment.yaml
metadata:
  name: {{ include "my-app.fullname" . }}
  labels:
    {{- include "chart.labels" . | nindent 4 }}
spec:
  selector:
    matchLabels:
      {{- include "my-app.selectorLabels" . | nindent 6 }}
  template:
    metadata:
      annotations:
        checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }}   # restart pods when the ConfigMap changes
    spec:
      containers:
        - name: app
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
          env:
            - name: DB_HOST
              value: {{ required "database.host is required" .Values.database.host | quote }}
          args: {{- toYaml .Values.args | nindent 12 }}
          command: {{ tpl .Values.commandTemplate . | quote }}   # values may reference {{ .Release.Name }}

Inside define, the dot is whatever the caller passed. include "x" . passes the full context; include "x" .Values.foo passes only that subtree, and .Release is then unreachable unless you use $, which always refers to the root context. range and with rebind the dot the same way.

The checksum annotation pattern is the standard way to restart pods on configuration change; Helm has no built-in trigger because the Deployment spec does not otherwise change.

Hooks and tests#

Hooks are ordinary manifests with a helm.sh/hook annotation. Helm applies them at the named phase, waits for Jobs and Pods to finish, and only then continues. A hook resource is not part of the release: helm uninstall does not remove it and helm get manifest does not list it (helm get hooks does).

apiVersion: batch/v1
kind: Job
metadata:
  name: {{ include "my-app.fullname" . }}-migrate
  annotations:
    "helm.sh/hook": pre-upgrade,pre-install
    "helm.sh/hook-weight": "-5"                                   # lower runs first, string-quoted integer
    "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded   # keep failed Jobs for kubectl logs
spec:
  backoffLimit: 0
  template:
    spec:
      restartPolicy: Never
      containers:
        - name: migrate
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
          args: ["migrate", "up"]
HookRuns
pre-install, post-installAround the first install of a release
pre-upgrade, post-upgradeAround every upgrade, before and after the manifests are applied
pre-delete, post-deleteAround helm uninstall
pre-rollback, post-rollbackAround helm rollback
testOnly on helm test

A failing hook fails the release. With the default before-hook-creation policy the failed Job stays until the next run, so kubectl logs job/my-app-migrate shows why. --no-hooks skips them, which is how to upgrade past a migration that has already been applied by hand.

Tests are Pods or Jobs with helm.sh/hook: test under templates/tests/. They run against the installed release and succeed when the container exits 0.

# templates/tests/connection.yaml
apiVersion: v1
kind: Pod
metadata:
  name: {{ include "my-app.fullname" . }}-test
  annotations:
    "helm.sh/hook": test
    "helm.sh/hook-delete-policy": hook-succeeded
spec:
  restartPolicy: Never
  containers:
    - name: curl
      image: curlimages/curl
      args: ["-fsS", "http://{{ include "my-app.fullname" . }}:{{ .Values.service.port }}/healthz"]
helm test my-app -n my-namespace --logs --timeout 2m   # --logs prints the test pod logs before cleanup
helm template my-app ./chart --skip-tests              # leave tests out of rendered output

OCI registries#

Charts push to any OCI registry as an artifact with media type application/vnd.cncf.helm.chart.content.v1.tar+gzip. The tag is the chart version; there is no latest. Prefer the digest form in CI so a re-pushed version cannot change what installs.

helm registry login registry.example.com -u "$REGISTRY_USER" --password-stdin <<< "$REGISTRY_TOKEN"
helm package ./chart --version 1.2.3 --app-version 4.5.6     # writes my-app-1.2.3.tgz
helm push my-app-1.2.3.tgz oci://registry.example.com/charts  # reference is the repository, no chart name or tag
helm pull oci://registry.example.com/charts/my-app --version 1.2.3 --untar
helm show chart oci://registry.example.com/charts/my-app --version 1.2.3
helm upgrade --install my-app oci://registry.example.com/charts/my-app@sha256:<digest> -n my-namespace
helm registry logout registry.example.com

Dependencies can point at oci:// repositories in Chart.yaml and helm dependency update resolves them with the same login. Classic HTTP repositories still work (helm repo add, helm repo update, helm search repo my-app -l) but need an index.yaml refresh before a new version is visible, which is the usual reason a version “does not exist”.

Values schema#

values.schema.json is JSON Schema applied to the final merged .Values, so --set typos and wrong types in an environment file fail before anything is rendered. Validation runs on install, upgrade, lint and template; --skip-schema-validation disables it.

{
  "$schema": "https://json-schema.org/draft-07/schema#",
  "type": "object",
  "required": ["image"],
  "properties": {
    "image": {
      "type": "object",
      "required": ["repository"],
      "properties": {
        "repository": { "type": "string", "minLength": 1 },
        "tag": { "type": "string" }
      }
    },
    "replicaCount": { "type": "integer", "minimum": 1 },
    "resources": { "type": "object" }
  },
  "additionalProperties": false
}

additionalProperties: false at the top level catches misspelt keys such as replicasCount, but it also rejects global and subchart keys unless you list them. Subcharts validate their own schema against their own subtree.

Rendering for a target cluster#

helm template has no cluster, so .Capabilities.APIVersions contains only the built-in kinds and .Capabilities.KubeVersion defaults to the Helm build’s Kubernetes version. Charts that branch on .Capabilities.APIVersions.Has "monitoring.coreos.com/v1" render differently offline than in the cluster.

helm template my-app ./chart \
  --kube-version 1.34.0 \
  --api-versions monitoring.coreos.com/v1 \
  --api-versions gateway.networking.k8s.io/v1 \
  --include-crds \                # CRDs from crds/ are otherwise left out of template output
  --is-upgrade                    # sets .Release.IsUpgrade for templates that branch on it

CRDs in the crds/ directory are installed once on install and never upgraded; helm upgrade skips them silently. Apply CRD changes with kubectl apply --server-side -f crds/ before upgrading a chart that ships them.

Objects annotated helm.sh/resource-policy: keep survive helm uninstall and are orphaned rather than deleted; the same is true when a template stops rendering them. Use it for PVCs and namespaces the chart creates.

Helm with GitOps#

Argo CD renders charts with helm template and applies the output itself, so no release Secret exists and helm list shows nothing. Helm hooks are mapped to Argo CD sync hooks, and lookup returns empty. Debug with helm template locally and argocd app diff, not helm get. Flux’s helm-controller does run real Helm releases, so helm history works there.

Troubleshooting Helm releases#

SymptomCauseCheck
another operation (install/upgrade/rollback) is in progressRelease left in a pending-* stateWhen a release is stuck
cannot patch ... field is immutableSelector or other immutable field changed between chart versionshelm diff upgrade, then delete and recreate that object deliberately
invalid ownership metadata / exists and cannot be importedObject already exists and was not created by this releaseAdd --take-ownership, or remove the object first
conflict with "<manager>" on upgradeServer-side apply field conflict with another controllerDrop the field from the chart or add --force-conflicts
<no value> or nil pointer evaluatingTemplate reads a values key that does not existhelm get values my-app --all or the values files, then add required or default
YAML parse error on templates/x.yamlIndentation or whitespace trimming wrongThe error names the file and line; render it alone with helm template ... -s templates/x.yaml
Upgrade “succeeds” but pods are brokenNo --wait, so Helm stopped after applyingAdd --wait; inspect with kubectl rollout status
Values change ignored on upgrade--reuse-values kept old values, or the key is under the wrong subcharthelm get values my-app --all
values don't meet the specifications of the schemavalues.schema.json rejected an overrideRead the path in the error, fix the type

Oneliners#

# Every release across the cluster, with chart versions
helm list -A -o json | jq -r '.[] | [.namespace, .name, .chart, .app_version, .status] | @tsv'

# Releases that are not deployed, including failed and pending
helm list -A --all -o json | jq -r '.[] | select(.status!="deployed") | [.namespace,.name,.status] | @tsv'

# What changed between two revisions
diff <(helm get manifest my-app --revision 6) <(helm get manifest my-app --revision 7)

# Images a chart would run, without installing
helm template my-app ./chart -f values.yaml | grep -E '^\s+image:' | sort -u

# Render one template
helm template my-app ./chart -s templates/deployment.yaml

# Find missing values in rendered output
helm template my-app ./chart 2>&1 | grep -n 'nil pointer\|<no value>'

# Pull a chart to inspect it
helm pull oci://registry.example.com/charts/postgresql --version 16.0.0 --untar

# Values a release was installed with, as a reusable file (may contain secrets)
helm get values my-app -o yaml > values.recovered.yaml

# Release ledger Secrets, oldest first (history is capped by --history-max, default 10)
kubectl get secret -n my-namespace -l owner=helm --sort-by=.metadata.creationTimestamp

Upstream reference: Helm 4 overview, chart template guide.