Prerequisites: configmap, pod-volume

A sidecar is a helper container that runs in the same Pod as the main application and adds a capability without putting that capability into the application image. It shares the Pod lifecycle and can share network or volumes with the main container.

The Grafana Pod’s main container renders dashboards and serves the web application. Its grafana-sc-dashboard sidecar performs a separate job: discover dashboard ConfigMaps and maintain files for Grafana to read.

The live sidecar is configured with these environment variables:

VariableLive valueMeaning
METHODWATCHKeep watching for additions, updates, and deletions
LABELgrafana_dashboardSelect ConfigMaps with this label
FOLDER/tmp/dashboardsDefault output directory
NAMESPACEALLSearch all namespaces allowed by its RBAC
FOLDER_ANNOTATIONgrafana_folderLet each ConfigMap choose a subfolder

Its loop is small:

watch API -> select labeled ConfigMap -> read data keys -> write files -> keep watching

This is the general sidecar pattern, implemented here as an ordinary long-running container. Kubernetes also documents a native sidecar form inside initContainers using container-level restart policy. That is not the form rendered by the Grafana Helm chart in devata. The distinction matters when reading the Pod YAML: behavior defines the role, while the field placement defines lifecycle semantics.

Why not an init container?

An init container would copy the dashboard once before Grafana starts and then exit. It would not handle a ConfigMap update unless the Pod restarted. The dashboard sidecar stays alive and watches, so a Git change can flow through Argo and the ConfigMap into the existing Pod.

Why not make Grafana watch Kubernetes directly?

Grafana’s file provider knows how to load dashboard files. It does not need Kubernetes API awareness. The sidecar is an adapter between two contracts:

  • Kubernetes contract: labeled ConfigMaps available through the API;
  • Grafana contract: dashboard JSON available on a local filesystem.

Separating the contracts keeps Grafana stock, keeps Kubernetes permissions out of the Grafana process, and makes failures easier to locate.

Observe the helper

pod=$(kubectl -n monitoring get pod \
  -l app.kubernetes.io/name=grafana \
  -o jsonpath='{.items[0].metadata.name}')
 
kubectl -n monitoring logs "$pod" -c grafana-sc-dashboard --tail=100
 
kubectl -n monitoring get pod "$pod" \
  -o jsonpath='{range .status.containerStatuses[*]}{.name}{" ready="}{.ready}{" restarts="}{.restartCount}{"\n"}{end}'

If the ConfigMap exists but the file does not, this sidecar and its permissions are the first place to inspect.

Official reference: Sidecar containers.