Longhorn and Velero, two different durability boundaries

Document contract

• Role: architecture and case-study successor
• Truth boundary: intended design from lab; failure and restore claims link dated experiment evidence
• Last verified: lab default branch and live read-only storage inspection, 2026-07-26

Prerequisites: kubernetes, longhorn-volume

Longhorn and Velero solve different loss scenarios. Longhorn keeps selected block volumes available when one worker fails. Velero copies selected Kubernetes resources and file data to private object storage outside the cluster. Calling both “backup” erases the failure boundaries that justify having both.

flowchart LR
  app[Stateful workload] --> pvc[PVC using longhorn]
  pvc --> volume[Longhorn volume]
  volume --> a[Replica on OptiPlex XFS volume]
  volume --> b[Replica on second worker]
  app -. selected file-system backup .-> agent[Velero node agent]
  agent --> kopia[Kopia]
  kopia --> r2[Private Cloudflare R2]

What Longhorn protects

The longhorn StorageClass is non-default and uses two replicas. Workloads move to it deliberately, one at a time, with their old local-path claims retained during the rollback window.

Two replicas on different workers allow a volume to remain usable while one worker is unavailable. A dated durability experiment forced the hosting worker down, moved the proof workload, read the same checksum while the volume was degraded, and watched two healthy replicas return after recovery.

That test proves worker-failure behavior for the tested volume. It does not prove survival from:

• deletion that propagates through the storage system;
• loss of both worker disks;
• a full cluster loss;
• corruption copied to both replicas;
• a Talos wipe of the worker whose data path lives inside EPHEMERAL storage.

What Velero protects

Velero writes Kubernetes resource backups and explicitly selected file-system volume data to a private R2 bucket. Kopia handles the file data. Native Longhorn snapshots are not the durable target because they remain inside the same cluster failure domain.

The R2 bucket has a strict operating ceiling. No unattended schedule is declared. A capacity guard measures current object usage, bounds the planned input, reserves space for metadata, and rejects unsafe work. Pod-volume backup is opt-in so an unbounded Prometheus TSDB cannot enter the bucket accidentally.

A dated restore experiment backed up a bounded namespace and Longhorn PVC, deleted the source namespace, restored it under a different name, and recovered the sentinel checksum. That proves the selected path, credentials, repository password, namespace mapping, and file restore worked together at that point.

It does not prove that every workload or PVC is covered.

The asymmetric local disks

The two storage workers do not have identical persistence beneath Longhorn:

• the OptiPlex uses a dedicated XFS Talos user volume at /var/mnt/longhorn;
• the second worker exposes /var/mnt/longhorn from Talos EPHEMERAL storage.

Both paths survive ordinary reboot and upgrade. Only the dedicated user volume is designed to survive a Talos reset independently of EPHEMERAL. The replicas still improve worker availability, but reconstruction planning must account for this asymmetry.

Inspect the current contract

These commands are read-only:

kubectl config current-context
kubectl get storageclass
kubectl get pvc -A
kubectl -n longhorn-system get nodes.longhorn.io
kubectl -n longhorn-system get volumes.longhorn.io
kubectl -n velero get backupstoragelocations.velero.io
kubectl -n velero get backups.velero.io

Read the output as separate questions:

• Which claims intentionally use Longhorn?
• Are both storage nodes schedulable and healthy?
• Does each important volume have the intended replica count and placement?
• Is the off-cluster backup location reachable?
• Which exact resources and volumes were selected by each backup?
• Has a restore, not only a backup, succeeded recently?

Recovery decision table

Check yourself

1. Why does a two-replica volume not replace off-cluster backup?
2. What does a completed Backup object fail to prove?
3. Which worker loss and wipe scenarios expose the difference between a Talos user volume and EPHEMERAL?
4. What transaction would you require before calling a restored workload usable?

References

• Longhorn concepts
• Longhorn on Talos Linux
• Velero file-system backup
• Velero restore reference