Storage

Cordium stores everything on Kubernetes persistent volumes provisioned by your cluster's CSI drivers. This page explains how each Cordium feature maps to Kubernetes storage, how to choose the storage backends, and how to troubleshoot storage issues. For the installation requirements, read here.

How Storage Is Used

All the storage objects live in the cordium Kubernetes namespace:

Cordium feature	Kubernetes objects	Requirement
Workspace storage	A `ReadWriteOnce` `PersistentVolumeClaim` named `ws-<WORKSPACE_UID>` per Workspace, sized by the Workspace's storage limit. It is kept while a persistent Workspace is stopped, and deleted when an ephemeral Workspace stops or when a Workspace is deleted.	A `StorageClass` with dynamic provisioning. Required.
`ACCESS_MODE_EXCLUSIVE` Volumes	A `ReadWriteOnce` `PersistentVolumeClaim` named `vol-<VOLUME_UID>` per Volume.	The same as above.
`ACCESS_MODE_SHARED` Volumes	A `ReadWriteMany` `PersistentVolumeClaim` named `vol-<VOLUME_UID>` per Volume.	A `StorageClass` of a multi-writer filesystem backend. Optional.
Template pre-builds and WorkspaceSnapshots	A `VolumeSnapshot` of the Workspace's claim, from which new claims are restored.	The `VolumeSnapshot` CRDs, the CSI snapshot controller, and a `VolumeSnapshotClass` for the CSI driver. Optional.
Growing Volumes	An expansion of the Volume's claim.	A `StorageClass` with `allowVolumeExpansion: true`. Optional.

Since each Workspace claim is attached to a single pod, block storage is the best fit for Workspaces: it is fast, and most block storage CSI drivers support copy-on-write snapshots, which makes pre-builds and WorkspaceSnapshots cheap and fast to take and to restore.

Choosing Storage Backends

Here are some common setups:

Environment	Workspaces and `EXCLUSIVE` Volumes	`SHARED` Volumes
Single-node or on-prem clusters	Longhorn (installed by the quick installer's `--longhorn` flag), OpenEBS or Rook-Ceph RBD.	Longhorn RWX volumes, Rook-CephFS or the NFS CSI driver.
AWS EKS	The EBS CSI driver (e.g. `gp3`).	The EFS CSI driver.
Google GKE	The Persistent Disk CSI driver (e.g. `pd-balanced` or `pd-ssd`).	Filestore CSI driver.
Azure AKS	The Azure Disk CSI driver.	The Azure Files CSI driver.

You can verify that the snapshot support is installed in your Kubernetes cluster as follows:

kubectl get crd volumesnapshots.snapshot.storage.k8s.io
kubectl get volumesnapshotclass

Selecting Storage Classes

If your Kubernetes cluster has several StorageClasses, you can choose the class of each Workspace and Volume, as well as the VolumeSnapshotClass of the snapshots, via CEL rules in the ClusterConfig. Here is an example for an EKS cluster that uses fast io2 volumes for large Workspaces, gp3 volumes for everything else and EFS for the SHARED Volumes:

kind: ClusterConfig
spec:
  workspace:
    storage:
      storageClass:
        rules:
          - condition:
              match: ctx.workspace.status.limit.storage.megabytes >= 100000
            storageClass: ebs-io2
          - condition:
              matchAny: true
            storageClass: ebs-gp3
      volumeSnapshotClass:
        rules:
          - condition:
              matchAny: true
            volumeSnapshotClass: ebs-csi-snapclass
  volume:
    storage:
      storageClass:
        rules:
          - condition:
              match: ctx.volume.spec.accessMode == "ACCESS_MODE_SHARED"
            storageClass: efs-sc
          - condition:
              matchAny: true
            storageClass: ebs-gp3

You can read more about these rules here.

note

A restored claim must use the same CSI driver as the snapshot that it is restored from. If you route Workspaces to StorageClasses of different CSI drivers, make sure that the Workspaces restored from a Template pre-build or a WorkspaceSnapshot are routed to the same driver as their source.

Regions

Every claim and snapshot belongs to the Region in which it was created. Cordium therefore runs a persistent Workspace in the Region of its storage, a Workspace that mounts Volumes in the Region of its Volumes, and a restored Workspace in the Region of its snapshot. The Region of a Volume can be chosen at creation time (e.g. cordium create volume --region eu-west), and it otherwise defaults to the creator's preferred Region.

Troubleshooting

Here are the most common storage issues:

A Workspace is stuck while initializing. The claim is typically waiting for the storage to be provisioned. Inspect it via kubectl -n cordium describe pvc ws-<WORKSPACE_UID>. You can get the Workspace's UID via cordium get ws <NAME> -o yaml.
A Volume stays STATE_PENDING. This is expected until the first Workspace that mounts it is scheduled with storage backends that delay the binding until the first consumer. If a Workspace that mounts it fails to start, inspect the claim via kubectl -n cordium describe pvc vol-<VOLUME_UID>. This is typically caused by a SHARED Volume whose StorageClass cannot provision ReadWriteMany volumes.
A WorkspaceSnapshot fails with an Unsupported failure. The VolumeSnapshot API is not installed or no VolumeSnapshotClass is available for the CSI driver. Install the CSI snapshot controller and a VolumeSnapshotClass, or select one explicitly via the ClusterConfig.
A Volume cannot be grown. The StorageClass must have allowVolumeExpansion: true, and some drivers only expand volumes that are not attached to a running pod.