Resources and Timeouts

Every Workspace run is constrained by CPU, memory and storage limits, and every running Workspace is subject to an inactivity timeout. This page explains how these values are resolved, as well as the Cluster-wide quotas that apply to Workspaces.

Compute Limits

The spec.limit field defines the CPU, memory and storage limits of a Workspace:

spec:
  limit:
    cpu:
      millicores: 4000   # 4 cores
    memory:
      megabytes: 8192    # 8 GB
    storage:
      megabytes: 50000   # 50 GB

Field	Description
`cpu.millicores`	The maximum CPU that the Workspace can use, in millicores (i.e. `1000` is 1 core).
`memory.megabytes`	The maximum memory of the Workspace. Processes are killed by the kernel's OOM killer if they exceed it.
`storage.megabytes`	The size of the Workspace's own storage, which holds its home directory, `/workspace` and its container layer.

You can also set them via the --cpu, --memory and --storage flags of cordium run, cordium create workspace and cordium create template:

cordium run --template api-dev.payments.cordium --cpu 4000 --memory 8192 --storage 50000

CPU and memory are enforced as limits rather than reservations, which means that Workspaces only consume the node resources that they actually use, and that idle Workspaces do not hold any reserved capacity.

Resolution

The effective limits of a run are resolved by the API server when the Workspace starts, and they are reported in the Workspace's status.limit field. They are resolved as follows:

The first level that sets a limit is used as the starting point, in the following order: the Workspace's own spec, its Template's spec, and its Space's default limit (only for ORGANIZATION Spaces).
Any field that is still unset is taken from the ClusterConfig's default limit for the Space type (i.e. defaultUserSpaceLimit or defaultOrganizationSpaceLimit), and otherwise from the built-in defaults: 2 cores, 6000 MB of memory and 20000 MB of storage.
The result is capped by the Space's maximum limit, and then by the ClusterConfig's maximum limit.

Here is an example: a Workspace requests 16 cores and does not set any memory, while its Space's maximum is 8 cores and the ClusterConfig's default memory for organization Spaces is 16 GB. The Workspace therefore runs with 8 cores, 16 GB of memory and 20 GB of storage.

note

Since the Workspace's own limit takes precedence as a whole, setting only limit.cpu in a Workspace means that its Template's memory and storage limits are not inherited. In such case, set all the fields that you need in the Workspace itself.

Space admins can define the default and maximum limits of an ORGANIZATION Space as follows (read more here):

spec:
  limit:
    defaultLimit:
      cpu:
        millicores: 2000
      memory:
        megabytes: 4096
    maxLimit:
      cpu:
        millicores: 8000
      memory:
        megabytes: 32768
      storage:
        megabytes: 100000

Template pre-builds and image builds use the ClusterConfig's buildLimit instead, which also defaults to 2 cores, 6000 MB of memory and 20000 MB of storage (read more here).

Inactivity Timeout

A running Workspace is automatically stopped by the Cluster once it has been inactive for longer than its inactivity timeout. The following count as activity:

Input in the web portal's terminals and in terminals opened via the APIs.
Requests to the Workspace's applications.
SSH connections to the Workspace.
Open connections of the Workspace API, including cordium exec, cordium cp, file operations and log streaming.

The timeout defaults to 30 hours, and Cluster administrators can change it, globally or per Space type, via the ClusterConfig (read more here). Activity is recorded with a granularity of a few minutes, which means that the actual stop time can be a few minutes later than the exact timeout.

Disabling the Timeout

Long-running Workspaces (e.g. a self-hosted CI runner or a long-lived AI agent) can opt out of the inactivity timeout as follows:

spec:
  runtime:
    timeout:
      mode: DISABLED

This is only honored when the ClusterConfig's spec.workspace.timeout.allowNoTimeout field is enabled. Otherwise, the Cluster's inactivity timeout still applies.

note

Pre-builds are never stopped by the inactivity timeout. Instead, Workspaces that are stuck in a transitional state are stopped automatically: after 5 minutes in INIT_REQUEST, after 3 hours in any of the initialization states (e.g. PULLING_IMAGE, BUILDING_IMAGE or PREPARING), and after 2 hours in a stopping state.

Quotas

The following quotas apply to every User regardless of the Space. They can be changed by Cluster administrators via the ClusterConfig (read more here):

Quota	Default	`ClusterConfig` field
The total number of Workspaces per User	1000	`spec.workspace.limit.maxPerUser`
The number of non-stopped Workspaces per User	128	`spec.workspace.limit.maxActivePerUser`
The total number of WorkspaceSnapshots per User	100	`spec.workspace.limit.maxSnapshotsPerUser`
The total number of Volumes per Space	64	`spec.volume.limit.maxPerSpace`

Starting a Workspace while the User already has the maximum number of non-stopped Workspaces fails with a permission error. You can list and stop your running Workspaces via cordium get ws and cordium stop.