# CI/CD Pipelines

> Cordium documentation. Canonical page: <https://octelium.com/docs/cordium/latest/examples/automation/cicd>.

Cordium *Workspaces* make capable CI/CD execution environments: each job runs in a fresh, ephemeral sandbox on your own infrastructure, with as much CPU and memory as it needs, shared caches, and secretless access to the private resources that integration tests and deployments need, such as databases, internal APIs and Kubernetes clusters. Since deployments run with the identity of the job's *Workspace*, there is no kubeconfig, cloud credential or database password to store in your CI system.

This example builds a test-and-deploy pipeline for a Go service that can be driven by any CI system, such as GitHub Actions, GitLab CI, Jenkins or Buildkite.

## The Test Template

The `ci` *Template* checks out the commit to test when the *Workspace* is created, and reuses the Go caches across runs via a shared *Volume* (read more about variables [here](https://octelium.com/docs/cordium/latest/workspaces/overview.md#variables) and about caches [here](https://octelium.com/docs/cordium/latest/examples/workflows/volumes.md#a-shared-build-cache)):

```yaml
spec:
  vars:
    - name: REF
      value: main
  image:
    registry:
      url: golang:1.26-bookworm
  repository:
    url: https://github.com/acme-corp/payments-api
    cloneOptions:
      branch: main
      disableLazyUnshallow: true
    authentication:
      http:
        username: x-access-token
        password:
          fromSecret: github-read-token.payments.cordium
  runtime:
    volumeMounts:
      - volumeRef:
          name: go-cache
        mountPath: /cache
    envVars:
      - key: GOMODCACHE
        value: /cache/mod
      - key: GOCACHE
        value: /cache/build
    tasks:
      - name: checkout
        type: ON_CREATE
        workingDir: /workspace/repo
        onFailure: ON_FAILURE_ABORT
        run: |
          git fetch -q --depth 1 origin "${{ vars.REF }}"
          git checkout -q FETCH_HEAD
          git log -1 --oneline
  limit:
    cpu:
      millicores: 8000
    memory:
      megabytes: 16384
    storage:
      megabytes: 30000
```

```bash
cordium create volume go-cache.payments.cordium --size 30000 --shared
cordium create template ci.payments.cordium --file ci.yaml
```

`REF` can be a branch, a tag or a commit SHA, since the `checkout` task fetches it explicitly.

## Running Jobs from Any CI System

The following Python script is all that a CI job needs to run. It starts an ephemeral *Workspace* for the commit, runs the pipeline's steps in it via `exec` while streaming their output to the CI job's log, stops at the first failing step with its exit code, and always deletes the *Workspace* (read more about authenticating CI jobs [here](https://octelium.com/docs/cordium/latest/use/cli.md#authentication) and [here](https://octelium.com/docs/cordium/latest/examples/automation/github-actions.md)):

```python
import sys

from cordium import Cordium

STEPS = [
    "go vet ./...",
    "go test -race -count 1 ./...",
    "go build -o /tmp/payments-api ./cmd/server",
]


def main(ref: str) -> int:
    with Cordium() as client:
        workspace = client.workspaces.run(
            template="ci.payments.cordium",
            ephemeral=True,
            display_name=f"CI {ref[:12]}",
            vars={"REF": ref},
            timeout=900,
        )
        try:
            for step in STEPS:
                print(f"::group::{step}", flush=True)
                with workspace.exec_stream(
                    step, cwd="/workspace/repo", interactive=False, timeout=3600
                ) as session:
                    for chunk in session:
                        out = sys.stdout if chunk.stream == "stdout" else sys.stderr
                        out.buffer.write(chunk.data)
                        out.flush()
                    result = session.wait()
                print("::endgroup::", flush=True)
                if result.exit_code != 0:
                    print(f"{step} failed with exit code {result.exit_code}", file=sys.stderr)
                    return result.exit_code
            return 0
        finally:
            workspace.delete()


if __name__ == "__main__":
    sys.exit(main(sys.argv[1]))
```

```bash
python ci_run.py "$GIT_COMMIT_SHA"
```

Since each step is a separate `exec` session, you get the exact exit code and output of every step, while the *Workspace*'s CPU, memory, caches and network access are shared by all of them. The `::group::` lines fold the output of each step in GitHub Actions and are harmless in other CI systems.

## Secretless Deployments

Deployment jobs need access to production-like infrastructure, which is exactly where long-lived CI credentials are the most dangerous. Instead, a *Cluster* administrator exposes the staging Kubernetes cluster as an Octelium `KUBERNETES` *Service* named `k8s-staging` (read more [here](https://octelium.com/docs/octelium/latest/management/core/service/kubernetes.md)), and only allows the *Workspaces* of the `deploy` *Template* to use it, and only within the `payments` Kubernetes namespace:

```yaml
kind: Policy
metadata:
  name: deploy-to-staging
spec:
  rules:
    - effect: ALLOW
      condition:
        all:
          of:
            - match: ctx.service.metadata.name == "k8s-staging.default"
            - match: '"ext" in ctx.session.status && "cordium" in ctx.session.status.ext'
            - match: ctx.session.status.ext.cordium.templateRef.name == "deploy.payments.cordium"
            - match: ctx.request.kubernetes.namespace == "payments"
```

The `deploy` *Template*, which uses Cordium's default Ubuntu-based image, then rolls out a new image with `kubectl`, which is authenticated by Octelium on a per-request basis. Since it is an auto-stopping `POST_START` task with `ON_FAILURE_ABORT`, the run fails if the rollout fails:

```yaml
spec:
  vars:
    - name: IMAGE
      value: ghcr.io/acme-corp/payments-api:latest
  runtime:
    autoStop: true
    tasks:
      - name: install-kubectl
        type: ON_CREATE
        runAsRoot: true
        onFailure: ON_FAILURE_ABORT
        run: |
          VERSION="$(curl -fsSL https://dl.k8s.io/release/stable.txt)"
          curl -fsSLo /usr/local/bin/kubectl "https://dl.k8s.io/release/$VERSION/bin/linux/$(dpkg --print-architecture)/kubectl"
          chmod +x /usr/local/bin/kubectl
      - name: deploy
        type: POST_START
        onFailure: ON_FAILURE_ABORT
        run: |
          set -e
          timeout 120 sh -c 'until getent hosts k8s-staging > /dev/null; do sleep 2; done'
          octelium cfg k8s-staging
          export KUBECONFIG="$HOME/.kube/k8s-staging.$OCTELIUM_DOMAIN"
          kubectl -n payments set image deployment/payments-api server="${{ vars.IMAGE }}"
          kubectl -n payments rollout status deployment/payments-api --timeout 10m
```

```bash
cordium create ws --template deploy.payments.cordium --ephemeral --start \
  --var IMAGE=ghcr.io/acme-corp/payments-api:3f9c2e71
```

Every `kubectl` request made by the job, including the image that was rolled out, appears in Octelium's access logs along with the identity of the *Workspace* and of the *User* who triggered it (read more [here](https://octelium.com/docs/octelium/latest/management/core/visibility.md)).