Using Helm to deploy Octelium clients and remotely serve Services

You can easily deploy octelium clients in any remote Kubernetes cluster to access and serve the Cluster Services (read more here). An easier and more automated alternative solution is to use the official Octelium Helm charts to deploy the octelium containers in any remote Kubernetes clusters.

note

Octelium clients do not need any special Kubernetes operator to remotely serve some Octelium Service from a remote Kubernetes service/deployment/pod. You can just deploy the octelium container inside your remote Kubernetes cluster and as long as it has access to the destination Kubernetes resource (i.e. there is no Kubernetes NetworkPolicy that prevents it from accessing the k8s resource), then it is capable of remotely serving it to the Octelium Cluster.

A minimal example should look as follows:

helm install my-octelium-chart oci://ghcr.io/octelium/helm-charts/octelium --set octelium.domain=<DOMAIN> --set octelium.auth.token=<AUTHENTICATION_TOKEN>

note

The chart values were reorganized in the chart version v1.0.0. If you are upgrading from an older chart, then the authentication values octelium.authToken, octelium.authTokenSecret and octelium.authTokenSecretKey are now octelium.auth.token, octelium.auth.existingSecret and octelium.auth.existingSecretKey respectively, while octelium.args is now octelium.extraArgs. Any value that is no longer recognized is now rejected by the chart instead of being silently ignored.

You can also use an authentication token stored in an existing Kubernetes secret as follows:

helm install my-octelium-chart oci://ghcr.io/octelium/helm-charts/octelium --set octelium.domain=<DOMAIN> --set octelium.auth.existingSecret=<K8S_SECRET_NAME>

By default, the Kubernetes secret key is assumed to be data. You can choose a custom secret key as follows:

helm install my-octelium-chart oci://ghcr.io/octelium/helm-charts/octelium --set octelium.domain=<DOMAIN> --set octelium.auth.existingSecret=<K8S_SECRET_NAME> --set octelium.auth.existingSecretKey=<KEY_NAME>

Note that Helm stores every value you pass via the --set flag in the release metadata of the Kubernetes cluster. Therefore, passing the authentication token via the octelium.auth.token value is mostly useful for quick trials, while for production deployments you should either use an existing Kubernetes secret managed by your own secret management tooling or, even better, use the secretless assertion-based authentication described below.

Secretless Authentication

Instead of using an authentication token Credential, the octelium client running inside your remote Kubernetes cluster can authenticate itself to the Cluster using a Kubernetes-issued OIDC identity token of its own Kubernetes service account (read more about assertion-based IdentityProviders here). This way no secret at all is stored inside the remote Kubernetes cluster.

First we create an oidcIdentityToken IdentityProvider for your remote Kubernetes cluster as follows:

kind: IdentityProvider
metadata:
  name: k8s-cluster-01
spec:
  oidcIdentityToken:
    issuerURL: https://oidc.example-k8s-cluster.com
    audience: https://example.com

note

Many managed Kubernetes providers publish their OIDC discovery endpoints publicly. If your remote Kubernetes cluster does not, then you can set the JWKS content of the cluster manually instead of the issuerURL field (read more here).

Now your WORKLOAD User can set the identity that corresponds to the Kubernetes service account used by the chart as follows:

kind: User
metadata:
  name: k8s-cluster-01
spec:
  type: WORKLOAD
  authentication:
    identities:
    - identityProvider: k8s-cluster-01
      identifier: system:serviceaccount:octelium:octelium-connector

And now you can deploy the chart without any authentication token as follows:

helm install my-octelium-chart oci://ghcr.io/octelium/helm-charts/octelium \
  --namespace octelium --create-namespace \
  --set octelium.domain=<DOMAIN> \
  --set serviceAccount.name=octelium-connector \
  --set octelium.auth.assertion.enabled=true \
  --set octelium.auth.assertion.audience=https://example.com

The chart mounts a short-lived, audience-bound projected Kubernetes service account token for the octelium container, and Kubernetes itself rotates that token before it expires. Note that the octelium.auth.assertion.audience value has to match the audience field of the IdentityProvider, and that the identifier of the User's identity is the sub claim of the Kubernetes service account token which takes the form system:serviceaccount:<K8S_NAMESPACE>:<K8S_SERVICE_ACCOUNT_NAME>. Setting an explicit serviceAccount.name value simply keeps that identifier stable regardless of the Helm release name.

Serving Services

You can serve one or more Services from your remote Kubernetes cluster via the --set octelium.serve flag. Here is an example:

helm install my-octelium-chart oci://ghcr.io/octelium/helm-charts/octelium --set octelium.domain=<DOMAIN> --set octelium.auth.token=<AUTHENTICATION_TOKEN> --set "octelium.serve={svc1}"

You can also serve multiple Services as follows:

helm install my-octelium-chart oci://ghcr.io/octelium/helm-charts/octelium --set octelium.domain=<DOMAIN> --set octelium.auth.token=<AUTHENTICATION_TOKEN> --set "octelium.serve={svc1,svc2,svc3}"

You can also serve every single Service that is assigned to the User via the octelium.serveAll value as follows:

helm install my-octelium-chart oci://ghcr.io/octelium/helm-charts/octelium --set octelium.domain=<DOMAIN> --set octelium.auth.token=<AUTHENTICATION_TOKEN> --set octelium.serveAll=true

Let us discuss a detailed example. Let us imagine that you have an nginx Kubernetes service in the default Kubernetes namespace of your remote Kubernetes cluster that you want to serve and expose to the Octelium Cluster. You will need to create an Octelium Service as follows:

kind: Service
metadata:
  name: remote-nginx
spec:
  mode: WEB
  isPublic: true
  config:
    upstream:
      url: http://nginx.default.svc
      user: k8s-cluster-01

Note that the upstream of the remote-nginx Service above is the URL of the nginx Kubernetes service http://nginx.default.svc, as if we're trying to access it from that remote Kubernetes cluster. Also note that we added the user field which means that the Service's upstream is served by the connect User k8s-cluster-01. You can actually create that Octelium User, before creating the Service, as follows:

kind: User
metadata:
  name: k8s-cluster-01
spec:
  type: WORKLOAD

And then you can create an authentication token for the k8s-cluster-01 User as follows (read more here):

octeliumctl create cred --user k8s-cluster-01 my-cred

And then you now deploy the octelium client in your remote Kubernetes cluster and instruct it to serve the remote-nginx Service as follows:

helm install my-octelium-chart oci://ghcr.io/octelium/helm-charts/octelium --set octelium.domain=<DOMAIN> --set octelium.auth.token=<AUTHENTICATION_TOKEN> --set "octelium.serve={remote-nginx}"

Publishing Services

Serving Services exposes your remote Kubernetes cluster's workloads to the Octelium Cluster. You might also want the exact opposite: to let the workloads of your remote Kubernetes cluster access the Cluster Services over ordinary Kubernetes service hostnames without having to deploy an octelium sidecar container next to each one of them. To do that, you can publish Services to the ports of the octelium pod via the octelium.publish value (read more about publishing Services here), and expose those ports via a Kubernetes service via the service.enabled value.

For such a case it is much easier to use a values file instead of a long list of --set flags. Here is an example values file:

octelium:
  domain: example.com
  auth:
    existingSecret: octelium-auth-token
  publish:
    - service: postgres
      port: 5432
    - service: redis.ns1
      port: 6379
      name: redis
    - service: coredns.ns1
      port: 5353
      protocol: UDP
      name: coredns

service:
  enabled: true

And now you can deploy the chart as follows:

helm install my-octelium-chart oci://ghcr.io/octelium/helm-charts/octelium -f /PATH/TO/VALUES.YAML

The workloads of your remote Kubernetes cluster can now access the postgres, redis.ns1 and coredns.ns1 Services at my-octelium-chart.<K8S_NAMESPACE>.svc on the ports 5432, 6379 and 5353 respectively.

note

The protocol field of a published Service defaults to TCP and it has to match the type of the Octelium Service. In other words, a UDP or DNS Service (read more about Service modes here) has to explicitly set protocol: UDP, otherwise Kubernetes exposes a TCP port that the traffic of the published Service can never traverse.

By default every published Service listens on 0.0.0.0 inside the pod so that the other pods of your remote Kubernetes cluster can reach it. You can restrict that via the address field of the published Service.

Replicas

You can also deploy multiple replicas of the octelium client and impose random load balancing among the different octelium serving the upstream of remote-nginx via the --set replicaCount flag as follows:

helm install my-octelium-chart oci://ghcr.io/octelium/helm-charts/octelium --set octelium.domain=<DOMAIN> --set octelium.auth.token=<AUTHENTICATION_TOKEN> --set "octelium.serve={remote-nginx}" --set replicaCount=3

Every replica is an entirely independent Octelium Session and Device, which is why upgrading the chart rolls the pods one at a time while keeping the already running ones connected.

Note that you are not restricted to remotely serving HTTP-based Services from any remote Kubernetes cluster. You can actually serve arbitrary TCP and UDP based Services too.

Embedded SSH and SOCKS5

You can also serve the embedded SSH server (read more here) as well as the embedded SOCKS5 server from the octelium pod as follows:

octelium:
  domain: example.com
  auth:
    existingSecret: octelium-auth-token
  essh:
    enabled: true
    listenAddresses: ["0.0.0.0"]
  esocks5:
    enabled: true
    listenAddresses: ["0.0.0.0"]

service:
  enabled: true

note

The listenAddresses values are not optional here. By default both embedded servers bind to the Octelium Device's own tunnel addresses which a Kubernetes service cannot route to. Also note that the embedded SOCKS5 server does not authenticate its own clients, and that an embedded SSH session has access to the octelium container itself, which is why you should keep the Kubernetes service of both of them internal to your remote Kubernetes cluster.

Networking Privileges

By default, the octelium container drops every Linux capability except NET_ADMIN, which the client needs in order to set up its tunnel interface, and it additionally runs with a read-only root filesystem, without privilege escalation and with the RuntimeDefault seccomp profile. If your remote Kubernetes cluster does not allow NET_ADMIN at all, then you can run the client entirely unprivileged by using the pure userspace tunnel implementation (read more here) as follows:

octelium:
  domain: example.com
  auth:
    existingSecret: octelium-auth-token
  network:
    implementation: gvisor

podSecurityContext:
  runAsNonRoot: true
  runAsUser: 1000
  runAsGroup: 1000
  fsGroup: 1000

securityContext:
  capabilities:
    drop: ["ALL"]
    add: []

Additional Arguments

You can also add any additional octelium connect flags as follows:

helm install my-octelium-chart oci://ghcr.io/octelium/helm-charts/octelium --set octelium.domain=<DOMAIN> --set octelium.auth.token=<AUTHENTICATION_TOKEN> --set "octelium.extraArgs={--no-dns}"

Note, however, that most of the commonly used octelium connect flags already have their own dedicated values which you should prefer over the octelium.extraArgs value. For example, the --scope flag is set via the octelium.auth.scopes value (read more about scopes here), the --ip-mode flag is set via the octelium.network.ipMode value (read more here), and the --localdns flag is set via the octelium.dns.local.enabled value (read more here). Here is an example:

octelium:
  domain: example.com
  auth:
    existingSecret: octelium-auth-token
    scopes: ["service:svc1.ns1", "service:ns2/*"]
  network:
    ipMode: v4
    mtu: 1380
  dns:
    local:
      enabled: true

You can read about every supported value of the chart here.