Octelium as a Kubernetes Ingress Controller

As Octelium Clusters operate on top of Kubernetes, the Octelium Cluster can seamlessly function as an advanced Kubernetes ingress controller that provides L7-based routing, filtering, access control, visibility as well as automatic TLS termination, load balancing, request and response manipulation including paths, headers, and body content, including serialized JSON body content. It is important, however, to distinguish Octelium from conventional ingress controllers. Octelium does not utilize or monitor native Kubernetes Ingress resources or their associated rules. Instead, it employs its own Services to implement ingress functionality (read more about Services here). Because Octelium operates independently of the standard Kubernetes Ingress API, it can be deployed alongside traditional ingress controllers within the same cluster without conflict

A Simple Example

In this guide we are going to mostly assume that your internal Kubernetes services require no authentication and thus use anonymous Services (read more here).

If your internal Kubernetes service has the name my-k8s-svc in the default Kubernetes namespace and listening over the TCP/8080 port, then you can expose it as follows:

kind: Service
metadata:
  name: my-svc
spec:
  mode: HTTP
  isPublic: true
  isAnonymous: true
  config:
    upstream:
      url: http://my-k8s-svc.default.svc:8080

In addition to being able to use internal Kubernetes services as upstreams for your Octelium Service, Octelium also enables you to directly deploy, manage and scale your containerized applications and serve them as Services by reusing the underlying Kubernetes infrastructure that runs the Octelium Cluster and deploying your Dockerized images on top of it. (read more about managed containers here)

kind: Service
metadata:
  name: my-api
spec:
  mode: HTTP
  isPublic: true
  isAnonymous: true
  config:
    upstream:
      container:
        port: 3000
        image: ghcr.io/<ORG>/<IMAGE>:<TAG>
        replicas: 3
        credentials:
          usernamePassword:
            username: linus
            password:
              fromSecret: reg-password
        resourceLimit:
          cpu:
            millicores: 2000
          memory:
            megabytes: 4000

You can now apply the creation of the Service as follows (read more here):

octeliumctl apply /PATH/TO/SERVICE.YAML

Now your internal my-k8s-svc Kubernetes service is exposed publicly at the https://my-svc.<DOMAIN>.

note

You might want to have a look at Namespaces (read more here) to group and organize your Services' FQDNs.

Octelium Services are implemented as Kubernetes deployment with a single replica by default. You can horizontally scale up or down your Service as follows:

kind: Service
metadata:
  name: my-svc
spec:
  mode: HTTP
  isPublic: true
  isAnonymous: true
  config:
    upstream:
      url: http://my-k8s-svc.default.svc:8080
  deployment:
    replicas: 5

Dynamic Routing

Octelium enables you to dynamically route to a specific upstream, not only based on request paths as in Kubernetes-native ingress controllers, but also based on request headers, method, query parameters, and even body content, including serialized JSON body content. You can use dynamic configuration (read more here) to define multiple configurations that can be chosen on a per-request basis via CEL/OPA policy-as-code rules. Here is a simple example:

kind: Service
metadata:
  name: my-api
spec:
  mode: HTTP
  isPublic: true
  isAnonymous: true
  dynamicConfig:
    configs:
      - name: v1
        upstream:
          url: http://apiv1.default.svc:8080
      - name: v2
        upstream:
          url: http://apiv2.default.svc:8080
    rules:
      - condition:
          match: 'ctx.request.http.path.startsWith("/v1")'
        configName: v1
      - condition:
          all:
            of:
              - match: 'ctx.request.http.path.startsWith("/v2")'
              - match: ctx.request.http.method in ["POST", "GET", "PUT"]
              - match: ctx.request.http.headers["user-agent"].toLower().contains("my-client")
              - match: int(ctx.request.http.queryParams.page) < 100
        configName: v2
      - condition:
          matchAny: true
        configName: v1

You can also simultaneously deploy multiple containers that correspond to multiple configurations (e.g. multiple API versions) and route among them on a per-request basis. Here is an example:

kind: Service
metadata:
  name: my-web-app
spec:
  mode: WEB
  dynamicConfig:
    configs:
    - name: c1
      upstream:
        container:
          port: 3000
          image: ghcr.io/org/image:v1
    - name: c2
      upstream:
        container:
          port: 3000
          image: ghcr.io/org/image:v2
    rules:
    - condition:
        match: ctx.request.http.path.startsWith("/v1")
      configName: c1
    - condition:
        match: ctx.request.http.path.startsWith("/v2")
      configName: c2

Request and Response Manipulation

Octelium also enables you to modify and rewrite your requests and responses. That includes header manipulation, path manipulation, CORS and retries. Here is a detailed example:

kind: Service
metadata:
  name: my-svc
spec:
  mode: HTTP
  isPublic: true
  isAnonymous: true
  config:
    upstream:
      url: http://my-k8s-svc.default.svc:8080
    http:
      header:
        addRequestHeaders:
        - key: X-Header-1
          value: VALUE-1
        removeRequestHeaders:
        - X-Header-2
        addResponseHeaders:
        - key: X-Header-3
          value: VALUE-3
        removeResponseHeaders:
        - X-Header-4
      path:
        removePrefix: /v1
        addPrefix: /v2
      enableRequestBuffering: true
      body:
        maxRequestSize: 100000
        mode: JSON
      retry:
        maxRetries: 10
        initialInterval:
          milliseconds: 500
        maxInterval:
          seconds: 5
        maxElapsedTime:
          seconds: 12
        multiplier: 1.5
        statusCodes: [503, 429]
        retryOnServerErrors: true
      cors:
        allowOriginStringMatch: ["https://example.com"]
        allowMethods: "POST, GET, OPTIONS"
        allowHeaders: "X-PINGOTHER, Content-Type"
        exposeHeaders: "Content-Encoding, Kuma-Revision"
        maxAge: "86400"
        allowCredentials: true

You can also use Octelium's HTTP plugins, including Lua scripts and Envoy's ExtProc compliant servers, to sanitize and manipulate your request and responses (read more about HTTP plugins here) in arbitrarily complex ways. For example, you can use this to manipulate the JSON body content of requests/responses and call external APIs to implement geolocation-based access control and/or rate limiting. Here is an example:

kind: Service
metadata:
  name: my-svc
spec:
  mode: HTTP
  isPublic: true
  isAnonymous: true
  config:
    upstream:
      url: http://my-k8s-svc.default.svc:8080
    http:
      plugins:
        - name: minute-rate-limit
          condition:
            matchAny: true
          ratelimit:
            limit: 100
            window:
              minutes: 2
        - name: hour-rate-limit
          condition:
            matchAny: true
          ratelimit:
            limit: 1000
            window:
              hours: 6
        - name: rate-limit-auth
          condition:
            match: ctx.request.http.path.startsWith("/auth")
          ratelimit:
            limit: 10
            window:
              minutes: 1
        - name: validate-users
          condition:
            all:
              of:
                - match: ctx.request.http.path == "/users"
                - match: ctx.request.http.method in ["POST", "PUT"]
          jsonSchema:
            inline: <YOUR_JSON_SCHEMA>
        - name: drop-php
          condition:
            match: ctx.request.http.path.toLower().endsWith(".php")
          direct:
            statusCode: 400
            body:
              inline: '{"error": "invalid path"}'
        - name: lua-cleanup
          condition:
            matchAny: true
          lua:
            inline: |
              function onRequest(ctx)
                if strings.hasPrefix(ctx.request.http.path, "/apis/v0") then
                  octelium.req.exit(400)
                end
              end


              function onResponse(ctx)
                octelium.req.deleteResponseHeader("X-Internal")
              end

Access Control

So far, we have used an anonymous Service, which allows all access publicly over the internet. This could be useful for hosting and testing public websites, APIs, webhooks, etc. Since Octelium is a zero trust access platform, it is primarily designed for identity-based access control you might want to only restrict the Service access to the Cluster's Users who can access the Service after authenticating to a web-based IdentityProvider (e.g. OpenID Connect, SAML 2.0, GitHub OAuth2) via their browsers (read more here). You can do so by removing the isAnonymous field and adding Policies for who is allowed to access the Service (read more about Policies and access control here). Here is an example:

kind: Service
metadata:
  name: my-svc
spec:
  mode: HTTP
  isPublic: true
  config:
    upstream:
      url: http://my-k8s-svc.default.svc:8080
  authorization:
    inlinePolicies:
      - spec:
          rules:
            - effect: ALLOW
              condition:
                all:
                  of:
                    - match: ctx.user.spec.email.endsWith("@example.com")
                    - match: ctx.request.http.path.startsWith("/apis")
                    - match: ctx.request.http.method in ["GET", "POST", "PUT", "DELETE"]
                    - match: ctx.request.http.uri == "/apis/users?name=john"
                    - match: ctx.request.http.queryParams.name == "john"
                    - match: ctx.request.http.headers["x-custom-header"] == "this-value"
                    - match: ctx.request.http.scheme == "http"
                    - match: string(ctx.request.http.body).toLower().contains("value1")
                    - match: ctx.request.http.bodyMap.key1 == "value1"

Moreover, Octelium also supports anonymous authorization where you can apply your Policies and InlinePolicies' rules in the anonymous mode. You can read more about anonymous authorization here.

Visibility

Octelium also provides OpenTelemetry-ready, application-layer L7 aware visibility and access logging in real time (see an example for HTTP here). You can read more about visibility here.

This was a very short guide to show you how to use Octelium to deploy, scale, route and provide dynamic zero trust secure access to your workloads. Here are a few more related features that you might be interested in:

Routing not just by request paths, but also by header keys and values, request body content including JSON (read more here).
Request/response header manipulation (read more here).
Cross-Origin Resource Sharing (CORS) (read more here).
gRPC mode (read more here).
Secretless access to upstreams and injecting bearer, basic, or custom authentication header credentials (read more here).
Exposing the API publicly for anonymous access (read more here).
Application layer-aware ABAC access control via policy-as-code using CEL and Open Policy Agent (read more here).
OpenTelemetry-ready, application-layer L7 aware auditing and visibility (read more here).