Octelium as an API Gateway

Overview

Octelium provides a unified, scalable, self-hosted, open source infrastructure to build your AI gateways. Octelium provides the following:

A scalable platform built on top of Kubernetes to provide access to your APIs whether they are internal resources behind NAT in multiple private clouds, on-prem, your own laptop, etc... as well as to publicly protected SaaS resources.
Deploy, scale and secure access to your containerized APIs (read more about managed containers here).
Centralized identity-based, application-layer (L7) aware ABAC access control on a per-request basis (read more about access control here).
Unified identity management for both HUMAN as well as WORKLOAD Users (read more here).
Advanced request/output sanitization and manipulation of headers and body content via Lua scripts and Envoy ExtProc compatible interfaces (read more here) to build you ad-hoc rate limiting, caching, and DLP logic.
Unified access and identity for WORKLOAD Users via OAuth2 client credentials (read more here) and standard bearer authentication (read more here).
OpenTelemetry-native, identity-based, L7 aware visibility and auditing (read more here).
Secretless access to upstreams that require bearer authentication, basic authentication, OAuth2 client credentials or AWS sigv4 (read more here).
Dynamic, identity-based, L7-aware routing to your upstreams (read more about dynamic configuration here).
Extremely dynamic plugin system, that includes Lua scripts and Envoy ExtProc compliant servers, to manipulate your request/response headers and body content, including serialized JSON (read more here).
GitOps-friendly declarative, programmable management (read more here).

API Gateway

A Simple Example

This is a very simple example for a Service to represent a certain API upstream server. The upstream server could be (read more about Service upstreams here):

A directly accessible resource by the Octelium Cluster (e.g. a Kubernetes service running on the same Kubernetes cluster).

kind: Service
metadata:
  name: my-api
spec:
  mode: HTTP
  isPublic: true
  port: 80
  config:
    upstream:
      url: http://my-api.default.svc:8080

A remotely accessible resource by the Octelium Cluster via a connected Session (read more here). This could be, in practice, any server that is listening behind NAT (e.g. private cloud, IoT, your own laptop). Here is an example:

kind: Service
metadata:
  name: my-api
spec:
  mode: HTTP
  isPublic: true
  port: 80
  config:
    upstream:
      url: http://my-api.default.svc:8080
      user: eu-central-2

A publicly protected resource via bearer or basic authentication (read more here). Here is an example for a public SaaS resource that is protected by a bearer access token:

kind: Service
metadata:
  name: my-gemini
spec:
  mode: HTTP
  isPublic: true
  config:
    upstream:
      url: https://generativelanguage.googleapis.com
      auth:
        bearer:
          fromSecret: gemini-api-key

A containerized/Docker application that can be deployed as a managed container (read more here). Here is an example:

kind: Service
metadata:
  name: my-api
spec:
  mode: HTTP
  isPublic: true
  config:
    upstream:
      container:
        port: 3000
        image: ghcr.io/<ORG>/<IMAGE>:<TAG>
        replicas: 3
        credentials:
          usernamePassword:
            username: linus
            password:
              fromSecret: reg-password
        resourceLimit:
          cpu:
            millicores: 2000
          memory:
            megabytes: 4000

You can now apply the creation of the Service as follows (read more here):

octeliumctl apply /PATH/TO/SERVICE.YAML

You can dynamic configuration (read more here) to dynamically route among multiple upstreams the Service based on identity and/or request context (e.g. HTTP request paths, body content, etc...). Here is a minimal example:

kind: Service
metadata:
  name: my-api
spec:
  mode: HTTP
  isPublic: true
  dynamicConfig:
    configs:
      - name: v1
        upstream:
          url: https://apiv1.example.com
        http:
          path:
            removePrefix: /v1
      - name: v2
        upstream:
          url: https://apiv2.example.com
        http:
          path:
            removePrefix: /v2
    rules:
      - condition:
          match: 'ctx.request.http.path.startsWith("/v1")'
        configName: v1
      - condition:
          match: 'ctx.request.http.path.startsWith("/v2")'
        configName: v2

Octelium doesn't just enable you to dynamically route to an upstream based on a request path, but since it's built as a zero trust architecture, it can dynamically route to an upstream based on the identity and the access context too. Here is an another example:

kind: Service
metadata:
  name: my-api
spec:
  mode: HTTP
  isPublic: true
  dynamicConfig:
    configs:
      - name: v1
        upstream:
          url: https://apiv1.example.com
        http:
          path:
            removePrefix: /v1
      - name: v2
        upstream:
          url: https://apiv2.example.com
        http:
          path:
            removePrefix: /v2
    rules:
      - condition:
          match: '"prod" in ctx.user.spec.groups'
        configName: v1
      - condition:
          match: '"dev" in ctx.user.spec.groups'
        configName: v2

You can also use upstreams that are protected by their own credentials such as bearer tokens (read more about secretless access in HTTP here). Here is an example:

kind: Service
metadata:
  name: my-api
spec:
  mode: HTTP
  isPublic: true
  dynamicConfig:
    configs:
      - name: v1
        upstream:
          url: https://apiv1.example.com
        http:
          auth:
            bearer:
              fromSecret: access-token-1
      - name: v2
        upstream:
          url: https://apiv2.example.com
        http:
          auth:
            bearer:
              fromSecret: access-token-2
    rules:
      - condition:
          match: 'ctx.request.http.path.startsWith("/v1")'
        configName: v1
      - condition:
          match: 'ctx.request.http.path.startsWith("/v2")'
        configName: v2

Containerized Applications

And you can also deploy and scale your own containerized microservices including those that are served via private registries that are protected with credentials. Here is an example:

First you create the Secret for the registry's password as follows:

octeliumctl create secret ghcr-token

And then you create the Service as follows:

kind: Service
metadata:
  name: my-api
spec:
  mode: HTTP
  isPublic: true
  dynamicConfig:
    configs:
      - name: v1
        upstream:
          container:
            image: ghcr.io/org/my-api:v1
            port: 80
            replicas: 2
            credentials:
              usernamePassword:
                username: org
                password:
                  fromSecret: ghcr-token
        http:
          path:
            removePrefix: /v1
      - name: v2
        upstream:
          container:
            image: ghcr.io/org/my-api:v2
            port: 80
            replicas: 2
            credentials:
              usernamePassword:
                username: org
                password:
                  fromSecret: ghcr-token
        http:
          path:
            removePrefix: /v2
    rules:
      - condition:
          match: 'ctx.request.http.path.startsWith("/v1")'
        configName: v1
      - condition:
          match: 'ctx.request.http.path.startsWith("/v2")'
        configName: v2

Anonymous Access

You can also opt to choose to provide anonymous public access instead of using Octelium to provide authentication and access control. You might want to use anonymous access, for example, for public APIs or whenever you want to handle your own authentication and authorization at the upstream's side. You can read more about the anonymous access mode here.

Access Control

When it comes to access control, Octelium provides a rich layer-7 aware, identity-based, context-aware ABAC access control on a per-request basis where you can control access based on the HTTP request's path, method, and even serialized JSON body content using policy-as-code with CEL and Open Policy Agent (OPA) (You can read more in detail about Policies and access control here). Here is a generic example:

kind: Service
metadata:
  name: my-api
spec:
  mode: HTTP
  config:
    upstream:
      url: https://api.example.com
  authorization:
    inlinePolicies:
      - spec:
          rules:
            - effect: ALLOW
              condition:
                all:
                  of:
                    - match: ctx.user.spec.groups.hasAll("dev", "ops")
                    - match: ctx.request.http.method in ["POST", "PUT"]
                    - match: ctx.request.http.path.startsWith("/users")
                    - match: ctx.request.http.headers["x-custom-header"] == "this-value"
                    # bodyMap contains the serialized JSON body content
                    - match: ctx.request.http.bodyMap.age > 18 && ctx.request.http.bodyMap.email.endsWith("@example.com")

Plugins

You can also use Octelium's plugins, including Lua scripts and Envoy ex_proc compliant servers, to sanitize and manipulate your request and responses (read more about HTTP plugins here) in arbitrarily complex ways. For example, you can use this to manipulate the JSON body content of requests/responses and call external APIs to implement advanced identity-based caching and rate limiting. Here is an example:

kind: Service
metadata:
  name: safe-api
spec:
  mode: HTTP
  isPublic: true
  config:
    upstream:
      url: https://api.example.com
      plugins:
        - name: main
          condition:
            matchAny: true
          lua:
            inline: |
              function onRequest(ctx)
                local body = json.decode(octelium.req.getRequestBody())

                if body.age > 100 or body.age < 18 then
                  octelium.req.exit(400)
                end

                if strings.lenUnicode(body.user.name) > 100 then
                  octelium.req.exit(400)
                end

                body.email = ctx.user.spec.email
                body.isBrowser = ctx.session.status.isBrowser

                if strings.toLower(body.action) == "create" then
                  local c = http.client()
                  c:setBaseURL("http://rate-limiting.default.svc")
                  local req = c:request()
                  local bodyReq = {}
                  bodyReq.userUID = ctx.user.metadata.uid
                  req:setBody(json.encode(bodyReq))
                  local resp, err = req:post("/v1/check")
                  if err then
                    octelium.req.exit(500)
                    return
                  end

                  if resp:code() == 200 then
                    local apiResp = json.decode(resp:body())
                    if not apiResp.isRateLimited then
                      octelium.req.exit(429)
                      return
                    end
                  end
                end

                octelium.req.setRequestBody(json.encode(body))
              end


              function onResponse(ctx)
                octelium.req.deleteResponseHeader("X-Internal")
                octelium.req.setResponseHeader("X-User-Uid", ctx.user.metadata.uid)
              end

Octelium also provides other plugins (read more about HTTP plugins here) such as rate limiting, caching, JSON schema validation and direct response that can be used to control your MCP client requests. Here is another example:

kind: Service
metadata:
  name: my-api
spec:
  mode: HTTP
  isPublic: true
  config:
    upstream:
      url: https://api.example.com
    http:
      plugins:
        - name: minute-rate-limit
          condition:
            matchAny: true
          ratelimit:
            limit: 100
            window:
              minutes: 2
        - name: hour-rate-limit
          condition:
            matchAny: true
          ratelimit:
            limit: 1000
            window:
              hours: 6
        - name: rate-limit-sensitive
          condition:
            match: ctx.request.http.path.startsWith("/sensitive")
          ratelimit:
            limit: 10
            window:
              minutes: 1
        - name: validate-users
          condition:
            match: ctx.request.http.path == "/users" && ctx.request.http.method in ["POST", "PUT"]
          jsonSchema:
            inline: <YOUR_JSON_SCHEMA>
        - name: validate-groups
          condition:
            match: ctx.request.http.path == "/groups" && ctx.request.http.method in ["POST", "PUT"]
          jsonSchema:
            inline: <YOUR_JSON_SCHEMA>
        - name: drop-php
          condition:
            match: ctx.request.http.path.toLower().endsWith(".php")
          direct:
            statusCode: 400
            body:
              inline: '{"error": "invalid path"}'

Visibility

Octelium also provides OpenTelemetry-ready, application-layer L7 aware visibility and access logging in real time (see an example for HTTP here). You can read more about visibility here.

This was a very short guide to show you how to use Octelium to deploy, scale, route and provide dynamic zero trust secure access to your workloads. Here are a few more related features that you might be interested in:

Routing not just by request paths, but also by header keys and values, request body content including JSON (read more here).
Request/response header manipulation (read more here).
Cross-Origin Resource Sharing (CORS) (read more here).
gRPC mode (read more here).
Secretless access to upstreams and injecting bearer, basic, or custom authentication header credentials (read more here).
Exposing the API publicly for anonymous access (read more here).
Application layer-aware ABAC access control via policy-as-code using CEL and Open Policy Agent (read more here).
OpenTelemetry-ready, application-layer L7 aware auditing and visibility (read more here).