Octelium as an API Gateway

Octelium provides a unified, scalable, self-hosted, open source infrastructure to build your AI gateways. Octelium provides the following:

A scalable platform built on top of Kubernetes to provide access to your APIs whether they are internal resources behind NAT in multiple private clouds, on-prem, your own laptop, etc... as well as to publicly protected SaaS resources.
Deploy, scale and secure access to your containerized APIs (read more about managed containers here).
Centralized identity-based, application-layer (L7) aware ABAC access control on a per-request basis (read more about access control here).
Unified identity management for both HUMAN as well as WORKLOAD Users (read more here).
Advanced request/output sanitization and manipulation of headers and body content via Lua scripts and Envoy ExtProc compatible interfaces (read more here) to build you ad-hoc rate limiting, caching, and DLP logic.
Unified access and identity for WORKLOAD Users via OAuth2 client credentials (read more here) and standard bearer authentication (read more here).
OpenTelemetry-native, identity-based, L7 aware visibility and auditing (read more here).
Secretless access to upstreams that require bearer authentication, basic authentication, OAuth2 client credentials or AWS sigv4 (read more here).
Dynamic, identity-based, L7-aware routing to your upstreams (read more about dynamic configuration here).
Extremely dynamic plugin system, that includes Lua scripts and Envoy ExtProc compliant servers, to manipulate your request/response headers and body content, including serialized JSON (read more here).
GitOps-friendly declarative, programmable management (read more here).

This is a very simple example for a Service to represent a certain API upstream server. The upstream server could be (read more about Service upstreams here):

A directly accessible resource by the Octelium Cluster (e.g. a Kubernetes service running on the same Kubernetes cluster).

1kind: Service
2metadata:
3  name: my-api
4spec:
5  mode: HTTP
6  isPublic: true
7  port: 80
8  config:
9    upstream:
10      url: http://my-api.default.svc:8080

A remotely accessible resource by the Octelium Cluster via a connected Session (read more here). This could be, in practice, any server that is listening behind NAT (e.g. private cloud, IoT, your own laptop). Here is an example:

1kind: Service
2metadata:
3  name: my-api
4spec:
5  mode: HTTP
6  isPublic: true
7  port: 80
8  config:
9    upstream:
10      url: http://my-api.default.svc:8080
11      user: eu-central-2

A publicly protected resource via bearer or basic authentication (read more here). Here is an example for a public SaaS resource that is protected by a bearer access token:

1kind: Service
2metadata:
3  name: my-gemini
4spec:
5  mode: HTTP
6  isPublic: true
7  config:
8    upstream:
9      url: https://generativelanguage.googleapis.com
10      auth:
11        bearer:
12          fromSecret: gemini-api-key

A containerized/Docker application that can be deployed as a managed container (read more here). Here is an example:

1kind: Service
2metadata:
3  name: my-api
4spec:
5  mode: HTTP
6  isPublic: true
7  config:
8    upstream:
9      container:
10        port: 3000
11        image: ghcr.io/<ORG>/<IMAGE>:<TAG>
12        replicas: 3
13        credentials:
14          usernamePassword:
15            username: linus
16            password:
17              fromSecret: reg-password
18        resourceLimit:
19          cpu:
20            millicores: 2000
21          memory:
22            megabytes: 4000

You can dynamic configuration (read more here) to dynamically route among multiple upstreams the Service based on identity and/or request context (e.g. HTTP request paths, body content, etc...). Here is a minimal example:

1kind: Service
2metadata:
3  name: my-api
4spec:
5  mode: HTTP
6  isPublic: true
7  dynamicConfig:
8    configs:
9      - name: v1
10        upstream:
11          url: https://apiv1.example.com
12        http:
13          path:
14            removePrefix: /v1
15      - name: v2
16        upstream:
17          url: https://apiv2.example.com
18        http:
19          path:
20            removePrefix: /v2
21    rules:
22      - condition:
23          match: 'ctx.request.http.path.startsWith("/v1")'
24        configName: v1
25      - condition:
26          match: 'ctx.request.http.path.startsWith("/v2")'
27        configName: v2

You can now apply the creation of the Service as follows (read more here):

octeliumctl apply /PATH/TO/SERVICE.YAML

Octelium doesn't just enable you to dynamically route to an upstream based on a request path, but since it's built as a zero trust architecture, it can dynamically route to an upstream based on the identity and the access context too. Here is an another example:

1kind: Service
2metadata:
3  name: my-api
4spec:
5  mode: HTTP
6  isPublic: true
7  dynamicConfig:
8    configs:
9      - name: v1
10        upstream:
11          url: https://apiv1.example.com
12        http:
13          path:
14            removePrefix: /v1
15      - name: v2
16        upstream:
17          url: https://apiv2.example.com
18        http:
19          path:
20            removePrefix: /v2
21    rules:
22      - condition:
23          match: '"prod" in ctx.user.spec.groups'
24        configName: v1
25      - condition:
26          match: '"dev" in ctx.user.spec.groups'
27        configName: v2

You can also use upstreams that are protected by their own credentials such as bearer tokens (read more about secretless access in HTTP here). Here is an example:

1kind: Service
2metadata:
3  name: my-api
4spec:
5  mode: HTTP
6  isPublic: true
7  dynamicConfig:
8    configs:
9      - name: v1
10        upstream:
11          url: https://apiv1.example.com
12        http:
13          auth:
14            bearer:
15              fromSecret: access-token-1
16      - name: v2
17        upstream:
18          url: https://apiv2.example.com
19        http:
20          auth:
21            bearer:
22              fromSecret: access-token-2
23    rules:
24      - condition:
25          match: 'ctx.request.http.path.startsWith("/v1")'
26        configName: v1
27      - condition:
28          match: 'ctx.request.http.path.startsWith("/v2")'
29        configName: v2

And you can also deploy and scale your own containerized microservices including those that are served via private registries that are protected with credentials. Here is an example:

First you create the Secret for the registry's password as follows:

octeliumctl create secret ghcr-token

And then you create the Service as follows:

1kind: Service
2metadata:
3  name: my-api
4spec:
5  mode: HTTP
6  isPublic: true
7  dynamicConfig:
8    configs:
9      - name: v1
10        upstream:
11          container:
12            image: ghcr.io/org/my-api:v1
13            port: 80
14            replicas: 2
15            credentials:
16              usernamePassword:
17                username: org
18                password:
19                  fromSecret: ghcr-token
20        http:
21          path:
22            removePrefix: /v1
23      - name: v2
24        upstream:
25          container:
26            image: ghcr.io/org/my-api:v2
27            port: 80
28            replicas: 2
29            credentials:
30              usernamePassword:
31                username: org
32                password:
33                  fromSecret: ghcr-token
34        http:
35          path:
36            removePrefix: /v2
37    rules:
38      - condition:
39          match: 'ctx.request.http.path.startsWith("/v1")'
40        configName: v1
41      - condition:
42          match: 'ctx.request.http.path.startsWith("/v2")'
43        configName: v2

You can also opt to choose to provide anonymous public access instead of using Octelium to provide authentication and access control. You might want to use anonymous access, for example, for public APIs or whenever you want to handle your own authentication and authorization at the upstream's side. You can read more about the anonymous access mode here.

NOTE

You can also add and remove request/response headers as in the guide here

When it comes to access control, Octelium provides a rich layer-7 aware, identity-based, context-aware ABAC access control on a per-request basis where you can control access based on the HTTP request's path, method, and even serialized JSON body content using policy-as-code with CEL and Open Policy Agent (OPA) (You can read more in detail about Policies and access control here). Here is a generic example:

1kind: Service
2metadata:
3  name: my-api
4spec:
5  mode: HTTP
6  config:
7    upstream:
8      url: https://api.example.com
9  authorization:
10    inlinePolicies:
11      - spec:
12          rules:
13            - effect: ALLOW
14              condition:
15                all:
16                  of:
17                    - match: ctx.user.spec.groups.hasAll("dev", "ops")
18                    - match: ctx.request.http.method in ["POST", "PUT"]
19                    - match: ctx.request.http.path.startsWith("/users")
20                    - match: ctx.request.http.headers["x-custom-header"] == "this-value"
21                    # bodyMap contains the serialized JSON body content
22                    - match: ctx.request.http.bodyMap.age > 18 && ctx.request.http.bodyMap.email.endsWith("@example.com")

You can also use Octelium's plugins, currently Lua scripts and Envoy's ExtProc, to sanitize and manipulate your request and responses (read more about HTTP plugins here) in arbitrarily complex ways. For example, you can use this to manipulate the JSON body content of requests/responses and call external APIs to implement advanced identity-based caching and rate limiting. Here is an example:

1kind: Service
2metadata:
3  name: safe-api
4spec:
5  mode: HTTP
6  isPublic: true
7  config:
8    upstream:
9      url: https://api.example.com
10      plugins:
11        - name: main
12          condition:
13            matchAny: true
14          lua:
15            inline: |
16              function onRequest(ctx)
17                local body = json.decode(octelium.req.getRequestBody())
18
19                if body.age > 100 or body.age < 18 then
20                  octelium.req.exit(400)
21                end
22
23                if strings.lenUnicode(body.user.name) > 100 then
24                  octelium.req.exit(400)
25                end
26
27                body.email = ctx.user.spec.email
28                body.isBrowser = ctx.session.status.isBrowser
29
30                if strings.toLower(body.action) == "create" then
31                  local c = http.client()
32                  c:setBaseURL("http://rate-limiting.default.svc")
33                  local req = c:request()
34                  local bodyReq = {}
35                  bodyReq.userUID = ctx.user.metadata.uid
36                  req:setBody(json.encode(bodyReq))
37                  local resp, err = req:post("/v1/check")
38                  if err then
39                    octelium.req.exit(500)
40                    return
41                  end
42
43                  if resp:code() == 200 then
44                    local apiResp = json.decode(resp:body())
45                    if not apiResp.isRateLimited then
46                      octelium.req.exit(429)
47                      return
48                    end
49                  end
50                end
51
52                octelium.req.setRequestBody(json.encode(body))
53              end
54
55
56              function onResponse(ctx)
57                octelium.req.deleteResponseHeader("X-Internal")
58                octelium.req.setResponseHeader("X-User-Uid", ctx.user.metadata.uid)
59              end

Octelium also provides other plugins (read more about HTTP plugins here) such as rate limiting, caching, JSON schema validation and direct response that can be used to control your MCP client requests. Here is another example:

1kind: Service
2metadata:
3  name: my-api
4spec:
5  mode: HTTP
6  isPublic: true
7  config:
8    upstream:
9      url: https://api.example.com
10    http:
11      plugins:
12        - name: minute-rate-limit
13          condition:
14            matchAny: true
15          ratelimit:
16            limit: 100
17            window:
18              minutes: 2
19        - name: hour-rate-limit
20          condition:
21            matchAny: true
22          ratelimit:
23            limit: 1000
24            window:
25              hours: 6
26        - name: rate-limit-sensitive
27          condition:
28            match: ctx.request.http.path.startsWith("/sensitive")
29          ratelimit:
30            limit: 10
31            window:
32              minutes: 1
33        - name: validate-users
34          condition:
35            match: ctx.request.http.path == "/users" && ctx.request.http.method in ["POST", "PUT"]
36          jsonSchema:
37            inline: <YOUR_JSON_SCHEMA>
38        - name: validate-groups
39          condition:
40            match: ctx.request.http.path == "/groups" && ctx.request.http.method in ["POST", "PUT"]
41          jsonSchema:
42            inline: <YOUR_JSON_SCHEMA>
43        - name: drop-php
44          condition:
45            match: ctx.request.http.path.toLower().endsWith(".php")
46          direct:
47            statusCode: 400
48            body:
49              inline: '{"error": "invalid path"}'

Octelium also provides OpenTelemetry-ready, application-layer L7 aware visibility and access logging in real time (see an example for HTTP here). You can read more about visibility here.

This was a very short guide to show you how to use Octelium to deploy, scale, route and provide dynamic zero trust secure access to your workloads. Here are a few more related features that you might be interested in:

Routing not just by request paths, but also by header keys and values, request body content including JSON (read more here).
Request/response header manipulation (read more here).
Cross-Origin Resource Sharing (CORS) (read more here).
gRPC mode (read more here).
Secretless access to upstreams and injecting bearer, basic, or custom authentication header credentials (read more here).
Exposing the API publicly for anonymous access (read more here).
Application layer-aware ABAC access control via policy-as-code using CEL and Open Policy Agent (read more here).
OpenTelemetry-ready, application-layer L7 aware auditing and visibility (read more here).