Identity-aware access control

Control who can use each model and what they can ask it to do

Every request is associated with an authenticated Octelium User. Authorization evaluates that identity together with normalized inference context before the request reaches the provider.

Apply CEL or OPA policy to human and workload identity, groups, session type, device posture and request attributes together with protocol, operation, model, streaming mode, tools and token limits.
Allow or deny access and constrain the effective model or reasoning configuration from the same request context used for authorization.
Select upstreams, provider accounts, credentials and LLM configuration dynamically; use semantic routing when model choice should depend on request intent.

Read about LLM access control

policy.yaml

Identity-aware policyAuthorize the model and operation against the person or workload making the request.

Identity-aware policy. Authorize the model and operation against the person or workload making the request.

kind: Policy
metadata:
  name: production-inference
spec:
  rules:
    - effect: ALLOW
      condition:
        all:
          of:
            - match: ctx.request.llm.operation == "GENERATE"
            - match: ctx.request.llm.model in ["auto", "gpt-5"]
            - match: '"ai-platform" in ctx.user.spec.groups'
            - match: ctx.session.status.type == "CLIENT"

Dynamic model and reasoning. Make model and reasoning policy decisions from identity and request context.

spec:
  mode: LLM
  config:
    upstream:
      url: https://api.openai.com
    llm:
      protocol: OPENAI
      model:
        eval: |
          "ai-platform" in ctx.user.spec.groups
            ? "gpt-5" : "gpt-5-mini"
      reasoning:
        eval: |
          ctx.user.spec.type == "WORKLOAD"
            ? "LOW" : "HIGH"

Semantic routing. Route by intent instead of brittle keywords, with an explicit fallback model.

llm:
  plugins:
    - name: route-by-intent
      condition:
        match: ctx.request.llm.model == "auto"
      semanticRouter:
        fallbackModel: gpt-5-mini
        routes:
          - name: complex-engineering
            description: Architecture and debugging
            examples:
              - Diagnose this distributed deadlock
            model: gpt-5
            minSimilarity: 0.45

AI security and governance

Apply LLM-aware controls to every inference request

Octelium understands the semantics of supported LLM protocols instead of treating inference as opaque HTTP. Inference-aware plugins provide a common control layer for safety, model selection, usage limits and optimization.

Explore LLM plugins and controls

Inference pipelinerequest authorized

01Guardrailsdetect · redact · deny
02Tool controlsallow · remove · replace
03Semantic routerintent → model
04Prompt controlsprepend · replace · reject
05Semantic cacheexact + vector match
06Token quotauser · session · tenant

Modelgpt-5.6

CacheMISS

Quota97.4K left

Provider credential isolation

Keep model-provider credentials out of applications and agents

Applications authenticate to Octelium with their own identity instead of receiving a shared provider key. After authorization, the gateway supplies the credential required by the selected upstream and prevents downstream authentication material from being forwarded to the provider.

Read about secretless access

openai.yaml

OpenAI bearer tokenThe provider credential stays in a Secret and is injected only after authorization.

OpenAI bearer token. The provider credential stays in a Secret and is injected only after authorization.

kind: Service
metadata:
  name: inference
spec:
  mode: LLM
  isPublic: true
  config:
    upstream:
      url: https://api.openai.com
    llm:
      protocol: OPENAI
      auth:
        bearer:
          fromSecret: openai-api-key

Gemini API key. Custom header authentication supports provider-specific API key conventions.

kind: Service
metadata:
  name: gemini
spec:
  mode: LLM
  config:
    upstream:
      url: https://generativelanguage.googleapis.com/v1beta
    llm:
      protocol: GEMINI
      auth:
        custom:
          header: x-goog-api-key
          value:
            fromSecret: gemini-api-key

Amazon Bedrock SigV4. Native AWS Signature Version 4 support signs Bedrock requests inside the gateway.

spec:
  mode: LLM
  config:
    upstream:
      url: https://bedrock-runtime.us-east-1.amazonaws.com
    llm:
      protocol: BEDROCK
      auth:
        sigv4:
          accessKeyID: AKIAEXAMPLE
          secretAccessKey:
            fromSecret: aws-secret-access-key
          region: us-east-1
          service: bedrock

Multi-provider AI gateway

Connect applications to OpenAI, Anthropic, Gemini and Bedrock through one gateway

Octelium natively understands all four supported inference protocols and can translate between them in any direction. Applications can keep the API format that fits them while the gateway selects a different provider or protocol upstream.

See supported providers and protocols

OpenAI

Anthropic

Gemini

AWS Bedrock

Mistral

Meta AI

DeepSeek

Kimi

Grok

Qwen

Z.AI

Ollama

AI observability and audit

Trace AI usage by identity, model, policy, tool and token

Every LLM Service emits structured, inference-specific AccessLogs linked to the authenticated User, Session, Service and authorization decision. Investigate individual requests or aggregate usage across identities, applications, models and policies.

See LLM visibility fields

LLM visibilityLast 24 hours · all Services

UserModelProtocol

Total tokens

18.7M14.2M in · 4.5M out

Reasoning

1.3M2,406 managed requests

Cache hit rate

38.4%6,182 requests served

Quota denials

2718,942 allowed

Top token consumersupport-agent42%

Top effective modelgpt-5-mini48%

Top called toolsearch_docs3.1K

gpt-6OPENAISemantic route3 tool callssupport-agent · inference.prodInput / output1.8K / 412

claude-opus-5.5ANTHROPIC1.2K cache readStreammaya@acme.com · research.aiInput / output8.2K / 1.4K

gemini-3.8-flashGEMINISemantic cacheGuardrail passeddocs-bot · assistants.aiInput / output940 / 206

Selected request

Requested modelauto

Effective modelgpt-6

Total tokens2,244

Reasoning tokens184

Latency · TTFT842ms · 188ms TTFT

Identitysupport-agent

Token quotaALLOWED

Finish reasonSTOP

Routing, caching and usage control

Route models intelligently and control inference consumption

LLM-native plugins let platform teams optimize model usage without moving those decisions into every application. Routing, caching and token controls can use the same identity and request context as access policy.

Explore routing, caching and token controls

openai.yaml

kind: Service

metadata:

spec:

mode: LLM

config:

upstream:

url: https://api.openai.com

llm:

protocol: OPENAI

auth:

bearer:

fromSecret: openai-api-key

plugins:

- name: house-rules

condition:

match: ctx.request.llm.operation == "GENERATE"

prompt:

system:

mode: PREPEND

content:

value: Follow Acme's security and data-handling rules.

- name: caller-identity

condition:

match: ctx.request.llm.operation == "GENERATE"

prompt:

message:

role: USER

position: NEW_BEFORE

content:

eval: '"The requesting User is " + ctx.user.metadata.name'

Capabilities of the LLM and MCP gateway modes, each shown as the Service configuration that enables it

LLM gateway, Prompt instructions: Insert, replace or reject system instructions and messages. openai.yaml reads as follows.
kind: Service
metadata:
name: openai
spec:
mode: LLM
config:
upstream:
url: https://api.openai.com
llm:
protocol: OPENAI
auth:
bearer:
fromSecret: openai-api-key
plugins:
- name: house-rules
condition:
match: ctx.request.llm.operation == "GENERATE"
prompt:
system:
mode: PREPEND
content:
value: Follow Acme's security and data-handling rules.
- name: caller-identity
condition:
match: ctx.request.llm.operation == "GENERATE"
prompt:
message:
role: USER
position: NEW_BEFORE
content:
eval: '"The requesting User is " + ctx.user.metadata.name'
mode: PREPEND: first instruction
eval: '"The requesting User is " + ctx.user.metadata.name': …User is support-bot
LLM gateway, Guardrails: Deny secrets and redact personal data before the model sees it. openai.yaml reads as follows.
kind: Service
metadata:
name: openai
spec:
mode: LLM
config:
upstream:
url: https://api.openai.com
llm:
protocol: OPENAI
auth:
bearer:
fromSecret: openai-api-key
plugins:
- name: input-dlp
condition:
match: ctx.request.llm.operation == "GENERATE"
guardrail:
leg: REQUEST
scopes:
- CONTENT
- TOOL_RESULTS
patterns:
- secrets: {}
action: DENY
- type: EMAIL
action: REDACT
denyMessage: The request contains restricted content
action: DENY: sk-proj-… → 403
action: REDACT: jane@acme.com → [REDACTED:EMAIL]
LLM gateway, Model and reasoning: Decide the model and how much it may reason, per identity. openai.yaml reads as follows.
kind: Service
metadata:
name: openai
spec:
mode: LLM
config:
upstream:
url: https://api.openai.com
llm:
protocol: OPENAI
model:
eval: '"ai-admins" in ctx.user.spec.groups ? "gpt-5" : "gpt-5-mini"'
reasoning:
level: MEDIUM
auth:
bearer:
fromSecret: openai-api-key
eval: '"ai-admins" in ctx.user.spec.groups ? "gpt-5" : "gpt-5-mini"': support-bot → gpt-5-mini
level: MEDIUM: reasoning_effort: medium
LLM gateway, Semantic routing and caching: Route by meaning and answer repeats from a per-User cache. openai.yaml reads as follows.
kind: Service
metadata:
name: openai
spec:
mode: LLM
config:
upstream:
url: https://api.openai.com
llm:
protocol: OPENAI
auth:
bearer:
fromSecret: openai-api-key
embedding:
source:
currentUpstream: true
model: text-embedding-3-small
plugins:
- name: auto-model
condition:
match: ctx.request.llm.model == "auto"
semanticRouter:
fallbackModel: gpt-5-mini
routes:
- name: engineering
examples:
- Why is this Go program deadlocking?
model: gpt-5
- name: user-cache
condition:
match: ctx.request.llm.operation == "GENERATE"
semanticCache:
scope:
perUser: true
minSimilarity: 0.92
useXCacheHeader: true
- name: engineering: similarity 0.81
model: gpt-5: auto → gpt-5
minSimilarity: 0.92: 0.97
useXCacheHeader: true: X-Cache: HIT
LLM gateway, Token rate limiting: Sliding-window token budgets per User, Session or tenant. openai.yaml reads as follows.
kind: Service
metadata:
name: openai
spec:
mode: LLM
config:
upstream:
url: https://api.openai.com
llm:
protocol: OPENAI
auth:
bearer:
fromSecret: openai-api-key
plugins:
- name: user-budget
condition:
match: ctx.request.llm.operation == "GENERATE"
tokenRateLimit:
key:
perUser: true
limit: 100000
window:
hours: 1
defaultOutputTokens: 2048
denyMessage: The hourly token budget has been exhausted
limit: 100000: 98,912 used
defaultOutputTokens: 2048: 2,048 reserved
denyMessage: The hourly token budget has been exhausted: 429 Too Many Requests
LLM gateway, Protocol translation: Serve the OpenAI API to agents from an Anthropic upstream. claude.yaml reads as follows.
kind: Service
metadata:
name: claude
spec:
mode: LLM
config:
upstream:
url: https://api.anthropic.com
llm:
protocol: OPENAI
translation:
upstreamProtocol: ANTHROPIC
defaultMaxOutputTokens: 4096
auth:
custom:
header: x-api-key
value:
fromSecret: anthropic-api-key
protocol: OPENAI: /v1/chat/completions
upstreamProtocol: ANTHROPIC: /v1/messages
fromSecret: anthropic-api-key: injected upstream
MCP gateway, Tool-level access control: Authorize every method, tool and argument per identity. finance-mcp.yaml reads as follows.
kind: Service
metadata:
name: finance-mcp
spec:
mode: MCP
config:
upstream:
url: http://10.0.5.18:8080/mcp
mcp:
endpoint: /mcp
auth:
bearer:
fromSecret: mcp-upstream-token
authorization:
inlinePolicies:
- spec:
rules:
- effect: ALLOW
condition:
match: ctx.request.mcp.method == "tools/list"
- effect: ALLOW
condition:
all:
of:
- match: ctx.request.mcp.method == "tools/call"
- match: ctx.request.mcp.name == "transfer"
- match: '"finance-agents" in ctx.user.spec.groups'
- match: ctx.request.mcp.http.bodyMap.params.arguments.amount <= 1000
match: ctx.request.mcp.method == "tools/list": tools/list
- match: ctx.request.mcp.method == "tools/call": tools/call
- match: ctx.request.mcp.name == "transfer": transfer
- match: '"finance-agents" in ctx.user.spec.groups': finance-agents
- match: ctx.request.mcp.http.bodyMap.params.arguments.amount <= 1000: 25000
MCP gateway, Guardrails: Catch leaked secrets in arguments and injections in results. finance-mcp.yaml reads as follows.
kind: Service
metadata:
name: finance-mcp
spec:
mode: MCP
config:
upstream:
url: http://10.0.5.18:8080/mcp
mcp:
endpoint: /mcp
auth:
bearer:
fromSecret: mcp-upstream-token
plugins:
- name: tool-content
condition:
match: ctx.request.mcp.method == "tools/call"
guardrail:
leg: BOTH
scopes:
- TOOL_ARGUMENTS
- TOOL_RESULTS
patterns:
- secrets: {}
action: DENY
- regex: '(?i)ignore (all )?previous instructions'
action: DENY
denyMessage: Restricted content was detected
- TOOL_ARGUMENTS: agent → server
- TOOL_RESULTS: server → agent
action: DENY: ghp_… in arguments
action: DENY: injection in a result
MCP gateway, Lua scripting: Rewrite JSON-RPC messages with the caller's identity. finance-mcp.yaml reads as follows.
kind: Service
metadata:
name: finance-mcp
spec:
mode: MCP
config:
upstream:
url: http://10.0.5.18:8080/mcp
mcp:
endpoint: /mcp
auth:
bearer:
fromSecret: mcp-upstream-token
plugins:
- name: add-user
condition:
all:
of:
- match: ctx.request.mcp.method == "tools/call"
- match: ctx.request.mcp.name == "lookup"
lua:
inline: |
function onRequest(ctx)
local body = json.decode(octelium.req.getRequestBody())
body.params.arguments.userUID = ctx.user.metadata.uid
octelium.req.setRequestBody(json.encode(body))
end
body.params.arguments.userUID = ctx.user.metadata.uid: 5f0e8b2a-…
octelium.req.setRequestBody(json.encode(body)): re-validated
MCP gateway, Rate limiting: Bound tool calls per User and answer with JSON-RPC errors. finance-mcp.yaml reads as follows.
kind: Service
metadata:
name: finance-mcp
spec:
mode: MCP
config:
upstream:
url: http://10.0.5.18:8080/mcp
mcp:
endpoint: /mcp
auth:
bearer:
fromSecret: mcp-upstream-token
plugins:
- name: user-rate
condition:
matchAny: true
rateLimit:
key:
perUser: true
limit: 100
window:
minutes: 2
statusCode: 429
body:
inline: '{"jsonrpc":"2.0","error":{"code":-32000,"message":"Rate limit exceeded"},"id":null}'
limit: 100: 100 used
statusCode: 429: 101st call
MCP gateway, Managed containers: Deploy and scale the MCP server itself inside the Cluster. finance-mcp.yaml reads as follows.
kind: Service
metadata:
name: finance-mcp
spec:
mode: MCP
config:
upstream:
container:
image: ghcr.io/acme/finance-mcp:1.4.0
port: 8080
replicas: 3
resourceLimit:
cpu:
millicores: 1000
memory:
megabytes: 2000
mcp:
endpoint: /mcp
replicas: 3: 3/3 ready

Self-hosted deployment

Run the AI gateway on infrastructure you control

Octelium runs on Kubernetes and does not require a hosted control plane in the inference request path. Keep the gateway, provider credentials and audit stream inside your own cloud or on-premises environment.

Read the management guide

cluster.yaml+14 −18

kind: Group

metadata:

spec: {}

---

kind: User

metadata:

spec:

type: HUMAN

email: alice@acme.com

groups: ["payments"]

---

kind: User

metadata:

spec:

type: HUMAN

groups: ["payments"]

---

kind: Service

metadata:

spec:

mode: HTTP

config:

upstream:

url: http://10.0.3.24:8080

---

kind: Service

metadata:

spec: ⋯

---

kind: User

metadata:

spec: ⋯

~/octeliummain

Cluster resources applied declaratively from YAML with octeliumctl and managed programmatically with the Go, TypeScript, Python and Rust SDKs

GitOps: cluster.yaml in the main branch of the ~/octelium repository describes the desired state of the Cluster. Its diff reads as follows, with + for added, ~ for changed and - for removed lines.
+ kind: Group
+ metadata:
+ name: payments
+ spec: {}
+ ---
+ kind: User
+ metadata:
+ name: alice
+ spec:
+ type: HUMAN
+ email: alice@acme.com
+ groups: ["payments"]
---
kind: User
metadata:
name: bob
spec:
type: HUMAN
~ groups: ["payments"]
---
kind: Service
metadata:
name: billing
spec:
mode: HTTP
config:
upstream:
~ url: http://10.0.3.24:8080
- ---
- kind: Service
- metadata:
- name: billing-legacy
- spec: ⋯
- ---
- kind: User
- metadata:
- name: john
- spec: ⋯
~/octelium ❯ octeliumctl apply --prune .
Group: payments Created
User: alice Created
User: bob Updated
Service: billing.default Updated
Service: billing-legacy.default Deleted
User: john Deleted
Cluster Core resources successfully applied
2 resources created
2 resources updated
2 resources deleted
Go: main.go uses the octelium-go SDK to list the Users of the Cluster and update the email and Policies of linus.
client, err := octelium.New(ctx)
if err != nil {⋯}
conn, err := client.Conn(ctx)
if err != nil {⋯}
api := corev1.NewMainServiceClient(conn)
users, err := api.ListUser(ctx, &corev1.ListUserOptions{})
if err != nil {⋯}
for _, u := range users.Items {
fmt.Println(u.Metadata.Name)
}
user, err := api.GetUser(ctx, &metav1.GetOptions{Name: "linus"})
if err != nil {⋯}
user.Spec.Email = "linus@acme.com"
user.Spec.Authorization = &corev1.User_Spec_Authorization{
Policies: []string{"db-readonly"},
}
if _, err := api.UpdateUser(ctx, user); err != nil {⋯}
fmt.Println("updated", user.Metadata.Name)
~/app ❯ go run .
alice
bob
linus
updated linus
TypeScript: main.ts uses the @octelium/sdk SDK to list the Users of the Cluster and update the email and Policies of linus.
const client = await OcteliumClient.create({
domain: "acme.com",
auth: {⋯},
});
const users = await client.coreV1.listUser(ListUserOptions.create());
for (const u of users.response.items) {
console.log(u.metadata?.name);
}
const { response: user } = await client.coreV1.getUser(
GetOptions.create({ name: "linus" }),
);
user.spec!.email = "linus@acme.com";
user.spec!.authorization = User_Spec_Authorization.create({
policies: ["db-readonly"],
});
await client.coreV1.updateUser(user);
console.log("updated", user.metadata?.name);
await client.close();
~/app ❯ npx tsx main.ts
alice
bob
linus
updated linus
Python: main.py uses the octelium-sdk SDK to list the Users of the Cluster and update the email and Policies of linus.
async with await OcteliumClient.create() as client:
users = await client.core_v1.list_user(ListUserOptions())
for u in users.items:
print(u.metadata.name)
user = await client.core_v1.get_user(GetOptions(name="linus"))
user.spec.email = "linus@acme.com"
user.spec.authorization.policies = ["db-readonly"]
await client.core_v1.update_user(user)
print("updated", user.metadata.name)
~/app ❯ python main.py
alice
bob
linus
updated linus
Rust: main.rs uses the octelium SDK to list the Users of the Cluster and update the email and Policies of linus.
let client = Client::from_env().await?;
let mut api = client.core_v1();
let users = api.list_user(ListUserOptions::default()).await?;
for u in users.into_inner().items {
println!("{}", u.metadata.unwrap_or_default().name);
}
let options = GetOptions {
name: "linus".into(),
..Default::default()
};
let mut user = api.get_user(options).await?.into_inner();
let spec = user.spec.get_or_insert_default();
spec.email = "linus@acme.com".into();
spec.authorization.get_or_insert_default().policies = vec!["db-readonly".into()];
let user = api.update_user(user).await?.into_inner();
println!("updated {}", user.metadata.unwrap_or_default().name);
~/app ❯ cargo run -q
alice
bob
linus
updated linus

AI gateway capabilities

Security and control across the inference path

Authorization, provider authentication, routing, inference controls and audit records operate on the same authenticated and normalized request context.

01
Identity and context
Human and workload identity, groups, session type, device posture and request attributes meet in one CEL or OPA authorization decision.
Read more
02
Dynamic models and reasoning
A static value, CEL expression, Rego result or ordered plugin can choose the effective model and a portable reasoning level, exact token budget or provider-native effort.
Read more
03
Prompt and tool governance
Prepend, append, replace, strip or reject instructions; inject messages; and allow, remove, deny, replace or add tool definitions.
Read more
04
Guardrails
Inspect request content, instructions, tool definitions, tool results and completed responses using regex, PII detectors and secret detection.
Read more
05
Token-based rate limiting
Reserve and reconcile input, output or total-token quotas over sliding windows, scoped per User, Session or an evaluated tenant key.
Read more
06
Semantic caching
Reuse exact or semantically equivalent successful generations within a per-Session, per-User, shared or tenant-defined security partition.
Read more
07
Semantic routing
Embed the request's semantic subject and select the highest-scoring model route, with a deterministic fallback when nothing matches.
Read more
08
Extensible request path
Compose LLM-native controls with Lua, ExtProc, JSON Schema, direct responses, request rate limits and path plugins.
Read more

Questions

Frequently asked

Which AI providers and APIs does Octelium support?: The LLM mode natively understands OpenAI, Anthropic, Gemini and Amazon Bedrock protocols and can translate between any of the four in either direction. It covers generation, streaming, embeddings, token counting, moderation and model discovery where the selected protocol exposes those operations. OpenAI-compatible servers such as vLLM and Ollama can also be used when they expose the supported OpenAI API surface.
What makes the AI gateway identity-aware?: Every request is tied to an Octelium human or workload User instead of relying on a shared provider key. Policies combine that identity, group membership, session type and device posture with normalized LLM attributes such as operation, requested model, streaming, modalities, tools and declared output tokens. The same context can choose an upstream account, model, reasoning level and plugin configuration dynamically.
Can Octelium route requests to different models dynamically?: Yes. Model and reasoning settings can be static, computed with CEL, returned by Open Policy Agent, or applied conditionally through plugins. Semantic routing classifies the meaning of a request against descriptions and examples and sends it to the best matching model. Dynamic Service configuration can additionally select a different upstream, provider account, credential, headers and complete LLM configuration.
How does token-based rate limiting work?: Token rate limiting applies a sliding-window quota to input, output or total inference tokens. Quotas can be keyed per User, per Session or by an evaluated value such as a tenant ID. Octelium reserves capacity before sending the request so concurrent requests observe one another, then reconciles the reservation with usage reported by the provider.
What does semantic caching store and how is it isolated?: The semantic cache can serve an exact repeat without an embedding call or find an earlier semantically equivalent request using vector similarity. The effective model, instructions, tools, reasoning, response schema and other execution context must still match. Cache partitions can be per Session, per User, shared or derived from context for tenant isolation. A cache hit uses no inference quota and still passes current response guardrails.
Which guardrails and prompt controls are available?: Guardrails inspect instructions, content, tool definitions, tool results and generated responses. They support RE2 patterns, built-in PII detectors and broad secret detection. Matching content can be denied, redacted, stripped or replaced where mutation is safe. Prompt plugins can prepend, append, replace, strip or reject system instructions and insert or modify user and assistant messages.
What can teams see about AI usage?: Identity-aware AccessLogs record who used which Service and model, requested and effective models, protocol, operation, routing decision, reasoning, finish reason, tools, guardrails, cache and quota outcomes, latency, time to first token and reported input, output, cached and reasoning token usage. The enterprise console aggregates these fields into trends, breakdowns and top users, models and tools.
Is Octelium self-hosted and horizontally scalable?: Yes. Octelium is free and open source, has no mandatory hosted control plane and runs on Kubernetes infrastructure you control. Production deployments distribute the data plane across Gateway nodes and scale it horizontally with Kubernetes. The same Cluster can deploy and scale containerized self-hosted inference servers as managed Service upstreams.
Do applications receive the upstream provider credentials?: No. Applications authenticate to Octelium with their own identity. Provider bearer tokens, custom API keys, OAuth2 client credentials, basic credentials and AWS SigV4 secrets stay in the Cluster and are injected only after authorization.

Keep reading

Run Octelium on your own infrastructure

Install a self-hosted Cluster and start applying identity-aware access control to your Services.

Quick install guide GitHub repo Enterprise package

Secure, govern and observe LLM access

Control who can use each model and what they can ask it to do

Apply LLM-aware controls to every inference request

Keep model-provider credentials out of applications and agents

Connect applications to OpenAI, Anthropic, Gemini and Bedrock through one gateway

Trace AI usage by identity, model, policy, tool and token

Route models intelligently and control inference consumption

Run the AI gateway on infrastructure you control

Security and control across the inference path

Identity and context

Dynamic models and reasoning

Prompt and tool governance

Guardrails

Token-based rate limiting

Semantic caching

Semantic routing

Extensible request path

Frequently asked

Related solutions

MCP gateway

Secure infrastructure for AI agents

Open source API gateway

Run Octelium on your own infrastructure