Secure and govern MCP access for AI agents

Capabilities of the LLM and MCP gateway modes, each shown as the Service configuration that enables it

• LLM gateway, Prompt instructions: Insert, replace or reject system instructions and messages. openai.yaml reads as follows.
• kind: Service
• metadata:
• name: openai
• spec:
• mode: LLM
• config:
• upstream:
• url: https://api.openai.com
• llm:
• protocol: OPENAI
• auth:
• bearer:
• fromSecret: openai-api-key
• plugins:
• - name: house-rules
• condition:
• match: ctx.request.llm.operation == "GENERATE"
• prompt:
• system:
• mode: PREPEND
• content:
• value: Follow Acme's security and data-handling rules.
• - name: caller-identity
• condition:
• match: ctx.request.llm.operation == "GENERATE"
• prompt:
• message:
• role: USER
• position: NEW_BEFORE
• content:
• eval: '"The requesting User is " + ctx.user.metadata.name'
• mode: PREPEND: first instruction
• eval: '"The requesting User is " + ctx.user.metadata.name': …User is support-bot
• LLM gateway, Guardrails: Deny secrets and redact personal data before the model sees it. openai.yaml reads as follows.
• kind: Service
• metadata:
• name: openai
• spec:
• mode: LLM
• config:
• upstream:
• url: https://api.openai.com
• llm:
• protocol: OPENAI
• auth:
• bearer:
• fromSecret: openai-api-key
• plugins:
• - name: input-dlp
• condition:
• match: ctx.request.llm.operation == "GENERATE"
• guardrail:
• leg: REQUEST
• scopes:
• - CONTENT
• - TOOL_RESULTS
• patterns:
• - secrets: {}
• action: DENY
• - type: EMAIL
• action: REDACT
• denyMessage: The request contains restricted content
• action: DENY: sk-proj-… → 403
• action: REDACT: jane@acme.com → [REDACTED:EMAIL]
• LLM gateway, Model and reasoning: Decide the model and how much it may reason, per identity. openai.yaml reads as follows.
• kind: Service
• metadata:
• name: openai
• spec:
• mode: LLM
• config:
• upstream:
• url: https://api.openai.com
• llm:
• protocol: OPENAI
• model:
• eval: '"ai-admins" in ctx.user.spec.groups ? "gpt-5" : "gpt-5-mini"'
• reasoning:
• level: MEDIUM
• auth:
• bearer:
• fromSecret: openai-api-key
• eval: '"ai-admins" in ctx.user.spec.groups ? "gpt-5" : "gpt-5-mini"': support-bot → gpt-5-mini
• level: MEDIUM: reasoning_effort: medium
• LLM gateway, Semantic routing and caching: Route by meaning and answer repeats from a per-User cache. openai.yaml reads as follows.
• kind: Service
• metadata:
• name: openai
• spec:
• mode: LLM
• config:
• upstream:
• url: https://api.openai.com
• llm:
• protocol: OPENAI
• auth:
• bearer:
• fromSecret: openai-api-key
• embedding:
• source:
• currentUpstream: true
• model: text-embedding-3-small
• plugins:
• - name: auto-model
• condition:
• match: ctx.request.llm.model == "auto"
• semanticRouter:
• fallbackModel: gpt-5-mini
• routes:
• - name: engineering
• examples:
• - Why is this Go program deadlocking?
• model: gpt-5
• - name: user-cache
• condition:
• match: ctx.request.llm.operation == "GENERATE"
• semanticCache:
• scope:
• perUser: true
• minSimilarity: 0.92
• useXCacheHeader: true
• - name: engineering: similarity 0.81
• model: gpt-5: auto → gpt-5
• minSimilarity: 0.92: 0.97
• useXCacheHeader: true: X-Cache: HIT
• LLM gateway, Token rate limiting: Sliding-window token budgets per User, Session or tenant. openai.yaml reads as follows.
• kind: Service
• metadata:
• name: openai
• spec:
• mode: LLM
• config:
• upstream:
• url: https://api.openai.com
• llm:
• protocol: OPENAI
• auth:
• bearer:
• fromSecret: openai-api-key
• plugins:
• - name: user-budget
• condition:
• match: ctx.request.llm.operation == "GENERATE"
• tokenRateLimit:
• key:
• perUser: true
• limit: 100000
• window:
• hours: 1
• defaultOutputTokens: 2048
• denyMessage: The hourly token budget has been exhausted
• limit: 100000: 98,912 used
• defaultOutputTokens: 2048: 2,048 reserved
• denyMessage: The hourly token budget has been exhausted: 429 Too Many Requests
• LLM gateway, Protocol translation: Serve the OpenAI API to agents from an Anthropic upstream. claude.yaml reads as follows.
• kind: Service
• metadata:
• name: claude
• spec:
• mode: LLM
• config:
• upstream:
• url: https://api.anthropic.com
• llm:
• protocol: OPENAI
• translation:
• upstreamProtocol: ANTHROPIC
• defaultMaxOutputTokens: 4096
• auth:
• custom:
• header: x-api-key
• value:
• fromSecret: anthropic-api-key
• protocol: OPENAI: /v1/chat/completions
• upstreamProtocol: ANTHROPIC: /v1/messages
• fromSecret: anthropic-api-key: injected upstream
• MCP gateway, Tool-level access control: Authorize every method, tool and argument per identity. finance-mcp.yaml reads as follows.
• kind: Service
• metadata:
• name: finance-mcp
• spec:
• mode: MCP
• config:
• upstream:
• url: http://10.0.5.18:8080/mcp
• mcp:
• endpoint: /mcp
• auth:
• bearer:
• fromSecret: mcp-upstream-token
• authorization:
• inlinePolicies:
• - spec:
• rules:
• - effect: ALLOW
• condition:
• match: ctx.request.mcp.method == "tools/list"
• - effect: ALLOW
• condition:
• all:
• of:
• - match: ctx.request.mcp.method == "tools/call"
• - match: ctx.request.mcp.name == "transfer"
• - match: '"finance-agents" in ctx.user.spec.groups'
• - match: ctx.request.mcp.http.bodyMap.params.arguments.amount <= 1000
• match: ctx.request.mcp.method == "tools/list": tools/list
• - match: ctx.request.mcp.method == "tools/call": tools/call
• - match: ctx.request.mcp.name == "transfer": transfer
• - match: '"finance-agents" in ctx.user.spec.groups': finance-agents
• - match: ctx.request.mcp.http.bodyMap.params.arguments.amount <= 1000: 25000
• MCP gateway, Guardrails: Catch leaked secrets in arguments and injections in results. finance-mcp.yaml reads as follows.
• kind: Service
• metadata:
• name: finance-mcp
• spec:
• mode: MCP
• config:
• upstream:
• url: http://10.0.5.18:8080/mcp
• mcp:
• endpoint: /mcp
• auth:
• bearer:
• fromSecret: mcp-upstream-token
• plugins:
• - name: tool-content
• condition:
• match: ctx.request.mcp.method == "tools/call"
• guardrail:
• leg: BOTH
• scopes:
• - TOOL_ARGUMENTS
• - TOOL_RESULTS
• patterns:
• - secrets: {}
• action: DENY
• - regex: '(?i)ignore (all )?previous instructions'
• action: DENY
• denyMessage: Restricted content was detected
• - TOOL_ARGUMENTS: agent → server
• - TOOL_RESULTS: server → agent
• action: DENY: ghp_… in arguments
• action: DENY: injection in a result
• MCP gateway, Lua scripting: Rewrite JSON-RPC messages with the caller's identity. finance-mcp.yaml reads as follows.
• kind: Service
• metadata:
• name: finance-mcp
• spec:
• mode: MCP
• config:
• upstream:
• url: http://10.0.5.18:8080/mcp
• mcp:
• endpoint: /mcp
• auth:
• bearer:
• fromSecret: mcp-upstream-token
• plugins:
• - name: add-user
• condition:
• all:
• of:
• - match: ctx.request.mcp.method == "tools/call"
• - match: ctx.request.mcp.name == "lookup"
• lua:
• inline: |
• function onRequest(ctx)
• local body = json.decode(octelium.req.getRequestBody())
• body.params.arguments.userUID = ctx.user.metadata.uid
• octelium.req.setRequestBody(json.encode(body))
• end
• body.params.arguments.userUID = ctx.user.metadata.uid: 5f0e8b2a-…
• octelium.req.setRequestBody(json.encode(body)): re-validated
• MCP gateway, Rate limiting: Bound tool calls per User and answer with JSON-RPC errors. finance-mcp.yaml reads as follows.
• kind: Service
• metadata:
• name: finance-mcp
• spec:
• mode: MCP
• config:
• upstream:
• url: http://10.0.5.18:8080/mcp
• mcp:
• endpoint: /mcp
• auth:
• bearer:
• fromSecret: mcp-upstream-token
• plugins:
• - name: user-rate
• condition:
• matchAny: true
• rateLimit:
• key:
• perUser: true
• limit: 100
• window:
• minutes: 2
• statusCode: 429
• body:
• inline: '{"jsonrpc":"2.0","error":{"code":-32000,"message":"Rate limit exceeded"},"id":null}'
• limit: 100: 100 used
• statusCode: 429: 101st call
• MCP gateway, Managed containers: Deploy and scale the MCP server itself inside the Cluster. finance-mcp.yaml reads as follows.
• kind: Service
• metadata:
• name: finance-mcp
• spec:
• mode: MCP
• config:
• upstream:
• container:
• image: ghcr.io/acme/finance-mcp:1.4.0
• port: 8080
• replicas: 3
• resourceLimit:
• cpu:
• millicores: 1000
• memory:
• megabytes: 2000
• mcp:
• endpoint: /mcp
• replicas: 3: 3/3 ready