Visibility

An LLM Service emits identity-aware, inference-specific AccessLogs in real time. The common part identifies the User, Session, Device, Service, Namespace, authorization decision and policy match. The entry.info.llm part records the normalized inference exchange (read more about visibility here).

Body and Header Visibility

Prompt and completion bodies are disabled by default because they routinely contain secrets, personal information, source code and proprietary data. Enable only the required fields:

spec:
  mode: LLM
  config:
    upstream:
      url: https://api.openai.com
    llm:
      visibility:
        enableRequestBody: true
        enableRequestBodyMap: true
        enableResponseBody: true
        enableResponseBodyMap: true
        includeRequestHeaders:
          - User-Agent
          - X-Request-ID
        includeResponseHeaders:
          - Content-Type
          - X-Provider-Request-ID

enableRequestBody records the request body as bytes.
enableRequestBodyMap records its serialized JSON representation.
enableResponseBody records a non-streamed response body as bytes.
enableResponseBodyMap records its serialized JSON representation.
includeRequestHeaders and includeResponseHeaders select individual headers.

All headers can be enabled with explicit exclusions:

llm:
  visibility:
    includeAllRequestHeaders: true
    includeAllResponseHeaders: true
    excludeRequestHeaders:
      - Cookie
      - X-Internal-Token
    excludeResponseHeaders:
      - Set-Cookie

Sensitive authentication and session headers are excluded even when all headers are enabled. Body and event capture is bounded, so a large body can be omitted. Streamed response bodies and body maps are not recorded through the response-body options.

note

Access-log visibility and semantic cache storage are independent. A Service that records no prompt or completion body still stores prompt-derived vectors, execution-context digests and generated responses while a semantic cache entry lives.

Entry Types

The type field describes the lifecycle of a response:

COMPLETE is a request with a finite response.
STREAM_START is emitted when a streamed response starts.
STREAM_END is the final record of that stream and carries accumulated response metadata and usage.

The source field identifies who produced the response:

UPSTREAM means the provider produced it and inference tokens were consumed.
SEMANTIC_CACHE means an earlier response was reused without invoking the provider.
OCTELIUM means Octelium produced it, including rejected requests.

isUpstreamInvoked is intentionally separate. A response guardrail can reject provider-generated output, resulting in source: OCTELIUM while isUpstreamInvoked: true and usage still show the tokens consumed.

Complete Response

The following is the LLM-specific part of an example non-streamed AccessLog. Common identity and policy fields are omitted for brevity:

{
  "entry": {
    "info": {
      "llm": {
        "type": "COMPLETE",
        "protocol": "OPENAI",
        "operation": "GENERATE",
        "route": "RESPONSES",
        "source": "UPSTREAM",
        "isUpstreamInvoked": true,
        "model": {
          "requested": "auto",
          "effective": "gpt-5",
          "reported": "gpt-5-2026-08-01",
          "source": "SEMANTIC_ROUTER"
        },
        "maxOutputTokens": "2048",
        "inputItemCount": 1,
        "estimatedInputTokens": "96",
        "estimateQuality": "COMPLETE",
        "usage": {
          "state": "COMPLETE",
          "inputTokens": "101",
          "outputTokens": "328",
          "totalTokens": "429",
          "cacheReadInputTokens": "0",
          "reasoningOutputTokens": "112"
        },
        "responseID": "resp_01HXYZ",
        "finishReason": "TOOL_CALL",
        "rawFinishReason": "tool_calls",
        "reasoning": {
          "effort": "high"
        },
        "tools": {
          "count": 2,
          "names": ["read_document", "search_knowledge_base"],
          "removedCount": 1,
          "removedNames": ["execute_shell"],
          "calledNames": ["search_knowledge_base"],
          "callCount": 1
        },
        "guardrails": [
          {
            "result": "MODIFIED",
            "leg": "REQUEST",
            "plugin": "input-dlp"
          },
          {
            "result": "PASS",
            "leg": "RESPONSE",
            "plugin": "output-safety"
          }
        ],
        "tokenRateLimit": {
          "result": "ALLOWED"
        },
        "semanticCache": {
          "result": "MISS",
          "plugin": "user-cache"
        },
        "semanticRouter": {
          "result": "MATCH",
          "route": "complex-engineering",
          "similarity": 0.81,
          "model": "gpt-5",
          "plugin": "automatic-model-selection"
        },
        "http": {
          "request": {
            "method": "POST",
            "path": "/v1/responses",
            "bodyBytes": "742"
          },
          "response": {
            "code": 200,
            "bodyBytes": "1480",
            "contentType": "application/json"
          },
          "httpVersion": "HTTP2"
        }
      }
    }
  }
}

Integer values whose protobuf type is 64-bit are represented as JSON strings.

Model and Reasoning

model distinguishes three names:

requested is what the downstream asked for.
effective is what Octelium sent to the provider.
reported is the provider's response value, which can resolve an alias or unpinned name.

Its source is unset when the requested model was preserved, or is CONFIG, PLUGIN or SEMANTIC_ROUTER. plugin names the model plugin when the source is PLUGIN.

reasoning is present when the Service decided the configuration. It contains isDisabled, a served ordinal effort, or a numeric tokenBudget. Octelium does not interpret a reasoning value controlled entirely by the downstream.

Provider Token Usage

usage contains provider-reported counts only. It is never populated with Octelium's estimate, and it is absent for a semantic cache hit. state is COMPLETE for final provider usage or PARTIAL when a stream ended before the final usage report.

inputTokens, outputTokens and totalTokens are accompanied by:

cacheReadInputTokens, which is a subset of input for OpenAI and additive for Anthropic.
cacheWriteInputTokens, which Anthropic reports additively.
reasoningOutputTokens, which OpenAI already includes in output.

Use estimatedInputTokens and estimateQuality when no provider usage is present, but never treat the estimate as billing truth.

Streamed Response

A streamed response emits a start record followed by an end record. The end record can include time to first token, event count, provider usage and finish reason:

{
  "entry": {
    "info": {
      "llm": {
        "type": "STREAM_END",
        "protocol": "ANTHROPIC",
        "operation": "GENERATE",
        "route": "MESSAGES",
        "source": "UPSTREAM",
        "isUpstreamInvoked": true,
        "model": {
          "requested": "claude-sonnet",
          "effective": "claude-sonnet",
          "reported": "claude-sonnet-2026-08-15"
        },
        "stream": true,
        "usage": {
          "state": "COMPLETE",
          "inputTokens": "220",
          "outputTokens": "640",
          "totalTokens": "860"
        },
        "finishReason": "STOP",
        "rawFinishReason": "end_turn",
        "timeToFirstToken": {
          "milliseconds": 480
        },
        "eventCount": "37"
      }
    }
  }
}

An incomplete stream can have usage.state: PARTIAL and an unset finish reason. Its observed token counts are a floor rather than a final total.

Semantic Cache Hit

An exact or semantic hit has no provider usage and does not spend a token quota:

{
  "entry": {
    "info": {
      "llm": {
        "type": "COMPLETE",
        "source": "SEMANTIC_CACHE",
        "semanticCache": {
          "result": "SEMANTIC_HIT",
          "similarity": 0.96,
          "isStored": false,
          "plugin": "user-cache"
        }
      }
    }
  }
}

Cache results are EXACT_HIT, SEMANTIC_HIT, MISS, BYPASS or ERROR. ERROR means an embedding or vector-store failure; the request fails open to the provider. isStored shows whether the observed response was added to the cache.

Rejections

A token quota rejection identifies the plugin and scope that denied the request:

{
  "entry": {
    "info": {
      "llm": {
        "type": "COMPLETE",
        "source": "OCTELIUM",
        "tokenRateLimit": {
          "result": "DENIED",
          "plugin": "user-total-budget",
          "scope": "TOTAL"
        },
        "http": {
          "response": {
            "code": 429,
            "contentType": "application/json"
          }
        }
      }
    }
  }
}

Each applied guardrail produces an entry with PASS, MODIFIED, DENIED or ERROR. ERROR distinguishes an inability to reach a verdict from content that matched a deny pattern.

Metrics and Export

AccessLogs are exported through OpenTelemetry and can be sent to an observability or SIEM backend. Useful dimensions include Service, User, requested and effective model, protocol, operation, route, finish reason, cache result, semantic route, guardrail result and quota result. Only sum usage when measuring provider consumption; it contains authoritative reported values and excludes cache hits.

Read more about the LLM configuration and LLM plugins.