Session Keeping

Group requests from one client session for sticky load balancing and threaded audit logs

Coding agents and chat apps send many requests that belong to one logical session — one conversation, one agent task. GoModel detects that session and uses it throughout routing and observability:

• Sticky load balancing — a load-balanced virtual model routes every request of a session to the target that served it first, which keeps provider prompt caches warm and model behavior consistent mid-conversation.
• Sticky API-key selection — when one provider has several keys, every request in a detected session uses the same key, preserving provider prompt-cache affinity.
• Threaded audit logs — the dashboard’s Audit Logs page groups a session’s requests into one thread: the latest request as the row, with an expander that unfolds the older requests beneath it.
• Per-session usage and cost — usage records carry the same scoped session id, so the dashboard and admin API can report the requests, tokens, and provider spend for one conversation or agent task.

Session keeping works with zero configuration. Defaults fit most setups; the options below exist for the rare cases they don’t.

How a session is identified

The first matching signal wins:

1. Session headers — a built-in registry covers the headers known tools send, plus gateway conventions:

   Header	Sent by
   X-Session-Id	OpenCode, Kilo Code, generic clients
   X-Claude-Code-Session-Id	Claude Code (documented for gateways)
   Session-Id / Session_id	Codex CLI (current / older), Roo Code
   X-Litellm-Session-Id	LiteLLM convention
   Helicone-Session-Id	Helicone convention
   Agent-Session-Id	Goose

2. Body fields — for clients that mark sessions in the request body: Anthropic metadata.user_id (Claude Code embeds its session UUID there, both current and legacy formats are parsed), session_id (OpenRouter convention), litellm_session_id, prompt_cache_key (Zed and other OpenAI clients that reuse a cache key per conversation), and /v1/responses conversation references.

3. Automatic detection — chat and responses requests with no explicit signal are grouped by their conversation opening: the model, system context, tools, and the messages through the first user turn. Follow-up requests resend that prefix unchanged, so an agent that simply replays its history (Aider, Cline, Continue, …) still gets a stable session id (auto-…) with no client changes.

Body-based signals and automatic detection read request bodies up to 1 MiB (chunked requests included); larger bodies fall back to header signals.

Client-provided session ids are scoped by user path, including UUID-shaped values, so one tenant cannot collide with another by reusing the same id.

Sticky load balancing

When a request carries a session and resolves through a virtual model with several targets, the first request picks a target via the redirect’s strategy (round_robin or cost) and later requests stick to it. If the pinned target becomes unavailable or saturated, the strategy picks again and the session re-pins — a session is never glued to a dead target, and the all-saturated honest-429 behavior is unchanged.

Affinity is on by default and can be disabled per redirect:

virtual_models:
  - source: "smart"
    strategy: round_robin
    session_affinity: false   # default: true
    targets:
      - model: "openai/gpt-4o"
      - model: "anthropic/claude-sonnet-5"

The same flag is available in the dashboard’s virtual model editor (“Session keeping”) and on the admin API (session_affinity on PUT /admin/virtual-models).

Pins are in-memory per instance (like rate-limit counters): after a restart or on another replica, the next request of a session simply re-pins. Idle sessions expire after 6 hours.

API-key affinity is deterministic rather than stored, so the same ordered key set selects the same key across replicas and restarts. Disable it per provider with session_sticky_keys: false, <PROVIDER>[_SUFFIX]_SESSION_STICKY_KEYS=false, or the dashboard’s Session-sticky API keys checkbox. Requests with no detected session remain round robin.

Threaded audit logs

Audit entries record the session id (session_id), and the Audit Logs page groups them by default (“Group by session” toggle). Each thread shows its latest request with a count badge; the expander on the left unfolds the older requests. The badge counts the complete session. Filters select which sessions appear and which matching request represents each thread. GET /admin/audit/sessions returns each thread’s request_count and latest matching audit-log entry. GET /admin/audit/log?session_id=… returns all requests in an expanded session.

Each grouped thread also has a usage-chart action. It opens Usage Analytics with that session selected; expanding an individual request exposes the same action on its session badge.

Per-session usage and cost

Usage Analytics includes a full-width Usage by Recent Session table. Each row represents one detected, user-path-scoped session and shows request count, input/output/total tokens, and input/output/total cost. Session chips in that table and the request log filter the whole usage page.

The admin API exposes the same bounded report:

curl -H "Authorization: Bearer $GOMODEL_MASTER_KEY" \
  "http://localhost:8080/admin/usage/sessions?days=7&limit=50&offset=0"

{
  "entries": [
    {
      "session_id": "scoped-0123456789abcdef0123456789abcdef",
      "user_path": "/team/alpha",
      "requests": 8,
      "input_tokens": 24000,
      "output_tokens": 4100,
      "total_tokens": 28100,
      "input_cost": 0.42,
      "output_cost": 0.16,
      "total_cost": 0.58
    }
  ],
  "total": 1,
  "limit": 50,
  "offset": 0
}

Requests and tokens include responses served by GoModel’s local response cache. Cost fields include provider-bound requests only, because cost stored on a local-cache hit is the cost that was avoided, not money actually spent. Results are ordered by latest session activity; limit defaults to 50 and is capped at 200. The normal usage filters (session_id, user_path, model, provider, label, and date range) apply.

Configuration

Everything is on by default.

# Master switch for session identification
SESSION_KEEPING_ENABLED=true
# Content-based detection for requests with no explicit signal
SESSION_AUTO_DETECT=true
# The built-in header/body-field registry
SESSION_BUILTIN_RULES=true
# Extra session headers (numbered from 1), merged over the built-ins.
# The optional transform "session-uuid" extracts a session_<uuid> value.
SESSION_HEADER_1=X-My-Session
SESSION_HEADER_1_TRANSFORM=

Or in config.yaml:

session:
  enabled: true
  auto_detect: true
  builtin_rules: true
  headers:
    - header: X-My-Session

Credential-bearing headers (Authorization, API-key headers, …) are rejected as session sources, since the session id is persisted on audit and usage records.

When to change the defaults: set SESSION_AUTO_DETECT=false if you only want explicitly-tagged sessions grouped; set session_affinity: false on a redirect when you prefer strict load spreading over cache affinity (for example pure round-robin capacity balancing across identical deployments).