0008 virtual models

ADR-0008: Virtual Models (unify aliases and access overrides)

Context

GoModel exposes two operator-defined ways to shape model routing:

• Aliases map a new, addressable name to one real model (fast -> openai/gpt-4o). They are resolved early, as a rewrite.
• Access overrides gate an existing, scoped selector (/, provider/, model, provider/model) by user_paths. They are enforced late, as an authorization decision on the already-resolved model.

These are stored in separate tables, served by separate services, and surfaced by separate admin endpoints, yet they are the same operator concept: a model the operator defines rather than one a provider advertises. The split duplicated user_path scoping and the database-migration handling, which caused real bugs (an alias user_paths feature that re-implemented matching the overrides already had, and shipped without a migration, breaking existing databases).

We also want load balancing — one name resolving to several real models, chosen per request — and there is no home for it today.

Decision

Introduce one entity, the virtual model, persisted in virtual_models and keyed uniquely by source.

• A row with targets is a redirect: source is a new name that rewrites to a real model. One target is an alias; many targets are load balancing, distributed by strategy (round_robin, honoring per-target weight, or cost). This was implemented as the additive follow-up the staging enabled — the targets, strategy, and weight columns were already persisted.
• A row without targets is an access policy: source is a scoped selector over existing models, gated by user_paths.

Behavior is derived from the presence of targets — there is no role column. Storage, the service object, the admin API, and the dashboard are unified, but resolution stays staged: redirect runs early, the access gate runs late, exactly as before.

Pricing overrides remain a separate subsystem.

Version 1 preserves today’s behavior exactly. The fields that enable load balancing (targets beyond one, strategy, per-target weight) and scoped redirects (user_paths on a redirect row) are stored but inert; they are turned on by later changes that need no migration.

Resolution Rule

1. Resolver (early). If the requested model exactly matches a redirect row’s source, rewrite it to that row’s single target.
2. Authorizer (late). Scope-match the resolved selector against the policy rows and enforce user_paths.

Redirect and policy rows never cross stages: a redirect source is a new name that does not scope-match a real model, and a policy row has no target, so the resolver ignores it.

Migration

A one-time, idempotent seed copied existing aliases rows (as redirects) and model_overrides rows (as policies) into virtual_models on first start when the table was empty. The seed shipped in v0.1.44 and was removed in v0.1.81; upgrading from a release older than v0.1.44 must pass through a version in that range so the seed runs once. The legacy aliases and model_overrides tables/collections are never read or written anymore and are left in place; operators may drop them.

Consequences

Positive

• One user_path scope, one migration path, one admin surface, one UI.
• Load balancing becomes an additive change (data + a picker), not a third subsystem.
• Less duplicated code; the class of bug from divergent re-implementations is removed.

Negative

• One table feeds two pipeline stages, mitigated by two independent in-memory indexes and by porting the existing, tested matching logic verbatim.
• source is a single namespace, so a redirect and a policy cannot share a name. This is structurally rare (aliases already forbid masking real models) and is accepted.
• Rollback is lossless only before the first virtual-model edit, because new writes go only to virtual_models.

Update — single native engine, authoritative Enabled, scoped redirects, unified UI

A follow-up change completed the unification the first version staged:

• One native engine. The composition over the legacy aliases and modeloverrides services was replaced by native redirect + policy matching inside virtualmodels, operating directly on VirtualModel rows behind a single in-memory snapshot. The internal/aliases and internal/modeloverrides packages were removed; their tested matching logic was ported.
• Enabled is authoritative. A policy row’s Enabled now governs access: a disabled policy turns its selector off for everyone, an enabled policy with user_paths restricts, and a selector with no row follows MODELS_ENABLED_BY_DEFAULT. This makes “disable a single model” expressible for the first time and lets the dashboard toggle any model on/off.
• Scoped redirects are enforced. user_paths on a redirect row are no longer inert: resolution consults the effective request user_path via the optional gateway.UserPathModelResolver (ResolveModelForUserPath). A redirect applies only for matching callers and falls through to the literal model name otherwise (the use case from the closed upstream PR #387). Exposure at /v1/models remains unscoped for redirects.
• One admin surface and UI. A single GET/PUT/DELETE /admin/virtual-models endpoint replaces /admin/aliases and /admin/model-overrides, and the dashboard collapses the separate alias and access-override modals into one virtual-model editor (Source — locked when editing an existing model — an always-present target field, user_paths, enabled, description) plus a per-row enable/disable toggle and alias-like styling for any model that carries a virtual model.