The Life of a Call

Entry point: somm.llm(project=...) and the workload contract

Policy first: privacy class, daily budget, and capability requirements

Workloads declared with SommLLM.register_workload(...) carry privacy, budget, and capability requirements (docs/errors/SOMM_WORKLOAD_UNREGISTERED.md:19, docs/BLUEPRINT.md:118). Before any provider is contacted, generate() merges the workload's declared capabilities with capabilities inferred from the request itself, then runs ordered, mutable pre_call hooks registered through hooks.register_hook(...) (packages/somm/src/somm/client.py:910, packages/somm/src/somm/hooks.py:160).

Routing: filter incapable models, pace quotas, cool transient failures

With policy satisfied, the call dispatches either to a pinned provider or to the preference-ordered router (packages/somm/src/somm/client.py:910). Pins are "try first" by default — if the pinned route fails, the router can still fall back — while no_fallback=True supplies pinned-or-bust semantics for experiments where rerouting would invalidate results (packages/somm/src/somm/client.py:1133).

flowchart TD
    A["SommLLM.generate()"] --> B["Resolve workload policy
(auto-register in observe mode)"]
    B --> C["Merge declared + inferred capabilities"]
    C --> D["Run mutable pre_call hooks
(fail open)"]
    D --> E{"Daily budget?"}
    E -- "exceeded" --> F["SOMM_BUDGET_EXCEEDED
fatal, no dispatch"]
    E -- "ok" --> G{"Provider pin?"}
    G -- "pinned" --> H["Try pin first"]
    G -- "unpinned" --> I["Preference-ordered router"]
    I --> J["Filter explicitly incapable
provider/model pairs"]
    J --> K["Apply plan governor:
defer over-paced, drop blocked"]
    K --> L["Skip cooled providers"]
    L --> M["Dispatch"]
    H --> M
    M -- "transient failure" --> N["Cool provider,
fall through to next"]
    N --> M
    M -- "success" --> O["Normalize result +
immutable Call record"]
    O --> P["Batched telemetry write"]
    P --> Q["post_call hooks (inline)
post_process (background)"]

The router applies three filters before any network access (packages/somm/src/somm/routing.py:158):

• Capability. Provider/model pairs that are explicitly incapable of the request's requirements are removed; pairs with unknown capability values remain eligible, so incomplete metadata never silently shrinks the candidate pool (docs/multimodal.md:75).
• Quota pacing. Provider plans classify usage as ok, over_pace, or exhausted; the plan governor keeps normal providers first, defers over-paced ones, and removes blocked ones (tests/test_plans.py:262). A broken or absent governor fails open to the original provider chain, preserving availability (tests/test_plans.py:272). Quota ceilings can even be inferred from observed 429 events in past telemetry (packages/somm-core/src/somm_core/plans.py:627).
• Cooldowns. Providers recently seen failing transiently are skipped. Cooldown state lives in SQLite — with separate model-level and provider-wide entries — so it survives process restarts (docs/BLUEPRINT.md:187).

Recording the call: tokens, cost, latency, outcome, provenance

Cost is computed locally: cost_for_call() converts token counts into per-call USD using pricing seeded from an offline bundled snapshot and cached for ten minutes (packages/somm-core/src/somm_core/pricing.py:233, packages/somm-core/src/somm_core/pricing.py:108). Missing pricing fails open to zero cost with a once-per-provider/model warning for paid providers — visible, but never blocking a call (packages/somm-core/src/somm_core/pricing.py:240, tests/test_pricing_safety.py:87).

What the record does not contain matters just as much. Prompt and response bodies are stored only through opt-in sampling (packages/somm-core/src/somm_core/repository.py:642); shadow body capture is opt-in, deterministically sampled, size-capped, and forbidden for private workloads (packages/somm/src/somm/client.py:850). The full privacy story is on The Data Substrate.

When things break: transient fallthrough vs. fatal SOMM_* errors

somm splits failure into two regimes. Transient failures — timeouts, rate limits, 5xx responses, insufficient credit — cool the offending provider and fall through to the next candidate. Fatal policy or configuration failures stop immediately, and each has a canonical SOMM_* page documenting the problem, cause, behavior, and fix (docs/errors/index.html:28). Budget exhaustion and privacy violations are on the fatal side precisely so that fallback can never launder a policy refusal into a successful call.

Failures are deliberately visible and bounded in telemetry rather than silently swallowed. — somm design decisions, README.md:162

Failed calls are still calls: they land in the same ledger with their error details, and somm calls --status error surfaces them from the CLI — it's the route bug reports are asked to include (.github/ISSUE_TEMPLATE/bug_report.md:28). Integrations follow the same philosophy: the LangChain adapter raises failures by default so retry and circuit-breaker middleware can react, with an opt-in mode that returns an empty message carrying failure metadata instead (packages/somm-langchain/src/somm_langchain/chat_model.py:63).

The write path: batched telemetry with JSONL spill for resilience

Telemetry never blocks the hot path. A per-process writer queue batches short writes into the project-local SQLite database — .somm/calls.sqlite (README.md:67) — keeping the no-service-required call path fast (packages/somm/src/somm/telemetry.py:38, notes/PLAN.md:1179). If database writes fail, records spill to permission-restricted JSONL files for later atomic replay, so telemetry degrades to disk rather than being lost (packages/somm/src/somm/telemetry.py:270).

The database underneath is built for this pattern: constructing a Repository creates a permission-restricted SQLite file, applies packaged migrations automatically, and configures WAL, foreign keys, and per-thread, fork-aware connections (packages/somm-core/src/somm_core/repository.py:177). Migrations commit DDL and version stamps together, so a database is never left partially upgraded (packages/somm-core/src/somm_core/schema.py:91).

The whole round trip is cheap enough to gate in CI:

The performance check runs the warmed hot path against a fake provider and a temporary repository, isolating somm's own overhead from network and provider latency; only median latency gates CI, reducing sensitivity to noisy outliers (scripts/check_perf_budget.py:69, scripts/check_perf_budget.py:100). Schema v19 is where the original workload/prompt/call ledger has evolved to (packages/somm-core/src/somm_core/version.py:7).

Once the row is written, this call's life is over — but its afterlife is the point. Every downstream service consumes calls without rewriting them: evaluation, model intelligence, and recommendations all read the same ledger (docs/BLUEPRINT.md:171).