diff --git a/docs/roadmap/invocation-scoped-token-usage.md b/docs/roadmap/invocation-scoped-token-usage.md index 2d5b8d4..437bcde 100644 --- a/docs/roadmap/invocation-scoped-token-usage.md +++ b/docs/roadmap/invocation-scoped-token-usage.md @@ -144,6 +144,12 @@ override on that backend continues to select its OpenRouter response contract. Other built-in backend selections report unknown cost initially. Additional provider adapters or cost-capability configuration are not needed for this scope. +This baseline selects by exact backend identity: a separately named or +collection-qualified backend pointing to OpenRouter would report unknown cost. +Backend configuration as an alternative to identity-based selection is now an +open discussion; settle that selection before implementing provider cost support. +The contextual-observation stage does not depend on that decision. + Use the cost included in the normal completion response. No additional usage lookup, pricing request, or opt-in provider request parameter is needed. @@ -201,6 +207,19 @@ Completeness describes coverage of reported values, not verification of the provider's billing. Preserve reported usable amounts without pricing inference or reconciliation. +For a successful run, result cost and terminal-event cost describe the same +dispatched attempts and must agree on value, presence, and completeness. Result +cost must also be the same with observation enabled or disabled. For example, +an unreported failed attempt followed by a successful attempt reporting USD +0.002 yields `Value: 0.002`, `Known: true`, and `Complete: false`. Apply the same +accounting through repairs and ordinary or prepared execution. + +Group cost completeness concerns the terminal observations included in its +subtotal. If two completed runs reported all their costs but a third observed +run has no terminal event, the subtotal may have `Complete: true` while +`TerminalCoverageComplete` is false. Preserve this distinction using the existing +coverage field; no additional completion flag or group lifecycle is needed. + ## Accounting And Responsibility Boundaries Reuse existing events, token presence/completeness, and group summaries, extending @@ -230,6 +249,10 @@ logging infrastructure are outside this scope. conversions, successful run results, attempt and terminal events, and existing group summaries. Keep result cost available without registering an observer. Preserve existing token fields and accounting. +- Verify that successful-result cost agrees with terminal-event cost and is + unchanged when observation is disabled, including the unreported-retry example + above. Verify group snapshots with fully reported completed runs alongside a + member without a terminal observation. - Cover ordinary and prepared execution, coordinated continuations, nested calls using derived contexts, concurrent calls, multiple engines sharing a scope, and independent scopes on a reused engine. @@ -250,5 +273,33 @@ logging infrastructure are outside this scope. the [integration contract](../integrations/openai-compatible-chat.md#response-handling), and update internal observation and model-client documentation for changed flows. -This roadmap defines the target behavior. A separate implementation plan can -divide the work into appropriately sized stages. +## Implementation Planning Guidance + +Plan two separately reviewable stages: + +1. Contextual observer selection, including ordinary, prepared, coordinated, + and nested execution boundaries and the existing group collector. +2. Provider-reported cost, carried through decoding, accounting, public results, + events, and group summaries, with the corresponding documentation. + +The second stage is more extensive than the context helper. Resolve the backend +selection discussion before finalizing that stage; avoid coupling delivery of +contextual observation to provider cost support. + +Today, successful results accumulate generation responses in the runner, while +optional observation recorders account for dispatched attempts. Adding cost to +response objects alone does not preserve completeness across unreported retries. +Reuse a small accounting component for result and observation paths so they +apply the same rules, including when no observer is selected. Preserve existing +token accounting rather than expanding this work into a redesign of its public +contract. + +Keep tolerant optional-cost decoding local to the model-client boundary. Adding +a plain `*float64` JSON field to the typed envelope would make a nonnumeric or +overflowing value fail decoding of the entire response. Decode the optional +field separately so an unusable cost becomes unknown without changing existing +framing or content validation. This needs no general numeric-conversion layer. + +Use the completion criteria above to select focused offline checks for each +stage. This roadmap owns target behavior; a separate implementation plan owns +the detailed work sequence.