eric/notarius
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Plan the policy alignment work

Browse Source
This commit is contained in:
Eric Rakestraw
2026-09-28 01:13:52 +00:00
parent ff9b05077c
commit f6d7c7cef4
2 changed files with 258 additions and 360 deletions
Download Patch File Download Diff File Expand all files Collapse all files

603
docs/roadmap/implementation.md
View File

@@ -1,403 +1,310 @@
# ConfigKit And PromptKit Upgrade Implementation Plan
# Policy Alignment Implementation Plan
## Purpose And Scope
## Purpose And Execution Contract
Implement [ConfigKit And PromptKit Upgrades](upgrade-kit.md). That document
owns the desired behavior and operator migration; this document owns execution
order. Each numbered stage is one bounded implementation prompt for a
gpt-5.6-terra coding agent. Complete and validate stages in order.
Implement [Standard Repository Policy Alignment](policy.md). That feature
roadmap owns the target policy choices and justified Notarius exceptions; this
document owns implementation order. It replaces the completed kit-upgrade plan,
not the implementation or contracts delivered by that earlier work.
The inspected baseline pins ConfigKit v0.3.0 and PromptKit v0.10.0. Adopt
exactly ConfigKit v0.4.0 and PromptKit v0.11.0, not whatever release happens to
be newest during implementation. The release guides and public APIs are
available in the sibling checkouts `../ConfigKit` and `../promptkit`; consult
the target tags or downloaded modules if those checkouts have advanced.
Build and test Notarius with `GOWORK=off`, without local dependency replacements.
Each numbered stage is one bounded prompt for a gpt-5.6-terra coding agent.
Complete stages in order. Inspect the worktree before each stage, preserve
unrelated work, validate the resulting documentation, and commit the coherent
stage under agent policy. Report changed files, checks, and any concrete blocker.
Do not push, tag, publish, or retire unrelated roadmaps.
Read `docs/development.md` and all five policies in `docs/policy/` before
implementation. In particular, simplify machinery without narrowing operator
choices or elevating incidental details into requirements. Preserve existing
contracts unless this roadmap expressly changes them. Use graph discovery for
code, then inspect the relevant source and tests; locations below identify the
reviewed baseline, not a requirement to preserve private file structure.
This is a documentation/policy migration. Do not change application code,
dependency pins, tests, release scripts, runtime contracts, or release gates.
No production credentials or model calls are needed.
## Decisions And Boundaries
## Fixed Sources And Adoption Rules
- Configuration selection becomes `--config-overlay` and `overlays/<name>.yml`,
with no legacy alias or fallback. Keep empty, omitted, and `default` selection
behavior. Do not rename PromptKit profiles or pipeline/runtime profile types.
- Keep configuration version 5, source-file-relative declarations, CLI override
precedence, strict loading, inspection provenance, and redaction.
- Add only the global `promptkit.transient_retries` setting: nonnegative integer,
default **2 additional attempts**, explicit zero disables. No arbitrary cap,
per-binding setting, new CLI retry flag, or new total-deadline setting.
- PromptKit owns transient HTTP retries and structural repairs; Notarius keeps
stage/validator retry budgets and disposition unchanged. The existing shared
scheduler retains its completion-scoped permit through upstream waiting.
- Add provider-neutral failure metadata and request-local execution summaries.
Do not change domain artifacts, result receipts, prompt content, validators,
acceptance policy, or introduce a new telemetry service/artifact.
- Successful retry recovery is ordinary success, not a warning. Debug remains
opt-in. Existing token fields keep their meaning; observed usage is separate.
- No release, tag, push, source-license change, or binary packaging is part of
these stages. Record upgrade implications in current integration/operator
documentation; the eventual release note is a separate release task.
Use upstream full commit
`d84a38ebab26f048cafbe59a2246700c35d29843` throughout.
Begin with its
[adoption guide](https://gitea.maximumdirect.net/eric/policy/src/commit/d84a38ebab26f048cafbe59a2246700c35d29843/docs/adoption.md).
Read applicable local policies through `docs/development.md`; when revising a
policy, also read the upstream counterpart and affected cross-policy provisions.
Do not reconstruct historical adoption baselines.
For each stage, inspect the starting worktree, implement code with its focused
tests and current-behavior documentation, run the listed relevant checks, and
make one coherent local commit under agent policy. Report checks and any gaps.
Do not repeat passed checks for unchanged content merely to make a commit.
Retrieve pinned files through public URLs or `git show` against that exact
commit in an available checkout. Do not use an unverified sibling working tree
or silently select newer upstream content. An unavailable source is a concrete
blocker to copying it accurately, not a reason to invent its text.
## Stage 1 — Cut Over Configuration Selection To Overlays
For every adopted policy:
**Status:** Completed 2026-09-25.
- Copy upstream wording verbatim except for required template removal and the
agreed, substantive project-specific customizations in the feature roadmap.
- Retain its upstream Base Policy Version; set Last Updated to the actual date
the local document is adopted, not the review date by habit.
- Add Upstream Source pointing to its own file at the full selected commit:
`https://gitea.maximumdirect.net/eric/policy/src/commit/d84a38ebab26f048cafbe59a2246700c35d29843/docs/policy/FILE.md`.
- Remove the entire Base Policy Template section and perform its instructions.
Keep upstream generic rules outside it, including commit-pinned example links.
- Explain substantive local exceptions briefly where they apply, or retain an
existing rationale/ADR link. Do not create an exception registry or preserve
stylistic paraphrases merely because they already exist.
**Goal:** Adopt ConfigKit v0.4.0 and finish its narrow operator-facing rename in
one coherent change. This is one prompt: it is a naming migration, not a new
configuration engine.
Current public project and adoption-guide links are discovery links; they may
follow main. Source, example, and attribution links stay commit-pinned. Preserve
accurate preexisting example pins supplied by the selected OSS template.
Metadata must reflect only documents actually adopted by the current stage.
The transition may temporarily contain mixed versions without claiming otherwise.
**Inspect:** `internal/core/config/load.go`, `loaded_config.go`, configuration
document/path/operational tests; `internal/cli/run.go`,
`configkit_commands.go`, CLI configuration and example contract tests;
`examples/profiles/development.yml`; upstream v0.4.0 migration guide.
## Stage 1 — Establish Provenance And Copied-Policy Licensing
**Work:**
**Goal:** Establish the self-contained adoption and licensing foundation.
One prompt limited to two short policies and their actual notice materials.
1. Upgrade ConfigKit to v0.4.0 in `go.mod`/`go.sum`. Change `LoadOptions.Profile`
to `Overlay` and pass it into `configkit.Options[Document].Overlay`. Rename
only helpers, fields, locals, fixtures, and descriptions representing the
selected configuration layer.
2. Replace `--config-profile` with `--config-overlay` for `run`, `config validate`,
`config effective`, `config explain`, and `pipelines list`. Update usage,
flag parsing, argument classification, explicit-empty handling, and error
paths together. Retired flags use normal unknown-flag behavior.
3. Move `examples/profiles/development.yml` to
`examples/overlays/development.yml`; update maintained-example discovery and
selected-layer fixtures. Check file-relative paths after moving. Leave
`examples/llm-profiles/`, PromptKit profile sources, and ordinary explicit
imports alone unless an import actually points at the moved example.
4. Consume upstream overlay provenance and diagnostic names directly. Do not
rebuild composition, implement directory fallback, or translate metadata
back to profile vocabulary.
5. Update `docs/config.md`, `docs/cli.md`, `docs/integrations/pkg-configkit.md`,
`docs/internal/configuration.md`, relevant portions of `docs/internal/cli.md`,
and `docs/consumers/subprocess.md`. Review README, operations, and other
consumer examples for affected invocations. Explain the clean migration and
distinction from LLM profiles. Preserve accepted ADR-0018 and release history.
**Read:** Selected upstream `docs/policy/meta.md` (1.0),
`docs/policy/oss.md` (1.8), `COPYRIGHT`,
`examples/licensing/policy-attribution.md`, and the adoption guide;
local `COPYRIGHT`, `LICENSE`, and the feature roadmap.
**Acceptance:** All five commands select overlays consistently. Representative
existing tests cover selected/base/empty/default behavior, overlay and explicit
override precedence, declaration-relative paths, redacted effective/explain
output, and source-aware failure labels. Include a selected legacy-only file
case proving there is no silent fallback. Retain ordinary-import behavior and
LLM profile coverage; do not duplicate every upstream parser case.
**Files:** Create `docs/policy/meta.md`, `docs/policy/oss.md`,
`docs/oss/attribution.md`, and
`docs/oss/licenses/CC-BY-SA-4.0.txt`; update `COPYRIGHT`.
**Validate:** `GOWORK=off go test ./internal/core/config ./internal/cli`,
`GOWORK=off go mod tidy -diff`, `git diff --check`. Review occurrences of
`config-profile` and `examples/profiles` in current docs/code semantically;
historical docs and explicit legacy-rejection tests are not defects.
1. Adopt meta and OSS using the fixed-source rules. Keep public discovery URLs
in meta and link to the existing local developer guide and new attribution.
Point project-specific OSS context to Notarius's grant and release runbook;
do not import generic example projects.
2. Create attribution with `# Open Source Attribution Notices` followed by
the canonical policy entry. Replace all three UPSTREAM_COMMIT placeholders.
Initially identify the two newly adopted policy files as adapted material.
Later stages expand this statement as their material is actually adopted;
do not imply that the older five files have already been refreshed.
3. Copy the complete CC BY-SA 4.0 standard text from the pinned reference asset.
Preserve the canonical copyright/SPDX/notices/source arrangement and its
modification indication. Do not add research or compliance prose.
4. Add a distinct scope statement to COPYRIGHT: policy material adapted from
Standard Repository Policies remains CC-BY-SA-4.0, with attribution defining
the affected material. Keep the existing GPL-3.0-or-later application grant,
warranty disclaimer, and third-party statement; leave root LICENSE unchanged.
5. Do not create placeholder dependency entries. Dependency inventory is assigned
to Stages 4–5, and remains tracked here rather than asserted complete.
## Stage 2 — Upgrade The PromptKit Runtime Baseline
**Acceptance:** Both policies have correct versions, date, file-specific source
links, and no template-only section. All new relative links resolve. Attribution
describes this stage's actual material, and code licensing is unchanged.
**Status:** Completed 2026-09-25.
**Validation:** Compare copied policy bodies and license text to the pinned
sources; inspect deliberate substitutions. Run `git diff --check` and review
the COPYRIGHT diff. No Go tests are needed.
**Goal:** Adopt the compatible upstream dependency and document its changed
default before adding Notarius controls. One small prompt.
## Stage 2 — Standardize General Policies And Selective Orientation
**Inspect:** `internal/framework/llm/promptkit_client.go`, asset/profile identity
helpers and tests, `go.mod`, and upstream v0.11.0 release/consumer documentation.
**Goal:** Adopt the general rules and their reading workflow. One prompt:
three largely verbatim policy updates plus small orientation edits.
**Work:**
**Read:** Local agent/testing/documentation policies; pinned upstream versions
1.3, 1.3, and 1.11 respectively; upstream `agents/AGENTS.md` and the adoption
guide's Policy Reading Guide; local `AGENTS.md` and `docs/development.md`.
1. Pin PromptKit v0.11.0 and tidy. Keep the Go version and provider catalog
selections unless the actual dependency graph requires a change; the target
release does not require either. Do not alter embedded D&D model settings.
2. Update `promptKitBuiltinProfileCatalogID` to the v0.11.0 identity in accordance
with the existing checkpoint fingerprint convention. No other cache policy
change or blanket invalidation is part of this work.
3. Update the pinned integration links in `docs/integrations/pkg-promptkit.md`,
including the relocated upstream `docs/policy/release.md`. Record the
LGPL-3.0-or-later dependency license and reference its license/notices; do not
relicense Notarius or invent a new distribution workflow.
4. Remove the obsolete no-provider-retry-loop statement in `docs/internal/llm.md`.
Describe the now-active upstream default and per-attempt timeouts in the
appropriate integration/operations sections. Do not yet describe the setting
or observations from later stages as implemented.
**Files:** Update `docs/policy/agents.md`, `testing.md`,
`documentation.md`, root `AGENTS.md`, `docs/development.md`, and
`docs/oss/attribution.md`.
**Acceptance:** The prepared-execution adapter builds and its existing tests
pass without prompt/catalog changes. Current documentation acknowledges the
dependency's default retry behavior and license accurately.
1. Adopt agent and testing substantive wording without local paraphrases.
Remove the redundant agent preamble, additional generic contract-preservation
prose, and ADR-0017 illustration. Its decision remains in architecture/ADR;
this is not permission to change it.
2. Adopt documentation wording and ownership rows. Customize the consumer row
for existing subprocess guidance, not a hypothetical public Go API.
Omit the inapplicable HTTP contract row; preserve applicable CLI/config/
operations/integration/internal ownership. Remove the local duplicate
implementation-details-versus-policy section.
3. Base root AGENTS.md on upstream `agents/AGENTS.md`, not upstream root
AGENTS.md. Preserve applicable existing instructions. Require orientation
through the developer guide and agent policy, but not full-suite reading.
4. Add the adoption guide's seven-row selective Policy Reading Guide and its
selection/reuse/scope-expansion rules to docs/development.md. Retain existing
subsystem navigation and validation guidance; remove redundant competing
policy-trigger rows from the old table rather than creating two authorities.
5. Expand the attribution material statement to identify the newly refreshed
policy files, adapted AGENTS.md, and copied policy-reading-guide portion of
docs/development.md. Do not license the unrelated remainder as copied policy.
Ensure the COPYRIGHT scope statement covers these identified portions.
**Validate:** `GOWORK=off go test ./internal/framework/llm ./internal/cli`,
`GOWORK=off go mod tidy -diff`, `git diff --check`.
**Acceptance:** General rules match upstream, except required documentation
ownership customization. All seven policy triggers are available; existing
subsystem links still work. Reading is selective and public discovery is reachable.
No test changes or stricter testing/commit requirements were introduced.
## Stage 3 — Wire The Explicit Transient Retry Setting
**Validation:** Diff each policy against its pinned source, inspect AGENTS and
developer-guide navigation, check attribution scope and relative links, and run
`git diff --check`. Search affected files for leftover template instructions.
**Status:** Completed 2026-09-25.
## Stage 3 — Align Architecture And Release Policies Without Contract Changes
**Goal:** Make retry policy inspectable and configurable through the one shared
engine. One prompt covering a single vertical configuration slice.
**Goal:** Complete the customized policies and release navigation. One prompt
focused on preserving existing decisions rather than redesigning them.
**Inspect:** `internal/core/config/{document,config,load,operational_document,
loaded_config,validation}.go`, effective-config tests;
`internal/cli/catalog.go:buildProductionLLMClient`;
`internal/framework/llm/promptkit_client.go:NewPromptKitClient` and HTTP tests.
**Read:** Local architecture and release policies, docs/release.md, relevant
linked ADRs; pinned upstream architecture 1.3 and release 1.4. Use
docs/internal/overview.md and existing canonical contracts only where needed to
resolve an ownership question.
**Work:**
**Files:** Update `docs/policy/architecture.md`, `docs/policy/release.md`,
`docs/release.md`, and `docs/oss/attribution.md`.
1. Add `TransientRetries *int` to `PromptKitDocument` and a resolved integer to
`PromptKitConfig`. Supply 2 through both the document default path and the
runtime `Default()` path. Preserve explicit zero; never use zero as omission.
Include it in document-to-runtime conversion and effective configuration.
2. Reject negative values through both existing document and direct runtime
validation entry points. Follow current source-aware diagnostics; do not
impose a new maximum. Keep ConfigKit's existing decoding behavior.
3. Add `TransientRetries *int` to `PromptKitClientConfig`: nil follows upstream
default, non-nil includes explicit zero. Production construction always
supplies the resolved configuration value. Pass it into `promptkit.Config`,
not per-request options. All calls through the injected shared client thus
share the setting without module changes.
4. Add the setting to `examples/dnd-complete.config.yml` and `docs/config.md`;
keep the minimal example minimal. Update configuration/LLM internal guides
for the new plumbing and operations for the zero-value opt-out.
1. Adopt upstream general architecture ownership/approach guidance. Replace its
application-specific template guidance with Notarius's existing decisions.
Preserve the feature roadmap's enumerated pipeline, module, assets, domain,
identity, validation/retry, scheduling, configuration/provenance, state,
secret-handling, permissions, platform, and distribution contracts.
Keep existing application-specific sections unless an edit is necessary for
the adopted structure; do not turn this into a broad architecture rewrite.
2. Adopt release wording with narrow project customizations: source only,
lightweight tags, main/origin, branch-before-tag, no prereleases, and no
hosted/asynchronous deliverables. Keep the native CLI/version check and
GOWORK=off rationale. Preserve the historical no-retrospective-notes decision.
Retain unchanged generic requirements verbatim.
3. Add the OSS footer rule for new notes. Update the runbook with a copyable
three-link COPYRIGHT | LICENSE | ATTRIBUTION footer using absolute
`https://gitea.maximumdirect.net/eric/notarius/src/tag/<version>/...` paths.
Explain substitution of the selected release tag and local path checking.
Do not require a future tag to resolve or change publication scripts.
4. Expand policy attribution to the complete adopted docs/policy set, retaining
the AGENTS/developer-guide scope from Stage 2 and any relevant adaptation
indications. All seven per-document source links now identify this adoption.
**Acceptance/tests:** Demonstrate omitted default, explicit zero, a positive
override, negative rejection, and effective/explain inspection. Use an existing
CLI factory test to prove the resolved value reaches construction. Exercise
built-in generation with a local HTTP server: eligible failure then success,
disabled retries, exhausted configured count, and a noneligible 400 response.
An injected `LLMClient` stub does not test built-in HTTP retry wiring. Avoid
testing the upstream jitter formula or every equivalent HTTP status.
**Acceptance:** Existing architecture and release choices are unchanged and their
local reasons remain clear. Upstream generic wording is used where applicable.
There are no new gates, platforms, artifacts, retrospective notes, or ADR edits.
**Validate:** `GOWORK=off go test ./internal/core/config ./internal/cli ./internal/framework/llm`
and `git diff --check`.
**Validation:** Review policy diffs specifically for added/removed requirements,
check runbook agreement and affected links, and run `git diff --check`.
Do not run a release/publication command. No runtime tests are warranted.
## Stage 4 — Preserve Typed Failure And Exhaustion Metadata
## Stage 4 — Attribute The Kit Dependencies And Catalogs
**Status:** Completed 2026-09-25.
**Goal:** Add the first bounded group of component notices: ConfigKit,
PromptKit, and their two backend catalogs. One prompt for four related projects.
**Goal:** Distinguish provider failures without coupling callers to PromptKit.
One prompt; defer execution observations to the next stage.
**Read:** Adopted OSS policy; go.mod/go.sum, resolved build dependency metadata;
exact selected module COPYRIGHT/LICENSE/NOTICE/attribution files and VCS metadata.
The reviewed pins are ConfigKit v0.4.1, PromptKit v0.11.1, and both catalog
modules v2.20260926.0; verify the actual selected versions before writing.
**Inspect:** `internal/framework/contracts/errors.go`, adapter error conversion
in `promptkit_client.go`, redaction helpers, and existing pipeline handling of
`ErrLLMGeneration`, `ErrLLMCapacityExceeded`, and invalid structured output.
**Files:** Update `docs/oss/attribution.md`; add applicable texts under
`docs/oss/licenses/`.
**Work:**
1. Use read-only discovery such as `GOWORK=off go list -m -json all`,
`GOWORK=off go list -deps ./cmd/notarius`, and selected module contents.
Dependency acquisition may populate the module cache but must not change
dependency pins or introduce a local replace directive.
2. Establish each selected version's exact source commit from module/VCS
metadata; peel annotated tags to commits. Do not substitute a tag object ID,
main-branch head, assumed sibling HEAD, or version tag in attribution URLs.
3. Add factual House Format entries with supplied copyright statements,
applicable SPDX expressions, local standard-text links, required upstream
notice links, and exact-commit Source links. Preserve alternatives and
exceptions; determine scope from contents rather than filenames.
4. Include LGPLv3 and its GPLv3 companion for the kit grants, and MIT for the
catalogs where confirmed by the pinned sources. Reuse shared local standard
texts; leave root project LICENSE unchanged.
5. Inspect upstream dependency inventories for leads, but do not copy them
wholesale or treat them as substitutes for Notarius's own inclusion scope.
Hand remaining selected third-party components to Stage 5 through the stage
report, not editorial prose in attribution.
1. Extend the existing application-owned `LLMGenerationError` rather than
creating a parallel provider-error hierarchy. Retain `StatusCode()` and
`errors.Is(err, ErrLLMGeneration)`. Add application-owned failure-kind,
exhaustion, and exhausted-generation attempt metadata/accessors; use uint64
for upstream attempt counts. Preserve the existing constructor's callers
with sensible unspecified/generation metadata, adding a metadata constructor
or private adapter helper as needed.
2. Define kinds for `transport`, `provider_status`, `malformed_response`, and
generic `generation`. Exhaustion is an independent boolean and attempt count,
not a replacement kind. Zero attempts means unavailable in this error
projection, not proof no request happened. The count comes from
`RetryExhaustedError.Attempts()` and covers the exhausted generation, not all
repair generations combined.
3. Use `errors.Is`/`errors.As` on upstream sentinels/types. Keep cancellation and
deadline identity first, capacity handling intact, and HTTP status available
through an exhaustion wrapper. Do not unwrap application errors to upstream
provider objects or classify by prose. Preserve sanitized diagnostic text.
4. A malformed provider envelope is a generation failure; schema-invalid model
content stays `ErrInvalidStructuredOutput`. Do not change outer retry or
validator disposition logic. Document classification at the integration and
internal LLM boundaries.
**Acceptance:** These four components have verified source/notice links and
complete applicable local texts. No dependency changes, guessed notices, or
new distribution obligations were invented.
**Acceptance/tests:** Adapter tests cover transport, malformed response,
noneligible status, exhausted eligible status with count/status preserved, and
cancellation/deadline/capacity regression behavior. Check typed fields and
identities, not full messages. Existing credential-redaction cases remain valid
for nested exhaustion diagnostics; use fake secrets only.
**Validation:** Check exact revision correspondence, upstream filenames, every
new local license link, and byte fidelity of standard texts. Run
`git diff --check`; confirm go.mod/go.sum are unchanged. Report discovery
commands and evidence; do not add a persistent inventory tool or audit artifact.
**Validate:** `GOWORK=off go test ./internal/framework/contracts ./internal/framework/llm ./internal/framework/pipeline`
and `git diff --check`.
## Stage 5 — Complete Attribution And Connect Reader Navigation
## Stage 5 — Project Request-Local Execution Observations
**Goal:** Finish remaining incorporated-component coverage and canonical
navigation. One prompt for the remaining small dependency set and doc links.
**Status:** Completed 2026-09-25.
**Read:** Adopted OSS policy, remaining selected module/source notices,
README.md, integration guides, and repository evidence of copied code/assets.
Start with jsonschema/v6 v6.0.2, x/text v0.40.0, yaml.v3 v3.0.1, and yaml/v4
v4.0.0-rc.6, then resolve any additional actual incorporation revealed by
Stage 4. A go.sum entry alone is not proof of incorporation.
**Goal:** Return a lightweight terminal summary even when generation fails.
One prompt focused on contracts and adapter mapping, without persistence changes.
**Files:** Update `docs/oss/attribution.md`, applicable
`docs/oss/licenses/*`, `README.md`,
`docs/integrations/pkg-configkit.md`, and
`docs/integrations/pkg-promptkit.md`.
**Inspect:** `internal/framework/contracts/contracts.go`, `CompleteStructured`,
`responseFromResult`, prepared execution tests, upstream `observations.go` and
prepared-execution observer documentation.
1. Inspect direct/transitive incorporation and copied material separately.
Apply the same exact-revision, factual-entry, and faithful-text rules as
Stage 4. Include relevant components, not every tool used in development.
Do not add generic entries with unresolved versions or copyrights.
2. Use the OSS format for omitted copyright/SPDX fields where the source
supplies no applicable value. Preserve required project-specific notices.
Raise only a concrete unresolved obligation or rights issue; do not seek
blanket approval for the already selected linked-notice arrangement.
3. End README with the prescribed GPL-v3-or-later COPYRIGHT link and the
Open Source attribution notices link. Do not imply that every incorporated
component or document has the application's license.
4. Replace competing license/notice paragraphs in both integration guides with
concise links to the relevant canonical attribution anchors. Keep dependency
pins, behavior, compatibility, and public API links in the guides.
5. Keep the attribution document to its title and factual component entries.
Do not include the inventory research, legal conclusions, or completion logs.
Do not copy unused upstream examples or standard licenses.
**Work:**
**Acceptance:** Attribution covers the verified relevant components and copied
policy material, with only applicable texts. README and integration links point
to the canonical inventory; no user-visible API or dependency settings change.
1. Add an optional `Execution *LLMExecutionSummary` field to
`StructuredCompletionResponse` (`execution,omitempty`). Own these DTOs in
Notarius contracts; do not alias or serialize upstream event types.
2. The summary contains uint64 `attempts`, `transient_retries`, `repair_calls`;
integer millisecond `capacity_wait_ms`, `generation_ms`, `retry_delay_ms`,
`duration_ms`; `outcome`, `failure_kind`, `status_code`, `retries_exhausted`;
and `usage`. Usage contains prompt/completion/total/cached/cache-write counts,
each with `value`, `known`, `complete` fields. Do not omit these booleans or
known-zero values. The summary needs no prompts, response text, endpoints,
credentials, raw errors, or new copies of run/profile identifiers.
3. Map terminal outcome/failure values explicitly to application-owned strings.
Preserve the distinctions exposed by v0.11.0 (including canceled, deadline,
capacity, invalid output, configuration/load/render failures); unknown future
failures map to `other`. Reuse relevant application failure vocabulary from
Stage 4, without forcing every execution error into a generation error.
4. Attach a local observer to the `RunRequest` before `PrepareExecution` freezes
it. Retain the `ExecutionFinished` projection only: upstream supplies aggregate
counts, durations, and usage. Do not sum attempt events and terminal totals,
retain an event log, or reconstruct a second accounting system. Synchronous
request-local collection needs no global state or asynchronous queue.
5. Attach that summary to every subsequent return from `RunPrepared`, including
context/capacity/provider failure and post-result decode/validation failure.
Preparation failure has no execution event and returns no summary. Preserve
upstream execution outcome as such: a later Notarius decode error remains a
separate returned error, not a fabricated upstream terminal event.
6. Keep existing successful-result token and repair fields unchanged. Observed
`repair_calls` represents execution activity and is not a renamed legacy
`RepairAttempts`. No observation field is a billable-cost guarantee.
**Validation:** Review coverage against resolved dependencies and inspected
copied-material evidence, validate affected relative/source links and text
fidelity, and run `git diff --check`. No Go suite, scanner, or SBOM is required
solely for documentation changes.
**Acceptance/tests:** Success, failed execution, cancellation, and no-observation
preparation failure retain the right presence. Test known zero versus unknown
and partial aggregate usage with controlled provider responses. One concurrent
completion case proves no summary cross-contamination. Check relationships and
nonnegative durations, not exact timing. Document the DTO and mapping in
`docs/internal/llm.md`; public dependency semantics stay in pkg-promptkit.
## Stage 6 — Verify The Complete Adoption
**Validate:** `GOWORK=off go test ./internal/framework/contracts ./internal/framework/llm`
and `GOWORK=off go test -race ./internal/framework/llm`.
**Goal:** Perform the cross-document check and correct scoped documentation
defects. One prompt; no fresh feature work or automated policy-sync system.
## Stage 6 — Carry Summaries Through Existing Debug Paths
**Status:** Completed 2026-09-25.
**Goal:** Make observations useful to operators without a new output channel.
One prompt, limited to debug propagation and terminal allowlists.
**Inspect:** `internal/framework/pipeline/debug.go`, its response-content/fallback
helpers, terminal recording tests, and `debugLLMScope.references`. Trace
`debugLLMCallReference` through the existing timed stage/terminal envelopes;
updating only the detailed response file is insufficient. The CLI's separate
run-summary files do not need a new run-wide accounting mechanism.
**Work:**
1. Project the optional execution summary into existing debug response artifacts
and per-call references used by stage/terminal records. Explicitly preserve it
when `Debug.Response` or `RunResult` is absent and through content splitting
and fallback construction. Keep copies detached if existing clone paths
could otherwise share mutable pointers.
2. Use explicit typed fields as the allowlist for this metadata-only terminal
projection. Do not carry raw upstream events, newly copied provider diagnostics, or model
content into safe terminal summaries. Preserve established redaction and
detailed-trace content boundaries.
3. Leave warnings, normal console output, receipts, domain artifacts, checkpoint
payloads, and no-debug behavior unchanged. No new telemetry file or aggregation
of observed token totals into legacy totals.
4. Update `docs/internal/state.md`/relevant debug internals and `docs/operations.md`
to identify where this diagnostic information is available. Link to the
canonical explanation rather than repeating every field in consumer docs.
**Acceptance/tests:** Through the real debug wrapper and existing recorder seam,
verify success and failed/canceled completions retain summaries in response and
safe terminal records. Verify content splitting/fallback does not discard them,
and debug disabled writes nothing. Reuse existing redaction tests, adding only
the new projection assertions. No new warnings result from recovered retries.
**Validate:** `GOWORK=off go test ./internal/framework/pipeline ./internal/cli`,
`GOWORK=off go test -race ./internal/framework/pipeline`, `git diff --check`.
## Stage 7 — Verify Retry Composition And Finish Operator Guidance
**Status:** Completed 2026-09-25.
**Goal:** Validate the cross-boundary behavior that individual stages cannot
prove, then reconcile current documentation. One prompt; no generalized retry
test framework or runtime redesign.
**Inspect:** Existing adapter HTTP fixtures, scheduler cancellation tests,
pipeline stage retry/debug tests, CLI production tests; current config, LLM,
operations, and consumer documentation.
**Read:** The feature roadmap; the seven resulting policies and their pinned
sources; AGENTS.md, developer guide, COPYRIGHT, README, attribution/license
materials, release runbook, and affected integration links.
**Work and acceptance:**
- Add one representative local-HTTP scenario combining a recoverable transient
status with structurally invalid generated content followed by a successful
repair. Prove transient budgets apply independently per generation and
terminal attempt/repair/usage accounting is not doubled. Use small explicit
budgets rather than executing the illustrative 18-request worst case.
- Exercise cancellation during provider-directed retry waiting without a long
real sleep: signal that the request was received, cancel a context while a
long Retry-After is pending, and wait with a bounded test timeout. Preserve
cancellation identity and the terminal observation. Do not build a second
backoff implementation or test exact random delays.
- Review shared scheduler tests for completion-scoped permit ownership and
cancellation/release. Add a focused integration case only where existing
coverage does not protect that boundary across the new retry path. Keep
upstream backend capacity algorithms upstream.
- Confirm a recovered generation passes through ordinary success without
producing process warnings. Preserve existing outer stage retries and semantic
feedback. Do not alter limits to hide budget multiplication.
- Finish operations guidance: distinguish transient, structural, and stage/
validator budgets; give the 18-attempt single-producer illustration with its
assumptions; explain per-attempt timeout versus overall caller cancellation,
Retry-After waits, cost/latency implications, incomplete usage accounting, and
the outer scheduler holding its permit during waits. No new total deadline or
exactly-once guarantee is implied.
- Review all canonical docs changed earlier for consistency. Keep exact fields
and defaults in config, flags in CLI, dependency semantics in integrations,
mechanism in internals, and operating implications in operations. Keep the
migration note about overlays distinct from unchanged LLM profiles. Retain
historical ADR/release content; no new ADR is needed for these upgrades.
1. Compare all seven policies directly against the selected revision. Classify
remaining differences as metadata, template removal, required customization,
or a feature-roadmap exception. Replace accidental paraphrases with upstream
wording. Preserve established local requirements.
2. Confirm versions 1.3/1.3/1.11/1.3/1.4/1.0/1.8 for agents, architecture,
documentation, testing, release, meta, and OSS respectively. Verify each
Upstream Source targets its own file and Last Updated reflects actual work.
3. Confirm discovery links follow public project/main-guide URLs while adopted
source, examples, and component attribution remain accurately commit-pinned.
Check no placeholder UPSTREAM_COMMIT, template-only section, or local
checkout-dependent reference remains in adopted materials.
4. Check selective policy routing, local authority, canonical documentation
ownership, copied-material scope/modification status, unchanged application
grant/root LICENSE, and directly reachable license/companion texts.
5. Verify source-only publication, selected gates, optional diagnostics,
release completion, and prospective-only footer guidance still agree.
Do not modify historical notes, accepted ADRs, or scripts to satisfy this check.
6. Review changed-file scope: only intended policy, notice, and supporting
documentation files should have changed. Correct in-scope documentation
defects, report unresolved concrete issues, and leave unrelated roadmap
retirement and release preparation to an explicit later request.
**Validate:** Run the focused adapter/pipeline/CLI tests and race tests covering
modified concurrency paths. Review tests for unnecessary upstream duplication
or exact-prose/change-detector assertions; consolidate rather than multiply them.
## Stage 8 — Final Integration Verification And Handoff
**Status:** Completed 2026-09-25.
**Goal:** Verify the complete upgraded application and documentation against the
feature roadmap. One final verification prompt; fix only defects within scope.
**Work:** Review all feature acceptance items and staged diffs. Confirm no
upstream types escaped the LLM boundary, no references to the retired config
flag remain in current examples/help, no unintended LLM profile rename occurred,
and no source-only/license/release scope expanded. Check edited relative links.
Leave the roadmap available for the user's completion review; do not publish a
release or delete planning documents automatically.
**Validation commands:**
```sh
GOWORK=off go test ./...
GOWORK=off go vet ./...
GOWORK=off go build ./...
GOWORK=off go mod tidy -diff
GOWORK=off go test -race ./internal/framework/llm ./internal/framework/pipeline
GOWORK=off go run ./cmd/notarius config validate --config examples/dnd-minimal.config.yml
GOWORK=off go run ./cmd/notarius config validate --config examples/dnd-complete.config.yml
GOWORK=off go run ./cmd/notarius config validate --config examples/dnd-complete.config.yml --config-overlay development
git diff --check
```
Reuse already-passed checks for the same unchanged candidate where appropriate.
The example/CLI tests own maintained examples; the commands above are a one-time
migration smoke check, not new permanent release gates. No live model calls or
real API keys are required. Report environment/dependency-access failures
separately from implementation failures; do not claim blocked checks passed.
**Completion criteria:** Both exact dependency pins are active; overlay migration,
retry setting, typed errors, observations, debug propagation, tests, examples,
and current documentation satisfy the feature roadmap. Report commits and
checks, plus any residual limitation. Future release preparation should explain
the breaking overlay cutover, retry default/opt-out, and upstream license change
in a minor-release note, but is not part of this implementation.
**Validation:** Run `git diff --check`; inspect file/link destinations and
compare local standard texts with verified sources. A search for
`Base Policy Template` or `UPSTREAM_COMMIT` in adopted files should be empty;
do not confuse explanatory roadmap text with an unfilled adopted template.
No production tests, new test framework, recurring synchronization gate, or
release execution is needed.
## Open Questions
None. The feature roadmap supplies the policy choices needed for implementation.
The DTO naming and stage boundaries above settle remaining implementation choices
without adding new product restrictions or compatibility promises.
None at the agreed baseline. Policy choices and exceptions are settled.
A newly discovered concrete notice/source or rights issue should be reported
with evidence and viable treatments rather than resolved by inventing a broader
requirement. That contingency does not authorize reopening settled policies.