notarius/docs/roadmap/promptkit.md

PromptKit 0.12 Configuration Migration

Purpose

Adopt PromptKit v0.12.0 after the completed ConfigKit upgrade, replacing the removed configuration APIs and asset formats with PromptKit's ConfigKit-backed configuration boundary. Make the new composition, inheritance, presence, and captured-configuration capabilities useful without introducing a second loader or changing Notarius's domain extraction and pipeline acceptance policies.

This is a feature roadmap, not an execution sequence. The operator-source and native-overlay decisions were accepted on 2026-10-04 and are incorporated below. A separate implementation plan will define the execution sequence. Release preparation and publication are separate work.

Reviewed Baseline

Reviewed on 2026-10-04 against Notarius commit af62b3b, with ConfigKit v0.5.0 and PromptKit v0.11.2 selected in go.mod. The remote PromptKit v0.12.0 tag resolves to full source commit 4b8169da286917fde7875425bb32b799b12c6af3, matching the sibling checkout. Primary upstream references are the release guide, consumer migration guide, and format reference. These were reviewed in the local checkout; the remote tag was verified directly.

The current integration has these migration-sensitive boundaries:

• internal/framework/llm/asset_registry.go flattens module-owned prompt, schema, and fallback-profile filesystems and constructs WithPromptFS, WithSchemaFS, and WithFallbackProfileFS options.
• promptkit_profiles.go shares profile-source construction between the execution client and credential-free explicit-profile inspection. It uses Config.ProfileDir, WithProfileFile, and the backend registration option. The inspection engine supplies PromptDir: "." without preparing a prompt.
• promptkit_client.go constructs the engine and uses prepared execution, semantic correction messages, structural repair overrides, observations, direct sessions, reasoning overrides, and provider-neutral error adaptation.
• promptkit_profile_fingerprint.go combines a pinned catalog marker with reread operator profile files and fallback asset bytes. The optional local endpoint contributes a separate identity; local concurrency is excluded.
• Embedded prompt definitions throughout assets/dnd/ and assets/generic/normalize/deduplication/ use top-level id and version. assets/dnd/profiles/dnd-extraction.yaml and the maintained deployment file examples/llm-profiles/dnd-extraction.yml use top-level id.
• The public profile contract says a matching operator profile completely replaces a fallback or catalog profile. It documents base_profile inheritance. These claims do not match v0.12.0's native semantics.

The current focused LLM, CLI, and module-integration tests pass with GOWORK=off. This establishes the baseline only: no trial dependency upgrade, live inference, or production model-quality evaluation has been performed.

Upstream Changes Relevant To Notarius

PromptKit v0.12.0 is an incompatible configuration/API release:

• Config.Sources accepts ordered ConfigKit underlays and an optional main input. Legacy prompt/profile directory fields and configuration-source options are removed. Schema and injected-runtime options remain available.
• Documents use keyed backends, profiles, and prompts mappings, with a second version mapping beneath each prompt ID. Embedded id, version, and base_profile fields are invalid; profile inheritance uses extends.
• Files within one source group compose strictly, not by filename precedence. Later groups overlay earlier groups recursively. Explicit null, zero, false, and empty values can differ from omission; {} does not clear a map.
• Engine construction validates every final effective definition, including unused definitions, and captures message content and static templates. Ordinary execution and inspection do not reload them. Schemas, request artifacts, credentials, and provider operations retain their runtime timing.
• InspectConfiguration exposes captured presence, sources, diagnostics, explanations, and redacted effective output. These reports are not automatically safe to publish: ordinary prompt/provider content remains data.
• Maintained catalogs move to /v3, both at v3.20260930.0. PromptKit and Notarius already share the required ConfigKit v0.5.0 and Go 1.25.5 baseline; no additional Go-version upgrade is required by this release.

Target End State

One Native Configuration Boundary

PromptKit owns the effective backend/profile/prompt configuration and all upstream composition and inheritance. Notarius selects its sources and adapts the resulting engine to the existing provider-neutral completion contract. ConfigKit and PromptKit types remain inside the LLM adapter's explicit integration boundary, not in domain modules or pipeline contracts.

Retain Notarius configuration version: 5, promptkit.profile_file, promptkit.profile_dir, their mutual exclusion and declaring-file-relative path handling, transient_retries, and the optional conventional local backend. Do not add replacement Notarius configuration fields merely because the upstream constructor has changed. Document the changed external file format.

Use one shared source-construction path for execution and profile preflight. The source precedence is:

1. PromptKit's maintained catalog layer, supplied by PromptKit itself.
2. Registered application prompt and fallback-profile assets.
3. The explicitly selected operator file or directory.
4. A programmatic local backend contribution when Notarius's promptkit.local_backend is configured, so those explicit Notarius values win over conflicting file declarations of those fields.

The existing selected file/directory accepts native combined profiles, backends, and prompts documents. Enable explicit ConfigKit imports on these operator inputs. Describe the familiar profile_file and profile_dir fields as PromptKit configuration sources whose principal workflow is profile selection; do not add a section whitelist or a custom merge language. Domain modules remain responsible for interpreting and validating model output, even when an operator supplies a prompt overlay.

Inputs within each group use native strict composition. Directory discovery is recursive over YAML files and has no hidden per-file priority. No unselected directory is scanned. Construct filesystem-backed inputs with usable resource contexts so referenced message files retain their declaring-source base. Do not combine host profile files with embedded content under a fabricated common filesystem root.

Document import layouts that do not accidentally select the same definitions through both directory discovery and imports: a file-root import layout is often the simplest choice. Use native import/resource rules rather than inventing deduplication, filename override order, or broader host access.

Use FilesystemInputs for the existing flattened embedded views, DirectoryInputs or an individual ConfigKit file input for the operator source, and BackendSource with the presence-aware local-backend helper for programmatic backend configuration. Keep schemas on the existing schema filesystem boundary. There is no need to deserialize catalog assets, register catalog backends individually, or recreate PromptKit inheritance.

Profile preflight does not need dummy prompt directories or synthetic prompts. Both construction paths include the same catalog, application prompt/fallback, operator, and local-backend layers. This matters for partial operator prompt overlays: inspecting a layer without its application base would validate a different effective definition. Call InspectProfile on the constructed engine without preparing a synthetic request. Construction still captures and validates the effective prompt definitions; preflight does not resolve credential values, prepare request artifacts, or contact providers.

Keyed Assets Without Prompt Behavior Changes

Every maintained prompt declaration becomes a keyed document of this shape:

prompts:
  dnd.spells:
    v1:
      # Existing definition body, without embedded id/version fields.

Profiles become keyed entries beneath profiles. Preserve literal prompt IDs, the v1 version, logical profile IDs, selected response schemas, content paths, message order, roles, shared fragment bytes, cache-control metadata, and configured structural repair budgets. Do not conflate this migration with prompt rewriting, schema changes, or durable artifact version changes.

The application-owned dnd-extraction fallback continues to explicitly select openrouter, openai/gpt-5.6-luna, timeout 240, and service tier flex, with reasoning effort unset. Do not replace that choice with an upstream catalog alias or adopt unrelated model defaults. Migrate the maintained operator example and relevant inline Go/YAML test fixtures as well as production assets.

Module ownership, ordered asset manifests, scoped mounting, and selected-asset hashing remain intact. Check virtual filenames and source-relative content_file resolution after wrapping definitions; do not move business logic into the root assets package.

Presence, Inheritance, And Operator Migration

The adopted policy is native recursive overlay, not a compatibility shim for complete-definition replacement. Operators can override only the fields they want. Omitted fields retain lower-layer values, and extends uses native literal sibling-key lookup. Changing a backend or model does not erase explicit inherited settings; backend-derived defaults are applied to the final backend after inheritance.

Document intentional clearing with null, including reasoning/service tier, sampling controls, credential lookup, and provider maps where supported. Distinguish null from zero/false/blank and from an empty mapping. For example, switching an operator's dnd-extraction overlay to a local model may also need service_tier: null to remove the fallback's explicit flex setting. Do not automatically scrub settings based on model-name heuristics.

Translate base_profile to extends in maintained examples and explain that operators must migrate their external files. Do not retain legacy aliases or an alternate parser. Preserve exact PromptKit keys; do not introduce a new identifier normalization pass. Update tests that assumed whole replacement or selection-time validation deliberately, rather than treating those intentional upstream changes as regressions.

Native eager validation applies to final effective values, not independently to each superseded contribution. Do not add a source-history audit or raw-layer validation that prevents legitimate overrides from repairing lower layers.

Native Operator Capabilities

Support the capabilities of the unified format directly, without new Notarius configuration knobs or model-specific adaptation:

• Custom or multiple backend definitions, including endpoint, credential environment name, provider parameters, concurrency, and queue capacity. Operators may also overlay maintained backend settings, for example changing OpenRouter's admission limit. Backend IDs are configuration keys, not a closed registration list. Notarius's global scheduler still applies.
• Profile inheritance, partial overrides, and recursive provider-parameter composition. Explicit null suppression and zero/false values follow native field rules. Native sibling inheritance is not limited by a Notarius profiles-only whitelist; prompt/backend mappings follow upstream rules too.
• File-configured api_key_required, with PromptKit's exact credential precedence and runtime checks. In particular, true suppresses a backend's implicit environment source; an explicitly configured profile environment source remains usable. This is an operator choice, not a new mandatory credential policy or a reason to require credentials during preflight.
• Deployment prompt overlays and declaring-source-relative message files, including imported content. Prompt version selection remains caller-owned; this upgrade does not infer a new module version from file order or add automatic latest-version selection.

The existing local_backend object remains a convenient highest-priority contribution for that conventional backend. Other backend settings remain available through the unified operator source; do not copy every upstream field into a parallel Notarius backend schema.

Keep the two maintained top-level configuration examples. Use the complete example's existing PromptKit deployment file and small canonical documentation snippets to demonstrate native inheritance/partial overrides, custom backend configuration, imports, null clearing, and prompt editing without adding a large unrelated example suite. Not every capability needs its own runnable pipeline.

Captured Execution And Checkpoint Identity

One engine represents one captured configuration for its lifetime. Editing operator definitions or message files requires constructing a new engine; there is no watcher or automatic hot reload in this scope. Preserve the normal CLI lifecycle without promising that independently constructed preflight and execution engines are one transactional filesystem snapshot.

Compute and retain checkpoint identity from the configuration actually captured by the execution engine. Do not identify an old captured engine by files reread after it was constructed. Identity must account for effective profile/backend semantics and the pinned PromptKit/catalog revisions. It must also account for operator prompt definitions and their captured prompt content, not just content_file path strings.

Use public captured inspection and prompt-definition equality values where appropriate; do not reach into PromptKit internals. InspectPrompt.PromptHash includes captured content and is an opaque equality value. Inspection reports alone do not embed referenced message-file bodies. Preserve application module/schema fingerprints, actual-used profile provenance, and the existing exclusion of concurrency-only local settings from semantic checkpoint identity. A conservative identity across effective profiles is acceptable; per-selected- profile optimization is not a prerequisite for this upgrade.

Raw sources, configuration history, prompt bodies, endpoints, and credential values must not be added to ordinary manifests or checkpoint metadata. The new inspection API is useful privately for the migration and captured identity; it does not require a new public CLI report or automatic dump into output or debug bundles. Hash-only identity is distinct from publishing an inspection report, and redaction is not a substitute for covering execution semantics.

Existing Execution Policies Remain Unchanged

Preserve prepared execution, the exact latest-response semantic correction pair, prompt-prefix caching, direct sessions, reasoning overrides, cancellation, the shared scheduler, local/backend admission, observer accounting, error classification, and credential redaction. Structural repairs, transient HTTP retries, outer producer attempts, and validator execution budgets remain separate. The existing one-repair maintained prompt default remains unchanged.

The migration does not change extraction/normalization semantics, warning classification, artifact contracts, subprocess receipts, or reusable-state eligibility. Construction errors may occur earlier under native eager validation; adapt them through the existing safe adapter/CLI error boundary, not by exposing arbitrary upstream configuration content.

Required Work

• Update go.mod and go.sum to PromptKit v0.12.0 and its selected /v3 catalogs. Keep ConfigKit v0.5.0 and unrelated dependencies unless module resolution requires a change. Remove obsolete /v2 requirements after tidy.
• Replace removed constructor/source/backend APIs in the LLM adapter and shared profile-preflight construction; retain schema and injected-client options. Remove obsolete adapter helpers rather than layering a parallel configuration system beside them.
• Migrate every registered prompt and fallback profile, the maintained deployment profile, and affected test fixtures to keyed definitions.
• Adopt the unified source policy, explicit operator imports, presence/null behavior, native operator capabilities, and captured lifetime; replace live-file checkpoint fingerprinting with captured identity.
• Update behavioral tests at the adapter, CLI preflight, asset registry, and representative assembled-module boundaries. Reuse real ConfigKit/PromptKit loading and offline injected clients or local HTTP fixtures.
• Update canonical documentation when implementation lands: docs/config.md owns selected sources, precedence, profile syntax and migration; docs/operations.md owns deployment and reload guidance; docs/internal/llm.md owns construction, capture, and checkpoint mechanics; docs/integrations/pkg-promptkit.md owns the supported upstream boundary and pinned links. Revise consumer/CLI examples only where actually affected.
• Update docs/oss/attribution.md for PromptKit and the two /v3 catalogs, checking their actual notices and exact source commits. PromptKit remains LGPL-3.0-or-later; the catalogs remain MIT at the reviewed revisions. Reuse unchanged standard texts and preserve the project's own license grant.

Native layered overrides change an existing documented operator contract. Add a concise Notarius ADR recording the accepted native composition and unified-source ownership/compatibility decision rather than duplicating the upstream manual. Existing historical ADRs need not be rewritten.

Verification And Acceptance

Acceptance is behavioral, not an inventory of private helper names:

• All registered D&D and generic prompts construct and prepare with migrated assets, resolving shared/module content in the correct source context. Request identities, message content/order, schemas, and cache controls retain their intended behavior. Do not add exact shared-prefix-length detector tests.
• Operator overrides, inheritance, explicit zero/null, duplicate definitions within a group, local backend precedence, endpoint-only profiles, and invalid unused final definitions follow the native configuration policy. Cover representative imported combined definitions, custom/backend catalog overlays, partial prompt overlays with their embedded base, and optional versus explicitly required credentials. Credential-free preflight and execution resolve the same source ordering and application definitions.
• Engine/file mutation tests demonstrate captured settings and content; fingerprints stay tied to that engine and change when a new engine captures different execution-relevant settings or operator prompt content.
• Structural repair, semantic feedback, sessions/reasoning, capacity failure, transient errors, safe diagnostics, and observations retain existing provider-neutral behavior using deterministic boundary tests.
• Maintained minimal, complete, and inherited development pipelines validate without credentials or provider calls. External legacy profile files receive clear migration guidance, not silent reinterpretation.

Use focused LLM, CLI, and module-integration tests during implementation, then GOWORK=off go test ./..., GOWORK=off go vet ./..., a temporary native build, and GOWORK=off go mod tidy -diff for this shared-boundary migration. Verify selected module versions, attribution links, maintained examples, and stale API/format claims. These are migration checks, not new permanent release gates. Live provider testing is optional follow-up evidence, not required acceptance.

This work warrants a minor Notarius release because the accepted external profile format and override behavior change. Keep Notarius configuration at version 5; select the actual release number during separate release preparation.