notarius/docs/roadmap/promptkit.md

PromptKit 0.12 Configuration Migration

Purpose

Adopt PromptKit v0.12.0 after the completed ConfigKit upgrade, replacing the removed configuration APIs and asset formats with PromptKit's ConfigKit-backed configuration boundary. Make the new composition, inheritance, presence, and captured-configuration capabilities useful without introducing a second loader or changing Notarius's domain extraction and pipeline acceptance policies.

This is a feature roadmap, not an execution sequence. A separate implementation plan should be prepared after the open decisions below are settled. Release preparation and publication are separate work.

Reviewed Baseline

Reviewed on 2026-10-04 against Notarius commit af62b3b, with ConfigKit v0.5.0 and PromptKit v0.11.2 selected in go.mod. The remote PromptKit v0.12.0 tag resolves to full source commit 4b8169da286917fde7875425bb32b799b12c6af3, matching the sibling checkout. Primary upstream references are the release guide, consumer migration guide, and format reference. These were reviewed in the local checkout; the remote tag was verified directly.

The current integration has these migration-sensitive boundaries:

• internal/framework/llm/asset_registry.go flattens module-owned prompt, schema, and fallback-profile filesystems and constructs WithPromptFS, WithSchemaFS, and WithFallbackProfileFS options.
• promptkit_profiles.go shares profile-source construction between the execution client and credential-free explicit-profile inspection. It uses Config.ProfileDir, WithProfileFile, and the backend registration option. The inspection engine supplies PromptDir: "." without preparing a prompt.
• promptkit_client.go constructs the engine and uses prepared execution, semantic correction messages, structural repair overrides, observations, direct sessions, reasoning overrides, and provider-neutral error adaptation.
• promptkit_profile_fingerprint.go combines a pinned catalog marker with reread operator profile files and fallback asset bytes. The optional local endpoint contributes a separate identity; local concurrency is excluded.
• Embedded prompt definitions throughout assets/dnd/ and assets/generic/normalize/deduplication/ use top-level id and version. assets/dnd/profiles/dnd-extraction.yaml and the maintained deployment file examples/llm-profiles/dnd-extraction.yml use top-level id.
• The public profile contract says a matching operator profile completely replaces a fallback or catalog profile. It documents base_profile inheritance. These claims do not match v0.12.0's native semantics.

The current focused LLM, CLI, and module-integration tests pass with GOWORK=off. This establishes the baseline only: no trial dependency upgrade, live inference, or production model-quality evaluation has been performed.

Upstream Changes Relevant To Notarius

PromptKit v0.12.0 is an incompatible configuration/API release:

• Config.Sources accepts ordered ConfigKit underlays and an optional main input. Legacy prompt/profile directory fields and configuration-source options are removed. Schema and injected-runtime options remain available.
• Documents use keyed backends, profiles, and prompts mappings, with a second version mapping beneath each prompt ID. Embedded id, version, and base_profile fields are invalid; profile inheritance uses extends.
• Files within one source group compose strictly, not by filename precedence. Later groups overlay earlier groups recursively. Explicit null, zero, false, and empty values can differ from omission; {} does not clear a map.
• Engine construction validates every final effective definition, including unused definitions, and captures message content and static templates. Ordinary execution and inspection do not reload them. Schemas, request artifacts, credentials, and provider operations retain their runtime timing.
• InspectConfiguration exposes captured presence, sources, diagnostics, explanations, and redacted effective output. These reports are not automatically safe to publish: ordinary prompt/provider content remains data.
• Maintained catalogs move to /v3, both at v3.20260930.0. PromptKit and Notarius already share the required ConfigKit v0.5.0 and Go 1.25.5 baseline; no additional Go-version upgrade is required by this release.

Target End State

One Native Configuration Boundary

PromptKit owns the effective backend/profile/prompt configuration and all upstream composition and inheritance. Notarius selects its sources and adapts the resulting engine to the existing provider-neutral completion contract. ConfigKit and PromptKit types remain inside the LLM adapter's explicit integration boundary, not in domain modules or pipeline contracts.

Retain Notarius configuration version: 5, promptkit.profile_file, promptkit.profile_dir, their mutual exclusion and declaring-file-relative path handling, transient_retries, and the optional conventional local backend. Do not add replacement Notarius configuration fields merely because the upstream constructor has changed. Document the changed external file format.

Use one shared source-construction path for execution and profile preflight. The proposed precedence, subject to the decisions below, is:

1. PromptKit's maintained catalog layer, supplied by PromptKit itself.
2. Registered application prompt and fallback-profile assets.
3. The explicitly selected operator file or directory.
4. A programmatic local backend contribution when Notarius's promptkit.local_backend is configured, so those explicit Notarius values win over conflicting file declarations of those fields.

Inputs within each group use native strict composition. Directory discovery is recursive over YAML files and has no hidden per-file priority. No unselected directory is scanned. Construct filesystem-backed inputs with usable resource contexts so referenced message files retain their declaring-source base. Do not combine host profile files with embedded content under a fabricated common filesystem root.

Use FilesystemInputs for the existing flattened embedded views, DirectoryInputs or an individual ConfigKit file input for the operator source, and BackendSource with the presence-aware local-backend helper for programmatic backend configuration. Keep schemas on the existing schema filesystem boundary. There is no need to deserialize catalog assets, register catalog backends individually, or recreate PromptKit inheritance.

The profile-only preflight engine does not need dummy prompt directories or synthetic prompts. It can construct an engine from catalog, fallback, operator, and local-backend sources, then call InspectProfile. If an operator source also supplies prompts under the selected source policy, native construction validation applies to them as well. Preflight does not resolve credential values, prepare request artifacts, or contact providers.

Keyed Assets Without Prompt Behavior Changes

Every maintained prompt declaration becomes a keyed document of this shape:

prompts:
  dnd.spells:
    v1:
      # Existing definition body, without embedded id/version fields.

Profiles become keyed entries beneath profiles. Preserve literal prompt IDs, the v1 version, logical profile IDs, selected response schemas, content paths, message order, roles, shared fragment bytes, cache-control metadata, and configured structural repair budgets. Do not conflate this migration with prompt rewriting, schema changes, or durable artifact version changes.

The application-owned dnd-extraction fallback continues to explicitly select openrouter, openai/gpt-5.6-luna, timeout 240, and service tier flex, with reasoning effort unset. Do not replace that choice with an upstream catalog alias or adopt unrelated model defaults. Migrate the maintained operator example and relevant inline Go/YAML test fixtures as well as production assets.

Module ownership, ordered asset manifests, scoped mounting, and selected-asset hashing remain intact. Check virtual filenames and source-relative content_file resolution after wrapping definitions; do not move business logic into the root assets package.

Presence, Inheritance, And Operator Migration

The recommended policy is native recursive overlay, not a compatibility shim for complete-definition replacement. Operators can override only the fields they want. Omitted fields retain lower-layer values, and extends uses native literal sibling-key lookup. Changing a backend or model does not erase explicit inherited settings; backend-derived defaults are applied to the final backend after inheritance.

Document intentional clearing with null, including reasoning/service tier, sampling controls, credential lookup, and provider maps where supported. Distinguish null from zero/false/blank and from an empty mapping. For example, switching an operator's dnd-extraction overlay to a local model may also need service_tier: null to remove the fallback's explicit flex setting. Do not automatically scrub settings based on model-name heuristics.

Translate base_profile to extends in maintained examples and explain that operators must migrate their external files. Do not retain legacy aliases or an alternate parser. Preserve exact PromptKit keys; do not introduce a new identifier normalization pass. Update tests that assumed whole replacement or selection-time validation deliberately, rather than treating those intentional upstream changes as regressions.

Native eager validation applies to final effective values, not independently to each superseded contribution. Do not add a source-history audit or raw-layer validation that prevents legitimate overrides from repairing lower layers.

Captured Execution And Checkpoint Identity

One engine represents one captured configuration for its lifetime. Editing operator definitions or message files requires constructing a new engine; there is no watcher or automatic hot reload in this scope. Preserve the normal CLI lifecycle without promising that independently constructed preflight and execution engines are one transactional filesystem snapshot.

Compute and retain checkpoint identity from the configuration actually captured by the execution engine. Do not identify an old captured engine by files reread after it was constructed. Identity must account for effective profile/backend semantics and the pinned PromptKit/catalog revisions; if operator prompt definitions are admitted, it must also account for their captured prompt content, not just content_file path strings.

Use public captured inspection and prompt-definition equality values where appropriate; do not reach into PromptKit internals. InspectPrompt.PromptHash includes captured content and is an opaque equality value. Inspection reports alone do not embed referenced message-file bodies. Preserve application module/schema fingerprints, actual-used profile provenance, and the existing exclusion of concurrency-only local settings from semantic checkpoint identity. A conservative identity across effective profiles is acceptable; per-selected- profile optimization is not a prerequisite for this upgrade.

Raw sources, configuration history, prompt bodies, endpoints, and credential values must not be added to ordinary manifests or checkpoint metadata. The new inspection API is useful privately for the migration and captured identity; it does not require a new public CLI report or automatic dump into output or debug bundles. Hash-only identity is distinct from publishing an inspection report, and redaction is not a substitute for covering execution semantics.

Existing Execution Policies Remain Unchanged

Preserve prepared execution, the exact latest-response semantic correction pair, prompt-prefix caching, direct sessions, reasoning overrides, cancellation, the shared scheduler, local/backend admission, observer accounting, error classification, and credential redaction. Structural repairs, transient HTTP retries, outer producer attempts, and validator execution budgets remain separate. The existing one-repair maintained prompt default remains unchanged.

The migration does not change extraction/normalization semantics, warning classification, artifact contracts, subprocess receipts, or reusable-state eligibility. Construction errors may occur earlier under native eager validation; adapt them through the existing safe adapter/CLI error boundary, not by exposing arbitrary upstream configuration content.

Required Work

• Update go.mod and go.sum to PromptKit v0.12.0 and its selected /v3 catalogs. Keep ConfigKit v0.5.0 and unrelated dependencies unless module resolution requires a change. Remove obsolete /v2 requirements after tidy.
• Replace removed constructor/source/backend APIs in the LLM adapter and shared profile-preflight construction; retain schema and injected-client options. Remove obsolete adapter helpers rather than layering a parallel configuration system beside them.
• Migrate every registered prompt and fallback profile, the maintained deployment profile, and affected test fixtures to keyed definitions.
• Adopt the agreed source policy, presence/null behavior, and captured lifetime; replace live-file checkpoint fingerprinting with captured identity.
• Update behavioral tests at the adapter, CLI preflight, asset registry, and representative assembled-module boundaries. Reuse real ConfigKit/PromptKit loading and offline injected clients or local HTTP fixtures.
• Update canonical documentation when implementation lands: docs/config.md owns selected sources, precedence, profile syntax and migration; docs/operations.md owns deployment and reload guidance; docs/internal/llm.md owns construction, capture, and checkpoint mechanics; docs/integrations/pkg-promptkit.md owns the supported upstream boundary and pinned links. Revise consumer/CLI examples only where actually affected.
• Update docs/oss/attribution.md for PromptKit and the two /v3 catalogs, checking their actual notices and exact source commits. PromptKit remains LGPL-3.0-or-later; the catalogs remain MIT at the reviewed revisions. Reuse unchanged standard texts and preserve the project's own license grant.

Native layered overrides change an existing documented operator contract. A concise Notarius ADR is appropriate if that recommendation is accepted, recording the ownership/compatibility decision rather than duplicating the upstream manual. Existing historical ADRs need not be rewritten.

Verification And Acceptance

Acceptance is behavioral, not an inventory of private helper names:

• All registered D&D and generic prompts construct and prepare with migrated assets, resolving shared/module content in the correct source context. Request identities, message content/order, schemas, and cache controls retain their intended behavior. Do not add exact shared-prefix-length detector tests.
• Operator overrides, inheritance, explicit zero/null, duplicate definitions within a group, local backend precedence, endpoint-only profiles, and invalid unused final definitions follow the agreed native configuration policy. Credential-free preflight and execution resolve the same source ordering.
• Engine/file mutation tests demonstrate captured settings and content; fingerprints stay tied to that engine and change when a new engine captures different execution-relevant settings or admitted prompt content.
• Structural repair, semantic feedback, sessions/reasoning, capacity failure, transient errors, safe diagnostics, and observations retain existing provider-neutral behavior using deterministic boundary tests.
• Maintained minimal, complete, and inherited development pipelines validate without credentials or provider calls. External legacy profile files receive clear migration guidance, not silent reinterpretation.

Use focused LLM, CLI, and module-integration tests during implementation, then GOWORK=off go test ./..., GOWORK=off go vet ./..., a temporary native build, and GOWORK=off go mod tidy -diff for this shared-boundary migration. Verify selected module versions, attribution links, maintained examples, and stale API/format claims. These are migration checks, not new permanent release gates. Live provider testing is optional follow-up evidence, not required acceptance.

This work warrants a minor Notarius release because the accepted external profile format and override behavior change. Keep Notarius configuration at version 5; select the actual release number during separate release preparation.

Open Decisions

1. Native Overlay Or Preserve Complete Profile Replacement?

Recommendation: adopt PromptKit's recursive field overlay and native extends/null semantics. It removes bespoke configuration machinery, makes small deployment overrides useful, and keeps Notarius aligned with upstream. The tradeoff is explicit operator migration: omitted fields no longer erase fallback settings, so local/model switches may require null clears.

Alternative: preserve whole-profile replacement by explicitly clearing lower definitions before applying operator definitions. This protects the old replacement expectation, but requires Notarius to inspect/transform selected sources and account for inheritance and origins. It undermines the simple native handoff and is harder to maintain. Merely placing files in a later underlay does not implement whole replacement.

2. Scope Of Operator PromptKit Documents

Recommendation: allow native combined profiles, backends, and prompts documents through the existing selected file/directory settings, and enable explicit ConfigKit imports on those operator inputs. Keep the familiar fields and describe them as PromptKit configuration sources whose principal workflow is profile selection. Do not invent a Notarius section whitelist or custom merge language. Custom backends and deployment prompt edits become useful native capabilities; source-relative content capture and checkpoint identity must cover them. Module response decoding and artifact validation still define what a supplied prompt can successfully produce.

Directory selection discovers all YAML beneath its root. Imported fragments that would also be discovered must not accidentally become duplicate inputs; document a file-root import layout or keep fragments outside the discovered directory, using upstream resource/import rules. Do not promise filename override order or unrestricted host resource access beyond upstream behavior.

Alternative: retain profile-only operator inputs and leave imports disabled for this upgrade, introducing broader source selection later. This narrows the initial migration and leaves module prompts exclusively embedded, but requires an explicit locally chosen restriction on the unified upstream format and postpones custom backend/configuration composition. It should be a deliberate product decision, not an accidental consequence of old helper names.