diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 7eea22a..ec4da54 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1,536 +1,285 @@ -# ConfigKit Adoption Implementation Plan +# Policy Alignment Implementation Plan -## Status, Authority, And Historical Plans +## Status, Scope, And Authority -Planned on 2026-09-26 against Scriptorium commit -`43eea5c7c41d2a8134b3dece60f3af5c98b8fd1c`. This document records the -ordered implementation and its completion evidence. The maintainer resolved -the inspection-policy question: -`extra_params` remain visible; only API-key values require secret protection. +Planned on 2026-09-28 against Scriptorium +`8452b54809b99425a97940da485daaa48c77aeb8`. All stages below are pending. +Execute them in integer order; each is sized as a focused implementation task +for a gpt-5.6-terra coding agent. -The [feature roadmap](configkit.md) owns scope, target behavior, and user intent. -This plan owns implementation steps and the interface choices needed to execute -them. Read [development](../development.md) and every policy before work: -[agents](../policy/agents.md), [architecture](../policy/architecture.md), -[documentation](../policy/documentation.md), [testing](../policy/testing.md), -[release](../policy/release.md), and [OSS](../policy/oss.md). -[ADR 0004](../adr/0004-definition-boundary.md) continues to govern Promptkit's -definition boundary. Do not rewrite accepted ADR decisions. +The [feature roadmap](policy.md) owns the agreed end state, scope, and reasons +for local exceptions. This plan owns execution steps and acceptance checks. +This is documentation work, not application, dependency, or release-tool work. -This filename previously contained completed Promptkit plans. Older roadmap -links to this file refer to these historical records, not the ConfigKit stages: +The selected public upstream revision is +`d84a38ebab26f048cafbe59a2246700c35d29843`. +Start with its +[adoption guide](https://gitea.maximumdirect.net/eric/policy/src/commit/d84a38ebab26f048cafbe59a2246700c35d29843/docs/adoption.md). +Retrieve source files at this revision, not whatever main contains when a +stage runs. A verified checkout of this exact revision is acceptable; no +sibling checkout is a prerequisite. If source retrieval fails, report that +blocker rather than inventing source content or provenance. -- Completed v0.11 implementation and release handoff: - `git show 43eea5c7c41d2a8134b3dece60f3af5c98b8fd1c:docs/roadmap/implementation.md`. -- Completed v0.9 implementation: - `git show e4405966c87c8a7843b340146e90cc1e515bef42:docs/roadmap/implementation.md`. +Read the current local agent policy and developer orientation before work, +then policies governing the stage. Once Stage 2 lands, use the new selective +reading guide. Reuse unchanged policy text already available in context. -Those deliveries must not be repeated or lost from upcoming release preparation. -These Git records are available locally without publishing the recent commits. +## Historical Implementation Records -## Working Rules And Selected Design +This filename previously held completed application plans. Preserve their +records through Git rather than carrying their implementation steps forward: -The dependency target is `gitea.maximumdirect.net/eric/ConfigKit v0.4.0`, -commit `e6ced0d3d1194b58e0b3dc8970089471ba175258`. Use its public root package, -imported as `configkit`. The roadmap links the exact upstream contracts. -Do not opportunistically upgrade Promptkit or other dependencies, commit a -workspace/replacement, or depend on sibling source at runtime. +- ConfigKit adoption and release handoff: + `git show 8452b54809b99425a97940da485daaa48c77aeb8:docs/roadmap/implementation.md`. +- Earlier PromptKit plans are referenced within that historical document. -This is a clean configuration cutover, not a compatibility project. Native -ConfigKit syntax, presence, nullability, resource limits, and precedence apply. -Do not add a pre-parser, YAML serialization round trip, alternate merge engine, -legacy mode, generic provider abstraction, or a new CLI framework. +Those stages must not be rerun. Existing release notes, including +[the v0.14.0 note](../releases/v0.14.0.md), record prior delivered features. +Do not infer a new release obligation from an old prospective handoff. +The new policy and licensing changes still need a concise summary in the next +release, including the already-selected GPL-3.0-or-later application grant and +the separately licensed policy material. Version selection and publication +are not part of this plan. -Keep application source discovery and domain rules in `internal/config`; -CLI parsing/output stays in `internal/adapter/cli`. Promptkit remains responsible -for prompt/profile/schema parsing, backend semantics, credentials, and engine -behavior. Keep one shared server engine and existing HTTP artifact policy. -No HTTP schema, execution-report, request-timeout, or retry-orchestration change -is part of this plan. +## Common Execution Rules -For every stage: +1. Inspect status and relevant diffs before each stage. Preserve unrelated + changes; do not silently include them in the stage commit. +2. Read the selected upstream files completely before adopting them. Compare + directly with local files; do not reconstruct historical adoption baselines. +3. Start from upstream text, remove the complete Base Policy Template section, + and apply only the customizations selected below. Preserve shared language + verbatim, including its meaning and strength. Do not retain old paraphrases + merely because they are familiar or add prohibitions for hypothetical risks. +4. For each policy adopted in that stage, set the target Base Policy Version, + the actual adoption date, and an Upstream Source link: + `https://gitea.maximumdirect.net/eric/policy/src/commit/d84a38ebab26f048cafbe59a2246700c35d29843/docs/policy/FILE.md`. + Substitute the actual filename. Do not stamp a source on a policy before + incorporating it. Intermediate commits may legitimately have mixed versions. +5. Update the single policy-attribution entry as its copied/adapted scope grows. + Do not claim later-stage material has already been adopted. +6. Validate affected text, links, notices, and diff; record concise completion + evidence under the completed stage here. Commit each coherent stage with a + short plain-English message under the agent policy. Do not push or publish. -1. Inspect the working tree and relevant code/tests through the repository's - code-discovery guidance. Preserve unrelated changes. -2. Implement the listed outcome and only supporting refactoring it needs. - Internal names below are selected implementation guidance, not public APIs - or immutable package-layout contracts. -3. Update canonical current-state docs when behavior lands. Until Stage 3 the - new loader is internal preparation, not an advertised executable feature. - Never claim later stages are implemented. -4. Run focused tests and the development guide's baseline validation for code - changes. Tests stay offline, deterministic, and free of real credentials. - Reuse existing coverage rather than cloning ConfigKit's parser test matrix. -5. Format changed Go files, inspect the diff, validate affected docs/examples, - record stage status/evidence here, and commit the coherent unit. Do not push, - tag, publish, or add new release gates. +Do not change Go code, module pins, tests, release scripts, CI, accepted ADR +decisions, or published release notes. Do not introduce recurring sync checks, +license approval rounds, release gates, or blanket source headers. Preserve +the application grant and dependency terms. Standard license text must remain +verbatim even if a generic whitespace check dislikes publisher formatting. -### Application Configuration Definition +## Stage 1: Adopt OSS Rules And Establish Licensing Notices -Use a new `Document` type in `internal/config` for the native configuration -shape. It replaces the old on-disk `Config` after CLI cutover. Keep the familiar -YAML field names: +**Status: Pending.** -| YAML path | Go shape | Default and selected meaning | -| --- | --- | --- | -| `prompt_dir`, `profile_dir` | `string` | Empty; no source configured. | -| `schema_dir` | `string` | Existing schema-directory default. Empty remains empty at the application boundary; Promptkit owns any subsequent framework default. | -| `transient_retries` | `*int` | Nil; omitted/null delegates, zero disables additional attempts. | -| `server` | Non-pointer struct | Its children receive typed defaults; null is invalid. | -| `server.addr` | `string` | Existing listen-address default when absent; explicit empty is passed unchanged to the standard HTTP server, not replaced with the Scriptorium default. | -| `server.artifact_root` | `string` | Empty; HTTP file input remains disabled. | -| Three `server.max_*_bytes` fields | `int64` | Existing defaults; explicit zero disables the respective limit; null is invalid. | -| `defaults` | Non-pointer struct | Contains `render_format`; null is invalid. | -| `defaults.render_format` | `string` | Existing text default; resolved value must be text or json. Explicit empty is invalid, not defaulted. | -| `backends` | `map[string]BackendDocument` | Nil; null is permitted, an empty mapping follows ConfigKit semantics. | +Files: `docs/policy/oss.md`, `docs/oss/attribution.md`, +`docs/oss/licenses/CC-BY-SA-4.0.txt`, `COPYRIGHT`, `README.md`, +and the footer portion of `docs/release.md`. -`BackendDocument` keeps `endpoint string`, `api_key_env string`, -`extra_params map[string]any`, `concurrency_limit int`, and -`queue_capacity *int`. Scalar nulls are invalid; nullable maps/pointers retain -native semantics. No catalog IDs or Promptkit fields are added. +1. Read upstream OSS v1.8, COPYRIGHT, the canonical + `examples/licensing/policy-attribution.md`, and referenced licensing + examples at their pinned URLs. Adopt OSS v1.8 with metadata as above. + Remove template-only content; retain upstream common wording and its + existing commit-pinned example links. Customize the README example to + Scriptorium and GPL v3 or later. No other OSS exception is selected. +2. Copy the complete CC-BY-SA-4.0 reference text from the selected revision + into the named local license file and compare it byte-for-byte with that + source. Do not copy the rest of the reference collection or add CC0 merely + because example formatting is used. +3. Reformat the attribution document to the exact v1.8 House Format. Use + `# Open Source Attribution Notices`, immediately followed by entries. + Place Standard Repository Policies first, using the canonical entry, + with its COPYRIGHT, LICENSE, and Source URLs pinned to the selected full + commit. In this stage identify `docs/policy/oss.md` as the newly adapted + material. Do not research or make claims about prior adoption provenance. +4. Preserve the eight existing dependency entries and their imported commits, + copyright wording, SPDX expressions, and required notices. Optional module + identification paragraphs may retain versions currently in source labels. + Remove the introductory prose, missing-copyright filler, and editorial + YAML license explanations. Preserve the factual MIT AND Apache-2.0 + expressions. Use actual upstream filename labels, local license names, + terminal periods, and the exact `[Source](COMMIT_URL).` source form. + Do not upgrade dependencies or relabel their grants. +5. Add a scoped paragraph to COPYRIGHT stating that material adapted from + Standard Repository Policies remains CC-BY-SA-4.0, with + `docs/oss/attribution.md` identifying affected material and links. This + covers the scoped material as stages add it; it does not relicense unrelated + application code or documentation. Keep the existing GPL grant and LICENSE. +6. Correct the README ending to link GPL v3 or later to COPYRIGHT and keep + the attribution link. Change the release-procedure footer to absolute + immutable-tag COPYRIGHT, LICENSE, and ATTRIBUTION links, updating the + references to the number of URLs. Change no publication commands. + Do not retrofit published notes or create a release note/version now. -The fresh defaults factory reads existing `internal/defaults` and format -constants, not a duplicated set of literals. Normalization trims surrounding -whitespace from application directory strings, artifact root, and server -address; cleans only nonempty application paths; and lowercases/trims -`defaults.render_format`. Empty paths remain empty: never let -`filepath.Clean("")` turn disabled/unset into `.`. Record actual changes with -`configkit.Normalize`, using value-free reasons. Do not normalize backend -fields or parameter values on Promptkit's behalf. +Acceptance: OSS text matches the pinned source except the approved project +example and metadata; attribution follows the prescribed format; the added +license text is faithful; project and dependency grants remain unchanged. +Check affected local paths and upstream notice/example destinations. Missing +copyright fields are omitted, not explained with agent-authored commentary. -The read-only validator reports negative application size limits, negative -nonnull application retries, and invalid render format through -`Validation.Error` at their field paths. It does not print rejected values. -Do not call the existing format parser's empty-as-default branch for this field. -Do not add endpoint, backend ID, credential-presence, directory-existence, -address-format, or positive-only-capacity checks. Existing Promptkit and -command-specific validation retains those responsibilities where applicable. +## Stage 2: Adopt Provenance, Agent Guidance, And Documentation Ownership -## Stage 1: Add The Dependency And Native Application Definition +**Status: Pending.** -**Status: Complete.** The pinned dependency, native definition, focused -presence/normalization/validation tests, and commit-specific attribution are in -place. The existing executable loader remains active pending the command cutover. +Files: new `docs/policy/meta.md`, `docs/policy/agents.md`, +`docs/policy/documentation.md`, root `AGENTS.md`, +`docs/development.md`, and the policy attribution entry. -Primary files: `go.mod`, `go.sum`, new focused files under `internal/config`, -and `docs/oss/attribution.md`. +1. Adopt meta v1.0 and agents v1.3 verbatim after template-section removal + and metadata substitution. Keep meta's live project and main-branch + adoption-guide discovery URLs; its own Upstream Source is commit-pinned. + Retain local relative links to developer guidance and attribution. +2. Adopt documentation v1.11. Customize only applicable topic-map rows: + configuration remains application configuration, framework formats belong + to PromptKit, consumers cover executable use and migration rather than a + Scriptorium public Go API, and integrations cover Scriptorium-owned + executable/process contracts. Keep upstream rows for meta, COPYRIGHT, + LICENSE, and attribution. Remove instructional “if applicable” qualifiers + from selected rows. Preserve the rest of the shared language. +3. Use the upstream ADR lifecycle and examples guidance without retaining + generic local expansions. This does not edit accepted ADRs or change their + decisions. Metadata maintenance follows the adoption guide through meta, + rather than a duplicate custom rule set. +4. Adapt upstream `agents/AGENTS.md` into root AGENTS.md. Preserve any actual + repository-specific code-discovery instructions; do not copy environment- + injected instructions into the file just because they appeared in a session. + Do not copy upstream's own root AGENTS.md. +5. Add the adoption guide's Policy Reading Guide table and selection bullets + verbatim to the developer guide with the seven local links. Replace its + unconditional architecture-reading instruction and duplicated policy-reading + prose with this selective guide. Retain project orientation, task-specific + contract rows, dependency boundaries, and baseline validation commands. + Avoid maintaining competing policy trigger rules in multiple places. +6. Expand the policy entry's Material sentence to identify the adopted OSS, + meta, agents, and documentation policies plus adapted root AGENTS.md and + the copied Policy Reading Guide section of docs/development.md. Mark them + adapted for this repository; do not imply the whole developer guide or + unrelated tooling instructions have changed license. -1. Add ConfigKit at the selected release. Retain Promptkit v0.11.0 and its - selected catalogs. Provision dependencies outside offline test execution. - ConfigKit's yaml/v4 release-candidate dependency is the reviewed upstream - choice, not an instruction to upgrade its parser independently. -2. Implement `Document`, `BackendDocument`, the fresh defaults factory, - recorded resolver, and read-only validator described above. Keep the old - loader operational until Stage 3 so this stage remains independently - buildable; this temporary development state is not a shipped compatibility - mode or a second user-selectable configuration path. -3. Test the new definition using real `configkit.Load[Document]` with - `Bytes` and `NoSource`. Cover explicit zero/empty/null distinctions, - retry/queue nil-versus-zero, safe application validation, and a normalized - path whose effective output agrees with the returned value. Include nested - unknown fields and a representative dynamic numeric parameter. -4. Add ConfigKit and yaml/v4 attribution from their selected source material. - Resolve the imported yaml/v4 commit from release/module metadata. Preserve - actual copyright statements; do not infer a project statement from the - FSF license-document copyright. Use applicable SPDX expressions, local - standard texts, and commit-pinned upstream license/notice links. Reuse GPL, - LGPL, MIT, and Apache texts where applicable, including companion links. - Retain yaml.v3 attribution because Promptkit still incorporates it. -5. Document the internal definition as preparation in the relevant internal - guide without claiming the CLI has switched. No current configuration - contract changes yet. +Acceptance: all stage policies have their own correct source link and version; +orientation selects policies by task, including meta for adoption work; local +links resolve; no upstream handbook or example collection was copied. +Generic language differences from upstream are eliminated, not rephrased. -Acceptance: the dependency and definition compile, focused tests demonstrate -native semantics and audited normalization, and attribution follows the OSS -policy. Existing executable behavior still passes its tests. +## Stage 3: Adopt Architecture And Testing With Narrow Local Decisions -## Stage 2: Build Discovery And The Shared Loading Boundary +**Status: Pending.** -**Status: Complete.** Ordered source discovery, one ConfigKit load, typed -diagnostics, and settings conversion are covered by focused file/default and -override tests. Existing commands still use the old entry point until cutover. +Files: `docs/policy/architecture.md`, `docs/policy/testing.md`, +policy attribution entry, and focused internal docs only if moving existing +implementation detail requires it. -Primary files: `internal/config` and its tests; `docs/internal/sources.md`. +1. Read the existing architecture, relevant accepted ADRs, source/adapter + internal guides, and the pinned architecture v1.3 template. Retain upstream + Purpose And Ownership, Architectural Approach, and Adopted Constraints And + Implemented Structure text verbatim. Replace the Application-Specific + Topics placeholder with Scriptorium's existing substantive decisions. +2. Preserve public PromptKit/ConfigKit responsibility boundaries, definition + support, strict decoding, presence handling, HTTP artifact containment, + request independence and concurrency-safe shared capacity, credential/error + boundaries, dependency restrictions, and explicit workflow non-goals. + Retain the existing lexical containment and privileged-host threat scope, + not a stronger filesystem-sandbox promise. Customize the trust discussion + to those decisions rather than retaining instructional example prose. +3. Correct system shape to include run, render, serve, inspect, and config. + Link to internal documentation for concrete package inventory/mechanics. + Where material moves, preserve it in its canonical internal owner without + duplicating it. Do not introduce new wrappers, ports, orchestration, or + package constraints merely to “implement” the architectural recommendation. +4. Adopt testing v1.3 common text verbatim. Add one clearly identified local + test-ownership section for configuration, CLI, HTTP, formatting, architecture + guards, and PromptKit/ConfigKit boundaries. Preserve the existing distinction + between adoption/integration fixtures and upstream exhaustive tests. + Retain a short application-specific note explaining HTTP containment and + shared-state risks where useful; do not replace upstream risk-based coverage + guidance or create percentage targets, quotas, or new test requirements. +5. Add accurate source/version/date metadata and extend the policy attribution + scope to architecture and testing. Record any moved adapted prose in the + Material scope if it is placed outside the already identified files. -1. Add `LoadOptions` with `Path string`, `Explicit bool`, `Overlay string`, - and `Overrides []configkit.Override`, plus - `Load(LoadOptions) (*configkit.Result[Document], error)`. - Discovery chooses a source; the one ConfigKit call owns actual file reading - and decoding. Return the result so consumers retain the frozen inspection - snapshot as well as the typed configuration. -2. With `Explicit=true`, pass the selected path to `configkit.File`; - an explicitly empty path is invalid, not implicit discovery or a fallback - to the old system filename. Do not silently trim a source filename into a - different filename. ConfigKit validates and resolves file sources. -3. With `Explicit=false`, ignore the option's path value and examine the - existing ordered implicit search locations. Only nonexistence permits - examining the next candidate. Other stat failures are errors. Select the - first existing candidate and let ConfigKit load it. A selected file's later - disappearance/read/parse failure does not restart discovery. -4. If neither implicit candidate exists, use `NoSource`. Initially leave - imports disabled and the overlay empty; Stage 4 enables composition. - Defaults-only loads still accept explicit application overrides. Never - reinterpret an empty file as `NoSource`. -5. Pass overrides into the ConfigKit call before decoding/validation; do not - validate a separately merged lower configuration first. Native parsing and - composition failures still follow the upstream lifecycle and are not - bypassed by CLI overrides. -6. Add a small `Settings(Document) AppSettings` conversion for existing - executable wiring. It copies resolved fields, maps the already-validated - format, and converts the backend map into ID-sorted backend settings. - It performs no file IO, normalization, credential lookup, semantic backend - checks, or additional defaults. Dynamic values retain their decoded shapes. -7. Preserve ConfigKit's typed `*LoadError` through wrapping so the CLI can - render safe structured diagnostics. Discovery-only errors are concise and - quote paths safely; no new public error taxonomy is needed. -8. Test ordered discovery with temporary paths rather than host system files, - explicit missing/empty paths, empty-file rejection versus `{}`, - defaults-only loading, source records, override-before-validation behavior, - sorted backend conversion, and preservation of zero/nullable values. - Remove no existing entry point until all its CLI callers migrate in Stage 3. +Acceptance: direct comparison isolates only the roadmap's justified local +decisions; shared text matches upstream. Architecture describes existing +behavior accurately without implementation work. Testing preserves ownership +without duplicating upstream matrices or imposing new coverage gates. -Acceptance: real files and defaults-only inputs produce one reusable result, -runtime conversion agrees with it, and tests do not depend on machine-wide -configuration or secrets. The old and new internals are temporarily buildable, -with no new user-visible mode. +## Stage 4: Adopt Release Policy Without Changing Publication Behavior -## Stage 3: Cut Existing Commands Over And Remove The Legacy Loader +**Status: Pending.** -**Status: Complete.** All five command modes use the shared ConfigKit loader; -the legacy decoder and merge path are removed. CLI and configuration contracts -now describe native YAML and override behavior. Full test, race, vet, build, -and diff checks passed. +Files: `docs/policy/release.md`, `docs/release.md`, attribution entry. -Primary files: `internal/adapter/cli/run.go`, `inspect.go`, -`internal/config/config.go`, and associated tests. +1. Read the current release policy, procedure, script, and hosted workflow; + inspect them read-only. Adopt upstream release v1.4 common wording. +2. Add a compact Scriptorium-specific section preserving the established + selections: main/origin, branch-before-tag publication with exact remote + candidate agreement, annotated tags derived from the checked-in note, + stable releases only, asynchronous Linux binaries/checksums, one race-enabled + offline test invocation, and source-boundary guards. Keep rationale for + selected gates in policy and command/target details in the procedure. +3. Retain upstream's general alternatives verbatim but state which existing + approach Scriptorium selects. Atomic push and source-only release remain + template-supported alternatives, not newly supported Scriptorium workflows. + Preserve successful tag publication as completion; no CI polling, artifact + availability, or consumer installation gate is added. +4. Align the procedure with this allocation without changing its commands, + required checks, supported platforms, or failure-recovery behavior. Confirm + the Stage 1 three-link footer survives hosted-body reuse. Do not run the + publication command or mutate remote state to validate documentation. +5. Add metadata and expand the policy attribution scope to all seven policies. + The final Material description also identifies adapted AGENTS.md and the + developer-guide reading section, plus any actually moved adapted text. -1. Route run, render, serve, inspect prompt, and inspect profile through the - shared `Load` and `Settings` boundary. Preserve the explicit presence of - `--config` in every parser. Centralize the small application-flag mapping - rather than reproduce it in each command. -2. Use `flag.FlagSet.Visit` (or the existing explicit-set helper) to create - one `configkit.Override` per explicitly supplied application flag. Use - `configkit.Keys`, built-in scalar values, and a safe `Flag` label containing - only the flag name. Standard flag parsing's last occurrence supplies the - single override; do not emit duplicate records or parser defaults. +Acceptance: the existing release workflow still satisfies the resulting policy +with no script/CI changes. Source completion, delivery status, immutability, +tag type, and validation obligations are unchanged. All seven source links +identify their own file at the selected full commit. - | Flag | Application path | Existing command scope after cutover | - | --- | --- | --- | - | `--prompt-dir` | `prompt_dir` | run, render, serve, inspect prompt | - | `--profile-dir` | `profile_dir` | run, render, serve, inspect profile | - | `--schema-dir` | `schema_dir` | run, render, serve | - | `--addr` | `server.addr` | serve | - | `--artifact-root` | `server.artifact_root` | serve | - | `--max-request-bytes` | `server.max_request_bytes` | serve | - | `--max-artifact-bytes` | `server.max_artifact_bytes` | serve | - | `--max-response-bytes` | `server.max_response_bytes` | serve | +## Stage 5: Verify Language Parity And Complete The Documentation Handoff - Register the schema-directory flag on render as well as run/serve so the - advertised application path override is consistently usable. This is not a - general expansion of runtime/provider flags to other commands. -3. Keep `--format`, `--out`, request `--transient-retries`, prompt/profile - selection, model overrides, timeouts, inputs, variables, and reporting - options outside the application configuration tree. In particular, - `render --format` overrides that invocation's output format without - rewriting `defaults.render_format`; omitted format inherits its resolved - application value. Existing invocation-only parsing remains unchanged. -4. Remove post-load directory trimming/cleaning/defaulting from argument - finalizers, including unconditional schema-path cleaning. Keep only - invocation-owned transformations such as output-file paths. Explicit - empty profile/artifact-root overrides must clear file-provided values; - explicit empty prompt directory must reach the existing execution - prerequisite rather than revive the lower value. -5. Keep existing command prerequisites: run/render/serve and prompt inspection - need their effective prompt directory as currently required; profile - inspection does not acquire a prompt-directory requirement. All commands - still perform the common application schema validation. Do not introduce - eager filesystem or credential validation for inspection. -6. Add one CLI diagnostic path using `errors.As` and - `configkit.WriteDiagnostics` for load errors. Carry successful load warnings - to the command boundary and render them once on stderr before execution or - output. Handle diagnostic writer errors as command failures. Do not flatten - structured failures into raw parser text or print rejected values. - Preserve existing process exit classifications; application/parse/load - failures are exit 1, not generated-output validation exit 2. -7. Remove the old YAML decoder, `strictInteger.UnmarshalYAML`, old on-disk - types, `applyConfig`, `ApplyCLIOverrides`, and unused compatibility - sentinels/helpers after migrating all callers/tests. Keep only one defaults - factory. Remove the direct yaml.v3 import; module tidy may retain yaml.v3 - as an indirect Promptkit dependency. -8. Update tests to the deliberate breaking semantics, not the legacy behavior. - Protect explicit empty path clearing, scalar-null rejection, unknown fields, - rejected empty/multi-document input, one representative YAML policy change, - unchanged request-versus-engine retry precedence, absent optional - credentials, and command-specific inspection prerequisites. -9. Update configuration and CLI contracts, source/adapter internals, architecture - ownership, and the contributor reading guide for the delivered cutover. - Preserve existing application defaults, Promptkit definition support, - HTTP containment, shared capacity, and report behavior. Adjust the current - CLI claim that every command requires a prompt directory to match actual - command-specific requirements. +**Status: Pending.** -Acceptance: all existing commands use ConfigKit, no legacy parser or second -application merge remains, relevant tests pass, and diagnostics and docs describe -the native breaking behavior. Imports/overlays and new inspection commands are -not yet advertised. +Files: this plan, the feature roadmap, and only affected documentation needing +correction within the preceding stages' scope. -## Stage 4: Enable Explicit Imports And Selected Configuration Overlays +1. Compare all seven adopted documents directly with their selected upstream + counterparts. Account for metadata, removed template sections, approved + substitutions, and the narrow local sections. Correct every remaining + non-substantive wording difference. Do not turn this comparison into a + permanent script, CI requirement, or release gate. +2. Check there are no adoption placeholders, copied template-only instructions, + absent local example links, stale BSD descriptions of Scriptorium itself, + or contradictory selective-reading directions. Legitimate BSD dependency + attribution remains. Validate links using Markdown-aware handling so + illustrative syntax in inline code is not treated as a broken real link. +3. Confirm per-document source metadata, the policy entry's exact scope, + COPYRIGHT's scoped license treatment, existing dependency grants/notices, + and the byte-for-byte new license text. Keep example/source links pinned + and public discovery links current. Check footer paths in the candidate; + unpublished release tags need not resolve online. +4. Review affected roadmap references to implementation.md. Where a link + specifically means an old completed plan, replace it with a clear historical + Git reference from this plan, not this new policy sequence. Preserve + deferred application features and historical completion evidence. + Current release directions link to the canonical procedure, not an old + two-link footer. Do not rewrite published release notes. +5. Mark only actually delivered stages complete with concise evidence here. + Update the feature roadmap's status to delivered and keep its end-state + focus. Record the next-release summary described above without choosing a + version, creating a tag, or claiming publication. +6. Run diff/whitespace and affected-document/link checks. Do not reformat + verbatim license text to satisfy a style check. No Go test/build rerun is + necessary for this documentation-only work; no executable example or + publication command needs changing. Inspect status and staged diff, commit, + and report the commit and remaining work under the agent policy. -**Status: Complete.** File-backed loads support explicit imports and one -selected overlay under the selected file's canonical directory. All five -existing command modes accept `--config-overlay`; real-file composition and a -render workflow are tested. The composed example and contracts are updated; -full test, race, vet, build, and diff checks passed. - -Primary files: shared loader, CLI source-selection wiring, configuration and -CLI tests, maintained configuration examples. - -1. Enable `Options.Imports` for file-backed loads. Leave it false for - `NoSource`, which rejects imports. Use ConfigKit's default canonical - selected-file directory as the root; do not supply a wider filesystem root - or add application traversal code. -2. Wire `LoadOptions.Overlay` into `Options.Overlay`. Register - `--config-overlay` on all five existing configuration-loading command - modes. Treat the value exactly as an upstream overlay name: no trimming, - case folding, extension fallback, path expansion, or local inheritance. - Empty and exact lowercase default mean no active overlay under v0.4.0. - A requested active overlay without a selected base source fails through - ConfigKit; do not manufacture an empty base file or root. -3. Use native `configkit.imports` directives for explicit paths/globs. The - application enables the feature without requiring a second CLI opt-in; - absent directives cause no discovery. Retain conflict-based base - composition and native overlay replacement/null semantics. -4. Preserve working-directory-relative application path values even when - authored in fragments or overlays. Configuration-source containment is - not the HTTP artifact policy and is not applied to prompt/schema paths. -5. Add focused real-file tests for disjoint backend fragments, an import - conflict, overlay replacement, explicit CLI precedence, missing selected - overlay, and no-base overlay failure. Verify `--profile` still selects a - Promptkit execution profile independently of `--config-overlay`. - One real command workflow proves wiring; upstream owns exhaustive glob, - symlink, cycle, and parser-limit tests. -6. Add a small synthetic import/overlay example under `examples/`; preserve - a simple single-file quickstart. Update config/CLI contracts and operations - guidance for the new features, root selection, and relative-value policy. - -Acceptance: supported commands load the same native composition pipeline; -imports are explicit, overlays remain distinct from execution profiles, and no -new scanning, remote acquisition, or path policy is introduced. - -## Stage 5: Add Application Validation, Effective Output, And Source Inspection - -**Status: Complete.** Application-only validation, effective YAML, and ordered -JSON source inspection use one shared ConfigKit result without an engine. Tests -cover source composition, CLI provenance, stream failures, and the credential -reference boundary. Public and internal contracts are updated; full test, -race, vet, build, and diff checks passed. - -Primary files: a focused `internal/adapter/cli/config.go`, shared config -definition, CLI dispatch/help/tests, and canonical CLI/config documentation. - -1. Apply the maintainer's selected inspection policy: no redaction for - `extra_params`, including historical values. API-key values are the only - relevant secrets, but this schema stores only `api_key_env` reference names. - Keep those names visible and never resolve them in the ConfigKit load. - Do not add secret tags to the reference name or parameter container, a - catch-all secret predicate, sensitive-key guessing, or a selective masking - interface. Actual credentials remain with Promptkit; raw API-key fields - remain rejected. Use upstream inspection renderers without adding a second - value-rendering path or a synthetic credential field. -2. Add a top-level `config` command family using the existing standard-library - flag style. The selected interfaces are: - - | Command | Success stdout | - | --- | --- | - | `scriptorium config validate [flags]` | Empty; exit status communicates application validation success. | - | `scriptorium config show [flags]` | `Result.WriteEffective` redacted YAML. | - | `scriptorium config sources [flags]` | One newline-terminated JSON array of source records. | - | `scriptorium config explain --path [flags]` | Added in Stage 6. | - - These are prospective interface specifications, not commands to execute - before their stages are implemented. -3. All configuration subcommands accept `--config`, `--config-overlay`, - and the eight application override flags in Stage 3's table, regardless - of which execution command normally uses them. They do not accept request, - credential, backend-editing, output-format, or output-file flags. Shell - redirection is sufficient for these inspection outputs. No positional - arguments are accepted. -4. Load exactly once through the common loader. Do not construct a Promptkit - engine, restricted artifact reader, listener, or client. Missing prompt - directories, unselected/absent credentials, and framework backend validity - are not additional application-inspection gates. Invalid application - types/ranges/format still fail. Validation describes application load - validity, not end-to-end executability. -5. For sources, map `Result.Sources()` to a small local DTO with JSON keys - `id`, `label`, `layer` (always present), and `path` (omit when empty). - Preserve upstream source order; emit `[]\n` for defaults-only input without - overrides. Include synthetic CLI sources where present. Do not marshal - opaque ConfigKit implementation objects. The standard JSON encoder escapes - control characters; no custom terminal rendering is necessary. -6. Diagnostics/warnings go only to stderr through the shared diagnostic helper. - Successful operations exit 0; parse/load/output/diagnostic-write failures - exit 1. No exit 2 meaning is introduced for configuration validation. - Ordinary writer failures may leave partial output; do not promise atomic - streams or buffering of an entire configuration. -7. Test defaults-only show/sources, a composed load with CLI provenance, - explicit empty clearing, normalized effective values, failed loads producing - no successful stdout, and writer failure. Set a synthetic API-key environment - variable: effective output must show its reference name, not its value. - Repeat with the variable absent to establish that inspection does not require - credentials. Assert that ordinary synthetic parameter values remain visible - and runtime ExtraParams stay intact; no real environment or provider is used. -8. Update CLI/config contracts, README navigation, subprocess integration, - operations, and internal inventory to describe only these delivered modes. - Retain the README's license/attribution ending. - -Acceptance: validation, effective output, and source inspection use the same -resolved configuration as execution, preserve stream separation, and honor the -selected API-key boundary without masking or changing backend parameter data. - -## Stage 6: Add Per-Path Explanation And Verify Credential Boundaries - -**Status: Complete.** `config explain` uses required JSON Pointer paths and -ConfigKit's snapshot renderer, including historical and normalized values. -Tests cover root, escaped/numeric keys, unknown paths, writer failures, and -synthetic credential-reference boundaries. Full test, race, vet, build, and -diff checks passed. - -Primary files: configuration command implementation/tests and CLI/config docs. - -1. Implement `config explain` with the same selectors and override flags as - Stage 5 plus a required, explicitly supplied `--path`. Parse using - `configkit.ParsePath`; syntax is JSON Pointer, not dotted notation, a file - path, or a URL fragment. Explicit `--path ""` selects the root; an omitted - flag is an error. Preserve escaped and numeric mapping-key semantics. -2. Use the successful shared result's `WriteExplanation`. Explain known - effective and historical paths, including removed values. An unknown or - malformed path exits 1 with no successful explanation output. Quote - user-supplied paths safely if included in application diagnostics. - Do not rebuild history, look up environment values, or reload sources. -3. Keep the upstream human-readable renderer and its redaction semantics rather - than inventing a JSON explanation schema or freezing the entire rendered - prose as an application guarantee. Writer failures exit 1 and can leave - partial output, as with effective output. -4. Test a value's default/base/overlay/CLI history, recorded normalization, - explicit equal-value writes, an escaped backend key, root selection, and - an unknown path. Cover a displaced historical backend parameter value: - it remains visible, including in a container/root explanation. Use distinct - synthetic environment names and key values to verify reference names remain - explainable but resolved API-key bytes are absent from all history/output. -5. Update CLI/config docs and operations guidance with path syntax, - historical-value meaning, representation differences, and sensitive - identifiers/paths. Do not claim that redaction detects arbitrary secrets - outside classified paths. - -Acceptance: explain uses the existing snapshot and upstream semantics; its -history matches the selected inputs and normalization, parameters remain -visible, and explanation neither resolves nor discloses API-key values. - -## Stage 7: Verify Integration, Finish Documentation, And Record Release Handoff - -**Status: Complete.** The feature roadmap, command paths, dependency boundary, -documentation, and attribution were audited. Maintained single-file and -composed examples passed configuration inspection and no-provider rendering; -the optional-credential/integer-parameter backend path passed with a local -provider. The full test, race, vet, build, formatting, link, and diff checks -passed. Release preparation is recorded below; no tag or publication occurred. - -1. Review the feature roadmap against actual code. Confirm all five original - command modes and all four new configuration subcommands use one definition - and load pipeline. Remove any transitional loader, duplicated defaults, - unused compatibility helpers, or post-snapshot configuration mutation. - Do not add architecture machinery merely to assert a particular file layout. -2. Exercise maintained single-file and import/overlay examples through the - implemented configuration commands and a no-provider render workflow. - Use focused backend integration evidence that dynamic integer parameters - reach Promptkit correctly and missing optional credentials retain their - established meaning. Use a local HTTP server only where provider serialization - must be checked; never a real service or credential. -3. Retain existing HTTP concurrency, containment, error mapping, retry, - timeout, and report tests. Add a representative configuration-to-server - wiring test only if existing coverage misses changed settings propagation; - do not reproduce all upstream scheduling or HTTP cases. -4. Run the development baseline on final code: `go test ./...`, - `go test -race ./...`, `go vet ./...`, and - `go build ./cmd/scriptorium`. Check formatting, affected examples/links, - and `git diff --check`. Do not stage the built binary or scratch files. - Compare any new standard license text byte-for-byte; preserve upstream - whitespace rather than editing license wording to satisfy a style check. -5. Ensure current configuration, CLI, subprocess, operations, internal docs, - contributor routing, and architecture ownership agree. Testing policy - should identify ConfigKit as the owner of exhaustive upstream loading tests, - while Scriptorium owns application adoption tests. Change policy dates when - revising policies, not base versions unless adopting new templates. - Keep accepted ADR text and published release notes immutable. -6. Record the release handoff below and mark only delivered stages complete. - Update the feature roadmap's status, including the resolved inspection - decision. Preserve deferred work. The next breaking release follows the - release policy; version selection, candidate publication validation, release - notes at that version, tag creation, and publishing are not this sprint. -7. Commit the final coherent handoff and report validation evidence, remaining - limitations, and any outstanding work without claiming source/binary - publication or complete legal compliance. - -Acceptance: the ConfigKit-native target is delivered with proportionate -regression evidence and accurate documentation, and the workspace contains no -temporary dependency substitutions or uncommitted implementation artifacts. - -### Release Handoff To Retain - -The upcoming release needs migration guidance for empty files versus `{}`, -explicit empty values, nullable versus scalar fields, strict YAML/scalars, -reserved keys and upstream size limits, and the invalid explicit-empty config -path. Include imports/overlays, application path interpretation, CLI presence, -inspection command scope, and the selected API-key-only secret policy. Explicit -empty server address reaches standard HTTP server behavior instead of restoring -Scriptorium's default; operators should supply their intended listen address. - -Keep request retry overrides and Promptkit credential semantics distinct from -application settings. Explain that configuration validation is not a provider -probe or complete Promptkit definition validator, and that inspection output -is not guaranteed reloadable. Include dependency/attribution changes and the -existing LICENSE | ATTRIBUTION footer. Preserve the still-relevant Promptkit -v0.11 release handoff identified above when preparing a combined next release. - -### Implementation And Release Preparation Record - -The ConfigKit v0.4.0 adoption is implemented in committed source. The next -breaking release still needs version selection, version-matched release notes, -candidate validation, tagging, and publication under the release policy; none -has occurred as part of this work. Keep the earlier Promptkit v0.11 handoff -linked above when preparing a combined release. - -The release note should summarize the deliberate configuration migration: -empty files now fail and `{}` means an empty mapping; explicitly empty strings -and zero values override lower values; `null` follows declared nullable types; -strict YAML rejects additional documents, anchors/aliases, unsupported scalar -forms, and unknown fields; `configkit` is reserved and ConfigKit's source and -resource limits apply. Explicit `--config ''` is invalid. Imports are opt-in -per file directive, while `--config-overlay` selects one named overlay under -the selected file's canonical directory; application path *values* remain -process-working-directory-relative. Only explicitly supplied application CLI -flags override configuration. An explicit empty server address is passed to -the standard HTTP server and does not restore Scriptorium's default. - -Document the four application-inspection modes and the selected policy: -`extra_params`, including history, and `api_key_env` reference names remain -visible, but actual API-key bytes are never resolved by ConfigKit. Inspection -output can be sensitive and is not guaranteed reloadable. Application -validation neither probes a provider nor completely validates Promptkit -definitions. Keep request retry overrides and Promptkit's credential rules -separate from application settings. Include the new dependency and attribution -notices, and retain the existing LICENSE | ATTRIBUTION footer. Deferred -secrets bootstrap, schema versioning, reload, and remote acquisition remain -deferred; do not imply source or binary publication has happened. +Acceptance: policy language is standardized with only justified local +customization; notice/provenance arrangements describe reality; no runtime, +dependency, publication, or historical-release change slipped into the diff. ## Open Questions -None remain for this scope. On 2026-09-26 the maintainer selected no redaction -for `extra_params` and specified that the API key is the only secret requiring -ConfigKit redaction. Stages 5–6 implement that decision through visible parameter -values/reference names and the existing separation of actual credentials from -application configuration. Do not reopen it or substitute blanket masking. - -This does not claim automatic detection of credentials embedded in arbitrary -strings or provider parameters. The application schema still rejects raw API-key -fields, and Promptkit remains responsible for resolving named credential sources. +None. The maintainer approved the public discovery URLs, the standard +CC-BY-SA arrangement for copied policy material, the selected upstream revision, +and the justified local variations. The existing application grant, dependency +grants, and release approach remain unchanged; do not reopen them merely to +perform this adoption. diff --git a/docs/roadmap/policy.md b/docs/roadmap/policy.md index 58f757a..7fdacf2 100644 --- a/docs/roadmap/policy.md +++ b/docs/roadmap/policy.md @@ -5,7 +5,8 @@ Agreed on 2026-09-28; implementation is pending. This roadmap replaces the earlier licensing-only proposal with alignment of the complete applicable policy set and its supporting documentation. It defines the targeted end state -and required changes, not staged implementation instructions. +and required changes; the [implementation plan](implementation.md) owns ordered +stages and their verification. The goal is identical policy language across adopting projects except where a substantive, locally significant requirement or decision warrants customization. @@ -204,10 +205,11 @@ Completion means: - Policy attribution and document-license scope match the material actually incorporated; required standard texts are complete and unmodified. - README, attribution, release navigation, and still-operative roadmap - handoffs agree with the new conventions. Supersede stale instructions in - [the implementation record](implementation.md) without rewriting historical - completion evidence. Preserve the earlier author-selected license change - in the next release's preparation context; publication remains separate work. + handoffs agree with the new conventions. Historical implementation evidence + remains available through the Git references in the implementation plan, + without making old footer directions current policy. The earlier + author-selected license change remains in the next release's preparation + context; publication remains separate work. - Review checks the resulting rules, source/example URLs, notice facts, local links, license-text fidelity, and final diff. Runtime tests are unnecessary for documentation-only changes. A Git adoption record is sufficient; no