From db8db5ffc5444ac2e0cd35d824f2c993cf168773 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sun, 26 Jul 2026 12:08:53 -0500 Subject: [PATCH] Add a plan to implement improved documentation and support for running notarius as a subprocess --- docs/roadmap/implementation.md | 284 +++++++++++++++++++++++++++++++++ docs/roadmap/subprocess.md | 208 ++++++++++++++++++++++++ 2 files changed, 492 insertions(+) create mode 100644 docs/roadmap/implementation.md create mode 100644 docs/roadmap/subprocess.md diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md new file mode 100644 index 0000000..b2dcdf2 --- /dev/null +++ b/docs/roadmap/implementation.md @@ -0,0 +1,284 @@ +# Subprocess Integration Implementation Plan + +## Status + +Ready for implementation. + +## Objective + +Implement the accepted [Subprocess Integration Contract](subprocess.md) as a +small, generic extension of the existing CLI boundary. The completed work must +let a subprocess caller discover a successful Notarius output bundle through a +versioned JSON receipt without parsing human prose, while preserving current +pipeline, output, rejection, warning, configuration, and interactive CLI +behavior. + +Complete the stages below in order. Each stage must leave its affected package +passing before the next begins. Do not add Narratio-specific production code, +a public Go client, a new output encoder, or any feature listed as out of scope +in the feature roadmap. + +## Decisions And Invariants + +- `--json` is a boolean flag on `notarius run`; it does not affect pipeline + resolution, execution, checkpoint identity, output encoding, or publication. +- The machine result is a private CLI Go type implementing the public + `notarius.run-result.v1` wire contract. Do not expose framework or CLI Go + packages for external import. +- The finalized run manifest is authoritative for `run_id`, `pipeline_id`, and + `validation_status`. Verify that its pipeline ID matches the effective + resolved pipeline rather than emitting conflicting provenance. +- Human-oriented stdout remains byte-for-byte governed by its current path when + `--json` is absent. Machine mode emits no human status or debug-path line on + stdout. +- Warnings and errors remain on stderr. Exit statuses remain 0 for success, 1 + for runtime failure, and 2 for syntax failure. +- Validation rejection remains a successful pipeline outcome. The receipt + reports counts but does not decide which lanes an orchestrator requires. +- The receipt points to the existing bundle; it does not duplicate output + descriptors, lane payloads, warnings, rejections, or manifest content. +- Reported filesystem paths are lexical absolute paths produced with + `filepath.Abs`. Do not resolve symlinks or change the physical directories + used for publication. +- For the production `json` output module, `index_file` is exactly + `index.json` and is included only after confirming that the runner returned + exactly one logical file with that clean name. For any injected or future + non-`json` output module, omit `index_file`; the machine receipt remains + generic. +- Machine-result construction and JSON encoding occur before debug success + terminalization, but stdout writing occurs only after output publication and + successful requested debug terminalization. +- Standard output is not transactional. A final writer failure returns exit 1 + and may leave partial bytes. Consumers must ignore stdout unless the process + exits 0. Because debug terminalization must precede receipt emission, a + result-delivery failure does not rewrite an already persisted successful run + report; it is additionally reported on stderr and the process exits 1. +- Do not add a stable JSON error envelope or promote diagnostic wording into a + compatibility contract. + +## Stage 1: Define The Run-Result Wire Model + +Add a focused private implementation under `internal/cli`, preferably +`run_result.go`, with: + +- a constant for `notarius.run-result.v1`; +- a private result struct whose JSON fields and requiredness match the feature + roadmap; +- a constructor that accepts the resolved pipeline, completed runner output, + run output directory, and optional debug directory; and +- serialization into an owned byte slice containing exactly one compact JSON + object followed by one newline; and +- a complete-write helper for delivering those prepared bytes. + +The result fields are: + +- required `schema_version`, `run_id`, `pipeline_id`, `output_directory`, + `normalized_output_count`, `rejected_output_count`, and `warning_count`; +- required non-empty `validation_status`, copied without reinterpretation from + the final run manifest; +- optional `index_file`, set to `index.json` for the resolved production JSON + output module; and +- optional `debug_directory`, present only when debug capture was requested and + its path is non-empty. + +The constructor must: + +1. validate the required run ID, pipeline ID, final validation status, and + non-empty run output directory rather than emit a malformed receipt; +2. verify that the final manifest pipeline ID matches the effective resolved + pipeline ID; +3. convert the output bundle and optional debug bundle paths to lexical absolute + paths; +4. derive all counts from the completed `pipeline.RunOutput`; +5. identify the output module through the resolved module key, not concrete + encoder types or D&D knowledge; and +6. for module key `pipeline.DefaultOutputModule`, verify exactly one returned + logical output file is named `index.json` before setting `index_file`. + +Treat a missing or duplicate `index.json` from the production JSON output as a +runtime contract error. Do not inspect or decode the index contents here; the +output encoder and its existing tests own that format. + +Implement the writer with `encoding/json` and complete-write handling. It must +surface encoding errors, zero-progress writes, short writes, and underlying +writer errors to the command boundary. The command must map a delivery failure +to a fixed, bounded result-write error without wrapping caller-supplied writer +text into user or debug diagnostics. Do not add an injected serializer or a +generic serialization framework merely to manufacture unreachable error cases +for tests. + +Add lean focused tests for the wire-model boundary: + +- required fields, counts, schema identity, and compact newline-terminated JSON; +- blank required manifest identities, blank validation status, and a mismatch + between final-manifest and effective pipeline IDs; +- lexical absolute conversion for relative output and debug paths; +- production JSON entry-point discovery; +- omission of `index_file` for a non-JSON output module; +- missing and duplicate production entry points; and +- a representative failing or zero-progress writer. + +Decode structured output in tests rather than snapshotting an entire JSON +string. Assert literal field names and the schema identity because they are the +public compatibility contract. + +Stage completion: + +- `go test ./internal/cli` + +## Stage 2: Integrate Machine Mode With Run Finalization + +Update the run command in `internal/cli/run.go`: + +- register `--json` with the existing `flag.FlagSet`; +- include it in root usage without adding it to `runFlagTakesValue`; +- leave all existing flag combinations valid; and +- do not add configuration or environment equivalents. + +After the runner succeeds and requested debug summaries have been written, +apply this exact finalization order: + +1. when `--json` is selected, construct and encode the complete receipt into + owned memory; fail before physical output publication if its required fields + or production entry point are invalid; +2. publish the runner's logical output files through the existing confined + output writer; +3. terminalize requested debug reporting as successful and stop without + writing stdout if terminalization fails; +4. in machine mode, write the prepared receipt to stdout and return a runtime + failure if that write does not complete; +5. otherwise, use the unchanged human-oriented success reporting path; and +6. report successful-run warning counts to stderr as today. + +The receipt's `output_directory` is the absolute path to the run-specific +bundle, not the configured output root. `debug_directory` is the absolute path +to the allocated run-specific debug bundle. + +On a machine-result stdout failure after terminalization: + +- report a fixed, bounded, code-owned result-write error on stderr through the + existing CLI failure reporting surface without including the underlying + writer error; +- include the existing debug-path discovery line when applicable; +- return exit 1; +- do not attempt terminal reporting a second time; and +- leave the successfully published output and debug bundle intact. + +Do not change `pipelineCommandState`, the debug run-report wire shape, the +output encoder contract, or `writeOutputFiles` solely to support this feature. +The machine receipt belongs to CLI reporting after the runner and output module +have completed their existing responsibilities. + +Stage completion: + +- `go test ./internal/cli` + +## Stage 3: Protect The Public CLI Contract + +Add or extend behavior-level CLI tests at the narrowest stable boundaries. +Reuse existing production components, fakes, state harnesses, and maintained +examples rather than creating a second subprocess fixture framework. + +Cover: + +- a representative maintained production invocation with `--json`, decoding + exactly one stdout document and verifying the schema identity, run and + pipeline IDs, absolute output directory, `index_file`, counts, validation + status, and the existence of the referenced bundle entry point; +- a warning-bearing debug run, proving a correct warning count, warning + reporting on stderr, an absolute optional debug path, and no human prose in + stdout; +- a rejection-bearing successful run using an explicit deterministic rejecting + validator, proving zero or partial normalized outputs and the correct + rejection count without changing exit status; +- one representative syntax failure and one representative runtime failure + with `--json`, proving the established exit class, stderr ownership, and no + completed success document; +- an injected stdout writer failure after successful publication, proving exit + 1, retained output files, fixed diagnostic reporting, and omission of a + recognizable writer-error sentinel from stderr and debug diagnostics; and +- the existing no-`--json` tests continuing to protect interactive output. + +For the writer-failure case, use a writer that fails before accepting bytes when +asserting empty stdout. The public contract nevertheless remains that arbitrary +writers may leave partial bytes and consumers must ignore stdout on nonzero +exit. + +Do not: + +- duplicate the JSON output encoder's lane, manifest, warning, or rejection + serialization matrix; +- assert complete human error strings; +- add a golden file for the small receipt; +- test private helper call order; or +- add a prompt-language or unrelated end-to-end test. + +Stage completion: + +- `go test ./internal/cli` +- `go test ./internal/modules/generic/output/json` +- `go test ./internal/modules/integration` + +## Stage 4: Publish The Consumer-Facing Documentation + +After the code and behavior tests pass, update the canonical current-behavior +documentation: + +- `docs/cli.md` + - add `--json` to run syntax and its flag table; + - define stdout/stderr behavior and the requirement to parse machine stdout + only after exit 0; + - preserve the existing exit-status table and link to the run-result + contract. +- `docs/integrations/run-result.md` + - own the complete `notarius.run-result.v1` field table, requiredness, + example, path semantics, production `index_file` rule, partial-write rule, + and additive compatibility policy; + - link to the published JSON bundle contract rather than repeating its + descriptors or payload schemas. +- `docs/integrations/json-output.md` + - add only a narrow cross-link explaining that a subprocess caller obtains + the physical bundle root from the run-result receipt before using + `index.json` for logical discovery; + - do not duplicate run-result fields or stream semantics. +- `docs/consumers/subprocess.md` + - provide a concise task workflow for preflight, invocation, separate stream + capture, exit checking, receipt decoding, confined index resolution, lane + lookup by ID, media-type and schema-identity checks, required-versus-optional + consumer policy, provenance retention, and sensitive-data handling; + - use a generic orchestrator example with placeholders, not private Narratio + paths, credentials, or a duplicate complete configuration. +- `docs/internal/cli.md` + - document receipt construction before publication, emission after + publication and debug terminalization, and the stdout failure behavior; + - preserve the CLI composition-root boundary. +- `README.md` and `docs/development.md` + - add only the smallest links needed to make subprocess consumer guidance and + its implementation owner discoverable. + +Do not otherwise change `docs/config.md`, `docs/operations.md`, or the JSON +output bundle contract unless implementation reveals an actual change to +behavior they own. The machine receipt adds no configuration field, filesystem +surface, retention policy, or lane-output shape. + +After all code, tests, and current-behavior documentation are complete: + +- change [the feature roadmap](subprocess.md) status to `Implemented`; +- change this plan's status to `Completed`; and +- ensure neither roadmap is used as the canonical description of current + behavior. + +Stage completion: + +- validate documentation links and command examples against the implementation; +- `git diff --check` +- `go test ./...` +- `go vet ./...` +- `go build ./cmd/notarius` +- `go test -race ./internal/cli` + +## Open Questions + +None. The decisions above resolve stdout partial-write behavior, output-module +generality, path normalization, partial pipeline success, terminal reporting, +documentation ownership, and test boundaries. diff --git a/docs/roadmap/subprocess.md b/docs/roadmap/subprocess.md new file mode 100644 index 0000000..403ed4b --- /dev/null +++ b/docs/roadmap/subprocess.md @@ -0,0 +1,208 @@ +# Subprocess Integration Contract + +## Status + +Accepted for implementation. + +## Purpose + +Make Notarius straightforward to invoke as a subprocess from an orchestrator +such as Narratio. A caller should be able to run a configured pipeline, discover +the published output bundle without parsing human prose or scanning a +directory, and hand selected structured artifacts to a later stage. + +This work strengthens the public CLI boundary. It does not turn Notarius into a +Go library, embed Narratio-specific behavior, or change pipeline execution and +artifact semantics. + +## Desired End State + +A subprocess caller can: + +1. validate a Notarius configuration and selected pipeline before execution; +2. invoke `notarius run` with explicit input, output-root, session, and + reference arguments; +3. request one versioned, machine-readable success result on standard output; +4. use that result to locate the published output bundle; +5. discover normalized lane payloads through the bundle's authoritative + `index.json`; +6. distinguish process failure from successful partial pipeline outcomes; and +7. record Notarius run provenance in its own manifest without depending on + internal packages, cache formats, debug formats, or human-readable messages. + +The existing human-oriented command output remains the default for interactive +use. + +## Machine-Readable Run Result + +`notarius run` supports `--json`. On success, the flag makes standard output +contain exactly one JSON object followed by a newline. No human-oriented status +line is mixed into that stream. + +The result uses the schema identity `notarius.run-result.v1` and contains: + +| Field | Presence | Meaning | +| --- | --- | --- | +| `schema_version` | Required | Exactly `notarius.run-result.v1`. | +| `run_id` | Required | The Notarius run identifier. | +| `pipeline_id` | Required | The effective pipeline identifier. | +| `output_directory` | Required | Absolute path to the successfully published output bundle. | +| `index_file` | Required for the production JSON output | Logical bundle path `index.json`. | +| `normalized_output_count` | Required | Number of final normalized lane outputs returned by the pipeline. | +| `rejected_output_count` | Required | Number of recorded rejected outputs. | +| `warning_count` | Required | Number of final run warnings returned by the pipeline. | +| `validation_status` | Required | The run manifest's final validation status without reinterpretation. | +| `debug_directory` | Optional | Absolute debug-bundle path when debug capture was requested and completed. | + +An illustrative successful result is: + +```json +{ + "schema_version": "notarius.run-result.v1", + "run_id": "run-1770000000000000000-0123456789abcdef0123456789abcdef", + "pipeline_id": "dnd-session", + "output_directory": "/srv/narratio/runs/session-7/notarius/run-1770000000000000000-0123456789abcdef0123456789abcdef", + "index_file": "index.json", + "normalized_output_count": 6, + "rejected_output_count": 2, + "warning_count": 1, + "validation_status": "approved" +} +``` + +The receipt is a discovery and summary document, not a duplicate output +envelope. It does not embed lane payloads, rejection entries, warnings, the run +manifest, or output-file contents. Consumers use `index_file` and the existing +published JSON output contract for those records. + +The result contract must tolerate future additive optional fields. Any +incompatible field or semantic change requires a new run-result schema version. + +## Stream, Publication, And Failure Semantics + +Machine-readable output is emitted only after: + +- the pipeline has completed without a framework error; +- all logical output files have been successfully published; +- requested debug terminal reporting has completed; and +- all result fields are known. + +Writing or encoding the machine-readable result is part of successful command +completion. Failure to write it produces the existing runtime-failure exit +class. + +With `--json`: + +- successful stdout is exclusively the run-result JSON document; +- successful warnings remain on stderr under the existing CLI contract; +- syntax and runtime errors retain their existing exit statuses and stderr + diagnostics; +- consumers treat stdout as a valid result only when the process exits with + status 0; failures before result writing emit no result, while a failure + during the stdout write may leave incomplete bytes that must be ignored; and +- human-readable diagnostic wording is not promoted into a machine contract. + +Without `--json`, current interactive stdout and stderr behavior remains +unchanged. + +Successful runs may contain rejected outputs or omit some normalized lanes. +That remains a valid pipeline outcome. The run result reports counts, while +`index.json`, `rejected.json`, and `warnings.json` remain authoritative for +details. Notarius will not add a generic `--fail-on-rejection` policy as part +of this work. + +## Output Discovery And Consumer Responsibilities + +The production JSON encoder's `index.json` remains the authoritative mapping +from lane IDs to published payloads. A subprocess consumer should: + +- resolve `index_file` beneath `output_directory` and reject path escape; +- locate expected outputs by `lane_id`, not by guessing filenames; +- check each selected descriptor's media type and schema identity; +- decode payloads according to their published integration contracts; +- decide which lanes are required or optional for its own later stages; and +- retain rejection, warning, and manifest files when they are needed for + provenance or review. + +For Narratio, required report inputs and partial-success policy remain Narratio +stage configuration and orchestration concerns. Notarius does not acquire +knowledge of Narratio stages, manifests, workspace layout, publication policy, +or report formats. + +## Invocation Guidance + +The consumer documentation recommends that subprocess callers: + +- use `notarius config validate --pipeline` as an optional preflight; +- pass explicit absolute paths for the input, configuration, output root, and + CLI-supplied references; +- use a stable, non-secret prompt session identifier when useful for provider + routing or caching; +- capture stdout and stderr separately; +- supply credentials through the configured environment mechanism rather than + command arguments or generated configuration containing secret values; +- place output, cache, debug, and subprocess logs under intentional + sensitivity and retention policies; and +- treat the Notarius manifest and run-result receipt as provenance while + leaving the caller's own manifest authoritative for its stage lifecycle. + +Notarius configuration remains owned by Notarius. An orchestrator may select a +configuration and pass supported operational overrides, but should not +duplicate the complete Notarius configuration schema. + +## Documentation End State + +- `docs/cli.md` owns `run --json`, stream behavior, and exit semantics; +- a new `docs/integrations/run-result.md` owns the versioned run-result wire + contract and compatibility policy; +- `docs/integrations/json-output.md` remains the sole owner of output-bundle + discovery and lane publication; +- a new `docs/consumers/subprocess.md` provides the task-oriented invocation and + consumption workflow; and +- `docs/internal/cli.md` describes how the CLI constructs and emits the result + only after successful publication. + +Other documents should link to these owners instead of repeating volatile +fields or command details. + +## Acceptance Criteria + +- An ordinary successful `run` retains its existing human-readable output. +- A successful `run --json` emits one valid `notarius.run-result.v1` document + and no human prose on stdout. +- Relative configured or overridden output and debug roots are reported as + absolute bundle paths. +- The receipt identifies the production JSON bundle entry point without + copying its lane descriptors or payloads. +- Warning-bearing and rejection-bearing runs remain successful and report + accurate counts. +- Syntax, configuration, provider, pipeline, publication, debug, and result + writing failures retain the correct nonzero exit class. Consumers are + explicitly required to ignore stdout from a nonzero invocation. +- The implementation does not expose internal Go types or couple generic CLI + code to D&D or Narratio concepts. +- Public and internal documentation assigns each new contract to one canonical + owner. +- Offline behavioral tests protect the structured-output contract, default + human behavior, absolute path reporting, stream separation, and failure to + serialize or write the success result without duplicating lower-level output + encoder tests. + +## Out Of Scope + +The following may be useful later but are not prerequisites for the Narratio +integration: + +- a result-file flag in addition to machine-readable stdout; +- a JSON failure envelope or stable machine-readable error taxonomy; +- caller-supplied Notarius run IDs or exact output-bundle paths; +- a generic `--fail-on-rejection` or required-lane CLI policy; +- a public Go client package or importable Narratio adapter; +- Narratio stage, configuration, manifest, or report-generation changes; +- `notarius version --json`; +- installable or queryable artifact JSON Schemas; +- signal-aware CLI contexts and graceful SIGINT or SIGTERM handling; +- packaged release artifacts and a broader application-versioning policy. + +These items should be promoted only in response to a demonstrated integration +need rather than bundled into the initial subprocess contract.