# Subprocess Integration Implementation Plan ## Status Completed. ## Objective Implement the accepted [Subprocess Integration Contract](subprocess.md) as a small, generic extension of the existing CLI boundary. The completed work must let a subprocess caller discover a successful Notarius output bundle through a versioned JSON receipt without parsing human prose, while preserving current pipeline, output, rejection, warning, configuration, and interactive CLI behavior. Complete the stages below in order. Each stage must leave its affected package passing before the next begins. Do not add Narratio-specific production code, a public Go client, a new output encoder, or any feature listed as out of scope in the feature roadmap. ## Decisions And Invariants - `--json` is a boolean flag on `notarius run`; it does not affect pipeline resolution, execution, checkpoint identity, output encoding, or publication. - The machine result is a private CLI Go type implementing the public `notarius.run-result.v1` wire contract. Do not expose framework or CLI Go packages for external import. - The finalized run manifest is authoritative for `run_id`, `pipeline_id`, and `validation_status`. Verify that its pipeline ID matches the effective resolved pipeline rather than emitting conflicting provenance. - Human-oriented stdout remains byte-for-byte governed by its current path when `--json` is absent. Machine mode emits no human status or debug-path line on stdout. - Warnings and errors remain on stderr. Exit statuses remain 0 for success, 1 for runtime failure, and 2 for syntax failure. - Validation rejection remains a successful pipeline outcome. The receipt reports counts but does not decide which lanes an orchestrator requires. - The receipt points to the existing bundle; it does not duplicate output descriptors, lane payloads, warnings, rejections, or manifest content. - Reported filesystem paths are lexical absolute paths produced with `filepath.Abs`. Do not resolve symlinks or change the physical directories used for publication. - For the production `json` output module, `index_file` is exactly `index.json` and is included only after confirming that the runner returned exactly one logical file with that clean name. For any injected or future non-`json` output module, omit `index_file`; the machine receipt remains generic. - Machine-result construction and JSON encoding occur before debug success terminalization, but stdout writing occurs only after output publication and successful requested debug terminalization. - Standard output is not transactional. A final writer failure returns exit 1 and may leave partial bytes. Consumers must ignore stdout unless the process exits 0. Because debug terminalization must precede receipt emission, a result-delivery failure does not rewrite an already persisted successful run report; it is additionally reported on stderr and the process exits 1. - Do not add a stable JSON error envelope or promote diagnostic wording into a compatibility contract. ## Stage 1: Define The Run-Result Wire Model Add a focused private implementation under `internal/cli`, preferably `run_result.go`, with: - a constant for `notarius.run-result.v1`; - a private result struct whose JSON fields and requiredness match the feature roadmap; - a constructor that accepts the resolved pipeline, completed runner output, run output directory, and optional debug directory; and - serialization into an owned byte slice containing exactly one compact JSON object followed by one newline; and - a complete-write helper for delivering those prepared bytes. The result fields are: - required `schema_version`, `run_id`, `pipeline_id`, `output_directory`, `normalized_output_count`, `rejected_output_count`, and `warning_count`; - required non-empty `validation_status`, copied without reinterpretation from the final run manifest; - optional `index_file`, set to `index.json` for the resolved production JSON output module; and - optional `debug_directory`, present only when debug capture was requested and its path is non-empty. The constructor must: 1. validate the required run ID, pipeline ID, final validation status, and non-empty run output directory rather than emit a malformed receipt; 2. verify that the final manifest pipeline ID matches the effective resolved pipeline ID; 3. convert the output bundle and optional debug bundle paths to lexical absolute paths; 4. derive all counts from the completed `pipeline.RunOutput`; 5. identify the output module through the resolved module key, not concrete encoder types or D&D knowledge; and 6. for module key `pipeline.DefaultOutputModule`, verify exactly one returned logical output file is named `index.json` before setting `index_file`. Treat a missing or duplicate `index.json` from the production JSON output as a runtime contract error. Do not inspect or decode the index contents here; the output encoder and its existing tests own that format. Implement the writer with `encoding/json` and complete-write handling. It must surface encoding errors, zero-progress writes, short writes, and underlying writer errors to the command boundary. The command must map a delivery failure to a fixed, bounded result-write error without wrapping caller-supplied writer text into user or debug diagnostics. Do not add an injected serializer or a generic serialization framework merely to manufacture unreachable error cases for tests. Add lean focused tests for the wire-model boundary: - required fields, counts, schema identity, and compact newline-terminated JSON; - blank required manifest identities, blank validation status, and a mismatch between final-manifest and effective pipeline IDs; - lexical absolute conversion for relative output and debug paths; - production JSON entry-point discovery; - omission of `index_file` for a non-JSON output module; - missing and duplicate production entry points; and - a representative failing or zero-progress writer. Decode structured output in tests rather than snapshotting an entire JSON string. Assert literal field names and the schema identity because they are the public compatibility contract. Stage completion: - `go test ./internal/cli` ## Stage 2: Integrate Machine Mode With Run Finalization Update the run command in `internal/cli/run.go`: - register `--json` with the existing `flag.FlagSet`; - include it in root usage without adding it to `runFlagTakesValue`; - leave all existing flag combinations valid; and - do not add configuration or environment equivalents. After the runner succeeds and requested debug summaries have been written, apply this exact finalization order: 1. when `--json` is selected, construct and encode the complete receipt into owned memory; fail before physical output publication if its required fields or production entry point are invalid; 2. publish the runner's logical output files through the existing confined output writer; 3. terminalize requested debug reporting as successful and stop without writing stdout if terminalization fails; 4. in machine mode, write the prepared receipt to stdout and return a runtime failure if that write does not complete; 5. otherwise, use the unchanged human-oriented success reporting path; and 6. report successful-run warning counts to stderr as today. The receipt's `output_directory` is the absolute path to the run-specific bundle, not the configured output root. `debug_directory` is the absolute path to the allocated run-specific debug bundle. On a machine-result stdout failure after terminalization: - report a fixed, bounded, code-owned result-write error on stderr through the existing CLI failure reporting surface without including the underlying writer error; - include the existing debug-path discovery line when applicable; - return exit 1; - do not attempt terminal reporting a second time; and - leave the successfully published output and debug bundle intact. Do not change `pipelineCommandState`, the debug run-report wire shape, the output encoder contract, or `writeOutputFiles` solely to support this feature. The machine receipt belongs to CLI reporting after the runner and output module have completed their existing responsibilities. Stage completion: - `go test ./internal/cli` ## Stage 3: Protect The Public CLI Contract Add or extend behavior-level CLI tests at the narrowest stable boundaries. Reuse existing production components, fakes, state harnesses, and maintained examples rather than creating a second subprocess fixture framework. Cover: - a representative maintained production invocation with `--json`, decoding exactly one stdout document and verifying the schema identity, run and pipeline IDs, absolute output directory, `index_file`, counts, validation status, and the existence of the referenced bundle entry point; - a warning-bearing debug run, proving a correct warning count, warning reporting on stderr, an absolute optional debug path, and no human prose in stdout; - a rejection-bearing successful run using an explicit deterministic rejecting validator, proving zero or partial normalized outputs and the correct rejection count without changing exit status; - one representative syntax failure and one representative runtime failure with `--json`, proving the established exit class, stderr ownership, and no completed success document; - an injected stdout writer failure after successful publication, proving exit 1, retained output files, fixed diagnostic reporting, and omission of a recognizable writer-error sentinel from stderr and debug diagnostics; and - the existing no-`--json` tests continuing to protect interactive output. For the writer-failure case, use a writer that fails before accepting bytes when asserting empty stdout. The public contract nevertheless remains that arbitrary writers may leave partial bytes and consumers must ignore stdout on nonzero exit. Do not: - duplicate the JSON output encoder's lane, manifest, warning, or rejection serialization matrix; - assert complete human error strings; - add a golden file for the small receipt; - test private helper call order; or - add a prompt-language or unrelated end-to-end test. Stage completion: - `go test ./internal/cli` - `go test ./internal/modules/generic/output/json` - `go test ./internal/modules/integration` ## Stage 4: Publish The Consumer-Facing Documentation After the code and behavior tests pass, update the canonical current-behavior documentation: - `docs/cli.md` - add `--json` to run syntax and its flag table; - define stdout/stderr behavior and the requirement to parse machine stdout only after exit 0; - preserve the existing exit-status table and link to the run-result contract. - `docs/integrations/run-result.md` - own the complete `notarius.run-result.v1` field table, requiredness, example, path semantics, production `index_file` rule, partial-write rule, and additive compatibility policy; - link to the published JSON bundle contract rather than repeating its descriptors or payload schemas. - `docs/integrations/json-output.md` - add only a narrow cross-link explaining that a subprocess caller obtains the physical bundle root from the run-result receipt before using `index.json` for logical discovery; - do not duplicate run-result fields or stream semantics. - `docs/consumers/subprocess.md` - provide a concise task workflow for preflight, invocation, separate stream capture, exit checking, receipt decoding, confined index resolution, lane lookup by ID, media-type and schema-identity checks, required-versus-optional consumer policy, provenance retention, and sensitive-data handling; - use a generic orchestrator example with placeholders, not private Narratio paths, credentials, or a duplicate complete configuration. - `docs/internal/cli.md` - document receipt construction before publication, emission after publication and debug terminalization, and the stdout failure behavior; - preserve the CLI composition-root boundary. - `README.md` and `docs/development.md` - add only the smallest links needed to make subprocess consumer guidance and its implementation owner discoverable. Do not otherwise change `docs/config.md`, `docs/operations.md`, or the JSON output bundle contract unless implementation reveals an actual change to behavior they own. The machine receipt adds no configuration field, filesystem surface, retention policy, or lane-output shape. After all code, tests, and current-behavior documentation are complete: - change [the feature roadmap](subprocess.md) status to `Implemented`; - change this plan's status to `Completed`; and - ensure neither roadmap is used as the canonical description of current behavior. Stage completion: - validate documentation links and command examples against the implementation; - `git diff --check` - `go test ./...` - `go vet ./...` - `go build ./cmd/notarius` - `go test -race ./internal/cli` ## Open Questions None. The decisions above resolve stdout partial-write behavior, output-module generality, path normalization, partial pipeline success, terminal reporting, documentation ownership, and test boundaries.