285 lines
13 KiB
Markdown
285 lines
13 KiB
Markdown
# Subprocess Integration Implementation Plan
|
|
|
|
## Status
|
|
|
|
Ready for implementation.
|
|
|
|
## Objective
|
|
|
|
Implement the accepted [Subprocess Integration Contract](subprocess.md) as a
|
|
small, generic extension of the existing CLI boundary. The completed work must
|
|
let a subprocess caller discover a successful Notarius output bundle through a
|
|
versioned JSON receipt without parsing human prose, while preserving current
|
|
pipeline, output, rejection, warning, configuration, and interactive CLI
|
|
behavior.
|
|
|
|
Complete the stages below in order. Each stage must leave its affected package
|
|
passing before the next begins. Do not add Narratio-specific production code,
|
|
a public Go client, a new output encoder, or any feature listed as out of scope
|
|
in the feature roadmap.
|
|
|
|
## Decisions And Invariants
|
|
|
|
- `--json` is a boolean flag on `notarius run`; it does not affect pipeline
|
|
resolution, execution, checkpoint identity, output encoding, or publication.
|
|
- The machine result is a private CLI Go type implementing the public
|
|
`notarius.run-result.v1` wire contract. Do not expose framework or CLI Go
|
|
packages for external import.
|
|
- The finalized run manifest is authoritative for `run_id`, `pipeline_id`, and
|
|
`validation_status`. Verify that its pipeline ID matches the effective
|
|
resolved pipeline rather than emitting conflicting provenance.
|
|
- Human-oriented stdout remains byte-for-byte governed by its current path when
|
|
`--json` is absent. Machine mode emits no human status or debug-path line on
|
|
stdout.
|
|
- Warnings and errors remain on stderr. Exit statuses remain 0 for success, 1
|
|
for runtime failure, and 2 for syntax failure.
|
|
- Validation rejection remains a successful pipeline outcome. The receipt
|
|
reports counts but does not decide which lanes an orchestrator requires.
|
|
- The receipt points to the existing bundle; it does not duplicate output
|
|
descriptors, lane payloads, warnings, rejections, or manifest content.
|
|
- Reported filesystem paths are lexical absolute paths produced with
|
|
`filepath.Abs`. Do not resolve symlinks or change the physical directories
|
|
used for publication.
|
|
- For the production `json` output module, `index_file` is exactly
|
|
`index.json` and is included only after confirming that the runner returned
|
|
exactly one logical file with that clean name. For any injected or future
|
|
non-`json` output module, omit `index_file`; the machine receipt remains
|
|
generic.
|
|
- Machine-result construction and JSON encoding occur before debug success
|
|
terminalization, but stdout writing occurs only after output publication and
|
|
successful requested debug terminalization.
|
|
- Standard output is not transactional. A final writer failure returns exit 1
|
|
and may leave partial bytes. Consumers must ignore stdout unless the process
|
|
exits 0. Because debug terminalization must precede receipt emission, a
|
|
result-delivery failure does not rewrite an already persisted successful run
|
|
report; it is additionally reported on stderr and the process exits 1.
|
|
- Do not add a stable JSON error envelope or promote diagnostic wording into a
|
|
compatibility contract.
|
|
|
|
## Stage 1: Define The Run-Result Wire Model
|
|
|
|
Add a focused private implementation under `internal/cli`, preferably
|
|
`run_result.go`, with:
|
|
|
|
- a constant for `notarius.run-result.v1`;
|
|
- a private result struct whose JSON fields and requiredness match the feature
|
|
roadmap;
|
|
- a constructor that accepts the resolved pipeline, completed runner output,
|
|
run output directory, and optional debug directory; and
|
|
- serialization into an owned byte slice containing exactly one compact JSON
|
|
object followed by one newline; and
|
|
- a complete-write helper for delivering those prepared bytes.
|
|
|
|
The result fields are:
|
|
|
|
- required `schema_version`, `run_id`, `pipeline_id`, `output_directory`,
|
|
`normalized_output_count`, `rejected_output_count`, and `warning_count`;
|
|
- required non-empty `validation_status`, copied without reinterpretation from
|
|
the final run manifest;
|
|
- optional `index_file`, set to `index.json` for the resolved production JSON
|
|
output module; and
|
|
- optional `debug_directory`, present only when debug capture was requested and
|
|
its path is non-empty.
|
|
|
|
The constructor must:
|
|
|
|
1. validate the required run ID, pipeline ID, final validation status, and
|
|
non-empty run output directory rather than emit a malformed receipt;
|
|
2. verify that the final manifest pipeline ID matches the effective resolved
|
|
pipeline ID;
|
|
3. convert the output bundle and optional debug bundle paths to lexical absolute
|
|
paths;
|
|
4. derive all counts from the completed `pipeline.RunOutput`;
|
|
5. identify the output module through the resolved module key, not concrete
|
|
encoder types or D&D knowledge; and
|
|
6. for module key `pipeline.DefaultOutputModule`, verify exactly one returned
|
|
logical output file is named `index.json` before setting `index_file`.
|
|
|
|
Treat a missing or duplicate `index.json` from the production JSON output as a
|
|
runtime contract error. Do not inspect or decode the index contents here; the
|
|
output encoder and its existing tests own that format.
|
|
|
|
Implement the writer with `encoding/json` and complete-write handling. It must
|
|
surface encoding errors, zero-progress writes, short writes, and underlying
|
|
writer errors to the command boundary. The command must map a delivery failure
|
|
to a fixed, bounded result-write error without wrapping caller-supplied writer
|
|
text into user or debug diagnostics. Do not add an injected serializer or a
|
|
generic serialization framework merely to manufacture unreachable error cases
|
|
for tests.
|
|
|
|
Add lean focused tests for the wire-model boundary:
|
|
|
|
- required fields, counts, schema identity, and compact newline-terminated JSON;
|
|
- blank required manifest identities, blank validation status, and a mismatch
|
|
between final-manifest and effective pipeline IDs;
|
|
- lexical absolute conversion for relative output and debug paths;
|
|
- production JSON entry-point discovery;
|
|
- omission of `index_file` for a non-JSON output module;
|
|
- missing and duplicate production entry points; and
|
|
- a representative failing or zero-progress writer.
|
|
|
|
Decode structured output in tests rather than snapshotting an entire JSON
|
|
string. Assert literal field names and the schema identity because they are the
|
|
public compatibility contract.
|
|
|
|
Stage completion:
|
|
|
|
- `go test ./internal/cli`
|
|
|
|
## Stage 2: Integrate Machine Mode With Run Finalization
|
|
|
|
Update the run command in `internal/cli/run.go`:
|
|
|
|
- register `--json` with the existing `flag.FlagSet`;
|
|
- include it in root usage without adding it to `runFlagTakesValue`;
|
|
- leave all existing flag combinations valid; and
|
|
- do not add configuration or environment equivalents.
|
|
|
|
After the runner succeeds and requested debug summaries have been written,
|
|
apply this exact finalization order:
|
|
|
|
1. when `--json` is selected, construct and encode the complete receipt into
|
|
owned memory; fail before physical output publication if its required fields
|
|
or production entry point are invalid;
|
|
2. publish the runner's logical output files through the existing confined
|
|
output writer;
|
|
3. terminalize requested debug reporting as successful and stop without
|
|
writing stdout if terminalization fails;
|
|
4. in machine mode, write the prepared receipt to stdout and return a runtime
|
|
failure if that write does not complete;
|
|
5. otherwise, use the unchanged human-oriented success reporting path; and
|
|
6. report successful-run warning counts to stderr as today.
|
|
|
|
The receipt's `output_directory` is the absolute path to the run-specific
|
|
bundle, not the configured output root. `debug_directory` is the absolute path
|
|
to the allocated run-specific debug bundle.
|
|
|
|
On a machine-result stdout failure after terminalization:
|
|
|
|
- report a fixed, bounded, code-owned result-write error on stderr through the
|
|
existing CLI failure reporting surface without including the underlying
|
|
writer error;
|
|
- include the existing debug-path discovery line when applicable;
|
|
- return exit 1;
|
|
- do not attempt terminal reporting a second time; and
|
|
- leave the successfully published output and debug bundle intact.
|
|
|
|
Do not change `pipelineCommandState`, the debug run-report wire shape, the
|
|
output encoder contract, or `writeOutputFiles` solely to support this feature.
|
|
The machine receipt belongs to CLI reporting after the runner and output module
|
|
have completed their existing responsibilities.
|
|
|
|
Stage completion:
|
|
|
|
- `go test ./internal/cli`
|
|
|
|
## Stage 3: Protect The Public CLI Contract
|
|
|
|
Add or extend behavior-level CLI tests at the narrowest stable boundaries.
|
|
Reuse existing production components, fakes, state harnesses, and maintained
|
|
examples rather than creating a second subprocess fixture framework.
|
|
|
|
Cover:
|
|
|
|
- a representative maintained production invocation with `--json`, decoding
|
|
exactly one stdout document and verifying the schema identity, run and
|
|
pipeline IDs, absolute output directory, `index_file`, counts, validation
|
|
status, and the existence of the referenced bundle entry point;
|
|
- a warning-bearing debug run, proving a correct warning count, warning
|
|
reporting on stderr, an absolute optional debug path, and no human prose in
|
|
stdout;
|
|
- a rejection-bearing successful run using an explicit deterministic rejecting
|
|
validator, proving zero or partial normalized outputs and the correct
|
|
rejection count without changing exit status;
|
|
- one representative syntax failure and one representative runtime failure
|
|
with `--json`, proving the established exit class, stderr ownership, and no
|
|
completed success document;
|
|
- an injected stdout writer failure after successful publication, proving exit
|
|
1, retained output files, fixed diagnostic reporting, and omission of a
|
|
recognizable writer-error sentinel from stderr and debug diagnostics; and
|
|
- the existing no-`--json` tests continuing to protect interactive output.
|
|
|
|
For the writer-failure case, use a writer that fails before accepting bytes when
|
|
asserting empty stdout. The public contract nevertheless remains that arbitrary
|
|
writers may leave partial bytes and consumers must ignore stdout on nonzero
|
|
exit.
|
|
|
|
Do not:
|
|
|
|
- duplicate the JSON output encoder's lane, manifest, warning, or rejection
|
|
serialization matrix;
|
|
- assert complete human error strings;
|
|
- add a golden file for the small receipt;
|
|
- test private helper call order; or
|
|
- add a prompt-language or unrelated end-to-end test.
|
|
|
|
Stage completion:
|
|
|
|
- `go test ./internal/cli`
|
|
- `go test ./internal/modules/generic/output/json`
|
|
- `go test ./internal/modules/integration`
|
|
|
|
## Stage 4: Publish The Consumer-Facing Documentation
|
|
|
|
After the code and behavior tests pass, update the canonical current-behavior
|
|
documentation:
|
|
|
|
- `docs/cli.md`
|
|
- add `--json` to run syntax and its flag table;
|
|
- define stdout/stderr behavior and the requirement to parse machine stdout
|
|
only after exit 0;
|
|
- preserve the existing exit-status table and link to the run-result
|
|
contract.
|
|
- `docs/integrations/run-result.md`
|
|
- own the complete `notarius.run-result.v1` field table, requiredness,
|
|
example, path semantics, production `index_file` rule, partial-write rule,
|
|
and additive compatibility policy;
|
|
- link to the published JSON bundle contract rather than repeating its
|
|
descriptors or payload schemas.
|
|
- `docs/integrations/json-output.md`
|
|
- add only a narrow cross-link explaining that a subprocess caller obtains
|
|
the physical bundle root from the run-result receipt before using
|
|
`index.json` for logical discovery;
|
|
- do not duplicate run-result fields or stream semantics.
|
|
- `docs/consumers/subprocess.md`
|
|
- provide a concise task workflow for preflight, invocation, separate stream
|
|
capture, exit checking, receipt decoding, confined index resolution, lane
|
|
lookup by ID, media-type and schema-identity checks, required-versus-optional
|
|
consumer policy, provenance retention, and sensitive-data handling;
|
|
- use a generic orchestrator example with placeholders, not private Narratio
|
|
paths, credentials, or a duplicate complete configuration.
|
|
- `docs/internal/cli.md`
|
|
- document receipt construction before publication, emission after
|
|
publication and debug terminalization, and the stdout failure behavior;
|
|
- preserve the CLI composition-root boundary.
|
|
- `README.md` and `docs/development.md`
|
|
- add only the smallest links needed to make subprocess consumer guidance and
|
|
its implementation owner discoverable.
|
|
|
|
Do not otherwise change `docs/config.md`, `docs/operations.md`, or the JSON
|
|
output bundle contract unless implementation reveals an actual change to
|
|
behavior they own. The machine receipt adds no configuration field, filesystem
|
|
surface, retention policy, or lane-output shape.
|
|
|
|
After all code, tests, and current-behavior documentation are complete:
|
|
|
|
- change [the feature roadmap](subprocess.md) status to `Implemented`;
|
|
- change this plan's status to `Completed`; and
|
|
- ensure neither roadmap is used as the canonical description of current
|
|
behavior.
|
|
|
|
Stage completion:
|
|
|
|
- validate documentation links and command examples against the implementation;
|
|
- `git diff --check`
|
|
- `go test ./...`
|
|
- `go vet ./...`
|
|
- `go build ./cmd/notarius`
|
|
- `go test -race ./internal/cli`
|
|
|
|
## Open Questions
|
|
|
|
None. The decisions above resolve stdout partial-write behavior, output-module
|
|
generality, path normalization, partial pipeline success, terminal reporting,
|
|
documentation ownership, and test boundaries.
|