Add a plan to implement improved documentation and support for running notarius as a subprocess
This commit is contained in:
208
docs/roadmap/subprocess.md
Normal file
208
docs/roadmap/subprocess.md
Normal file
@@ -0,0 +1,208 @@
|
||||
# Subprocess Integration Contract
|
||||
|
||||
## Status
|
||||
|
||||
Accepted for implementation.
|
||||
|
||||
## Purpose
|
||||
|
||||
Make Notarius straightforward to invoke as a subprocess from an orchestrator
|
||||
such as Narratio. A caller should be able to run a configured pipeline, discover
|
||||
the published output bundle without parsing human prose or scanning a
|
||||
directory, and hand selected structured artifacts to a later stage.
|
||||
|
||||
This work strengthens the public CLI boundary. It does not turn Notarius into a
|
||||
Go library, embed Narratio-specific behavior, or change pipeline execution and
|
||||
artifact semantics.
|
||||
|
||||
## Desired End State
|
||||
|
||||
A subprocess caller can:
|
||||
|
||||
1. validate a Notarius configuration and selected pipeline before execution;
|
||||
2. invoke `notarius run` with explicit input, output-root, session, and
|
||||
reference arguments;
|
||||
3. request one versioned, machine-readable success result on standard output;
|
||||
4. use that result to locate the published output bundle;
|
||||
5. discover normalized lane payloads through the bundle's authoritative
|
||||
`index.json`;
|
||||
6. distinguish process failure from successful partial pipeline outcomes; and
|
||||
7. record Notarius run provenance in its own manifest without depending on
|
||||
internal packages, cache formats, debug formats, or human-readable messages.
|
||||
|
||||
The existing human-oriented command output remains the default for interactive
|
||||
use.
|
||||
|
||||
## Machine-Readable Run Result
|
||||
|
||||
`notarius run` supports `--json`. On success, the flag makes standard output
|
||||
contain exactly one JSON object followed by a newline. No human-oriented status
|
||||
line is mixed into that stream.
|
||||
|
||||
The result uses the schema identity `notarius.run-result.v1` and contains:
|
||||
|
||||
| Field | Presence | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `schema_version` | Required | Exactly `notarius.run-result.v1`. |
|
||||
| `run_id` | Required | The Notarius run identifier. |
|
||||
| `pipeline_id` | Required | The effective pipeline identifier. |
|
||||
| `output_directory` | Required | Absolute path to the successfully published output bundle. |
|
||||
| `index_file` | Required for the production JSON output | Logical bundle path `index.json`. |
|
||||
| `normalized_output_count` | Required | Number of final normalized lane outputs returned by the pipeline. |
|
||||
| `rejected_output_count` | Required | Number of recorded rejected outputs. |
|
||||
| `warning_count` | Required | Number of final run warnings returned by the pipeline. |
|
||||
| `validation_status` | Required | The run manifest's final validation status without reinterpretation. |
|
||||
| `debug_directory` | Optional | Absolute debug-bundle path when debug capture was requested and completed. |
|
||||
|
||||
An illustrative successful result is:
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": "notarius.run-result.v1",
|
||||
"run_id": "run-1770000000000000000-0123456789abcdef0123456789abcdef",
|
||||
"pipeline_id": "dnd-session",
|
||||
"output_directory": "/srv/narratio/runs/session-7/notarius/run-1770000000000000000-0123456789abcdef0123456789abcdef",
|
||||
"index_file": "index.json",
|
||||
"normalized_output_count": 6,
|
||||
"rejected_output_count": 2,
|
||||
"warning_count": 1,
|
||||
"validation_status": "approved"
|
||||
}
|
||||
```
|
||||
|
||||
The receipt is a discovery and summary document, not a duplicate output
|
||||
envelope. It does not embed lane payloads, rejection entries, warnings, the run
|
||||
manifest, or output-file contents. Consumers use `index_file` and the existing
|
||||
published JSON output contract for those records.
|
||||
|
||||
The result contract must tolerate future additive optional fields. Any
|
||||
incompatible field or semantic change requires a new run-result schema version.
|
||||
|
||||
## Stream, Publication, And Failure Semantics
|
||||
|
||||
Machine-readable output is emitted only after:
|
||||
|
||||
- the pipeline has completed without a framework error;
|
||||
- all logical output files have been successfully published;
|
||||
- requested debug terminal reporting has completed; and
|
||||
- all result fields are known.
|
||||
|
||||
Writing or encoding the machine-readable result is part of successful command
|
||||
completion. Failure to write it produces the existing runtime-failure exit
|
||||
class.
|
||||
|
||||
With `--json`:
|
||||
|
||||
- successful stdout is exclusively the run-result JSON document;
|
||||
- successful warnings remain on stderr under the existing CLI contract;
|
||||
- syntax and runtime errors retain their existing exit statuses and stderr
|
||||
diagnostics;
|
||||
- consumers treat stdout as a valid result only when the process exits with
|
||||
status 0; failures before result writing emit no result, while a failure
|
||||
during the stdout write may leave incomplete bytes that must be ignored; and
|
||||
- human-readable diagnostic wording is not promoted into a machine contract.
|
||||
|
||||
Without `--json`, current interactive stdout and stderr behavior remains
|
||||
unchanged.
|
||||
|
||||
Successful runs may contain rejected outputs or omit some normalized lanes.
|
||||
That remains a valid pipeline outcome. The run result reports counts, while
|
||||
`index.json`, `rejected.json`, and `warnings.json` remain authoritative for
|
||||
details. Notarius will not add a generic `--fail-on-rejection` policy as part
|
||||
of this work.
|
||||
|
||||
## Output Discovery And Consumer Responsibilities
|
||||
|
||||
The production JSON encoder's `index.json` remains the authoritative mapping
|
||||
from lane IDs to published payloads. A subprocess consumer should:
|
||||
|
||||
- resolve `index_file` beneath `output_directory` and reject path escape;
|
||||
- locate expected outputs by `lane_id`, not by guessing filenames;
|
||||
- check each selected descriptor's media type and schema identity;
|
||||
- decode payloads according to their published integration contracts;
|
||||
- decide which lanes are required or optional for its own later stages; and
|
||||
- retain rejection, warning, and manifest files when they are needed for
|
||||
provenance or review.
|
||||
|
||||
For Narratio, required report inputs and partial-success policy remain Narratio
|
||||
stage configuration and orchestration concerns. Notarius does not acquire
|
||||
knowledge of Narratio stages, manifests, workspace layout, publication policy,
|
||||
or report formats.
|
||||
|
||||
## Invocation Guidance
|
||||
|
||||
The consumer documentation recommends that subprocess callers:
|
||||
|
||||
- use `notarius config validate --pipeline` as an optional preflight;
|
||||
- pass explicit absolute paths for the input, configuration, output root, and
|
||||
CLI-supplied references;
|
||||
- use a stable, non-secret prompt session identifier when useful for provider
|
||||
routing or caching;
|
||||
- capture stdout and stderr separately;
|
||||
- supply credentials through the configured environment mechanism rather than
|
||||
command arguments or generated configuration containing secret values;
|
||||
- place output, cache, debug, and subprocess logs under intentional
|
||||
sensitivity and retention policies; and
|
||||
- treat the Notarius manifest and run-result receipt as provenance while
|
||||
leaving the caller's own manifest authoritative for its stage lifecycle.
|
||||
|
||||
Notarius configuration remains owned by Notarius. An orchestrator may select a
|
||||
configuration and pass supported operational overrides, but should not
|
||||
duplicate the complete Notarius configuration schema.
|
||||
|
||||
## Documentation End State
|
||||
|
||||
- `docs/cli.md` owns `run --json`, stream behavior, and exit semantics;
|
||||
- a new `docs/integrations/run-result.md` owns the versioned run-result wire
|
||||
contract and compatibility policy;
|
||||
- `docs/integrations/json-output.md` remains the sole owner of output-bundle
|
||||
discovery and lane publication;
|
||||
- a new `docs/consumers/subprocess.md` provides the task-oriented invocation and
|
||||
consumption workflow; and
|
||||
- `docs/internal/cli.md` describes how the CLI constructs and emits the result
|
||||
only after successful publication.
|
||||
|
||||
Other documents should link to these owners instead of repeating volatile
|
||||
fields or command details.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- An ordinary successful `run` retains its existing human-readable output.
|
||||
- A successful `run --json` emits one valid `notarius.run-result.v1` document
|
||||
and no human prose on stdout.
|
||||
- Relative configured or overridden output and debug roots are reported as
|
||||
absolute bundle paths.
|
||||
- The receipt identifies the production JSON bundle entry point without
|
||||
copying its lane descriptors or payloads.
|
||||
- Warning-bearing and rejection-bearing runs remain successful and report
|
||||
accurate counts.
|
||||
- Syntax, configuration, provider, pipeline, publication, debug, and result
|
||||
writing failures retain the correct nonzero exit class. Consumers are
|
||||
explicitly required to ignore stdout from a nonzero invocation.
|
||||
- The implementation does not expose internal Go types or couple generic CLI
|
||||
code to D&D or Narratio concepts.
|
||||
- Public and internal documentation assigns each new contract to one canonical
|
||||
owner.
|
||||
- Offline behavioral tests protect the structured-output contract, default
|
||||
human behavior, absolute path reporting, stream separation, and failure to
|
||||
serialize or write the success result without duplicating lower-level output
|
||||
encoder tests.
|
||||
|
||||
## Out Of Scope
|
||||
|
||||
The following may be useful later but are not prerequisites for the Narratio
|
||||
integration:
|
||||
|
||||
- a result-file flag in addition to machine-readable stdout;
|
||||
- a JSON failure envelope or stable machine-readable error taxonomy;
|
||||
- caller-supplied Notarius run IDs or exact output-bundle paths;
|
||||
- a generic `--fail-on-rejection` or required-lane CLI policy;
|
||||
- a public Go client package or importable Narratio adapter;
|
||||
- Narratio stage, configuration, manifest, or report-generation changes;
|
||||
- `notarius version --json`;
|
||||
- installable or queryable artifact JSON Schemas;
|
||||
- signal-aware CLI contexts and graceful SIGINT or SIGTERM handling;
|
||||
- packaged release artifacts and a broader application-versioning policy.
|
||||
|
||||
These items should be promoted only in response to a demonstrated integration
|
||||
need rather than bundled into the initial subprocess contract.
|
||||
Reference in New Issue
Block a user