Internal: Extract Stage

Responsibility

extract runs after trim and before render. It converts the canonical narratio.transcript.final_trimmed JSON into configured Notarius lane artifacts. An omitted or disabled Notarius section makes the stage succeed as a no-op and does not require a Notarius runner.

The external protocol is documented in the Notarius integration contract. Configuration fields belong in Configuration, and physical paths and force procedures belong in Operations.

Lifecycle

internal/stage/extract.go:

resolves the final trimmed transcript from the shared artifact catalog;
resolves and fingerprints the Notarius invocation contract;
creates a run-local staging directory and invokes the injected notarius.Runner;
validates the successful receipt, confined index, configured required lane descriptors, and regular payload files;
atomically promotes the complete bundle to its immutable durable location;
records one non-selectable notarius_index output and one selectable notarius_lane output per configured lane; and
registers each lane as narratio.extraction.<output_key> for downstream Scriptorium and publish resolution.

Lane records retain checksum, contract, producer run ID, and Notarius system, run, pipeline, and lane provenance. Stage metadata retains the durable bundle root, receipt, diagnostic paths, rejection/warning summaries, producing Narratio run ID, and invocation fingerprint. Validation completes before promotion, so a rejected result cannot expose a partial durable bundle.

Resume Validation

internal/stage/extract_resume.go permits a skip only when the existing stage record succeeded and still matches the current invocation fingerprint. The fingerprint covers the resolved executable and config paths, pipeline ID, timeout, working directory, and sorted configured output contracts.

The validator then checks the producing run identity, canonical immutable bundle root, path confinement and absence of symlink components, receipt identity, exactly one canonical index, the exact configured source set, contracts and provenance, regular-file status, and stored checksums. Missing or obsolete results are non-resumable and run again; unsafe filesystem conditions return an error rather than silently accepting or replacing data.

The fingerprint cannot observe files imported by Notarius configuration, profile contents, prompt/module definitions, or other transitive inputs. Operators must force extraction after changing any such input.

Failure Behavior

Adapter startup, timeout, nonzero exit, receipt decoding, path confinement, index compatibility, required-lane rejection, payload inspection, checksum, or promotion errors fail the stage through ordinary manifest transition handling. Stdout receipt and stderr diagnostics remain separate. Downstream stages are not given selectable extraction sources unless the complete configured result has passed validation and promotion.

Implementation And Focused Tests

Stage execution and selection: internal/stage/extract.go, internal/stage/extract_test.go
Resume validation: internal/stage/extract_resume.go, internal/stage/extract_resume_test.go
Subprocess boundary: internal/adapters/notarius/subprocess.go, internal/adapters/notarius/subprocess_test.go
Catalog hydration: internal/artifacts/extraction_catalog.go, internal/artifacts/extraction_catalog_test.go
Composition and downstream behavior: internal/app/runner_test.go, internal/stage/analyze_test.go, internal/stage/publish_test.go

3.6 KiB Raw Blame History