Document and validate Notarius extraction workflows

docs/internal/adapters.md

@@ -41,11 +41,12 @@ Adapters do not own:
 - Seriatim subprocess runner.
 - Audita subprocess runner.
 - Scriptorium subprocess runner.
+- Notarius subprocess runner when extraction is enabled.
 - Noop notifier (`notify.NoopSender`).
 - Object store only when required by selected stages/config.
 
-The Notarius subprocess adapter is implemented independently of stage policy;
-application composition and extract-stage wiring are not yet implemented.
+Notarius is composed only when extraction is enabled; the extract stage owns
+receipt, bundle, and configured-lane policy rather than the adapter.
 
 Object-store construction goes through `newCommandObjectStore`, which loads
 configured filesystem secrets before adapter initialization.

docs/internal/artifacts.md

@@ -24,12 +24,15 @@ Registry entries bind each ID to its producer, output kind, canonical fallback,
 and content validator. The focused stage documents own their input/output flow;
 [Configuration](../config.md) owns where operators may select these IDs.
 
-## Configured and Previous-Session Sources
+## Configured, Extraction, And Previous-Session Sources
 
 - configured source ID format: `narratio.artifact.<artifact_key>`
+- extraction source ID format: `narratio.extraction.<output_key>`
 - previous-session source ID format: `narratio.previous_session.artifact.<artifact_key>`
 
-Both formats are validated by strict source-policy rules.
+All formats are validated by strict source-policy rules. Extraction sources are
+registered only from `pipeline.notarius.outputs`; the Notarius index has no
+selectable source ID.
 
 ## Runtime Catalog
 
@@ -58,6 +61,15 @@ Configured sources (`narratio.artifact.*`):
 
 - resolve only through runtime catalog availability.
 
+Extraction sources (`narratio.extraction.*`):
+
+- use the shared registration and manifest hydration path in
+  `extraction_catalog.go`;
+- require a current successful extract record with the exact configured source,
+  compatible contract and Notarius provenance, a confined regular durable
+  payload, and matching checksum; and
+- are never inferred by scanning the Notarius bundle directory.
+
 Previous-session sources (`narratio.previous_session.artifact.*`):
 
 - resolve only from local `previous/` cache state;
@@ -127,19 +139,23 @@ physical layout.
 
 - source ID formats are stable contracts;
 - artifact resolution is deterministic and manifest-aware;
+- extraction sources are available only from a compatible successful manifest
+  record;
 - previous-session source resolution in `analyze` is local-only;
 - remote current-state key construction remains centralized in artifacts helpers.
 
 ## Implementation And Tests
 
 - Registry and resolution: `internal/artifacts/artifact_resolver.go`,
-  `internal/artifacts/catalog.go`, `internal/artifacts/transcripts.go`
+  `internal/artifacts/catalog.go`, `internal/artifacts/transcripts.go`,
+  `internal/artifacts/extraction_catalog.go`
 - Current state: `internal/artifacts/current_state.go`
 - Paths and keys: `internal/artifacts/paths.go`,
   `internal/artifacts/s3_keys.go`
 - Previous requirements: `internal/artifacts/previous_requirements.go`
 - Tests: `internal/artifacts/artifact_resolver_test.go`,
   `internal/artifacts/catalog_test.go`,
+  `internal/artifacts/extraction_catalog_test.go`,
   `internal/artifacts/current_state_test.go`,
   `internal/artifacts/paths_model_test.go`,
   `internal/artifacts/previous_requirements_test.go`

docs/internal/manifest.md

@@ -50,13 +50,23 @@ The model admits these stage states:
 
 The application runner marks an executing stage running and then succeeded or
 failed in both manifests, persisting each transition. On success it records
-outputs, logs, generated configuration references, and metadata. A successful
-forced rerun marks only succeeded downstream session-stage records stale.
+outputs, logs, generated configuration references, and metadata. Artifact
+records may include optional contract and external provenance objects; old
+manifests remain compatible when those fields are absent. A successful forced
+rerun marks only succeeded downstream session-stage records stale.
+
+A stage may explicitly return a skipped disposition and stable reason. The
+runner persists that outcome in both manifests, clears older outputs for the
+session-stage record, and continues. This self-skip is distinct from deciding
+not to execute an already-succeeded stage and is reconsidered on later runs.
+Skipped results cannot contain outputs.
 
 When an already-succeeded stage is skipped, the invocation run manifest records
 the `skip` action and reason. The session manifest deliberately retains its
 existing succeeded record because it remains the cross-invocation progress
-authority.
+authority. Stages with a resume validator, currently extraction, may reject an
+otherwise eligible skip when the recorded durable result is obsolete; the
+runner marks it stale and executes it.
 
 Session manifest is the authoritative stage-progress ledger across invocations.
 Run manifest is invocation-scoped audit state.
@@ -64,6 +74,7 @@ Run manifest is invocation-scoped audit state.
 ## Invariants
 
 - stage resume/skip decisions are session-manifest driven.
+- self-skipped stages do not retain stale outputs and are reconsidered.
 - force reruns stale downstream succeeded stages.
 - run manifest does not replace session manifest as progress authority.
 

docs/internal/overview.md

@@ -53,7 +53,7 @@ The implemented canonical order is:
 4. [`polish`](stage-polish.md)
 5. [`normalize`](stage-normalize.md)
 6. [`trim`](stage-trim.md)
-7. `extract`
+7. [`extract`](stage-extract.md)
 8. [`render`](stage-render.md)
 9. [`analyze`](stage-analyze.md)
 10. [`publish`](stage-publish.md)
@@ -84,6 +84,7 @@ and execution semantics.
 - [`polish`](stage-polish.md)
 - [`normalize`](stage-normalize.md)
 - [`trim`](stage-trim.md)
+- [`extract`](stage-extract.md)
 - [`render`](stage-render.md)
 - [`analyze`](stage-analyze.md)
 - [`publish`](stage-publish.md)

docs/internal/stage-extract.md

@@ -0,0 +1,76 @@
+# Internal: Extract Stage
+
+## Responsibility
+
+`extract` runs after `trim` and before `render`. It converts the canonical
+`narratio.transcript.final_trimmed` JSON into configured Notarius lane artifacts.
+An omitted or disabled Notarius section makes the stage succeed as a no-op and
+does not require a Notarius runner.
+
+The external protocol is documented in the
+[Notarius integration contract](../integrations/notarius.md). Configuration
+fields belong in [Configuration](../config.md), and physical paths and force
+procedures belong in [Operations](../operations.md).
+
+## Lifecycle
+
+`internal/stage/extract.go`:
+
+1. resolves the final trimmed transcript from the shared artifact catalog;
+2. resolves and fingerprints the Notarius invocation contract;
+3. creates a run-local staging directory and invokes the injected
+   `notarius.Runner`;
+4. validates the successful receipt, confined index, configured required lane
+   descriptors, and regular payload files;
+5. atomically promotes the complete bundle to its immutable durable location;
+6. records one non-selectable `notarius_index` output and one selectable
+   `notarius_lane` output per configured lane; and
+7. registers each lane as `narratio.extraction.<output_key>` for downstream
+   Scriptorium and publish resolution.
+
+Lane records retain checksum, contract, producer run ID, and Notarius system,
+run, pipeline, and lane provenance. Stage metadata retains the durable bundle
+root, receipt, diagnostic paths, rejection/warning summaries, producing
+Narratio run ID, and invocation fingerprint. Validation completes before
+promotion, so a rejected result cannot expose a partial durable bundle.
+
+## Resume Validation
+
+`internal/stage/extract_resume.go` permits a skip only when the existing stage
+record succeeded and still matches the current invocation fingerprint. The
+fingerprint covers the resolved executable and config paths, pipeline ID,
+timeout, working directory, and sorted configured output contracts.
+
+The validator then checks the producing run identity, canonical immutable
+bundle root, path confinement and absence of symlink components, receipt
+identity, exactly one canonical index, the exact configured source set,
+contracts and provenance, regular-file status, and stored checksums. Missing or
+obsolete results are non-resumable and run again; unsafe filesystem conditions
+return an error rather than silently accepting or replacing data.
+
+The fingerprint cannot observe files imported by Notarius configuration,
+profile contents, prompt/module definitions, or other transitive inputs.
+Operators must force extraction after changing any such input.
+
+## Failure Behavior
+
+Adapter startup, timeout, nonzero exit, receipt decoding, path confinement,
+index compatibility, required-lane rejection, payload inspection, checksum, or
+promotion errors fail the stage through ordinary manifest transition handling.
+Stdout receipt and stderr diagnostics remain separate. Downstream stages are
+not given selectable extraction sources unless the complete configured result
+has passed validation and promotion.
+
+## Implementation And Focused Tests
+
+- Stage execution and selection: `internal/stage/extract.go`,
+  `internal/stage/extract_test.go`
+- Resume validation: `internal/stage/extract_resume.go`,
+  `internal/stage/extract_resume_test.go`
+- Subprocess boundary: `internal/adapters/notarius/subprocess.go`,
+  `internal/adapters/notarius/subprocess_test.go`
+- Catalog hydration: `internal/artifacts/extraction_catalog.go`,
+  `internal/artifacts/extraction_catalog_test.go`
+- Composition and downstream behavior: `internal/app/runner_test.go`,
+  `internal/stage/analyze_test.go`, `internal/stage/publish_test.go`
+

docs/internal/workspace.md

@@ -24,6 +24,14 @@ materialized into canonical session paths before stage success. Managed
 previous-session cache paths remain session-durable and are never redirected
 into run-local output space.
 
+Extraction uses run-local receipt, stderr, and output-root helpers, then
+promotes the validated external bundle to the unique immutable Notarius bundle
+path supplied by `internal/artifacts`. `internal/fileops.PromoteDirectory`
+copies only regular files and directories to a same-filesystem temporary
+sibling and atomically renames it without following symlinks or replacing an
+existing destination. Exact physical paths belong in
+[Operations](../operations.md#extraction-workflow).
+
 ## Locking
 
 `artifacts.LocalStore` enforces the single-writer session lock via `.lock`
@@ -54,9 +62,11 @@ deletion scope belong in [CLI](../cli.md#clean) and
 - Path model and local store: `internal/artifacts/paths.go`,
   `internal/artifacts/local.go`
 - Run-local materialization: `internal/stage/run_local.go`
+- Immutable bundle promotion: `internal/fileops/directory.go`
 - Cleanup confinement: `internal/app/cleanup_targets.go`,
   `internal/app/post_publish_cleanup.go`
 - Tests: `internal/artifacts/paths_model_test.go`,
   `internal/artifacts/local_test.go`, `internal/stage/run_local_test.go`,
+  `internal/fileops/directory_test.go`,
   `internal/app/cleanup_targets_test.go`,
   `internal/app/post_publish_cleanup_test.go`