|
|
|
|
@@ -2,11 +2,13 @@
|
|
|
|
|
|
|
|
|
|
## Status And Audience
|
|
|
|
|
|
|
|
|
|
Complete.
|
|
|
|
|
Remediation planned. Stages 1 through 9 are complete; Stages 10 through 14
|
|
|
|
|
address gaps found during the post-implementation architecture review.
|
|
|
|
|
|
|
|
|
|
This plan is written for a GPT-5.6 Terra coding agent. Implement the stages in
|
|
|
|
|
strict numerical order. Do not skip ahead, combine stages merely to reduce the
|
|
|
|
|
number of prompts, or describe unfinished behavior as current functionality.
|
|
|
|
|
This plan is written for a GPT-5.6 Terra coding agent. Implement incomplete
|
|
|
|
|
stages in strict numerical order. Do not skip ahead, combine stages merely to
|
|
|
|
|
reduce the number of prompts, or describe unfinished behavior as current
|
|
|
|
|
functionality.
|
|
|
|
|
|
|
|
|
|
The target state and acceptance criteria are owned by
|
|
|
|
|
[notarius-extract-stage.md](notarius-extract-stage.md). This document owns the
|
|
|
|
|
@@ -25,6 +27,11 @@ implementation sequence.
|
|
|
|
|
| Stage 7 | Complete |
|
|
|
|
|
| Stage 8 | Complete |
|
|
|
|
|
| Stage 9 | Complete |
|
|
|
|
|
| Stage 10 | Complete |
|
|
|
|
|
| Stage 11 | Pending |
|
|
|
|
|
| Stage 12 | Pending |
|
|
|
|
|
| Stage 13 | Pending |
|
|
|
|
|
| Stage 14 | Pending |
|
|
|
|
|
|
|
|
|
|
After completing and validating a stage, update only that stage's row to
|
|
|
|
|
`Complete` and record any material deviation in the relevant stage section.
|
|
|
|
|
@@ -32,7 +39,7 @@ Do not mark a stage complete while required tests or exit criteria remain.
|
|
|
|
|
|
|
|
|
|
## Working Rules
|
|
|
|
|
|
|
|
|
|
Before Stage 1, read:
|
|
|
|
|
Before Stage 10, read:
|
|
|
|
|
|
|
|
|
|
- `docs/development.md` and its task-specific references;
|
|
|
|
|
- `docs/roadmap/notarius-extract-stage.md` completely;
|
|
|
|
|
@@ -40,538 +47,314 @@ Before Stage 1, read:
|
|
|
|
|
receipt, JSON-output, and lane-contract documentation; and
|
|
|
|
|
- the focused Narratio documents and tests named by the current stage.
|
|
|
|
|
|
|
|
|
|
For every stage:
|
|
|
|
|
For every incomplete stage:
|
|
|
|
|
|
|
|
|
|
1. Inspect the current implementation before editing. Prefer the codebase
|
|
|
|
|
knowledge graph for code discovery as required by `AGENTS.md`.
|
|
|
|
|
2. Preserve unrelated worktree changes.
|
|
|
|
|
3. Implement the complete stage scope, including focused tests. Do not leave
|
|
|
|
|
TODO implementations, compatibility shims without owners, or knowingly
|
|
|
|
|
dead code for a later stage.
|
|
|
|
|
4. Use real internal collaborators in tests where fast and deterministic; fake
|
|
|
|
|
only subprocess, remote, clock, randomness, or other external boundaries.
|
|
|
|
|
5. Keep tests offline and independent of real Notarius, PromptKit, LLM
|
|
|
|
|
providers, credentials, and mutable services.
|
|
|
|
|
6. Run the focused package tests listed for the stage. Fix failures before
|
|
|
|
|
proceeding.
|
|
|
|
|
7. Run `go test ./...` after every stage. Also run `go vet ./...` and
|
|
|
|
|
`go build ./cmd/narratio` when the stage changes shared contracts,
|
|
|
|
|
composition, CLI behavior, or documentation examples.
|
|
|
|
|
8. Keep proposed behavior in `docs/roadmap/` until Stage 9. Intermediate code
|
|
|
|
|
is development work and must not cause current-behavior documentation to
|
|
|
|
|
claim that the end-to-end feature is complete.
|
|
|
|
|
3. Implement the complete stage scope, including focused regression tests. Do
|
|
|
|
|
not leave TODO implementations or defer a known part of the stage.
|
|
|
|
|
4. Keep tests offline and independent of real Notarius, PromptKit, LLM
|
|
|
|
|
providers, credentials, network services, and mutable external state.
|
|
|
|
|
5. Run the focused package tests listed for the stage, then `go test ./...`.
|
|
|
|
|
6. Run `go vet ./...` and `go build ./cmd/narratio` whenever shared contracts,
|
|
|
|
|
composition, CLI behavior, portability, or documentation examples change.
|
|
|
|
|
7. Do not broaden the feature beyond the target architecture while fixing a
|
|
|
|
|
remediation. Prefer a general runner or filesystem invariant when the defect
|
|
|
|
|
is general, but avoid unrelated cleanup.
|
|
|
|
|
|
|
|
|
|
Intermediate stages must compile and pass the repository test suite. The work
|
|
|
|
|
is not release-ready until Stage 9 is complete.
|
|
|
|
|
## Completed Stages 1-9 Summary
|
|
|
|
|
|
|
|
|
|
## Stage 1: Establish Shared Stage-Outcome And Artifact-Provenance Contracts
|
|
|
|
|
1. **Stage 1 — Shared contracts:** Added explicit stage self-skip semantics,
|
|
|
|
|
artifact source identity, contract metadata, external provenance, and
|
|
|
|
|
backward-compatible manifest serialization.
|
|
|
|
|
2. **Stage 2 — Configuration and source policy:** Added strict optional
|
|
|
|
|
Notarius configuration, deterministic defaults/path resolution, output-lane
|
|
|
|
|
contracts, and the `narratio.extraction.<key>` source family.
|
|
|
|
|
3. **Stage 3 — Immutable promotion:** Added regular-file-only bundle copying,
|
|
|
|
|
symlink and special-file rejection, bounded permissions, syncing, cleanup,
|
|
|
|
|
and atomic no-replace installation on Linux.
|
|
|
|
|
4. **Stage 4 — Notarius adapter:** Added the subprocess boundary, exact CLI
|
|
|
|
|
invocation, separate receipt/diagnostic streams, bounded tolerant decoding,
|
|
|
|
|
path confinement, and generic receipt/index discovery.
|
|
|
|
|
5. **Stage 5 — Extract execution:** Added final-trimmed transcript consumption,
|
|
|
|
|
configured required-lane validation, immutable bundle promotion, checksums,
|
|
|
|
|
manifest-ready outputs, provenance, diagnostics, and invocation
|
|
|
|
|
fingerprinting.
|
|
|
|
|
6. **Stage 6 — Lifecycle and resume:** Registered `extract` between `trim` and
|
|
|
|
|
`render`, added full and single-stage CLI support, resume validation, force
|
|
|
|
|
ordering, and old-manifest compatibility.
|
|
|
|
|
7. **Stage 7 — Analyze integration:** Added shared manifest-backed extraction
|
|
|
|
|
catalog hydration and explicit required/optional Scriptorium input
|
|
|
|
|
resolution without directory scanning or implicit inputs.
|
|
|
|
|
8. **Stage 8 — Publish and inspection integration:** Added explicit extraction
|
|
|
|
|
publish rules, metadata round-tripping, restore compatibility, and operator
|
|
|
|
|
availability reporting.
|
|
|
|
|
9. **Stage 9 — Documentation and examples:** Added maintained ten-lane and
|
|
|
|
|
subset examples, integration and maintainer documentation, operational
|
|
|
|
|
guidance, and repository-wide validation.
|
|
|
|
|
|
|
|
|
|
Introduce the reusable internal contracts needed by extraction without adding
|
|
|
|
|
Notarius configuration or a registered stage.
|
|
|
|
|
## Stage 10: Correct Downstream Invalidation For Every Replaced Stage Result
|
|
|
|
|
|
|
|
|
|
Fix runner semantics so a newly produced extraction result cannot coexist with
|
|
|
|
|
downstream stages that are still considered current from an older result. Make
|
|
|
|
|
the correction at the general stage-runner boundary rather than adding an
|
|
|
|
|
`extract`-specific exception.
|
|
|
|
|
|
|
|
|
|
Implementation:
|
|
|
|
|
|
|
|
|
|
1. Under `internal/artifactmodel`, add optional neutral models for:
|
|
|
|
|
- contract metadata: `media_type`, `schema_id`, `schema_version`, and
|
|
|
|
|
optional `module_key`;
|
|
|
|
|
- external provenance: `system`, `run_id`, `pipeline_id`, and
|
|
|
|
|
`artifact_id`.
|
|
|
|
|
2. Extend `artifacts.Ref` with:
|
|
|
|
|
- optional explicit `SourceID`;
|
|
|
|
|
- optional contract metadata; and
|
|
|
|
|
- optional external provenance.
|
|
|
|
|
3. Extend `manifest.ArtifactRecord` with the same nested optional metadata.
|
|
|
|
|
Use backward-compatible `omitempty` JSON fields. Old manifests must load
|
|
|
|
|
and round-trip without fabricated metadata.
|
|
|
|
|
4. Refactor `internal/app.mapResultOutputs` to prefer `Ref.SourceID` when set,
|
|
|
|
|
copy both metadata structures, and retain existing output-kind and analyze
|
|
|
|
|
inference only as fallback behavior.
|
|
|
|
|
5. Add an explicit `StageResult` disposition with succeeded and skipped
|
|
|
|
|
values. Preserve zero-value success so existing stages do not need
|
|
|
|
|
mechanical edits.
|
|
|
|
|
6. Add a stable skip-reason field. Reject or fail safely if a skipped result
|
|
|
|
|
contains outputs.
|
|
|
|
|
7. Teach `executeStages` to persist a self-skipped stage as skipped in both
|
|
|
|
|
session and run manifests, apply bounded result metadata and diagnostics,
|
|
|
|
|
clear any older session-stage outputs, and continue to later stages.
|
|
|
|
|
Distinguish this from the runner's existing decision to skip an already
|
|
|
|
|
succeeded stage.
|
|
|
|
|
8. Keep successful and failed behavior unchanged for existing stages.
|
|
|
|
|
1. In `internal/app`, capture the prior session-stage outcome before changing a
|
|
|
|
|
selected stage to running. Preserve enough prior state to distinguish a
|
|
|
|
|
reused result, a repeated identical self-skip, and a replaced result.
|
|
|
|
|
2. Define replacement and invalidation behavior as follows:
|
|
|
|
|
- an already-succeeded stage skipped without execution does not invalidate
|
|
|
|
|
anything;
|
|
|
|
|
- a resume-validation error still stops without mutating the succeeded stage
|
|
|
|
|
or downstream stages;
|
|
|
|
|
- a non-resumable succeeded result is marked stale and invalidates succeeded
|
|
|
|
|
downstream stages before rerun, preserving the current behavior;
|
|
|
|
|
- any forced execution invalidates succeeded downstream stages even if the
|
|
|
|
|
execution later self-skips or fails;
|
|
|
|
|
- a non-forced execution that changes a skipped, failed, stale, pending, or
|
|
|
|
|
absent stage to succeeded invalidates succeeded downstream stages;
|
|
|
|
|
- an executed failure invalidates succeeded downstream stages before the
|
|
|
|
|
failed manifest state is committed; and
|
|
|
|
|
- a self-skip invalidates succeeded downstream stages only when it replaces
|
|
|
|
|
a different effective outcome. Repeating the same stable skip reason with
|
|
|
|
|
no outputs must not make every disabled full-pipeline invocation rerun all
|
|
|
|
|
downstream stages.
|
|
|
|
|
3. Persist invalidation with the same session-manifest transition that makes the
|
|
|
|
|
upstream replacement observable. Do not leave a committed failed or skipped
|
|
|
|
|
upstream result with succeeded downstream records that depend on its former
|
|
|
|
|
output.
|
|
|
|
|
4. Continue to use canonical stage order for invalidation. For `extract`, the
|
|
|
|
|
downstream set remains `render`, `analyze`, `publish`, and `notify`.
|
|
|
|
|
5. Recheck each precomputed stage decision immediately before acting on it.
|
|
|
|
|
When an earlier stage invalidates a later succeeded stage during the same
|
|
|
|
|
full run, the later stage must execute instead of honoring its original skip
|
|
|
|
|
decision.
|
|
|
|
|
6. Use stable bounded stale reasons that distinguish forced replacement,
|
|
|
|
|
changed upstream result, upstream failure, and upstream self-skip where that
|
|
|
|
|
distinction is operationally useful.
|
|
|
|
|
7. Remove or consolidate the old success-only force invalidation path after the
|
|
|
|
|
new invariant owns all replacement cases. Do not invalidate stages earlier
|
|
|
|
|
than the replaced stage.
|
|
|
|
|
|
|
|
|
|
Tests:
|
|
|
|
|
|
|
|
|
|
- `internal/artifactmodel`: JSON behavior for complete and omitted metadata.
|
|
|
|
|
- `internal/manifest`: old manifest compatibility and self-skip clearing of
|
|
|
|
|
earlier outputs.
|
|
|
|
|
- `internal/app`: explicit source ID precedence, fallback inference,
|
|
|
|
|
provenance copying, self-skip persistence, continuation after self-skip,
|
|
|
|
|
and rejection of skipped results containing outputs.
|
|
|
|
|
- extend `internal/app/extract_lifecycle_test.go` with a full-plan or focused
|
|
|
|
|
multi-stage fixture in which disabled `extract` and downstream `analyze` first
|
|
|
|
|
complete, Notarius is enabled, and the next ordinary run executes both
|
|
|
|
|
`extract` and the previously succeeded downstream stages;
|
|
|
|
|
- a failed extract followed by a successful ordinary retry invalidates and
|
|
|
|
|
reruns previously succeeded downstream stages;
|
|
|
|
|
- `run-stage extract --force` followed by `notarius_disabled` leaves downstream
|
|
|
|
|
succeeded stages stale;
|
|
|
|
|
- forced extraction failure leaves downstream succeeded stages stale;
|
|
|
|
|
- repeated identical disabled self-skip does not repeatedly stale downstream
|
|
|
|
|
stages;
|
|
|
|
|
- successful resume reuse does not invalidate downstream stages;
|
|
|
|
|
- resume-validation error preserves both the prior extract success and
|
|
|
|
|
downstream state; and
|
|
|
|
|
- forcing `render` still does not stale `extract`.
|
|
|
|
|
|
|
|
|
|
Exit criteria:
|
|
|
|
|
|
|
|
|
|
- Existing stage outputs and manifests remain compatible.
|
|
|
|
|
- A synthetic stage can self-skip without being recorded as succeeded.
|
|
|
|
|
- A later invocation will reconsider a self-skipped stage.
|
|
|
|
|
- `go test ./...`, `go vet ./...`, and `go build ./cmd/narratio` pass.
|
|
|
|
|
|
|
|
|
|
## Stage 2: Add Notarius Configuration And Extraction Source Policy
|
|
|
|
|
|
|
|
|
|
Add strict configuration and stable source identity, but do not invoke
|
|
|
|
|
Notarius or register `extract` yet.
|
|
|
|
|
|
|
|
|
|
Implementation:
|
|
|
|
|
|
|
|
|
|
1. Add an optional `Notarius *NotariusConfig` field to `PipelineConfig`.
|
|
|
|
|
2. Define:
|
|
|
|
|
- `enabled` with false default;
|
|
|
|
|
- `binary` with `notarius` default;
|
|
|
|
|
- `config_path`;
|
|
|
|
|
- `pipeline_id`;
|
|
|
|
|
- `timeout` with `3h` default;
|
|
|
|
|
- optional `working_directory`; and
|
|
|
|
|
- `outputs map[string]NotariusOutputConfig`.
|
|
|
|
|
3. Define each output with required `lane_id`, `media_type`, `schema_id`, and
|
|
|
|
|
`schema_version`, plus optional `module_key`.
|
|
|
|
|
4. Apply defaults centrally. When enabled, resolve `config_path` to an
|
|
|
|
|
absolute path using the same configuration-source semantics as comparable
|
|
|
|
|
adapter paths. Default `working_directory` to the resolved config file's
|
|
|
|
|
directory, and resolve an explicitly configured working directory to an
|
|
|
|
|
absolute path.
|
|
|
|
|
5. Validate enabled configuration before stage execution:
|
|
|
|
|
- non-empty binary, config path, pipeline ID, and output map;
|
|
|
|
|
- positive duration;
|
|
|
|
|
- non-empty output contract fields;
|
|
|
|
|
- output keys accepted by the existing configured-artifact key policy;
|
|
|
|
|
- unique normalized extraction source IDs;
|
|
|
|
|
- unique lane IDs;
|
|
|
|
|
- no collision with built-in, configured Scriptorium, or previous-session
|
|
|
|
|
source identities.
|
|
|
|
|
6. Add `SourceKindExtraction` and helpers for the exact
|
|
|
|
|
`narratio.extraction.<key>` family in `internal/artifactpolicy` and
|
|
|
|
|
`internal/artifacts`. Do not accept arbitrary unconfigured extraction
|
|
|
|
|
sources at configuration validation boundaries.
|
|
|
|
|
7. Extend Scriptorium input and publish source validation to accept only
|
|
|
|
|
extraction keys declared by the same effective pipeline configuration.
|
|
|
|
|
8. Do not expose a Notarius session-ID setting, lane-selection flag, prompt,
|
|
|
|
|
profile, model, retry, or reference configuration.
|
|
|
|
|
|
|
|
|
|
Tests:
|
|
|
|
|
|
|
|
|
|
- strict YAML decoding and unknown-field rejection;
|
|
|
|
|
- omitted and disabled behavior;
|
|
|
|
|
- defaults and path resolution from pipeline-file location;
|
|
|
|
|
- enabled required fields and positive timeout;
|
|
|
|
|
- invalid/safe keys, normalized collisions, duplicate lanes, incomplete
|
|
|
|
|
contracts, and source-family collisions;
|
|
|
|
|
- valid and unknown extraction sources in Scriptorium inputs and publish
|
|
|
|
|
rules; and
|
|
|
|
|
- existing configurations and maintained examples continue to load.
|
|
|
|
|
|
|
|
|
|
Exit criteria:
|
|
|
|
|
|
|
|
|
|
- A complete Notarius configuration loads and normalizes deterministically.
|
|
|
|
|
- Invalid downstream extraction references fail during config validation.
|
|
|
|
|
- No runtime code invokes Notarius yet.
|
|
|
|
|
- `go test ./internal/config ./internal/artifactpolicy ./internal/artifacts`
|
|
|
|
|
and `go test ./...` pass.
|
|
|
|
|
|
|
|
|
|
## Stage 3: Add Safe Immutable Directory Promotion
|
|
|
|
|
|
|
|
|
|
Implement the filesystem primitive used to promote a validated external bundle
|
|
|
|
|
without exposing a partial or mixed durable directory.
|
|
|
|
|
|
|
|
|
|
Implementation:
|
|
|
|
|
|
|
|
|
|
1. Add a narrowly named directory-promotion operation to `internal/fileops`.
|
|
|
|
|
It accepts an existing source directory and a new destination directory.
|
|
|
|
|
2. Require a non-empty source and destination. The destination must not exist.
|
|
|
|
|
3. Create a temporary sibling of the destination so final installation uses a
|
|
|
|
|
same-filesystem rename.
|
|
|
|
|
4. Recursively walk with `Lstat` semantics:
|
|
|
|
|
- permit directories and regular files only;
|
|
|
|
|
- reject symlinks, devices, sockets, named pipes, and other special files;
|
|
|
|
|
- never follow a symlink;
|
|
|
|
|
- use deterministic lexical traversal;
|
|
|
|
|
- preserve relative layout; and
|
|
|
|
|
- use bounded safe permissions rather than copying unsafe mode bits.
|
|
|
|
|
5. Copy and sync each regular file, sync the completed temporary tree where
|
|
|
|
|
practical, and rename the temporary directory to the unique destination.
|
|
|
|
|
6. On any failure, remove only the operation's validated temporary sibling.
|
|
|
|
|
Never remove or overwrite the source, destination, workspace root, or a
|
|
|
|
|
broad caller-provided directory.
|
|
|
|
|
7. Return an error if a concurrent actor creates the destination before final
|
|
|
|
|
installation.
|
|
|
|
|
|
|
|
|
|
Tests using `t.TempDir()`:
|
|
|
|
|
|
|
|
|
|
- nested regular-tree success and byte-for-byte preservation;
|
|
|
|
|
- deterministic layout;
|
|
|
|
|
- destination already exists;
|
|
|
|
|
- source is not a directory;
|
|
|
|
|
- symlink to a file, symlink to a directory, and escaping symlink rejection;
|
|
|
|
|
- representative special-file rejection where portable;
|
|
|
|
|
- copy/install failure cleanup with no partial destination; and
|
|
|
|
|
- source preservation after success and failure.
|
|
|
|
|
|
|
|
|
|
Exit criteria:
|
|
|
|
|
|
|
|
|
|
- A promoted directory appears only as a complete unique tree.
|
|
|
|
|
- The operation cannot follow symlinks or replace an existing destination.
|
|
|
|
|
- `go test ./internal/fileops` and `go test ./...` pass.
|
|
|
|
|
|
|
|
|
|
## Stage 4: Implement The Notarius Adapter
|
|
|
|
|
|
|
|
|
|
Create the external subprocess boundary independently of stage policy.
|
|
|
|
|
|
|
|
|
|
Implementation:
|
|
|
|
|
|
|
|
|
|
1. Add `internal/adapters/notarius` with:
|
|
|
|
|
- a `Runner` interface;
|
|
|
|
|
- transport-neutral request and result models;
|
|
|
|
|
- receipt, index, lane-descriptor, and pipeline-wide-descriptor models;
|
|
|
|
|
- a production subprocess runner; and
|
|
|
|
|
- a small configurable fake for stage tests.
|
|
|
|
|
2. Build exactly:
|
|
|
|
|
|
|
|
|
|
```text
|
|
|
|
|
notarius run <pipeline-id> --config <path> --input <path>
|
|
|
|
|
--output-dir <path> --json
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Use separate argument elements, absolute supplied paths, the configured
|
|
|
|
|
working directory, inherited environment, and Narratio's shared subprocess
|
|
|
|
|
timeout/cancellation behavior. Do not pass `--session-id`.
|
|
|
|
|
3. Write stdout directly to the configured receipt path and stderr to the
|
|
|
|
|
configured log path. Do not combine streams.
|
|
|
|
|
4. On nonzero exit, cancellation, or timeout, return the subprocess error and
|
|
|
|
|
do not parse stdout.
|
|
|
|
|
5. After exit zero, read receipt bytes with an explicit upper bound, decode
|
|
|
|
|
`notarius.run-result.v1`, tolerate unknown fields, and validate required
|
|
|
|
|
fields and matching pipeline ID.
|
|
|
|
|
6. Require an absolute `output_directory` and confine it beneath the absolute
|
|
|
|
|
output root from the request. Confine the relative `index_file` beneath the
|
|
|
|
|
returned directory using Narratio's root-scoped path helpers. Reject
|
|
|
|
|
absolute logical paths, traversal, symlinks at consumed paths, and
|
|
|
|
|
non-regular files.
|
|
|
|
|
7. Decode the supported production JSON index tolerantly. Validate required
|
|
|
|
|
management paths and lane descriptors, reject duplicate lane IDs, and
|
|
|
|
|
confine every lane and pipeline-wide descriptor file beneath the bundle
|
|
|
|
|
root.
|
|
|
|
|
8. Load `rejected.json` and `warnings.json` with explicit size bounds and
|
|
|
|
|
tolerant decoding. Return only their generic summaries, including lane ID
|
|
|
|
|
and reason code where present; do not read lane payload bodies or interpret
|
|
|
|
|
D&D fields. Keep free-form external messages bounded for immediate errors
|
|
|
|
|
and out of manifest metadata.
|
|
|
|
|
9. Return descriptors and paths without deciding which lanes are required or
|
|
|
|
|
whether their configured contracts match.
|
|
|
|
|
10. Do not implement automatic `notarius config validate` execution.
|
|
|
|
|
|
|
|
|
|
Tests:
|
|
|
|
|
|
|
|
|
|
- exact argument order and omission of `--session-id`;
|
|
|
|
|
- working directory, environment inheritance, stream separation, cancellation,
|
|
|
|
|
timeout, and nonzero exit;
|
|
|
|
|
- stdout ignored after failure;
|
|
|
|
|
- receipt size limit, malformed JSON, unsupported version, missing fields,
|
|
|
|
|
pipeline mismatch, and tolerated unknown fields;
|
|
|
|
|
- output-directory escape from the requested output root;
|
|
|
|
|
- index malformed/unsupported shapes, duplicate lanes, missing management
|
|
|
|
|
paths, and tolerated optional fields;
|
|
|
|
|
- malformed or oversized rejection/warning documents and tolerated unknown
|
|
|
|
|
summary fields;
|
|
|
|
|
- absolute logical paths, lexical traversal, root-prefix confusion, symlink
|
|
|
|
|
paths, and descriptor escape attempts; and
|
|
|
|
|
- valid lane plus chunk-map/evidence-context discovery.
|
|
|
|
|
|
|
|
|
|
Use a test helper subprocess or injected shared runner. Do not require a real
|
|
|
|
|
Notarius binary.
|
|
|
|
|
|
|
|
|
|
Exit criteria:
|
|
|
|
|
|
|
|
|
|
- The adapter fully enforces the generic Notarius subprocess and discovery
|
|
|
|
|
boundary without D&D policy.
|
|
|
|
|
- `go test ./internal/adapters/notarius ./internal/adapters/subprocess` and
|
|
|
|
|
`go test ./...` pass.
|
|
|
|
|
|
|
|
|
|
## Stage 5: Implement Extract-Stage Execution And Bundle Materialization
|
|
|
|
|
|
|
|
|
|
Implement the stage behind direct package tests, but do not yet add it to the
|
|
|
|
|
canonical `stage.All()` registry.
|
|
|
|
|
|
|
|
|
|
Implementation:
|
|
|
|
|
|
|
|
|
|
1. Add `Notarius notarius.Runner` to `stage.Env` and default production
|
|
|
|
|
composition in `internal/app/runner.go` when enabled extraction is selected.
|
|
|
|
|
2. Add canonical path helpers for:
|
|
|
|
|
- the run-local extract directory;
|
|
|
|
|
- receipt and stderr paths;
|
|
|
|
|
- the run-local Notarius output root; and
|
|
|
|
|
- immutable `artifacts/notarius/<narratio-run-id>` destinations.
|
|
|
|
|
3. Implement `extractStage` with name `extract` and declarations matching its
|
|
|
|
|
final-trimmed input and structured outputs.
|
|
|
|
|
4. When configuration is absent or disabled, return the explicit skipped
|
|
|
|
|
result with `notarius_disabled` and no outputs.
|
|
|
|
|
5. Resolve `narratio.transcript.final_trimmed` through the manifest-aware
|
|
|
|
|
resolver. Do not fall back to guessed filenames outside that resolver.
|
|
|
|
|
6. Invoke the adapter once with resolved absolute paths.
|
|
|
|
|
7. Apply required-output policy:
|
|
|
|
|
- find each configured output by exact lane ID;
|
|
|
|
|
- require exactly one descriptor;
|
|
|
|
|
- compare media type, schema ID/version, and optional module key;
|
|
|
|
|
- treat rejected or absent configured lanes as failure;
|
|
|
|
|
- tolerate unconfigured lanes;
|
|
|
|
|
- require each selected file to be regular, non-empty, and syntactically
|
|
|
|
|
valid JSON; and
|
|
|
|
|
- compute a staging checksum for later copy verification.
|
|
|
|
|
8. Validate the complete bundle tree for promotion, then use Stage 3's helper
|
|
|
|
|
to copy it to the unique immutable destination.
|
|
|
|
|
9. Re-resolve the promoted index and configured descriptor paths rather than
|
|
|
|
|
retaining staging paths. Compute authoritative SHA-256 checksums from the
|
|
|
|
|
promoted files and fail if promoted bytes differ from the validated source.
|
|
|
|
|
10. Return one `artifacts.Ref` per configured lane with explicit extraction
|
|
|
|
|
source ID, checksum, contract metadata, and Notarius external provenance.
|
|
|
|
|
11. Return the promoted `index.json` as a `notarius_index` output without a
|
|
|
|
|
selectable source ID.
|
|
|
|
|
12. Return bounded stage metadata containing durable bundle root, receipt and
|
|
|
|
|
diagnostic paths, receipt counts/status, rejection/warning summaries, the
|
|
|
|
|
producing Narratio run ID, and a deterministic fingerprint of the resolved
|
|
|
|
|
binary, config path, pipeline ID, timeout, working directory, and sorted
|
|
|
|
|
output-contract map.
|
|
|
|
|
13. Do not delete run-local diagnostics or failed bundles.
|
|
|
|
|
|
|
|
|
|
Tests:
|
|
|
|
|
|
|
|
|
|
- disabled self-skip;
|
|
|
|
|
- missing/invalid final-trimmed input before adapter invocation;
|
|
|
|
|
- exact adapter request construction;
|
|
|
|
|
- required, missing, rejected, duplicate, incompatible, and unconfigured
|
|
|
|
|
lanes;
|
|
|
|
|
- non-empty and JSON-syntax validation without D&D decoding;
|
|
|
|
|
- immutable promotion and promoted-path re-resolution;
|
|
|
|
|
- checksums, explicit source IDs, nested metadata, and external provenance;
|
|
|
|
|
- adapter and promotion failures do not return successful outputs; and
|
|
|
|
|
- complete bundle files are preserved while unknown files are not registered
|
|
|
|
|
as sources.
|
|
|
|
|
|
|
|
|
|
Exit criteria:
|
|
|
|
|
|
|
|
|
|
- Direct extract-stage execution produces a complete manifest-ready result
|
|
|
|
|
using only a fake Notarius runner.
|
|
|
|
|
- The stage is not yet in normal CLI plans.
|
|
|
|
|
- `go test ./internal/stage ./internal/artifacts ./internal/fileops` and
|
|
|
|
|
`go test ./...` pass.
|
|
|
|
|
|
|
|
|
|
## Stage 6: Integrate Canonical Ordering, CLI Lifecycle, Force, And Resume
|
|
|
|
|
|
|
|
|
|
Make `extract` a real pipeline stage and complete its orchestration semantics.
|
|
|
|
|
|
|
|
|
|
Implementation:
|
|
|
|
|
|
|
|
|
|
1. Register `extractStage` in `stage.All()` immediately after `trim` and before
|
|
|
|
|
`render`.
|
|
|
|
|
2. Update all hard-coded or user-visible stage inventories, stage-name
|
|
|
|
|
validation, CLI help, planning, prerequisite checks, and test fixtures.
|
|
|
|
|
3. Confirm full and single-stage commands can select `extract`. Do not add a
|
|
|
|
|
lane-selection flag and do not broaden `--artifacts` beyond Scriptorium
|
|
|
|
|
artifact selection.
|
|
|
|
|
4. Add an optional stage resume-validation contract. Use a result containing:
|
|
|
|
|
- resumable boolean; and
|
|
|
|
|
- bounded non-resumable reason.
|
|
|
|
|
Reserve an ordinary error for cases where a safe decision cannot be made.
|
|
|
|
|
5. Before skipping a succeeded stage, `executeStages` invokes the optional
|
|
|
|
|
validator. A non-resumable result marks the session stage stale with its
|
|
|
|
|
reason and schedules it to run. A validation error stops execution without
|
|
|
|
|
mutating the succeeded record.
|
|
|
|
|
6. Implement extraction resume validation against:
|
|
|
|
|
- enabled state;
|
|
|
|
|
- stored invocation/config-contract fingerprint;
|
|
|
|
|
- succeeded extract record;
|
|
|
|
|
- immutable bundle and index paths;
|
|
|
|
|
- exact configured source set;
|
|
|
|
|
- regular confined lane files;
|
|
|
|
|
- checksums;
|
|
|
|
|
- contracts; and
|
|
|
|
|
- Notarius external provenance.
|
|
|
|
|
Require the producing Narratio run ID recorded in extract metadata, output
|
|
|
|
|
records, and the immutable bundle path to agree. Do not compare it with the
|
|
|
|
|
session manifest's top-level `run_id`, which identifies the latest
|
|
|
|
|
invocation and may legitimately differ after resume.
|
|
|
|
|
7. Do not inspect or hash Notarius's transitive reference/profile closure.
|
|
|
|
|
8. Verify canonical downstream invalidation:
|
|
|
|
|
- force through `trim` stales extract and later succeeded stages;
|
|
|
|
|
- force `extract` stales render and later succeeded stages;
|
|
|
|
|
- force `render` does not stale extract.
|
|
|
|
|
9. Verify existing session manifests without an extract record normalize and
|
|
|
|
|
execute the new stage rather than failing migration.
|
|
|
|
|
|
|
|
|
|
Tests:
|
|
|
|
|
|
|
|
|
|
- exact canonical order and CLI stage inventory;
|
|
|
|
|
- full-plan and `run-stage extract` behavior;
|
|
|
|
|
- disabled skip followed by later enablement;
|
|
|
|
|
- successful resume skip;
|
|
|
|
|
- missing, tampered, incompatible, or config-mismatched outputs trigger stale
|
|
|
|
|
and rerun;
|
|
|
|
|
- unsafe validation errors stop without corrupting prior success;
|
|
|
|
|
- force and downstream invalidation in each direction named above; and
|
|
|
|
|
- old manifest compatibility.
|
|
|
|
|
|
|
|
|
|
Exit criteria:
|
|
|
|
|
|
|
|
|
|
- Normal execution includes `extract` in the agreed position.
|
|
|
|
|
- Disabled and resume behavior cannot make obsolete outputs current.
|
|
|
|
|
- `go test ./internal/stage ./internal/app ./internal/manifest`,
|
|
|
|
|
- Enabling, retrying, forcing, failing, or disabling extraction cannot leave an
|
|
|
|
|
incompatible succeeded analyze/publish result current.
|
|
|
|
|
- Repeated unchanged disabled extraction remains inexpensive and stable.
|
|
|
|
|
- `go test ./internal/app ./internal/stage ./internal/manifest`,
|
|
|
|
|
`go test ./...`, `go vet ./...`, and `go build ./cmd/narratio` pass.
|
|
|
|
|
|
|
|
|
|
## Stage 7: Integrate Extraction Sources With The Runtime Catalog And Analyze
|
|
|
|
|
## Stage 11: Exclude Run-Local Notarius Bundles From Implicit Publication
|
|
|
|
|
|
|
|
|
|
Make succeeded extraction lanes selectable by configured Scriptorium
|
|
|
|
|
artifacts.
|
|
|
|
|
Enforce the existing explicit-only publication boundary while preserving
|
|
|
|
|
ordinary run-record diagnostics.
|
|
|
|
|
|
|
|
|
|
Implementation:
|
|
|
|
|
|
|
|
|
|
1. Extend `ArtifactCatalog` with extraction definitions registered from the
|
|
|
|
|
effective Notarius output map.
|
|
|
|
|
2. Add one shared manifest-backed hydration operation for extraction sources.
|
|
|
|
|
It accepts session paths, manifest, and configured extraction definitions
|
|
|
|
|
and marks entries available only after verifying the roadmap's full
|
|
|
|
|
successful-stage, source, path, checksum, contract, and provenance rules.
|
|
|
|
|
3. Add a distinct extraction provenance value such as
|
|
|
|
|
`manifest.current_extract_run`.
|
|
|
|
|
4. Extend `ResolveSessionArtifactWithCatalog` to resolve configured extraction
|
|
|
|
|
sources through catalog availability. It must never scan the Notarius
|
|
|
|
|
artifact directory or accept an incidental file.
|
|
|
|
|
5. Refactor catalog construction enough that extraction registration and
|
|
|
|
|
hydration are reusable by analyze and publish. Do not create a generic
|
|
|
|
|
workflow abstraction or move Scriptorium execution policy into artifacts.
|
|
|
|
|
6. Update analyze catalog creation to include effective extraction
|
|
|
|
|
definitions and current manifest state before planning Scriptorium inputs.
|
|
|
|
|
7. Preserve required/optional input behavior. Required unavailable extraction
|
|
|
|
|
inputs fail with actionable extract configuration/rerun guidance; optional
|
|
|
|
|
inputs are omitted normally.
|
|
|
|
|
8. Do not automatically add extraction inputs to any configured artifact.
|
|
|
|
|
1. Update publish run-file collection so the canonical run-local
|
|
|
|
|
`extract/notarius-output/` subtree is never included in the run-record upload.
|
|
|
|
|
Match the exact slash-normalized relative subtree; do not use a broad
|
|
|
|
|
substring rule that could suppress unrelated files.
|
|
|
|
|
2. Keep the Notarius receipt and stderr files eligible for the existing run
|
|
|
|
|
archive. They are diagnostics, not the published bundle. Preserve current
|
|
|
|
|
audio exclusion and all unrelated run-file behavior.
|
|
|
|
|
3. Do not scan or upload the durable `artifacts/notarius/<run-id>/` directory.
|
|
|
|
|
A configured lane may still be uploaded only through an explicit
|
|
|
|
|
`pipeline.publish.outputs` rule resolved through the artifact catalog.
|
|
|
|
|
4. Preserve locks, required/optional output handling, upload ordering, current
|
|
|
|
|
manifest publication, and the final current-run pointer commit.
|
|
|
|
|
5. Update publish metadata so excluded staging-bundle files are not counted in
|
|
|
|
|
`run_files_uploaded` or listed in `run_uploaded_paths`.
|
|
|
|
|
|
|
|
|
|
Tests:
|
|
|
|
|
|
|
|
|
|
- registration, lookup, deterministic ordering, and source-family collision;
|
|
|
|
|
- successful current-manifest hydration;
|
|
|
|
|
- missing stage, skipped, failed, stale, interrupted, missing-source,
|
|
|
|
|
internally inconsistent producer identity, incompatible-contract,
|
|
|
|
|
missing-file, tampered-checksum, and unsafe-path rejection;
|
|
|
|
|
- required and optional analyze input behavior;
|
|
|
|
|
- selected Scriptorium artifact execution receives only explicitly configured
|
|
|
|
|
extraction inputs; and
|
|
|
|
|
- no object-store calls or directory scanning occur during analyze
|
|
|
|
|
extraction resolution.
|
|
|
|
|
- materialize a realistic current-run `extract/notarius-output/<external-run>/`
|
|
|
|
|
tree containing index, management, lane, pipeline-wide, and unknown regular
|
|
|
|
|
files, then prove none are uploaded under the run prefix;
|
|
|
|
|
- prove receipt and stderr diagnostics in the same extract directory remain
|
|
|
|
|
ordinary run uploads;
|
|
|
|
|
- prove no Notarius bundle member is uploaded when there is no explicit
|
|
|
|
|
extraction publish rule;
|
|
|
|
|
- prove one explicit extraction rule uploads only its durable selected lane to
|
|
|
|
|
the configured session destination; and
|
|
|
|
|
- retain assertions for commit-marker order, locks, previous-cache publication,
|
|
|
|
|
and unrelated run files.
|
|
|
|
|
|
|
|
|
|
Exit criteria:
|
|
|
|
|
|
|
|
|
|
- A fake successful extract run can feed selected JSON files into a fake
|
|
|
|
|
Scriptorium analysis run.
|
|
|
|
|
- No failed or incidental bundle can become available through the catalog.
|
|
|
|
|
- `go test ./internal/artifacts ./internal/stage ./internal/config` and
|
|
|
|
|
`go test ./...` pass.
|
|
|
|
|
|
|
|
|
|
## Stage 8: Integrate Extraction Sources With Publish And Operator Inspection
|
|
|
|
|
|
|
|
|
|
Complete downstream artifact handling without automatically publishing the
|
|
|
|
|
bundle.
|
|
|
|
|
|
|
|
|
|
Implementation:
|
|
|
|
|
|
|
|
|
|
1. Use Stage 7's shared extraction catalog registration and hydration in
|
|
|
|
|
publish planning and source resolution.
|
|
|
|
|
2. Permit explicit publish rules whose source is a configured extraction
|
|
|
|
|
source. Preserve existing destination policy, lock behavior, required
|
|
|
|
|
output policy, selected-Scriptorium behavior, and remote commit order.
|
|
|
|
|
3. Do not publish the complete Notarius bundle unless individual explicit
|
|
|
|
|
rules name selectable sources. `notarius_index` remains non-selectable in
|
|
|
|
|
the initial feature.
|
|
|
|
|
4. Ensure the published session manifest retains extraction artifact contract
|
|
|
|
|
and external provenance metadata through ordinary serialization.
|
|
|
|
|
5. Extend existing artifact/status/inspection output only where necessary so
|
|
|
|
|
configured extraction sources report planned, available, unavailable, and
|
|
|
|
|
published states consistently. Do not expose payload contents.
|
|
|
|
|
6. Confirm restore safely round-trips explicitly published extraction files
|
|
|
|
|
through existing manifest and confined-path behavior; add code only if a
|
|
|
|
|
real incompatibility is found.
|
|
|
|
|
|
|
|
|
|
Tests:
|
|
|
|
|
|
|
|
|
|
- successful explicit lane publication;
|
|
|
|
|
- required missing or invalid extraction source failure;
|
|
|
|
|
- optional/unselected behavior remains consistent;
|
|
|
|
|
- publish lock and commit-marker order remain unchanged;
|
|
|
|
|
- `--artifacts` does not partially select Notarius lanes;
|
|
|
|
|
- manifest metadata survives publish serialization and restore loading; and
|
|
|
|
|
- operator inspection never reads or prints extraction payload bodies.
|
|
|
|
|
|
|
|
|
|
Exit criteria:
|
|
|
|
|
|
|
|
|
|
- Explicit extraction publish rules work through the shared catalog.
|
|
|
|
|
- No implicit whole-bundle publication is introduced.
|
|
|
|
|
- No run-local or durable Notarius bundle is published wholesale.
|
|
|
|
|
- Explicit configured lane publication remains functional and auditable.
|
|
|
|
|
- `go test ./internal/stage ./internal/app ./internal/artifacts` and
|
|
|
|
|
`go test ./...` pass.
|
|
|
|
|
|
|
|
|
|
## Stage 9: Add Maintained Examples, Current Documentation, And Final Validation
|
|
|
|
|
## Stage 12: Enforce Canonical Notarius Management-File Semantics
|
|
|
|
|
|
|
|
|
|
Finish the public and maintainer contract only after the implementation is
|
|
|
|
|
complete.
|
|
|
|
|
Make the adapter's accepted production contract agree with extract, resume, and
|
|
|
|
|
catalog invariants.
|
|
|
|
|
|
|
|
|
|
Implementation:
|
|
|
|
|
|
|
|
|
|
1. Read Notarius's current published lane contracts and add exact media type,
|
|
|
|
|
schema ID, schema version, and module-key constraints for all ten lanes to
|
|
|
|
|
a maintained Narratio complete example. Do not guess or copy stale roadmap
|
|
|
|
|
placeholders.
|
|
|
|
|
2. Add a small maintained Scriptorium example that consumes a purpose-specific
|
|
|
|
|
subset of extraction sources rather than all ten.
|
|
|
|
|
3. Add `docs/integrations/notarius.md` as Narratio's external-consumer
|
|
|
|
|
contract. Link to Notarius's canonical documents instead of duplicating
|
|
|
|
|
complete schemas.
|
|
|
|
|
4. Add `docs/internal/stage-extract.md` for implemented stage mechanics,
|
|
|
|
|
lifecycle, resume validation, failure behavior, and focused tests.
|
|
|
|
|
5. Update the canonical owners listed in the roadmap's Documentation
|
|
|
|
|
Deliverables section. Keep configuration fields/defaults only in
|
|
|
|
|
`docs/config.md`, commands only in `docs/cli.md`, physical layout and force
|
|
|
|
|
procedures only in `docs/operations.md`, and implementation mechanics only
|
|
|
|
|
in internal documents.
|
|
|
|
|
6. Update every implemented stage inventory to show `extract` between `trim`
|
|
|
|
|
and `render`.
|
|
|
|
|
7. Add troubleshooting guidance for missing Notarius, nonzero exit,
|
|
|
|
|
receipt/index incompatibility, required-lane rejection, resume
|
|
|
|
|
invalidation, and the requirement to force extraction after changing
|
|
|
|
|
Notarius's transitive configuration inputs.
|
|
|
|
|
8. Review the roadmap status. If every acceptance criterion is implemented,
|
|
|
|
|
mark it complete or move completed planning material according to repository
|
|
|
|
|
convention without deleting useful decision context prematurely.
|
|
|
|
|
9. Check all changed links, examples, commands, field names, defaults, schemas,
|
|
|
|
|
and paths against the implementation.
|
|
|
|
|
1. Re-read Notarius's current `run-result.md` and `json-output.md` before editing
|
|
|
|
|
and keep tolerant decoding of unknown fields within the supported schema.
|
|
|
|
|
2. For `notarius.run-result.v1` using the production JSON output, require the
|
|
|
|
|
receipt's logical `index_file` to be exactly `index.json`. A different safe
|
|
|
|
|
relative path is semantically incompatible and must fail adapter discovery,
|
|
|
|
|
not produce a successful result that later becomes non-resumable.
|
|
|
|
|
3. Require the production index management fields to be exactly:
|
|
|
|
|
- `manifest_file: manifest.json`;
|
|
|
|
|
- `rejected_file: rejected.json`; and
|
|
|
|
|
- `warnings_file: warnings.json`.
|
|
|
|
|
4. Retain all existing confinement, regular-file, symlink, size-limit, and
|
|
|
|
|
tolerant unknown-field checks after the exact semantic checks. Do not tighten
|
|
|
|
|
optional lane descriptor fields beyond Notarius's published contract.
|
|
|
|
|
5. Keep the canonical index invariant shared by stage output, resume validation,
|
|
|
|
|
and catalog hydration. Avoid introducing a second configurable or inferred
|
|
|
|
|
management-path model.
|
|
|
|
|
6. Return errors that identify the incompatible field and observed value without
|
|
|
|
|
including transcript-derived payload content.
|
|
|
|
|
|
|
|
|
|
Validation:
|
|
|
|
|
Tests:
|
|
|
|
|
|
|
|
|
|
- focused example/configuration validation tests;
|
|
|
|
|
- `go test ./...`;
|
|
|
|
|
- `go vet ./...`;
|
|
|
|
|
- `go build ./cmd/narratio`;
|
|
|
|
|
- inspect `git diff --check`;
|
|
|
|
|
- confirm ordinary tests use no live Notarius, PromptKit, LLM provider,
|
|
|
|
|
credentials, network service, or mutable external state; and
|
|
|
|
|
- confirm no secrets or private campaign content were added.
|
|
|
|
|
- reject alternate but confined values such as `nested/index.json`,
|
|
|
|
|
`./index.json`, and renamed management files;
|
|
|
|
|
- continue rejecting absolute paths, traversal, symlinks, and missing files;
|
|
|
|
|
- continue accepting canonical documents with unknown optional fields;
|
|
|
|
|
- prove a successful adapter result promotes to canonical `index.json`; and
|
|
|
|
|
- prove a successful extract record is immediately accepted by resume validation
|
|
|
|
|
and catalog hydration without normalization or path rewriting.
|
|
|
|
|
|
|
|
|
|
Exit criteria:
|
|
|
|
|
|
|
|
|
|
- Every feature-roadmap acceptance criterion is satisfied.
|
|
|
|
|
- Maintained examples load under strict configuration validation.
|
|
|
|
|
- Current-behavior documentation accurately describes the shipped feature
|
|
|
|
|
without duplicating Notarius-owned schema definitions.
|
|
|
|
|
- The repository-wide validation suite passes.
|
|
|
|
|
- Every adapter success satisfies the canonical management-file assumptions of
|
|
|
|
|
extraction, resume, and catalog code.
|
|
|
|
|
- `go test ./internal/adapters/notarius ./internal/stage ./internal/artifacts`
|
|
|
|
|
and `go test ./...` pass.
|
|
|
|
|
|
|
|
|
|
## Stage 13: Complete And Document The Atomic-Promotion Platform Contract
|
|
|
|
|
|
|
|
|
|
Replace the accidental Linux-only behavior with an explicit, tested support
|
|
|
|
|
boundary. Support Linux, macOS, and Windows; fail early and clearly on other
|
|
|
|
|
platforms until they gain an atomic no-replace primitive.
|
|
|
|
|
|
|
|
|
|
Implementation:
|
|
|
|
|
|
|
|
|
|
1. Keep Linux installation based on `renameat2(RENAME_NOREPLACE)`.
|
|
|
|
|
2. Add a Darwin implementation using `renamex_np(RENAME_EXCL)` from
|
|
|
|
|
`golang.org/x/sys/unix`.
|
|
|
|
|
3. Add a Windows implementation using the no-replace form of
|
|
|
|
|
`golang.org/x/sys/windows.MoveFileEx`. The sibling temporary directory keeps
|
|
|
|
|
the move on one volume; do not set the replace-existing flag.
|
|
|
|
|
4. Split directory-sync behavior by platform where necessary. Unsupported
|
|
|
|
|
directory syncing may be treated as best effort only for documented
|
|
|
|
|
platform-specific unsupported-operation errors; do not suppress ordinary I/O
|
|
|
|
|
or permission failures.
|
|
|
|
|
5. Add a small build-specific capability check so an unsupported platform fails
|
|
|
|
|
before copying the bundle into a temporary tree. Retain safe cleanup if any
|
|
|
|
|
later platform operation fails.
|
|
|
|
|
6. Preserve the core invariant on every supported platform: a concurrent actor
|
|
|
|
|
that creates the destination wins or causes a clean error; Narratio never
|
|
|
|
|
replaces that destination.
|
|
|
|
|
7. Document Linux, macOS, and Windows as the supported atomic-promotion
|
|
|
|
|
platforms and the explicit extraction limitation on other operating systems.
|
|
|
|
|
Do not imply that all of Narratio has a broader support guarantee than its
|
|
|
|
|
existing documentation establishes.
|
|
|
|
|
|
|
|
|
|
Tests and validation:
|
|
|
|
|
|
|
|
|
|
- retain the portable promotion conformance tests for successful nested copy,
|
|
|
|
|
source preservation, existing destination, cleanup, symlink rejection, and
|
|
|
|
|
install collision;
|
|
|
|
|
- add platform-specific no-replace tests that run on their native CI platform;
|
|
|
|
|
- cross-compile the fileops tests for Linux, Darwin, and Windows;
|
|
|
|
|
- confirm the unsupported-platform implementation returns its capability error
|
|
|
|
|
before creating a temporary sibling; and
|
|
|
|
|
- run `go test ./internal/fileops`, `go test ./...`, `go vet ./...`, and
|
|
|
|
|
`go build ./cmd/narratio` on the development platform.
|
|
|
|
|
|
|
|
|
|
Exit criteria:
|
|
|
|
|
|
|
|
|
|
- Directory promotion is functional and no-replace on Linux, macOS, and
|
|
|
|
|
Windows rather than merely compiling there.
|
|
|
|
|
- Unsupported platforms fail before expensive copying and have an explicit
|
|
|
|
|
documented boundary.
|
|
|
|
|
- All available native and cross-compilation checks pass.
|
|
|
|
|
|
|
|
|
|
## Stage 14: Reconcile Documentation And Perform Final Remediation Validation
|
|
|
|
|
|
|
|
|
|
Make current-behavior documentation match the corrected implementation and
|
|
|
|
|
close the remediation effort only after end-to-end verification.
|
|
|
|
|
|
|
|
|
|
Implementation:
|
|
|
|
|
|
|
|
|
|
1. Correct `docs/internal/stage-extract.md` to say that omitted or disabled
|
|
|
|
|
Notarius explicitly self-skips with `notarius_disabled`; do not describe the
|
|
|
|
|
manifest result as succeeded.
|
|
|
|
|
2. Replace the nonexistent `internal/stage/extract_resume_test.go` reference with
|
|
|
|
|
the actual focused test owner, or create that file only if tests were
|
|
|
|
|
intentionally reorganized during remediation.
|
|
|
|
|
3. Update operations, publish, and extraction documentation to state that:
|
|
|
|
|
- changed extraction outcomes stale affected downstream stages;
|
|
|
|
|
- repeated identical disabled self-skip does not cause perpetual reruns;
|
|
|
|
|
- the run-local Notarius bundle is excluded from run-record upload; and
|
|
|
|
|
- only explicit configured extraction lanes are published.
|
|
|
|
|
4. Update platform-support documentation from Stage 13 and troubleshooting for
|
|
|
|
|
unsupported atomic promotion.
|
|
|
|
|
5. Reconcile `docs/roadmap/notarius-extract-stage.md` with the remediated current
|
|
|
|
|
state. Mark the feature complete only if every original and remediation
|
|
|
|
|
acceptance criterion is satisfied.
|
|
|
|
|
6. Recheck maintained examples, internal links, stage inventories, command
|
|
|
|
|
examples, field names, schema identities, defaults, and paths against code.
|
|
|
|
|
7. Keep roadmap history concise; do not restore the former detailed completed
|
|
|
|
|
Stage 1-9 instructions to this file.
|
|
|
|
|
|
|
|
|
|
Validation:
|
|
|
|
|
|
|
|
|
|
- run focused lifecycle, adapter, fileops, catalog, analyze, publish, restore,
|
|
|
|
|
and operator tests without live external services;
|
|
|
|
|
- run focused race tests covering the changed extraction, artifact, publish,
|
|
|
|
|
fileops, and runner paths;
|
|
|
|
|
- run `go test -count=1 ./...`;
|
|
|
|
|
- run `go vet ./...`;
|
|
|
|
|
- run `go build ./cmd/narratio`;
|
|
|
|
|
- run the Stage 13 cross-compilation checks;
|
|
|
|
|
- run `git diff --check`; and
|
|
|
|
|
- confirm no secrets, private campaign content, generated binaries, or test
|
|
|
|
|
artifacts were added to the repository.
|
|
|
|
|
|
|
|
|
|
Exit criteria:
|
|
|
|
|
|
|
|
|
|
- All post-implementation review findings are corrected or explicitly bounded
|
|
|
|
|
by the documented platform contract.
|
|
|
|
|
- Current documentation accurately describes stage outcomes, invalidation,
|
|
|
|
|
publication, adapter compatibility, and supported platforms.
|
|
|
|
|
- Every progress row is `Complete`, the repository-wide validation suite passes,
|
|
|
|
|
and the target feature roadmap can truthfully remain complete.
|
|
|
|
|
|
|
|
|
|
## Open Questions
|
|
|
|
|
|
|
|
|
|
None. The target architecture, defaults, lifecycle behavior, compatibility
|
|
|
|
|
boundary, storage layout, source identity, downstream selection model, and
|
|
|
|
|
implementation order are decision-complete for implementation. If repository
|
|
|
|
|
inspection exposes a contradiction with an implemented invariant, stop at the
|
|
|
|
|
affected stage and document the concrete conflict rather than silently
|
|
|
|
|
changing this plan's target behavior.
|
|
|
|
|
None. The remediation stages adopt the long-term-maintainable defaults: general
|
|
|
|
|
runner invalidation semantics, explicit-only bundle publication, exact current
|
|
|
|
|
Notarius management-file semantics, and atomic promotion support on Linux,
|
|
|
|
|
macOS, and Windows with an explicit early failure elsewhere.
|
|
|
|
|
|
