narratio/docs/roadmap/cleanup.md

Roadmap: Legacy Config Cleanup

Status: Implemented

Problem

Narratio's current pipeline config schema still accepts fields that predate the current storage, artifact, and previous-session models:

• pipeline.storage.bucket
• pipeline.storage.prefix
• pipeline.analyzer.*
• previous_session_artifact

These names make the config reference harder to trust because they suggest supported behavior that operators should no longer use. The modern interface is:

• pipeline.storage.s3.* for remote storage.
• Scriptorium configured artifacts under pipeline.scriptorium.artifacts.
• Canonical artifact source IDs such as narratio.artifact.<configured_artifact_key>.
• Canonical previous-session artifact sources such as narratio.previous_session.artifact.<configured_artifact_key>.

Strict YAML decoding should reject removed legacy fields once this cleanup lands.

Current State

pipeline.storage.bucket and pipeline.storage.prefix were inert compatibility fields and have been removed:

• They are no longer present on config.StorageConfig.
• Strict decoding rejects them.
• Runtime S3 behavior uses pipeline.storage.s3.bucket and pipeline.storage.s3.root_prefix.
• No current code reads the top-level storage bucket or prefix fields.

pipeline.analyzer.* was legacy code surface and has been removed:

• config.PipelineConfig no longer includes analyzer config.
• Strict decoding rejects pipeline.analyzer.
• stage.Env no longer exposes an analyzer runner, and internal/adapters/analyzer has been deleted.
• Modern analyze execution is Scriptorium-backed; the analyzer adapter is not used by current stage execution.

previous_session_artifact was a live legacy behavior and has been removed:

• Config validation rejects it as an unsupported Scriptorium input source.
• The analyze stage no longer has path-based previous-artifact resolution through inputs.<name>.path.
• Tests cover canonical previous-session sources and the rejection of the legacy source.
• The canonical replacement is narratio.previous_session.artifact.<configured_artifact_key>, resolved through the previous-session cache/catalog model.

Target Model

The pipeline config schema should expose only current behavior:

• Remote storage is configured only through pipeline.storage.s3.*.
• Generated artifacts are configured only through pipeline.scriptorium.artifacts.
• Scriptorium artifact inputs use canonical source IDs.
• Previous-session artifact inputs use narratio.previous_session.artifact.<configured_artifact_key>.
• Unknown legacy fields fail strict YAML decoding.

No compatibility aliases should remain unless a future migration requirement explicitly reintroduces them.

Cleanup Order

Stage 1: Remove Inert Storage Compatibility Fields

Status: Implemented

Remove pipeline.storage.bucket and pipeline.storage.prefix.

Implementation requirements:

• Delete StorageConfig.Bucket and StorageConfig.Prefix.
• Keep StorageConfig.Backend and StorageConfig.S3.
• Confirm all runtime storage paths continue to use storage.s3.bucket and storage.s3.root_prefix.
• Update examples and docs to remove top-level storage bucket and prefix.
• Add or update strict-decode tests proving pipeline.storage.bucket and pipeline.storage.prefix are rejected.

Acceptance criteria:

• Existing S3 workflows still pass with pipeline.storage.s3.bucket.
• Pipeline configs containing top-level storage.bucket or storage.prefix fail to load.
• No docs or examples present those fields as available.

Stage 2: Remove Legacy Analyzer Schema and Adapter Surface

Status: Implemented

Remove the unused analyzer configuration and adapter contract.

Implementation requirements:

• Delete PipelineConfig.Analyzer.
• Delete AnalyzerConfig and ArtifactSettings.
• Remove analyzer timeout validation.
• Remove stage.Env.Analyzer.
• Delete internal/adapters/analyzer if no remaining code imports it.
• Remove pipeline.analyzer.* from tests, examples, and docs.
• Add or update strict-decode tests proving pipeline.analyzer is rejected.

Acceptance criteria:

• Analyze behavior remains fully Scriptorium-backed.
• No runtime code imports internal/adapters/analyzer.
• Pipeline configs containing pipeline.analyzer fail to load.
• Contributor and internal adapter docs no longer list the analyzer adapter.

Stage 3: Remove Path-Based Previous Session Artifact Source

Status: Implemented

Remove previous_session_artifact and require canonical previous-session artifact sources.

Implementation requirements:

• Remove previous_session_artifact from supported Scriptorium input sources.
• Remove analyze-stage special-case handling that resolves inputs.<name>.path for previous artifacts.
• Keep canonical handling for narratio.previous_session.artifact.<configured_artifact_key>.
• Rewrite tests that use previous_session_artifact to use canonical sources and prepared previous-cache fixtures.
• Add validation tests proving previous_session_artifact is rejected.
• Update docs to remove the legacy path-based source and document only canonical previous-session sources.

Acceptance criteria:

• pipeline.scriptorium.artifacts.*.inputs.*.source: previous_session_artifact fails validation.
• Canonical previous-session sources continue to work for required and optional inputs.
• Prepare/restore previous-cache behavior remains unchanged.
• No docs or examples mention previous_session_artifact as supported.

Test Guidance

Run focused tests after each stage:

• go test ./internal/config -v
• go test ./internal/stage -run Analyze -v
• go test ./internal/app -v
• go test ./...

For Stage 1, focus on config load/strict-decode and S3 workflow regression tests.

For Stage 2, focus on compile-time removal, config strict-decode tests, and full app/stage tests to catch stale adapter references.

For Stage 3, focus on Scriptorium config validation, analyze-stage input resolution, previous-cache behavior, and restore/analyze workflows.

Documentation Updates

Update current-behavior docs only after the corresponding code removal lands:

• docs/config.md
• docs/cli.md, only if command behavior text references removed fields.
• docs/operations.md, only if operator workflow text references removed fields.
• docs/internal/stage-analyze.md
• docs/internal/adapters.md
• examples/pipeline.full.annotated.yml
• examples/pipeline.production.yml

Do not preserve removed fields in examples as compatibility notes. The goal is to make strict config behavior and documentation line up.

Assumptions

• This is a hard cleanup; no backward-compatible aliases are retained.
• Current production configs can be migrated to storage.s3.*, Scriptorium artifacts, and canonical previous-session sources before this lands.
• Removing the unused analyzer adapter does not block any active stage behavior.
• The cleanup should be implemented in the listed order so inert schema removal is separated from behavior removal.