narratio/docs/roadmap/implementation.md

Documentation Policy And Structure Implementation Plan

Status

Proposed.

Objective

Implement the target documentation model defined in Documentation Policy And Structure Alignment. This is a documentation-only effort: do not change Go code, runtime behavior, configuration schemas, generated assets, or application features while following this plan.

Complete the stages below in order. Treat each stage as an independently reviewable change boundary and run its stated checks before proceeding. Preserve the technical intent of the feature roadmap and the existing current-behavior documentation unless repository evidence shows that it is stale.

Governing Decisions

• Use ../notarius/docs/policy/documentation.md as the structural and policy baseline for Narratio's documentation policy, adapting project names, canonical paths, and Narratio-specific document types.
• Keep architecture, documentation, and testing under docs/policy/.
• Move contributor orientation from docs/policy/development.md to docs/development.md.
• Rename docs/internal/README.md to docs/internal/overview.md.
• Replace the root AGENTS.md with the exact two-line content specified in Stage 4.
• Keep docs/troubleshooting.md as a Narratio-specific canonical owner even though the current Notarius ownership table has no corresponding row.
• Add docs/integrations/whisperx.md; WhisperX is an implemented external HTTP boundary and is the only current first-class integration missing a focused integration contract.
• Keep directory index filenames not otherwise selected by this plan unchanged. In particular, do not rename docs/integrations/README.md.
• Do not create docs/adr/ until Narratio has an ADR to record. Do not create placeholder API or consumer documentation.
• Do not copy Notarius product or runtime contracts. Use Notarius only as the policy and information-architecture model.
• Outside docs/roadmap/, describe implemented behavior only. Roadmaps may describe proposed behavior but must not be cited as current contracts.
• When documentation and implementation disagree, inspect the implementation and tests and make current-behavior documentation accurate. Do not alter code in this effort. Record a genuine runtime defect or unresolved ambiguity for separate work rather than silently documenting desired behavior as current.
• Do not add a documentation tool dependency solely for this migration. Use repository scripts only if they already exist; otherwise use the validation procedure in Stage 8.

Stage 1: Replace The Documentation Policy

Goal

Make docs/policy/documentation.md the authoritative Narratio documentation ownership and lifecycle policy before changing the rest of the documentation.

Work

1. Read the complete current files:
   • ../notarius/docs/policy/documentation.md;
   • docs/roadmap/documentation.md;
   • docs/policy/documentation.md; and
   • the current Narratio documentation tree.
2. Replace docs/policy/documentation.md with a Narratio-adapted version of the current Notarius policy. Preserve the Notarius section model:
   • Purpose;
   • Core Rules;
   • Canonical Ownership;
   • Boundary Rules;
   • Architecture Decision Records; and
   • Maintenance.
3. Make these Narratio-specific adaptations:
   • use Narratio rather than Notarius throughout;
   • set the contributor owner to docs/development.md;
   • set architecture, documentation, and testing owners to their files under docs/policy/;
   • set the implemented component inventory to docs/internal/overview.md;
   • retain the Notarius conditional rows for public HTTP and consumer documentation;
   • add a troubleshooting row assigning symptom-driven diagnosis and safe remedies to docs/troubleshooting.md while keeping commands, configuration, and operational procedures with their existing owners;
   • retain docs/integrations/ as the owner of external and durable contracts; and
   • retain the rule that nonexistent interface-specific documents are not required until the interface exists.
4. In the boundary rules, explicitly separate:
   • README orientation, contributor routing, normative architecture, and the implemented internal inventory;
   • CLI invocation, configuration meaning, operations, and troubleshooting;
   • integration contracts and internal implementation; and
   • documentation security rules, application security invariants, credential-supply mechanisms, operational handling, and internal mechanisms.
5. Preserve the lightweight Nygard ADR format and supersession rules from the Notarius policy. Do not create an ADR directory in this stage.
6. Remove the old documentation profiles, prescribed per-document outlines, and obsolete paths rather than appending the new policy beneath them.

Checks

• Search the finished policy for Notarius, docs/architecture.md, docs/policy/development.md, and docs/internal/README.md; none should remain.
• Confirm every canonical owner in the feature roadmap appears consistently in the policy.
• Run git diff --check -- docs/policy/documentation.md.

Completion Criteria

• The policy closely follows the current Notarius policy while expressing the Narratio-specific ownership model.
• It describes target canonical paths even when a later stage will perform the physical rename.
• It contains no implementation sequence or future runtime claim.

Stage 2: Refocus The Architecture Policy

Goal

Make docs/policy/architecture.md the concise normative owner of Narratio's current system shape, dependency boundaries, invariants, safety properties, and non-goals.

Work

1. Review the current architecture policy, the feature roadmap, the new documentation policy, ../notarius/docs/policy/architecture.md, and the current internal overview and focused internal documents.
2. Rewrite or reorganize docs/policy/architecture.md using the Notarius architecture policy as a style and ownership reference, not as a source of Narratio runtime facts.
3. Preserve and clearly state Narratio's existing normative decisions:
   • a small, explicit, stage-driven orchestrator rather than a generic DAG or workflow engine;
   • Narratio ownership of orchestration, configuration resolution, run and session state, artifact and path modeling, manifests, resume, and publish;
   • isolation of WhisperX, Seriatim, Audita, Scriptorium, notification, and object-storage details behind adapters;
   • strict, centralized configuration and narrow templating;
   • centralized local and remote path construction and confined writes;
   • manifest-driven stage progress and transactional stage completion;
   • the remote publish commit boundary in which current/run_id.txt is written last;
   • private campaign-data and secret-handling invariants;
   • deterministic ordering and repeatable orchestration where practical; and
   • the existing architectural non-goals.
4. Keep package names and concrete component inventories out of architecture except where a name is essential to express a boundary. Route concrete ownership to ../internal/overview.md and focused internal docs.
5. Replace the general testing inventory with a short architectural testing expectation and a link to testing.md. Package- or subsystem-specific test details belong in internal docs when useful.
6. Replace documentation-rule duplication with a short requirement and a link to documentation.md.
7. Link operational layout and lifecycle statements to ../operations.md and external contract statements to ../integrations/ where useful. Do not duplicate their complete contracts.
8. Remove all obsolete references, especially docs/documentation/policy.md and docs/architecture.md.
9. Do not invent ADR links. Mention docs/adr/ only as the future home of significant decision history.

Checks

• Search the file for docs/documentation/policy.md, docs/architecture.md, and docs/internal/README.md; none should remain.
• Confirm links resolve relative to docs/policy/.
• Compare every retained invariant with the existing architecture policy and relevant current internal documentation; no invariant may be weakened by omission merely to shorten the document.
• Run git diff --check -- docs/policy/architecture.md.

Completion Criteria

• Architecture owns normative system rules but not the package inventory, general testing policy, documentation policy, CLI reference, configuration reference, or operational procedures.
• Existing Narratio safety and publish invariants remain explicit.

Stage 3: Move And Rewrite The Contributor Guide

Goal

Create the canonical first-read contributor landing page at docs/development.md and remove its obsolete policy location.

Work

1. Move docs/policy/development.md to docs/development.md using a normal repository rename so history remains recognizable.
2. Rewrite it as a concise, table-of-contents-adjacent landing page modeled on ../notarius/docs/development.md, adapted entirely to Narratio.
3. Begin with a short orientation stating that Narratio is a stage-driven Go orchestrator and route readers to:
   • README.md for product context;
   • policy/architecture.md for normative system boundaries; and
   • internal/overview.md for implemented component ownership.
4. Add a What To Read table with task-specific routes for at least:
   • locating current component ownership;
   • architecture, boundaries, invariants, or safety changes;
   • documentation changes;
   • adding, reviewing, rewriting, or deleting tests;
   • CLI behavior;
   • configuration loading or user-visible configuration;
   • session workflow, restore, status, cleanup, and object storage;
   • stage behavior and pipeline sequencing;
   • adapters and external contracts;
   • manifests, artifacts, workspace paths, and publish behavior;
   • examples; and
   • proposed or unimplemented behavior.
5. For each row, link both the internal implementation guide and public or external contract when those have distinct ownership. Do not restate their rules in the table.
6. End with concise validation guidance:
   • use focused package tests while iterating;
   • run go test ./...;
   • run go vet ./...; and
   • run go build ./cmd/narratio for repository-wide changes.
7. Remove the old repository-layout inventory, detailed change playbooks, dependency policy, and duplicated coding conventions. Route those topics to architecture, internal documentation, public contracts, examples, and the testing or documentation policy instead.
8. Update direct references to the contributor guide in files already changed in Stages 1 and 2 if any transitional link remains. Broader navigation is handled in Stage 5.

Checks

• Confirm docs/development.md exists and docs/policy/development.md does not.
• Confirm all links in the new file resolve relative to docs/.
• Confirm the guide contains no package inventory or second definition of a public contract.
• Run git diff --check -- docs/development.md.

Completion Criteria

• A new maintainer or coding agent can choose the correct canonical reading path from one short document.
• The contributor guide is navigational rather than a fourth normative policy.

Stage 4: Replace The Repository Agent Instruction

Goal

Make AGENTS.md a stable pointer to the task-specific contributor guide.

Work

Replace the entire root AGENTS.md contents with exactly these two lines, including the line break and final period:

Please review `docs/development.md` for initial orientation in this repository
and follow its task-specific reading guide.

Do not retain the previous instruction, add headings, add generated markers, or append repository policy details.

Checks

• Run git diff --check -- AGENTS.md.

• Verify exact content with:

  diff -u <(printf '%s\n' 'Please review `docs/development.md` for initial orientation in this repository' 'and follow its task-specific reading guide.') AGENTS.md
  

Completion Criteria

• The comparison is empty and AGENTS.md contains no other text.

Stage 5: Align Structural Entry Points And Navigation

Goal

Complete the selected structural parity changes and make every primary entry point use the new canonical paths.

Work

1. Rename docs/internal/README.md to docs/internal/overview.md.
2. Refocus the renamed file as the implemented component inventory:
   • state its boundary against architecture and public/operator contracts;
   • show the high-level execution path from cmd/narratio through internal/app, configuration and composition, stages and adapters, manifests and artifacts, and durable outputs;
   • provide a compact component table for the executable, app orchestration, configuration, stages, adapters, manifests, artifacts/path safety, previous-session cache, logging, and artifact policy; and
   • retain links to every focused internal document.
3. Keep the canonical stage order in the internal overview only as an implemented inventory summary. Link to stage documents for mechanics and to CLI/operations for user-visible execution semantics.
4. Update README.md while keeping it short:
   • preserve Narratio's product description and minimal successful command;
   • link to docs/development.md and docs/internal/overview.md;
   • include the documentation, architecture, and testing policies in a maintainer-oriented portion of the link list without turning the README into contributor guidance; and
   • retain user links to CLI, configuration, operations, troubleshooting, integrations, and examples.
5. Update all references throughout README.md, docs/, and roadmap files from docs/policy/development.md to docs/development.md, and from docs/internal/README.md to docs/internal/overview.md. Fix relative links according to the referencing file's directory rather than performing a blind textual replacement.
6. Correct other obsolete structural references already known in docs/roadmap/notarius-extract-stage.md, including docs/architecture.md, so it points to docs/policy/architecture.md and the now-existing docs/development.md. Do not change that roadmap's feature design or claim that extraction is implemented.
7. Keep docs/integrations/README.md as the integrations index for now; update its links and ownership language as necessary, but do not rename it.

Checks

• Search the repository documentation, excluding this implementation plan and its feature roadmap, for docs/policy/development.md, docs/internal/README.md, docs/architecture.md, and docs/documentation/policy.md; none should remain. The two planning documents may name old paths when specifying the required migrations.
• Confirm README.md, docs/development.md, and docs/internal/overview.md form a coherent orientation chain.
• Run git diff --check for all changed Markdown files.

Completion Criteria

• The physical file layout matches the canonical ownership table.
• All primary entry points and known inbound links use the new locations.
• No compatibility stub is left at either old path.

Stage 6: Audit User, Operator, And Integration Contracts

Goal

Give volatile public and external facts one canonical owner while preserving useful task-oriented guidance.

Work

1. Review docs/cli.md, docs/config.md, docs/operations.md, docs/troubleshooting.md, every file under docs/integrations/, maintained examples, and the corresponding Go implementations and tests.
2. For code discovery, follow repository instructions: use the codebase knowledge graph first, then use text search for literal commands, fields, defaults, paths, messages, Markdown, YAML, and other non-code content.
3. Make ownership conform to policy:
   • CLI owns commands, arguments, flags, invocation semantics, output conventions, and exits;
   • configuration owns discovery, precedence, fields, defaults, environment overrides, and validation;
   • operations owns runtime workflows, physical local and remote layout, resume, cleanup, permissions, recovery, and operational limits;
   • troubleshooting owns symptom-led diagnosis and safe fixes, linking rather than redefining other contracts; and
   • integrations own external protocols, file formats, request/result shapes, logical durable outputs, and compatibility behavior.
4. Preserve concise cross-cutting workflows in the document whose audience owns the task. Replace repeated volatile definitions with stable summaries and direct links.
5. Create docs/integrations/whisperx.md. Derive its current contract from internal/adapters/whisperx, internal/stage/transcribe.go, configuration, and tests. Cover only:
   • Narratio's purpose for the integration;
   • the HTTP adapter boundary;
   • request and response expectations;
   • retry, timeout, cancellation, validation, and failure semantics;
   • deterministic or concurrency behavior that is externally relevant; and
   • links to configuration for operator-selected values and to internal docs for implementation mechanics. Do not duplicate the configuration field table or physical workspace paths.
6. Add WhisperX to docs/integrations/README.md and correct the index's description: integration contracts are externally observable boundaries, not merely implementation-level references.
7. Review the Audita, Scriptorium, and Seriatim documents for the same boundary. Keep their protocol and data contracts; move or replace configuration field definitions and internal runner wiring with links when duplicated.
8. Verify every documented command, flag, field, default, environment override, exit behavior, path, and integration guarantee against code or a focused test. If a current fact cannot be verified, remove unsupported specificity or record the issue for separate follow-up.

Checks

• Compare CLI command and flag inventories with internal/app command parsing and its tests.
• Compare configuration tables with internal/config structs, defaults, normalization, loading, validation, and tests.
• Compare operational path and lifecycle claims with internal/artifacts, internal/manifest, internal/app, internal/stage, and their tests.
• Compare each integration contract with its adapter and stage boundary tests.
• Confirm troubleshooting entries use symptom, likely cause, diagnostic step, safe fix, and relevant links.
• Run focused tests only when needed to verify a disputed contract; no code changes are authorized.
• Run git diff --check for changed documentation.

Completion Criteria

• Each volatile user, operator, and integration fact has one authoritative definition.
• The new WhisperX contract is discoverable and implementation-accurate.
• Task-oriented workflows remain usable without maintaining parallel reference tables.

Stage 7: Audit Internal Documentation And Examples

Goal

Make internal documents explain implementation without redefining architecture, public contracts, external contracts, or future work, and confirm maintained examples remain canonical.

Work

1. Review every focused file under docs/internal/ against its current package implementation and focused tests.
2. For each internal document, retain or establish:
   • purpose and implemented owner;
   • internal collaborators and execution flow;
   • inputs, outputs, and state transitions expressed at the internal boundary;
   • package-local guarantees, failures, and safety invariants;
   • links to relevant tests or test areas when they materially help a maintainer; and
   • links to public, operator, configuration, or integration contracts rather than duplicate definitions.
3. Audit the highest-drift topics explicitly:
   • canonical stage order, prerequisites, invalidation, skip, force, resume, and failure behavior;
   • session and run manifest states and transitions;
   • artifact identities, resolution rules, checksums, and current-state pointers;
   • workspace, cache, spool, logs, reports, and object-storage keys;
   • restore discovery, planning, conflict handling, execution, and reporting;
   • publish upload ordering and commit semantics;
   • adapter composition and subprocess behavior; and
   • path confinement, deletion, and cleanup safety.
4. Where the same internal fact appears in several focused documents, select the component that owns it and use links from consumers. Do not over-prune a short invariant when repeating it is necessary to prevent an unsafe local change; in that case, state the invariant briefly and link to its normative owner.
5. Review all complete YAML and input artifacts under examples/:
   • keep complete copyable files there rather than in prose;
   • ensure documentation links to them rather than maintaining full duplicate examples;
   • confirm they contain no credentials or private campaign content; and
   • verify they load and validate through existing tests where such coverage already exists.
6. Do not modify application code or invent new test infrastructure. A missing example test may be recorded for later code work; it is not in scope for this documentation-only implementation.

Checks

• Search internal docs for duplicated configuration field/default tables, CLI flag tables, complete external schemas, and physical operations procedures; replace inappropriate copies with links.
• Compare the stage inventory in docs/internal/overview.md with the implemented stage registry.
• Run the existing example/config validation tests identified in docs/development.md or the current test suite.
• Run git diff --check for changed documentation and examples.

Completion Criteria

• Internal documentation is an accurate component-level implementation guide.
• Architecture and public/external contracts remain authoritative at their respective boundaries.
• Complete examples are centralized, valid, discoverable, and secret-free.

Stage 8: Perform Repository-Wide Validation And Close The Roadmap

Goal

Verify the migrated documentation as a coherent whole and record completion only after every acceptance criterion is satisfied.

Work

1. Review README.md, AGENTS.md, docs/development.md, all policy files, all user/operator references, all integration and internal documents, all roadmaps, and all maintained examples as one navigation system.

2. Search for obsolete paths and project-name leakage:

   rg -n 'docs/policy/development\.md|docs/internal/README\.md|docs/architecture\.md|docs/documentation/policy\.md' README.md AGENTS.md docs --glob '!docs/roadmap/implementation.md' --glob '!docs/roadmap/documentation.md'
   rg -n 'Notarius' README.md AGENTS.md docs --glob '!docs/roadmap/notarius-extract-stage.md'
   

   The first search must return no matches. Review every match from the second; Narratio may legitimately mention the Notarius product or integration, but copied Notarius policy language must not remain.

3. Validate local Markdown file and directory links with a reproducible script that scans README.md, AGENTS.md, and docs/**/*.md, resolves relative targets from each source file, strips anchors and query fragments, ignores external URLs and pure anchors, and fails for missing local targets. Use an existing repository checker if present. Otherwise run a temporary script without adding it or a new dependency to the repository.

4. Manually inspect anchor links and directory links because a basic local-path checker may not validate them fully.

5. Check formatting and whitespace:

   git diff --check
   

6. Run the complete existing validation suite:

   go test ./...
   go vet ./...
   go build ./cmd/narratio
   

7. Review the feature roadmap acceptance criteria one by one. Confirm that:

   • canonical ownership is explicit and reflected by actual content;
   • current behavior and future roadmap work are separated;
   • examples and commands match implementation;
   • no sensitive data was introduced;
   • no obsolete path or compatibility stub remains; and
   • this effort changed documentation only.

8. If all criteria and checks pass, change the status in docs/roadmap/documentation.md from Proposed. to Completed. and change this plan's status from Proposed. to Completed.. Do not mark either complete while required work or an unexplained validation failure remains.

9. If a validation failure predates this work and is demonstrably unrelated, record the exact command, failure, and evidence in the implementation handoff rather than changing application code. Documentation-link or content failures introduced or exposed by this migration must be fixed before completion.

Completion Criteria

• All Stage 8 searches and documentation checks pass.
• go test ./..., go vet ./..., and go build ./cmd/narratio pass, or a precisely evidenced unrelated pre-existing failure is reported.
• Both roadmap documents are marked Completed. only after the intended final state is present.
• The final handoff lists changed documents, validation commands and results, and any explicitly out-of-scope follow-up without claiming unperformed work.