narratio/docs/roadmap/implementation.md

# Documentation Policy And Structure Implementation Plan

## Status

Proposed.

## Objective

Implement the target documentation model defined in
[Documentation Policy And Structure Alignment](documentation.md). This is a
documentation-only effort: do not change Go code, runtime behavior,
configuration schemas, generated assets, or application features while
following this plan.

Complete the stages below in order. Treat each stage as an independently
reviewable change boundary and run its stated checks before proceeding. Preserve
the technical intent of the feature roadmap and the existing current-behavior
documentation unless repository evidence shows that it is stale.

## Governing Decisions

- Use `../notarius/docs/policy/documentation.md` as the structural and policy
  baseline for Narratio's documentation policy, adapting project names,
  canonical paths, and Narratio-specific document types.
- Keep architecture, documentation, and testing under `docs/policy/`.
- Move contributor orientation from `docs/policy/development.md` to
  `docs/development.md`.
- Rename `docs/internal/README.md` to `docs/internal/overview.md`.
- Replace the root `AGENTS.md` with the exact two-line content specified in
  Stage 4.
- Keep `docs/troubleshooting.md` as a Narratio-specific canonical owner even
  though the current Notarius ownership table has no corresponding row.
- Add `docs/integrations/whisperx.md`; WhisperX is an implemented external HTTP
  boundary and is the only current first-class integration missing a focused
  integration contract.
- Keep directory index filenames not otherwise selected by this plan unchanged.
  In particular, do not rename `docs/integrations/README.md`.
- Do not create `docs/adr/` until Narratio has an ADR to record. Do not create
  placeholder API or consumer documentation.
- Do not copy Notarius product or runtime contracts. Use Notarius only as the
  policy and information-architecture model.
- Outside `docs/roadmap/`, describe implemented behavior only. Roadmaps may
  describe proposed behavior but must not be cited as current contracts.
- When documentation and implementation disagree, inspect the implementation
  and tests and make current-behavior documentation accurate. Do not alter code
  in this effort. Record a genuine runtime defect or unresolved ambiguity for
  separate work rather than silently documenting desired behavior as current.
- Do not add a documentation tool dependency solely for this migration. Use
  repository scripts only if they already exist; otherwise use the validation
  procedure in Stage 8.

## Stage 1: Replace The Documentation Policy

### Goal

Make `docs/policy/documentation.md` the authoritative Narratio documentation
ownership and lifecycle policy before changing the rest of the documentation.

### Work

1. Read the complete current files:
   - `../notarius/docs/policy/documentation.md`;
   - `docs/roadmap/documentation.md`;
   - `docs/policy/documentation.md`; and
   - the current Narratio documentation tree.
2. Replace `docs/policy/documentation.md` with a Narratio-adapted version of the
   current Notarius policy. Preserve the Notarius section model:
   - Purpose;
   - Core Rules;
   - Canonical Ownership;
   - Boundary Rules;
   - Architecture Decision Records; and
   - Maintenance.
3. Make these Narratio-specific adaptations:
   - use Narratio rather than Notarius throughout;
   - set the contributor owner to `docs/development.md`;
   - set architecture, documentation, and testing owners to their files under
     `docs/policy/`;
   - set the implemented component inventory to
     `docs/internal/overview.md`;
   - retain the Notarius conditional rows for public HTTP and consumer
     documentation;
   - add a troubleshooting row assigning symptom-driven diagnosis and safe
     remedies to `docs/troubleshooting.md` while keeping commands,
     configuration, and operational procedures with their existing owners;
   - retain `docs/integrations/` as the owner of external and durable contracts;
     and
   - retain the rule that nonexistent interface-specific documents are not
     required until the interface exists.
4. In the boundary rules, explicitly separate:
   - README orientation, contributor routing, normative architecture, and the
     implemented internal inventory;
   - CLI invocation, configuration meaning, operations, and troubleshooting;
   - integration contracts and internal implementation; and
   - documentation security rules, application security invariants,
     credential-supply mechanisms, operational handling, and internal
     mechanisms.
5. Preserve the lightweight Nygard ADR format and supersession rules from the
   Notarius policy. Do not create an ADR directory in this stage.
6. Remove the old documentation profiles, prescribed per-document outlines,
   and obsolete paths rather than appending the new policy beneath them.

### Checks

- Search the finished policy for `Notarius`, `docs/architecture.md`,
  `docs/policy/development.md`, and `docs/internal/README.md`; none should
  remain.
- Confirm every canonical owner in the feature roadmap appears consistently in
  the policy.
- Run `git diff --check -- docs/policy/documentation.md`.

### Completion Criteria

- The policy closely follows the current Notarius policy while expressing the
  Narratio-specific ownership model.
- It describes target canonical paths even when a later stage will perform the
  physical rename.
- It contains no implementation sequence or future runtime claim.

## Stage 2: Refocus The Architecture Policy

### Goal

Make `docs/policy/architecture.md` the concise normative owner of Narratio's
current system shape, dependency boundaries, invariants, safety properties,
and non-goals.

### Work

1. Review the current architecture policy, the feature roadmap, the new
   documentation policy, `../notarius/docs/policy/architecture.md`, and the
   current internal overview and focused internal documents.
2. Rewrite or reorganize `docs/policy/architecture.md` using the Notarius
   architecture policy as a style and ownership reference, not as a source of
   Narratio runtime facts.
3. Preserve and clearly state Narratio's existing normative decisions:
   - a small, explicit, stage-driven orchestrator rather than a generic DAG or
     workflow engine;
   - Narratio ownership of orchestration, configuration resolution, run and
     session state, artifact and path modeling, manifests, resume, and publish;
   - isolation of WhisperX, Seriatim, Audita, Scriptorium, notification, and
     object-storage details behind adapters;
   - strict, centralized configuration and narrow templating;
   - centralized local and remote path construction and confined writes;
   - manifest-driven stage progress and transactional stage completion;
   - the remote publish commit boundary in which `current/run_id.txt` is
     written last;
   - private campaign-data and secret-handling invariants;
   - deterministic ordering and repeatable orchestration where practical; and
   - the existing architectural non-goals.
4. Keep package names and concrete component inventories out of architecture
   except where a name is essential to express a boundary. Route concrete
   ownership to `../internal/overview.md` and focused internal docs.
5. Replace the general testing inventory with a short architectural testing
   expectation and a link to `testing.md`. Package- or subsystem-specific test
   details belong in internal docs when useful.
6. Replace documentation-rule duplication with a short requirement and a link
   to `documentation.md`.
7. Link operational layout and lifecycle statements to `../operations.md` and
   external contract statements to `../integrations/` where useful. Do not
   duplicate their complete contracts.
8. Remove all obsolete references, especially
   `docs/documentation/policy.md` and `docs/architecture.md`.
9. Do not invent ADR links. Mention `docs/adr/` only as the future home of
   significant decision history.

### Checks

- Search the file for `docs/documentation/policy.md`, `docs/architecture.md`,
  and `docs/internal/README.md`; none should remain.
- Confirm links resolve relative to `docs/policy/`.
- Compare every retained invariant with the existing architecture policy and
  relevant current internal documentation; no invariant may be weakened by
  omission merely to shorten the document.
- Run `git diff --check -- docs/policy/architecture.md`.

### Completion Criteria

- Architecture owns normative system rules but not the package inventory,
  general testing policy, documentation policy, CLI reference, configuration
  reference, or operational procedures.
- Existing Narratio safety and publish invariants remain explicit.

## Stage 3: Move And Rewrite The Contributor Guide

### Goal

Create the canonical first-read contributor landing page at
`docs/development.md` and remove its obsolete policy location.

### Work

1. Move `docs/policy/development.md` to `docs/development.md` using a normal
   repository rename so history remains recognizable.
2. Rewrite it as a concise, table-of-contents-adjacent landing page modeled on
   `../notarius/docs/development.md`, adapted entirely to Narratio.
3. Begin with a short orientation stating that Narratio is a stage-driven Go
   orchestrator and route readers to:
   - `README.md` for product context;
   - `policy/architecture.md` for normative system boundaries; and
   - `internal/overview.md` for implemented component ownership.
4. Add a `What To Read` table with task-specific routes for at least:
   - locating current component ownership;
   - architecture, boundaries, invariants, or safety changes;
   - documentation changes;
   - adding, reviewing, rewriting, or deleting tests;
   - CLI behavior;
   - configuration loading or user-visible configuration;
   - session workflow, restore, status, cleanup, and object storage;
   - stage behavior and pipeline sequencing;
   - adapters and external contracts;
   - manifests, artifacts, workspace paths, and publish behavior;
   - examples; and
   - proposed or unimplemented behavior.
5. For each row, link both the internal implementation guide and public or
   external contract when those have distinct ownership. Do not restate their
   rules in the table.
6. End with concise validation guidance:
   - use focused package tests while iterating;
   - run `go test ./...`;
   - run `go vet ./...`; and
   - run `go build ./cmd/narratio` for repository-wide changes.
7. Remove the old repository-layout inventory, detailed change playbooks,
   dependency policy, and duplicated coding conventions. Route those topics to
   architecture, internal documentation, public contracts, examples, and the
   testing or documentation policy instead.
8. Update direct references to the contributor guide in files already changed
   in Stages 1 and 2 if any transitional link remains. Broader navigation is
   handled in Stage 5.

### Checks

- Confirm `docs/development.md` exists and
  `docs/policy/development.md` does not.
- Confirm all links in the new file resolve relative to `docs/`.
- Confirm the guide contains no package inventory or second definition of a
  public contract.
- Run `git diff --check -- docs/development.md`.

### Completion Criteria

- A new maintainer or coding agent can choose the correct canonical reading
  path from one short document.
- The contributor guide is navigational rather than a fourth normative policy.

## Stage 4: Replace The Repository Agent Instruction

### Goal

Make `AGENTS.md` a stable pointer to the task-specific contributor guide.

### Work

Replace the entire root `AGENTS.md` contents with exactly these two lines,
including the line break and final period:

```text
Please review `docs/development.md` for initial orientation in this repository
and follow its task-specific reading guide.
```

Do not retain the previous instruction, add headings, add generated markers, or
append repository policy details.

### Checks

- Run `git diff --check -- AGENTS.md`.
- Verify exact content with:

  ```sh
  diff -u <(printf '%s\n' 'Please review `docs/development.md` for initial orientation in this repository' 'and follow its task-specific reading guide.') AGENTS.md
  ```

### Completion Criteria

- The comparison is empty and `AGENTS.md` contains no other text.

## Stage 5: Align Structural Entry Points And Navigation

### Goal

Complete the selected structural parity changes and make every primary entry
point use the new canonical paths.

### Work

1. Rename `docs/internal/README.md` to `docs/internal/overview.md`.
2. Refocus the renamed file as the implemented component inventory:
   - state its boundary against architecture and public/operator contracts;
   - show the high-level execution path from `cmd/narratio` through
     `internal/app`, configuration and composition, stages and adapters,
     manifests and artifacts, and durable outputs;
   - provide a compact component table for the executable, app orchestration,
     configuration, stages, adapters, manifests, artifacts/path safety,
     previous-session cache, logging, and artifact policy; and
   - retain links to every focused internal document.
3. Keep the canonical stage order in the internal overview only as an
   implemented inventory summary. Link to stage documents for mechanics and to
   CLI/operations for user-visible execution semantics.
4. Update `README.md` while keeping it short:
   - preserve Narratio's product description and minimal successful command;
   - link to `docs/development.md` and `docs/internal/overview.md`;
   - include the documentation, architecture, and testing policies in a
     maintainer-oriented portion of the link list without turning the README
     into contributor guidance; and
   - retain user links to CLI, configuration, operations, troubleshooting,
     integrations, and examples.
5. Update all references throughout `README.md`, `docs/`, and roadmap files from
   `docs/policy/development.md` to `docs/development.md`, and from
   `docs/internal/README.md` to `docs/internal/overview.md`. Fix relative links
   according to the referencing file's directory rather than performing a
   blind textual replacement.
6. Correct other obsolete structural references already known in
   `docs/roadmap/notarius-extract-stage.md`, including `docs/architecture.md`,
   so it points to `docs/policy/architecture.md` and the now-existing
   `docs/development.md`. Do not change that roadmap's feature design or claim
   that extraction is implemented.
7. Keep `docs/integrations/README.md` as the integrations index for now; update
   its links and ownership language as necessary, but do not rename it.

### Checks

- Search the repository documentation, excluding this implementation plan and
  its feature roadmap, for `docs/policy/development.md`,
  `docs/internal/README.md`, `docs/architecture.md`, and
  `docs/documentation/policy.md`; none should remain. The two planning documents
  may name old paths when specifying the required migrations.
- Confirm `README.md`, `docs/development.md`, and
  `docs/internal/overview.md` form a coherent orientation chain.
- Run `git diff --check` for all changed Markdown files.

### Completion Criteria

- The physical file layout matches the canonical ownership table.
- All primary entry points and known inbound links use the new locations.
- No compatibility stub is left at either old path.

## Stage 6: Audit User, Operator, And Integration Contracts

### Goal

Give volatile public and external facts one canonical owner while preserving
useful task-oriented guidance.

### Work

1. Review `docs/cli.md`, `docs/config.md`, `docs/operations.md`,
   `docs/troubleshooting.md`, every file under `docs/integrations/`, maintained
   examples, and the corresponding Go implementations and tests.
2. For code discovery, follow repository instructions: use the codebase
   knowledge graph first, then use text search for literal commands, fields,
   defaults, paths, messages, Markdown, YAML, and other non-code content.
3. Make ownership conform to policy:
   - CLI owns commands, arguments, flags, invocation semantics, output
     conventions, and exits;
   - configuration owns discovery, precedence, fields, defaults, environment
     overrides, and validation;
   - operations owns runtime workflows, physical local and remote layout,
     resume, cleanup, permissions, recovery, and operational limits;
   - troubleshooting owns symptom-led diagnosis and safe fixes, linking rather
     than redefining other contracts; and
   - integrations own external protocols, file formats, request/result shapes,
     logical durable outputs, and compatibility behavior.
4. Preserve concise cross-cutting workflows in the document whose audience
   owns the task. Replace repeated volatile definitions with stable summaries
   and direct links.
5. Create `docs/integrations/whisperx.md`. Derive its current contract from
   `internal/adapters/whisperx`, `internal/stage/transcribe.go`, configuration,
   and tests. Cover only:
   - Narratio's purpose for the integration;
   - the HTTP adapter boundary;
   - request and response expectations;
   - retry, timeout, cancellation, validation, and failure semantics;
   - deterministic or concurrency behavior that is externally relevant; and
   - links to configuration for operator-selected values and to internal docs
     for implementation mechanics.
   Do not duplicate the configuration field table or physical workspace paths.
6. Add WhisperX to `docs/integrations/README.md` and correct the index's
   description: integration contracts are externally observable boundaries,
   not merely implementation-level references.
7. Review the Audita, Scriptorium, and Seriatim documents for the same boundary.
   Keep their protocol and data contracts; move or replace configuration field
   definitions and internal runner wiring with links when duplicated.
8. Verify every documented command, flag, field, default, environment
   override, exit behavior, path, and integration guarantee against code or a
   focused test. If a current fact cannot be verified, remove unsupported
   specificity or record the issue for separate follow-up.

### Checks

- Compare CLI command and flag inventories with `internal/app` command parsing
  and its tests.
- Compare configuration tables with `internal/config` structs, defaults,
  normalization, loading, validation, and tests.
- Compare operational path and lifecycle claims with `internal/artifacts`,
  `internal/manifest`, `internal/app`, `internal/stage`, and their tests.
- Compare each integration contract with its adapter and stage boundary tests.
- Confirm troubleshooting entries use symptom, likely cause, diagnostic step,
  safe fix, and relevant links.
- Run focused tests only when needed to verify a disputed contract; no code
  changes are authorized.
- Run `git diff --check` for changed documentation.

### Completion Criteria

- Each volatile user, operator, and integration fact has one authoritative
  definition.
- The new WhisperX contract is discoverable and implementation-accurate.
- Task-oriented workflows remain usable without maintaining parallel reference
  tables.

## Stage 7: Audit Internal Documentation And Examples

### Goal

Make internal documents explain implementation without redefining architecture,
public contracts, external contracts, or future work, and confirm maintained
examples remain canonical.

### Work

1. Review every focused file under `docs/internal/` against its current package
   implementation and focused tests.
2. For each internal document, retain or establish:
   - purpose and implemented owner;
   - internal collaborators and execution flow;
   - inputs, outputs, and state transitions expressed at the internal boundary;
   - package-local guarantees, failures, and safety invariants;
   - links to relevant tests or test areas when they materially help a
     maintainer; and
   - links to public, operator, configuration, or integration contracts rather
     than duplicate definitions.
3. Audit the highest-drift topics explicitly:
   - canonical stage order, prerequisites, invalidation, skip, force, resume,
     and failure behavior;
   - session and run manifest states and transitions;
   - artifact identities, resolution rules, checksums, and current-state
     pointers;
   - workspace, cache, spool, logs, reports, and object-storage keys;
   - restore discovery, planning, conflict handling, execution, and reporting;
   - publish upload ordering and commit semantics;
   - adapter composition and subprocess behavior; and
   - path confinement, deletion, and cleanup safety.
4. Where the same internal fact appears in several focused documents, select
   the component that owns it and use links from consumers. Do not over-prune a
   short invariant when repeating it is necessary to prevent an unsafe local
   change; in that case, state the invariant briefly and link to its normative
   owner.
5. Review all complete YAML and input artifacts under `examples/`:
   - keep complete copyable files there rather than in prose;
   - ensure documentation links to them rather than maintaining full duplicate
     examples;
   - confirm they contain no credentials or private campaign content; and
   - verify they load and validate through existing tests where such coverage
     already exists.
6. Do not modify application code or invent new test infrastructure. A missing
   example test may be recorded for later code work; it is not in scope for
   this documentation-only implementation.

### Checks

- Search internal docs for duplicated configuration field/default tables, CLI
  flag tables, complete external schemas, and physical operations procedures;
  replace inappropriate copies with links.
- Compare the stage inventory in `docs/internal/overview.md` with the
  implemented stage registry.
- Run the existing example/config validation tests identified in
  `docs/development.md` or the current test suite.
- Run `git diff --check` for changed documentation and examples.

### Completion Criteria

- Internal documentation is an accurate component-level implementation guide.
- Architecture and public/external contracts remain authoritative at their
  respective boundaries.
- Complete examples are centralized, valid, discoverable, and secret-free.

## Stage 8: Perform Repository-Wide Validation And Close The Roadmap

### Goal

Verify the migrated documentation as a coherent whole and record completion
only after every acceptance criterion is satisfied.

### Work

1. Review `README.md`, `AGENTS.md`, `docs/development.md`, all policy files,
   all user/operator references, all integration and internal documents, all
   roadmaps, and all maintained examples as one navigation system.
2. Search for obsolete paths and project-name leakage:

   ```sh
   rg -n 'docs/policy/development\.md|docs/internal/README\.md|docs/architecture\.md|docs/documentation/policy\.md' README.md AGENTS.md docs --glob '!docs/roadmap/implementation.md' --glob '!docs/roadmap/documentation.md'
   rg -n 'Notarius' README.md AGENTS.md docs --glob '!docs/roadmap/notarius-extract-stage.md'
   ```

   The first search must return no matches. Review every match from the second;
   Narratio may legitimately mention the Notarius product or integration, but
   copied Notarius policy language must not remain.
3. Validate local Markdown file and directory links with a reproducible script
   that scans `README.md`, `AGENTS.md`, and `docs/**/*.md`, resolves relative
   targets from each source file, strips anchors and query fragments, ignores
   external URLs and pure anchors, and fails for missing local targets. Use an
   existing repository checker if present. Otherwise run a temporary script
   without adding it or a new dependency to the repository.
4. Manually inspect anchor links and directory links because a basic local-path
   checker may not validate them fully.
5. Check formatting and whitespace:

   ```sh
   git diff --check
   ```
6. Run the complete existing validation suite:

   ```sh
   go test ./...
   go vet ./...
   go build ./cmd/narratio
   ```
7. Review the feature roadmap acceptance criteria one by one. Confirm that:
   - canonical ownership is explicit and reflected by actual content;
   - current behavior and future roadmap work are separated;
   - examples and commands match implementation;
   - no sensitive data was introduced;
   - no obsolete path or compatibility stub remains; and
   - this effort changed documentation only.
8. If all criteria and checks pass, change the status in
   `docs/roadmap/documentation.md` from `Proposed.` to `Completed.` and change
   this plan's status from `Proposed.` to `Completed.`. Do not mark either
   complete while required work or an unexplained validation failure remains.
9. If a validation failure predates this work and is demonstrably unrelated,
   record the exact command, failure, and evidence in the implementation handoff
   rather than changing application code. Documentation-link or content failures
   introduced or exposed by this migration must be fixed before completion.

### Completion Criteria

- All Stage 8 searches and documentation checks pass.
- `go test ./...`, `go vet ./...`, and `go build ./cmd/narratio` pass, or a
  precisely evidenced unrelated pre-existing failure is reported.
- Both roadmap documents are marked `Completed.` only after the intended final
  state is present.
- The final handoff lists changed documents, validation commands and results,
  and any explicitly out-of-scope follow-up without claiming unperformed work.