narratio/docs/roadmap/documentation.md

# Documentation Policy And Structure Alignment

## Status

Proposed.

## Purpose

Align Narratio's documentation policy, organization, and maintenance practices
with the current Notarius approach while preserving Narratio-specific product,
operational, and integration needs.

The resulting documentation set should assign each authoritative topic to one
canonical owner, distinguish current behavior from future work, make audience
boundaries explicit, and minimize duplicated contracts that can drift as the
application changes.

This roadmap defines the scope and intended final state. Execution guidance is
maintained separately in the [Implementation Plan](implementation.md).

## Motivation

Narratio already has substantial user, operator, integration, and internal
documentation, but its documentation policy predates the current Notarius
policy. The existing policy prescribes document profiles and detailed outlines
without defining ownership boundaries precisely enough. Several policy files
also refer to obsolete or nonexistent paths, and some concrete inventories and
contract details are repeated across documents.

The alignment should retain useful Narratio documentation while adopting the
stronger Notarius principles:

- one canonical owner for each authoritative fact;
- current behavior outside the roadmap and future behavior inside it;
- audience-appropriate detail;
- maintained, valid, secret-free examples;
- explicit boundaries among user, operator, integration, architecture, and
  internal documentation;
- a defined lifecycle for architectural decision records; and
- verification of documentation against implemented behavior.

## Target Documentation Model

### Canonical Ownership

The final documentation policy should assign the following responsibilities.

| Topic | Canonical owner | Intended responsibility |
| --- | --- | --- |
| Product orientation and minimal end-to-end quickstart | `README.md` | Explain what Narratio is, why it is useful, show the shortest successful invocation, and route readers onward. |
| Contributor entry point | `docs/development.md` | Provide a task-oriented reading guide, baseline validation commands, and links to canonical policies and contracts. |
| Current application architecture | `docs/policy/architecture.md` | Define system shape, normative ownership, dependency direction, boundaries, invariants, safety properties, and non-goals. |
| Documentation organization | `docs/policy/documentation.md` | Define canonical ownership, audience boundaries, maintenance rules, and the ADR and document lifecycle. |
| Testing policy | `docs/policy/testing.md` | Define test philosophy, risk-based sufficiency, test boundaries, doubles, coverage guidance, and test lifecycle decisions. |
| CLI contract | `docs/cli.md` | Define commands, arguments, flags, invocation semantics, output conventions, and exit behavior. |
| Configuration contract | `docs/config.md` | Define discovery, precedence, schemas, fields, defaults, environment overrides, and validation rules. |
| Operations | `docs/operations.md` | Define runtime workflows, physical state layout, resume, cleanup, permissions, recovery, and operational limits. |
| Troubleshooting | `docs/troubleshooting.md` | Provide symptom-driven diagnosis and safe remedies, linking to the owning CLI, configuration, operations, or integration contract. |
| External and durable integration contracts | `docs/integrations/` | Define the external formats, protocols, logical artifact shapes, and compatibility behavior Narratio relies on. |
| Implemented component inventory | `docs/internal/overview.md` | Inventory current packages and components, summarize their implemented responsibilities, and route readers to focused internal documents. |
| Internal component behavior | Other files under `docs/internal/` | Explain implementation flow, internal collaborators, state transitions, package-local guarantees and failures, and relevant tests. |
| Architectural decision history | `docs/adr/` | Record significant decisions, alternatives, rationale, consequences, and supersession history when ADRs are warranted. |
| Future work and implementation status | `docs/roadmap/` | Describe proposed, accepted, deferred, rejected, or completed work and its implementation status. |
| Complete copyable artifacts | `examples/` | Hold maintained configuration, inputs, and other files intended to be copied or run. |

Documents for interfaces that Narratio does not expose should not be created as
placeholders. In particular, public API or consumer documentation should be
added only if a corresponding public interface exists.

### Structural Parity With Notarius

Narratio should follow the current Notarius policy layout where the same
responsibility exists. In the aligned structure:

- `docs/internal/overview.md` is the internal component inventory;
- `docs/development.md` is the first-read contributor landing page;
- all navigation and relative links use those canonical paths;
- `docs/adr/` and sequentially numbered ADR filenames are used when architectural
  decisions need durable records; and
- Narratio-specific canonical documents remain, including the troubleshooting
  guide, stage documentation, and integration contracts.

Structural parity does not require placeholder documents, removal of useful
Narratio-specific material, or identical prose where the applications have
different contracts.

## Policy Alignment

### Documentation Policy

The documentation policy should follow the current Notarius policy, adapted to
Narratio. It should:

- define one canonical owner for every contract or authoritative fact;
- identify volatile details that must not be maintained in multiple places;
- permit non-owning documents to provide only short, stable summaries with
  links;
- describe implemented behavior only outside `docs/roadmap/`;
- allow accepted ADRs to precede implementation without presenting the
  decision as implemented behavior;
- distinguish user, operator, contributor, integration, and internal detail;
- require complete copyable files to live under `examples/`;
- define documentation security and privacy requirements;
- specify ownership boundaries for orientation, CLI, configuration,
  operations, troubleshooting, integrations, architecture, and internals;
- define the ADR format and supersession rules; and
- require contract, example, link, and sensitive-data checks when behavior or
  documentation changes.

### Architecture Policy

The architecture policy should remain the normative owner of Narratio's system
boundaries and invariants rather than a concrete package inventory or secondary
testing and documentation policy. It should:

- preserve Narratio's explicit, stage-driven orchestration model;
- preserve adapter, manifest, artifact, path-safety, publish-commit, security,
  privacy, and determinism invariants;
- link to the testing and documentation policies for their general rules;
- link to internal documentation for implemented component mechanics;
- link to external contracts rather than redefining them; and
- use the canonical `docs/policy/` paths consistently.

### Testing Policy

The copied Notarius testing policy is Narratio's canonical testing policy. Its
place in the documentation set requires:

- linking it from architecture and contributor guidance;
- removing or reducing duplicated general test philosophy elsewhere;
- retaining subsystem-specific test guidance only where it helps maintain a
  concrete contract; and
- ensuring project commands and examples remain accurate for Narratio.

The alignment should not require Narratio and Notarius to have identical test
suites. The shared policy governs how Narratio evaluates test value,
boundaries, doubles, regression protection, coverage, and sufficiency.

### Contributor Policy

The development document should be a concise contributor entry point. It should
route maintainers and coding agents to canonical documentation based on the
task at hand and provide the minimum repository orientation and validation
commands needed to begin work.

Detailed package inventories belong in `docs/internal/overview.md`;
architecture rules belong in the architecture policy; application contracts
belong in their user, operator, or integration documents; and general testing
rules belong in the testing policy. Any retained change recipes should avoid
redefining those owners.

## Documentation Set Alignment

### README And Navigation

Keep the README short and outward-facing. It should own product orientation and
one minimal successful workflow, then link to the CLI, configuration,
operations, troubleshooting, contributor, architecture, testing, integration,
and internal entry points as appropriate.

Navigation should use the final canonical paths and should not rely on obsolete
aliases or duplicate index files.

### CLI, Configuration, And Operations

Separate these contracts consistently:

- CLI documentation answers how Narratio is invoked and what its command-line
  interface means;
- configuration documentation answers how configuration is discovered,
  interpreted, defaulted, overridden, and validated; and
- operations documentation answers what happens to runtime state and how an
  operator runs, resumes, cleans, diagnoses, or recovers the application.

Cross-cutting workflows should have one task-oriented owner and link to the
other contracts rather than copying their flags, fields, defaults, or path
definitions.

### Troubleshooting

Keep troubleshooting as a Narratio-specific canonical document. Each entry
should begin from an observable symptom and provide a likely cause, a safe
diagnostic step, a safe remedy, and links to the canonical contract or
operational procedure.

Troubleshooting should not become a second CLI, configuration, or operations
reference.

### Integrations

Integration documents should own the externally observable contracts Narratio
uses: subprocess behavior, file formats, protocols, logical artifact paths and
schemas, compatibility expectations, and upstream or downstream
responsibilities.

They should describe only the portions of WhisperX, Seriatim, Audita,
Scriptorium, object storage, or future integrations that Narratio actually
depends on. Internal adapter mechanics belong under `docs/internal/`, while
configuration defaults, CLI syntax, and physical runtime placement remain with
their respective owners.

### Internal Documentation

`docs/internal/overview.md` is the canonical implemented component inventory. It
should summarize current components and route readers to focused documents
without restating normative architecture.

Focused internal documents should describe implemented behavior at useful
component boundaries, including collaborators, data and state transitions,
failures, and tests worth consulting. They may identify an external field,
file, or protocol when explaining a dependency, but should link to the
canonical contract for its definition.

The internal set should be reviewed for duplicate definitions of:

- stage order and stage contracts;
- manifest states and transitions;
- artifact identities, paths, and schemas;
- workspace and object-storage layout;
- restore, resume, cleanup, and publish behavior;
- adapter invocation and compatibility rules; and
- configuration fields and defaults.

### Examples

Complete copyable configuration and input files should remain under
`examples/`. Documentation may include small illustrative fragments, but it
should link to maintained examples instead of embedding alternate complete
files.

Examples should remain secret-free, loadable, valid, and covered by automated
tests where practical. Their commands, filenames, fields, defaults, and
templates must agree with implemented Narratio behavior.

### Roadmaps And ADRs

Roadmaps should own future work, implementation status, and sequencing. They
must not be treated as current behavior references. When roadmap work lands,
the relevant current-behavior documents should be updated in the same change,
and the roadmap status should accurately reflect completion.

Use ADRs for significant architectural decisions whose context, alternatives,
and consequences should remain durable. Use the lightweight Nygard structure:
title, status, date, context, decision, alternatives considered, and
consequences. Accepted decision content is immutable; a later change should
supersede it with a new ADR. Rejected architectural alternatives belong in the
ADR, while rejected product ideas remain roadmap material.

No empty `docs/adr/` directory or retrospective ADR catalog is required merely
to satisfy structural parity.

## Canonical-Ownership Audit

The existing documentation should be audited for repeated authoritative facts,
with priority given to details most likely to drift:

- commands, arguments, flags, output conventions, and exit behavior;
- configuration discovery, fields, defaults, environment variables, and
  validation rules;
- stage order, prerequisites, invalidation, skip, force, resume, and failure
  semantics;
- manifest states, artifact identities, and publish commit behavior;
- local workspace, cache, spool, log, report, and remote object paths;
- integration arguments, formats, schemas, timeouts, and compatibility rules;
  and
- security, credential, permission, and sensitive-artifact handling.

For each repeated fact, choose the canonical owner defined by policy. Remove
the duplicate definition or reduce it to the smallest stable summary needed
for orientation, with a link to the owner.

This audit should preserve useful task-oriented guidance. Canonical ownership
means eliminating parallel contract definitions, not forcing readers to
assemble every workflow from isolated reference fragments.

## Intended Final State

When this roadmap is complete:

- Narratio's documentation policy closely matches the current Notarius policy
  in principles, organization, terminology, and lifecycle rules;
- every authoritative documentation topic has one stated canonical owner;
- `docs/internal/overview.md` is the internal component index and no references
  to `docs/internal/README.md` remain;
- all policy and navigation links use real canonical paths;
- architecture, documentation, testing, and contributor policies have distinct
  responsibilities and link to one another;
- current-behavior documentation contains no unimplemented claims;
- roadmap documents clearly own future behavior and implementation status;
- user and operator documents avoid unnecessary implementation detail;
- internal documents do not redefine external or user-facing contracts;
- volatile commands, fields, defaults, schemas, paths, and guarantees are not
  maintained authoritatively in multiple places;
- complete examples live under `examples/` and remain valid and secret-free;
- the documentation remains concise enough to navigate but complete enough for
  users, operators, developers, integrators, and coding agents; and
- future behavior changes can identify the documentation that must change by
  consulting the ownership table.

## Validation

Completion should include:

- a repository-wide review of Markdown links and canonical paths;
- verification of documented commands and flags against the CLI
  implementation;
- verification of configuration fields, defaults, environment overrides, and
  validation rules against the configuration implementation;
- verification of stage, manifest, artifact, restore, resume, cleanup, and
  publish claims against implemented behavior;
- validation of maintained examples through existing runtime config paths and
  tests;
- `go test ./...`;
- confirmation that non-owning documents summarize and link rather than
  redefine volatile contracts;
- confirmation that unimplemented behavior appears only under
  `docs/roadmap/`, subject to the accepted-ADR exception; and
- a review for credentials, private campaign content, sensitive environment
  data, and private infrastructure details.

If the repository has no automated Markdown link checker, validation should use
a focused, reproducible scripted link check supplemented by manual review of
directory links and anchors.

## Acceptance Criteria

- `docs/policy/documentation.md` expresses the Narratio-adapted canonical
  ownership model and boundary rules from the current Notarius policy.
- `docs/policy/architecture.md`, `docs/policy/testing.md`, and
  `docs/development.md` have clear, non-overlapping responsibilities.
- `docs/policy/development.md` has moved to `docs/development.md`, with all
  inbound links updated.
- `docs/internal/README.md` has been renamed to
  `docs/internal/overview.md`, with all inbound links updated.
- README, CLI, configuration, operations, troubleshooting, integration,
  internal, roadmap, example, and ADR responsibilities match the ownership
  model.
- No known obsolete documentation paths remain.
- High-volatility contracts have one authoritative definition.
- Current and future behavior are clearly separated.
- Maintained examples and documented commands agree with the implementation.
- Documentation validation and the full Go test suite pass, or any unrelated
  pre-existing failure is recorded precisely.

## Non-Goals

- Changing Narratio runtime behavior solely to make existing documentation
  true.
- Implementing features described by other roadmap documents.
- Rewriting every document for stylistic uniformity when its ownership and
  content are already correct.
- Copying Notarius product, CLI, configuration, integration, or internal
  contracts into Narratio.
- Creating placeholder API, consumer, ADR, or integration documents for
  interfaces that do not exist.