Consolidate documentation ownership

This commit is contained in:
2026-07-17 03:12:35 +00:00
parent 6e6375521d
commit 3b07b64a0f
10 changed files with 65 additions and 237 deletions

View File

@@ -7,13 +7,14 @@ built-in defaults, then environment overrides are applied.
## Discovery ## Discovery
Commands that accept `--config` load configuration in this order: Commands that load configuration use this order:
1. the `--config` path, when provided; 1. an explicit path supplied through the CLI, when provided;
2. `NOTARIUS_CONFIG`, when set to a non-empty path; 2. `NOTARIUS_CONFIG`, when set to a non-empty path;
3. `/usr/local/etc/notarius/config.yml`. 3. `/usr/local/etc/notarius/config.yml`.
If none is available, the command fails with a config file not found error. If none is available, the command fails with a config file not found error.
The explicit-path option is defined in the [CLI reference](cli.md).
## Maintained Examples ## Maintained Examples
@@ -108,11 +109,8 @@ profiles.
## Pipelines ## Pipelines
A pipeline defines the fixed Notarius workflow: A pipeline selects implementations for the fixed workflow defined by
[Architecture](policy/architecture.md#system-shape).
```text
input -> chunk -> extract -> merge -> normalize -> output
```
Pipeline fields: Pipeline fields:
@@ -319,8 +317,8 @@ casts still must be present in the source transcript.
`workspace` fields: `workspace` fields:
- `directory`: optional workspace root for Notarius-owned local state. - `directory`: optional workspace root for Notarius-owned local state.
- `resume.enabled`: boolean resume checkpointing setting. Default: `false`. - `resume.enabled`: boolean resume checkpointing setting.
- `debug.enabled`: boolean debug artifact setting. Default: `false`. - `debug.enabled`: boolean debug artifact setting.
- `diagnostics`: optional diagnostics settings defined below. - `diagnostics`: optional diagnostics settings defined below.
`workspace.resume.enabled` and `workspace.debug.enabled` are independent. `workspace.resume.enabled` and `workspace.debug.enabled` are independent.
@@ -331,11 +329,12 @@ permissions, and sensitive content, see [Operations](operations.md).
Preferred workspace diagnostics fields: Preferred workspace diagnostics fields:
- `workspace.directory`: workspace root for Notarius-owned local state.
- `workspace.diagnostics.enabled`: set to `false` to skip creating diagnostics - `workspace.diagnostics.enabled`: set to `false` to skip creating diagnostics
run directories and diagnostics artifacts. Default: `true`. run directories and diagnostics artifacts.
- `workspace.diagnostics.retention`: `auto`, `always`, or `never`. - `workspace.diagnostics.retention`: `auto`, `always`, or `never`.
When unset, the effective diagnostics retention default is `auto`.
Defaults for workspace and diagnostics fields are listed in
[Defaults](#defaults).
`workspace.diagnostics.retention` overrides legacy diagnostics retention when `workspace.diagnostics.retention` overrides legacy diagnostics retention when
set. set.
@@ -343,9 +342,8 @@ set.
`diagnostics` fields: `diagnostics` fields:
- `work_dir`: deprecated compatibility directory for per-run diagnostics. - `work_dir`: deprecated compatibility directory for per-run diagnostics.
Default: `/tmp/notarius`.
- `retention`: deprecated compatibility retention mode. `auto`, `always`, or - `retention`: deprecated compatibility retention mode. `auto`, `always`, or
`never`. Empty uses `auto`. `never`.
Existing `diagnostics.work_dir`, `diagnostics.retention`, `NOTARIUS_WORK_DIR`, Existing `diagnostics.work_dir`, `diagnostics.retention`, `NOTARIUS_WORK_DIR`,
and `NOTARIUS_DIAGNOSTICS_RETENTION` inputs remain supported for compatibility. and `NOTARIUS_DIAGNOSTICS_RETENTION` inputs remain supported for compatibility.
@@ -371,7 +369,7 @@ Pipeline resolution additionally checks:
- the pipeline ID exists; - the pipeline ID exists;
- at least one artifact lane is declared and selected; - at least one artifact lane is declared and selected;
- selected lanes exist when `--only` is used; - lanes selected through the CLI exist in the resolved pipeline;
- required module keys are present; - required module keys are present;
- module keys are registered for the expected slot; - module keys are registered for the expected slot;
- module capability requirements are satisfied; - module capability requirements are satisfied;

View File

@@ -1,11 +1,11 @@
# D&D Spell Raw Output # D&D Spell Raw Output
This document is the durable raw output contract for the implemented This document is the durable raw output contract for the production D&D spell
`dnd/spells` extractor. extractor. Selectable extractor keys are cataloged in
[Configuration](../config.md#implemented-production-modules).
## Identity ## Identity
- Extractor key: `dnd/spells`
- Prompt ID: `dnd.spells` - Prompt ID: `dnd.spells`
- Response schema key: `dnd_spells` - Response schema key: `dnd_spells`
- Response schema ID: `notarius.dnd.spells` - Response schema ID: `notarius.dnd.spells`
@@ -27,8 +27,7 @@ structure is:
``` ```
`spell_casts` must be present. It may be empty when no spell casts are found. `spell_casts` must be present. It may be empty when no spell casts are found.
With the default configured merger, multiple chunk results are concatenated in When multiple chunk results are combined, spell casts remain in chunk order.
chunk order.
## Spell-Cast Fields ## Spell-Cast Fields

View File

@@ -1,7 +1,9 @@
# JSON Output # JSON Output
This document is the durable JSON output file-format contract produced by the This document is the durable JSON output file-format contract produced by the
implemented `json` output module and written by the CLI. production JSON encoder and written by the CLI. Selectable output-encoder keys
are cataloged in
[Configuration](../config.md#implemented-production-modules).
The output module produces the logical bundle described here. The CLI's The output module produces the logical bundle described here. The CLI's
physical placement and lifecycle for that bundle are defined in physical placement and lifecycle for that bundle are defined in
@@ -9,7 +11,7 @@ physical placement and lifecycle for that bundle are defined in
## Files ## Files
The `json` output module writes: The encoder writes:
- `index.json` - `index.json`
- `manifest.json` - `manifest.json`

View File

@@ -1,11 +1,11 @@
# Seriatim Transcript JSON # Seriatim Transcript JSON
This document is the external input contract for the implemented `seriatim` This document is the external input contract consumed by the production
input adapter. Seriatim input adapter. Selectable input-adapter keys are cataloged in
[Configuration](../config.md#implemented-production-modules).
## Adapter ## Adapter
- Module key: `seriatim`
- Source format: `application/vnd.seriatim+json` - Source format: `application/vnd.seriatim+json`
## Accepted Shape ## Accepted Shape

View File

@@ -127,11 +127,11 @@ the relevant source, chunk, prior-stage, schema, reference, session, LLM, option
and run context. Validators execute in resolved order and stop at the first and run context. Validators execute in resolved order and stop at the first
error or rejection. An empty chain approves the result. error or rejection. An empty chain approves the result.
`runWithRetry` performs the initial module call plus the configured additional `runWithRetry` applies the effective retry policy around module execution and
attempts. Each attempt includes module execution and its complete validation its complete validation chain. A module or validator error becomes a framework
chain. A module or validator error retries and becomes a framework error after error when attempts are exhausted. A rejection becomes a recorded
the final attempt. A rejection retries and becomes a recorded `RejectedOutput` `RejectedOutput` when attempts are exhausted. Cancellation stops retry
after the final attempt. Cancellation stops retry processing immediately. processing immediately.
Rejected output is a non-fatal pipeline outcome and does not advance. Warnings Rejected output is a non-fatal pipeline outcome and does not advance. Warnings
from discarded attempts are not promoted. Configuration owns retry counts and from discarded attempts are not promoted. Configuration owns retry counts and

View File

@@ -66,17 +66,20 @@ Implemented diagnostics artifacts:
## Checkpoints ## Checkpoints
When `workspace.resume.enabled: true` and `workspace.directory` is set, runs When checkpoint writing is enabled for a configured workspace, runs write
write checkpoints under: checkpoints under:
```text ```text
<workspace.directory>/checkpoints/<pipeline-id>/<input-key>-<source-or-input-digest>/<pipeline-digest>/<identity-digest>/ <workspace.directory>/checkpoints/<pipeline-id>/<input-key>-<source-or-input-digest>/<pipeline-digest>/<identity-digest>/
``` ```
Each workflow step owns its own manifest and payload files. There is no Each workflow step owns its own manifest and payload files. There is no
root-level checkpoint summary. Ordinary `notarius run` invocations execute the root-level checkpoint summary. Ordinary invocations execute the pipeline
pipeline normally and refresh checkpoints. `notarius run --resume` reuses valid normally and refresh checkpoints. An explicit resume invocation reuses valid
checkpoints and executes any missing, invalid, or incompatible step normally. checkpoints and executes any missing, invalid, or incompatible step normally.
Configuration controls checkpoint writing, while the explicit resume option is
defined in the [Configuration](config.md#workspace) and
[CLI](cli.md#run) references.
Checkpoints do not include raw prompts, raw reference contents, raw LLM request Checkpoints do not include raw prompts, raw reference contents, raw LLM request
payloads, or debug traces. They can still contain source text, intermediate payloads, or debug traces. They can still contain source text, intermediate
@@ -88,14 +91,14 @@ digests match the current invocation. Changes to input bytes, the resolved
pipeline, selected lanes, the runtime LLM profile override, or bound reference pipeline, selected lanes, the runtime LLM profile override, or bound reference
content invalidate reuse. content invalidate reuse.
Plain `notarius run` does not reuse checkpoints. It executes the workflow and Runs do not reuse checkpoints unless explicitly requested. Without reuse, the
refreshes checkpoint files when checkpointing is enabled. `notarius run workflow executes normally and refreshes checkpoint files when checkpointing is
--resume` is the explicit reuse path. enabled.
## Debug ## Debug
When `workspace.debug.enabled: true` and `workspace.directory` is set, runs When debug recording is enabled for a configured workspace, runs write debug
write debug artifacts under: artifacts under:
```text ```text
<workspace.directory>/debug/<run-id>/ <workspace.directory>/debug/<run-id>/
@@ -112,13 +115,12 @@ a retry or validator attempt
write `prompt-000N.json`, `response-000N.json`, and write `prompt-000N.json`, `response-000N.json`, and
`response-content-000N.*` files under that attempt directory and are linked from `response-content-000N.*` files under that attempt directory and are linked from
the attempt `llm_calls` array. Prompt content is written inline in the prompt the attempt `llm_calls` array. Prompt content is written inline in the prompt
artifact. Response metadata is written to `response-000N.json`, while the artifact. The response metadata and body use the paired files described above;
response body is written separately as pretty-printed JSON when possible or as the body is pretty-printed JSON when possible and raw text otherwise. Debug
raw text otherwise. Debug artifacts may contain source material, reference artifacts may contain source material, reference material, prompt inputs, model
material, prompt inputs, model outputs, and other sensitive data. API keys are outputs, and other sensitive data. API keys are not written, and obvious
not written, and obvious credential-shaped values and sensitive map keys are credential-shaped values and sensitive map keys are redacted, but debug
redacted, but debug directories should still be protected as sensitive local directories should still be protected as sensitive local state.
state.
## Retention ## Retention
@@ -138,13 +140,11 @@ an unusable diagnostics work directory, are printed to stderr and may not have a
diagnostics run directory. diagnostics run directory.
Failures after diagnostics directory creation are printed to stderr and written Failures after diagnostics directory creation are printed to stderr and written
to `error.log`. Depending on where the failure occurred, diagnostics may also to `error.log`. Depending on where the failure occurred, the directory may also
include invocation metadata, redacted effective config, resolved pipeline data, contain artifacts written before the failure.
the run manifest, warnings, and a run report.
If durable output writing fails after the pipeline completes, diagnostics are If durable output writing fails after the pipeline completes, diagnostics are
retained for inspection and may include `run-manifest.json`, `warnings.json`, retained for inspection.
`run-report.json`, and `error.log`.
## Warnings ## Warnings
@@ -177,9 +177,6 @@ directories unless they are part of your own operational policy.
## Operational Limits ## Operational Limits
If `--resume` cannot reuse a checkpoint, Notarius executes that step and writes
a fresh checkpoint when checkpointing is enabled.
Provider retries and timeouts are handled by Scriptorium according to the Provider retries and timeouts are handled by Scriptorium according to the
selected execution profile. Pipeline module retry settings are defined in selected execution profile. Pipeline module retry settings are defined in
[Configuration](config.md#module-bindings). There is no separate CLI retry [Configuration](config.md#module-bindings). There is no separate CLI retry

View File

@@ -1,130 +0,0 @@
# Documentation Policy Migration
The documentation ownership policy has been revised to assign each topic to one
canonical document. This roadmap organizes the likely migration work into three
ordered passes.
This is a planning inventory, not a review of the current documents. Each item
must be verified before content is moved, removed, or rewritten. Complete the
passes in order so canonical content is established before non-owning copies are
removed.
## Pass 1: User And Operator Contracts
Establish the canonical externally observable facts first. These documents will
become the targets linked from developer and internal documentation in the next
pass.
### Product And CLI
- Audit `README.md` so it owns product orientation and the single minimal
end-to-end quickstart without becoming a command or configuration reference.
- Audit `docs/cli.md` for material owned by the README, configuration reference,
or operations guide.
- Keep commands, arguments, flags, invocation semantics, and exit codes
canonical in `docs/cli.md`.
### Configuration And Examples
- Audit `docs/config.md` for complete example files, CLI syntax, runtime state
lifecycle, or implementation detail.
- Keep fields, defaults, precedence, environment overrides, validation, and
user-selectable module and validator keys canonical in `docs/config.md`.
- Audit minimal and production-oriented configuration examples, moving complete
copyable files under `examples/` and leaving only small illustrative fragments
in reference documentation.
- Inventory complete configuration, input, command, and output examples embedded
in prose documents.
- Select one maintained copy for each complete artifact under `examples/` where
practical, and verify important examples through automated tests.
### Operations And Integrations
- Audit `docs/operations.md` for duplicated CLI syntax, configuration field
definitions, logical output schemas, or implementation mechanics.
- Keep runtime workflows, physical state, retention, recovery, permissions, and
operational limits canonical in `docs/operations.md`.
- Audit `docs/integrations/` so external formats, protocols, logical output
paths, schemas, media types, and compatibility rules have one canonical home.
- Separate logical output bundle contracts from physical runtime placement and
lifecycle.
### Pass 1 Completion
- Confirm that README, CLI, configuration, operations, integrations, and
examples have non-overlapping ownership.
- Validate user-facing commands, fields, defaults, keys, schemas, paths, and
maintained examples against implemented behavior.
- Validate links among the user and operator documents.
## Pass 2: Developer And Internal Documentation
Use the canonical contracts established in Pass 1 to remove duplicated facts
from contributor and implementation documentation.
### Orientation And Architecture
- Audit `docs/development.md` so it routes contributors without maintaining a
parallel package inventory or architectural description.
- Audit `docs/internal/overview.md` so it owns the implemented component map
without restating normative architecture.
- Audit `docs/policy/architecture.md` so it contains current normative
architecture without implementation inventory, decision history, or future
behavior.
### Internal Components
- Audit internal component documents for repeated configuration definitions,
external input or output schemas, operator procedures, and global
architectural invariants.
- Replace duplicated external schemas or field definitions with links to the
canonical configuration or integration contracts from Pass 1.
- Keep implementation flow, internal collaborators, state transitions,
package-local guarantees, failures, and relevant tests in focused internal
documents.
- Identify task-specific contributor recipes that need a focused internal home
rather than the developer landing page.
- Retain production module and validator implementation details in module
internals while linking user-selectable keys to `docs/config.md`.
### Pass 2 Completion
- Confirm that development routes, architecture governs, internal overview
inventories, and focused internal docs explain implementation.
- Confirm that developer documents link to configuration, operations, and
integration contracts rather than redefining them.
- Validate developer-facing links and relevant focused tests.
## Pass 3: Lifecycle And Final Deduplication
Finish the migration by reconciling historical and future documentation, then
perform a repository-wide ownership audit.
### ADR And Roadmap Lifecycle
- Audit roadmap files for implemented behavior or completed status summaries
that should be removed or replaced with links to current canonical docs.
- Ensure ADRs own architectural rationale and supersession history without
becoming current behavior references or implementation trackers.
- Ensure accepted but unimplemented ADR decisions link to roadmap-owned
implementation status where appropriate.
- Distinguish rejected architectural alternatives in ADRs from rejected product
ideas in roadmap files.
### Repository-Wide Audit
- Inventory repeated commands, flags, defaults, module keys, validator keys,
file names, paths, schemas, retry semantics, and runtime guarantees.
- Assign each repeated fact to the canonical owner defined by the policy.
- Replace non-owning copies with short summaries and links where navigation is
useful.
- Retain only minimal illustrative snippets in prose and link them to maintained
examples.
- Validate local links and remove references to deleted or relocated material.
### Pass 3 Completion
- Confirm that every contractual or volatile fact has one canonical owner.
- Confirm that roadmap files contain future work and implementation status, ADRs
contain decision rationale, and current docs contain implemented behavior.
- Run final link, example, and documentation consistency checks.

View File

@@ -25,7 +25,7 @@ future work only.
- Production LLM-backed validators when there is a concrete review policy that - Production LLM-backed validators when there is a concrete review policy that
benefits from model judgment. benefits from model judgment.
- Validator diagnostics and timing summaries if operators need more detail than - Validator diagnostics and timing summaries if operators need more detail than
`manifest.json`, `rejected.json`, and `warnings.json` provide. the current [durable output bundle](../integrations/json-output.md) provides.
- Media-type validators for non-JSON module outputs when such modules are - Media-type validators for non-JSON module outputs when such modules are
introduced. introduced.
- Validator compatibility metadata if real deployments need config-time - Validator compatibility metadata if real deployments need config-time
@@ -42,8 +42,7 @@ future work only.
- Optional generated example output fixtures with a regeneration procedure. - Optional generated example output fixtures with a regeneration procedure.
- Additional diagnostics or reporting views if operator workflows need them. - Additional diagnostics or reporting views if operator workflows need them.
## Non-Goals To Revisit Deliberately ## Candidate Workspace Work
- A general workflow language. Workspace storage, cleanup, archival, and reuse candidates are tracked in the
- Structural module selection through ad hoc run flags. [Workspace Future Work](workspace.md) roadmap.
- Storing secrets in config files, diagnostics, manifests, or examples.

View File

@@ -1,25 +0,0 @@
# Workspace Implementation Status
The workspace implementation described by this roadmap has landed. Current
behavior is documented in the canonical current-behavior docs:
- [Configuration](../config.md)
- [CLI Reference](../cli.md)
- [Operations](../operations.md)
- [Diagnostics Internals](../internal/diagnostics.md)
- [Pipeline Internals](../internal/pipeline.md)
Implemented behavior includes:
- `workspace.directory` as the root for Notarius-owned local state;
- workspace diagnostics under `<workspace.directory>/diagnostics/<run-id>/`;
- compatibility for legacy `diagnostics.work_dir`, `diagnostics.retention`,
`NOTARIUS_WORK_DIR`, and `NOTARIUS_DIAGNOSTICS_RETENTION`;
- checkpoint writes under `<workspace.directory>/checkpoints/` when resume
checkpointing is enabled;
- explicit checkpoint reuse through `notarius run --resume`;
- debug artifacts under `<workspace.directory>/debug/<run-id>/` when debug
output is enabled;
- independent resume and debug settings.
Deferred workspace ideas remain in [Workspace Roadmap](workspace.md).

View File

@@ -1,26 +1,14 @@
# Workspace Roadmap Status # Workspace Future Work
The local workspace feature has been implemented. Current behavior is documented Current workspace settings and operating behavior are documented in
in [Configuration](../config.md), [CLI Reference](../cli.md), [Configuration](../config.md#workspace) and
[Operations](../operations.md), and the relevant internal docs. [Operations](../operations.md). This roadmap contains only candidate additions
to that behavior.
The implemented workspace provides one configurable root for Notarius-owned ## Candidate Work
local state:
```text - Default-idempotent run behavior with an explicit force override.
<workspace.directory>/ - Remote workspace storage.
diagnostics/ - Workspace garbage collection.
checkpoints/ - Workspace archival policy.
debug/ - Cross-machine checkpoint reuse.
```
Implemented behavior includes workspace-backed diagnostics, checkpoint writing,
explicit checkpoint reuse through `notarius run --resume`, workspace debug
artifacts, safe workspace-relative writes, and compatibility for legacy
diagnostics configuration.
## Deferred Work
Default-idempotent `run` behavior with a force override, remote workspace
storage, workspace garbage collection, archival policy, and cross-machine resume
remain deferred.