eric/notarius
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Consolidate documentation ownership

Browse Source
This commit is contained in:
Eric Rakestraw
2026-07-17 03:12:35 +00:00
parent 6e6375521d
commit 3b07b64a0f
10 changed files with 65 additions and 237 deletions
Download Patch File Download Diff File Expand all files Collapse all files

28
docs/config.md
View File

@@ -7,13 +7,14 @@ built-in defaults, then environment overrides are applied.
## Discovery
Commands that accept `--config` load configuration in this order:
Commands that load configuration use this order:
1. the `--config` path, when provided;
1. an explicit path supplied through the CLI, when provided;
2. `NOTARIUS_CONFIG`, when set to a non-empty path;
3. `/usr/local/etc/notarius/config.yml`.
If none is available, the command fails with a config file not found error.
The explicit-path option is defined in the [CLI reference](cli.md).
## Maintained Examples
@@ -108,11 +109,8 @@ profiles.
## Pipelines
A pipeline defines the fixed Notarius workflow:
```text
input -> chunk -> extract -> merge -> normalize -> output
```
A pipeline selects implementations for the fixed workflow defined by
[Architecture](policy/architecture.md#system-shape).
Pipeline fields:
@@ -319,8 +317,8 @@ casts still must be present in the source transcript.
`workspace` fields:
- `directory`: optional workspace root for Notarius-owned local state.
- `resume.enabled`: boolean resume checkpointing setting. Default: `false`.
- `debug.enabled`: boolean debug artifact setting. Default: `false`.
- `resume.enabled`: boolean resume checkpointing setting.
- `debug.enabled`: boolean debug artifact setting.
- `diagnostics`: optional diagnostics settings defined below.
`workspace.resume.enabled` and `workspace.debug.enabled` are independent.
@@ -331,11 +329,12 @@ permissions, and sensitive content, see [Operations](operations.md).
Preferred workspace diagnostics fields:
- `workspace.directory`: workspace root for Notarius-owned local state.
- `workspace.diagnostics.enabled`: set to `false` to skip creating diagnostics
run directories and diagnostics artifacts. Default: `true`.
run directories and diagnostics artifacts.
- `workspace.diagnostics.retention`: `auto`, `always`, or `never`.
When unset, the effective diagnostics retention default is `auto`.
Defaults for workspace and diagnostics fields are listed in
[Defaults](#defaults).
`workspace.diagnostics.retention` overrides legacy diagnostics retention when
set.
@@ -343,9 +342,8 @@ set.
`diagnostics` fields:
- `work_dir`: deprecated compatibility directory for per-run diagnostics.
Default: `/tmp/notarius`.
- `retention`: deprecated compatibility retention mode. `auto`, `always`, or
`never`. Empty uses `auto`.
`never`.
Existing `diagnostics.work_dir`, `diagnostics.retention`, `NOTARIUS_WORK_DIR`,
and `NOTARIUS_DIAGNOSTICS_RETENTION` inputs remain supported for compatibility.
@@ -371,7 +369,7 @@ Pipeline resolution additionally checks:
- the pipeline ID exists;
- at least one artifact lane is declared and selected;
- selected lanes exist when `--only` is used;
- lanes selected through the CLI exist in the resolved pipeline;
- required module keys are present;
- module keys are registered for the expected slot;
- module capability requirements are satisfied;

9
docs/integrations/dnd-spell-artifacts.md
View File

@@ -1,11 +1,11 @@
# D&D Spell Raw Output
This document is the durable raw output contract for the implemented
`dnd/spells` extractor.
This document is the durable raw output contract for the production D&D spell
extractor. Selectable extractor keys are cataloged in
[Configuration](../config.md#implemented-production-modules).
## Identity
- Extractor key: `dnd/spells`
- Prompt ID: `dnd.spells`
- Response schema key: `dnd_spells`
- Response schema ID: `notarius.dnd.spells`
@@ -27,8 +27,7 @@ structure is:
```
`spell_casts` must be present. It may be empty when no spell casts are found.
With the default configured merger, multiple chunk results are concatenated in
chunk order.
When multiple chunk results are combined, spell casts remain in chunk order.
## Spell-Cast Fields

6
docs/integrations/json-output.md
View File

@@ -1,7 +1,9 @@
# JSON Output
This document is the durable JSON output file-format contract produced by the
implemented `json` output module and written by the CLI.
production JSON encoder and written by the CLI. Selectable output-encoder keys
are cataloged in
[Configuration](../config.md#implemented-production-modules).
The output module produces the logical bundle described here. The CLI's
physical placement and lifecycle for that bundle are defined in
@@ -9,7 +11,7 @@ physical placement and lifecycle for that bundle are defined in
## Files
The `json` output module writes:
The encoder writes:
- `index.json`
- `manifest.json`

6
docs/integrations/seriatim.md
View File

@@ -1,11 +1,11 @@
# Seriatim Transcript JSON
This document is the external input contract for the implemented `seriatim`
input adapter.
This document is the external input contract consumed by the production
Seriatim input adapter. Selectable input-adapter keys are cataloged in
[Configuration](../config.md#implemented-production-modules).
## Adapter
- Module key: `seriatim`
- Source format: `application/vnd.seriatim+json`
## Accepted Shape

10
docs/internal/pipeline.md
View File

@@ -127,11 +127,11 @@ the relevant source, chunk, prior-stage, schema, reference, session, LLM, option
and run context. Validators execute in resolved order and stop at the first
error or rejection. An empty chain approves the result.
`runWithRetry` performs the initial module call plus the configured additional
attempts. Each attempt includes module execution and its complete validation
chain. A module or validator error retries and becomes a framework error after
the final attempt. A rejection retries and becomes a recorded `RejectedOutput`
after the final attempt. Cancellation stops retry processing immediately.
`runWithRetry` applies the effective retry policy around module execution and
its complete validation chain. A module or validator error becomes a framework
error when attempts are exhausted. A rejection becomes a recorded
`RejectedOutput` when attempts are exhausted. Cancellation stops retry
processing immediately.
Rejected output is a non-fatal pipeline outcome and does not advance. Warnings
from discarded attempts are not promoted. Configuration owns retry counts and

45
docs/operations.md
View File

@@ -66,17 +66,20 @@ Implemented diagnostics artifacts:
## Checkpoints
When `workspace.resume.enabled: true` and `workspace.directory` is set, runs
write checkpoints under:
When checkpoint writing is enabled for a configured workspace, runs write
checkpoints under:
```text
<workspace.directory>/checkpoints/<pipeline-id>/<input-key>-<source-or-input-digest>/<pipeline-digest>/<identity-digest>/
```
Each workflow step owns its own manifest and payload files. There is no
root-level checkpoint summary. Ordinary `notarius run` invocations execute the
pipeline normally and refresh checkpoints. `notarius run --resume` reuses valid
root-level checkpoint summary. Ordinary invocations execute the pipeline
normally and refresh checkpoints. An explicit resume invocation reuses valid
checkpoints and executes any missing, invalid, or incompatible step normally.
Configuration controls checkpoint writing, while the explicit resume option is
defined in the [Configuration](config.md#workspace) and
[CLI](cli.md#run) references.
Checkpoints do not include raw prompts, raw reference contents, raw LLM request
payloads, or debug traces. They can still contain source text, intermediate
@@ -88,14 +91,14 @@ digests match the current invocation. Changes to input bytes, the resolved
pipeline, selected lanes, the runtime LLM profile override, or bound reference
content invalidate reuse.
Plain `notarius run` does not reuse checkpoints. It executes the workflow and
refreshes checkpoint files when checkpointing is enabled. `notarius run
--resume` is the explicit reuse path.
Runs do not reuse checkpoints unless explicitly requested. Without reuse, the
workflow executes normally and refreshes checkpoint files when checkpointing is
enabled.
## Debug
When `workspace.debug.enabled: true` and `workspace.directory` is set, runs
write debug artifacts under:
When debug recording is enabled for a configured workspace, runs write debug
artifacts under:
```text
<workspace.directory>/debug/<run-id>/
@@ -112,13 +115,12 @@ a retry or validator attempt
write `prompt-000N.json`, `response-000N.json`, and
`response-content-000N.*` files under that attempt directory and are linked from
the attempt `llm_calls` array. Prompt content is written inline in the prompt
artifact. Response metadata is written to `response-000N.json`, while the
response body is written separately as pretty-printed JSON when possible or as
raw text otherwise. Debug artifacts may contain source material, reference
material, prompt inputs, model outputs, and other sensitive data. API keys are
not written, and obvious credential-shaped values and sensitive map keys are
redacted, but debug directories should still be protected as sensitive local
state.
artifact. The response metadata and body use the paired files described above;
the body is pretty-printed JSON when possible and raw text otherwise. Debug
artifacts may contain source material, reference material, prompt inputs, model
outputs, and other sensitive data. API keys are not written, and obvious
credential-shaped values and sensitive map keys are redacted, but debug
directories should still be protected as sensitive local state.
## Retention
@@ -138,13 +140,11 @@ an unusable diagnostics work directory, are printed to stderr and may not have a
diagnostics run directory.
Failures after diagnostics directory creation are printed to stderr and written
to `error.log`. Depending on where the failure occurred, diagnostics may also
include invocation metadata, redacted effective config, resolved pipeline data,
the run manifest, warnings, and a run report.
to `error.log`. Depending on where the failure occurred, the directory may also
contain artifacts written before the failure.
If durable output writing fails after the pipeline completes, diagnostics are
retained for inspection and may include `run-manifest.json`, `warnings.json`,
`run-report.json`, and `error.log`.
retained for inspection.
## Warnings
@@ -177,9 +177,6 @@ directories unless they are part of your own operational policy.
## Operational Limits
If `--resume` cannot reuse a checkpoint, Notarius executes that step and writes
a fresh checkpoint when checkpointing is enabled.
Provider retries and timeouts are handled by Scriptorium according to the
selected execution profile. Pipeline module retry settings are defined in
[Configuration](config.md#module-bindings). There is no separate CLI retry

130
docs/roadmap/documentation.md
View File

@@ -1,130 +0,0 @@
# Documentation Policy Migration
The documentation ownership policy has been revised to assign each topic to one
canonical document. This roadmap organizes the likely migration work into three
ordered passes.
This is a planning inventory, not a review of the current documents. Each item
must be verified before content is moved, removed, or rewritten. Complete the
passes in order so canonical content is established before non-owning copies are
removed.
## Pass 1: User And Operator Contracts
Establish the canonical externally observable facts first. These documents will
become the targets linked from developer and internal documentation in the next
pass.
### Product And CLI
- Audit `README.md` so it owns product orientation and the single minimal
end-to-end quickstart without becoming a command or configuration reference.
- Audit `docs/cli.md` for material owned by the README, configuration reference,
or operations guide.
- Keep commands, arguments, flags, invocation semantics, and exit codes
canonical in `docs/cli.md`.
### Configuration And Examples
- Audit `docs/config.md` for complete example files, CLI syntax, runtime state
lifecycle, or implementation detail.
- Keep fields, defaults, precedence, environment overrides, validation, and
user-selectable module and validator keys canonical in `docs/config.md`.
- Audit minimal and production-oriented configuration examples, moving complete
copyable files under `examples/` and leaving only small illustrative fragments
in reference documentation.
- Inventory complete configuration, input, command, and output examples embedded
in prose documents.
- Select one maintained copy for each complete artifact under `examples/` where
practical, and verify important examples through automated tests.
### Operations And Integrations
- Audit `docs/operations.md` for duplicated CLI syntax, configuration field
definitions, logical output schemas, or implementation mechanics.
- Keep runtime workflows, physical state, retention, recovery, permissions, and
operational limits canonical in `docs/operations.md`.
- Audit `docs/integrations/` so external formats, protocols, logical output
paths, schemas, media types, and compatibility rules have one canonical home.
- Separate logical output bundle contracts from physical runtime placement and
lifecycle.
### Pass 1 Completion
- Confirm that README, CLI, configuration, operations, integrations, and
examples have non-overlapping ownership.
- Validate user-facing commands, fields, defaults, keys, schemas, paths, and
maintained examples against implemented behavior.
- Validate links among the user and operator documents.
## Pass 2: Developer And Internal Documentation
Use the canonical contracts established in Pass 1 to remove duplicated facts
from contributor and implementation documentation.
### Orientation And Architecture
- Audit `docs/development.md` so it routes contributors without maintaining a
parallel package inventory or architectural description.
- Audit `docs/internal/overview.md` so it owns the implemented component map
without restating normative architecture.
- Audit `docs/policy/architecture.md` so it contains current normative
architecture without implementation inventory, decision history, or future
behavior.
### Internal Components
- Audit internal component documents for repeated configuration definitions,
external input or output schemas, operator procedures, and global
architectural invariants.
- Replace duplicated external schemas or field definitions with links to the
canonical configuration or integration contracts from Pass 1.
- Keep implementation flow, internal collaborators, state transitions,
package-local guarantees, failures, and relevant tests in focused internal
documents.
- Identify task-specific contributor recipes that need a focused internal home
rather than the developer landing page.
- Retain production module and validator implementation details in module
internals while linking user-selectable keys to `docs/config.md`.
### Pass 2 Completion
- Confirm that development routes, architecture governs, internal overview
inventories, and focused internal docs explain implementation.
- Confirm that developer documents link to configuration, operations, and
integration contracts rather than redefining them.
- Validate developer-facing links and relevant focused tests.
## Pass 3: Lifecycle And Final Deduplication
Finish the migration by reconciling historical and future documentation, then
perform a repository-wide ownership audit.
### ADR And Roadmap Lifecycle
- Audit roadmap files for implemented behavior or completed status summaries
that should be removed or replaced with links to current canonical docs.
- Ensure ADRs own architectural rationale and supersession history without
becoming current behavior references or implementation trackers.
- Ensure accepted but unimplemented ADR decisions link to roadmap-owned
implementation status where appropriate.
- Distinguish rejected architectural alternatives in ADRs from rejected product
ideas in roadmap files.
### Repository-Wide Audit
- Inventory repeated commands, flags, defaults, module keys, validator keys,
file names, paths, schemas, retry semantics, and runtime guarantees.
- Assign each repeated fact to the canonical owner defined by the policy.
- Replace non-owning copies with short summaries and links where navigation is
useful.
- Retain only minimal illustrative snippets in prose and link them to maintained
examples.
- Validate local links and remove references to deleted or relocated material.
### Pass 3 Completion
- Confirm that every contractual or volatile fact has one canonical owner.
- Confirm that roadmap files contain future work and implementation status, ADRs
contain decision rationale, and current docs contain implemented behavior.
- Run final link, example, and documentation consistency checks.

9
docs/roadmap/future.md
View File

@@ -25,7 +25,7 @@ future work only.
- Production LLM-backed validators when there is a concrete review policy that
benefits from model judgment.
- Validator diagnostics and timing summaries if operators need more detail than
`manifest.json`, `rejected.json`, and `warnings.json` provide.
the current [durable output bundle](../integrations/json-output.md) provides.
- Media-type validators for non-JSON module outputs when such modules are
introduced.
- Validator compatibility metadata if real deployments need config-time
@@ -42,8 +42,7 @@ future work only.
- Optional generated example output fixtures with a regeneration procedure.
- Additional diagnostics or reporting views if operator workflows need them.
## Non-Goals To Revisit Deliberately
## Candidate Workspace Work
- A general workflow language.
- Structural module selection through ad hoc run flags.
- Storing secrets in config files, diagnostics, manifests, or examples.
Workspace storage, cleanup, archival, and reuse candidates are tracked in the
[Workspace Future Work](workspace.md) roadmap.

25
docs/roadmap/implementation.md
View File

@@ -1,25 +0,0 @@
# Workspace Implementation Status
The workspace implementation described by this roadmap has landed. Current
behavior is documented in the canonical current-behavior docs:
- [Configuration](../config.md)
- [CLI Reference](../cli.md)
- [Operations](../operations.md)
- [Diagnostics Internals](../internal/diagnostics.md)
- [Pipeline Internals](../internal/pipeline.md)
Implemented behavior includes:
- `workspace.directory` as the root for Notarius-owned local state;
- workspace diagnostics under `<workspace.directory>/diagnostics/<run-id>/`;
- compatibility for legacy `diagnostics.work_dir`, `diagnostics.retention`,
`NOTARIUS_WORK_DIR`, and `NOTARIUS_DIAGNOSTICS_RETENTION`;
- checkpoint writes under `<workspace.directory>/checkpoints/` when resume
checkpointing is enabled;
- explicit checkpoint reuse through `notarius run --resume`;
- debug artifacts under `<workspace.directory>/debug/<run-id>/` when debug
output is enabled;
- independent resume and debug settings.
Deferred workspace ideas remain in [Workspace Roadmap](workspace.md).

34
docs/roadmap/workspace.md
View File

@@ -1,26 +1,14 @@
# Workspace Roadmap Status
# Workspace Future Work
The local workspace feature has been implemented. Current behavior is documented
in [Configuration](../config.md), [CLI Reference](../cli.md),
[Operations](../operations.md), and the relevant internal docs.
Current workspace settings and operating behavior are documented in
[Configuration](../config.md#workspace) and
[Operations](../operations.md). This roadmap contains only candidate additions
to that behavior.
The implemented workspace provides one configurable root for Notarius-owned
local state:
## Candidate Work
```text
<workspace.directory>/
diagnostics/
checkpoints/
debug/
```
Implemented behavior includes workspace-backed diagnostics, checkpoint writing,
explicit checkpoint reuse through `notarius run --resume`, workspace debug
artifacts, safe workspace-relative writes, and compatibility for legacy
diagnostics configuration.
## Deferred Work
Default-idempotent `run` behavior with a force override, remote workspace
storage, workspace garbage collection, archival policy, and cross-machine resume
remain deferred.
- Default-idempotent run behavior with an explicit force override.
- Remote workspace storage.
- Workspace garbage collection.
- Workspace archival policy.
- Cross-machine checkpoint reuse.