Reorganize user and operator documentation

docs/operations.md

@@ -4,24 +4,16 @@ This is the canonical reference for operating implemented Notarius runs.
 
 ## Normal Run
 
-A run reads one source file, resolves one configured pipeline, calls the
-configured Scriptorium-backed LLM runtime, writes durable JSON output, and
-writes diagnostics for inspection.
+A run reads one source file, resolves one configured pipeline, executes its
+modules, writes durable output, and writes diagnostics when enabled. Start with
+the [README quickstart](../README.md), then use the [CLI reference](cli.md) for
+invocation options.
 
-```sh
-go run ./cmd/notarius run dnd-session \
-  --config examples/dnd-spells.config.yml \
-  --input examples/seriatim-minimal-transcript.json \
-  --output-dir ./notarius-output \
-  --diagnostics-dir /tmp/notarius
-```
-
-The command prints a success line with the pipeline ID, normalized output count,
-rejected output count, and the output path.
-
-For production, configure a workspace such as `/var/lib/notarius` and ensure the
-Notarius process can create files below it. For local development, prefer an
-ignored project-local workspace such as `./.notarius/workspace`.
+For production, configure an application-owned workspace such as
+`/var/lib/notarius` and ensure the Notarius process can create files below it.
+For local development, prefer an ignored project-local workspace such as
+`./.notarius/workspace`. See [Configuration](config.md#workspace) for workspace
+fields.
 
 ## Output Directory
 
@@ -31,23 +23,10 @@ Durable output is written to:
 <output-root>/<run-id>/
 ```
 
-The default output root is `./notarius-output`. Use `--output-dir` to choose a
-different root.
-
-The `json` output module writes these files:
-
-- `index.json`: file index with paths to the manifest, lane output files,
-  rejected outputs, and warnings.
-- `manifest.json`: run manifest with resolved pipeline provenance, top-level
-  module metadata, module keys, reference provenance, validation status, and
-  timing.
-- `lanes/<lane-id>.json`: normalized raw JSON output payloads, one file per
-  lane. For the current D&D spell extractor, this includes `lanes/spells.json`.
-- `rejected.json`: rejected raw output records.
-- `warnings.json`: warnings reported by pipeline modules or the output encoder.
-
-Output writes are atomic per file. Logical output file names must be clean,
-relative, slash-separated paths and must not contain `..`.
+The output root and its invocation-specific override are defined in the
+[CLI reference](cli.md#run). Output writes are atomic per file. The
+[JSON output contract](integrations/json-output.md) defines the logical files,
+paths, schemas, and media types inside each run directory.
 
 ## Diagnostics Directory
 
@@ -57,17 +36,15 @@ Diagnostics are written under:
 <diagnostics-work-dir>/<run-id>/
 ```
 
-The default diagnostics work directory is `/tmp/notarius`. It can be set with
-`workspace.directory`, `NOTARIUS_WORKSPACE_DIR`, legacy
-`diagnostics.work_dir`, legacy `NOTARIUS_WORK_DIR`, or `--diagnostics-dir`.
-When a workspace directory is set, diagnostics are written under
-`<workspace.directory>/diagnostics/<run-id>/` unless `--diagnostics-dir`
-overrides the diagnostics work directory for that invocation.
+When a workspace directory is configured, diagnostics are written under
+`<workspace.directory>/diagnostics/<run-id>/`. An invocation-specific override
+changes only the diagnostics root, not the workspace root. Configuration and
+environment controls are defined in [Configuration](config.md); the override
+flag is defined in the [CLI reference](cli.md#run).
 
-Set `workspace.diagnostics.enabled: false` or
-`NOTARIUS_WORKSPACE_DIAGNOSTICS_ENABLED=false` to skip diagnostics directory
-creation and diagnostics artifact writes. Concise failures are still printed to
-stderr.
+Diagnostics can be disabled through configuration. When disabled, Notarius
+does not create a diagnostics run directory or write diagnostics artifacts;
+concise failures are still printed to stderr.
 
 Implemented diagnostics artifacts:
 
@@ -87,9 +64,6 @@ Implemented diagnostics artifacts:
 - `error.log`: failure message, written after diagnostics directory creation
   when a run fails.
 
-`source-document.json` is supported by the diagnostics writer but is not written
-by the current CLI run workflow.
-
 ## Checkpoints
 
 When `workspace.resume.enabled: true` and `workspace.directory` is set, runs
@@ -104,19 +78,15 @@ root-level checkpoint summary. Ordinary `notarius run` invocations execute the
 pipeline normally and refresh checkpoints. `notarius run --resume` reuses valid
 checkpoints and executes any missing, invalid, or incompatible step normally.
 
-Checkpoint payloads preserve byte content with base64 envelopes, media type,
-metadata, warnings, and content digests where applicable. Checkpoints do not
-include raw prompts, raw reference contents, raw LLM request payloads, or debug
-traces. They can still contain source text, intermediate extracted content,
-rejected outputs, metadata, and warnings. Treat checkpoint directories as
-sensitive local state.
+Checkpoints do not include raw prompts, raw reference contents, raw LLM request
+payloads, or debug traces. They can still contain source text, intermediate
+extracted content, rejected outputs, metadata, warnings, and content digests.
+Treat checkpoint directories as sensitive local state.
 
-A checkpoint is reused only when its workspace schema version, checkpoint
-identity digest, step status, dependency fingerprints, payload files, and
-payload digests match the current invocation. Changes to input bytes, resolved
-pipeline digest, selected lanes, runtime LLM profile override, materialized
-reference digests, or other identity material invalidate reuse and use a
-separate checkpoint directory.
+A checkpoint is reused only when its stored status, dependencies, payloads, and
+digests match the current invocation. Changes to input bytes, the resolved
+pipeline, selected lanes, the runtime LLM profile override, or bound reference
+content invalidate reuse.
 
 Plain `notarius run` does not reuse checkpoints. It executes the workflow and
 refreshes checkpoint files when checkpointing is enabled. `notarius run
@@ -135,10 +105,10 @@ Debug output is per invocation. It is independent of checkpointing and is not
 used for resume. Enabling debug does not write checkpoints, and enabling resume
 checkpointing does not write debug output.
 
-Debug artifacts include framework-boundary inputs and outputs for source,
-chunk, extract, merge, normalize, and output work, structured LLM request and
-response data from Notarius contracts, validator requests and results, timing,
-and retry attempt metadata. LLM calls made inside a retry or validator attempt
+Debug artifacts include inputs and outputs for source, chunk, extract, merge,
+normalize, and output work, structured LLM request and response data, validator
+requests and results, timing, and retry attempt metadata. LLM calls made inside
+a retry or validator attempt
 write `prompt-000N.json`, `response-000N.json`, and
 `response-content-000N.*` files under that attempt directory and are linked from
 the attempt `llm_calls` array. Prompt content is written inline in the prompt
@@ -147,22 +117,20 @@ response body is written separately as pretty-printed JSON when possible or as
 raw text otherwise. Debug artifacts may contain source material, reference
 material, prompt inputs, model outputs, and other sensitive data. API keys are
 not written, and obvious credential-shaped values and sensitive map keys are
-redacted in framework envelopes, but debug directories should still be protected
-as sensitive local state.
+redacted, but debug directories should still be protected as sensitive local
+state.
 
 ## Retention
 
-Diagnostics retention is configured with `workspace.diagnostics.retention`,
-`NOTARIUS_WORKSPACE_DIAGNOSTICS_RETENTION`, legacy `diagnostics.retention`,
-legacy `NOTARIUS_DIAGNOSTICS_RETENTION`, or the default `auto`.
+Diagnostics retention uses the effective mode selected through configuration;
+see [Configuration](config.md#diagnostics) for the fields, environment
+overrides, precedence, and default.
 
 - `auto`: keep failed runs and successful runs with warnings; remove successful
   warning-free runs.
 - `always`: keep every diagnostics run directory.
 - `never`: remove successful run directories; failed runs are still retained.
 
-Unknown retention values are rejected during config validation.
-
 ## Failures
 
 Failures before diagnostics directory creation, such as a missing config file or
@@ -183,11 +151,8 @@ retained for inspection and may include `run-manifest.json`, `warnings.json`,
 A successful run with warnings exits with code `0`, prints a warning count to
 stderr, and writes warnings to durable output and diagnostics when retained.
 
-The run manifest `validation_status` indicates whether raw outputs were
-approved or rejected after validation.
-
-Reference-related warnings include empty bound reference files. Empty references
-are still passed to extractors so optional slots can be intentionally blank.
+The [JSON output contract](integrations/json-output.md) defines durable warning
+and validation-status fields.
 
 ## Cleanup
 
@@ -216,9 +181,9 @@ If `--resume` cannot reuse a checkpoint, Notarius executes that step and writes
 a fresh checkpoint when checkpointing is enabled.
 
 Provider retries and timeouts are handled by Scriptorium according to the
-selected execution profile. Pipeline module retries are controlled by module
-binding `retries` values in config for chunk, extract, merge, and normalize.
-There is no separate CLI retry command.
+selected execution profile. Pipeline module retry settings are defined in
+[Configuration](config.md#module-bindings). There is no separate CLI retry
+command.
 
 Notarius writes local files only. Remote storage and archive management are not
 part of the implemented CLI.