diff --git a/docs/roadmap/5-seriatim-input-module.md b/docs/roadmap/5-seriatim-input-module.md index d561452..44e048e 100644 --- a/docs/roadmap/5-seriatim-input-module.md +++ b/docs/roadmap/5-seriatim-input-module.md @@ -23,8 +23,7 @@ In scope: - input adapter registry wiring; - module metadata/capabilities for pipeline validation; - fixtures and tests; -- config path to use the input module through a named pipeline profile if - config loading exists. +- config compatibility through named pipeline profiles. Out of scope: @@ -33,13 +32,16 @@ Out of scope: - transcript-specific behavior in runner/core packages; - support for every possible Seriatim schema variant. -## Proposed Stages +## Target End State -### Seriatim Source Model +The repository should contain a real Seriatim input-stage module that translates +Seriatim minimal transcript JSON into the generic source model. -Define module-local structs for the Seriatim minimal output schema. +The Seriatim module should be registered under the stable input adapter key +`seriatim`. It should be selectable through the existing input adapter registry +and through pipeline-profile resolution when a profile binds `input: seriatim`. -Expected external shape: +The module should accept the Seriatim minimal output shape: - top-level `metadata`; - top-level `segments`; @@ -49,57 +51,36 @@ Expected external shape: - segment `speaker`; - segment `text`. -Keep these structs in the Seriatim input module package. - -### Parse And Validate - -Implement parser and validation behavior. - -Validation should cover: - -- valid JSON; -- required metadata fields; -- required segment fields; -- unique segment IDs; -- non-empty segment text; -- valid start/end values as appropriate. - -Prefer clear module-specific errors. - -### Map To SourceDocument - -Map Seriatim data into the generic source model: +The module should map Seriatim data into generic source values: - segment `id` becomes `SourceUnit.ID`; - segment `text` becomes `SourceUnit.Text`; -- unit kind should identify transcript-like units without requiring core - packages to know transcript semantics; -- `speaker`, `start`, and `end` become unit metadata; -- Seriatim metadata becomes document metadata. +- the document and unit kind strings identify transcript-like source material + without adding transcript-specific fields or types to core packages; +- `speaker`, `start`, and `end` become source-unit metadata; +- top-level Seriatim metadata becomes source-document metadata; +- the resulting source document passes core source validation. -The resulting `SourceDocument` should pass core source validation. +The module should reject invalid Seriatim input with clear module-specific +errors. Validation should cover: -### Registry And CLI Wiring +- valid JSON; +- required top-level metadata and segments; +- required segment fields; +- unique segment IDs; +- non-empty segment text; +- valid start and end values. -Register the module under a stable input adapter key, likely `seriatim`. +The module should declare flat capabilities for pipeline validation. Initial +capabilities should describe transcript-oriented source properties preserved by +the adapter, including speaker and timestamp metadata. -Declare module metadata for pipeline validation. Initial provided capabilities -should include transcript-oriented metadata such as `speaker` and `timestamps` -if those fields are preserved from Seriatim input. +Implementation staging belongs in +[`implementation.md`](implementation.md). -If config and CLI support exist, add a minimal pipeline-profile fixture or test -config using the Seriatim input module: +## Fixtures And Tests -```sh -notarius run dnd-session --input ./transcript.json --only spells -``` - -The command may still use fake extract, merge, normalize, and output modules -until checkpoint 6. - -### Fixtures And Tests - -Add fixtures and tests for: +The checkpoint should add synthetic fixtures and focused tests for: - valid Seriatim minimal transcript; - malformed JSON; diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index ad23c69..3f302fd 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1,15 +1,15 @@ -# Implementation Plan: Checkpoint 4 Portable Audita Infrastructure +# Implementation Plan: Checkpoint 5 Seriatim Input Module ## Status This is a staged implementation plan for -[`4-portable-audita-infrastructure.md`](4-portable-audita-infrastructure.md). -It is intended for an LLM coding agent to follow stage by stage. +[`5-seriatim-input-module.md`](5-seriatim-input-module.md). It is intended for +an LLM coding agent to follow stage by stage. -This plan implements only checkpoint 4. Do not add real Seriatim parsing, real -D&D extraction, real domain prompts or schemas, production extraction modules, -durable final artifact writing, or `notarius run` execution behavior in this -checkpoint. +This plan implements only checkpoint 5. Do not add D&D extraction, real domain +prompts or schemas, LLM extraction calls, a `notarius run` command, broad +Seriatim schema support, or transcript-specific behavior in core framework +packages in this checkpoint. ## Policy Context @@ -20,896 +20,424 @@ Follow: Required boundaries: -- copy only reusable Audita infrastructure, not Audita's transcript-correction - model; -- keep framework code source-agnostic and domain-agnostic; -- keep provider-specific HTTP request/response details inside - `internal/framework/llm`; -- keep prompt construction and prompt assets out of provider adapters; -- keep structural pipeline wiring in named pipeline config, not ad hoc CLI - flags; -- preserve the fixed pipeline shape: - `input -> chunk -> extract -> merge -> normalize -> output`; -- keep capabilities as flat strings; -- redact configured secrets from diagnostics, effective config, and surfaced - provider errors. - -Audita files useful for adaptation: - -- `../audita/internal/framework/llm/*` -- `../audita/internal/framework/responseschema/registry.go` -- `../audita/internal/prompts/registry.go` -- `../audita/internal/prompts/render.go` -- `../audita/internal/core/diagnostics/*` -- `../audita/internal/core/config/*` -- `../audita/internal/cli/*` -- `../audita/examples/*.yml` +- keep Seriatim JSON schema details inside `internal/modules/input/seriatim`; +- keep core source, runner, pipeline, extractor, validator, LLM, and config + packages source-agnostic and domain-agnostic; +- do not add transcript-specific typed fields to `SourceDocument`, + `SourceUnit`, runner contracts, or pipeline contracts; +- preserve transcript-specific values only as source metadata conventions; +- register the input module through the existing input adapter registry instead + of adding ad hoc conditionals; +- use flat capability strings in module metadata; +- keep future or planned behavior in `docs/roadmap/` until implemented. ## Global Implementation Decisions -- Add `gopkg.in/yaml.v3` as the only new third-party dependency in this - checkpoint. Audita already uses it, and YAML is the architecture-policy - preference for Notarius config. -- Put LLM client, scheduler, structured-output decoding helpers, secret - redaction, and response-schema registry code in `internal/framework/llm`. - Do not create separate `responseschema`, `structuredoutput`, or `warnings` - framework packages. -- Put prompt registry and rendering code in `internal/framework/prompt`. -- Put config structs, defaults, file/env/flag precedence helpers, redaction, and - validation in `internal/core/config`. -- Put diagnostics run-directory code in `internal/core/diagnostics`. This - package is justified by real durable diagnostic state in checkpoint 4; keep it - extraction-oriented and do not copy correction-ledger concepts. -- Keep CLI support limited to discovery/validation commands: - `notarius config validate` and `notarius pipelines list`. -- Do not add embedded built-in pipeline profiles. Config files are the only - source of user-defined pipeline profiles in this checkpoint. -- Do not add real D&D prompt/schema assets. Use inert placeholder assets only - for registry tests. -- Use JSON for diagnostics artifacts even when the input config is YAML. -- Use `sha256:` for content digests written to diagnostics or registries, - unless an existing Notarius type already expects bare hex. Prefer the prefixed - form for new Notarius code. -- Run `gofmt` on touched Go files after every stage. - -## Stage 1: LLM Runtime - -### Goal - -Adapt Audita's OpenAI-compatible structured-output client, secret redaction, and -scheduler to Notarius's existing `contracts.StructuredLLMClient`. - -### Files To Add - -- `internal/framework/llm/client_common.go` -- `internal/framework/llm/openai_compatible_client.go` -- `internal/framework/llm/openai_compatible_client_test.go` -- `internal/framework/llm/scheduler.go` -- `internal/framework/llm/scheduler_test.go` -- `internal/framework/llm/secrets.go` -- `internal/framework/llm/secrets_test.go` - -### Required API - -Add: +- Add no new third-party dependency. Use `encoding/json` with + `Decoder.UseNumber` for Seriatim JSON parsing. +- Use `seriatim` as the stable input adapter key. +- Put all concrete Seriatim input code under + `internal/modules/input/seriatim`. +- Expose a small module API: ```go -type OpenAICompatibleClientConfig struct { - BaseURL string - Model string - APIKey string - MaxRetries int - HTTPClient *http.Client - RequestTimeout time.Duration -} +const Key = "seriatim" -type OpenAICompatibleClient struct { /* unexported fields */ } - -func NewOpenAICompatibleClient(cfg OpenAICompatibleClientConfig) (*OpenAICompatibleClient, error) -func (c *OpenAICompatibleClient) CompleteStructured(ctx context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) +func New() *Adapter +func ModuleSpec() pipeline.ModuleSpec +func Register(registry *pipeline.InputAdapterRegistry) error ``` -Add: - -```go -type Scheduler struct { /* unexported fields */ } -func NewScheduler(maxConcurrency int) (*Scheduler, error) -func (s *Scheduler) Acquire(ctx context.Context) (func(), error) -func (s *Scheduler) Run(ctx context.Context, fn func(context.Context) error) error -``` - -Add secret helpers: - -```go -func RedactSecrets(message string, secrets []string) string -func ErrorWithSecretsRedacted(err error, secrets []string) error -``` - -### Notarius Adaptations - -Audita's client expects a typed response schema object. Notarius currently uses: - -```go -type StructuredCompletionRequest struct { - ResponseSchemaName string - ResponseSchema json.RawMessage -} -``` - -Therefore the Notarius client must: - -- require `ResponseSchemaName` to be non-empty; -- require `ResponseSchema` to be non-empty valid JSON; -- send OpenAI-compatible `response_format.type=json_schema`; -- send `json_schema.name` from `ResponseSchemaName`; -- send `json_schema.schema` from `ResponseSchema`; -- decode the assistant message content into `out`; -- return raw assistant content in `StructuredCompletionResponse.Content`; -- populate provider/model/token metadata when available; -- use `req.Model` when present, otherwise the configured default model. - -Provider request/response structs must be unexported. - -### Required Behavior - -- Validate `out` is a non-nil pointer. -- Reject empty base URL, empty default model, negative retries, empty model at - call time, missing schema name, and invalid schema JSON. -- Trim message roles/content and reject empty roles or content. -- Retry transport errors, malformed provider envelopes, malformed assistant JSON, - HTTP 429, and HTTP 5xx up to `MaxRetries`. -- Do not retry non-retryable 4xx provider errors. -- Respect `context.Context` cancellation. -- Redact API keys and bearer tokens from returned errors. -- Scheduler must bound concurrent calls, release permits exactly once, handle - cancellation while queued, and avoid leaking permits when cancellation races - with grant. - -### Required Tests - -Port and adapt Audita LLM tests using local `httptest.Server` fakes: - -- successful structured completion; -- request body includes model, messages, schema name, and schema JSON; -- default model fallback and request model override; -- invalid output target; -- missing/invalid schema; -- provider non-2xx behavior; -- retry behavior for 429/5xx and malformed retryable responses; -- no retry for non-retryable 4xx; -- provider error redacts configured API key; -- scheduler max concurrency; -- scheduler cancellation while queued; -- scheduler release function is idempotent. - -### Validation - -Run: - -```sh -gofmt -w internal/framework/llm -go test ./internal/framework/llm -go test ./... -``` - -## Stage 2: Embedded Response Schema Registry - -### Goal - -Adapt Audita's embedded response-schema registry pattern into -`internal/framework/llm` without adding real extractor schemas. - -### Files To Add - -- `internal/framework/llm/schema_registry.go` -- `internal/framework/llm/schema_registry_test.go` -- `internal/framework/llm/assets/schemas/test_artifact.v1.json` -- `internal/framework/llm/assets/schemas/test_validator_decision.v1.json` - -### Required API - -Add: - -```go -type ResponseSchemaKey string - -const ( - TestArtifactSchemaKey ResponseSchemaKey = "test_artifact" - TestValidatorDecisionSchemaKey ResponseSchemaKey = "test_validator_decision" -) - -type ResponseSchema struct { - Key ResponseSchemaKey `json:"key"` - ID string `json:"id"` - Version string `json:"version"` - Name string `json:"name"` - JSONSchema json.RawMessage `json:"json_schema"` - SHA256 string `json:"sha256"` -} - -func RegisteredResponseSchemas() []ResponseSchema -func LookupResponseSchema(key ResponseSchemaKey) (ResponseSchema, bool) -func MustLookupResponseSchema(key ResponseSchemaKey) ResponseSchema -func (s ResponseSchema) DiagnosticsMap() map[string]any -``` - -### Required Behavior - -- Embed schema assets using `embed.FS`. -- Validate assets are valid JSON at package initialization. -- Compute `SHA256` from exact asset bytes and format as `sha256:`. -- Return defensive copies of `json.RawMessage`. -- Return registered schemas sorted by key. -- Use Notarius-neutral IDs and names, for example - `notarius.test_artifact` and `notarius_test_artifact_v1`. -- Keep test schemas inert and clearly non-domain-specific. - -### Required Tests - -- lookup succeeds for both test schemas; -- unknown lookup returns false; -- `MustLookupResponseSchema` panics for unknown keys; -- registered schemas are sorted; -- JSON schema content is valid JSON; -- returned schema JSON is mutation-safe; -- diagnostics map omits raw schema content and includes ID, version, name, and - hash. - -### Validation - -Run: - -```sh -gofmt -w internal/framework/llm -go test ./internal/framework/llm -go test ./... -``` - -## Stage 3: Prompt Registry - -### Goal - -Adapt Audita's embedded prompt registry and rendering pattern into -`internal/framework/prompt` using only generic test prompts. - -### Files To Add - -- `internal/framework/prompt/registry.go` -- `internal/framework/prompt/render.go` -- `internal/framework/prompt/registry_test.go` -- `internal/framework/prompt/render_test.go` -- `internal/framework/prompt/assets/shared/prompt_hardening.md` -- `internal/framework/prompt/assets/test/generic/system.md` -- `internal/framework/prompt/assets/test/generic/user.md` - -### Required API - -Add: - -```go -const ( - SourceBuiltin = "builtin" - VersionV1 = "v1" - TestGenericPromptID = "test.generic" -) - -type Metadata struct { - PromptID string `json:"prompt_id"` - PromptVersion string `json:"prompt_version"` - PromptSource string `json:"prompt_source"` - EmbeddedPath string `json:"embedded_path"` - SHA256 string `json:"sha256"` -} - -func LookupMetadata(promptID string) (Metadata, bool) -func MustLookupMetadata(promptID string) Metadata -func RegisteredMetadata() []Metadata -func HardeningText() string -func RenderUserSystem(promptID string, data any) (system string, user string, metadata Metadata, err error) -func (m Metadata) DiagnosticsMap() map[string]any -``` - -### Required Behavior - -- Compile embedded system/user prompts with `text/template`. -- Use `Option("missingkey=error")`. -- Provide a `{{ hardening }}` template function backed by the shared hardening - asset. -- Trim rendered system/user text. -- Compute `SHA256` from combined system and user source text and format as +- `ModuleSpec()` must return stage `pipeline.StageInput`, no required + capabilities, and these provided capabilities: + `source.transcript`, `transcript.speaker`, and `transcript.timestamps`. +- The Seriatim adapter should satisfy `contracts.InputAdapter`. +- Use `source.SourceDocument.Kind = "transcript"`. +- Use `source.SourceDocument.Format = + "application/vnd.seriatim.minimal+json"`. +- Use `source.SourceUnit.Kind = "transcript_segment"`. +- Compute `SourceDocument.Digest` from the exact raw input bytes as `sha256:`. -- Return prompt metadata sorted by prompt ID. -- Keep assets generic and non-domain-specific. +- Resolve `SourceDocument.ID` in this order: + 1. trimmed `contracts.ParseRequest.SourceID`, if non-empty; + 2. trimmed string `metadata.id`, if present and non-empty; + 3. trimmed string `metadata.source_id`, if present and non-empty; + 4. deterministic fallback `seriatim:`. +- Segment IDs become source unit IDs exactly after validation. Reject segment + IDs with leading or trailing whitespace rather than silently rewriting them. +- Copy top-level Seriatim `metadata` into `SourceDocument.Metadata`. +- Store segment `speaker`, `start`, and `end` in `SourceUnit.Metadata` under + keys with those exact names. +- Store `start` and `end` as `json.Number` values so JSON serialization remains + numeric and the original decimal representation is preserved. +- Require top-level `metadata` to be present and be an object, but do not + require any specific metadata keys in checkpoint 5. +- Require top-level `segments` to be present and contain at least one segment. +- Reject unknown or extra JSON fields only if they prevent parsing the minimal + shape. Otherwise ignore them so the module can tolerate compatible Seriatim + additions. +- Return module-specific errors prefixed with useful Seriatim context, for + example `seriatim input: segment "s1" text must not be empty`. +- Keep examples free of private transcript content. Use synthetic fixture text. -### Required Tests - -- metadata lookup succeeds for `test.generic`; -- unknown lookup returns false; -- `MustLookupMetadata` panics for unknown prompt ID; -- registered metadata is sorted; -- render returns system/user text and metadata; -- missing template data returns an error; -- hardening text is available and appears when referenced; -- diagnostics map includes metadata but no rendered prompt text. - -### Validation - -Run: - -```sh -gofmt -w internal/framework/prompt -go test ./internal/framework/prompt -go test ./... -``` - -## Stage 4: Diagnostics Run Directory +## Stage 1: Seriatim Package Skeleton And External Model ### Goal -Adapt Audita's diagnostics run-directory pattern using extraction-oriented -artifact names and Notarius types. +Create the Seriatim input module package, define the module-local JSON model, +and add registry-facing module metadata without changing framework contracts. ### Files To Add -- `internal/core/diagnostics/artifacts.go` -- `internal/core/diagnostics/run_dir.go` -- `internal/core/diagnostics/run_dir_test.go` -- `internal/core/diagnostics/artifacts_test.go` +- `internal/modules/input/seriatim/adapter.go` +- `internal/modules/input/seriatim/model.go` +- `internal/modules/input/seriatim/metadata.go` +- `internal/modules/input/seriatim/registry_test.go` ### Required API -Add artifact names: - -```go -const ( - ArtifactInvocationMetadata = "invocation.json" - ArtifactEffectiveConfig = "effective-config.json" - ArtifactResolvedPipeline = "resolved-pipeline.json" - ArtifactSourceDocument = "source-document.json" - ArtifactRunManifest = "run-manifest.json" - ArtifactRunReport = "run-report.json" - ArtifactWarnings = "warnings.json" - ArtifactErrorLog = "error.log" -) -``` - Add: ```go -type RunDirectory struct { /* unexported fields */ } +package seriatim + +const Key = "seriatim" -type RetentionMode string const ( - RetentionAuto RetentionMode = "auto" - RetentionAlways RetentionMode = "always" - RetentionNever RetentionMode = "never" + DocumentKind = "transcript" + UnitKind = "transcript_segment" + Format = "application/vnd.seriatim.minimal+json" ) -type RetentionDecisionInput struct { - RetentionMode RetentionMode - RunSucceeded bool - HasWarnings bool -} +const ( + MetadataSpeaker = "speaker" + MetadataStart = "start" + MetadataEnd = "end" +) -func ShouldRetainRunDirectory(input RetentionDecisionInput) bool -func NewRunDirectory(workDir string, retention RetentionMode) (*RunDirectory, error) -func (r *RunDirectory) Path() string -func (r *RunDirectory) RunID() string -func (r *RunDirectory) WriteInvocationMetadata(metadata InvocationMetadata) error -func (r *RunDirectory) WriteRedactedEffectiveConfig(payload any) error -func (r *RunDirectory) WriteResolvedPipeline(payload any) error -func (r *RunDirectory) WriteSourceDocument(payload any) error -func (r *RunDirectory) WriteRunManifest(manifest artifacts.RunManifest) error -func (r *RunDirectory) WriteRunReport(payload any) error -func (r *RunDirectory) WriteWarnings(warnings []contracts.Warning) error -func (r *RunDirectory) WriteErrorLog(errorMessage string) error -func (r *RunDirectory) WriteJSONArtifact(name string, payload any) error -func (r *RunDirectory) ApplyRetention(input RetentionDecisionInput) error +type Adapter struct{} + +func New() *Adapter +func (a *Adapter) Key() string +func (a *Adapter) Parse(ctx context.Context, req contracts.ParseRequest) (*source.SourceDocument, error) +func ModuleSpec() pipeline.ModuleSpec +func Register(registry *pipeline.InputAdapterRegistry) error ``` -Add: +Add typed metadata helpers: ```go -type InvocationMetadata struct { - Operation string `json:"operation"` - PipelineID string `json:"pipeline_id,omitempty"` - PipelineDigest string `json:"pipeline_digest,omitempty"` - InputPath string `json:"input_path,omitempty"` - ConfigPath string `json:"config_path,omitempty"` - ConfigSource string `json:"config_source,omitempty"` - OnlyLanes []string `json:"only_lanes,omitempty"` - RunID string `json:"run_id"` - StartedAt time.Time `json:"started_at"` +func Speaker(unit source.SourceUnit) (string, bool) +func Start(unit source.SourceUnit) (json.Number, bool) +func End(unit source.SourceUnit) (json.Number, bool) +``` + +### Seriatim JSON Shape + +Define module-local structs for the minimal external shape: + +```go +type transcript struct { + Metadata map[string]any `json:"metadata"` + Segments []segment `json:"segments"` +} + +type segment struct { + ID string `json:"id"` + Start json.Number `json:"start"` + End json.Number `json:"end"` + Speaker string `json:"speaker"` + Text string `json:"text"` } ``` +Use an internal decode helper based on `json.NewDecoder(bytes.NewReader(raw))` +and `UseNumber`. + ### Required Behavior -- Default work directory is `/tmp/notarius`. -- Create work directory with `0o755`. -- Create unique run directories named `run-`. -- Write JSON artifacts with `json.MarshalIndent` and a trailing newline. -- Write error logs as plain text with a trailing newline. -- Reject artifact names that are empty, absolute paths, contain path - separators, or resolve outside the run directory. -- Retention behavior: - - failed runs are always retained; - - `always` retains successful runs; - - `never` removes successful runs; - - `auto` retains successful runs only when warnings exist. -- Do not copy Audita names such as source transcript, normalized transcript, - correction ledger, proposal, or replacement. +- `New()` returns a non-nil adapter. +- `Adapter.Key()` returns `Key`. +- `ModuleSpec()` returns defensive slices and the capability set listed in the + global decisions. +- `Register()` calls `InputAdapterRegistry.RegisterWithSpec(ModuleSpec(), ...)`. +- `Register(nil)` returns an error from the registry path rather than panicking. +- Keep external JSON structs unexported. ### Required Tests -- run directory creation and run ID shape; -- default work directory behavior using a temporary working directory where - possible; -- JSON artifact writes are indented and newline-terminated; -- invocation metadata fills missing run ID and start time; -- error log write; -- artifact path rejection for unsafe names; -- retention decisions for failed/successful runs across modes; -- `ApplyRetention` removes only the run directory for removable cases. +- `New()` returns an adapter whose key is `seriatim`. +- `ModuleSpec()` uses input stage and declares the required provided + capabilities. +- `Register()` makes the adapter buildable from an `InputAdapterRegistry`. +- Registry lookup returns the Seriatim module spec. ### Validation Run: ```sh -gofmt -w internal/core/diagnostics -go test ./internal/core/diagnostics +gofmt -w internal/modules/input/seriatim +go test ./internal/modules/input/seriatim go test ./... ``` -## Stage 5: Config Model, YAML Loading, Redaction, And Environment Overrides +## Stage 2: Parse, Validate, And Map To SourceDocument ### Goal -Add Notarius config structs and loading primitives for named pipeline profiles, -LLM profiles, operational settings, YAML files, redaction, and environment -overrides. +Implement the Seriatim parser and mapper from minimal Seriatim JSON into the +generic source model. -### Files To Add +### Files To Add Or Update -- `internal/core/config/config.go` -- `internal/core/config/file_config.go` -- `internal/core/config/env.go` -- `internal/core/config/redaction.go` -- `internal/core/config/config_test.go` -- `internal/core/config/file_config_test.go` -- `internal/core/config/env_test.go` -- `internal/core/config/redaction_test.go` - -### Dependency Change - -Add: - -```sh -go get gopkg.in/yaml.v3@v3.0.1 -``` - -Commit the resulting `go.mod` and `go.sum` changes. - -### Required API - -Add core config types: - -```go -const SupportedFileConfigVersion = 1 - -type Config struct { - LLMProfiles map[string]LLMProfile - Pipelines map[string]pipeline.PipelineProfile - Concurrency ConcurrencyConfig - Diagnostics DiagnosticsConfig -} - -type LLMProfile struct { - Provider string - BaseURL string - Model string - APIKey string - APIKeyEnv string - TimeoutSeconds int - MaxRetries int - MaxConcurrency int -} - -type ConcurrencyConfig struct { - TotalLLM int -} - -type DiagnosticsConfig struct { - WorkDir string - Retention diagnostics.RetentionMode -} - -func Default() Config -func (c Config) Redacted() Config -func (c *Config) ApplyEnvOverrides() error -func LoadFromEnv() (Config, error) -``` - -Add file config loading: - -```go -type FileConfig struct { /* YAML-facing fields */ } -func LoadFileConfig(path string) (FileConfig, error) -func ParseFileConfigYAML(data []byte) (FileConfig, error) -func (c *Config) ApplyFileConfig(fileCfg FileConfig) error -``` - -Add unexported lookup seams for tests: - -```go -func (c *Config) applyFileConfigWithLookup(fileCfg FileConfig, lookup func(string) (string, bool)) error -func (c *Config) applyEnvOverridesWithLookup(lookup func(string) (string, bool)) error -``` - -### YAML Shape - -Support this shape: - -```yaml -version: 1 -llm_profiles: - default: - provider: openai-compatible - base_url: https://example.invalid/v1 - model: test-model - api_key_env: NOTARIUS_TEST_API_KEY - timeout: 600s - max_retries: 3 - max_concurrency: 1 -pipelines: - example: - input: fake/input - chunk: generic - artifacts: - events: - extract: - module: fake/extract - llm_profile: default - options: - temperature: 0 - merge: appendorder - normalize: noop - validators: - - fake/validator - output: json -concurrency: - total_llm: 1 -diagnostics: - work_dir: /tmp/notarius - retention: auto -``` - -Module bindings must support both string shorthand and object form: - -```yaml -extract: fake/extract -extract: - module: fake/extract - llm_profile: fast - options: - key: value -``` - -Validator bindings must support a sequence of string or object bindings. - -### Defaults - -`Default()` must provide: - -- `LLMProfiles["default"]` with provider `openai-compatible`, - `TimeoutSeconds: 600`, `MaxRetries: 3`, and `MaxConcurrency: 1`; -- no built-in pipeline profiles; -- `Concurrency.TotalLLM: 1`; -- `Diagnostics.WorkDir: "/tmp/notarius"`; -- `Diagnostics.Retention: diagnostics.RetentionAuto`. - -Config validation will later decide whether a selected LLM profile is complete -enough to build a real client. This stage should not require default `BaseURL` -or `Model` values. - -### Environment Variables - -Support these environment overrides: - -- `NOTARIUS_LLM_DEFAULT_API_KEY` -- `NOTARIUS_LLM_DEFAULT_BASE_URL` -- `NOTARIUS_LLM_DEFAULT_MODEL` -- `NOTARIUS_LLM_DEFAULT_TIMEOUT_SECONDS` -- `NOTARIUS_LLM_DEFAULT_MAX_RETRIES` -- `NOTARIUS_LLM_DEFAULT_MAX_CONCURRENCY` -- `NOTARIUS_TOTAL_LLM_CONCURRENCY` -- `NOTARIUS_WORK_DIR` -- `NOTARIUS_DIAGNOSTICS_RETENTION` - -Do not support environment variables that structurally change pipeline module -selection. - -### Required Behavior - -- YAML parsing must use `KnownFields(true)`. -- Config version is required and must equal `SupportedFileConfigVersion`. -- Duration fields accept integer seconds or Go duration strings resolving to - whole seconds. -- `api_key_env` must be a valid environment-variable name and must resolve via - the lookup function used by tests. -- Redaction must replace all API key values with `[REDACTED]` while preserving - non-secret fields. -- Applying a file config must merge into defaults, not replace unspecified - sections with zero values. -- Environment overrides apply after file config. -- Object-form `options` must preserve YAML scalar/list/map values as - `map[string]any` suitable for `pipeline.ModuleBinding.Options`. - -### Required Tests - -- default values; -- parse minimal valid config; -- reject unknown YAML fields; -- reject missing/unsupported version; -- string and object module-binding forms; -- validator binding list with mixed shorthand/object forms; -- duration parsing; -- API key env resolution and invalid env-name rejection; -- file config merges with defaults; -- environment overrides operational and LLM values; -- environment variables cannot change pipeline module wiring; -- redacted config removes API key values. - -### Validation - -Run: - -```sh -gofmt -w internal/core/config -go test ./internal/core/config -go test ./... -``` - -## Stage 6: Config Validation And Pipeline Resolution - -### Goal - -Validate named pipeline profiles against registered module metadata, LLM -profiles, lane selections, and resolved pipeline digest behavior. - -### Files To Add - -- `internal/core/config/validation.go` -- `internal/core/config/effective_config.go` -- `internal/core/config/validation_test.go` -- `internal/core/config/effective_config_test.go` - -### Required API - -Add: - -```go -type ResolveInput struct { - PipelineID string - Only []string - Catalog pipeline.ModuleCatalog -} - -type EffectiveConfig struct { - Config Config - PipelineID string - Only []string - ResolvedPipeline pipeline.ResolvedPipeline -} - -func (c Config) Validate() error -func (c Config) Resolve(input ResolveInput) (EffectiveConfig, error) -func (c Config) LLMProfile(id string) (LLMProfile, bool) -func (c Config) OpenAICompatibleClientConfig(profileID string) (llm.OpenAICompatibleClientConfig, error) -``` - -### Required Behavior - -`Validate()` must check config-internal rules that do not need a module catalog: - -- LLM profile IDs trim to non-empty unique IDs; -- pipeline IDs trim to non-empty unique IDs; -- LLM profile provider must be empty or `openai-compatible`; -- LLM profile numeric values must be positive where applicable; -- diagnostics work dir must not be empty; -- diagnostics retention must be one of `auto`, `always`, or `never`; -- `TotalLLM` must be greater than zero; -- every LLM profile referenced by a pipeline binding must exist. - -`Resolve(input)` must: - -- call `Validate()`; -- trim `PipelineID`; -- reject empty or unknown pipeline IDs; -- pass `Only` through to `pipeline.ResolvePipeline`; -- return the resolved pipeline and digest; -- surface errors that name the pipeline and lane where possible. - -`OpenAICompatibleClientConfig(profileID)` must: - -- reject unknown profile IDs; -- reject unsupported providers; -- require base URL and model; -- pass API key, timeout, and max retries to the LLM client config. - -### Required Tests - -- `Validate` success for valid config; -- unknown LLM profile referenced by binding; -- invalid provider; -- invalid numeric fields; -- invalid diagnostics retention; -- empty/unknown selected pipeline ID; -- lane filtering success and failure; -- unknown module key through module catalog; -- missing capability through module catalog; -- resolved pipeline digest changes when effective config changes; -- `OpenAICompatibleClientConfig` rejects incomplete default profile and succeeds - when base URL/model are set. - -Use fake module registries and fake module specs. Do not add real modules. - -### Validation - -Run: - -```sh -gofmt -w internal/core/config -go test ./internal/core/config -go test ./... -``` - -## Stage 7: CLI Config Validation And Pipeline Listing - -### Goal - -Add minimal CLI support for config discovery/validation and pipeline profile -listing without adding run execution. - -### Files To Update - -- `internal/cli/run.go` -- `internal/cli/run_test.go` - -### Required CLI - -Support: - -```sh -notarius config validate --config path/to/config.yml -notarius config validate --config path/to/config.yml --pipeline pipeline-id -notarius config validate --config path/to/config.yml --pipeline pipeline-id --only lane-a,lane-b -notarius pipelines list --config path/to/config.yml -notarius pipelines list --config path/to/config.yml --json -``` - -Also support the default config search path from architecture policy: - -1. `--config` path, when provided; -2. `NOTARIUS_CONFIG`, when set; -3. `/usr/local/etc/notarius/config.yml`, when present. - -Do not add `notarius run` in this checkpoint. - -### Required Implementation Shape - -The CLI needs module metadata to validate structural pipeline config. Add an -internal test seam rather than hard-coding fake modules: - -```go -type Options struct { - Catalog pipeline.ModuleCatalog - LookupEnv func(string) (string, bool) -} - -func RunWithOptions(args []string, stdout, stderr io.Writer, opts Options) int -``` - -Keep the existing `Run(args, stdout, stderr)` as a wrapper that passes default -options. Default production options may contain empty registries until real -modules are registered by later checkpoints. - -### Required Behavior - -- `config validate` exits `0` and writes a concise success message on valid - config. -- `config validate` exits `1` on config load/validation errors and writes the - error to stderr. -- `config validate --pipeline` resolves that pipeline against the supplied - catalog. -- `--only` is valid only when `--pipeline` is present. -- `pipelines list` exits `0` and lists pipeline IDs sorted alphabetically. -- `pipelines list --json` emits stable JSON, for example: - -```json -{"pipelines":["a","b"]} -``` - -- Unknown commands and invalid flags exit `2`. -- Usage text must include the new commands. -- CLI flags must not allow ad hoc structural module overrides such as - `--extractor`, `--chunker`, `--input`, `--merge`, or `--normalize`. - -### Required Tests - -- existing help/unknown-command tests still pass; -- `config validate` success with fake catalog; -- `config validate` reports config parse errors; -- `config validate --pipeline --only` success and invalid lane failure; -- `--only` without `--pipeline` fails; -- `pipelines list` sorted text output; -- `pipelines list --json` stable JSON output; -- `NOTARIUS_CONFIG` is used when `--config` is absent; -- missing config path produces actionable error; -- no ad hoc structural flags are accepted. - -### Validation - -Run: - -```sh -gofmt -w internal/cli -go test ./internal/cli -go test ./... -go build ./cmd/notarius -rm -f ./notarius -``` - -## Stage 8: Final Checkpoint 4 Review Pass - -### Goal - -Remove accidental Audita coupling, confirm scope boundaries, and verify all -runtime pieces compose. - -### Required Review - -Check: - -- no correction proposal, replacement policy, transcript mutation, correction - ledger, or Audita module behavior was copied; -- no package names or artifact names mention corrections, proposals, - replacements, or transcripts unless they refer to Audita source files in - comments inside tests; -- no real D&D prompts or schemas were added; -- no Seriatim input adapter was added; -- no `notarius run` behavior was added; -- provider-specific HTTP structs stay unexported inside `internal/framework/llm`; -- response schemas live inside `internal/framework/llm`, not a new - `responseschema` package; -- prompt assets live inside `internal/framework/prompt`; -- config is centered on named pipeline profiles; -- module selection comes from config, not ad hoc CLI flags; -- diagnostics artifact names are extraction-oriented; -- secrets are redacted from effective config and surfaced provider errors. +- `internal/modules/input/seriatim/adapter.go` +- `internal/modules/input/seriatim/model.go` +- `internal/modules/input/seriatim/adapter_test.go` +- `internal/modules/input/seriatim/testdata/valid_minimal.json` +- `internal/modules/input/seriatim/testdata/duplicate_segment_id.json` ### Required Validation +Reject: + +- nil or canceled context before parsing; +- empty raw input; +- malformed JSON; +- valid JSON with trailing non-whitespace data; +- missing, null, or non-object top-level `metadata`; +- missing, null, empty, or non-array top-level `segments`; +- segment IDs that are empty after trimming; +- segment IDs with leading or trailing whitespace; +- duplicate segment IDs; +- missing or empty `speaker`; +- missing, empty, non-numeric, negative, or non-finite `start`; +- missing, empty, non-numeric, negative, or non-finite `end`; +- segments where `end < start`; +- missing or empty `text`. + +The parser may preserve leading and trailing whitespace in segment text as long +as the text is not empty after trimming. + +### Mapping Rules + +- `segment.id` becomes `SourceUnit.ID`. +- `segment.text` becomes `SourceUnit.Text`. +- `speaker`, `start`, and `end` become unit metadata under the exact keys + defined in stage 1. +- The document metadata is a shallow copy of top-level Seriatim metadata. +- The document digest is based on raw input bytes, not normalized JSON. +- Call `source.ValidateDocument` before returning the document and wrap any + validation failure with Seriatim context. + +### Required Tests + +Add tests for: + +- valid minimal transcript parses to a source document with expected ID, kind, + format, digest, units, and metadata; +- `ParseRequest.SourceID` overrides metadata-derived IDs; +- fallback document ID is deterministic and has prefix `seriatim:`; +- malformed JSON returns an actionable Seriatim parse error; +- missing metadata is rejected; +- missing or empty segments is rejected; +- duplicate segment IDs are rejected; +- empty segment text is rejected; +- missing speaker is rejected; +- invalid timestamp values are rejected; +- `end < start` is rejected; +- typed metadata helpers return the expected speaker and timestamp values; +- a `source.SourceRef` using the first and last generated unit IDs validates + with `source.ValidateRef`. + +### Validation + Run: ```sh -gofmt -w internal +gofmt -w internal/modules/input/seriatim +go test ./internal/modules/input/seriatim go test ./... -go vet ./... -go build ./cmd/notarius -rm -f ./notarius -git status --short ``` -The implementation response should summarize: +## Stage 3: Pipeline Resolution And Config Compatibility -- files/packages added; -- tests run; -- any deviations from this plan and why. +### Goal + +Prove the Seriatim input module participates in pipeline-profile resolution and +capability validation through existing registries and config loading. + +### Files To Add Or Update + +- `internal/modules/input/seriatim/config_test.go` +- `internal/modules/input/seriatim/testdata/pipeline.yml` + +### Required Test Catalog + +Build a test-only module catalog with: + +- Seriatim input registered through `seriatim.Register`; +- a fake chunker requiring `source.transcript` and providing `chunks`; +- a fake extractor requiring `chunks`, `transcript.speaker`, and + `transcript.timestamps`, and providing `fake.artifacts`; +- `pipeline.AppendOrderMerger` registered as `appendorder`, requiring + `fake.artifacts`; +- `pipeline.NoopNormalizer` registered as `noop`; +- a fake `json` output encoder registered as output stage. + +Do not add real extract, chunk, normalize, or output modules for this checkpoint. + +### YAML Fixture + +Use a synthetic pipeline fixture shaped like: + +```yaml +version: 1 +pipelines: + seriatim-fixture: + input: seriatim + chunk: fake/chunk + artifacts: + events: + extract: fake/extract + merge: appendorder + normalize: noop + output: json +``` + +The default LLM profile supplied by `config.Default()` is sufficient. Do not +add real provider settings to this fixture. + +### Required Tests + +- `config.ParseFileConfigYAML` and `Config.ApplyFileConfig` load the fixture. +- `Config.Resolve` succeeds with pipeline ID `seriatim-fixture` and the + test-only catalog. +- The resolved pipeline input module is `seriatim`. +- The resolved pipeline digest is non-empty and stable across repeated + resolution. +- Removing `transcript.timestamps` from the Seriatim module spec in the + test-only catalog causes resolution to fail with a missing capability error. +- Selecting an unknown `--only` lane still fails through existing resolution + behavior. + +### Validation + +Run: + +```sh +gofmt -w internal/modules/input/seriatim +go test ./internal/modules/input/seriatim +go test ./internal/core/config +go test ./... +``` + +## Stage 4: Runner Integration With Fake Downstream Stages + +### Goal + +Prove real Seriatim input can flow through the existing runner into fake +downstream stages while preserving source-unit IDs and metadata. + +### Files To Add Or Update + +- `internal/modules/input/seriatim/runner_test.go` + +### Required Behavior + +Use the same Seriatim fixture from stage 2 and a resolved pipeline from stage 3. +Register fake downstream stages only inside the test. + +The fake extractor should: + +- inspect the received `SourceDocument` and `SourceChunk`; +- assert that unit IDs match Seriatim segment IDs; +- assert that speaker and timestamp metadata are present; +- return one generic artifact candidate with a source reference pointing at + existing Seriatim-derived unit IDs. + +The test should then assert: + +- `Runner.Run` succeeds; +- the manifest records input module `seriatim`; +- the manifest source digest equals the parsed document digest; +- approved artifacts preserve valid source references; +- no transcript-specific type has been added outside the module. + +### Required Tests + +- successful runner execution from Seriatim JSON through fake chunk, extract, + merge, normalize, and output stages; +- runner failure when the Seriatim adapter returns an invalid source document, + using a malformed fixture or test input; +- validation of the fake candidate's source reference with + `source.ValidateRef`. + +### Validation + +Run: + +```sh +gofmt -w internal/modules/input/seriatim +go test ./internal/modules/input/seriatim +go test ./internal/framework/pipeline +go test ./... +``` + +## Stage 5: Documentation And Final Verification + +### Goal + +Document the implemented Seriatim integration contract once the module exists, +without describing unimplemented D&D extraction or run-command behavior. + +### Files To Add Or Update + +- `docs/integrations/seriatim.md` +- `docs/roadmap/5-seriatim-input-module.md` + +### Required Documentation + +Create `docs/integrations/seriatim.md` as implemented-behavior documentation +with: + +- accepted minimal JSON shape; +- required fields and validation rules; +- mapping from Seriatim fields to `SourceDocument` and `SourceUnit`; +- metadata key conventions for `speaker`, `start`, and `end`; +- capability strings declared by the module; +- note that broader Seriatim schema variants are not yet supported. + +Update `docs/roadmap/5-seriatim-input-module.md` only if implementation +reveals a real scope or policy correction. Keep future D&D extraction behavior +out of the integration doc. + +### Final Validation + +Run: + +```sh +gofmt -w internal/modules/input/seriatim +go test ./... +go build ./cmd/notarius +rm -f ./notarius +``` + +### Done Criteria + +- `go test ./...` passes. +- `go build ./cmd/notarius` passes. +- Seriatim minimal transcript JSON maps into `SourceDocument`. +- Unit IDs are stable and validate in source references. +- Transcript fields do not appear in core runner contracts. +- The input module is selectable through the input registry and pipeline-profile + resolution. +- The input module declares transcript-oriented flat capabilities for pipeline + validation. +- Tests prove transcript-specific assumptions are isolated to + `internal/modules/input/seriatim`. ## Open Questions -None. The plan intentionally defers real extractor prompts/schemas, concrete -input modules, real built-in module catalogs, and `notarius run` execution to -later checkpoints. +None. This plan chooses the checkpoint-5 behavior needed to implement the +feature without requiring additional product decisions.