Add a staged implementation plan for the initial framework and module composition architecture
This commit is contained in:
@@ -32,65 +32,26 @@ Out of scope:
|
|||||||
- diagnostics run directory;
|
- diagnostics run directory;
|
||||||
- real D&D artifact schemas.
|
- real D&D artifact schemas.
|
||||||
|
|
||||||
## Proposed Stages
|
## Target End State
|
||||||
|
|
||||||
### Stage 1: Input Adapter Registry
|
The repository should contain a minimal framework composition layer:
|
||||||
|
|
||||||
Add a registry for `InputAdapter` constructors or instances.
|
- `internal/framework/inputregistry` registers and builds input adapter
|
||||||
|
constructors by stable key.
|
||||||
|
- `internal/framework/extractorregistry` registers and builds extractor
|
||||||
|
constructors by stable key.
|
||||||
|
- `internal/framework/validators` provides shared validator decision helpers and
|
||||||
|
cardinality checks.
|
||||||
|
- `internal/framework/runner` executes configured extractors against a
|
||||||
|
`SourceDocument`, applies validator chains, and returns approved and rejected
|
||||||
|
artifacts.
|
||||||
|
|
||||||
The registry should:
|
The runner should operate on already parsed source documents in this checkpoint.
|
||||||
|
Raw input parsing and concrete input adapter behavior remain deferred to the
|
||||||
|
Seriatim adapter checkpoint.
|
||||||
|
|
||||||
- reject empty keys;
|
Implementation staging belongs in
|
||||||
- reject duplicate registrations;
|
[`implementation.md`](implementation.md).
|
||||||
- return clear errors for unknown keys;
|
|
||||||
- avoid importing concrete adapter packages from core contracts.
|
|
||||||
|
|
||||||
### Stage 2: Extractor Registry
|
|
||||||
|
|
||||||
Add a registry for extractor constructors or instances.
|
|
||||||
|
|
||||||
The registry should:
|
|
||||||
|
|
||||||
- support stable extractor keys;
|
|
||||||
- support repeated extractor instances if needed later;
|
|
||||||
- return clear errors for unknown keys;
|
|
||||||
- avoid domain-specific logic.
|
|
||||||
|
|
||||||
### Stage 3: Validator Decisions
|
|
||||||
|
|
||||||
Add validator decision types and cardinality checks.
|
|
||||||
|
|
||||||
Each validator should return exactly one decision for each candidate artifact it
|
|
||||||
receives.
|
|
||||||
|
|
||||||
Decision fields should include:
|
|
||||||
|
|
||||||
- candidate index;
|
|
||||||
- approved flag;
|
|
||||||
- reason code;
|
|
||||||
- message;
|
|
||||||
- optional diagnostics path.
|
|
||||||
|
|
||||||
### Stage 4: Minimal Runner
|
|
||||||
|
|
||||||
Add a runner that can:
|
|
||||||
|
|
||||||
1. receive a `SourceDocument`;
|
|
||||||
2. execute configured extractors;
|
|
||||||
3. validate candidate artifacts;
|
|
||||||
4. return approved and rejected artifacts.
|
|
||||||
|
|
||||||
Keep source chunking optional or stubbed at this checkpoint. The runner may
|
|
||||||
operate on whole documents only until source-unit chunking is added later.
|
|
||||||
|
|
||||||
### Stage 5: Runner Tests With Fakes
|
|
||||||
|
|
||||||
Add tests with fake components that prove:
|
|
||||||
|
|
||||||
- registered fake extractors run in configured order;
|
|
||||||
- validators filter candidates deterministically;
|
|
||||||
- decision cardinality failures are surfaced;
|
|
||||||
- approved and rejected artifacts are returned in stable order.
|
|
||||||
|
|
||||||
## Done Criteria
|
## Done Criteria
|
||||||
|
|
||||||
|
|||||||
@@ -1,14 +1,15 @@
|
|||||||
# Implementation Plan: Checkpoint 1 Core Contracts And Skeleton
|
# Implementation Plan: Checkpoint 2 Framework Composition
|
||||||
|
|
||||||
## Status
|
## Status
|
||||||
|
|
||||||
This is a staged implementation plan for
|
This is a staged implementation plan for
|
||||||
[`1-core-contracts-and-skeleton.md`](1-core-contracts-and-skeleton.md). It is
|
[`2-framework-composition.md`](2-framework-composition.md). It is intended for
|
||||||
intended for an LLM coding agent to follow stage by stage.
|
an LLM coding agent to follow stage by stage.
|
||||||
|
|
||||||
This plan implements only checkpoint 1. Do not implement real input adapters,
|
This plan implements only checkpoint 2. Do not implement real input adapters,
|
||||||
real extractors, LLM provider clients, prompt assets, response schema registries,
|
real extractors, source chunking, LLM provider clients, prompt assets, response
|
||||||
diagnostics run directories, or production config loading in this pass.
|
schema registries, diagnostics run directories, production config loading, or
|
||||||
|
D&D artifact schemas in this pass.
|
||||||
|
|
||||||
## Policy Context
|
## Policy Context
|
||||||
|
|
||||||
@@ -19,481 +20,505 @@ Follow:
|
|||||||
|
|
||||||
Required boundaries:
|
Required boundaries:
|
||||||
|
|
||||||
- core packages must stay source-agnostic and domain-agnostic;
|
- framework packages must stay source-agnostic and domain-agnostic;
|
||||||
- transcript-specific concepts must not appear in core or framework contracts;
|
- input adapter registry code must not import concrete adapter packages;
|
||||||
- D&D concepts must not appear in core or framework contracts;
|
- extractor registry and runner code must not import concrete extractor
|
||||||
- source references must target generic source units;
|
packages;
|
||||||
|
- runner code should operate on `SourceDocument`, not transcript-specific
|
||||||
|
structures;
|
||||||
|
- validators should be independently testable and composable;
|
||||||
- planned behavior outside this checkpoint must remain in roadmap docs.
|
- planned behavior outside this checkpoint must remain in roadmap docs.
|
||||||
|
|
||||||
## Global Implementation Decisions
|
## Global Implementation Decisions
|
||||||
|
|
||||||
- Use module path `gitea.maximumdirect.net/eric/notarius`.
|
- Add no third-party dependencies in checkpoint 2.
|
||||||
- Use Go `1.24.0`, matching the nearby Audita project.
|
- Do not change public core source/artifact/contract types unless a stage cannot
|
||||||
- Add no third-party dependencies in checkpoint 1.
|
compile without a small compatibility adjustment.
|
||||||
- Keep CLI behavior minimal: root help/usage and unknown-command handling only.
|
- Use constructor-based registries. Constructors can close over future
|
||||||
- Use package names exactly as listed in this plan unless implementation reveals
|
dependencies without changing registry APIs.
|
||||||
a compile-time conflict.
|
- Registries normalize keys with `strings.TrimSpace`.
|
||||||
- Use `encoding/json.RawMessage` for generic artifact payloads.
|
- Registries reject empty keys, duplicate keys, nil constructors, nil built
|
||||||
- Store contract tests close to the contracts they exercise.
|
instances, and key/name mismatches.
|
||||||
- Run `gofmt` on all Go files before validation.
|
- The minimal runner receives an already parsed `*source.SourceDocument`.
|
||||||
|
- The minimal runner does not use input adapters yet; input adapter registry is
|
||||||
|
tested directly.
|
||||||
|
- The runner assigns global candidate indices in extractor execution order.
|
||||||
|
- If a candidate supplies non-empty extractor metadata that conflicts with its
|
||||||
|
owning extractor, the runner returns an error.
|
||||||
|
- If a candidate leaves extractor metadata empty, the runner fills
|
||||||
|
`ExtractorKey`, `ArtifactType`, and `SchemaVersion` from the owning extractor.
|
||||||
|
- Validator chains run in the order returned by `Extractor.Validators()`.
|
||||||
|
- A validator result must use the same `ValidatorName` as `Validator.Name()`.
|
||||||
|
- Each validator must return exactly one decision for each currently eligible
|
||||||
|
candidate.
|
||||||
|
- If an extractor has no validators, its candidates are approved.
|
||||||
|
- On setup, extraction, or validation errors, return the current partial
|
||||||
|
`RunOutput` with a wrapped error.
|
||||||
|
- Run `gofmt` on all touched Go files before validation.
|
||||||
|
|
||||||
## Stage 1: Bootstrap Go Project And CLI Shell
|
## Stage 1: Input Adapter Registry
|
||||||
|
|
||||||
### Goal
|
### Goal
|
||||||
|
|
||||||
Create a compileable Go application shell with a minimal CLI entrypoint.
|
Add a constructor-based registry for `contracts.InputAdapter`.
|
||||||
|
|
||||||
### Files To Add
|
### Files To Add
|
||||||
|
|
||||||
- `go.mod`
|
- `internal/framework/inputregistry/registry.go`
|
||||||
- `cmd/notarius/main.go`
|
- `internal/framework/inputregistry/registry_test.go`
|
||||||
- `internal/cli/run.go`
|
|
||||||
- `internal/cli/run_test.go`
|
|
||||||
|
|
||||||
### Required Implementation
|
### Required API
|
||||||
|
|
||||||
Create `go.mod`:
|
Create package `inputregistry`.
|
||||||
|
|
||||||
```go
|
|
||||||
module gitea.maximumdirect.net/eric/notarius
|
|
||||||
|
|
||||||
go 1.24.0
|
|
||||||
```
|
|
||||||
|
|
||||||
Create `cmd/notarius/main.go`:
|
|
||||||
|
|
||||||
- import `os`;
|
|
||||||
- import `gitea.maximumdirect.net/eric/notarius/internal/cli`;
|
|
||||||
- call `os.Exit(cli.Run(os.Args[1:], os.Stdout, os.Stderr))`.
|
|
||||||
|
|
||||||
Create `internal/cli.Run(args []string, stdout, stderr io.Writer) int`.
|
|
||||||
|
|
||||||
Initial behavior:
|
|
||||||
|
|
||||||
- `notarius`, `notarius help`, `notarius --help`, and `notarius -h` write root
|
|
||||||
usage to `stdout` and return `0`;
|
|
||||||
- unknown commands write `notarius: unknown command "<cmd>"` plus usage to
|
|
||||||
`stderr` and return `2`;
|
|
||||||
- no `extract` command is implemented in checkpoint 1.
|
|
||||||
|
|
||||||
Root usage should be short and should not document unimplemented behavior as if
|
|
||||||
it exists. Acceptable text:
|
|
||||||
|
|
||||||
```text
|
|
||||||
Usage:
|
|
||||||
notarius help
|
|
||||||
```
|
|
||||||
|
|
||||||
### Required Tests
|
|
||||||
|
|
||||||
In `internal/cli/run_test.go`, cover:
|
|
||||||
|
|
||||||
- no args returns `0` and writes usage to stdout;
|
|
||||||
- `help`, `--help`, and `-h` return `0`;
|
|
||||||
- unknown command returns `2`, writes to stderr, and does not write meaningful
|
|
||||||
stdout.
|
|
||||||
|
|
||||||
### Validation
|
|
||||||
|
|
||||||
Run:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
gofmt -w cmd internal
|
|
||||||
go test ./...
|
|
||||||
go build ./cmd/notarius
|
|
||||||
```
|
|
||||||
|
|
||||||
## Stage 2: Core Source Model
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Add the generic source-document model and validation helpers.
|
|
||||||
|
|
||||||
### Files To Add
|
|
||||||
|
|
||||||
- `internal/core/source/source.go`
|
|
||||||
- `internal/core/source/validation.go`
|
|
||||||
- `internal/core/source/source_test.go`
|
|
||||||
|
|
||||||
### Required Types
|
|
||||||
|
|
||||||
Create package `source`.
|
|
||||||
|
|
||||||
Define:
|
Define:
|
||||||
|
|
||||||
```go
|
```go
|
||||||
type SourceDocument struct {
|
type Constructor func() (contracts.InputAdapter, error)
|
||||||
ID string `json:"id"`
|
|
||||||
Kind string `json:"kind"`
|
type Registry struct {
|
||||||
Format string `json:"format"`
|
// unexported fields
|
||||||
Digest string `json:"digest"`
|
|
||||||
Units []SourceUnit `json:"units"`
|
|
||||||
Metadata map[string]any `json:"metadata,omitempty"`
|
|
||||||
}
|
}
|
||||||
|
|
||||||
type SourceUnit struct {
|
func New() *Registry
|
||||||
ID string `json:"id"`
|
func (r *Registry) Register(key string, constructor Constructor) error
|
||||||
Kind string `json:"kind"`
|
func (r *Registry) Build(key string) (contracts.InputAdapter, error)
|
||||||
Text string `json:"text"`
|
func (r *Registry) RegisteredKeys() []string
|
||||||
Metadata map[string]any `json:"metadata,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type SourceRef struct {
|
|
||||||
SourceID string `json:"source_id"`
|
|
||||||
StartUnitID string `json:"start_unit_id"`
|
|
||||||
EndUnitID string `json:"end_unit_id"`
|
|
||||||
}
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### Required Helpers
|
### Required Behavior
|
||||||
|
|
||||||
Implement:
|
`Register` must:
|
||||||
|
|
||||||
```go
|
- return an error if the registry is nil;
|
||||||
func ValidateDocument(doc *SourceDocument) error
|
- trim `key`;
|
||||||
func ValidateRef(doc *SourceDocument, ref SourceRef) error
|
- reject empty keys;
|
||||||
func UnitIndex(doc *SourceDocument, unitID string) (int, bool)
|
- reject nil constructors;
|
||||||
```
|
- reject duplicate keys;
|
||||||
|
- store constructors by normalized key.
|
||||||
|
|
||||||
Validation rules:
|
`Build` must:
|
||||||
|
|
||||||
- document must be non-nil;
|
- return an error if the registry is nil;
|
||||||
- document `ID`, `Kind`, `Format`, and `Digest` must be non-empty after
|
- trim `key`;
|
||||||
trimming whitespace;
|
- reject empty keys;
|
||||||
- document must contain at least one unit;
|
- return a clear unknown-key error for unregistered keys;
|
||||||
- each unit `ID`, `Kind`, and `Text` must be non-empty after trimming
|
- call the constructor;
|
||||||
whitespace;
|
- wrap constructor errors with key context;
|
||||||
- unit IDs must be unique within a document;
|
- reject nil adapters;
|
||||||
- `ValidateRef` requires non-empty `SourceID`, `StartUnitID`, and `EndUnitID`;
|
- reject adapters whose `Key()` does not match the normalized key.
|
||||||
- `ValidateRef` requires `ref.SourceID == doc.ID`;
|
|
||||||
- start and end unit IDs must exist;
|
|
||||||
- start unit index must be less than or equal to end unit index.
|
|
||||||
|
|
||||||
Do not add transcript-specific fields or helpers.
|
`RegisteredKeys` must:
|
||||||
|
|
||||||
### Error Style
|
- return nil for a nil registry;
|
||||||
|
- return registered keys in sorted order;
|
||||||
Return ordinary Go errors with actionable field context, such as:
|
- return a copy that callers cannot mutate to affect registry state.
|
||||||
|
|
||||||
- `source document id must not be empty`;
|
|
||||||
- `source unit[2].id must not be empty`;
|
|
||||||
- `source unit id "u1" is duplicated`;
|
|
||||||
- `source ref start_unit_id "u9" was not found`.
|
|
||||||
|
|
||||||
### Required Tests
|
### Required Tests
|
||||||
|
|
||||||
Cover:
|
Cover:
|
||||||
|
|
||||||
- valid document;
|
- successful registration and build;
|
||||||
- nil document;
|
- key trimming;
|
||||||
- missing document fields;
|
- empty key rejection;
|
||||||
- empty units;
|
- duplicate key rejection;
|
||||||
- missing unit fields;
|
- nil constructor rejection;
|
||||||
- duplicate unit IDs;
|
- unknown key build error;
|
||||||
- valid source reference;
|
- constructor error wrapping;
|
||||||
- source ID mismatch;
|
- nil adapter rejection;
|
||||||
- missing source-ref unit ID;
|
- adapter key mismatch rejection;
|
||||||
- reversed source-ref unit order.
|
- sorted `RegisteredKeys`;
|
||||||
|
- nil registry behavior.
|
||||||
|
|
||||||
|
Use fake adapters only. Do not add concrete adapter packages.
|
||||||
|
|
||||||
### Validation
|
### Validation
|
||||||
|
|
||||||
Run:
|
Run:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
gofmt -w internal/core/source
|
gofmt -w internal/framework/inputregistry
|
||||||
go test ./internal/core/source
|
go test ./internal/framework/inputregistry
|
||||||
go test ./...
|
go test ./...
|
||||||
```
|
```
|
||||||
|
|
||||||
## Stage 3: Core Artifact Model
|
## Stage 2: Extractor Registry
|
||||||
|
|
||||||
### Goal
|
### Goal
|
||||||
|
|
||||||
Add extractor-neutral artifact and manifest types.
|
Add a constructor-based registry for `contracts.Extractor`.
|
||||||
|
|
||||||
### Files To Add
|
### Files To Add
|
||||||
|
|
||||||
- `internal/core/artifacts/artifacts.go`
|
- `internal/framework/extractorregistry/registry.go`
|
||||||
- `internal/core/artifacts/artifacts_test.go`
|
- `internal/framework/extractorregistry/registry_test.go`
|
||||||
|
|
||||||
### Required Types
|
### Required API
|
||||||
|
|
||||||
Create package `artifacts`.
|
Create package `extractorregistry`.
|
||||||
|
|
||||||
Define:
|
Define:
|
||||||
|
|
||||||
```go
|
```go
|
||||||
type Candidate struct {
|
type Constructor func() (contracts.Extractor, error)
|
||||||
Index int `json:"index"`
|
|
||||||
ExtractorKey string `json:"extractor_key"`
|
type Registry struct {
|
||||||
ArtifactType string `json:"artifact_type"`
|
// unexported fields
|
||||||
SchemaVersion string `json:"schema_version"`
|
|
||||||
Payload json.RawMessage `json:"payload"`
|
|
||||||
SourceRefs []source.SourceRef `json:"source_refs,omitempty"`
|
|
||||||
Metadata map[string]any `json:"metadata,omitempty"`
|
|
||||||
}
|
}
|
||||||
|
|
||||||
type Artifact struct {
|
func New() *Registry
|
||||||
ExtractorKey string `json:"extractor_key"`
|
func (r *Registry) Register(key string, constructor Constructor) error
|
||||||
ArtifactType string `json:"artifact_type"`
|
func (r *Registry) Build(key string) (contracts.Extractor, error)
|
||||||
SchemaVersion string `json:"schema_version"`
|
func (r *Registry) RegisteredKeys() []string
|
||||||
Payload json.RawMessage `json:"payload"`
|
```
|
||||||
SourceRefs []source.SourceRef `json:"source_refs,omitempty"`
|
|
||||||
Metadata map[string]any `json:"metadata,omitempty"`
|
### Required Behavior
|
||||||
|
|
||||||
|
Mirror `inputregistry` behavior, but for `contracts.Extractor`.
|
||||||
|
|
||||||
|
`Build` must reject extractors whose `Key()` does not match the normalized key.
|
||||||
|
It should not validate artifact type, schema version, or validator chain. Those
|
||||||
|
are extractor/runtime concerns.
|
||||||
|
|
||||||
|
### Required Tests
|
||||||
|
|
||||||
|
Cover the same cases as the input adapter registry:
|
||||||
|
|
||||||
|
- successful registration and build;
|
||||||
|
- key trimming;
|
||||||
|
- empty key rejection;
|
||||||
|
- duplicate key rejection;
|
||||||
|
- nil constructor rejection;
|
||||||
|
- unknown key build error;
|
||||||
|
- constructor error wrapping;
|
||||||
|
- nil extractor rejection;
|
||||||
|
- extractor key mismatch rejection;
|
||||||
|
- sorted `RegisteredKeys`;
|
||||||
|
- nil registry behavior.
|
||||||
|
|
||||||
|
Use fake extractors only. Do not add concrete extractor packages.
|
||||||
|
|
||||||
|
### Validation
|
||||||
|
|
||||||
|
Run:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
gofmt -w internal/framework/extractorregistry
|
||||||
|
go test ./internal/framework/extractorregistry
|
||||||
|
go test ./...
|
||||||
|
```
|
||||||
|
|
||||||
|
## Stage 3: Validator Runtime Helpers
|
||||||
|
|
||||||
|
### Goal
|
||||||
|
|
||||||
|
Add shared validator decision helpers and decision-cardinality enforcement.
|
||||||
|
|
||||||
|
### Files To Add
|
||||||
|
|
||||||
|
- `internal/framework/validators/validators.go`
|
||||||
|
- `internal/framework/validators/validators_test.go`
|
||||||
|
|
||||||
|
### Required API
|
||||||
|
|
||||||
|
Create package `validators`.
|
||||||
|
|
||||||
|
Define reason constants:
|
||||||
|
|
||||||
|
```go
|
||||||
|
const (
|
||||||
|
ReasonApproved = "approved"
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
Define helpers:
|
||||||
|
|
||||||
|
```go
|
||||||
|
func Approved(candidateIndex int) contracts.ValidationDecision
|
||||||
|
func Rejected(candidateIndex int, reasonCode string, message string) contracts.ValidationDecision
|
||||||
|
func EnforceDecisionCardinality(candidates []artifacts.Candidate, decisions []contracts.ValidationDecision) error
|
||||||
|
```
|
||||||
|
|
||||||
|
### Required Behavior
|
||||||
|
|
||||||
|
`Approved` returns an approved decision with:
|
||||||
|
|
||||||
|
- `CandidateIndex` set to the input;
|
||||||
|
- `Approved` set to true;
|
||||||
|
- `ReasonCode` set to `ReasonApproved`;
|
||||||
|
- `Message` set to `approved`.
|
||||||
|
|
||||||
|
`Rejected` returns a rejected decision with:
|
||||||
|
|
||||||
|
- `CandidateIndex` set to the input;
|
||||||
|
- `Approved` set to false;
|
||||||
|
- `ReasonCode` set to the trimmed reason code;
|
||||||
|
- `Message` set to the trimmed message.
|
||||||
|
|
||||||
|
`EnforceDecisionCardinality` must:
|
||||||
|
|
||||||
|
- require exactly one decision per candidate index;
|
||||||
|
- reject decisions for unknown candidate indices;
|
||||||
|
- reject duplicate decisions for the same candidate index;
|
||||||
|
- reject missing decisions;
|
||||||
|
- work with candidate indices, not slice positions;
|
||||||
|
- allow an empty candidate slice only when decisions are also empty.
|
||||||
|
|
||||||
|
Candidate indices are expected to be unique by the time validators run. If the
|
||||||
|
candidate slice itself contains duplicate indices, return an error.
|
||||||
|
|
||||||
|
### Required Tests
|
||||||
|
|
||||||
|
Cover:
|
||||||
|
|
||||||
|
- approved helper shape;
|
||||||
|
- rejected helper trims reason/message;
|
||||||
|
- cardinality success with non-zero candidate indices;
|
||||||
|
- empty candidates and empty decisions success;
|
||||||
|
- unknown decision index error;
|
||||||
|
- duplicate decision index error;
|
||||||
|
- missing decision index error;
|
||||||
|
- duplicate candidate index error.
|
||||||
|
|
||||||
|
### Validation
|
||||||
|
|
||||||
|
Run:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
gofmt -w internal/framework/validators
|
||||||
|
go test ./internal/framework/validators
|
||||||
|
go test ./...
|
||||||
|
```
|
||||||
|
|
||||||
|
## Stage 4: Minimal Runner Types And Constructor
|
||||||
|
|
||||||
|
### Goal
|
||||||
|
|
||||||
|
Add runner package types and constructor without implementing the full run loop
|
||||||
|
yet.
|
||||||
|
|
||||||
|
### Files To Add
|
||||||
|
|
||||||
|
- `internal/framework/runner/runner.go`
|
||||||
|
- `internal/framework/runner/runner_test.go`
|
||||||
|
|
||||||
|
### Required API
|
||||||
|
|
||||||
|
Create package `runner`.
|
||||||
|
|
||||||
|
Define:
|
||||||
|
|
||||||
|
```go
|
||||||
|
type ExtractorFactory interface {
|
||||||
|
Build(key string) (contracts.Extractor, error)
|
||||||
}
|
}
|
||||||
|
|
||||||
type RejectedArtifact struct {
|
type Runner struct {
|
||||||
Candidate Candidate `json:"candidate"`
|
// unexported fields
|
||||||
ValidatorName string `json:"validator_name"`
|
|
||||||
ReasonCode string `json:"reason_code"`
|
|
||||||
Message string `json:"message"`
|
|
||||||
}
|
}
|
||||||
|
|
||||||
type RunManifest struct {
|
func New(extractors ExtractorFactory) *Runner
|
||||||
RunID string `json:"run_id,omitempty"`
|
```
|
||||||
InputAdapter string `json:"input_adapter,omitempty"`
|
|
||||||
SourceDigests []string `json:"source_digests,omitempty"`
|
Define input/output structs:
|
||||||
Extractors []string `json:"extractors,omitempty"`
|
|
||||||
SchemaVersion string `json:"schema_version,omitempty"`
|
```go
|
||||||
ValidationStatus string `json:"validation_status,omitempty"`
|
type RunInput struct {
|
||||||
StartedAt *time.Time `json:"started_at,omitempty"`
|
Source *source.SourceDocument
|
||||||
CompletedAt *time.Time `json:"completed_at,omitempty"`
|
ExtractorKeys []string
|
||||||
|
LLMClient contracts.StructuredLLMClient
|
||||||
|
Metadata map[string]any
|
||||||
|
}
|
||||||
|
|
||||||
|
type RunOutput struct {
|
||||||
|
Approved []artifacts.Artifact `json:"approved,omitempty"`
|
||||||
|
Rejected []artifacts.RejectedArtifact `json:"rejected,omitempty"`
|
||||||
|
Warnings []contracts.Warning `json:"warnings,omitempty"`
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
Imports should include:
|
Define package errors as ordinary returned errors. Do not introduce custom error
|
||||||
|
types in this checkpoint unless tests require behavior that ordinary errors
|
||||||
|
cannot express.
|
||||||
|
|
||||||
- `encoding/json`;
|
### Required Behavior
|
||||||
- `time`;
|
|
||||||
- `gitea.maximumdirect.net/eric/notarius/internal/core/source`.
|
At this stage, `New` should only store the extractor factory.
|
||||||
|
|
||||||
|
If you add `Run` in this stage as a stub, it must return an explicit
|
||||||
|
`runner is not implemented` error and must be completed in stage 5. Prefer
|
||||||
|
adding the real `Run` method only in stage 5.
|
||||||
|
|
||||||
|
### Required Tests
|
||||||
|
|
||||||
|
Cover:
|
||||||
|
|
||||||
|
- `New` accepts a fake extractor factory and returns a non-nil runner;
|
||||||
|
- `RunInput` and `RunOutput` can be constructed with the expected fields.
|
||||||
|
|
||||||
|
Do not add registry imports to runner tests unless needed for interface
|
||||||
|
assertions. Runner should depend only on framework contracts and core packages.
|
||||||
|
|
||||||
|
### Validation
|
||||||
|
|
||||||
|
Run:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
gofmt -w internal/framework/runner
|
||||||
|
go test ./internal/framework/runner
|
||||||
|
go test ./...
|
||||||
|
```
|
||||||
|
|
||||||
|
## Stage 5: Minimal Runner Execution
|
||||||
|
|
||||||
|
### Goal
|
||||||
|
|
||||||
|
Implement the minimal runner flow from `SourceDocument` to approved/rejected
|
||||||
|
artifacts.
|
||||||
|
|
||||||
|
### Files To Update
|
||||||
|
|
||||||
|
- `internal/framework/runner/runner.go`
|
||||||
|
- `internal/framework/runner/runner_test.go`
|
||||||
|
|
||||||
|
### Required API
|
||||||
|
|
||||||
Add:
|
Add:
|
||||||
|
|
||||||
```go
|
```go
|
||||||
func ArtifactFromCandidate(candidate Candidate) Artifact
|
func (r *Runner) Run(ctx context.Context, input RunInput) (RunOutput, error)
|
||||||
```
|
```
|
||||||
|
|
||||||
This helper should copy candidate fields into an approved artifact. It may share
|
### Required Behavior
|
||||||
immutable `json.RawMessage` bytes at this checkpoint, but copying slices is
|
|
||||||
preferred where cheap.
|
`Run` must:
|
||||||
|
|
||||||
|
1. reject nil runner or nil extractor factory;
|
||||||
|
2. validate `input.Source` with `source.ValidateDocument`;
|
||||||
|
3. reject an empty `ExtractorKeys` list;
|
||||||
|
4. process extractor keys in input order;
|
||||||
|
5. build each extractor through the factory;
|
||||||
|
6. call `Extractor.Extract` with the source, LLM client, and metadata;
|
||||||
|
7. append extractor warnings to `RunOutput.Warnings`;
|
||||||
|
8. normalize candidate metadata for each returned candidate;
|
||||||
|
9. run validators in `Extractor.Validators()` order;
|
||||||
|
10. approve candidates that survive all validators;
|
||||||
|
11. reject candidates denied by validators;
|
||||||
|
12. return approved/rejected artifacts in deterministic order.
|
||||||
|
|
||||||
|
Candidate normalization must:
|
||||||
|
|
||||||
|
- assign `Candidate.Index` using a global monotonically increasing counter;
|
||||||
|
- fill empty `ExtractorKey`, `ArtifactType`, and `SchemaVersion` from the
|
||||||
|
owning extractor;
|
||||||
|
- return an error if a candidate's non-empty `ExtractorKey` differs from
|
||||||
|
`Extractor.Key()`;
|
||||||
|
- return an error if a candidate's non-empty `ArtifactType` differs from
|
||||||
|
`Extractor.ArtifactType()`;
|
||||||
|
- return an error if a candidate's non-empty `SchemaVersion` differs from
|
||||||
|
`Extractor.SchemaVersion()`.
|
||||||
|
|
||||||
|
Validation behavior:
|
||||||
|
|
||||||
|
- if an extractor has no validators, all normalized candidates are approved;
|
||||||
|
- each validator receives only currently eligible candidates;
|
||||||
|
- each validator result must have `ValidatorName == Validator.Name()`;
|
||||||
|
- call `validators.EnforceDecisionCardinality` for every validator result;
|
||||||
|
- approved decisions keep candidates eligible for the next validator;
|
||||||
|
- rejected decisions create `artifacts.RejectedArtifact` records using the
|
||||||
|
candidate, validator name, reason code, and message;
|
||||||
|
- candidates rejected by one validator are not passed to later validators;
|
||||||
|
- append validator warnings to `RunOutput.Warnings`.
|
||||||
|
|
||||||
|
Error behavior:
|
||||||
|
|
||||||
|
- return current partial `RunOutput` plus an error for setup, extraction,
|
||||||
|
validation, cardinality, or metadata mismatch failures;
|
||||||
|
- wrap errors with enough context to identify the extractor key or validator
|
||||||
|
name.
|
||||||
|
|
||||||
|
Do not validate source references inside the runner in checkpoint 2. Source
|
||||||
|
reference validation will be a deterministic validator in a later checkpoint.
|
||||||
|
|
||||||
### Required Tests
|
### Required Tests
|
||||||
|
|
||||||
Cover:
|
Cover:
|
||||||
|
|
||||||
- `ArtifactFromCandidate` preserves extractor key, artifact type, schema
|
- nil runner/factory error;
|
||||||
version, payload, source refs, and metadata;
|
- invalid source document error;
|
||||||
- JSON marshaling uses the expected field names;
|
- empty extractor list error;
|
||||||
- empty optional manifest fields are omitted.
|
- extractor keys run in configured order;
|
||||||
|
- runner assigns global candidate indices across extractors;
|
||||||
|
- runner fills empty candidate extractor metadata;
|
||||||
|
- candidate extractor-key mismatch errors;
|
||||||
|
- candidate artifact-type mismatch errors;
|
||||||
|
- candidate schema-version mismatch errors;
|
||||||
|
- no validators approves all candidates;
|
||||||
|
- validator approval produces approved artifacts;
|
||||||
|
- validator rejection produces rejected artifacts and removes candidate from
|
||||||
|
later validators;
|
||||||
|
- validator name mismatch errors;
|
||||||
|
- validator cardinality error is surfaced;
|
||||||
|
- extractor warnings and validator warnings are collected;
|
||||||
|
- partial output is returned when a later extractor or validator fails.
|
||||||
|
|
||||||
Do not add D&D-specific payload structs.
|
Use fake extractors, validators, and factories only.
|
||||||
|
|
||||||
### Validation
|
### Validation
|
||||||
|
|
||||||
Run:
|
Run:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
gofmt -w internal/core/artifacts
|
gofmt -w internal/framework/runner
|
||||||
go test ./internal/core/artifacts
|
go test ./internal/framework/runner
|
||||||
go test ./...
|
go test ./...
|
||||||
```
|
```
|
||||||
|
|
||||||
## Stage 4: Framework Contract Types
|
## Stage 6: Registry And Runner Integration Tests
|
||||||
|
|
||||||
### Goal
|
### Goal
|
||||||
|
|
||||||
Define the interfaces and request/response types later checkpoints will build
|
Prove the extractor registry and runner compose without adding real extractors.
|
||||||
against.
|
|
||||||
|
|
||||||
### Files To Add
|
### Files To Add
|
||||||
|
|
||||||
- `internal/framework/contracts/contracts.go`
|
- `internal/framework/runner/registry_integration_test.go`
|
||||||
- `internal/framework/contracts/contracts_test.go`
|
|
||||||
|
|
||||||
### Required Types And Interfaces
|
|
||||||
|
|
||||||
Create package `contracts`.
|
|
||||||
|
|
||||||
Define LLM-neutral message and completion types:
|
|
||||||
|
|
||||||
```go
|
|
||||||
type LLMMessage struct {
|
|
||||||
Role string `json:"role"`
|
|
||||||
Content string `json:"content"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type StructuredCompletionRequest struct {
|
|
||||||
StageName string `json:"stage_name"`
|
|
||||||
Messages []LLMMessage `json:"messages"`
|
|
||||||
Model string `json:"model,omitempty"`
|
|
||||||
ResponseSchemaName string `json:"response_schema_name,omitempty"`
|
|
||||||
ResponseSchema json.RawMessage `json:"response_schema,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type StructuredCompletionResponse struct {
|
|
||||||
Content json.RawMessage `json:"content"`
|
|
||||||
Provider string `json:"provider,omitempty"`
|
|
||||||
Model string `json:"model,omitempty"`
|
|
||||||
PromptTokens int `json:"prompt_tokens,omitempty"`
|
|
||||||
CompletionTokens int `json:"completion_tokens,omitempty"`
|
|
||||||
TotalTokens int `json:"total_tokens,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type StructuredLLMClient interface {
|
|
||||||
CompleteStructured(ctx context.Context, req StructuredCompletionRequest, out any) (StructuredCompletionResponse, error)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Define input adapter contract:
|
|
||||||
|
|
||||||
```go
|
|
||||||
type ParseRequest struct {
|
|
||||||
SourceID string `json:"source_id,omitempty"`
|
|
||||||
Path string `json:"path,omitempty"`
|
|
||||||
Raw []byte `json:"-"`
|
|
||||||
Metadata map[string]any `json:"metadata,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type InputAdapter interface {
|
|
||||||
Key() string
|
|
||||||
Parse(ctx context.Context, req ParseRequest) (*source.SourceDocument, error)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Define extractor contract:
|
|
||||||
|
|
||||||
```go
|
|
||||||
type ExtractionRequest struct {
|
|
||||||
Source *source.SourceDocument `json:"-"`
|
|
||||||
LLMClient StructuredLLMClient `json:"-"`
|
|
||||||
Metadata map[string]any `json:"metadata,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type ExtractionResult struct {
|
|
||||||
Candidates []artifacts.Candidate `json:"candidates,omitempty"`
|
|
||||||
Warnings []Warning `json:"warnings,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type Extractor interface {
|
|
||||||
Key() string
|
|
||||||
ArtifactType() string
|
|
||||||
SchemaVersion() string
|
|
||||||
Validators() []Validator
|
|
||||||
Extract(ctx context.Context, req ExtractionRequest) (ExtractionResult, error)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Define validator contract:
|
|
||||||
|
|
||||||
```go
|
|
||||||
type ValidationRequest struct {
|
|
||||||
Source *source.SourceDocument `json:"-"`
|
|
||||||
Candidates []artifacts.Candidate `json:"candidates"`
|
|
||||||
Metadata map[string]any `json:"metadata,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type ValidationDecision struct {
|
|
||||||
CandidateIndex int `json:"candidate_index"`
|
|
||||||
Approved bool `json:"approved"`
|
|
||||||
ReasonCode string `json:"reason_code"`
|
|
||||||
Message string `json:"message"`
|
|
||||||
DiagnosticArtifactPath string `json:"diagnostic_artifact_path,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type ValidationResult struct {
|
|
||||||
ValidatorName string `json:"validator_name"`
|
|
||||||
Decisions []ValidationDecision `json:"decisions"`
|
|
||||||
Warnings []Warning `json:"warnings,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type Validator interface {
|
|
||||||
Name() string
|
|
||||||
Validate(ctx context.Context, req ValidationRequest) (ValidationResult, error)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Define shared warning type:
|
|
||||||
|
|
||||||
```go
|
|
||||||
type Warning struct {
|
|
||||||
Scope string `json:"scope,omitempty"`
|
|
||||||
ReasonCode string `json:"reason_code"`
|
|
||||||
Message string `json:"message"`
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Required Tests
|
|
||||||
|
|
||||||
Use compile-time interface assertions with fake implementations:
|
|
||||||
|
|
||||||
```go
|
|
||||||
var _ InputAdapter = fakeAdapter{}
|
|
||||||
var _ Extractor = fakeExtractor{}
|
|
||||||
var _ Validator = fakeValidator{}
|
|
||||||
var _ StructuredLLMClient = fakeLLMClient{}
|
|
||||||
```
|
|
||||||
|
|
||||||
Also test that a fake extractor can return a candidate and expose a validator
|
|
||||||
without importing concrete adapter or extractor packages.
|
|
||||||
|
|
||||||
### Validation
|
|
||||||
|
|
||||||
Run:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
gofmt -w internal/framework/contracts
|
|
||||||
go test ./internal/framework/contracts
|
|
||||||
go test ./...
|
|
||||||
```
|
|
||||||
|
|
||||||
## Stage 5: Cross-Package Contract Composition Tests
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Prove the checkpoint 1 contracts compose at a minimal end-to-end type level
|
|
||||||
without implementing checkpoint 2 runner behavior.
|
|
||||||
|
|
||||||
### Files To Add
|
|
||||||
|
|
||||||
- `internal/framework/contracts/composition_test.go`
|
|
||||||
|
|
||||||
This can live in package `contracts_test` to verify public contracts from the
|
|
||||||
outside.
|
|
||||||
|
|
||||||
### Required Test Scenario
|
### Required Test Scenario
|
||||||
|
|
||||||
Create fakes in the test:
|
Use `extractorregistry.New()` to register two fake extractor constructors.
|
||||||
|
|
||||||
1. fake adapter parses a `ParseRequest` into a valid `SourceDocument`;
|
Run the runner with:
|
||||||
2. fake extractor receives that source and returns one `artifacts.Candidate`;
|
|
||||||
3. fake validator approves that candidate;
|
- a valid `SourceDocument`;
|
||||||
4. test manually calls those fakes in sequence.
|
- extractor keys in a deliberate order;
|
||||||
|
- fake extractors that each return one candidate;
|
||||||
|
- fake validators that approve or reject candidates.
|
||||||
|
|
||||||
Assertions:
|
Assertions:
|
||||||
|
|
||||||
- parsed source passes `source.ValidateDocument`;
|
- registered fake extractors are built by key;
|
||||||
- candidate source refs pass `source.ValidateRef`;
|
- extractor execution follows configured key order;
|
||||||
- validator returns one approved decision for the candidate index;
|
- approved artifacts are in deterministic order;
|
||||||
- no test type or assertion depends on transcript or D&D concepts.
|
- rejected artifacts are in deterministic order;
|
||||||
|
- no concrete adapter or extractor packages are imported.
|
||||||
Do not add a registry or runner in this stage. Those belong to checkpoint 2.
|
|
||||||
|
|
||||||
### Validation
|
### Validation
|
||||||
|
|
||||||
Run:
|
Run:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
gofmt -w internal/framework/contracts
|
gofmt -w internal/framework/runner
|
||||||
go test ./internal/framework/contracts
|
go test ./internal/framework/runner
|
||||||
go test ./...
|
go test ./...
|
||||||
go build ./cmd/notarius
|
go build ./cmd/notarius
|
||||||
```
|
```
|
||||||
|
|
||||||
## Stage 6: Final Checkpoint 1 Review Pass
|
## Stage 7: Final Checkpoint 2 Review Pass
|
||||||
|
|
||||||
### Goal
|
### Goal
|
||||||
|
|
||||||
Clean up naming, formatting, and accidental scope creep before checkpoint 1 is
|
Clean up naming, formatting, and accidental scope creep before checkpoint 2 is
|
||||||
considered complete.
|
considered complete.
|
||||||
|
|
||||||
### Required Review
|
### Required Review
|
||||||
@@ -504,8 +529,11 @@ Check:
|
|||||||
combat;
|
combat;
|
||||||
- no concrete adapter or extractor package exists;
|
- no concrete adapter or extractor package exists;
|
||||||
- no LLM provider code exists;
|
- no LLM provider code exists;
|
||||||
|
- no prompt, response schema, diagnostics, config, or source chunking package
|
||||||
|
was added;
|
||||||
- no third-party dependency was added;
|
- no third-party dependency was added;
|
||||||
- `README.md` was not updated to describe unimplemented behavior;
|
- runner operates on `SourceDocument`;
|
||||||
|
- input adapter registry is not wired into runner yet;
|
||||||
- roadmap docs remain the only place describing future behavior.
|
- roadmap docs remain the only place describing future behavior.
|
||||||
|
|
||||||
### Required Validation
|
### Required Validation
|
||||||
@@ -513,12 +541,15 @@ Check:
|
|||||||
Run:
|
Run:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
gofmt -w cmd internal
|
gofmt -w internal/framework
|
||||||
go test ./...
|
go test ./...
|
||||||
go build ./cmd/notarius
|
go build ./cmd/notarius
|
||||||
git status --short
|
git status --short
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Remove the root `./notarius` binary produced by `go build ./cmd/notarius` before
|
||||||
|
finishing the implementation turn.
|
||||||
|
|
||||||
The implementation response for this checkpoint should summarize:
|
The implementation response for this checkpoint should summarize:
|
||||||
|
|
||||||
- files added;
|
- files added;
|
||||||
@@ -527,8 +558,9 @@ The implementation response for this checkpoint should summarize:
|
|||||||
|
|
||||||
## Open Questions
|
## Open Questions
|
||||||
|
|
||||||
No blocking open questions remain for checkpoint 1.
|
No blocking open questions remain for checkpoint 2.
|
||||||
|
|
||||||
The plan intentionally chooses a narrow generic model and fake-only composition
|
The plan intentionally keeps raw input parsing outside the runner. The input
|
||||||
tests. More complete orchestration, registries, real adapters, real extractors,
|
adapter registry is added and tested now because it is part of framework
|
||||||
and LLM runtime behavior are deferred to later checkpoint implementation plans.
|
composition, but concrete parsing and adapter-runner wiring remain deferred
|
||||||
|
until the Seriatim adapter checkpoint.
|
||||||
|
|||||||
Reference in New Issue
Block a user