eric/scriptorium
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 9 Wiki Activity

Align architecture and testing policies with upstream

Browse Source
This commit is contained in:
Eric Rakestraw
2026-09-28 01:43:29 +00:00
parent ba0d99f107
commit b3bb36fc86
4 changed files with 179 additions and 202 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
docs/oss/attribution.md
View File

@@ -3,7 +3,8 @@
## Standard Repository Policies ## Standard Repository Policies
Material: `docs/policy/oss.md`, `docs/policy/meta.md`, Material: `docs/policy/oss.md`, `docs/policy/meta.md`,
`docs/policy/agents.md`, `docs/policy/documentation.md`, `AGENTS.md`, and the `docs/policy/agents.md`, `docs/policy/documentation.md`,
`docs/policy/architecture.md`, `docs/policy/testing.md`, `AGENTS.md`, and the
Policy Reading Guide section of `docs/development.md`, adapted for this Policy Reading Guide section of `docs/development.md`, adapted for this
repository. repository.

230
docs/policy/architecture.md
View File

@@ -1,147 +1,109 @@
# Architecture Policy # Architecture Policy
**Base Policy Version:** 1.1 **Base Policy Version:** 1.3
**Last Updated:** 2026-09-26 **Last Updated**: 2026-09-28
**Upstream Source:** https://gitea.maximumdirect.net/eric/policy/src/commit/d84a38ebab26f048cafbe59a2246700c35d29843/docs/policy/architecture.md
## Purpose And Ownership ## Purpose And Ownership
This policy defines Scriptorium's purpose, responsibility and dependency The adopted policy defines the application's purpose and the architectural
boundaries, established invariants, and non-goals. Concrete component inventory constraints contributors must follow. It owns system boundaries, responsibility
and implementation mechanics belong in [internal documentation](../internal/overview.md); and dependency rules, established invariants, and explicit non-goals.
public behavior belongs in its canonical contracts.
Adopted constraints govern future development without implying that every Describe the concrete package inventory and implementation mechanics in internal
capability they could govern exists or is planned. Keep proposed structure and documentation. Link to canonical public contracts for their exact behavior and
implementation status in roadmaps, and decision rationale in ADRs. Acceptance to ADRs for decision rationale.
of a decision is not proof of implementation. Do not turn incidental choices
into architectural invariants.
## System Shape ## Architectural Approach
Scriptorium is an executable application with four command entry paths: CLI Hexagonal architecture is strongly recommended, including encapsulating external
`run`, CLI `render`, CLI `inspect`, and the HTTP service started by `serve`. integrations and APIs within internal adapters. Keep domain and application
The `inspect` command has prompt and profile modes. Scriptorium does not expose logic independent of external integration details, and make dependency direction
a reusable root Go package. clear in the adopted policy.
The application consumes Implement that separation with proportionate, idiomatic code. Choose interfaces
and package layout to serve the application's actual boundaries. Use adapters
for collaborators with external integration responsibilities.
## Adopted Constraints And Implemented Structure
Architectural constraints govern current and future development. Record
implemented structure accurately, proposals and implementation status in the
roadmap, and significant decisions and rationale in ADRs. Describe a target
structure as implemented when its changes land.
Constraints must reflect deliberately selected requirements and decisions.
Do not promote incidental implementation choices into architectural invariants.
## Scriptorium's Purpose And System Shape
Scriptorium is an executable application with CLI `run`, `render`,
`inspect`, and `config` command families and an HTTP service started by
`serve`. It offers simple, request-independent execution of predefined
prompts, not a reusable Scriptorium root Go API or a workflow framework.
The [internal overview](../internal/overview.md) owns the concrete package
inventory; [adapter internals](../internal/adapters.md) and
[source internals](../internal/sources.md) own implementation mechanics.
## Framework And Application Boundaries
Scriptorium consumes
[Promptkit v0.11.1](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.1/docs/consumers/pkg-promptkit.md) [Promptkit v0.11.1](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.1/docs/consumers/pkg-promptkit.md)
through its supported root package. Promptkit owns prompt execution, through its supported public root package. Promptkit owns prompt execution,
preparation, source formats, built-in profiles, model-client behavior, and preparation, prompt and profile formats, built-in profiles, model-client
validation. Scriptorium owns application configuration, executable adapters, behavior, validation, and framework defaults. Scriptorium owns application
prepared-run and definition-inspection presentation, process behavior, and HTTP configuration, executable CLI and HTTP adapters, prepared-run and
deployment policy. definition-inspection presentation, process behavior, and HTTP deployment
policy. The [format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.1/docs/formats.md)
and [outbound integration contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.1/docs/integrations/openai-compatible-chat.md)
remain Promptkit's contracts.
Scriptorium also consumes ConfigKit v0.4.1 for application configuration Scriptorium uses ConfigKit v0.4.1 for strict application YAML decoding,
decoding, resolution, and validation flow. Scriptorium owns its application resolution, and validation flow. Scriptorium retains its application schema,
schema, defaults, normalization, validation rules, source selection, and CLI defaults, normalization, validation rules, source selection, and CLI override
flag mapping; ConfigKit does not own Promptkit framework sources or HTTP mapping. ConfigKit does not own Promptkit framework sources or HTTP artifact
artifact containment. containment. The [configuration contract](../config.md) owns externally
observable configuration behavior.
The concrete package inventory is maintained in the Application code may import Promptkit's public root package, but not its
[internal overview](../internal/overview.md). `internal` packages or catalog modules. It must not recreate the removed
Scriptorium framework facade or former framework package families. Use
consumer-owned interfaces only where substitution at an adapter boundary is
needed. Omitted framework settings pass as zero values so Promptkit applies
its own defaults. Cross-project changes land and release in Promptkit before
Scriptorium adopts its tagged dependency; do not commit Go workspaces, local
replacements, vendored Promptkit source, or unpublished dependency assumptions.
The repository's architecture guard protects these import and removal
boundaries. Adapter separation does not require an interface for every type,
extra wrappers, or a fixed directory tree.
As established by [ADR 0004](../adr/0004-definition-boundary.md), Scriptorium ## Definition Support And Request State
aims to support the complete set of features in valid Promptkit prompt and
profile definitions for its selected upstream version. Promptkit remains the
canonical parser and validator. Library-only features are considered
individually against the simple predefined-prompt execution model.
Maintain adapter boundaries with proportionate, idiomatic code. This approach As decided in [ADR 0004](../adr/0004-definition-boundary.md), Scriptorium aims
does not require an interface for every type, additional wrappers, or a fixed to support the complete set of features in valid Promptkit prompt and profile
directory tree for integrations that do not exist. definitions for its selected upstream version. Promptkit remains the canonical
parser and validator. Library-only features are considered individually
against Scriptorium's simple predefined-prompt execution model.
## Dependency Direction External YAML and JSON decoding remains strict. Adapters map external values
through Promptkit public requests and results without taking over framework
```text orchestration. Explicit numeric request overrides preserve presence, including
cmd/scriptorium zero. Adapters map Promptkit public error identities to CLI exits or HTTP
| statuses without classifying by message text. Raw API keys are not accepted
v in Scriptorium configuration, CLI arguments, or HTTP payloads, and resolved
CLI and HTTP adapters, configuration, defaults, and formatting
|
v
Promptkit (execution) and ConfigKit (application configuration)
```
- Retained application packages may import Promptkit's root package.
- They must not import Promptkit `internal` packages.
- They must not import Promptkit catalog modules directly.
- They must not import the removed Scriptorium root facade or recreate former
framework package families.
- Adapter-owned interfaces use Promptkit public values when a consumer-side
substitution boundary is needed.
- Scriptorium passes omitted framework settings as zero values so Promptkit
applies its own defaults.
The repository architecture guard enforces these import and removal
invariants.
Cross-project changes land and release in Promptkit before Scriptorium adopts
the tagged dependency. Do not commit Go workspaces, local replacements,
vendored Promptkit implementation, or unpublished dependency assumptions.
## Retained Boundaries
- `internal/adapter/cli` owns commands, flags, configuration precedence,
process streams, output files, summaries, and exit codes.
- `internal/adapter/http` owns routes, strict JSON DTOs, size limits, response
mapping, status mapping, and the restricted artifact reader.
- `internal/config` owns discovery and Scriptorium's application definition;
ConfigKit owns strict YAML decoding and the load/override pipeline.
- `internal/defaults` owns Scriptorium application and HTTP defaults only.
- `internal/format` owns deterministic prepared-run and definition-inspection
text and JSON presentation.
- Promptkit owns framework orchestration and contracts. Its
[format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.1/docs/formats.md)
and
[outbound integration contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.1/docs/integrations/openai-compatible-chat.md)
are canonical.
## HTTP Artifact Security Boundary
Ordinary CLI file loading is provided by Promptkit. Scriptorium's HTTP adapter
injects a restricted `promptkit.ArtifactReader` for inbound HTTP requests.
That reader denies file references without an artifact root, enforces the
configured byte limit, and applies Scriptorium's lexical root-containment rule.
The operating system still follows symlinks after the lexical check.
This boundary handles untrusted request references under the documented
deployment model; it is not a filesystem sandbox. Defending against a
privileged actor concurrently modifying the host filesystem is outside the
adopted scope, not a claim that such activity cannot occur. Do not infer
additional defenses or guarantees from hypothetical threats; raise unsettled
scope decisions under the [agent policy](agents.md).
The [HTTP API](../api.md) owns observable request outcomes, and
[operations](../operations.md) owns deployment permissions and root selection.
## State, Errors, And Secrets
Scriptorium has no durable run-state store, checkpoint, cache, or resume
mechanism. Recovery is a new request after correcting inputs, configuration, or
environment.
Each request is self-contained from the caller's perspective, without
conversation or workflow state retained from completed requests. This does
not mean the process is internally stateless. Shared engines, backend capacity
and queue accounting, in-flight coordination, cancellation, and server
lifecycle resources are permitted and must be shared where necessary to
enforce configured backend limits across concurrent requests. Such state must
be concurrency-safe and does not create a durable user-workflow contract.
Adapters map Promptkit public error identities into CLI exits or HTTP statuses
without classifying by message text. Raw API keys are not accepted in
Scriptorium configuration, CLI arguments, or HTTP payloads, and resolved
secrets must not be emitted. secrets must not be emitted.
## Architectural Invariants Each execution request is self-contained from the caller's perspective; no
conversation or workflow state is retained from completed requests. This
- External YAML and JSON decoding remains strict. does not preclude process-scoped operational state. A server shares its
- CLI and HTTP behavior remains presentation and transport logic rather than Promptkit engine, backend registry, capacity admission and queue accounting
framework orchestration. across relevant concurrent requests so configured backend limits cannot be
- Explicit numeric request overrides preserve presence, including zero. bypassed by per-request construction. In-flight coordination, cancellation,
- HTTP artifact containment and byte limits remain Scriptorium policy. and server lifecycle state must be concurrency-safe. None of this creates a
- No application package depends on Promptkit implementation packages. durable workflow, checkpoint, cache, or resume contract; recovery is a new
request after correcting inputs, configuration, or environment.
## Non-Goals ## Non-Goals
@@ -153,5 +115,19 @@ secrets must not be emitted.
- Do not add durable workflow, archive, or resume behavior. - Do not add durable workflow, archive, or resume behavior.
- Do not add application-managed prompt modification, follow-up message flows, - Do not add application-managed prompt modification, follow-up message flows,
conversation management, or multi-step workflow orchestration. conversation management, or multi-step workflow orchestration.
## Trust Boundaries And Threat Scope
Work that is not implemented belongs in `docs/roadmap/`. Ordinary CLI file loading belongs to Promptkit. For untrusted inbound HTTP
artifact references, Scriptorium injects a restricted
`promptkit.ArtifactReader`. It denies file references without an artifact
root, applies the configured byte limit, and enforces lexical root containment.
The operating system follows symlinks after that lexical check. This is not a
filesystem sandbox or a defense against a privileged actor concurrently
modifying the host filesystem; that threat is outside the adopted scope.
Do not infer stronger defenses from hypothetical threats. Raise unsettled
scope decisions under the [agent policy](agents.md).
The [HTTP API](../api.md) owns observable request outcomes, the
[configuration contract](../config.md) owns credential supply, and
[operations](../operations.md) owns deployment permissions and root selection.
The [source internals](../internal/sources.md) describe reader mechanics.

143
docs/policy/testing.md
View File

@@ -1,14 +1,16 @@
# Testing Policy # Testing Policy
**Base Policy Version:** 1.1 **Base Policy Version:** 1.3
**Last Updated:** 2026-09-26 **Last Updated**: 2026-09-28
**Upstream Source:** https://gitea.maximumdirect.net/eric/policy/src/commit/d84a38ebab26f048cafbe59a2246700c35d29843/docs/policy/testing.md
## Purpose ## Purpose
Our tests exist to make **incorrect changes expensive and correct changes cheap**. Our tests exist to make **incorrect changes expensive and correct changes cheap**.
We do not optimize for test count, line coverage, exhaustive isolation, or the fewest possible tests. We optimize for sufficient confidence in important behavior while imposing as little unnecessary friction as possible on future development. We optimize for sufficient confidence in important behavior while minimizing friction on future development.
## Every test has a cost ## Every test has a cost
@@ -22,9 +24,9 @@ Evaluate that tradeoff at two levels:
1. **Per test:** What realistic defect does this test detect, how consequential would that defect be, and is that protection worth the test's lifetime cost? 1. **Per test:** What realistic defect does this test detect, how consequential would that defect be, and is that protection worth the test's lifetime cost?
2. **Across the suite:** Does this collection provide materially more confidence than a smaller, simpler suite would? 2. **Across the suite:** Does this collection provide materially more confidence than a smaller, simpler suite would?
Some friction is intentional. Tests should make dangerous changes—such as breaking compatibility, corrupting data, violating security boundaries, or reintroducing subtle bugs—require deliberate review. They should not make ordinary internal changes needlessly expensive. Some friction is intentional. Tests should make dangerous changes—such as breaking compatibility, corrupting data, violating security boundaries, or reintroducing subtle bugs—require deliberate review. Keep ordinary internal changes inexpensive.
The cost of a test is not a reason to omit testing by default. Do not cite maintenance cost abstractly. When omitting a plausible test, be able to state why the protected failure is low-risk, already covered, obvious, reversible, or cheaper to detect elsewhere. For consequential, subtle, or difficult-to-observe behavior, the presumption should favor testing. Evaluate maintenance cost against the specific failure being protected. When omitting a plausible test, be able to state why the protected failure is low-risk, already covered, obvious, reversible, or cheaper to detect elsewhere. For consequential, subtle, or difficult-to-observe behavior, the presumption should favor testing.
## Default testing style ## Default testing style
@@ -56,7 +58,7 @@ To the extent the following features are present in the application, prioritize
6. Previously observed or plausible regressions. 6. Previously observed or plausible regressions.
7. Representative integration and end-to-end workflows. 7. Representative integration and end-to-end workflows.
A package-level contract is behavior relied upon by another package or major collaborator, not every observable detail of a package implementation. A package-level contract comprises behavior relied upon by another package or major collaborator.
Choose tests for requested behavior and established contracts. Tests must not Choose tests for requested behavior and established contracts. Tests must not
introduce new restrictions, guarantees, or threat scenarios merely because they introduce new restrictions, guarantees, or threat scenarios merely because they
@@ -75,14 +77,8 @@ Do not add tests merely because a function, branch, or line exists. Do not add a
## Choose the right test boundary ## Choose the right test boundary
Test through the narrowest stable boundary that expresses the behavior clearly. Choose the narrowest stable boundary that expresses the behavior clearly and
provides durable confidence with minimal incidental coupling.
For application configuration, ConfigKit owns exhaustive YAML grammar,
source traversal, import/overlay composition, provenance, and inspection
renderer tests. Scriptorium tests its selected definition, source discovery,
CLI override presence, command prerequisites, runtime conversion, credential
boundary, and representative execution/inspection workflows with the tagged
library. Do not copy the upstream parser test matrix into this repository.
This is often the package API, but it may instead be: This is often the package API, but it may instead be:
@@ -90,22 +86,15 @@ This is often the package API, but it may instead be:
- a package-level operation when several internal collaborators jointly produce the behavior; or - a package-level operation when several internal collaborators jointly produce the behavior; or
- a larger integration boundary when correctness emerges from interaction with a real dependency. - a larger integration boundary when correctness emerges from interaction with a real dependency.
Do not force all behavior through oversized end-to-end tests. Do not test every private helper merely because it exists. Choose the boundary that gives durable confidence with the least incidental coupling.
## Test behavior, not implementation ## Test behavior, not implementation
A test should protect a decision, contract, or invariant. A test must not memorialize the current implementation. A test should protect a selected decision, contract, or invariant.
A test is suspect when its main purpose is to detect that someone: A test is suspect when it primarily detects changes to private constants or
helpers, equivalent operation ordering, incidental formatting, equivalent
algorithms, or internal structure that preserves behavior.
- changed an internal constant; Behavioral tests should normally remain stable through internal refactoring that preserves the contract.
- renamed or split a private helper;
- reordered equivalent internal operations;
- changed incidental formatting;
- replaced one correct algorithm with another; or
- refactored internal object structure without changing behavior.
Refactoring should normally require no test edits unless the refactored structure is itself part of the contract.
## Expected effects of different changes ## Expected effects of different changes
@@ -122,7 +111,7 @@ A test failing is not the same as a test needing to be edited. Many tests may co
## Separate mechanism from policy ## Separate mechanism from policy
Configurable thresholds and defaults must not be duplicated throughout the test suite. Derive expectations from test-controlled configuration and relationships.
For this example, assume the selected contract rejects a request to start more For this example, assume the selected contract rejects a request to start more
processes than a configured limit. Do not encode its production default indirectly: processes than a configured limit. Do not encode its production default indirectly:
@@ -158,33 +147,52 @@ limit solely to reproduce this example.
The production default should be tested exactly only when its literal value is itself a public, operational, safety, protocol, or compatibility requirement. The production default should be tested exactly only when its literal value is itself a public, operational, safety, protocol, or compatibility requirement.
Apply the same rule to limits, timeouts, capacities, retry counts, and ranges: test relationships and behavior, not duplicated literals. Apply the same rule to limits, timeouts, capacities, retry counts, and ranges: test relationships and behavior.
For established concurrency limits, distinguish these behaviors where the contract requires them: For established concurrency limits, distinguish these behaviors where the contract requires them:
1. **Configuration handling:** values are accepted or rejected according to the selected contract. 1. **Configuration handling:** values are accepted or rejected according to the selected contract.
2. **Runtime enforcement:** observed peak concurrency never exceeds the configured limit. 2. **Runtime enforcement:** observed peak concurrency never exceeds the configured limit.
Use a test-controlled limit and measure the behavior relative to that limit. Do not merely assert today's default value. Use a test-controlled limit and measure the behavior relative to that limit.
## Avoid semantic duplication across layers ## Assign each behavior a test owner
Each behavior should have a clear test owner. Where these layers exist, assign ownership as follows:
- Configuration tests own application YAML, strict decoding, discovery, - Parser tests own parsing cases.
precedence, and application defaults. - Validator tests own validation rules.
- CLI tests own argument mapping, streams, summaries, exit behavior, and - Domain tests own transformations and invariants.
representative command workflows. - Adapter tests own external integration behavior.
- HTTP tests own DTOs, strict decoding, limits, status mapping, and restricted - Orchestrator tests own coordination and failure propagation.
artifact policy. - CLI tests own argument and configuration mapping.
- Formatter tests own prepared-run and definition-inspection text and JSON - End-to-end tests prove that representative assembled workflows work.
presentation.
- Architecture tests own dependency direction and removal invariants. Give each layer its distinct responsibility and keep expectation updates localized to the contract being changed.
## Scriptorium Test Ownership
- Application configuration tests own Scriptorium's definition, strict
decoding at its boundary, source discovery and selection, explicit CLI
override presence, application defaults, runtime conversion, and safe
diagnostics. ConfigKit owns exhaustive YAML grammar, source traversal,
import and overlay composition, provenance, and inspection renderers.
- CLI tests own argument mapping, streams, output, exit behavior, and
representative run, render, inspect, serve, and config workflows.
- HTTP tests own DTO decoding, transport limits, status and error mapping,
restricted artifact reading, and representative server workflows.
- Formatting tests own prepared-run and definition-inspection text and JSON
presentation. Architecture guards own dependency direction and removed
framework-package boundaries.
- Promptkit owns framework parsing, orchestration, validation, profiles, and - Promptkit owns framework parsing, orchestration, validation, profiles, and
model-client behavior. Scriptorium's definition compatibility fixtures protect model-client behavior. Scriptorium's definition and execution fixtures
its adapter plumbing without duplicating the upstream format contract. protect adoption through its public adapter boundary without duplicating
Promptkit's exhaustive format matrix.
Higher-level tests should not repeat every lower-level case. A single intentional policy change should not require unrelated edits across many test files. HTTP lexical containment, size limits, and shared engine and backend capacity
across concurrent requests merit focused adapter coverage because violations
cross a security or concurrency boundary. Choose cases by realistic risk
rather than a coverage quota or a duplicated upstream matrix.
## Use test doubles deliberately ## Use test doubles deliberately
@@ -205,12 +213,9 @@ Mocks are appropriate when the contract includes facts such as:
- an expensive API is called no more than once; or - an expensive API is called no more than once; or
- a security audit event is emitted. - a security audit event is emitted.
Do not use mocks merely to isolate every object or reproduce the implementation's call graph.
## Go-specific guidance ## Go-specific guidance
Select techniques according to the behavior and risk being tested. The following Select techniques by behavior and risk:
are options, not a checklist that every package or application must satisfy:
- table-driven tests for meaningful behavioral categories and boundaries; - table-driven tests for meaningful behavioral categories and boundaries;
- `t.TempDir()` for real filesystem behavior; - `t.TempDir()` for real filesystem behavior;
@@ -220,55 +225,48 @@ are options, not a checklist that every package or application must satisfy:
- integration tests where correctness depends on component interaction; and - integration tests where correctness depends on component interaction; and
- a small number of representative end-to-end tests when assembled workflows expose meaningful risks not adequately covered at narrower boundaries. - a small number of representative end-to-end tests when assembled workflows expose meaningful risks not adequately covered at narrower boundaries.
Avoid exact error-string assertions unless the wording is itself contractual. Prefer `errors.Is`, `errors.As`, typed errors, or structured error fields. Prefer `errors.Is`, `errors.As`, typed errors, or structured error fields. Use exact error-wording assertions only when the wording is contractual.
At CLI boundaries, prefer exit classifications, structured output, and the smallest stable semantic fragment needed to identify the error. Do not snapshot complete diagnostic wording unless it is contractual. At CLI boundaries, prefer exit classifications, structured output, and the smallest stable semantic fragment needed to identify the error. Use complete diagnostic snapshots only when the wording is contractual.
Golden-file updates must require an explicit local flag. CI must not update golden files automatically, and reviewers must inspect the semantic diff before accepting an update. Golden-file updates must require an explicit local flag. CI must not update golden files automatically, and reviewers must inspect the semantic diff before accepting an update.
Keep tests readable and direct. Test helpers and fixture frameworks must earn their own maintenance cost; do not build elaborate test infrastructure for small or isolated needs. Keep tests readable and direct. Size test helpers and fixture frameworks to the maintenance benefit they provide.
## Coverage ## Coverage
Coverage is a diagnostic, not a target. Use coverage to locate untested critical branches and unexpectedly weak packages. Evaluate test quality through meaningful behavioral protection and choose coverage according to risk.
Use it to find untested critical branches and unexpectedly weak packages. Do not write low-value tests solely to increase a percentage, and do not infer test quality from coverage alone. Pure domain logic will often warrant higher coverage than CLI wiring or external adapters. Uneven coverage is acceptable when it reflects risk.
Security-sensitive HTTP containment and external mappings may warrant denser
coverage than straightforward process wiring. Uneven coverage is acceptable
when it reflects risk.
## Regression tests ## Regression tests
A bug fix should normally include a regression test that fails before the fix and passes afterward. A bug fix should normally include a regression test that fails before the fix and passes afterward.
Retain the test when the defect could realistically recur and its consequences justify the ongoing cost. Prefer the narrowest durable test of the violated contract or invariant; do not preserve accidental implementation details from the original bug. Retain the test when the defect could realistically recur and its consequences justify the ongoing cost. Prefer the narrowest durable test of the violated contract or invariant.
Not every historical bug requires a permanent test. If the underlying design has made recurrence impossible, the test has become redundant, or a stronger invariant test now subsumes it, remove or consolidate it. Remove or consolidate a test when the underlying design has made recurrence impossible, its protection is redundant, or a stronger invariant test subsumes it.
## Deleting or rewriting tests ## Deleting or rewriting tests
Tests are maintained code, not permanent historical artifacts. Delete or rewrite tests when maintenance cost exceeds their continuing
protective value.
Delete or rewrite a test when its maintenance cost exceeds the confidence it provides.
Strong candidates include tests that: Strong candidates include tests that:
- require updates after harmless internal changes; - require updates after harmless internal changes;
- directly assert private constants without protecting a real contract; - assert incidental private constants or snapshot large amounts of incidental output;
- duplicate the same policy across several layers;
- verify mock choreography rather than outcomes; - verify mock choreography rather than outcomes;
- snapshot large amounts of incidental output; - duplicate policy across layers, retest trivial helpers covered by package
- test trivial private helpers already exercised through stable package behavior; behavior, or protect risks covered more effectively elsewhere;
- protect risks already covered more effectively elsewhere;
- are flaky, misleading, obsolete, or disproportionately expensive to diagnose; or - are flaky, misleading, obsolete, or disproportionately expensive to diagnose; or
- no longer correspond to a plausible failure mode. - no longer correspond to a plausible failure mode.
Several brittle tests may encode one genuine requirement. Replace them with one durable behavior-level or invariant test rather than preserving all of them. Several brittle tests may encode one genuine requirement. Consolidate them into one durable behavior-level or invariant test.
## Reviewing a proposed test ## Reviewing a proposed test
Use the following questions when the value, boundary, or durability of a proposed test is not self-evident. Significant test additions should be reviewable against them, but written answers are not required for every routine test. Use the following questions when a proposed test needs closer evaluation. Significant test additions should be reviewable against them; written answers are optional for routine tests.
1. Which requested behavior or established contract does it protect, and what realistic defect would violate it? 1. Which requested behavior or established contract does it protect, and what realistic defect would violate it?
2. How likely and consequential is that defect, and is it already protected elsewhere? 2. How likely and consequential is that defect, and is it already protected elsewhere?
@@ -282,10 +280,9 @@ A test suite is sufficient when:
- important requested behavior and established contracts are protected against realistic, consequential failures; - important requested behavior and established contracts are protected against realistic, consequential failures;
- boundaries and component interactions have coverage where they introduce meaningful risks not protected elsewhere; - boundaries and component interactions have coverage where they introduce meaningful risks not protected elsewhere;
- failures provide useful signal rather than redundant noise; - failures provide useful, distinct signals;
- legitimate internal changes usually do not require test edits; and - tests usually remain stable through legitimate internal changes; and
- additional tests would mostly repeat existing protection or preserve inconsequential implementation details. - additional tests would mostly repeat existing protection or preserve inconsequential implementation details.
Sufficiency is a risk judgment, not a coverage percentage, test count, or required Reassess sufficiency as the application, its users, and the consequences of
mix of testing techniques. Reassess it as the application, its users, and the failure evolve.
consequences of failure evolve.

5
docs/roadmap/implementation.md
View File

@@ -169,7 +169,10 @@ Generic language differences from upstream are eliminated, not rephrased.
## Stage 3: Adopt Architecture And Testing With Narrow Local Decisions ## Stage 3: Adopt Architecture And Testing With Narrow Local Decisions
**Status: Pending.** **Status: Complete.** The three shared architecture sections and the common
testing text match the pinned source. Local architecture decisions were checked
against the accepted ADRs and internal guides; focused test ownership and the
expanded attribution scope were reviewed without changing application code.
Files: `docs/policy/architecture.md`, `docs/policy/testing.md`, Files: `docs/policy/architecture.md`, `docs/policy/testing.md`,
policy attribution entry, and focused internal docs only if moving existing policy attribution entry, and focused internal docs only if moving existing