Align architecture and testing policies with upstream

docs/policy/testing.md

@@ -1,14 +1,16 @@
 # Testing Policy
 
-**Base Policy Version:** 1.1
+**Base Policy Version:** 1.3
 
-**Last Updated:** 2026-09-26
+**Last Updated**: 2026-09-28
+
+**Upstream Source:** https://gitea.maximumdirect.net/eric/policy/src/commit/d84a38ebab26f048cafbe59a2246700c35d29843/docs/policy/testing.md
 
 ## Purpose
 
 Our tests exist to make **incorrect changes expensive and correct changes cheap**.
 
-We do not optimize for test count, line coverage, exhaustive isolation, or the fewest possible tests. We optimize for sufficient confidence in important behavior while imposing as little unnecessary friction as possible on future development.
+We optimize for sufficient confidence in important behavior while minimizing friction on future development.
 
 ## Every test has a cost
 
@@ -22,9 +24,9 @@ Evaluate that tradeoff at two levels:
 1. **Per test:** What realistic defect does this test detect, how consequential would that defect be, and is that protection worth the test's lifetime cost?
 2. **Across the suite:** Does this collection provide materially more confidence than a smaller, simpler suite would?
 
-Some friction is intentional. Tests should make dangerous changes—such as breaking compatibility, corrupting data, violating security boundaries, or reintroducing subtle bugs—require deliberate review. They should not make ordinary internal changes needlessly expensive.
+Some friction is intentional. Tests should make dangerous changes—such as breaking compatibility, corrupting data, violating security boundaries, or reintroducing subtle bugs—require deliberate review. Keep ordinary internal changes inexpensive.
 
-The cost of a test is not a reason to omit testing by default. Do not cite maintenance cost abstractly. When omitting a plausible test, be able to state why the protected failure is low-risk, already covered, obvious, reversible, or cheaper to detect elsewhere. For consequential, subtle, or difficult-to-observe behavior, the presumption should favor testing.
+Evaluate maintenance cost against the specific failure being protected. When omitting a plausible test, be able to state why the protected failure is low-risk, already covered, obvious, reversible, or cheaper to detect elsewhere. For consequential, subtle, or difficult-to-observe behavior, the presumption should favor testing.
 
 ## Default testing style
 
@@ -56,7 +58,7 @@ To the extent the following features are present in the application, prioritize
 6. Previously observed or plausible regressions.
 7. Representative integration and end-to-end workflows.
 
-A package-level contract is behavior relied upon by another package or major collaborator, not every observable detail of a package implementation.
+A package-level contract comprises behavior relied upon by another package or major collaborator.
 
 Choose tests for requested behavior and established contracts. Tests must not
 introduce new restrictions, guarantees, or threat scenarios merely because they
@@ -75,14 +77,8 @@ Do not add tests merely because a function, branch, or line exists. Do not add a
 
 ## Choose the right test boundary
 
-Test through the narrowest stable boundary that expresses the behavior clearly.
-
-For application configuration, ConfigKit owns exhaustive YAML grammar,
-source traversal, import/overlay composition, provenance, and inspection
-renderer tests. Scriptorium tests its selected definition, source discovery,
-CLI override presence, command prerequisites, runtime conversion, credential
-boundary, and representative execution/inspection workflows with the tagged
-library. Do not copy the upstream parser test matrix into this repository.
+Choose the narrowest stable boundary that expresses the behavior clearly and
+provides durable confidence with minimal incidental coupling.
 
 This is often the package API, but it may instead be:
 
@@ -90,22 +86,15 @@ This is often the package API, but it may instead be:
 - a package-level operation when several internal collaborators jointly produce the behavior; or
 - a larger integration boundary when correctness emerges from interaction with a real dependency.
 
-Do not force all behavior through oversized end-to-end tests. Do not test every private helper merely because it exists. Choose the boundary that gives durable confidence with the least incidental coupling.
-
 ## Test behavior, not implementation
 
-A test should protect a decision, contract, or invariant. A test must not memorialize the current implementation.
+A test should protect a selected decision, contract, or invariant.
 
-A test is suspect when its main purpose is to detect that someone:
+A test is suspect when it primarily detects changes to private constants or
+helpers, equivalent operation ordering, incidental formatting, equivalent
+algorithms, or internal structure that preserves behavior.
 
-- changed an internal constant;
-- renamed or split a private helper;
-- reordered equivalent internal operations;
-- changed incidental formatting;
-- replaced one correct algorithm with another; or
-- refactored internal object structure without changing behavior.
-
-Refactoring should normally require no test edits unless the refactored structure is itself part of the contract.
+Behavioral tests should normally remain stable through internal refactoring that preserves the contract.
 
 ## Expected effects of different changes
 
@@ -122,7 +111,7 @@ A test failing is not the same as a test needing to be edited. Many tests may co
 
 ## Separate mechanism from policy
 
-Configurable thresholds and defaults must not be duplicated throughout the test suite.
+Derive expectations from test-controlled configuration and relationships.
 
 For this example, assume the selected contract rejects a request to start more
 processes than a configured limit. Do not encode its production default indirectly:
@@ -158,33 +147,52 @@ limit solely to reproduce this example.
 
 The production default should be tested exactly only when its literal value is itself a public, operational, safety, protocol, or compatibility requirement.
 
-Apply the same rule to limits, timeouts, capacities, retry counts, and ranges: test relationships and behavior, not duplicated literals.
+Apply the same rule to limits, timeouts, capacities, retry counts, and ranges: test relationships and behavior.
 
 For established concurrency limits, distinguish these behaviors where the contract requires them:
 
 1. **Configuration handling:** values are accepted or rejected according to the selected contract.
 2. **Runtime enforcement:** observed peak concurrency never exceeds the configured limit.
 
-Use a test-controlled limit and measure the behavior relative to that limit. Do not merely assert today's default value.
+Use a test-controlled limit and measure the behavior relative to that limit.
 
-## Avoid semantic duplication across layers
+## Assign each behavior a test owner
 
-Each behavior should have a clear test owner.
+Where these layers exist, assign ownership as follows:
 
-- Configuration tests own application YAML, strict decoding, discovery,
-  precedence, and application defaults.
-- CLI tests own argument mapping, streams, summaries, exit behavior, and
-  representative command workflows.
-- HTTP tests own DTOs, strict decoding, limits, status mapping, and restricted
-  artifact policy.
-- Formatter tests own prepared-run and definition-inspection text and JSON
-  presentation.
-- Architecture tests own dependency direction and removal invariants.
+- Parser tests own parsing cases.
+- Validator tests own validation rules.
+- Domain tests own transformations and invariants.
+- Adapter tests own external integration behavior.
+- Orchestrator tests own coordination and failure propagation.
+- CLI tests own argument and configuration mapping.
+- End-to-end tests prove that representative assembled workflows work.
+
+Give each layer its distinct responsibility and keep expectation updates localized to the contract being changed.
+
+## Scriptorium Test Ownership
+
+- Application configuration tests own Scriptorium's definition, strict
+  decoding at its boundary, source discovery and selection, explicit CLI
+  override presence, application defaults, runtime conversion, and safe
+  diagnostics. ConfigKit owns exhaustive YAML grammar, source traversal,
+  import and overlay composition, provenance, and inspection renderers.
+- CLI tests own argument mapping, streams, output, exit behavior, and
+  representative run, render, inspect, serve, and config workflows.
+- HTTP tests own DTO decoding, transport limits, status and error mapping,
+  restricted artifact reading, and representative server workflows.
+- Formatting tests own prepared-run and definition-inspection text and JSON
+  presentation. Architecture guards own dependency direction and removed
+  framework-package boundaries.
 - Promptkit owns framework parsing, orchestration, validation, profiles, and
-  model-client behavior. Scriptorium's definition compatibility fixtures protect
-  its adapter plumbing without duplicating the upstream format contract.
+  model-client behavior. Scriptorium's definition and execution fixtures
+  protect adoption through its public adapter boundary without duplicating
+  Promptkit's exhaustive format matrix.
 
-Higher-level tests should not repeat every lower-level case. A single intentional policy change should not require unrelated edits across many test files.
+HTTP lexical containment, size limits, and shared engine and backend capacity
+across concurrent requests merit focused adapter coverage because violations
+cross a security or concurrency boundary. Choose cases by realistic risk
+rather than a coverage quota or a duplicated upstream matrix.
 
 ## Use test doubles deliberately
 
@@ -205,12 +213,9 @@ Mocks are appropriate when the contract includes facts such as:
 - an expensive API is called no more than once; or
 - a security audit event is emitted.
 
-Do not use mocks merely to isolate every object or reproduce the implementation's call graph.
-
 ## Go-specific guidance
 
-Select techniques according to the behavior and risk being tested. The following
-are options, not a checklist that every package or application must satisfy:
+Select techniques by behavior and risk:
 
 - table-driven tests for meaningful behavioral categories and boundaries;
 - `t.TempDir()` for real filesystem behavior;
@@ -220,55 +225,48 @@ are options, not a checklist that every package or application must satisfy:
 - integration tests where correctness depends on component interaction; and
 - a small number of representative end-to-end tests when assembled workflows expose meaningful risks not adequately covered at narrower boundaries.
 
-Avoid exact error-string assertions unless the wording is itself contractual. Prefer `errors.Is`, `errors.As`, typed errors, or structured error fields.
+Prefer `errors.Is`, `errors.As`, typed errors, or structured error fields. Use exact error-wording assertions only when the wording is contractual.
 
-At CLI boundaries, prefer exit classifications, structured output, and the smallest stable semantic fragment needed to identify the error. Do not snapshot complete diagnostic wording unless it is contractual.
+At CLI boundaries, prefer exit classifications, structured output, and the smallest stable semantic fragment needed to identify the error. Use complete diagnostic snapshots only when the wording is contractual.
 
 Golden-file updates must require an explicit local flag. CI must not update golden files automatically, and reviewers must inspect the semantic diff before accepting an update.
 
-Keep tests readable and direct. Test helpers and fixture frameworks must earn their own maintenance cost; do not build elaborate test infrastructure for small or isolated needs.
+Keep tests readable and direct. Size test helpers and fixture frameworks to the maintenance benefit they provide.
 
 ## Coverage
 
-Coverage is a diagnostic, not a target.
+Use coverage to locate untested critical branches and unexpectedly weak packages. Evaluate test quality through meaningful behavioral protection and choose coverage according to risk.
 
-Use it to find untested critical branches and unexpectedly weak packages. Do not write low-value tests solely to increase a percentage, and do not infer test quality from coverage alone.
-
-Security-sensitive HTTP containment and external mappings may warrant denser
-coverage than straightforward process wiring. Uneven coverage is acceptable
-when it reflects risk.
+Pure domain logic will often warrant higher coverage than CLI wiring or external adapters. Uneven coverage is acceptable when it reflects risk.
 
 ## Regression tests
 
 A bug fix should normally include a regression test that fails before the fix and passes afterward.
 
-Retain the test when the defect could realistically recur and its consequences justify the ongoing cost. Prefer the narrowest durable test of the violated contract or invariant; do not preserve accidental implementation details from the original bug.
+Retain the test when the defect could realistically recur and its consequences justify the ongoing cost. Prefer the narrowest durable test of the violated contract or invariant.
 
-Not every historical bug requires a permanent test. If the underlying design has made recurrence impossible, the test has become redundant, or a stronger invariant test now subsumes it, remove or consolidate it.
+Remove or consolidate a test when the underlying design has made recurrence impossible, its protection is redundant, or a stronger invariant test subsumes it.
 
 ## Deleting or rewriting tests
 
-Tests are maintained code, not permanent historical artifacts.
-
-Delete or rewrite a test when its maintenance cost exceeds the confidence it provides.
+Delete or rewrite tests when maintenance cost exceeds their continuing
+protective value.
 
 Strong candidates include tests that:
 
 - require updates after harmless internal changes;
-- directly assert private constants without protecting a real contract;
-- duplicate the same policy across several layers;
+- assert incidental private constants or snapshot large amounts of incidental output;
 - verify mock choreography rather than outcomes;
-- snapshot large amounts of incidental output;
-- test trivial private helpers already exercised through stable package behavior;
-- protect risks already covered more effectively elsewhere;
+- duplicate policy across layers, retest trivial helpers covered by package
+  behavior, or protect risks covered more effectively elsewhere;
 - are flaky, misleading, obsolete, or disproportionately expensive to diagnose; or
 - no longer correspond to a plausible failure mode.
 
-Several brittle tests may encode one genuine requirement. Replace them with one durable behavior-level or invariant test rather than preserving all of them.
+Several brittle tests may encode one genuine requirement. Consolidate them into one durable behavior-level or invariant test.
 
 ## Reviewing a proposed test
 
-Use the following questions when the value, boundary, or durability of a proposed test is not self-evident. Significant test additions should be reviewable against them, but written answers are not required for every routine test.
+Use the following questions when a proposed test needs closer evaluation. Significant test additions should be reviewable against them; written answers are optional for routine tests.
 
 1. Which requested behavior or established contract does it protect, and what realistic defect would violate it?
 2. How likely and consequential is that defect, and is it already protected elsewhere?
@@ -282,10 +280,9 @@ A test suite is sufficient when:
 
 - important requested behavior and established contracts are protected against realistic, consequential failures;
 - boundaries and component interactions have coverage where they introduce meaningful risks not protected elsewhere;
-- failures provide useful signal rather than redundant noise;
-- legitimate internal changes usually do not require test edits; and
+- failures provide useful, distinct signals;
+- tests usually remain stable through legitimate internal changes; and
 - additional tests would mostly repeat existing protection or preserve inconsequential implementation details.
 
-Sufficiency is a risk judgment, not a coverage percentage, test count, or required
-mix of testing techniques. Reassess it as the application, its users, and the
-consequences of failure evolve.
+Reassess sufficiency as the application, its users, and the consequences of
+failure evolve.