audita/docs/roadmap/library-boundary-findings.md

Shared Library Boundary: Decisions and Evidence

Status: Boundary and offline prototype completed, 2026-09-23. Selected prototype contract and proposed production migration target; not the current Audita configuration contract. The bounded roadmap owns scope and completion criteria.

Subsequent scope decision: The ConfigKit overlay plan adds optional selected overlays, with v0.4.0 as the migration baseline. It supersedes the original no-overlay decision and adds an input layer between the main file and explicit setting overrides. The v0.3.1 prototype results below remain historical evidence; they do not establish overlay integration.

Selected Boundary

The former prototype is retained in Git history at 590604b; it was a separate Go module using published ConfigKit v0.4.0, PromptKit v0.11.0, and Audita v1.1.0. The Audita dependency lets this consumer reuse actual internal domain helpers through its Audita module namespace without a local replacement. It does not make those internal packages a new public API. Remove this dependency along with the prototype after equivalent production integration tests landed.

The initial prototype results below were recorded with ConfigKit v0.3.1. On 2026-09-24, upgrading only its ConfigKit pin and sums to v0.4.0 required no source changes. With Go 1.26.8 on linux/amd64, the nested offline go test ./... passed and go run . still reported one accepted correction, four attempts, zero transient retries, and two structural repairs. This verifies the dependency upgrade, not selected-overlay behavior.

ConfigKit owns loading, presence, explicit overrides, secret resolution, and redacted effective settings. PromptKit owns profile parsing, rendering, provider execution, transient retries, structural repair, and backend admission. Audita owns file selection, setting meanings, role selection, domain acceptance, and mapping execution facts into reports. Production CLI and runtime remain unchanged.

Configuration and Profile Contract

Selected for the prototype and recommended for production:

• Main YAML requires integer version: 2; absent, null, unknown fields, and older versions fail. No legacy parser, aliases, environment overlays, or automatic conversion. Defaults precede the selected file; explicitly supplied CLI settings become ConfigKit overrides. Validation runs on the resolved result, so an override may repair an invalid lower-precedence setting when decoding succeeds.
• Production file selection: explicit --config, then /usr/local/etc/audita/config.yml, then /etc/audita/config.yml. Select one file; do not merge them. Missing system files are skipped; explicit missing or empty paths and inaccessible selected files fail. No selected file fails with a request to configure backend/profile selection: no hidden provider or model default. Remove AUDITA_CONFIG. The prototype loads its explicit fixture only; discovery and command parsing belong to the ConfigKit implementation stage.
• profiles_dir defaults to profiles, resolved relative to the selected main configuration file, with absolute paths accepted. PromptKit reads that directory. Prompts and schemas are application-embedded, not operator-selected prompt files. The prototype uses complete application-owned profile files. Catalog profiles and profile inheritance are not needed for this migration; do not add catalog wiring.
• backends.<id>.endpoint is required. Use one application backend identity per service capacity pool. Do not split proposal and validation into different backend IDs when they use the same service. Limits are engine-local, not shared across multiple Audita processes or other applications.
• backends.<id>.concurrency defaults to 1, must be positive, and is shared by both roles and all initial/retry/repair attempts. Operators can select 4 for the described local backend or a suitable finite paid API limit. There is no additional global or per-role limiter. queue_capacity defaults to 1024, following PromptKit; non-negative, with zero meaning no waiting admission. This is an operational escape hatch rather than a second scheduling hierarchy.
• backends.<id>.credential is an optional ConfigKit secret reference: a scalar environment variable name. Omission or null means no credential. A supplied reference must resolve to a nonempty value, including during validation and inspection. There is no direct key field or CLI key flag. ConfigKit resolves it once and Audita supplies RunRequest.APIKey. Profiles must not introduce another api_key_env lookup. Backend credentials also support unauthenticated local use.
• Profiles own model, temperature, output token limit, top-p, per-generation timeout, service tier, reasoning effort, and extra provider parameters using PromptKit's format. They select the application backend; their effective endpoint must equal that backend's endpoint. Endpoint-only execution is rejected because it bypasses the requested finite capacity boundary. A named registered backend is mandatory even if both roles use the same profile.
• llm.proposal_profile is required. Omitted llm.validation_profile inherits the final proposal profile as a whole, with ConfigKit recording the derivation. Explicit empty or null values fail; no individual model/endpoint/key inheritance.
• llm.transient_retries defaults to 2 additional attempts per generation; explicit zero disables retries; negative is invalid. PromptKit owns which failures are eligible. Do not add an Audita retry loop.
• llm.repair_attempts defaults to 2 additional generations for both roles; explicit zero disables repair when deliberately requested. PromptKit supports 0–3. Positive defaults are exercised in the prototype. Audita owns this common setting and supplies a complete request output contract with the role's embedded schema, avoiding separate user-facing repair defaults in each prompt.
• llm.operation_timeout defaults to 15m, must be positive, and uses a Go duration string. It starts before each logical proposal/assessment call and covers preparation, capacity waiting, transient backoff, generation, and repairs. It is separate from whole-process cancellation. Profile timeout_seconds defaults to PromptKit's 600 seconds and is applied afresh to each built-in HTTP attempt. It does not bound the combined retry/repair sequence. A stricter caller/transport timeout can also end execution. The two defaults are selected starting points, not throughput measurements. With the default T=2 and R=2, each logical call dispatches at most 9 attempts before deadline/cancellation; proposal plus assessment may use 18.
• Omission retains static defaults or invokes the recorded role/backend derivations. Null is not a general synonym for omission: nonoptional scalars reject it. Optional credentials and the future optional section target accept null. Explicit empty module lists will mean an empty correction pipeline, not “restore defaults.” Repeated modules retain their existing ordered meaning.
• The original prototype disabled ConfigKit imports, selected configuration layers (then called configuration profiles), and secret-directory bootstrap. Selected overlays are now verified under the linked overlay decision; the other two features remain disabled. PromptKit execution profiles retain their meaning.

Configuration validation/inspection must assemble and inspect both selected profiles without provider calls. ConfigKit's effective YAML describes only its loaded state; a separate selected-profile view shows effective model controls/backend selection. Do not imply either snapshot alone represents the full execution configuration. The prototype exercises the two APIs but does not implement production subcommands. Do not serialize raw profile/library objects as a permanent Audita report schema.

Migration Inventory

This inventory covers the current configuration and process flags documented in configuration and CLI, including environment-only fields. “Retain” below means retain domain meaning in the new schema, not accept old files. All legacy environment overlays listed afterward are removed.

Current file field / setting	Current CLI flag(s)	Target and rationale
File selection	--config	Retain explicit invocation flag; discovery described above.
Transcript input	positional path	Invocation only; retain one transcript path.
Glossary input	--glossary	Invocation only; retain required glossary path.
Corrected output	--output	Invocation only; retain stdout fallback.
Report output	--report-json	Invocation only; retain separate report file.
pipeline.modules	--modules	Retain sequence/default [glossary, homophones, glossary, spoken_word, grammar]; explicit empty clears rather than inherits.
output.schema	--output-schema	Retain encoder selection/default bare-segments.
llm.proposal.base_url, llm.validation.base_url	--base-url, --validation-base-url	Replace with backend endpoint; remove per-role endpoint flags.
Both llm.*.model	--model, --validation-model	Replace with profile model; new --proposal-profile / --validation-profile select whole profiles.
Both llm.*.api_key_env, direct runtime keys	--llm-api-key, --validation-llm-api-key	Replace with backend ConfigKit secret reference; remove direct key flags.
Both llm.*.timeout	--llm-timeout-seconds, --validation-llm-timeout-seconds	Replace with profile timeout_seconds, plus llm.operation_timeout; no duplicate timeout flags.
Both llm.*.max_retries	--max-retries, --validation-max-retries	Replace with common llm.transient_retries and distinct llm.repair_attempts; remove misleading old flags.
concurrency.total_llm	--total-llm-concurrency, --llm-concurrency	Replace with per-backend concurrency; remove global/alias flags.
concurrency.proposal_llm, concurrency.validation_llm	--proposal-llm-concurrency, --validation-llm-concurrency	Remove role budgets; shared backend handles service capacity.
Validation prompt budget (no old YAML field)	--validation-max-prompt-tokens	Add validation.max_prompt_tokens, default 2048, positive; retain explicit flag override. Audita owns batching.
chunking.target_sections	--target-sections	Retain optional positive target; omission/null means automatic, zero invalid.
chunking.max_section_tokens	--max-section-tokens	Retain 8192, positive.
chunking.min_section_tokens	--min-section-tokens	Retain 2048, positive, no greater than maximum.
normalization.max_segment_gap	--normalize-max-segment-gap	Retain 4.0 numeric seconds, non-negative; drop alternate YAML duration syntax for these domain numeric fields.
normalization.ellipsis_gap	--normalize-ellipsis-gap	Retain 3.5 numeric seconds, non-negative.
normalization.max_segment_duration	--normalize-max-segment-duration	Retain 60.0 numeric seconds, positive.
normalization.max_segment_tokens	--normalize-max-segment-tokens	Retain 2048, positive.
thresholds.glossary	--glossary-confidence-threshold	Retain 0.8 in [0,1].
thresholds.homophones	--homophones-confidence-threshold	Retain 0.8 in [0,1].
thresholds.spoken_word	--spoken-word-confidence-threshold	Retain 0.8 in [0,1].
thresholds.grammar	--grammar-confidence-threshold	Retain 0.8 in [0,1].
context.description	--transcript-description	Retain trimming and 500-byte limit; empty clears context.
diagnostics.work_dir	--work-dir	Retain /tmp/audita, nonempty; artifact lifecycle remains Audita-owned.
diagnostics.retention	--work-dir-retention	Retain auto default and selection; existing never follow-up is separate.
Help/version/config commands	--help, -h	Retain command purpose; help should parse before loading configuration. Effective-config output intentionally becomes ConfigKit YAML, not legacy JSON.

Retained setting flags become explicit ConfigKit overrides only when supplied; CLI parser defaults must not overwrite file values. Normalization flag values stay numeric seconds. Domain values convert into existing domain structs after validation; do not spread ConfigKit or PromptKit types into correction algorithms.

Complete environment removal map (names remain permissible if explicitly named as secret references; none have implicit configuration meaning):

Old environment names	Replacement
AUDITA_CONFIG	File selection above.
AUDITA_MODULES	pipeline.modules / explicit flag.
AUDITA_LLM_API_KEY, OPENROUTER_API_KEY, AUDITA_VALIDATION_LLM_API_KEY	Explicit backend credential references; no implicit fallback.
AUDITA_MODEL, AUDITA_VALIDATION_MODEL	Profile files and role selection.
AUDITA_BASE_URL, AUDITA_VALIDATION_BASE_URL	Backend endpoints.
AUDITA_LLM_TIMEOUT_SECONDS, AUDITA_VALIDATION_LLM_TIMEOUT_SECONDS	Profile timeout and operation deadline.
AUDITA_MAX_RETRIES, AUDITA_VALIDATION_MAX_RETRIES	Common transient/repair settings.
AUDITA_TOTAL_LLM_CONCURRENCY, AUDITA_LLM_CONCURRENCY, AUDITA_PROPOSAL_LLM_CONCURRENCY, AUDITA_VALIDATION_LLM_CONCURRENCY	Backend concurrency; remove alias/role budgets.
AUDITA_VALIDATION_MAX_PROMPT_TOKENS	validation.max_prompt_tokens / explicit flag.
AUDITA_TARGET_SECTIONS, AUDITA_MAX_SECTION_TOKENS, AUDITA_MIN_SECTION_TOKENS	Corresponding chunking fields / flags.
AUDITA_NORMALIZE_MAX_SEGMENT_GAP, AUDITA_NORMALIZE_ELLIPSIS_GAP, AUDITA_NORMALIZE_MAX_SEGMENT_DURATION, AUDITA_NORMALIZE_MAX_SEGMENT_TOKENS	Corresponding normalization fields / flags.
AUDITA_GLOSSARY_CONFIDENCE_THRESHOLD, AUDITA_HOMOPHONES_CONFIDENCE_THRESHOLD, AUDITA_SPOKEN_WORD_CONFIDENCE_THRESHOLD, AUDITA_GRAMMAR_CONFIDENCE_THRESHOLD	Corresponding threshold fields / flags.
AUDITA_WORK_DIR, AUDITA_WORK_DIR_RETENTION	Diagnostics fields / flags.

Execution and Failure Mapping

The executable uses grammar proposal plus grammar review, current response schemas, current hardening text, transcript-section/description helpers, proposal preview, and actual decision-cardinality enforcement from Audita v1.1.0. PromptKit renders the adapted templates; the custom HTTP transport supplies OpenAI-compatible responses. The prototype pre-expands Audita's hardening template function into the embedded PromptKit definitions; it passes existing named string variables unchanged. No provider, paid call, or injected high-level LLM client participates.

Generated JSON is structurally validated/repaired by PromptKit before Audita decodes it. Audita then checks proposal applicability, confidence, and decision identity and cardinality. The demonstration previews corrections without mutation. Production retains every module's full validator chain, glossary protection, batching, and apply-time checks; the representative prototype does not claim to replace them.

Outcome	Section/batch effect	Overall run
Structural repair succeeds	Continue domain validation.	Normal execution.
Proposal repair exhausted	No proposals from this section; malformed-output warning.	Continue other work; successful completion may include warnings.
Assessment repair exhausted	Reject this batch; malformed-output warning.	Continue other work.
Schema-valid invalid/missing/duplicate decision indices	Reject batch and warn; no semantic retry/repair.	Continue other work.
Inapplicable/low-confidence proposal or negative assessment	Reject affected proposal; preserve domain reasons.	Continue other work.
Provider failure, retry exhaustion, transport or malformed HTTP response	Stop affected operation; retain available observations.	Runtime failure; cancel remaining work, retain existing Audita diagnostic handling.
Deadline, caller cancellation, capacity admission failure	Stop affected operation; no outer retry or fabricated output.	Runtime failure with distinct classified cause.
Invalid settings/profile/schema setup	Fail before processing where detectable.	Configuration error for user input, runtime error for broken embedded assets.

Zero transient retries produces one ordinary provider failure, not ErrRetriesExhausted; the latter signals an exhausted positive budget. Provider failures retain a typed cause. Do not expose raw provider messages through user diagnostics: continue Audita's separate secret-redaction boundary. A failed logical call may return no result while its observations still record completed attempts or an earlier successful generation. Already completed domain results can remain in diagnostics; failure must not be presented as a fully corrected output.

Observation and Report Handoff

The prototype maps library events into Audita-owned records under a short mutex. Records contain logical run/role/backend/profile identity, generation and attempt indices, dispatch state, queue/generation/retry/wall timing, classified outcomes, HTTP status, exhaustion, repair/retry counts, and each usage field's value, knownness, and completeness. It omits URLs, credentials, payloads, raw provider errors, and prompt text. Identifiers themselves are metadata, not a secret channel.

Use terminal records once for logical totals; retain attempt records for diagnosis without adding them to terminal totals. A known zero is distinct from unknown; partial totals remain incomplete after unmetered failed attempts. These observations are not a billing ledger. Callback code must not wait on its own run's progress.

Production report changes must distinguish logical calls from dispatched provider attempts. Replace scheduler utilization fields based on global/proposal/validation pools with per-backend admission/active timing. Preserve deterministic ledger and validator ordering. Map prompt definition/rendered hashes and selected profile/model into request metadata, but do not promise equality with legacy prompt hashes or serialize PromptKit's raw event struct as the stable report contract. Update the output integration reference, report mapper tests, and downstream migration guide when the production representation is implemented.

Verification and Remaining Handoff

The focused matrix is implemented in the nested module:

Roadmap scenario	Evidence
Successful assembled path	go run .; TestWorkflowBoundaries/repair_both_roles; actual Audita domain helpers and both selected profiles.
Presence, overrides, inheritance	TestConfigurationContract: omitted role/backend values, explicit zero, null/empty, version, unknown keys, and explicit overrides.
Credentials	Missing/unset and empty references fail; optional null accepted; effective/profile inspection redacted; HTTP authorization verified; no secret/prompt text in observations. TestProfilesRespectSharedBackendBoundary rejects competing lookup and capacity bypass.
Both roles repaired	repair_both_roles: four attempts, two repairs, one accepted correction.
Repair exhausted	Proposal and assessment exhaustion subtests: bounded counts and selected warning/rejection outcomes.
Semantic boundary	Missing span, missing/duplicate decision indices, negative assessment; no semantic repair loop.
Retry/zero/exhaustion	429 recovery, zero-budget 503, positive-budget exhaustion after completed proposal; typed error assertions.
Combined budgets	503 → invalid JSON → repair 503 → valid proposal → assessment: five attempts, two transient retries, one repair.
Shared capacity	TestSharedCapacityAndRecovery: caps four and two, both roles, a retry and repair, queue rejection, measured peak equal to cap, later successful call.
Waiting/cancellation/deadline	TestQueuedCancellationAndDeadline; TestDeadlineStopsRetryAfter: no queued dispatch after cancellation/deadline and a 60-second provider retry delay interrupted by the operation deadline.
Failed-run observations	TestPartialUsageOnFailedRun: valid HTTP output with known-zero usage followed by repair failure; nil result, retained events, incomplete known aggregate, correct counts.
Representative assets	TestRealPromptRenderingParity compares both rendered message pairs to Audita v1.1.0; TestSchemaFixturesMatchAudita compares schema bytes.

Commands executed from examples/library-adoption on Linux amd64 with Go 1.26.8:

GOWORK=off go mod download all
GOWORK=off GOPROXY=off GOSUMDB=off go mod verify
GOWORK=off GOPROXY=off GOSUMDB=off go run .
GOWORK=off GOPROXY=off GOSUMDB=off go test -race ./...
GOWORK=off GOPROXY=off GOSUMDB=off go vet ./...

All passed. The demonstration reports accepted=1, rejected=0, attempts=4, transient_retries=0, structural_repairs=2. Module verification reports all modules verified. Published tag availability was checked with git ls-remote --tags for ConfigKit v0.3.1 and PromptKit v0.11.0; successful module acquisition verifies this is not merely a local-checkout experiment. The module has no replacements.

The minimum Go 1.25.5 toolchain was also exercised with offline tests and the same successful demonstration. The exact installed-toolchain invocation was:

GOTOOLCHAIN=local GOWORK=off GOPROXY=off GOSUMDB=off /home/eric/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.5.linux-amd64/bin/go test ./...
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GOSUMDB=off /home/eric/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.5.linux-amd64/bin/go run .

An initial attempt to select that cached toolchain through GOTOOLCHAIN=go1.25.5 with checksum verification disabled was rejected by Go's toolchain verifier. Using the installed binary with GOTOOLCHAIN=local proved offline execution without weakening dependency checks. The portable README commands assume an installed sufficiently new Go executable on PATH and disable toolchain switching.

From the Audita root, go test ./... and go build -o ./bin/audita ./cmd/audita passed. These check existing production code; they do not cover the nested module. Affected relative documentation links, inventory coverage, and git diff --check were checked separately.

Findings and Limits

No adoption-blocking upstream API gap was found. Two integration details deserve explicit attention: PromptKit's profile GoDoc calls timeout_seconds a generation deadline, but its built-in HTTP path applies it separately to each attempt; the consumer therefore needs the selected total-operation deadline. Also, APIKeyRequired is an in-memory profile facility, not a profile-YAML field. The file-profile design uses the presence of ConfigKit's optional credential reference to require local secret resolution; it does not invent a new PromptKit YAML key.

The chosen defaults are proposed operational policy, not measured model-quality or cost recommendations. Tests use controlled transport responses and synthetic data; they do not establish real provider compatibility, model quality, network behavior, throughput, cross-process coordination, complete billing, or release compliance. Only the representative LLM-facing subset is implemented in the prototype; the inventory specifies the remaining domain fields for the ConfigKit migration. Production file discovery, CLI parsing, full module chains, all nine prompts, filesystem diagnostics/redaction, and serialized report migration remain their assigned production tasks. Those are deliberate scope boundaries, not missing claims of completion for the prototype.

The new profile/credential/capacity and deadline choices are explicit above; no unresolved upstream requirement prevents starting the ConfigKit implementation plan. Production report field names and final runtime wiring are selected in the implementation plan, following the outcome and observation contract here. The coordinated refactor targets both libraries directly; intermediate commits do not require a working legacy runtime.

Production Implementation Reference

The implementation plan owns ordered production work, component changes, tests, and acceptance criteria. The combined roadmap and configuration roadmap own the final target. This evidence does not require an interim runtime bridge or separately runnable configuration migration.

Retained domain scope includes transcript/glossary schemas, normalization/chunking, output encoding, proposals and apply-time checks, deterministic validators, ordered modules, LLM-validator batching/cardinality, ledger ordering, and artifact lifecycle. The production integration replaces loading, provider execution, prompt rendering, and scheduler composition while preserving those responsibilities. Retention and timestamp changes remain separate work.

Distribution Follow-Up

The pinned graph includes ConfigKit, PromptKit, both PromptKit backend catalogs, jsonschema/v6, yaml.v3, yaml.v4, x/text, and kr/text (including module/test dependency edges). Before distribution, review the actual production dependency graph and its license files, including PromptKit/catalog LGPL-3.0-or-later terms. Decide and verify the notices, license copies, corresponding library source/modifications, and rebuild/relink materials needed for Audita's Go binary/container distribution. This unit does not publish binaries or claim that release compliance is complete. No upstream relicensing request or library implementation change is required for this prototype.

Selected-Overlay Extension: 2026-09-24

The isolated prototype now pins published ConfigKit v0.4.0. ConfigKit loads overlays/testing.yml relative to the selected main file; the overlay selects complete PromptKit execution profiles from the separate profiles/ directory. The original v0.3.1 observations above are historical and do not prove this extension. No production Audita behavior changed.

With Go 1.26.8 on linux/amd64, from examples/library-adoption/, these offline checks passed with GOTOOLCHAIN=local GOWORK=off GOPROXY=off GOSUMDB=off:

go test ./...
go test -race ./...
go vet ./...
go run .
go run . --overlay testing
go run . --overlay testing --proposal-profile proposal

Each demonstration reported one accepted correction, four attempts, zero transient retries, and two structural repairs. The base run selected synthetic-grammar/synthetic-review; testing selected synthetic-grammar-testing/synthetic-review-testing; the explicit proposal override restored only synthetic-grammar. These are synthetic routing IDs, not price, latency, quality, or provider-compatibility measurements.

Requirement	Consumer evidence
Base, empty/default, testing selection, role overrides, final inheritance, and relative profile resolution	TestSelectedOverlayConfiguration loads temporary on-disk layouts and assembles both roles.
Actual base/testing/override HTTP models and repairs	TestDemonstrationSelectsOutboundModels captures every request body and matches model IDs to the reported selections and domain outcome.
Missing/invalid selection, version/directive rejection, and ordinary backend-setting validation	TestOverlayFailuresAndApplicationValidation checks ConfigKit diagnostic codes and effective backend limits.
Surviving credential resolution, absent/empty failures, null unauthenticated use, and profile-only switching	TestOverlayCredentialResolutionAndInspection controls synthetic environment values and observes HTTP Authorization.
Redacted effective output, explanation and metadata origin, profile inspection, and observations	TestOverlayCredentialResolutionAndInspection and TestOverlaySelectionProvenance inspect safe output and overlay/CLI origins.
Existing retry, repair, domain, capacity, and concurrent-observation behavior	TestWorkflowBoundaries, TestSharedCapacityAndRecovery, TestQueuedCancellationAndDeadline, TestDeadlineStopsRetryAfter, and TestPartialUsageOnFailedRun pass, including under -race.

Changing a profile does not remove a backend credential reference from the effective configuration. ConfigKit validates a surviving reference when only profile selection changes. Replacing the reference in an overlay resolves only the replacement; explicit null allows unauthenticated use. This distinction is part of the selected configuration boundary, not an application fallback. Imports and secret-directory bootstrap remain disabled.

The tests use a controlled built-in HTTP transport and synthetic credentials; there were no live provider calls. The prototype still covers one representative proposal/assessment pair, not all production modules, prompts, CLI paths, or diagnostic artifacts. Root go test ./... does not run this nested module. No upstream API gap appeared in this extension. The production implementation plan keeps implementation and release pending.