218 lines
16 KiB
Markdown
218 lines
16 KiB
Markdown
# Library Adoption: Shared Boundary and Prototype
|
|
|
|
**Last updated:** 2026-09-23
|
|
|
|
**Status:** Completed bounded boundary-design and offline-prototype unit.
|
|
Production migration has not started. The work packages below retain the original
|
|
scope; [decisions, evidence, limitations, and handoff](library-boundary-findings.md)
|
|
record the outcome. The [executable prototype](../../examples/library-adoption/README.md)
|
|
is isolated from production. No publication or upstream changes were made.
|
|
|
|
**Subsequent scope decision:** Enable ConfigKit v0.4.0 selected overlays for testing
|
|
in the next configuration unit. The [overlay plan](configkit.md#selected-overlays-for-testing)
|
|
supersedes the original optional-feature exclusion below; the completed prototype
|
|
has not yet exercised overlays.
|
|
|
|
## Objective
|
|
|
|
Define and prove the boundary between Audita, ConfigKit, and PromptKit before
|
|
replacing Audita's production configuration or LLM runtime. This unit covers the
|
|
shared design and representative prototype in stages 1 and 2 of the
|
|
[coordinated adoption roadmap](library-adoption.md). The
|
|
[ConfigKit roadmap](configkit.md) continues to own the broader configuration scope.
|
|
|
|
The deliverable is a concrete target contract supported by an offline executable
|
|
prototype, sufficient to implement the ConfigKit migration without subsequently
|
|
redesigning its LLM-facing settings. A successful prototype is not completion of
|
|
either production migration.
|
|
|
|
## Established Direction
|
|
|
|
- This is an intentional breaking migration with a migration guide, not legacy
|
|
configuration support, aliases, or a permanent compatibility mode.
|
|
- ConfigKit owns shared configuration mechanics; PromptKit owns shared prompt and
|
|
generation mechanics; Audita retains domain meaning and transcript correction.
|
|
- Structural repair is enabled from the outset for proposal and LLM-validator calls.
|
|
Positive default repair budgets must be selected, not silently left at zero.
|
|
- Transient retries and structural repair are separate. Explicit zero disables
|
|
transient retries; Audita must not add a second automatic retry loop.
|
|
- One finite concurrency limit per backend protects local capacity and bounds paid
|
|
API activity. Calls from both roles share that limit. Fine-grained role budgets
|
|
and an additional global concurrency control are unnecessary.
|
|
- Keep the sequence: boundary/prototype, ConfigKit migration, PromptKit migration,
|
|
then combined major release. No intermediate public release is planned.
|
|
|
|
## Scope and Deliverables
|
|
|
|
| Deliverable | Required result |
|
|
| --- | --- |
|
|
| Configuration inventory and migration map | Every existing setting and invocation option has a proposed destination, replacement, or removal, with rationale. |
|
|
| Boundary specification | One authoritative source for each setting, explicit precedence/defaults, credential flow, runtime conversion, and failure handling. |
|
|
| Representative executable prototype | New-style configuration drives one proposal and one assessment through released libraries and a controlled provider. |
|
|
| Focused behavioral evidence | Retry, repair, capacity, credential, observation, and domain-boundary scenarios below are demonstrated. |
|
|
| Decision and integration handoff | Concrete choices, measured limitations, affected components, removal plan, and next implementation tasks are recorded. |
|
|
|
|
Do not switch the production CLI to the prototype, migrate every module/prompt,
|
|
redesign transcript/glossary formats, change correction algorithms, modify retention
|
|
or timestamp behavior, or publish a release in this unit. Do not reproduce the
|
|
libraries' full test suites. Upstream changes require a separately scoped task if
|
|
an actual gap is found.
|
|
|
|
## Work Package 1: Inventory and Decide Ownership
|
|
|
|
Read the current [configuration](../config.md), [CLI](../cli.md),
|
|
[LLM runtime](../internal/llm-runtime.md), [prompts](../internal/prompts.md),
|
|
[validators](../internal/validators.md), and [output contracts](../integrations/output-files.md).
|
|
Use graph-first discovery to verify the relevant code and tests, especially
|
|
configuration application, prompt construction, structured responses, scheduling,
|
|
and report assembly.
|
|
|
|
Build a migration table grouped by application/domain settings, invocation state,
|
|
backend settings, execution profiles, output contracts, and inspection. Include
|
|
current environment settings and aliases without treating their existence as a
|
|
requirement to preserve them. Identify report fields whose meanings change when
|
|
one logical generation contains multiple provider attempts.
|
|
|
|
Resolve these design decisions explicitly:
|
|
|
|
| Decision | Proposed direction to prove |
|
|
| --- | --- |
|
|
| File selection and version | Audita selects the main configuration and validates a new schema version; specify discovery and the treatment of old versions. |
|
|
| Application versus invocation state | ConfigKit loads persistent settings; Audita parses invocation-only input/output paths and translates explicitly supplied setting flags into supported overrides. |
|
|
| Backend identity | One registration represents the service capacity shared by proposal and validation calls. Avoid separate identities for the same service merely because roles differ. |
|
|
| Endpoint and capacity ownership | Define them once per backend, with finite limits and an explicit policy for waiting capacity/exhaustion. Do not expose endpoint-only unlimited execution accidentally. |
|
|
| Execution profiles | Prefer complete application-owned profiles using PromptKit conventions. Decide their source and selection without duplicating model/settings fields in application configuration. Catalog selection remains a deliberate option. |
|
|
| Role selection and inheritance | Specify how proposal and validation choose profiles, including omission and any inheritance. Do not inherit old behavior solely for compatibility. |
|
|
| Credentials | Prefer one ConfigKit-owned secret-resolution path passed through PromptKit's supported request credential field. Define backend/profile credential association and avoid competing environment lookups. Validate this against profile requirements. |
|
|
| Retry, repair, and deadlines | Assign one owner and explicit defaults for each control. Use positive structural-repair defaults and document combined limits and total-operation scope. |
|
|
| Optional library features | The original prototype omitted imports, selected configuration layers, and secret-directory bootstrap. Selected overlays are now planned for testing; see the [overlay plan](configkit.md#selected-overlays-for-testing). Imports and secret-directory bootstrap remain out. |
|
|
| Inspection | Define effective configuration and selected-profile inspection without implying that one library's snapshot contains the other's complete state. |
|
|
|
|
Choose concrete field names, example values, defaults, units, and omission/null/zero
|
|
semantics for the prototype. In particular, decide what validation and inspection
|
|
do when configured credentials are unavailable. Classify choices as proposed,
|
|
selected, or unresolved; do not present candidates as implemented contracts.
|
|
|
|
## Work Package 2: Specify Execution and Reporting Semantics
|
|
|
|
Document the flow from selected configuration through backend/profile assembly,
|
|
prompt inputs, generation, structural validation/repair, typed decoding, and Audita
|
|
domain validation. Identify which values cross the boundary and which component
|
|
owns each conversion. Keep library types out of unrelated domain code where a
|
|
small existing domain value is sufficient.
|
|
|
|
Use the current correction-set and validator-decision schemas as representative
|
|
inputs. Reuse domain checks where practical. Select one real proposal/validator
|
|
pair; account for prompt helpers, template variables, shared fragments, and schema
|
|
references rather than testing only an unrelated hello-world prompt.
|
|
|
|
Record the following distinctions:
|
|
|
|
- Repair seeks output conforming to its structural contract. Success does not
|
|
establish semantic correctness or authorize applying a correction.
|
|
- A schema-valid decision set still requires Audita's cardinality and proposal
|
|
identity checks. Semantic rejection is not automatically another repair request.
|
|
- Exhausted structural repair yields failed content validation; provider failures,
|
|
retry exhaustion, cancellation, and capacity rejection have separate outcomes.
|
|
- Specify each outcome's intended effect on an Audita section/batch and overall run,
|
|
including warnings, rejected work, partial domain results, and terminal failures.
|
|
Preserve domain intent without preserving obsolete transport behavior by accident.
|
|
- For transient retry budget T and structural repair budget R, one logical call can
|
|
dispatch up to `(T + 1) * (R + 1)` provider attempts. For example, T=2 and R=2
|
|
permits nine attempts; these illustrative values do not select production defaults.
|
|
- Choose the total-operation deadline scope for a logical proposal or assessment;
|
|
distinguish it from per-attempt timeouts and whole-process cancellation. Provider
|
|
retry timing may exceed local backoff, so deadline behavior must be explicit.
|
|
|
|
Design the Audita observation record and aggregation rules. Include logical-call
|
|
and attempt identity, role/backend correlation, queue/generation/retry timing,
|
|
repair and retry counts, outcomes, and available usage. Preserve unknown/incomplete
|
|
usage rather than inventing zeros or complete costs. Do not add terminal totals to
|
|
attempt totals. Collect events safely across concurrent calls, without blocking
|
|
callbacks on progress by the same run or persisting raw library events as a stable
|
|
Audita schema. Transcript-bearing artifacts remain separately governed by Audita.
|
|
|
|
## Work Package 3: Build the Bounded Offline Prototype
|
|
|
|
Use ConfigKit v0.3.1 and PromptKit v0.11.0 as reviewed starting points; verify their
|
|
published availability and APIs before pinning the prototype. Recheck the combined
|
|
Go minimum (the reviewed PromptKit release requires Go 1.25.5). Do not depend on
|
|
sibling checkouts, an active Go workspace, or committed local module replacements.
|
|
|
|
A proposed home is `examples/library-adoption/` as an explicitly experimental,
|
|
standalone Go module, isolating prototype dependencies from the production binary.
|
|
Confirm this placement during execution. Its README must give exact offline run
|
|
and test commands after dependency acquisition; root `go test ./...` must not be
|
|
claimed to cover a nested module. Keep complete proposed configuration and prompt
|
|
fixtures with it and clearly label them as prototype-only.
|
|
|
|
The prototype should:
|
|
|
|
1. Load representative application settings with ConfigKit, using synthetic secret
|
|
values supplied by the test environment and an application-owned version rule.
|
|
2. Assemble shared backend registrations and proposal/validation profile selections.
|
|
3. Supply representative transcript/proposal inputs to PromptKit definitions and
|
|
actual response schemas using supported sources.
|
|
4. Use the built-in HTTP generation path with a controlled local endpoint or
|
|
transport. An injected LLM client alone cannot prove built-in HTTP retry behavior.
|
|
5. Run proposal generation, decode and check candidates, run assessment, and perform
|
|
the selected Audita decision checks without modifying real transcript files.
|
|
6. Collect selected safe observations and show a compact result understandable to a
|
|
reviewer, including repaired output and final domain acceptance/rejection.
|
|
|
|
No paid calls, real credentials, host environment assumptions, persistent caches,
|
|
or general-purpose test framework are needed. Make concurrency and cancellation
|
|
checks deterministic through controlled synchronization. Keep delays bounded and
|
|
use library-supported controls; do not patch library internals to make tests pass.
|
|
|
|
## Focused Verification Matrix
|
|
|
|
| Scenario | Evidence required |
|
|
| --- | --- |
|
|
| Successful assembled path | ConfigKit-loaded selections drive both real prompt roles through PromptKit and Audita checks. |
|
|
| Presence, overrides, and inheritance | Representative omission, explicit values, and role selection follow the proposed contract; unknown fields and unsupported versions fail clearly. |
|
|
| Credentials | Required missing/empty references fail as intended; controlled calls receive the chosen synthetic credential; inspection and observations do not expose it. |
|
|
| Structural repair succeeds | Invalid proposal output and invalid assessment output are repaired in their respective workflows; positive budgets are actually exercised. |
|
|
| Repair exhausted | Repeated invalid output produces the selected Audita outcome and accurate accounting, without an unbounded outer retry. |
|
|
| Domain rejection | Structurally valid but unacceptable correction or decision is rejected by Audita, demonstrating the semantic boundary. |
|
|
| Transient retry and explicit zero | An eligible temporary status recovers; zero makes one attempt. Exhaustion retains a useful classified cause. |
|
|
| Combined repair and retry | A repaired generation encountering a transient failure respects both budgets and reports distinct counts. |
|
|
| Shared capacity | Concurrent calls from both roles, including retry/repair activity, stay within one selected backend cap; test a representative local cap of four and a finite API cap. |
|
|
| Waiting, cancellation, and deadline | Cancellation/deadline stops pending work and retry waits; capacity exhaustion has the intended application outcome and later calls can still proceed. |
|
|
| Failed-run observations | Completed work remains observable after later failure; unknown usage, known zero, and aggregate counts are mapped correctly. |
|
|
|
|
Tests should protect Audita's integration decisions and important assembled behavior.
|
|
Rely on PromptKit's tests for exhaustive HTTP-status, backoff, and scheduler details;
|
|
rely on ConfigKit's tests for its parser and presence machinery. Use race testing for
|
|
new concurrent observation collection. Keep consumer checks offline and credential-free.
|
|
|
|
## Completion Criteria and Handoff
|
|
|
|
This unit is complete only when:
|
|
|
|
- The settings/ownership table and migration inventory are concrete enough to start
|
|
the ConfigKit migration, including file/profile sources, CLI scope, credentials,
|
|
defaults, finite backend limits, positive repair budgets, and deadline policy.
|
|
- The offline prototype and focused matrix demonstrate the selected boundary with
|
|
both libraries together, including both LLM roles and Audita domain checks.
|
|
- Exact verification commands, results, limitations, and unresolved upstream gaps
|
|
are recorded. Missing evidence is not reported as successful integration.
|
|
- No adoption-critical decision remains hidden in prototype code. If a gap prevents
|
|
settling the boundary, record it and revise the plan before declaring readiness.
|
|
- The existing Audita production behavior remains intact in this unit. Temporary
|
|
code is clearly isolated and has a removal/integration destination during the
|
|
production migrations; no second supported runtime is introduced.
|
|
- A component-level follow-up identifies code to replace, domain code to retain,
|
|
tests to update, documentation/migration-guide work, and any temporary bridge
|
|
needed while ConfigKit lands ahead of PromptKit.
|
|
- Licensing/distribution follow-up for the selected dependency graph is identified
|
|
for the later release stage; this prototype does not claim release compliance.
|
|
- Roadmap links and examples resolve, appropriate checks pass, and commits contain
|
|
only this unit's work. Significant settled architectural choices may be recorded
|
|
in an ADR under the [documentation policy](../policy/documentation.md), while
|
|
implementation status stays in the roadmaps.
|
|
|
|
Retain the overall adoption order unless evidence or the other planned large changes
|
|
justifies revising it. Completing this unit enables detailed production migration;
|
|
it does not authorize an intermediate release or require unrelated runtime fixes.
|