294 lines
16 KiB
Markdown
294 lines
16 KiB
Markdown
# Source Extraction Inventory
|
|
|
|
This inventory records reusable implementation evidence from sibling applications.
|
|
Those checkouts are review inputs only: ConfigKit does not import them, and its
|
|
build and tests must not require them. ConfigKit's current contracts and roadmap
|
|
govern every adaptation.
|
|
|
|
## Inspected revisions
|
|
|
|
| Application | Revision | Worktree | Role |
|
|
| --- | --- | --- | --- |
|
|
| Narratio | `b68435128be1e24b94fc7ec259d6e9093669fd99` | Clean `main` tracking `origin/main` | Primary parsing, composition, import, profile, presence, defaults, resolution, and inspection source |
|
|
| WeatherReporter | `5d306a202471e10fe98923a1df27d0b0fce5c666` | Clean `main` tracking `origin/main` | Secrets bootstrap and simple-consumer/defaults/options evidence |
|
|
|
|
These revisions were inspected on 2026-09-06. Paths below are relative to each
|
|
application repository. Record later source revisions and adaptations here when
|
|
the owning implementation work extracts a symbol or case.
|
|
|
|
## Adaptation rules
|
|
|
|
- Copy and generalize only the smallest applicable mechanism or behavioral case.
|
|
- Replace upstream YAML nodes, application types, commands, logging, digests, and
|
|
provider registries with ConfigKit-owned boundaries.
|
|
- Preserve strictness, deterministic behavior, and useful failure coverage while
|
|
changing expectations that conflict with ConfigKit's accepted contracts.
|
|
- Use sanitized local fixtures only. Audio, transcripts, credentials, private
|
|
configuration, and whole application package trees are application-only.
|
|
- Record the exact copied/generalized symbols and tests in the owning section when
|
|
extraction occurs. An entry below assigns evidence; it does not claim extraction
|
|
or behavior is implemented.
|
|
|
|
## Narratio evidence
|
|
|
|
### Source parsing and strict input
|
|
|
|
Assigned to the YAML/source implementation:
|
|
|
|
- Production: `internal/config/load.go` functions `decodeStrictYAML`,
|
|
`decodeStrictYAMLFromReader`, and `LoadSessionBytesWithOptions`; and
|
|
`internal/config/composition.go` functions `parseCompositionDocument`,
|
|
`parseCompositionBytes`, and `buildCompositionNode`.
|
|
- Cases: `TestLoadersRejectTrailingYAMLDocuments`,
|
|
`TestStrictYAMLRejectsParserExposedEmptyTrailingDocument`, and
|
|
`TestStrictYAMLAcceptsSingleDocumentAndRejectsUnknownFields` in
|
|
`internal/config/loader_strict_test.go`; plus duplicate-key, alias, non-mapping,
|
|
malformed-input, scalar spelling, presence, declaration-order, and source-label
|
|
coverage in `TestParseCompositionDocumentRetainsPresenceOwnershipAndDeclarationOrder`
|
|
and `TestParseCompositionDocumentRejectsAmbiguousOrMalformedYAML`.
|
|
- Preserve: labeled reader/byte input, strict unknown-field intent, rejection of
|
|
trailing documents (including an empty second document), duplicate detection,
|
|
and source-aware failures.
|
|
- Adapt: translate v4 parser nodes into ConfigKit's private document tree, apply
|
|
ConfigKit's scalar grammar and limits, and replace raw parser errors with safe
|
|
diagnostics. Narratio's application decoders and YAML node types are not copied.
|
|
|
|
Implementation record (2026-09-06): the labeled byte/reader ownership and strict
|
|
single-document cases above were generalized into `source.go`, `parse.go`, and
|
|
`internal/yaml`. No production line was copied verbatim. ConfigKit adds byte and
|
|
tree ceilings, shared atomic reader claims, regular-file checks, exact numeric
|
|
lexemes, parser-depth calibration, ConfigKit-owned locations, safe diagnostic
|
|
codes, and an AST import-boundary regression. Application typed decoding, parser
|
|
errors, YAML node types, and session-specific loaders were not retained.
|
|
|
|
### Strict composition
|
|
|
|
Assigned to document composition:
|
|
|
|
- Production: `mergeAdditiveComposition`, `mergeAdditiveCompositions`,
|
|
`validateAdditiveClaims`, `appendCompositionClaims`, `mergeAdditiveNodes`,
|
|
`newCompositionConflict`, and cloning/path/source helpers in
|
|
`internal/config/composition.go`.
|
|
- Cases: `TestMergeAdditiveCompositionJoinsOnlyDisjointMappings`,
|
|
`TestMergeAdditiveCompositionRejectsEveryDuplicateClass`, and
|
|
`TestMergeAdditiveCompositionReportsAllClaimingSources`.
|
|
- Preserve: recursive joining of disjoint mappings, collisions for every overlapping
|
|
non-mapping value (including equal values and sequences), all claimant sources,
|
|
deterministic traversal, and nonmutation of inputs.
|
|
- Adapt: empty mappings follow ConfigKit's no-child operation rule; paths use the
|
|
ConfigKit semantic path model; both precise locations and composite origins are
|
|
retained. Default values never participate in strict composition.
|
|
|
|
### Imports
|
|
|
|
Assigned to explicit import traversal and glob handling:
|
|
|
|
- Production: `loadPipelineCompositionSources`,
|
|
`splitPipelineCompositionEnvelope`, `loadPipelineImports`, and related ownership
|
|
structures in `internal/config/pipeline_composition.go`.
|
|
- Cases: `TestSplitPipelineCompositionEnvelopePreservesInputAndDeclarationOrder`,
|
|
`TestSplitPipelineCompositionEnvelopeRejectsMalformedShapesWithoutMutation`,
|
|
`TestLoadPipelineCompositionImportsDisjointFieldsAndTracksOwnership`,
|
|
`TestLoadPipelineCompositionMergesDisjointKeyedEntries`,
|
|
`TestLoadPipelineCompositionRejectsBaseConflictsWithAllSources`,
|
|
`TestLoadPipelineCompositionRejectsUnsafeOrInvalidImports`,
|
|
`TestLoadPipelineCompositionRejectsSameFileAliasesAndImportedComposition`,
|
|
`TestLoadPipelineCompositionReportsImportedParseAndSchemaSources`, and
|
|
`TestLoadPipelineCompositionKeepsRelativePathsRootBased`.
|
|
- Preserve: explicit declarations, declaration-order traversal, disjoint
|
|
composition, conflict attribution, malformed/unreadable target failures, directive
|
|
removal, and imported parse/schema source labels.
|
|
- Adapt: replace the root-only `composition` envelope with recursive
|
|
`_configkit.imports`; add path and glob declarations, bounded depth-first graph
|
|
traversal, canonical containment, cycle/diamond routes, and fixed source IDs.
|
|
Canonical paths—not `os.SameFile`—define identity, so hard links remain distinct.
|
|
|
|
### Profiles and overlay
|
|
|
|
Assigned to selected-profile overlay:
|
|
|
|
- Production: `mergeOverlayComposition`, `mergeOverlayNodes`,
|
|
`selectPipelineProfile`, `loadPipelineProfileOverlays`, and
|
|
`loadedProfileOverlay`.
|
|
- Cases: `TestLoadPipelineProfilesSelectDefaultAndExplicitOverlay`,
|
|
`TestLoadPipelineProfileSelectionErrors`,
|
|
`TestPipelineProfileNamesRejectControlCharacters`,
|
|
`TestLoadPipelineProfilesValidateEveryDeclaredOverlay`,
|
|
`TestLoadPipelineProfileOverlaySemantics`,
|
|
`TestLoadPipelineProfileRejectsNullAndKindChanges`, and
|
|
`TestLoadPipelineProfilePairResolvesIndependentEffectivePipelines`.
|
|
- Preserve: recursive mapping overlay, atomic scalar/sequence replacement,
|
|
deterministic selection failures, source ownership, and independent results.
|
|
- Adapt: load only the selected fixed profile path; lowercase `default` means no
|
|
overlay. ConfigKit permits null and cross-kind replacement and does not support
|
|
declared defaults, inheritance, arbitrary profile paths, or validating every
|
|
unselected overlay.
|
|
|
|
### Effective inspection and provenance
|
|
|
|
Assigned to layered metadata, redaction, rendering, and explanation:
|
|
|
|
- Production: `EffectivePipelineValues`, `EffectivePipelineSources`,
|
|
`MarshalEffectivePipeline`, `effectivePipelineFieldPaths`,
|
|
`effectivePipelineOwners`, and composition `semanticRecords`/
|
|
`compactSemanticRecords`.
|
|
- Cases: `TestCompositionCanonicalOutputsAreDeterministic`,
|
|
`TestEffectivePipelineValuesIgnoreEquivalentSourceLayout`,
|
|
`TestConfigValidateAndShowPipelineOnlyAreSideEffectFree`,
|
|
`TestConfigSourcesReportsStableOwnership`,
|
|
`TestConfigSourcesReportsCanonicalFamilyAndPublishOrigins`,
|
|
`TestConfigSourcesPreservesProfileOwnershipThroughFamilyExpansion`, and
|
|
`TestConfigSourcesReportsRootImportProfileAndDefaultOwnership`.
|
|
- Preserve: deterministic effective projection, stable logical paths, source
|
|
ownership, side-effect-free inspection, and domain-attribution evidence.
|
|
- Adapt: project directly from ConfigKit-owned typed values without YAML round trips;
|
|
add line/column locations, composite origins, complete operation history, detached
|
|
records, and redaction. Narratio commands, diff/digest APIs, generated artifact
|
|
attribution, campaign logic, and party expansion remain application-owned.
|
|
|
|
### Defaults, presence, and domain resolution
|
|
|
|
Assigned across schema, decoding, resolution, validation, and consumer examples:
|
|
|
|
- Production: `NormalizeConfig.UnmarshalYAML`, `outputPathWasSet`,
|
|
`applyPipelineDefaults` and its component default helpers, `Resolve`,
|
|
`ResolveLoadedPipelineCampaign`, `mergeCampaignSession`, `selectStableInput`, and
|
|
application-supplied byte loading in `internal/app/config_loader.go`.
|
|
- Cases: `TestNormalizeLoadAndValidate`,
|
|
`TestRepresentativeLoadDefaultsAndValidation`,
|
|
`TestMarshalEffectivePipelineRendersStablePublicConfiguration`,
|
|
`TestPipelineCampaignRegistryStrictDecode`,
|
|
`TestCampaignSessionMergeFillsStableInputs`,
|
|
`TestCampaignSessionMergeSessionOverridesStableInputs`,
|
|
`TestLoadWithSessionOptionsCarriesExplicitProfilePresence`, and the maintained
|
|
extraction-example coverage in `TestMaintainedExtractionExamplesPreservePublishedContracts`.
|
|
- Preserve: fresh defaults, explicit-presence-sensitive domain decisions, strict
|
|
nested types, stable input selection, application-supplied labeled bytes, and
|
|
validation after resolution.
|
|
- Adapt: presence comes from metadata rather than custom YAML hooks or pointer/zero
|
|
guesses; explicit empty and null values are retained; defaults are cloned and
|
|
application-owned; every domain mutation is recorded. Campaign/session fallback,
|
|
search paths, fetching, storage, and concrete default values remain in Narratio.
|
|
|
|
Implementation record (2026-09-06): ConfigKit generalized Narratio's fresh-default,
|
|
strict nested-type, and explicit-presence cases into `decode.go` and
|
|
`decode_test.go`. The decoder applies the owned document tree over a detached typed
|
|
baseline, treats mappings recursively and sequences atomically, preserves explicit
|
|
false/zero/empty/null, rejects unknown nested fields, and returns private presence
|
|
and default/input origin artifacts. Unlike Narratio's custom YAML hooks and
|
|
pointer/zero inference, presence comes directly from the parsed tree and equal-valued
|
|
input remains explicit. ConfigKit also adds destination-width numeric checks,
|
|
dynamic `int64`/`float64`, exact arrays, duration strings, paired text codecs, safe
|
|
multi-error collection, and repeated-result isolation. Resolution, validation,
|
|
consumer defaults, and all campaign/session algorithms remain assigned to later
|
|
work.
|
|
|
|
Lifecycle integration record (2026-09-06): the public single-source `Load` boundary
|
|
combines the already extracted labeled-source and strict-decoding mechanisms with
|
|
Narratio's fresh-default and presence expectations. ConfigKit adds an application-
|
|
defined pre-decode string-version check, defaults-only `NoSource` operation,
|
|
detached base/default metadata and source reports, and explicit gates for every
|
|
later lifecycle option. No application loader or production line was copied.
|
|
Narratio's resolution, validation, import/profile, campaign/session, and command
|
|
behavior remains assigned to its later owning work.
|
|
|
|
Resolution implementation record (2026-09-06): ConfigKit adapted the intent of
|
|
Narratio's `Resolve`, `applyPipelineDefaults`, component default helpers,
|
|
`mergeCampaignSession`, and `selectStableInput` into a generic recorded resolver in
|
|
`resolution.go`. The synthetic timeout/artifact cases preserve explicit-input
|
|
decisions, derived dependencies, normalization, and map copy/writeback without
|
|
copying campaign/session algorithms or concrete defaults. ConfigKit adds structural
|
|
mutation auditing, container coverage, helper/path consistency checks, equal-value
|
|
intent, coalesced repeated writes, private before/after snapshots, deterministic
|
|
record ordering, and safe hook diagnostics. Semantic validation remains assigned to
|
|
the next lifecycle work.
|
|
|
|
Validation implementation record (2026-09-06): ConfigKit generalized Narratio's
|
|
`TestNormalizeLoadAndValidate`, `TestRepresentativeLoadDefaultsAndValidation`,
|
|
strict registry decoding, and maintained extraction-example expectations into the
|
|
post-resolution `Validation` lifecycle and synthetic consumer examples. ConfigKit
|
|
keeps domain rules application-owned, resolves schema-valid missing/dynamic report
|
|
paths to nearest input locations, collects independent warnings/errors, and audits
|
|
the full typed projection to reject validator mutation. Live resources, registries,
|
|
campaign/session logic, commands, and application runtime initialization were not
|
|
extracted.
|
|
|
|
### Secrets bootstrap
|
|
|
|
Assigned to the explicit bootstrap operation:
|
|
|
|
- Production: `loadSecretsFromConfigWithDependencies`, `validateSecretDirectory`,
|
|
`validateSecretFile`, `readAPIKeyFile`, and
|
|
`internal/fileops/read_bounded.go` function `ReadFileBounded`.
|
|
- Cases: `TestLoadSecretsFromConfigLoadsValidFiles`,
|
|
`TestLoadSecretsFromConfigPreservesExistingEnv`,
|
|
`TestLoadSecretsFromConfigRelativeDirUsesCWD`,
|
|
`TestLoadSecretsFromConfigMissingDirFails`,
|
|
`TestLoadSecretsFromConfigUnreadableValidEntryFails`,
|
|
`TestLoadSecretsFromConfigRejectsUnsafeModes`,
|
|
`TestLoadSecretsFromConfigRejectsNonRegularAndOversizedEntries`, and
|
|
`TestLoadSecretsFromConfigRejectsSymlinkAndAncestorReplacement`.
|
|
- Preserve: bounded reads, valid environment names, existing variables (including
|
|
empty values) winning, regular-file checks, symlink rejection, and failure cases.
|
|
- Adapt: use ConfigKit's deterministic report/preflight/apply state machine and
|
|
fixed budgets. Logging, configuration-driven implicit invocation, private-mode
|
|
conventions, and application API-key plumbing remain in Narratio.
|
|
|
|
## WeatherReporter evidence
|
|
|
|
### Secrets bootstrap
|
|
|
|
Assigned to the explicit bootstrap operation:
|
|
|
|
- Production: `loadSecrets`, `applySecrets`, `restoreEnvironment`, `secretValue`,
|
|
and `environmentValue` in `internal/config/secrets.go`.
|
|
- Cases: `TestLoadSecretsDisabledLeavesEnvironmentUnchanged`,
|
|
`TestLoadSecretsOverwritesExistingEnvironment`,
|
|
`TestLoadSecretsTrimsOneTrailingLineEnding`,
|
|
`TestLoadSecretsRejectsInvalidDirectoryEntries`,
|
|
`TestLoadSecretsRejectsMissingDirectory`,
|
|
`TestLoadFileSecretDirectoryFailureLeavesEnvironmentUnchanged`,
|
|
`TestLoadFileValidationFailureLeavesSecretEnvironmentUnset`, and
|
|
`TestApplySecretsRollsBackOnEnvironmentFailure`.
|
|
- Preserve: separate read/preflight and apply mechanisms, one-newline handling,
|
|
invalid-name/type/read coverage, no writes after preflight failure, and controlled
|
|
environment-write failure coverage.
|
|
- Adapt: existing variables—including empty ones—are skipped, never overwritten;
|
|
application failures produce an accurate partial report and no rollback. Skip
|
|
entries are not read. Bootstrap is explicitly invoked and remains independent of
|
|
YAML loading and semantic validation.
|
|
|
|
### Defaults, explicit presence, and module options
|
|
|
|
Assigned to consumer-shaped examples and migration checks:
|
|
|
|
- Production: `Load`, `mergeFile`, `normalizeReportModules`,
|
|
`traverseReportModules`, `normalizeModuleOptions`, and `decodeKnownOptions`.
|
|
- Cases: `TestDefaults`, `TestOutputDirectoryLoading`,
|
|
`TestOutputDirectoryValidationIsConsistentForLoadedAndConstructedConfigs`,
|
|
`TestLoadMinimalExampleConfig`,
|
|
`TestReportDistributorPathTemplatesSetTracksExplicitEmptyList`,
|
|
`TestValidateReportModuleKeysWithoutMutatingOptions`,
|
|
`TestReportModuleOverridesNormalizesConstructedOptionsWithoutMutatingConfig`,
|
|
`TestReportModuleOverridesCanonicalizesTypedPointerOptions`,
|
|
`TestReportDistributorPathOverridesConsistentForLoadedAndConstructedConfig`,
|
|
`TestReportModuleValidationConsistentForLoadedAndConstructedConfig`,
|
|
`TestConstructedReportOverridesUseExportedFields`, and
|
|
`TestConstructedExplicitEmptyReportOverridesAreRejected`.
|
|
- Preserve: defaults-only use, consistent loaded/constructed outcomes, explicit
|
|
empty-list domain rejection, typed module-specific options, normalization without
|
|
mutating caller input, and unknown-field rejection.
|
|
- Adapt: ConfigKit permits empty lists structurally and exposes their presence to
|
|
application validation. Object bindings replace shorthand, explicitly set flags
|
|
become CLI override records, and general environment overlays retire. Report
|
|
semantics, registries, rendering, and module option validation remain application
|
|
responsibilities.
|
|
|
|
## Application-only material
|
|
|
|
Do not extract application commands, filesystem discovery, logging, runtime
|
|
initialization, remote clients, LLM/provider setup, audio/transcript fixtures,
|
|
configuration digests, report rendering, campaign/session algorithms, or consumer
|
|
resource validation into ConfigKit. Later consumer examples may use small synthetic
|
|
types and fixtures to prove that these responsibilities remain straightforward.
|