74 lines
4.6 KiB
Markdown
74 lines
4.6 KiB
Markdown
# Destination State Internals
|
|
|
|
Audience: developers and LLM coding agents changing `internal/state`.
|
|
|
|
## Purpose
|
|
|
|
`internal/state` parses, validates, serializes, and updates `.distributor.json` destination catalog state records.
|
|
|
|
## Inputs And Outputs
|
|
|
|
Inputs are destination state JSON, constructed catalog values, owner scopes, managed output paths, timestamps, and prune policy inputs. Outputs are validated catalog values, JSON bytes, managed path lists, owner-filtered output lists, missing-output repair results, and prune candidate plans.
|
|
|
|
## Boundaries
|
|
|
|
The package does not inspect storage backends, mutate files, choose workflow actions, build publish outputs, generate URLs, or parse config. Publish planning consumes parsed catalog state and catalog output helpers.
|
|
|
|
The external destination state contract is documented in `docs/integrations/destination-state.md`.
|
|
|
|
## Config Fields Used
|
|
|
|
`internal/state` uses shared constants for catalog mode, output kinds, slug-like id validation, link validation, storage path validation, and source manifest validation. Destination ids, pipeline ids, and link URLs originate from config but are supplied as values by callers.
|
|
|
|
## Adapters Used
|
|
|
|
None.
|
|
|
|
## State And Manifest Behavior
|
|
|
|
Current `.distributor.json` publish output uses schema version `4` catalog state. Required top-level fields are `schema_version`, `created_at`, `updated_at`, `state.mode`, and `outputs`; `distributor_version` is optional.
|
|
|
|
Each catalog output record requires a clean path, pipeline id, destination id, source identity, `source` or `generated` kind, lowercase SHA-256 digest, non-negative size, and created/updated timestamps. Generated outputs require `source_path` and `transform`; copied source outputs must omit both. Stored URLs are optional and must pass `internal/link` validation.
|
|
|
|
Embedded source identity records contain source manifest id, digest, and creation timestamp. Full source manifests are not embedded in catalog state.
|
|
|
|
The package identifies schema versions older than the current catalog schema as superseded legacy state for publish planning. It rejects invalid JSON, malformed catalog state, and unsupported future schema versions.
|
|
|
|
The package provides helpers for finding catalog outputs by path, filtering outputs by owner, listing managed output paths, removing missing output records for one owner or every owner, and building owner-scoped prune candidates.
|
|
|
|
Publish execution owns catalog output projection and timestamp preservation for rewritten outputs. State helpers only parse, validate, filter, and remove catalog records supplied by callers.
|
|
|
|
## Skip And Resume Behavior
|
|
|
|
Catalog parsing and helper transformations are pure. State code does not decide whether to skip, upsert, replace, force, or fail; publish planning maps parsed state and storage observations to actions.
|
|
|
|
Missing-output removal helpers remove matching output records only and leave storage inspection, timestamp updates, validation, and state rewrites to callers.
|
|
|
|
Prune planning helpers are pure. They select managed output candidates, sort deterministically by `updated_at` and path, preserve the newest `keep_latest` candidates before evaluating `older_than`, and return planned prune/preserve lists without mutating state. App-level prune execution uses missing-output removal helpers to remove only confirmed deleted records after storage deletion succeeds.
|
|
|
|
## Failure Behavior
|
|
|
|
Parsing rejects invalid JSON, trailing data, missing required fields, invalid timestamps, invalid catalog mode, duplicate outputs, invalid output paths, unsupported output kinds, missing generated transform metadata, invalid URLs, invalid digests, and negative sizes.
|
|
|
|
## Tests To Inspect
|
|
|
|
- `internal/state/catalog_test.go`
|
|
- `internal/state/prune_test.go`
|
|
- `internal/app/reconcile_state_test.go`
|
|
- `internal/cli/reconcile_state_test.go`
|
|
- `internal/publish/*_test.go`
|
|
|
|
## Architectural Invariants
|
|
|
|
- `.distributor.json` is the destination sentinel and state record.
|
|
- State helpers do not inspect or mutate storage.
|
|
- Source identity uses the source bundle contract.
|
|
- Newly written publish state uses schema version `4`.
|
|
- Superseded legacy schema handling is limited to identifying older state for publish planning.
|
|
- Missing-output repair helpers preserve unrelated owner records and outputs.
|
|
- Prune planning uses output `updated_at` and preserves unrelated owners.
|
|
- Generated outputs always record a transform id and source path.
|
|
- Output records always carry created and updated timestamps after parsing.
|
|
- Stored URLs are optional and must be absolute HTTP or HTTPS URLs when present.
|
|
- `distributor_version` is diagnostic metadata, not a comparison key.
|