Add shared-root destination state model
This commit is contained in:
@@ -255,7 +255,7 @@ Source bundle digest mismatches fail validation before destination writes occur.
|
|||||||
|
|
||||||
## Destination Fields
|
## Destination Fields
|
||||||
|
|
||||||
Each destination embeds a backend config at the destination level and may also configure publishing, transforms, path mapping, links, reconciliation, and transfer behavior.
|
Each destination embeds a backend config at the destination level and may also configure publishing, transforms, path mapping, links, state, reconciliation, and transfer behavior.
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
destinations:
|
destinations:
|
||||||
@@ -267,6 +267,8 @@ destinations:
|
|||||||
html: false
|
html: false
|
||||||
path_mapping:
|
path_mapping:
|
||||||
mode: preserve_relative
|
mode: preserve_relative
|
||||||
|
state:
|
||||||
|
mode: single_owner
|
||||||
reconciliation:
|
reconciliation:
|
||||||
mode: replace
|
mode: replace
|
||||||
transfer:
|
transfer:
|
||||||
@@ -282,6 +284,7 @@ destinations:
|
|||||||
- `transform`: required only when publishing generated HTML.
|
- `transform`: required only when publishing generated HTML.
|
||||||
- `path_mapping`: optional destination path mapping policy.
|
- `path_mapping`: optional destination path mapping policy.
|
||||||
- `links`: optional public URL metadata policy.
|
- `links`: optional public URL metadata policy.
|
||||||
|
- `state`: optional destination state ownership policy.
|
||||||
- `reconciliation`: optional managed-output reconciliation policy.
|
- `reconciliation`: optional managed-output reconciliation policy.
|
||||||
- `transfer`: optional destination comparison action policy.
|
- `transfer`: optional destination comparison action policy.
|
||||||
|
|
||||||
@@ -359,6 +362,19 @@ Primary URL policies:
|
|||||||
|
|
||||||
If no output matches the primary policy, per-output URLs may still be recorded and the top-level primary URL is omitted.
|
If no output matches the primary policy, per-output URLs may still be recorded and the top-level primary URL is omitted.
|
||||||
|
|
||||||
|
## Destination State Policy
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
state:
|
||||||
|
mode: single_owner
|
||||||
|
```
|
||||||
|
|
||||||
|
- `state.mode`: optional. Accepted values are `single_owner` and `shared_root`; default is `single_owner`.
|
||||||
|
|
||||||
|
`single_owner` state records one pipeline/destination owner for each destination bundle path and is the state mode written by `run`.
|
||||||
|
|
||||||
|
`shared_root` is accepted by configuration validation and by destination state parsing for shared-root `.distributor.json` files. Current publish execution writes single-owner destination state.
|
||||||
|
|
||||||
## Reconciliation Policy
|
## Reconciliation Policy
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
@@ -448,6 +464,7 @@ Defaults are applied after YAML decoding and before validation:
|
|||||||
- `transform.markdown_to_html.mode: sidecar` when a Markdown transform block is present and mode is omitted
|
- `transform.markdown_to_html.mode: sidecar` when a Markdown transform block is present and mode is omitted
|
||||||
- `path_mapping.mode: preserve_relative`
|
- `path_mapping.mode: preserve_relative`
|
||||||
- `links.primary: auto` when a `links` block is present and `primary` is omitted
|
- `links.primary: auto` when a `links` block is present and `primary` is omitted
|
||||||
|
- `state.mode: single_owner`
|
||||||
- `reconciliation.mode: replace`
|
- `reconciliation.mode: replace`
|
||||||
- `transfer.on_destination_same: skip`
|
- `transfer.on_destination_same: skip`
|
||||||
- `transfer.on_destination_older: replace`
|
- `transfer.on_destination_older: replace`
|
||||||
|
|||||||
@@ -4,9 +4,9 @@ Audience: operators, integrators, and maintainers who inspect or reason about de
|
|||||||
|
|
||||||
Each managed destination bundle path contains `.distributor.json`. This file is the destination sentinel and state record used for comparison, skip, replacement, and recovery decisions.
|
Each managed destination bundle path contains `.distributor.json`. This file is the destination sentinel and state record used for comparison, skip, replacement, and recovery decisions.
|
||||||
|
|
||||||
## State Schema
|
## Single-Owner State Schema
|
||||||
|
|
||||||
Current schema version: `2`.
|
Current state written by `run` uses schema version `2`.
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{
|
{
|
||||||
@@ -119,7 +119,80 @@ Normal replacement deletes only managed output paths recorded in `outputs` plus
|
|||||||
|
|
||||||
`distributor` can read schema version `1` destination state for compatibility. When v1 state is read, it is treated as single-owner state with `reconciliation.mode: replace`. Missing top-level `created_at` and `updated_at` are inferred from `published_at`, and missing per-output timestamps are also inferred from `published_at`.
|
`distributor` can read schema version `1` destination state for compatibility. When v1 state is read, it is treated as single-owner state with `reconciliation.mode: replace`. Missing top-level `created_at` and `updated_at` are inferred from `published_at`, and missing per-output timestamps are also inferred from `published_at`.
|
||||||
|
|
||||||
Newly written destination state uses schema version `2`.
|
Newly written destination state from publish execution uses schema version `2`.
|
||||||
|
|
||||||
|
## Shared-Root State Schema
|
||||||
|
|
||||||
|
`distributor` can parse and validate shared-root destination state with schema version `3`. Publish execution currently writes single-owner state.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"schema_version": 3,
|
||||||
|
"distributor_version": "dev",
|
||||||
|
"created_at": "2026-06-04T12:00:00Z",
|
||||||
|
"updated_at": "2026-06-04T12:10:00Z",
|
||||||
|
"state": {
|
||||||
|
"mode": "shared_root"
|
||||||
|
},
|
||||||
|
"owners": [
|
||||||
|
{
|
||||||
|
"pipeline_id": "reports",
|
||||||
|
"destination_id": "archive",
|
||||||
|
"reconciliation": {
|
||||||
|
"mode": "merge"
|
||||||
|
},
|
||||||
|
"source": {
|
||||||
|
"manifest": {
|
||||||
|
"schema_version": 1,
|
||||||
|
"id": "reports.example.2026-06-04",
|
||||||
|
"digest": "sha256:...",
|
||||||
|
"created": "2026-06-04T11:55:00Z",
|
||||||
|
"files": [
|
||||||
|
{"path": "report.md", "sha256": "sha256:...", "size": 1234}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"links": {
|
||||||
|
"primary_url": "https://reports.example.com/archive/report.html"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"outputs": [
|
||||||
|
{
|
||||||
|
"path": "report.html",
|
||||||
|
"kind": "generated",
|
||||||
|
"source_path": "report.md",
|
||||||
|
"transform": "markdown_to_html",
|
||||||
|
"url": "https://reports.example.com/archive/report.html",
|
||||||
|
"sha256": "sha256:...",
|
||||||
|
"size": 2345,
|
||||||
|
"pipeline_id": "reports",
|
||||||
|
"destination_id": "archive",
|
||||||
|
"source_id": "reports.example.2026-06-04",
|
||||||
|
"source_digest": "sha256:...",
|
||||||
|
"source_created": "2026-06-04T11:55:00Z",
|
||||||
|
"created_at": "2026-06-04T12:00:00Z",
|
||||||
|
"updated_at": "2026-06-04T12:10:00Z"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Shared-root required fields:
|
||||||
|
|
||||||
|
- `schema_version`: must be `3`.
|
||||||
|
- `created_at`: RFC3339 timestamp for when this shared-root state record was first created.
|
||||||
|
- `updated_at`: RFC3339 timestamp for the latest shared-root state update.
|
||||||
|
- `state.mode`: must be `shared_root`.
|
||||||
|
- `owners`: owner records keyed by `pipeline_id` and `destination_id`; each owner records its latest source manifest and reconciliation mode.
|
||||||
|
- `outputs`: output records for every managed path under the shared destination root.
|
||||||
|
|
||||||
|
Shared-root optional fields:
|
||||||
|
|
||||||
|
- `distributor_version`: application version string when available.
|
||||||
|
- `owners[].links.primary_url`: absolute HTTP or HTTPS URL selected by that owner's destination link policy.
|
||||||
|
|
||||||
|
Shared-root output records carry the same `path`, `kind`, `source_path`, `transform`, `url`, `sha256`, `size`, `created_at`, and `updated_at` fields as single-owner outputs. They also include the owner `pipeline_id` and `destination_id`, plus compact source identity fields `source_id`, `source_digest`, and `source_created`.
|
||||||
|
|
||||||
## Boundaries
|
## Boundaries
|
||||||
|
|
||||||
|
|||||||
@@ -18,7 +18,7 @@ The external destination state contract is documented in `docs/integrations/dest
|
|||||||
|
|
||||||
## Config Fields Used
|
## Config Fields Used
|
||||||
|
|
||||||
`internal/state` uses config reconciliation mode constants for destination state validation and legacy state normalization. Destination ids, pipeline ids, and link URLs originate from config but are supplied as values by callers.
|
`internal/state` uses config state mode and reconciliation mode constants for destination state validation and legacy state normalization. Destination ids, pipeline ids, and link URLs originate from config but are supplied as values by callers.
|
||||||
|
|
||||||
## Adapters Used
|
## Adapters Used
|
||||||
|
|
||||||
@@ -30,21 +30,26 @@ None.
|
|||||||
|
|
||||||
Schema version `1` state remains readable. Parsing infers `state.mode: single_owner`, `reconciliation.mode: replace`, top-level `created_at` and `updated_at` from `published_at`, and per-output timestamps from `published_at`.
|
Schema version `1` state remains readable. Parsing infers `state.mode: single_owner`, `reconciliation.mode: replace`, top-level `created_at` and `updated_at` from `published_at`, and per-output timestamps from `published_at`.
|
||||||
|
|
||||||
|
Schema version `3` is shared-root state. It records `state.mode: shared_root`, shared state timestamps, owner records keyed by pipeline id and destination id, each owner's latest source manifest and reconciliation metadata, optional owner primary links, and output records for every managed path. Shared-root output records include owner ids and compact source identity fields for source id, digest, and creation time.
|
||||||
|
|
||||||
Embedded source manifests are parsed and validated through `internal/bundle`, which delegates source manifest semantics to `pkg/bundle`. Output records require clean paths, `source` or `generated` kind, valid source paths, lowercase SHA-256 digests, non-negative sizes, created and updated timestamps, and transform ids for generated outputs. Stored URLs must pass `internal/link` validation.
|
Embedded source manifests are parsed and validated through `internal/bundle`, which delegates source manifest semantics to `pkg/bundle`. Output records require clean paths, `source` or `generated` kind, valid source paths, lowercase SHA-256 digests, non-negative sizes, created and updated timestamps, and transform ids for generated outputs. Stored URLs must pass `internal/link` validation.
|
||||||
|
|
||||||
The package also provides helpers for finding output records by path, projecting planned publish outputs into timestamped state outputs, merging retained and newly planned output records, and computing managed output paths from single-owner state.
|
The package also provides helpers for finding output records by path, projecting planned publish outputs into timestamped state outputs, merging retained and newly planned output records, and computing managed output paths from single-owner state.
|
||||||
|
|
||||||
|
For shared-root state, helpers parse either state shape, identify the current owner scope, return an owner's latest source manifest, list managed paths for one owner or all owners, detect path ownership conflicts, project planned owner outputs, merge one owner's planned outputs while preserving unrelated owners, and replace one owner's outputs by removing that owner's omitted outputs.
|
||||||
|
|
||||||
## Skip And Resume Behavior
|
## Skip And Resume Behavior
|
||||||
|
|
||||||
Comparison is pure. It returns outcomes for absent state, unmanaged content, invalid state, pipeline/destination mismatch, same source manifest, older destination, newer destination, same-created digest conflict, and different source id conflict. It does not decide whether to skip, replace, force, or fail; publish planning maps outcomes to actions.
|
Comparison is pure. It returns outcomes for absent state, unmanaged content, invalid state, pipeline/destination mismatch, same source manifest, older destination, newer destination, same-created digest conflict, and different source id conflict. It does not decide whether to skip, replace, force, or fail; publish planning maps outcomes to actions.
|
||||||
|
|
||||||
## Failure Behavior
|
## Failure Behavior
|
||||||
|
|
||||||
Parsing rejects invalid JSON, trailing data, missing required fields, invalid timestamps, invalid state mode, invalid reconciliation mode, invalid embedded manifests, duplicate outputs, invalid output paths, unsupported output kinds, missing generated transforms, invalid URLs, invalid digests, and negative sizes.
|
Parsing rejects invalid JSON, trailing data, missing required fields, invalid timestamps, invalid state mode, invalid reconciliation mode, invalid embedded manifests, duplicate owners, duplicate outputs, invalid output paths, unsupported output kinds, missing generated transforms, invalid URLs, invalid digests, negative sizes, and shared-root outputs whose owner is not registered.
|
||||||
|
|
||||||
## Tests To Inspect
|
## Tests To Inspect
|
||||||
|
|
||||||
- `internal/state/distributor_test.go`
|
- `internal/state/distributor_test.go`
|
||||||
|
- `internal/state/shared_root_test.go`
|
||||||
- `internal/state/compare_test.go`
|
- `internal/state/compare_test.go`
|
||||||
- `internal/publish/*_test.go`
|
- `internal/publish/*_test.go`
|
||||||
|
|
||||||
@@ -55,6 +60,8 @@ Parsing rejects invalid JSON, trailing data, missing required fields, invalid ti
|
|||||||
- Embedded source manifests use the source bundle contract.
|
- Embedded source manifests use the source bundle contract.
|
||||||
- Newly written single-owner state uses schema version `2`.
|
- Newly written single-owner state uses schema version `2`.
|
||||||
- Schema version `1` state remains readable as replacement-mode single-owner state.
|
- Schema version `1` state remains readable as replacement-mode single-owner state.
|
||||||
|
- Schema version `3` shared-root state is parsed and validated without converting unrelated single-owner state.
|
||||||
|
- Shared-root owner updates preserve unrelated owners and reject planned path collisions with other owners.
|
||||||
- Generated outputs always record a transform id.
|
- Generated outputs always record a transform id.
|
||||||
- Output records always carry created and updated timestamps after parsing.
|
- Output records always carry created and updated timestamps after parsing.
|
||||||
- Stored URLs are optional and must be absolute HTTP or HTTPS URLs when present.
|
- Stored URLs are optional and must be absolute HTTP or HTTPS URLs when present.
|
||||||
|
|||||||
@@ -94,7 +94,11 @@ The source manifest should remain minimal. Routing, destination selection, publi
|
|||||||
|
|
||||||
`manifest.json` from the source bundle is not copied to destinations as destination state.
|
`manifest.json` from the source bundle is not copied to destinations as destination state.
|
||||||
|
|
||||||
Each destination bundle path is managed by `.distributor.json`. This file is both the destination sentinel and the destination state record. It records:
|
Each destination bundle path is managed by `.distributor.json`. This file is both the destination sentinel and the destination state record.
|
||||||
|
|
||||||
|
Publish execution currently writes single-owner destination state. `internal/state` also parses and validates shared-root destination state, where one `.distributor.json` records multiple pipeline/destination owners and every managed output carries its owner identity.
|
||||||
|
|
||||||
|
Single-owner state records:
|
||||||
|
|
||||||
- `distributor` state schema version;
|
- `distributor` state schema version;
|
||||||
- pipeline id;
|
- pipeline id;
|
||||||
|
|||||||
@@ -55,6 +55,7 @@ type Destination struct {
|
|||||||
Transform Transform `yaml:"transform"`
|
Transform Transform `yaml:"transform"`
|
||||||
PathMap PathMapping `yaml:"path_mapping"`
|
PathMap PathMapping `yaml:"path_mapping"`
|
||||||
Links *Links `yaml:"links"`
|
Links *Links `yaml:"links"`
|
||||||
|
State StatePolicy `yaml:"state"`
|
||||||
Reconciliation ReconciliationPolicy `yaml:"reconciliation"`
|
Reconciliation ReconciliationPolicy `yaml:"reconciliation"`
|
||||||
Transfer TransferPolicy `yaml:"transfer"`
|
Transfer TransferPolicy `yaml:"transfer"`
|
||||||
}
|
}
|
||||||
@@ -123,6 +124,10 @@ type ReconciliationPolicy struct {
|
|||||||
Mode string `yaml:"mode"`
|
Mode string `yaml:"mode"`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
type StatePolicy struct {
|
||||||
|
Mode string `yaml:"mode"`
|
||||||
|
}
|
||||||
|
|
||||||
type TransferPolicy struct {
|
type TransferPolicy struct {
|
||||||
OnDestinationSame string `yaml:"on_destination_same"`
|
OnDestinationSame string `yaml:"on_destination_same"`
|
||||||
OnDestinationOlder string `yaml:"on_destination_older"`
|
OnDestinationOlder string `yaml:"on_destination_older"`
|
||||||
|
|||||||
@@ -47,6 +47,11 @@ const (
|
|||||||
ReconciliationModeMerge = "merge"
|
ReconciliationModeMerge = "merge"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
StateModeSingleOwner = "single_owner"
|
||||||
|
StateModeSharedRoot = "shared_root"
|
||||||
|
)
|
||||||
|
|
||||||
const DefaultS3Region = "us-east-1"
|
const DefaultS3Region = "us-east-1"
|
||||||
|
|
||||||
const (
|
const (
|
||||||
@@ -84,6 +89,9 @@ func ApplyDefaults(cfg *Config) {
|
|||||||
if destination.Links != nil && destination.Links.Primary == "" {
|
if destination.Links != nil && destination.Links.Primary == "" {
|
||||||
destination.Links.Primary = LinkPrimaryAuto
|
destination.Links.Primary = LinkPrimaryAuto
|
||||||
}
|
}
|
||||||
|
if destination.State.Mode == "" {
|
||||||
|
destination.State.Mode = StateModeSingleOwner
|
||||||
|
}
|
||||||
if destination.Reconciliation.Mode == "" {
|
if destination.Reconciliation.Mode == "" {
|
||||||
destination.Reconciliation.Mode = ReconciliationModeReplace
|
destination.Reconciliation.Mode = ReconciliationModeReplace
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -36,6 +36,9 @@ pipelines:
|
|||||||
if got, want := destination.Reconciliation.Mode, ReconciliationModeReplace; got != want {
|
if got, want := destination.Reconciliation.Mode, ReconciliationModeReplace; got != want {
|
||||||
t.Fatalf("reconciliation mode default = %q, want %q", got, want)
|
t.Fatalf("reconciliation mode default = %q, want %q", got, want)
|
||||||
}
|
}
|
||||||
|
if got, want := destination.State.Mode, StateModeSingleOwner; got != want {
|
||||||
|
t.Fatalf("state mode default = %q, want %q", got, want)
|
||||||
|
}
|
||||||
if cfg.Secrets.Directory != "" {
|
if cfg.Secrets.Directory != "" {
|
||||||
t.Fatalf("secrets.directory = %q, want empty", cfg.Secrets.Directory)
|
t.Fatalf("secrets.directory = %q, want empty", cfg.Secrets.Directory)
|
||||||
}
|
}
|
||||||
@@ -196,6 +199,35 @@ pipelines:
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestLoadFileAcceptsExplicitStateModes(t *testing.T) {
|
||||||
|
cfg := loadConfig(t, `
|
||||||
|
pipelines:
|
||||||
|
- id: reports
|
||||||
|
source:
|
||||||
|
backend: local
|
||||||
|
path: /source
|
||||||
|
destinations:
|
||||||
|
- id: archive
|
||||||
|
backend: local
|
||||||
|
path: /archive
|
||||||
|
state:
|
||||||
|
mode: single_owner
|
||||||
|
- id: web
|
||||||
|
backend: local
|
||||||
|
path: /web
|
||||||
|
state:
|
||||||
|
mode: shared_root
|
||||||
|
`)
|
||||||
|
|
||||||
|
destinations := cfg.Pipelines[0].Destinations
|
||||||
|
if got, want := destinations[0].State.Mode, StateModeSingleOwner; got != want {
|
||||||
|
t.Fatalf("archive state mode = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if got, want := destinations[1].State.Mode, StateModeSharedRoot; got != want {
|
||||||
|
t.Fatalf("web state mode = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestLoadFileAcceptsFixedPathMapping(t *testing.T) {
|
func TestLoadFileAcceptsFixedPathMapping(t *testing.T) {
|
||||||
cfg := loadConfig(t, `
|
cfg := loadConfig(t, `
|
||||||
pipelines:
|
pipelines:
|
||||||
|
|||||||
@@ -72,6 +72,7 @@ func Validate(cfg Config) error {
|
|||||||
errs = validatePublishTransformPolicy(errs, destinationContext, destination.Publish, destination.Transform)
|
errs = validatePublishTransformPolicy(errs, destinationContext, destination.Publish, destination.Transform)
|
||||||
errs = validatePathMapping(errs, destinationContext+".path_mapping", destination.PathMap)
|
errs = validatePathMapping(errs, destinationContext+".path_mapping", destination.PathMap)
|
||||||
errs = validateLinks(errs, destinationContext+".links", destination.Links)
|
errs = validateLinks(errs, destinationContext+".links", destination.Links)
|
||||||
|
errs = validateStatePolicy(errs, destinationContext+".state", destination.State)
|
||||||
errs = validateReconciliationPolicy(errs, destinationContext+".reconciliation", destination.Reconciliation)
|
errs = validateReconciliationPolicy(errs, destinationContext+".reconciliation", destination.Reconciliation)
|
||||||
errs = validateTransferPolicy(errs, destinationContext+".transfer", destination.Transfer)
|
errs = validateTransferPolicy(errs, destinationContext+".transfer", destination.Transfer)
|
||||||
}
|
}
|
||||||
@@ -322,6 +323,13 @@ func validateReconciliationPolicy(errs ValidationErrors, context string, policy
|
|||||||
return errs
|
return errs
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func validateStatePolicy(errs ValidationErrors, context string, policy StatePolicy) ValidationErrors {
|
||||||
|
if policy.Mode != StateModeSingleOwner && policy.Mode != StateModeSharedRoot {
|
||||||
|
errs = append(errs, context+".mode must be "+StateModeSingleOwner+" or "+StateModeSharedRoot)
|
||||||
|
}
|
||||||
|
return errs
|
||||||
|
}
|
||||||
|
|
||||||
func validateTransferPolicy(errs ValidationErrors, context string, policy TransferPolicy) ValidationErrors {
|
func validateTransferPolicy(errs ValidationErrors, context string, policy TransferPolicy) ValidationErrors {
|
||||||
if policy.OnDestinationSame != TransferActionSkip && policy.OnDestinationSame != TransferActionFail {
|
if policy.OnDestinationSame != TransferActionSkip && policy.OnDestinationSame != TransferActionFail {
|
||||||
errs = append(errs, context+".on_destination_same must be skip or fail")
|
errs = append(errs, context+".on_destination_same must be skip or fail")
|
||||||
|
|||||||
@@ -141,6 +141,40 @@ func TestValidateReconciliationPolicy(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestValidateStatePolicy(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
mode string
|
||||||
|
wantErr bool
|
||||||
|
}{
|
||||||
|
{name: "single owner", mode: StateModeSingleOwner},
|
||||||
|
{name: "shared root", mode: StateModeSharedRoot},
|
||||||
|
{name: "invalid", mode: "shared", wantErr: true},
|
||||||
|
}
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
cfg := Config{Pipelines: []Pipeline{{
|
||||||
|
ID: "reports",
|
||||||
|
Source: Backend{Backend: BackendLocal, Path: "/source"},
|
||||||
|
Destinations: []Destination{{
|
||||||
|
ID: "archive",
|
||||||
|
Backend: BackendLocal,
|
||||||
|
Path: "/destination",
|
||||||
|
State: StatePolicy{Mode: tt.mode},
|
||||||
|
}},
|
||||||
|
}}}
|
||||||
|
ApplyDefaults(&cfg)
|
||||||
|
err := Validate(cfg)
|
||||||
|
if tt.wantErr && err == nil {
|
||||||
|
t.Fatal("Validate() error = nil, want error")
|
||||||
|
}
|
||||||
|
if !tt.wantErr && err != nil {
|
||||||
|
t.Fatalf("Validate() error = %v", err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestValidateLinks(t *testing.T) {
|
func TestValidateLinks(t *testing.T) {
|
||||||
tests := []struct {
|
tests := []struct {
|
||||||
name string
|
name string
|
||||||
|
|||||||
@@ -12,9 +12,11 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
SchemaVersion = 2
|
SchemaVersion = 2
|
||||||
legacySchemaVersion = 1
|
SharedRootSchemaVersion = 3
|
||||||
StateModeSingleOwner = "single_owner"
|
legacySchemaVersion = 1
|
||||||
|
StateModeSingleOwner = config.StateModeSingleOwner
|
||||||
|
StateModeSharedRoot = config.StateModeSharedRoot
|
||||||
)
|
)
|
||||||
|
|
||||||
type DistributorState struct {
|
type DistributorState struct {
|
||||||
|
|||||||
@@ -3,6 +3,8 @@ package state
|
|||||||
import (
|
import (
|
||||||
"fmt"
|
"fmt"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/distributor/internal/bundle"
|
||||||
)
|
)
|
||||||
|
|
||||||
type OutputProjection struct {
|
type OutputProjection struct {
|
||||||
@@ -80,3 +82,185 @@ func ProjectOutputs(outputs []OutputProjection, existing []OutputFile, now time.
|
|||||||
}
|
}
|
||||||
return files
|
return files
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func CurrentOwnerScope(pipelineID, destinationID string) OwnerScope {
|
||||||
|
return OwnerScope{PipelineID: pipelineID, DestinationID: destinationID}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s SharedRootState) Owner(scope OwnerScope) (OwnerRecord, bool) {
|
||||||
|
for _, owner := range s.Owners {
|
||||||
|
if owner.Scope == scope {
|
||||||
|
return owner, true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return OwnerRecord{}, false
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s SharedRootState) SourceManifest(scope OwnerScope) (bundle.Manifest, bool) {
|
||||||
|
owner, ok := s.Owner(scope)
|
||||||
|
if !ok {
|
||||||
|
return bundle.Manifest{}, false
|
||||||
|
}
|
||||||
|
return owner.Source.Manifest, true
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s SharedRootState) ManagedOutputPaths(scope OwnerScope) []string {
|
||||||
|
paths := make([]string, 0, len(s.Outputs))
|
||||||
|
for _, output := range s.Outputs {
|
||||||
|
if output.Owner == scope {
|
||||||
|
paths = append(paths, output.Path)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return paths
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s SharedRootState) AllManagedOutputPaths() []string {
|
||||||
|
paths := make([]string, 0, len(s.Outputs))
|
||||||
|
for _, output := range s.Outputs {
|
||||||
|
paths = append(paths, output.Path)
|
||||||
|
}
|
||||||
|
return paths
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s SharedRootState) OutputOwner(path string) (OwnerScope, bool) {
|
||||||
|
for _, output := range s.Outputs {
|
||||||
|
if output.Path == path {
|
||||||
|
return output.Owner, true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return OwnerScope{}, false
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s SharedRootState) PathOwnershipConflict(scope OwnerScope, paths []string) (PathOwnershipConflict, bool) {
|
||||||
|
for _, path := range paths {
|
||||||
|
owner, exists := s.OutputOwner(path)
|
||||||
|
if exists && owner != scope {
|
||||||
|
return PathOwnershipConflict{Path: path, Owner: owner}, true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return PathOwnershipConflict{}, false
|
||||||
|
}
|
||||||
|
|
||||||
|
func ProjectSharedRootOutputs(outputs []OutputProjection, existing []SharedRootOutputFile, scope OwnerScope, source bundle.Manifest, now time.Time) []SharedRootOutputFile {
|
||||||
|
now = now.UTC()
|
||||||
|
files := make([]SharedRootOutputFile, 0, len(outputs))
|
||||||
|
for _, output := range outputs {
|
||||||
|
createdAt := now
|
||||||
|
if existingOutput, ok := findSharedRootOutput(existing, output.Path); ok && existingOutput.Owner == scope {
|
||||||
|
createdAt = existingOutput.CreatedAt
|
||||||
|
}
|
||||||
|
files = append(files, SharedRootOutputFile{
|
||||||
|
Path: output.Path,
|
||||||
|
Kind: output.Kind,
|
||||||
|
SourcePath: output.SourcePath,
|
||||||
|
Transform: output.Transform,
|
||||||
|
URL: output.URL,
|
||||||
|
SHA256: output.SHA256,
|
||||||
|
Size: output.Size,
|
||||||
|
Owner: scope,
|
||||||
|
SourceID: source.ID,
|
||||||
|
SourceDigest: source.Digest,
|
||||||
|
SourceCreated: source.Created,
|
||||||
|
CreatedAt: createdAt,
|
||||||
|
UpdatedAt: now,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
return files
|
||||||
|
}
|
||||||
|
|
||||||
|
func ReplaceOwnerOutputs(s SharedRootState, scope OwnerScope, owner OwnerRecord, planned []SharedRootOutputFile) (SharedRootState, error) {
|
||||||
|
if conflict, ok := s.PathOwnershipConflict(scope, sharedRootOutputPaths(planned)); ok {
|
||||||
|
return SharedRootState{}, fmt.Errorf("state output path %q is owned by %s/%s", conflict.Path, conflict.Owner.PipelineID, conflict.Owner.DestinationID)
|
||||||
|
}
|
||||||
|
if err := validatePlannedSharedRootOutputs(scope, planned); err != nil {
|
||||||
|
return SharedRootState{}, err
|
||||||
|
}
|
||||||
|
next := s
|
||||||
|
next.Owners = upsertOwner(s.Owners, owner)
|
||||||
|
next.Outputs = make([]SharedRootOutputFile, 0, len(s.Outputs)+len(planned))
|
||||||
|
for _, output := range s.Outputs {
|
||||||
|
if output.Owner != scope {
|
||||||
|
next.Outputs = append(next.Outputs, output)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
next.Outputs = append(next.Outputs, planned...)
|
||||||
|
return next, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func MergeOwnerOutputs(s SharedRootState, scope OwnerScope, owner OwnerRecord, planned []SharedRootOutputFile) (SharedRootState, error) {
|
||||||
|
if conflict, ok := s.PathOwnershipConflict(scope, sharedRootOutputPaths(planned)); ok {
|
||||||
|
return SharedRootState{}, fmt.Errorf("state output path %q is owned by %s/%s", conflict.Path, conflict.Owner.PipelineID, conflict.Owner.DestinationID)
|
||||||
|
}
|
||||||
|
if err := validatePlannedSharedRootOutputs(scope, planned); err != nil {
|
||||||
|
return SharedRootState{}, err
|
||||||
|
}
|
||||||
|
next := s
|
||||||
|
next.Owners = upsertOwner(s.Owners, owner)
|
||||||
|
outputs := make([]SharedRootOutputFile, 0, len(s.Outputs)+len(planned))
|
||||||
|
indexByPath := make(map[string]int, len(s.Outputs)+len(planned))
|
||||||
|
for _, output := range s.Outputs {
|
||||||
|
indexByPath[output.Path] = len(outputs)
|
||||||
|
outputs = append(outputs, output)
|
||||||
|
}
|
||||||
|
for _, output := range planned {
|
||||||
|
if index, exists := indexByPath[output.Path]; exists {
|
||||||
|
outputs[index] = output
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
indexByPath[output.Path] = len(outputs)
|
||||||
|
outputs = append(outputs, output)
|
||||||
|
}
|
||||||
|
next.Outputs = outputs
|
||||||
|
return next, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func findSharedRootOutput(outputs []SharedRootOutputFile, path string) (SharedRootOutputFile, bool) {
|
||||||
|
for _, output := range outputs {
|
||||||
|
if output.Path == path {
|
||||||
|
return output, true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return SharedRootOutputFile{}, false
|
||||||
|
}
|
||||||
|
|
||||||
|
func sharedRootOutputPaths(outputs []SharedRootOutputFile) []string {
|
||||||
|
paths := make([]string, 0, len(outputs))
|
||||||
|
for _, output := range outputs {
|
||||||
|
paths = append(paths, output.Path)
|
||||||
|
}
|
||||||
|
return paths
|
||||||
|
}
|
||||||
|
|
||||||
|
func rejectDuplicateSharedRootOutputs(outputs []SharedRootOutputFile) error {
|
||||||
|
seen := make(map[string]struct{}, len(outputs))
|
||||||
|
for _, output := range outputs {
|
||||||
|
if _, exists := seen[output.Path]; exists {
|
||||||
|
return fmt.Errorf("state output path %q is duplicated", output.Path)
|
||||||
|
}
|
||||||
|
seen[output.Path] = struct{}{}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func validatePlannedSharedRootOutputs(scope OwnerScope, outputs []SharedRootOutputFile) error {
|
||||||
|
if err := rejectDuplicateSharedRootOutputs(outputs); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for _, output := range outputs {
|
||||||
|
if output.Owner != scope {
|
||||||
|
return fmt.Errorf("state output path %q is owned by %s/%s, not %s/%s", output.Path, output.Owner.PipelineID, output.Owner.DestinationID, scope.PipelineID, scope.DestinationID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func upsertOwner(owners []OwnerRecord, owner OwnerRecord) []OwnerRecord {
|
||||||
|
next := append([]OwnerRecord(nil), owners...)
|
||||||
|
for index, existing := range next {
|
||||||
|
if existing.Scope == owner.Scope {
|
||||||
|
next[index] = owner
|
||||||
|
return next
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return append(next, owner)
|
||||||
|
}
|
||||||
|
|||||||
519
internal/state/shared_root.go
Normal file
519
internal/state/shared_root.go
Normal file
@@ -0,0 +1,519 @@
|
|||||||
|
package state
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/distributor/internal/bundle"
|
||||||
|
"gitea.maximumdirect.net/eric/distributor/internal/config"
|
||||||
|
"gitea.maximumdirect.net/eric/distributor/internal/link"
|
||||||
|
"gitea.maximumdirect.net/eric/distributor/internal/storage"
|
||||||
|
)
|
||||||
|
|
||||||
|
type StateDocument struct {
|
||||||
|
SingleOwner *DistributorState
|
||||||
|
SharedRoot *SharedRootState
|
||||||
|
}
|
||||||
|
|
||||||
|
type SharedRootState struct {
|
||||||
|
SchemaVersion int
|
||||||
|
DistributorVersion string
|
||||||
|
CreatedAt time.Time
|
||||||
|
UpdatedAt time.Time
|
||||||
|
State StatePolicy
|
||||||
|
Owners []OwnerRecord
|
||||||
|
Outputs []SharedRootOutputFile
|
||||||
|
}
|
||||||
|
|
||||||
|
type OwnerScope struct {
|
||||||
|
PipelineID string
|
||||||
|
DestinationID string
|
||||||
|
}
|
||||||
|
|
||||||
|
type OwnerRecord struct {
|
||||||
|
Scope OwnerScope
|
||||||
|
Reconciliation ReconciliationPolicy
|
||||||
|
Source SourceState
|
||||||
|
Links *LinkState
|
||||||
|
}
|
||||||
|
|
||||||
|
type SharedRootOutputFile struct {
|
||||||
|
Path string
|
||||||
|
Kind string
|
||||||
|
SourcePath string
|
||||||
|
Transform string
|
||||||
|
URL string
|
||||||
|
SHA256 string
|
||||||
|
Size int64
|
||||||
|
Owner OwnerScope
|
||||||
|
SourceID string
|
||||||
|
SourceDigest string
|
||||||
|
SourceCreated time.Time
|
||||||
|
CreatedAt time.Time
|
||||||
|
UpdatedAt time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
type PathOwnershipConflict struct {
|
||||||
|
Path string
|
||||||
|
Owner OwnerScope
|
||||||
|
}
|
||||||
|
|
||||||
|
type rawSharedRootState struct {
|
||||||
|
SchemaVersion *int `json:"schema_version"`
|
||||||
|
DistributorVersion string `json:"distributor_version"`
|
||||||
|
CreatedAt *string `json:"created_at"`
|
||||||
|
UpdatedAt *string `json:"updated_at"`
|
||||||
|
State *rawStatePolicy `json:"state"`
|
||||||
|
Owners []rawOwnerRecord `json:"owners"`
|
||||||
|
Outputs []rawSharedRootOutput `json:"outputs"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type rawOwnerRecord struct {
|
||||||
|
PipelineID *string `json:"pipeline_id"`
|
||||||
|
DestinationID *string `json:"destination_id"`
|
||||||
|
Reconciliation *rawReconciliationPolicy `json:"reconciliation"`
|
||||||
|
Source *rawSourceState `json:"source"`
|
||||||
|
Links *rawLinkState `json:"links"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type rawSharedRootOutput struct {
|
||||||
|
Path *string `json:"path"`
|
||||||
|
Kind *string `json:"kind"`
|
||||||
|
SourcePath *string `json:"source_path"`
|
||||||
|
Transform string `json:"transform"`
|
||||||
|
URL string `json:"url"`
|
||||||
|
SHA256 *string `json:"sha256"`
|
||||||
|
Size *int64 `json:"size"`
|
||||||
|
PipelineID *string `json:"pipeline_id"`
|
||||||
|
DestinationID *string `json:"destination_id"`
|
||||||
|
SourceID *string `json:"source_id"`
|
||||||
|
SourceDigest *string `json:"source_digest"`
|
||||||
|
SourceCreated *string `json:"source_created"`
|
||||||
|
CreatedAt *string `json:"created_at"`
|
||||||
|
UpdatedAt *string `json:"updated_at"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func ParseDocument(data []byte) (StateDocument, error) {
|
||||||
|
schemaVersion, err := parseSchemaVersion(data)
|
||||||
|
if err != nil {
|
||||||
|
return StateDocument{}, err
|
||||||
|
}
|
||||||
|
if schemaVersion == SharedRootSchemaVersion {
|
||||||
|
sharedRoot, err := ParseSharedRoot(data)
|
||||||
|
if err != nil {
|
||||||
|
return StateDocument{}, err
|
||||||
|
}
|
||||||
|
return StateDocument{SharedRoot: &sharedRoot}, nil
|
||||||
|
}
|
||||||
|
singleOwner, err := Parse(data)
|
||||||
|
if err != nil {
|
||||||
|
return StateDocument{}, err
|
||||||
|
}
|
||||||
|
return StateDocument{SingleOwner: &singleOwner}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseSchemaVersion(data []byte) (int, error) {
|
||||||
|
decoder := json.NewDecoder(bytes.NewReader(data))
|
||||||
|
var raw struct {
|
||||||
|
SchemaVersion *int `json:"schema_version"`
|
||||||
|
}
|
||||||
|
if err := decoder.Decode(&raw); err != nil {
|
||||||
|
return 0, fmt.Errorf("parse distributor state: %w", err)
|
||||||
|
}
|
||||||
|
if raw.SchemaVersion == nil {
|
||||||
|
return 0, fmt.Errorf("state schema_version is required")
|
||||||
|
}
|
||||||
|
return *raw.SchemaVersion, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func ParseSharedRoot(data []byte) (SharedRootState, error) {
|
||||||
|
decoder := json.NewDecoder(bytes.NewReader(data))
|
||||||
|
var raw rawSharedRootState
|
||||||
|
if err := decoder.Decode(&raw); err != nil {
|
||||||
|
return SharedRootState{}, fmt.Errorf("parse distributor state: %w", err)
|
||||||
|
}
|
||||||
|
var extra any
|
||||||
|
if err := decoder.Decode(&extra); err != io.EOF {
|
||||||
|
return SharedRootState{}, fmt.Errorf("parse distributor state: trailing data")
|
||||||
|
}
|
||||||
|
state, err := parseSharedRootRaw(raw)
|
||||||
|
if err != nil {
|
||||||
|
return SharedRootState{}, err
|
||||||
|
}
|
||||||
|
if err := ValidateSharedRoot(state); err != nil {
|
||||||
|
return SharedRootState{}, err
|
||||||
|
}
|
||||||
|
return state, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseSharedRootRaw(raw rawSharedRootState) (SharedRootState, error) {
|
||||||
|
if raw.SchemaVersion == nil {
|
||||||
|
return SharedRootState{}, fmt.Errorf("state schema_version is required")
|
||||||
|
}
|
||||||
|
state := SharedRootState{SchemaVersion: *raw.SchemaVersion}
|
||||||
|
if state.SchemaVersion != SharedRootSchemaVersion {
|
||||||
|
return SharedRootState{}, fmt.Errorf("state schema_version must be %d", SharedRootSchemaVersion)
|
||||||
|
}
|
||||||
|
state.DistributorVersion = raw.DistributorVersion
|
||||||
|
createdAt, err := parseRequiredTime("state created_at", raw.CreatedAt)
|
||||||
|
if err != nil {
|
||||||
|
return SharedRootState{}, err
|
||||||
|
}
|
||||||
|
updatedAt, err := parseRequiredTime("state updated_at", raw.UpdatedAt)
|
||||||
|
if err != nil {
|
||||||
|
return SharedRootState{}, err
|
||||||
|
}
|
||||||
|
state.CreatedAt = createdAt
|
||||||
|
state.UpdatedAt = updatedAt
|
||||||
|
if raw.State == nil || raw.State.Mode == "" {
|
||||||
|
return SharedRootState{}, fmt.Errorf("state state.mode is required")
|
||||||
|
}
|
||||||
|
state.State.Mode = raw.State.Mode
|
||||||
|
if raw.Owners == nil {
|
||||||
|
return SharedRootState{}, fmt.Errorf("state owners is required")
|
||||||
|
}
|
||||||
|
owners, err := parseOwnerRecords(raw.Owners)
|
||||||
|
if err != nil {
|
||||||
|
return SharedRootState{}, err
|
||||||
|
}
|
||||||
|
state.Owners = owners
|
||||||
|
if raw.Outputs == nil {
|
||||||
|
return SharedRootState{}, fmt.Errorf("state outputs is required")
|
||||||
|
}
|
||||||
|
outputs, err := parseSharedRootOutputs(raw.Outputs)
|
||||||
|
if err != nil {
|
||||||
|
return SharedRootState{}, err
|
||||||
|
}
|
||||||
|
state.Outputs = outputs
|
||||||
|
return state, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseOwnerRecords(rawOwners []rawOwnerRecord) ([]OwnerRecord, error) {
|
||||||
|
owners := make([]OwnerRecord, 0, len(rawOwners))
|
||||||
|
for index, raw := range rawOwners {
|
||||||
|
owner, err := parseOwnerRecord(index, raw)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
owners = append(owners, owner)
|
||||||
|
}
|
||||||
|
return owners, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseOwnerRecord(index int, raw rawOwnerRecord) (OwnerRecord, error) {
|
||||||
|
if raw.PipelineID == nil || *raw.PipelineID == "" {
|
||||||
|
return OwnerRecord{}, fmt.Errorf("state owners[%d].pipeline_id is required", index)
|
||||||
|
}
|
||||||
|
if raw.DestinationID == nil || *raw.DestinationID == "" {
|
||||||
|
return OwnerRecord{}, fmt.Errorf("state owners[%d].destination_id is required", index)
|
||||||
|
}
|
||||||
|
if raw.Reconciliation == nil || raw.Reconciliation.Mode == "" {
|
||||||
|
return OwnerRecord{}, fmt.Errorf("state owners[%d].reconciliation.mode is required", index)
|
||||||
|
}
|
||||||
|
if raw.Source == nil || len(raw.Source.Manifest) == 0 {
|
||||||
|
return OwnerRecord{}, fmt.Errorf("state owners[%d].source.manifest is required", index)
|
||||||
|
}
|
||||||
|
manifest, err := bundle.ParseManifest(raw.Source.Manifest)
|
||||||
|
if err != nil {
|
||||||
|
return OwnerRecord{}, fmt.Errorf("state owners[%d].source.manifest: %w", index, err)
|
||||||
|
}
|
||||||
|
owner := OwnerRecord{
|
||||||
|
Scope: OwnerScope{
|
||||||
|
PipelineID: *raw.PipelineID,
|
||||||
|
DestinationID: *raw.DestinationID,
|
||||||
|
},
|
||||||
|
Reconciliation: ReconciliationPolicy{Mode: raw.Reconciliation.Mode},
|
||||||
|
Source: SourceState{Manifest: manifest},
|
||||||
|
}
|
||||||
|
if raw.Links != nil {
|
||||||
|
owner.Links = &LinkState{PrimaryURL: raw.Links.PrimaryURL}
|
||||||
|
}
|
||||||
|
return owner, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseSharedRootOutputs(rawOutputs []rawSharedRootOutput) ([]SharedRootOutputFile, error) {
|
||||||
|
outputs := make([]SharedRootOutputFile, 0, len(rawOutputs))
|
||||||
|
for index, raw := range rawOutputs {
|
||||||
|
output, err := parseSharedRootOutput(index, raw)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
outputs = append(outputs, output)
|
||||||
|
}
|
||||||
|
return outputs, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseSharedRootOutput(index int, raw rawSharedRootOutput) (SharedRootOutputFile, error) {
|
||||||
|
if raw.Path == nil || *raw.Path == "" {
|
||||||
|
return SharedRootOutputFile{}, fmt.Errorf("state outputs[%d].path is required", index)
|
||||||
|
}
|
||||||
|
if raw.Kind == nil || *raw.Kind == "" {
|
||||||
|
return SharedRootOutputFile{}, fmt.Errorf("state outputs[%d].kind is required", index)
|
||||||
|
}
|
||||||
|
if raw.SourcePath == nil || *raw.SourcePath == "" {
|
||||||
|
return SharedRootOutputFile{}, fmt.Errorf("state outputs[%d].source_path is required", index)
|
||||||
|
}
|
||||||
|
if raw.SHA256 == nil || *raw.SHA256 == "" {
|
||||||
|
return SharedRootOutputFile{}, fmt.Errorf("state outputs[%d].sha256 is required", index)
|
||||||
|
}
|
||||||
|
if raw.Size == nil {
|
||||||
|
return SharedRootOutputFile{}, fmt.Errorf("state outputs[%d].size is required", index)
|
||||||
|
}
|
||||||
|
if raw.PipelineID == nil || *raw.PipelineID == "" {
|
||||||
|
return SharedRootOutputFile{}, fmt.Errorf("state outputs[%d].pipeline_id is required", index)
|
||||||
|
}
|
||||||
|
if raw.DestinationID == nil || *raw.DestinationID == "" {
|
||||||
|
return SharedRootOutputFile{}, fmt.Errorf("state outputs[%d].destination_id is required", index)
|
||||||
|
}
|
||||||
|
if raw.SourceID == nil || *raw.SourceID == "" {
|
||||||
|
return SharedRootOutputFile{}, fmt.Errorf("state outputs[%d].source_id is required", index)
|
||||||
|
}
|
||||||
|
if raw.SourceDigest == nil || *raw.SourceDigest == "" {
|
||||||
|
return SharedRootOutputFile{}, fmt.Errorf("state outputs[%d].source_digest is required", index)
|
||||||
|
}
|
||||||
|
sourceCreated, err := parseRequiredTime(fmt.Sprintf("state outputs[%d].source_created", index), raw.SourceCreated)
|
||||||
|
if err != nil {
|
||||||
|
return SharedRootOutputFile{}, err
|
||||||
|
}
|
||||||
|
createdAt, err := parseRequiredTime(fmt.Sprintf("state outputs[%d].created_at", index), raw.CreatedAt)
|
||||||
|
if err != nil {
|
||||||
|
return SharedRootOutputFile{}, err
|
||||||
|
}
|
||||||
|
updatedAt, err := parseRequiredTime(fmt.Sprintf("state outputs[%d].updated_at", index), raw.UpdatedAt)
|
||||||
|
if err != nil {
|
||||||
|
return SharedRootOutputFile{}, err
|
||||||
|
}
|
||||||
|
return SharedRootOutputFile{
|
||||||
|
Path: *raw.Path,
|
||||||
|
Kind: *raw.Kind,
|
||||||
|
SourcePath: *raw.SourcePath,
|
||||||
|
Transform: raw.Transform,
|
||||||
|
URL: raw.URL,
|
||||||
|
SHA256: *raw.SHA256,
|
||||||
|
Size: *raw.Size,
|
||||||
|
Owner: OwnerScope{
|
||||||
|
PipelineID: *raw.PipelineID,
|
||||||
|
DestinationID: *raw.DestinationID,
|
||||||
|
},
|
||||||
|
SourceID: *raw.SourceID,
|
||||||
|
SourceDigest: *raw.SourceDigest,
|
||||||
|
SourceCreated: sourceCreated,
|
||||||
|
CreatedAt: createdAt,
|
||||||
|
UpdatedAt: updatedAt,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s SharedRootState) CreatedAtString() string {
|
||||||
|
return s.CreatedAt.UTC().Format(time.RFC3339)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s SharedRootState) UpdatedAtString() string {
|
||||||
|
return s.UpdatedAt.UTC().Format(time.RFC3339)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (o SharedRootOutputFile) SourceCreatedString() string {
|
||||||
|
return o.SourceCreated.UTC().Format(time.RFC3339)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (o SharedRootOutputFile) CreatedAtString() string {
|
||||||
|
return o.CreatedAt.UTC().Format(time.RFC3339)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (o SharedRootOutputFile) UpdatedAtString() string {
|
||||||
|
return o.UpdatedAt.UTC().Format(time.RFC3339)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s SharedRootState) MarshalJSON() ([]byte, error) {
|
||||||
|
type stateJSON struct {
|
||||||
|
SchemaVersion int `json:"schema_version"`
|
||||||
|
DistributorVersion string `json:"distributor_version,omitempty"`
|
||||||
|
CreatedAt string `json:"created_at"`
|
||||||
|
UpdatedAt string `json:"updated_at"`
|
||||||
|
State StatePolicy `json:"state"`
|
||||||
|
Owners []OwnerRecord `json:"owners"`
|
||||||
|
Outputs []SharedRootOutputFile `json:"outputs"`
|
||||||
|
}
|
||||||
|
return json.Marshal(stateJSON{
|
||||||
|
SchemaVersion: s.SchemaVersion,
|
||||||
|
DistributorVersion: s.DistributorVersion,
|
||||||
|
CreatedAt: s.CreatedAtString(),
|
||||||
|
UpdatedAt: s.UpdatedAtString(),
|
||||||
|
State: s.State,
|
||||||
|
Owners: s.Owners,
|
||||||
|
Outputs: s.Outputs,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (o OwnerRecord) MarshalJSON() ([]byte, error) {
|
||||||
|
type sourceJSON struct {
|
||||||
|
Manifest bundle.Manifest `json:"manifest"`
|
||||||
|
}
|
||||||
|
type ownerJSON struct {
|
||||||
|
PipelineID string `json:"pipeline_id"`
|
||||||
|
DestinationID string `json:"destination_id"`
|
||||||
|
Reconciliation ReconciliationPolicy `json:"reconciliation"`
|
||||||
|
Source sourceJSON `json:"source"`
|
||||||
|
Links *LinkState `json:"links,omitempty"`
|
||||||
|
}
|
||||||
|
return json.Marshal(ownerJSON{
|
||||||
|
PipelineID: o.Scope.PipelineID,
|
||||||
|
DestinationID: o.Scope.DestinationID,
|
||||||
|
Reconciliation: o.Reconciliation,
|
||||||
|
Source: sourceJSON{Manifest: o.Source.Manifest},
|
||||||
|
Links: o.Links,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (o SharedRootOutputFile) MarshalJSON() ([]byte, error) {
|
||||||
|
type outputJSON struct {
|
||||||
|
Path string `json:"path"`
|
||||||
|
Kind string `json:"kind"`
|
||||||
|
SourcePath string `json:"source_path"`
|
||||||
|
Transform string `json:"transform,omitempty"`
|
||||||
|
URL string `json:"url,omitempty"`
|
||||||
|
SHA256 string `json:"sha256"`
|
||||||
|
Size int64 `json:"size"`
|
||||||
|
PipelineID string `json:"pipeline_id"`
|
||||||
|
DestinationID string `json:"destination_id"`
|
||||||
|
SourceID string `json:"source_id"`
|
||||||
|
SourceDigest string `json:"source_digest"`
|
||||||
|
SourceCreated string `json:"source_created"`
|
||||||
|
CreatedAt string `json:"created_at"`
|
||||||
|
UpdatedAt string `json:"updated_at"`
|
||||||
|
}
|
||||||
|
return json.Marshal(outputJSON{
|
||||||
|
Path: o.Path,
|
||||||
|
Kind: o.Kind,
|
||||||
|
SourcePath: o.SourcePath,
|
||||||
|
Transform: o.Transform,
|
||||||
|
URL: o.URL,
|
||||||
|
SHA256: o.SHA256,
|
||||||
|
Size: o.Size,
|
||||||
|
PipelineID: o.Owner.PipelineID,
|
||||||
|
DestinationID: o.Owner.DestinationID,
|
||||||
|
SourceID: o.SourceID,
|
||||||
|
SourceDigest: o.SourceDigest,
|
||||||
|
SourceCreated: o.SourceCreatedString(),
|
||||||
|
CreatedAt: o.CreatedAtString(),
|
||||||
|
UpdatedAt: o.UpdatedAtString(),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func ValidateSharedRoot(s SharedRootState) error {
|
||||||
|
if s.SchemaVersion != SharedRootSchemaVersion {
|
||||||
|
return fmt.Errorf("state schema_version must be %d", SharedRootSchemaVersion)
|
||||||
|
}
|
||||||
|
if s.CreatedAt.IsZero() {
|
||||||
|
return fmt.Errorf("state created_at is required")
|
||||||
|
}
|
||||||
|
if s.UpdatedAt.IsZero() {
|
||||||
|
return fmt.Errorf("state updated_at is required")
|
||||||
|
}
|
||||||
|
if s.State.Mode != StateModeSharedRoot {
|
||||||
|
return fmt.Errorf("state state.mode must be %s", StateModeSharedRoot)
|
||||||
|
}
|
||||||
|
if s.Owners == nil {
|
||||||
|
return fmt.Errorf("state owners is required")
|
||||||
|
}
|
||||||
|
owners := make(map[OwnerScope]OwnerRecord, len(s.Owners))
|
||||||
|
for index, owner := range s.Owners {
|
||||||
|
if err := validateOwnerRecord(index, owner); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if _, exists := owners[owner.Scope]; exists {
|
||||||
|
return fmt.Errorf("state owners[%d] duplicates owner %s/%s", index, owner.Scope.PipelineID, owner.Scope.DestinationID)
|
||||||
|
}
|
||||||
|
owners[owner.Scope] = owner
|
||||||
|
}
|
||||||
|
if s.Outputs == nil {
|
||||||
|
return fmt.Errorf("state outputs is required")
|
||||||
|
}
|
||||||
|
seenPaths := make(map[string]struct{}, len(s.Outputs))
|
||||||
|
for index, output := range s.Outputs {
|
||||||
|
if err := validateSharedRootOutput(index, output, owners); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if _, exists := seenPaths[output.Path]; exists {
|
||||||
|
return fmt.Errorf("state outputs[%d].path duplicates %q", index, output.Path)
|
||||||
|
}
|
||||||
|
seenPaths[output.Path] = struct{}{}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateOwnerRecord(index int, owner OwnerRecord) error {
|
||||||
|
if owner.Scope.PipelineID == "" {
|
||||||
|
return fmt.Errorf("state owners[%d].pipeline_id is required", index)
|
||||||
|
}
|
||||||
|
if owner.Scope.DestinationID == "" {
|
||||||
|
return fmt.Errorf("state owners[%d].destination_id is required", index)
|
||||||
|
}
|
||||||
|
if owner.Reconciliation.Mode != config.ReconciliationModeReplace && owner.Reconciliation.Mode != config.ReconciliationModeMerge {
|
||||||
|
return fmt.Errorf("state owners[%d].reconciliation.mode must be %s or %s", index, config.ReconciliationModeReplace, config.ReconciliationModeMerge)
|
||||||
|
}
|
||||||
|
if err := validateEmbeddedManifest(owner.Source.Manifest); err != nil {
|
||||||
|
return fmt.Errorf("state owners[%d].source.manifest: %w", index, err)
|
||||||
|
}
|
||||||
|
if owner.Links != nil && owner.Links.PrimaryURL != "" {
|
||||||
|
if err := link.ValidateHTTPURL(owner.Links.PrimaryURL); err != nil {
|
||||||
|
return fmt.Errorf("state owners[%d].links.primary_url: %w", index, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateSharedRootOutput(index int, output SharedRootOutputFile, owners map[OwnerScope]OwnerRecord) error {
|
||||||
|
if err := storage.ValidatePath(output.Path); err != nil {
|
||||||
|
return fmt.Errorf("state outputs[%d].path: %w", index, err)
|
||||||
|
}
|
||||||
|
switch output.Kind {
|
||||||
|
case OutputKindSource, OutputKindGenerated:
|
||||||
|
default:
|
||||||
|
return fmt.Errorf("state outputs[%d].kind must be source or generated", index)
|
||||||
|
}
|
||||||
|
if err := storage.ValidatePath(output.SourcePath); err != nil {
|
||||||
|
return fmt.Errorf("state outputs[%d].source_path: %w", index, err)
|
||||||
|
}
|
||||||
|
if output.Kind == OutputKindGenerated && output.Transform == "" {
|
||||||
|
return fmt.Errorf("state outputs[%d].transform is required for generated output", index)
|
||||||
|
}
|
||||||
|
if output.URL != "" {
|
||||||
|
if err := link.ValidateHTTPURL(output.URL); err != nil {
|
||||||
|
return fmt.Errorf("state outputs[%d].url: %w", index, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := bundle.ValidateDigest(output.SHA256); err != nil {
|
||||||
|
return fmt.Errorf("state outputs[%d].sha256: %w", index, err)
|
||||||
|
}
|
||||||
|
if output.Size < 0 {
|
||||||
|
return fmt.Errorf("state outputs[%d].size must be non-negative", index)
|
||||||
|
}
|
||||||
|
if output.Owner.PipelineID == "" {
|
||||||
|
return fmt.Errorf("state outputs[%d].pipeline_id is required", index)
|
||||||
|
}
|
||||||
|
if output.Owner.DestinationID == "" {
|
||||||
|
return fmt.Errorf("state outputs[%d].destination_id is required", index)
|
||||||
|
}
|
||||||
|
if _, exists := owners[output.Owner]; !exists {
|
||||||
|
return fmt.Errorf("state outputs[%d] references unknown owner %s/%s", index, output.Owner.PipelineID, output.Owner.DestinationID)
|
||||||
|
}
|
||||||
|
if output.SourceID == "" {
|
||||||
|
return fmt.Errorf("state outputs[%d].source_id is required", index)
|
||||||
|
}
|
||||||
|
if err := bundle.ValidateDigest(output.SourceDigest); err != nil {
|
||||||
|
return fmt.Errorf("state outputs[%d].source_digest: %w", index, err)
|
||||||
|
}
|
||||||
|
if output.SourceCreated.IsZero() {
|
||||||
|
return fmt.Errorf("state outputs[%d].source_created is required", index)
|
||||||
|
}
|
||||||
|
if output.CreatedAt.IsZero() {
|
||||||
|
return fmt.Errorf("state outputs[%d].created_at is required", index)
|
||||||
|
}
|
||||||
|
if output.UpdatedAt.IsZero() {
|
||||||
|
return fmt.Errorf("state outputs[%d].updated_at is required", index)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
262
internal/state/shared_root_test.go
Normal file
262
internal/state/shared_root_test.go
Normal file
@@ -0,0 +1,262 @@
|
|||||||
|
package state
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/distributor/internal/bundle"
|
||||||
|
"gitea.maximumdirect.net/eric/distributor/internal/config"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestParseDocumentHandlesSingleOwnerAndSharedRoot(t *testing.T) {
|
||||||
|
singleOwner, err := ParseDocument([]byte(validStateJSON(t)))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ParseDocument(single owner) error = %v", err)
|
||||||
|
}
|
||||||
|
if singleOwner.SingleOwner == nil || singleOwner.SharedRoot != nil {
|
||||||
|
t.Fatalf("single owner document = %#v", singleOwner)
|
||||||
|
}
|
||||||
|
|
||||||
|
sharedRoot, err := ParseDocument([]byte(validSharedRootStateJSON(t)))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ParseDocument(shared root) error = %v", err)
|
||||||
|
}
|
||||||
|
if sharedRoot.SharedRoot == nil || sharedRoot.SingleOwner != nil {
|
||||||
|
t.Fatalf("shared root document = %#v", sharedRoot)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestParseSharedRootState(t *testing.T) {
|
||||||
|
state, err := ParseSharedRoot([]byte(validSharedRootStateJSON(t)))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ParseSharedRoot() error = %v", err)
|
||||||
|
}
|
||||||
|
if got, want := state.SchemaVersion, SharedRootSchemaVersion; got != want {
|
||||||
|
t.Fatalf("schema version = %d, want %d", got, want)
|
||||||
|
}
|
||||||
|
if got, want := state.CreatedAtString(), "2026-05-30T11:12:00Z"; got != want {
|
||||||
|
t.Fatalf("created_at = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if got, want := state.State.Mode, StateModeSharedRoot; got != want {
|
||||||
|
t.Fatalf("state mode = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if got, want := len(state.Owners), 2; got != want {
|
||||||
|
t.Fatalf("owner count = %d, want %d", got, want)
|
||||||
|
}
|
||||||
|
if got, want := len(state.Outputs), 2; got != want {
|
||||||
|
t.Fatalf("output count = %d, want %d", got, want)
|
||||||
|
}
|
||||||
|
scope := CurrentOwnerScope("reports", "archive")
|
||||||
|
manifest, ok := state.SourceManifest(scope)
|
||||||
|
if !ok {
|
||||||
|
t.Fatal("SourceManifest() ok = false, want true")
|
||||||
|
}
|
||||||
|
if got, want := manifest.ID, validManifest(t).ID; got != want {
|
||||||
|
t.Fatalf("source manifest id = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestParseSharedRootRejectsInvalidMetadata(t *testing.T) {
|
||||||
|
tests := map[string]func(string) string{
|
||||||
|
"schema": func(body string) string {
|
||||||
|
return strings.Replace(body, `"schema_version": 3`, `"schema_version": 2`, 1)
|
||||||
|
},
|
||||||
|
"state mode": func(body string) string {
|
||||||
|
return strings.Replace(body, `"mode": "shared_root"`, `"mode": "single_owner"`, 1)
|
||||||
|
},
|
||||||
|
"duplicate owner": func(body string) string {
|
||||||
|
return strings.Replace(body, `"destination_id": "html"`, `"destination_id": "archive"`, 1)
|
||||||
|
},
|
||||||
|
"unknown output owner": func(body string) string {
|
||||||
|
return strings.Replace(body, `"destination_id": "html",`, `"destination_id": "missing",`, 1)
|
||||||
|
},
|
||||||
|
"duplicate output": func(body string) string {
|
||||||
|
return strings.Replace(body, `"path": "report.html"`, `"path": "report.md"`, 1)
|
||||||
|
},
|
||||||
|
"invalid source digest": func(body string) string {
|
||||||
|
return strings.Replace(body, `"source_digest": "sha256:`, `"source_digest": "SHA256:`, 1)
|
||||||
|
},
|
||||||
|
"invalid owner link": func(body string) string {
|
||||||
|
return strings.Replace(body, `"primary_url": "https://reports.example.com/archive/report.md"`, `"primary_url": "file:///tmp/report.md"`, 1)
|
||||||
|
},
|
||||||
|
}
|
||||||
|
for name, mutate := range tests {
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
_, err := ParseSharedRoot([]byte(mutate(validSharedRootStateJSON(t))))
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("ParseSharedRoot() error = nil, want error")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSharedRootMarshalNormalizesTimestamps(t *testing.T) {
|
||||||
|
state := validSharedRootState(t)
|
||||||
|
state.CreatedAt = time.Date(2026, 5, 30, 13, 12, 0, 0, time.FixedZone("offset", 2*60*60))
|
||||||
|
state.UpdatedAt = state.CreatedAt
|
||||||
|
state.Outputs[0].SourceCreated = state.CreatedAt
|
||||||
|
state.Outputs[0].CreatedAt = state.CreatedAt
|
||||||
|
state.Outputs[0].UpdatedAt = state.CreatedAt
|
||||||
|
|
||||||
|
data, err := json.Marshal(state)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Marshal() error = %v", err)
|
||||||
|
}
|
||||||
|
for _, want := range []string{
|
||||||
|
`"created_at":"2026-05-30T11:12:00Z"`,
|
||||||
|
`"updated_at":"2026-05-30T11:12:00Z"`,
|
||||||
|
`"source_created":"2026-05-30T11:12:00Z"`,
|
||||||
|
} {
|
||||||
|
if !strings.Contains(string(data), want) {
|
||||||
|
t.Fatalf("json = %s, want %s", data, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSharedRootOutputHelpers(t *testing.T) {
|
||||||
|
state := validSharedRootState(t)
|
||||||
|
archive := CurrentOwnerScope("reports", "archive")
|
||||||
|
html := CurrentOwnerScope("reports", "html")
|
||||||
|
|
||||||
|
if got, want := state.ManagedOutputPaths(archive), []string{"report.md"}; strings.Join(got, ",") != strings.Join(want, ",") {
|
||||||
|
t.Fatalf("archive paths = %#v, want %#v", got, want)
|
||||||
|
}
|
||||||
|
if got, want := state.AllManagedOutputPaths(), []string{"report.md", "report.html"}; strings.Join(got, ",") != strings.Join(want, ",") {
|
||||||
|
t.Fatalf("all paths = %#v, want %#v", got, want)
|
||||||
|
}
|
||||||
|
conflict, ok := state.PathOwnershipConflict(archive, []string{"report.html"})
|
||||||
|
if !ok || conflict.Owner != html {
|
||||||
|
t.Fatalf("conflict = %#v ok=%t, want html owner conflict", conflict, ok)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSharedRootProjectAndMergeOwnerOutputs(t *testing.T) {
|
||||||
|
state := validSharedRootState(t)
|
||||||
|
archive := CurrentOwnerScope("reports", "archive")
|
||||||
|
owner, ok := state.Owner(archive)
|
||||||
|
if !ok {
|
||||||
|
t.Fatal("Owner() ok = false, want true")
|
||||||
|
}
|
||||||
|
now := time.Date(2026, 5, 30, 12, 30, 0, 0, time.UTC)
|
||||||
|
planned := ProjectSharedRootOutputs([]OutputProjection{{
|
||||||
|
Path: "report.md",
|
||||||
|
Kind: OutputKindSource,
|
||||||
|
SourcePath: "report.md",
|
||||||
|
SHA256: validManifest(t).Files[0].SHA256,
|
||||||
|
Size: validManifest(t).Files[0].Size,
|
||||||
|
}, {
|
||||||
|
Path: "summary.txt",
|
||||||
|
Kind: OutputKindSource,
|
||||||
|
SourcePath: "summary.txt",
|
||||||
|
SHA256: validManifest(t).Files[1].SHA256,
|
||||||
|
Size: validManifest(t).Files[1].Size,
|
||||||
|
}}, state.Outputs, archive, validManifest(t), now)
|
||||||
|
|
||||||
|
merged, err := MergeOwnerOutputs(state, archive, owner, planned)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("MergeOwnerOutputs() error = %v", err)
|
||||||
|
}
|
||||||
|
if got, want := merged.AllManagedOutputPaths(), []string{"report.md", "report.html", "summary.txt"}; strings.Join(got, ",") != strings.Join(want, ",") {
|
||||||
|
t.Fatalf("merged paths = %#v, want %#v", got, want)
|
||||||
|
}
|
||||||
|
if merged.Outputs[0].CreatedAt.Equal(now) {
|
||||||
|
t.Fatalf("merged updated output created_at = %s, want preserved timestamp", merged.Outputs[0].CreatedAt)
|
||||||
|
}
|
||||||
|
if !merged.Outputs[0].UpdatedAt.Equal(now) {
|
||||||
|
t.Fatalf("merged updated output updated_at = %s, want %s", merged.Outputs[0].UpdatedAt, now)
|
||||||
|
}
|
||||||
|
|
||||||
|
replaced, err := ReplaceOwnerOutputs(state, archive, owner, planned)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ReplaceOwnerOutputs() error = %v", err)
|
||||||
|
}
|
||||||
|
if got, want := replaced.AllManagedOutputPaths(), []string{"report.html", "report.md", "summary.txt"}; strings.Join(got, ",") != strings.Join(want, ",") {
|
||||||
|
t.Fatalf("replaced paths = %#v, want %#v", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSharedRootOwnerOutputHelpersRejectConflicts(t *testing.T) {
|
||||||
|
state := validSharedRootState(t)
|
||||||
|
archive := CurrentOwnerScope("reports", "archive")
|
||||||
|
owner, ok := state.Owner(archive)
|
||||||
|
if !ok {
|
||||||
|
t.Fatal("Owner() ok = false, want true")
|
||||||
|
}
|
||||||
|
planned := []SharedRootOutputFile{{
|
||||||
|
Path: "report.html",
|
||||||
|
Kind: OutputKindSource,
|
||||||
|
Owner: archive,
|
||||||
|
CreatedAt: time.Date(2026, 5, 30, 12, 30, 0, 0, time.UTC),
|
||||||
|
UpdatedAt: time.Date(2026, 5, 30, 12, 30, 0, 0, time.UTC),
|
||||||
|
}}
|
||||||
|
|
||||||
|
if _, err := MergeOwnerOutputs(state, archive, owner, planned); err == nil {
|
||||||
|
t.Fatal("MergeOwnerOutputs() error = nil, want owner conflict")
|
||||||
|
}
|
||||||
|
if _, err := ReplaceOwnerOutputs(state, archive, owner, planned); err == nil {
|
||||||
|
t.Fatal("ReplaceOwnerOutputs() error = nil, want owner conflict")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func validSharedRootStateJSON(t *testing.T) string {
|
||||||
|
t.Helper()
|
||||||
|
data, err := json.MarshalIndent(validSharedRootState(t), "", " ")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("marshal shared root state: %v", err)
|
||||||
|
}
|
||||||
|
return string(data)
|
||||||
|
}
|
||||||
|
|
||||||
|
func validSharedRootState(t *testing.T) SharedRootState {
|
||||||
|
t.Helper()
|
||||||
|
source := validManifest(t)
|
||||||
|
htmlSource := source
|
||||||
|
htmlSource.Files = append([]bundle.ManifestFile(nil), source.Files...)
|
||||||
|
createdAt := time.Date(2026, 5, 30, 11, 12, 0, 0, time.UTC)
|
||||||
|
return SharedRootState{
|
||||||
|
SchemaVersion: SharedRootSchemaVersion,
|
||||||
|
DistributorVersion: "dev",
|
||||||
|
CreatedAt: createdAt,
|
||||||
|
UpdatedAt: createdAt,
|
||||||
|
State: StatePolicy{Mode: StateModeSharedRoot},
|
||||||
|
Owners: []OwnerRecord{{
|
||||||
|
Scope: CurrentOwnerScope("reports", "archive"),
|
||||||
|
Reconciliation: ReconciliationPolicy{Mode: config.ReconciliationModeReplace},
|
||||||
|
Source: SourceState{Manifest: source},
|
||||||
|
Links: &LinkState{PrimaryURL: "https://reports.example.com/archive/report.md"},
|
||||||
|
}, {
|
||||||
|
Scope: CurrentOwnerScope("reports", "html"),
|
||||||
|
Reconciliation: ReconciliationPolicy{Mode: config.ReconciliationModeMerge},
|
||||||
|
Source: SourceState{Manifest: htmlSource},
|
||||||
|
}},
|
||||||
|
Outputs: []SharedRootOutputFile{{
|
||||||
|
Path: "report.md",
|
||||||
|
Kind: OutputKindSource,
|
||||||
|
SourcePath: "report.md",
|
||||||
|
SHA256: source.Files[0].SHA256,
|
||||||
|
Size: source.Files[0].Size,
|
||||||
|
Owner: CurrentOwnerScope("reports", "archive"),
|
||||||
|
SourceID: source.ID,
|
||||||
|
SourceDigest: source.Digest,
|
||||||
|
SourceCreated: source.Created,
|
||||||
|
CreatedAt: createdAt,
|
||||||
|
UpdatedAt: createdAt,
|
||||||
|
}, {
|
||||||
|
Path: "report.html",
|
||||||
|
Kind: OutputKindGenerated,
|
||||||
|
SourcePath: "report.md",
|
||||||
|
Transform: "markdown_to_html",
|
||||||
|
URL: "https://reports.example.com/html/report.html",
|
||||||
|
SHA256: "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||||
|
Size: 128,
|
||||||
|
Owner: CurrentOwnerScope("reports", "html"),
|
||||||
|
SourceID: htmlSource.ID,
|
||||||
|
SourceDigest: htmlSource.Digest,
|
||||||
|
SourceCreated: htmlSource.Created,
|
||||||
|
CreatedAt: createdAt,
|
||||||
|
UpdatedAt: createdAt,
|
||||||
|
}},
|
||||||
|
}
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user