Add reconcile state planning core

docs/internal/app.md

@@ -4,13 +4,13 @@ Audience: developers and LLM coding agents changing `internal/app`.
 
 ## Purpose
 
-`internal/app` owns top-level application use cases: run, single-pipeline run, staged-source run, validate, inspect, manifest creation, and HTTP upload serving. It coordinates config loading, secret resolution, backend construction, source discovery, destination selection, publish planning/execution, notification handoff, output projection, and upload coordination.
+`internal/app` owns top-level application use cases: run, single-pipeline run, staged-source run, validate, inspect, manifest creation, reconcile-state planning/repair, and HTTP upload serving. It coordinates config loading, secret resolution, backend construction, source discovery, destination selection, publish planning/execution, state repair reporting, notification handoff, output projection, and upload coordination.
 
 ## Inputs And Outputs
 
 Inputs include app option structs, contexts, config paths, pipeline ids, local source roots, dry-run/force flags, output format, stdout writers, HTTP requests, and optional notifier implementations.
 
-Outputs include `RunReport`, validate/inspect/manifest results, CLI text/JSON projections, HTTP upload responses, upload status records, and errors. Destination-scoped failures can return a partial run report plus an aggregated error; fatal setup failures return before a complete report exists.
+Outputs include `RunReport`, `ReconcileStateReport`, validate/inspect/manifest results, CLI text/JSON projections, HTTP upload responses, upload status records, and errors. Destination-scoped failures can return a partial run report plus an aggregated error; fatal setup failures return before a complete report exists.
 
 ## Boundaries
 
@@ -34,13 +34,15 @@ The app layer registers default transforms, including Markdown-to-HTML, and supp
 
 Run workflows discover and validate source bundles through `internal/bundle`. Destination state actions are prepared and written through `internal/publish` and `internal/state`; the app layer records report projections of those actions and results.
 
+Reconcile-state workflows load one configured pipeline/destination selector, open that destination root, parse the root `.distributor.json`, and report missing managed output records plus unmanaged storage entries. Managed output existence checks use storage `Stat`; unmanaged reporting uses bounded storage `Walk` and excludes `.distributor.json` plus all paths already recorded as managed. Apply mode removes missing managed output records from state and rewrites valid state only; dry-run reports the same repair without writing. It does not validate output digests, delete destination files, adopt unmanaged files, or rewrite invalid or mismatched state.
+
 HTTP uploads stage and validate archives before enqueueing a pipeline run with a local staged source root. Go producers can use the public `pkg/upload` package to create client-side gzip tar uploads for this server contract; `internal/app` remains the server-side orchestration boundary and does not import that producer package.
 
 Upload idempotency is owned by the upload coordinator. Optional `Idempotency-Key` values are scoped to token id, pipeline id, and key. The coordinator reserves a key while staging is in progress, records the accepted run id with the validated source manifest identity after staging succeeds, returns the original accepted record for the same scoped key and same manifest, and rejects the same scoped key with a different manifest as a conflict.
 
 ## Skip And Resume Behavior
 
-Fan-out destinations are independent. A destination failure is recorded and does not prevent later destinations from being attempted. Dry-run builds plans and reports without destination writes, destination state writes, notifier calls, or SSH known-host persistence.
+Fan-out destinations are independent. A destination failure is recorded and does not prevent later destinations from being attempted. Run dry-run builds plans and reports without destination writes, destination state writes, notifier calls, or SSH known-host persistence. Reconcile-state dry-run reports missing managed records and unmanaged entries without rewriting state.
 
 HTTP upload status is in memory. Accepted jobs move through accepted, queued, running, succeeded, or failed states and expire after configured retention. Upload idempotency records are also memory-only, expire with the completed status record for their accepted run, and are cleared by process restart.
 
@@ -48,6 +50,8 @@ HTTP upload status is in memory. Accepted jobs move through accepted, queued, ru
 
 Runtime setup fails for config load, config validation, secret loading, or credential resolution errors. Source setup failures stop the affected run before destination planning. Destination open, planning, execution, and notification failures are recorded as destination failures where a partial result exists.
 
+Reconcile-state setup fails unless the caller supplies a pipeline id and destination id that select one configured destination root. Single-owner state must match that pipeline/destination owner. Shared-root all-owner repair still uses the selected destination to identify the root, then applies repair across owners inside that root. Invalid, unreadable, or ambiguous state fails before any rewrite.
+
 HTTP upload startup fails if upload tokens are missing, empty, or duplicated. Upload requests can fail during authentication, idempotency-key validation, content-type validation, idempotency conflict checks, queue admission, archive staging, source validation, or later publish execution.
 
 ## Tests To Inspect
@@ -67,3 +71,4 @@ HTTP upload startup fails if upload tokens are missing, empty, or duplicated. Up
 - Upload admission stages and validates a bundle before returning a run id.
 - Idempotent upload retries compare normalized source manifest identity, not archive bytes.
 - Runtime backend registration remains app-owned.
+- Reconcile-state repairs state records only; it never deletes or adopts destination files.

docs/internal/state.md

@@ -36,9 +36,9 @@ Shared-root publish conversion is explicit. Compatible single-owner state for th
 
 Embedded source manifests are parsed and validated through `internal/bundle`, which delegates source manifest semantics to `pkg/bundle`. Output records require clean paths, `source` or `generated` kind, valid source paths, lowercase SHA-256 digests, non-negative sizes, created and updated timestamps, and transform ids for generated outputs. Stored URLs must pass `internal/link` validation.
 
-The package also provides helpers for finding output records by path, projecting planned publish outputs into timestamped state outputs, merging retained and newly planned output records, and computing managed output paths from single-owner state.
+The package also provides helpers for finding output records by path, projecting planned publish outputs into timestamped state outputs, merging retained and newly planned output records, computing managed output paths from single-owner state, and removing missing managed output records from single-owner state.
 
-For shared-root state, helpers parse either state shape, identify the current owner scope, return an owner's latest source manifest, list managed paths for one owner or all owners, detect path ownership conflicts, project planned owner outputs, merge one owner's planned outputs while preserving unrelated owners, and replace one owner's outputs by removing that owner's omitted outputs.
+For shared-root state, helpers parse either state shape, identify the current owner scope, return an owner's latest source manifest, list managed paths for one owner or all owners, detect path ownership conflicts, project planned owner outputs, merge one owner's planned outputs while preserving unrelated owners, replace one owner's outputs by removing that owner's omitted outputs, and remove missing managed output records for either the current owner or every owner.
 
 Shared-root helper projections preserve output `created_at` for existing managed paths and use the current publication time for rewritten `updated_at`. Root-level `created_at` preservation is owned by publish execution.
 
@@ -48,6 +48,8 @@ Comparison is pure. It returns outcomes for absent state, unmanaged content, inv
 
 Shared-root comparison is owner-scoped. It compares only the owner keyed by the current pipeline id and destination id, treats an absent owner as absent destination state for that owner, and can compare compatible single-owner state for the current owner without converting unrelated single-owner state.
 
+Missing-output removal helpers are pure state transformations. They remove matching output records only and leave storage inspection, timestamp updates, validation, and state rewrites to callers.
+
 ## Failure Behavior
 
 Parsing rejects invalid JSON, trailing data, missing required fields, invalid timestamps, invalid state mode, invalid reconciliation mode, invalid embedded manifests, duplicate owners, duplicate outputs, invalid output paths, unsupported output kinds, missing generated transforms, invalid URLs, invalid digests, negative sizes, and shared-root outputs whose owner is not registered.
@@ -68,6 +70,7 @@ Parsing rejects invalid JSON, trailing data, missing required fields, invalid ti
 - Schema version `1` state remains readable as replacement-mode single-owner state.
 - Schema version `3` shared-root state is parsed and validated without converting unrelated single-owner state.
 - Shared-root owner updates preserve unrelated owners and reject planned path collisions with other owners.
+- Missing-output repair helpers preserve unrelated owner records and outputs.
 - Generated outputs always record a transform id.
 - Output records always carry created and updated timestamps after parsing.
 - Stored URLs are optional and must be absolute HTTP or HTTPS URLs when present.

internal/app/reconcile_state.go

@@ -0,0 +1,395 @@
+package app
+
+import (
+	"context"
+	"encoding/json"
+	"fmt"
+	"io"
+	"sort"
+	"time"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/config"
+	"gitea.maximumdirect.net/eric/distributor/internal/state"
+	"gitea.maximumdirect.net/eric/distributor/internal/storage"
+)
+
+const reconcileStateWalkLimit = 10000
+
+type ReconcileStateOptions struct {
+	ConfigPath    string
+	PipelineID    string
+	DestinationID string
+	AllOwners     bool
+	DryRun        bool
+	Stdout        io.Writer
+	OutputFormat  OutputFormat
+}
+
+type ReconcileStateReport struct {
+	PipelineID            string                    `json:"pipeline_id"`
+	DestinationID         string                    `json:"destination_id"`
+	Backend               string                    `json:"backend"`
+	RootPath              string                    `json:"root_path"`
+	StateSchema           int                       `json:"state_schema"`
+	OwnerScope            *ReconcileStateOwnerScope `json:"owner_scope,omitempty"`
+	CheckedCount          int                       `json:"checked_count"`
+	MissingManagedOutputs []ReconcileStatePath      `json:"missing_managed_outputs"`
+	UnmanagedEntries      []ReconcileStateEntry     `json:"unmanaged_entries"`
+	Changed               bool                      `json:"changed"`
+	WouldChange           bool                      `json:"would_change"`
+	DryRun                bool                      `json:"dry_run"`
+}
+
+type ReconcileStateOwnerScope struct {
+	PipelineID    string `json:"pipeline_id"`
+	DestinationID string `json:"destination_id"`
+	AllOwners     bool   `json:"all_owners,omitempty"`
+}
+
+type ReconcileStatePath struct {
+	Path          string                    `json:"path"`
+	OwnerScope    *ReconcileStateOwnerScope `json:"owner_scope,omitempty"`
+	StorageStatus string                    `json:"storage_status"`
+}
+
+type ReconcileStateEntry struct {
+	Path string `json:"path"`
+	Type string `json:"type"`
+	Size int64  `json:"size,omitempty"`
+}
+
+func ReconcileState(ctx context.Context, options ReconcileStateOptions) (ReconcileStateReport, error) {
+	if err := ValidateOutputFormat(options.OutputFormat); err != nil {
+		return ReconcileStateReport{}, err
+	}
+	if err := ctx.Err(); err != nil {
+		return ReconcileStateReport{}, err
+	}
+
+	setup, err := loadRuntimeSetup(options.ConfigPath)
+	if err != nil {
+		return ReconcileStateReport{}, err
+	}
+	return reconcileStateSetup(ctx, setup, options)
+}
+
+func reconcileStateConfigWithBackendFactory(ctx context.Context, cfg config.Config, options ReconcileStateOptions, provider backendFactoryProvider) (ReconcileStateReport, error) {
+	setup, err := runtimeSetupFromConfig("", cfg)
+	if err != nil {
+		return ReconcileStateReport{}, err
+	}
+	return reconcileStateSetupWithBackendFactory(ctx, setup, options, provider)
+}
+
+func reconcileStateSetup(ctx context.Context, setup runtimeSetup, options ReconcileStateOptions) (ReconcileStateReport, error) {
+	return reconcileStateSetupWithBackendFactory(ctx, setup, options, newBackendFactoryWithEnvironment)
+}
+
+func reconcileStateSetupWithBackendFactory(ctx context.Context, setup runtimeSetup, options ReconcileStateOptions, provider backendFactoryProvider) (ReconcileStateReport, error) {
+	if err := requireReconcileStateScope(options); err != nil {
+		return ReconcileStateReport{}, err
+	}
+	pipeline, ok := findPipeline(setup.Config, options.PipelineID)
+	if !ok {
+		return ReconcileStateReport{}, PipelineNotFoundError{ID: options.PipelineID}
+	}
+	destination, ok := findDestination(pipeline, options.DestinationID)
+	if !ok {
+		return ReconcileStateReport{}, fmt.Errorf("pipeline %s destination %s not found", options.PipelineID, options.DestinationID)
+	}
+
+	backends := provider(setup.Environment)
+	destinationBackend, err := backends.openDestination(ctx, destination)
+	if err != nil {
+		return ReconcileStateReport{}, err
+	}
+	defer closeBackend(destinationBackend)
+
+	report, err := buildReconcileStateReport(ctx, destinationBackend, pipeline, destination, options)
+	if err != nil {
+		return ReconcileStateReport{}, err
+	}
+	if err := WriteReconcileStateReport(options.Stdout, options.OutputFormat, report); err != nil {
+		return ReconcileStateReport{}, err
+	}
+	return report, nil
+}
+
+func requireReconcileStateScope(options ReconcileStateOptions) error {
+	if options.PipelineID == "" {
+		return fmt.Errorf("pipeline id is required")
+	}
+	if options.DestinationID == "" {
+		return fmt.Errorf("destination id is required")
+	}
+	return nil
+}
+
+func findDestination(pipeline config.Pipeline, id string) (config.Destination, bool) {
+	for _, destination := range pipeline.Destinations {
+		if destination.ID == id {
+			return destination, true
+		}
+	}
+	return config.Destination{}, false
+}
+
+func buildReconcileStateReport(ctx context.Context, backend storage.Backend, pipeline config.Pipeline, destination config.Destination, options ReconcileStateOptions) (ReconcileStateReport, error) {
+	statePath, err := storage.StatePath("")
+	if err != nil {
+		return ReconcileStateReport{}, err
+	}
+	data, err := backend.ReadFile(ctx, statePath)
+	if err != nil {
+		return ReconcileStateReport{}, err
+	}
+	document, err := state.ParseDocument(data)
+	if err != nil {
+		return ReconcileStateReport{}, err
+	}
+
+	report := ReconcileStateReport{
+		PipelineID:            pipeline.ID,
+		DestinationID:         destination.ID,
+		Backend:               destination.Backend,
+		RootPath:              destinationRootPath(destination),
+		MissingManagedOutputs: []ReconcileStatePath{},
+		UnmanagedEntries:      []ReconcileStateEntry{},
+		DryRun:                options.DryRun,
+	}
+	scope := state.CurrentOwnerScope(pipeline.ID, destination.ID)
+	if document.SingleOwner != nil {
+		return reconcileSingleOwnerState(ctx, backend, statePath, *document.SingleOwner, scope, report, options)
+	}
+	return reconcileSharedRootState(ctx, backend, statePath, *document.SharedRoot, scope, report, options)
+}
+
+func reconcileSingleOwnerState(ctx context.Context, backend storage.Backend, statePath string, destinationState state.DistributorState, scope state.OwnerScope, report ReconcileStateReport, options ReconcileStateOptions) (ReconcileStateReport, error) {
+	if destinationState.PipelineID != scope.PipelineID || destinationState.DestinationID != scope.DestinationID {
+		return ReconcileStateReport{}, fmt.Errorf("state owner is %s/%s, not %s/%s", destinationState.PipelineID, destinationState.DestinationID, scope.PipelineID, scope.DestinationID)
+	}
+	report.StateSchema = destinationState.SchemaVersion
+	report.OwnerScope = &ReconcileStateOwnerScope{PipelineID: scope.PipelineID, DestinationID: scope.DestinationID}
+	managed := state.ManagedOutputPaths(destinationState)
+	missing, err := missingSingleOwnerOutputs(ctx, backend, destinationState.Outputs)
+	if err != nil {
+		return ReconcileStateReport{}, err
+	}
+	report.CheckedCount = len(managed)
+	report.MissingManagedOutputs = missing
+	unmanaged, err := unmanagedEntries(ctx, backend, managed)
+	if err != nil {
+		return ReconcileStateReport{}, err
+	}
+	report.UnmanagedEntries = unmanaged
+	report.WouldChange = options.DryRun && len(missing) > 0
+
+	if !options.DryRun && len(missing) > 0 {
+		missingPaths := missingReportPaths(missing)
+		next, changed := state.RemoveMissingOutputs(destinationState, missingPaths)
+		report.Changed = changed
+		if changed {
+			next.UpdatedAt = time.Now().UTC()
+			if err := state.Validate(next); err != nil {
+				return ReconcileStateReport{}, err
+			}
+			if err := writeRepairedState(ctx, backend, statePath, next); err != nil {
+				return ReconcileStateReport{}, err
+			}
+		}
+	}
+	return report, nil
+}
+
+func reconcileSharedRootState(ctx context.Context, backend storage.Backend, statePath string, sharedRoot state.SharedRootState, scope state.OwnerScope, report ReconcileStateReport, options ReconcileStateOptions) (ReconcileStateReport, error) {
+	report.StateSchema = sharedRoot.SchemaVersion
+	report.OwnerScope = &ReconcileStateOwnerScope{
+		PipelineID:    scope.PipelineID,
+		DestinationID: scope.DestinationID,
+		AllOwners:     options.AllOwners,
+	}
+	managed := sharedRoot.AllManagedOutputPaths()
+	outputs := sharedRoot.Outputs
+	if !options.AllOwners {
+		outputs = sharedRootOutputsForOwner(sharedRoot.Outputs, scope)
+	}
+	missing, err := missingSharedRootOutputs(ctx, backend, outputs)
+	if err != nil {
+		return ReconcileStateReport{}, err
+	}
+	report.CheckedCount = len(outputs)
+	report.MissingManagedOutputs = missing
+	unmanaged, err := unmanagedEntries(ctx, backend, managed)
+	if err != nil {
+		return ReconcileStateReport{}, err
+	}
+	report.UnmanagedEntries = unmanaged
+	report.WouldChange = options.DryRun && len(missing) > 0
+
+	if !options.DryRun && len(missing) > 0 {
+		missingPaths := missingReportPaths(missing)
+		var next state.SharedRootState
+		var changed bool
+		if options.AllOwners {
+			next, changed = state.RemoveMissingSharedRootOutputs(sharedRoot, missingPaths)
+		} else {
+			next, changed = state.RemoveMissingSharedRootOwnerOutputs(sharedRoot, scope, missingPaths)
+		}
+		report.Changed = changed
+		if changed {
+			next.UpdatedAt = time.Now().UTC()
+			if err := state.ValidateSharedRoot(next); err != nil {
+				return ReconcileStateReport{}, err
+			}
+			if err := writeRepairedState(ctx, backend, statePath, next); err != nil {
+				return ReconcileStateReport{}, err
+			}
+		}
+	}
+	return report, nil
+}
+
+func missingSingleOwnerOutputs(ctx context.Context, backend storage.Backend, outputs []state.OutputFile) ([]ReconcileStatePath, error) {
+	missing := make([]ReconcileStatePath, 0)
+	for _, output := range outputs {
+		if err := checkManagedOutput(ctx, backend, output.Path); err != nil {
+			if storage.IsNotFound(err) {
+				missing = append(missing, ReconcileStatePath{Path: output.Path, StorageStatus: "missing"})
+				continue
+			}
+			return nil, err
+		}
+	}
+	return missing, nil
+}
+
+func missingSharedRootOutputs(ctx context.Context, backend storage.Backend, outputs []state.SharedRootOutputFile) ([]ReconcileStatePath, error) {
+	missing := make([]ReconcileStatePath, 0)
+	for _, output := range outputs {
+		if err := checkManagedOutput(ctx, backend, output.Path); err != nil {
+			if storage.IsNotFound(err) {
+				missing = append(missing, ReconcileStatePath{
+					Path: output.Path,
+					OwnerScope: &ReconcileStateOwnerScope{
+						PipelineID:    output.Owner.PipelineID,
+						DestinationID: output.Owner.DestinationID,
+					},
+					StorageStatus: "missing",
+				})
+				continue
+			}
+			return nil, err
+		}
+	}
+	return missing, nil
+}
+
+func checkManagedOutput(ctx context.Context, backend storage.Backend, path string) error {
+	_, err := backend.Stat(ctx, path)
+	return err
+}
+
+func unmanagedEntries(ctx context.Context, backend storage.Backend, managedPaths []string) ([]ReconcileStateEntry, error) {
+	managed := make(map[string]struct{}, len(managedPaths)+1)
+	for _, path := range managedPaths {
+		managed[path] = struct{}{}
+	}
+	managed[storage.StateFileName] = struct{}{}
+
+	entries := make([]ReconcileStateEntry, 0)
+	err := backend.Walk(ctx, "", storage.WalkOptions{Recursive: true, Limit: reconcileStateWalkLimit}, func(entry storage.Entry) error {
+		if entry.Type == storage.EntryTypeDirectory {
+			return nil
+		}
+		if _, ok := managed[entry.Path]; ok {
+			return nil
+		}
+		entries = append(entries, ReconcileStateEntry{
+			Path: entry.Path,
+			Type: string(entry.Type),
+			Size: entry.Size,
+		})
+		return nil
+	})
+	if err != nil {
+		return nil, err
+	}
+	sort.Slice(entries, func(i, j int) bool {
+		return entries[i].Path < entries[j].Path
+	})
+	return entries, nil
+}
+
+func sharedRootOutputsForOwner(outputs []state.SharedRootOutputFile, scope state.OwnerScope) []state.SharedRootOutputFile {
+	selected := make([]state.SharedRootOutputFile, 0, len(outputs))
+	for _, output := range outputs {
+		if output.Owner == scope {
+			selected = append(selected, output)
+		}
+	}
+	return selected
+}
+
+func missingReportPaths(missing []ReconcileStatePath) []string {
+	paths := make([]string, 0, len(missing))
+	for _, item := range missing {
+		paths = append(paths, item.Path)
+	}
+	return paths
+}
+
+func writeRepairedState(ctx context.Context, backend storage.Backend, path string, value any) error {
+	data, err := json.MarshalIndent(value, "", "  ")
+	if err != nil {
+		return err
+	}
+	data = append(data, '\n')
+	_, err = backend.WriteFile(ctx, path, data, storage.WriteOptions{Overwrite: true, PreferAtomic: true})
+	return err
+}
+
+func destinationRootPath(destination config.Destination) string {
+	switch destination.Backend {
+	case config.BackendS3:
+		if destination.Prefix == "" {
+			return "."
+		}
+		return destination.Prefix
+	default:
+		if destination.Path == "" {
+			return "."
+		}
+		return destination.Path
+	}
+}
+
+func WriteReconcileStateReport(w io.Writer, format OutputFormat, report ReconcileStateReport) error {
+	if IsJSONOutput(format) {
+		return WriteJSONEnvelope(w, "reconcile-state", true, nil, report, nil)
+	}
+	return writeReconcileStateReportText(w, report)
+}
+
+func writeReconcileStateReportText(w io.Writer, report ReconcileStateReport) error {
+	if w == nil {
+		return nil
+	}
+	status := "unchanged"
+	if report.Changed {
+		status = "changed"
+	} else if report.WouldChange {
+		status = "would_change"
+	}
+	_, err := fmt.Fprintf(w, "Reconcile state: pipeline=%s destination=%s backend=%s root=%s status=%s checked=%d missing=%d unmanaged=%d dry_run=%t\n",
+		report.PipelineID,
+		report.DestinationID,
+		report.Backend,
+		report.RootPath,
+		status,
+		report.CheckedCount,
+		len(report.MissingManagedOutputs),
+		len(report.UnmanagedEntries),
+		report.DryRun,
+	)
+	return err
+}

internal/app/reconcile_state_test.go

@@ -0,0 +1,269 @@
+package app
+
+import (
+	"context"
+	"encoding/json"
+	"strings"
+	"testing"
+	"time"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/config"
+	"gitea.maximumdirect.net/eric/distributor/internal/state"
+	"gitea.maximumdirect.net/eric/distributor/internal/storage"
+	"gitea.maximumdirect.net/eric/distributor/internal/storage/fake"
+	"gitea.maximumdirect.net/eric/distributor/internal/testutil"
+)
+
+func TestReconcileStateDryRunReportsMissingManagedOutputsWithoutRewrite(t *testing.T) {
+	backend := fake.New()
+	cfg := reconcileStateS3Config(t)
+	manifest := testutil.ValidManifest(testutil.BundleOptions{})
+	testutil.WriteFakeDestinationState(t, backend, "", manifest, testutil.DestinationStateOptions{})
+	if err := backend.DeleteManagedOutputs(context.Background(), "", []string{"summary.txt"}, storage.DeleteOptions{}); err != nil {
+		t.Fatalf("delete managed output: %v", err)
+	}
+	testutil.WriteFakeFile(t, backend, "extra.txt", "unmanaged")
+
+	report, err := reconcileStateConfigWithBackendFactory(context.Background(), cfg, ReconcileStateOptions{
+		PipelineID:    "reports",
+		DestinationID: "archive",
+		DryRun:        true,
+	}, fakeBackendFactoryProvider(t, map[string]storage.Backend{"s3:reports": backend}))
+	if err != nil {
+		t.Fatalf("reconcileStateConfigWithBackendFactory() error = %v", err)
+	}
+	if !report.WouldChange || report.Changed {
+		t.Fatalf("report changed=%t would_change=%t, want dry-run pending change", report.Changed, report.WouldChange)
+	}
+	if got := reportPathList(report.MissingManagedOutputs); got != "summary.txt" {
+		t.Fatalf("missing outputs = %q, want summary.txt", got)
+	}
+	if got := entryPathList(report.UnmanagedEntries); got != "extra.txt" {
+		t.Fatalf("unmanaged entries = %q, want extra.txt", got)
+	}
+	destinationState := readFakeSingleOwnerState(t, backend)
+	if got := strings.Join(state.ManagedOutputPaths(destinationState), ","); got != "report.md,summary.txt" {
+		t.Fatalf("state outputs = %q, want original outputs", got)
+	}
+}
+
+func TestReconcileStateApplyRemovesMissingRecordsAndPreservesUnmanagedFiles(t *testing.T) {
+	backend := fake.New()
+	cfg := reconcileStateS3Config(t)
+	manifest := testutil.ValidManifest(testutil.BundleOptions{})
+	testutil.WriteFakeDestinationState(t, backend, "", manifest, testutil.DestinationStateOptions{})
+	if err := backend.DeleteManagedOutputs(context.Background(), "", []string{"summary.txt"}, storage.DeleteOptions{}); err != nil {
+		t.Fatalf("delete managed output: %v", err)
+	}
+	testutil.WriteFakeFile(t, backend, "extra.txt", "unmanaged")
+
+	report, err := reconcileStateConfigWithBackendFactory(context.Background(), cfg, ReconcileStateOptions{
+		PipelineID:    "reports",
+		DestinationID: "archive",
+	}, fakeBackendFactoryProvider(t, map[string]storage.Backend{"s3:reports": backend}))
+	if err != nil {
+		t.Fatalf("reconcileStateConfigWithBackendFactory() error = %v", err)
+	}
+	if !report.Changed || report.WouldChange {
+		t.Fatalf("report changed=%t would_change=%t, want applied change", report.Changed, report.WouldChange)
+	}
+	destinationState := readFakeSingleOwnerState(t, backend)
+	if err := state.Validate(destinationState); err != nil {
+		t.Fatalf("Validate() repaired state error = %v", err)
+	}
+	if got := strings.Join(state.ManagedOutputPaths(destinationState), ","); got != "report.md" {
+		t.Fatalf("state outputs = %q, want report.md", got)
+	}
+	testutil.AssertFakeFile(t, backend, "extra.txt", "unmanaged")
+}
+
+func TestReconcileStateInvalidStateFailsWithoutRewrite(t *testing.T) {
+	backend := fake.New()
+	cfg := reconcileStateS3Config(t)
+	invalid := `{"schema_version":2,"pipeline_id":"reports"}`
+	testutil.WriteFakeFile(t, backend, storage.StateFileName, invalid)
+
+	_, err := reconcileStateConfigWithBackendFactory(context.Background(), cfg, ReconcileStateOptions{
+		PipelineID:    "reports",
+		DestinationID: "archive",
+	}, fakeBackendFactoryProvider(t, map[string]storage.Backend{"s3:reports": backend}))
+	if err == nil {
+		t.Fatal("reconcileStateConfigWithBackendFactory() error = nil, want invalid state error")
+	}
+	data, readErr := backend.ReadFile(context.Background(), storage.StateFileName)
+	if readErr != nil {
+		t.Fatalf("read invalid state: %v", readErr)
+	}
+	if string(data) != invalid {
+		t.Fatalf("state data = %q, want original invalid data", data)
+	}
+}
+
+func TestReconcileStateSharedRootOwnerScopeRepairsCurrentOwnerOnly(t *testing.T) {
+	backend := fake.New()
+	cfg := reconcileStateS3Config(t)
+	sharedRoot := reconcileSharedRootFixture(t)
+	writeFakeSharedRootStateForApp(t, backend, sharedRoot)
+	if err := backend.DeleteManagedOutputs(context.Background(), "", []string{"report.md", "report.html"}, storage.DeleteOptions{}); err != nil {
+		t.Fatalf("delete managed outputs: %v", err)
+	}
+
+	report, err := reconcileStateConfigWithBackendFactory(context.Background(), cfg, ReconcileStateOptions{
+		PipelineID:    "reports",
+		DestinationID: "archive",
+	}, fakeBackendFactoryProvider(t, map[string]storage.Backend{"s3:reports": backend}))
+	if err != nil {
+		t.Fatalf("reconcileStateConfigWithBackendFactory() error = %v", err)
+	}
+	if !report.Changed {
+		t.Fatal("report changed = false, want true")
+	}
+	repaired := readFakeSharedRootStateForApp(t, backend)
+	if got := strings.Join(repaired.AllManagedOutputPaths(), ","); got != "report.html" {
+		t.Fatalf("shared-root outputs = %q, want other owner output preserved", got)
+	}
+}
+
+func TestReconcileStateSharedRootAllOwnersRepairsEveryOwner(t *testing.T) {
+	backend := fake.New()
+	cfg := reconcileStateS3Config(t)
+	sharedRoot := reconcileSharedRootFixture(t)
+	writeFakeSharedRootStateForApp(t, backend, sharedRoot)
+	if err := backend.DeleteManagedOutputs(context.Background(), "", []string{"report.md", "report.html"}, storage.DeleteOptions{}); err != nil {
+		t.Fatalf("delete managed outputs: %v", err)
+	}
+
+	report, err := reconcileStateConfigWithBackendFactory(context.Background(), cfg, ReconcileStateOptions{
+		PipelineID:    "reports",
+		DestinationID: "archive",
+		AllOwners:     true,
+	}, fakeBackendFactoryProvider(t, map[string]storage.Backend{"s3:reports": backend}))
+	if err != nil {
+		t.Fatalf("reconcileStateConfigWithBackendFactory() error = %v", err)
+	}
+	if !report.Changed || report.CheckedCount != 2 {
+		t.Fatalf("report changed=%t checked=%d, want all-owner repair", report.Changed, report.CheckedCount)
+	}
+	repaired := readFakeSharedRootStateForApp(t, backend)
+	if got := repaired.AllManagedOutputPaths(); len(got) != 0 {
+		t.Fatalf("shared-root outputs = %#v, want none", got)
+	}
+}
+
+func reconcileStateS3Config(t *testing.T) config.Config {
+	t.Helper()
+	cfg := config.Config{Pipelines: []config.Pipeline{{
+		ID:     "reports",
+		Source: config.Backend{Backend: config.BackendLocal, Path: t.TempDir()},
+		Destinations: []config.Destination{{
+			ID:      "archive",
+			Backend: config.BackendS3,
+			Bucket:  "reports",
+		}},
+	}}}
+	config.ApplyDefaults(&cfg)
+	return cfg
+}
+
+func readFakeSingleOwnerState(t *testing.T, backend *fake.Backend) state.DistributorState {
+	t.Helper()
+	data, err := backend.ReadFile(context.Background(), storage.StateFileName)
+	if err != nil {
+		t.Fatalf("read state: %v", err)
+	}
+	destinationState, err := state.Parse(data)
+	if err != nil {
+		t.Fatalf("parse state: %v", err)
+	}
+	return destinationState
+}
+
+func writeFakeSharedRootStateForApp(t *testing.T, backend *fake.Backend, sharedRoot state.SharedRootState) {
+	t.Helper()
+	data, err := json.MarshalIndent(sharedRoot, "", "  ")
+	if err != nil {
+		t.Fatalf("marshal shared-root state: %v", err)
+	}
+	testutil.WriteFakeFile(t, backend, storage.StateFileName, string(append(data, '\n')))
+	for _, output := range sharedRoot.Outputs {
+		testutil.WriteFakeFile(t, backend, output.Path, "old")
+	}
+}
+
+func readFakeSharedRootStateForApp(t *testing.T, backend *fake.Backend) state.SharedRootState {
+	t.Helper()
+	data, err := backend.ReadFile(context.Background(), storage.StateFileName)
+	if err != nil {
+		t.Fatalf("read shared-root state: %v", err)
+	}
+	sharedRoot, err := state.ParseSharedRoot(data)
+	if err != nil {
+		t.Fatalf("parse shared-root state: %v", err)
+	}
+	return sharedRoot
+}
+
+func reconcileSharedRootFixture(t *testing.T) state.SharedRootState {
+	t.Helper()
+	source := testutil.ValidManifest(testutil.BundleOptions{})
+	htmlSource := source
+	createdAt := time.Date(2026, 5, 30, 11, 12, 0, 0, time.UTC)
+	return state.SharedRootState{
+		SchemaVersion:      state.SharedRootSchemaVersion,
+		DistributorVersion: "test",
+		CreatedAt:          createdAt,
+		UpdatedAt:          createdAt,
+		State:              state.StatePolicy{Mode: state.StateModeSharedRoot},
+		Owners: []state.OwnerRecord{{
+			Scope:          state.CurrentOwnerScope("reports", "archive"),
+			Reconciliation: state.ReconciliationPolicy{Mode: config.ReconciliationModeReplace},
+			Source:         state.SourceState{Manifest: source},
+		}, {
+			Scope:          state.CurrentOwnerScope("reports", "html"),
+			Reconciliation: state.ReconciliationPolicy{Mode: config.ReconciliationModeMerge},
+			Source:         state.SourceState{Manifest: htmlSource},
+		}},
+		Outputs: []state.SharedRootOutputFile{{
+			Path:          "report.md",
+			Kind:          state.OutputKindSource,
+			SourcePath:    "report.md",
+			SHA256:        source.Files[0].SHA256,
+			Size:          source.Files[0].Size,
+			Owner:         state.CurrentOwnerScope("reports", "archive"),
+			SourceID:      source.ID,
+			SourceDigest:  source.Digest,
+			SourceCreated: source.Created,
+			CreatedAt:     createdAt,
+			UpdatedAt:     createdAt,
+		}, {
+			Path:          "report.html",
+			Kind:          state.OutputKindGenerated,
+			SourcePath:    "report.md",
+			Transform:     "markdown_to_html",
+			SHA256:        "sha256:" + strings.Repeat("a", 64),
+			Size:          128,
+			Owner:         state.CurrentOwnerScope("reports", "html"),
+			SourceID:      htmlSource.ID,
+			SourceDigest:  htmlSource.Digest,
+			SourceCreated: htmlSource.Created,
+			CreatedAt:     createdAt,
+			UpdatedAt:     createdAt,
+		}},
+	}
+}
+
+func reportPathList(paths []ReconcileStatePath) string {
+	values := make([]string, 0, len(paths))
+	for _, path := range paths {
+		values = append(values, path.Path)
+	}
+	return strings.Join(values, ",")
+}
+
+func entryPathList(entries []ReconcileStateEntry) string {
+	values := make([]string, 0, len(entries))
+	for _, entry := range entries {
+		values = append(values, entry.Path)
+	}
+	return strings.Join(values, ",")
+}

internal/state/distributor_test.go

@@ -61,6 +61,38 @@ func TestParseValidStateWithLinks(t *testing.T) {
 	}
 }
 
+func TestRemoveMissingOutputs(t *testing.T) {
+	state, err := Parse([]byte(validStateJSON(t)))
+	if err != nil {
+		t.Fatalf("Parse() error = %v", err)
+	}
+	state.Outputs = append(state.Outputs, OutputFile{
+		Path:       "summary.txt",
+		Kind:       OutputKindSource,
+		SourcePath: "summary.txt",
+		SHA256:     "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
+		Size:       10,
+		CreatedAt:  state.CreatedAt,
+		UpdatedAt:  state.UpdatedAt,
+	})
+
+	next, changed := RemoveMissingOutputs(state, []string{"summary.txt"})
+	if !changed {
+		t.Fatal("RemoveMissingOutputs() changed = false, want true")
+	}
+	if got, want := ManagedOutputPaths(next), []string{"report.md"}; strings.Join(got, ",") != strings.Join(want, ",") {
+		t.Fatalf("paths = %#v, want %#v", got, want)
+	}
+
+	unchanged, changed := RemoveMissingOutputs(next, []string{"missing.txt"})
+	if changed {
+		t.Fatal("RemoveMissingOutputs() changed = true, want false")
+	}
+	if got, want := ManagedOutputPaths(unchanged), []string{"report.md"}; strings.Join(got, ",") != strings.Join(want, ",") {
+		t.Fatalf("unchanged paths = %#v, want %#v", got, want)
+	}
+}
+
 func TestParseNormalizesPublishedAtOffset(t *testing.T) {
 	body := strings.Replace(validStateJSON(t), `"published_at": "2026-05-30T11:12:00Z"`, `"published_at": "2026-05-30T13:12:00+02:00"`, 1)
 	state, err := Parse([]byte(body))

internal/state/outputs.go

@@ -60,6 +60,24 @@ func ManagedOutputPaths(s DistributorState) []string {
 	return paths
 }
 
+func RemoveMissingOutputs(s DistributorState, missingPaths []string) (DistributorState, bool) {
+	if len(missingPaths) == 0 {
+		return s, false
+	}
+	missing := pathSet(missingPaths)
+	next := s
+	next.Outputs = make([]OutputFile, 0, len(s.Outputs))
+	changed := false
+	for _, output := range s.Outputs {
+		if _, remove := missing[output.Path]; remove {
+			changed = true
+			continue
+		}
+		next.Outputs = append(next.Outputs, output)
+	}
+	return next, changed
+}
+
 func ProjectOutputs(outputs []OutputProjection, existing []OutputFile, now time.Time) []OutputFile {
 	now = now.UTC()
 	files := make([]OutputFile, 0, len(outputs))
@@ -122,6 +140,44 @@ func (s SharedRootState) AllManagedOutputPaths() []string {
 	return paths
 }
 
+func RemoveMissingSharedRootOwnerOutputs(s SharedRootState, scope OwnerScope, missingPaths []string) (SharedRootState, bool) {
+	if len(missingPaths) == 0 {
+		return s, false
+	}
+	missing := pathSet(missingPaths)
+	next := s
+	next.Outputs = make([]SharedRootOutputFile, 0, len(s.Outputs))
+	changed := false
+	for _, output := range s.Outputs {
+		if output.Owner == scope {
+			if _, remove := missing[output.Path]; remove {
+				changed = true
+				continue
+			}
+		}
+		next.Outputs = append(next.Outputs, output)
+	}
+	return next, changed
+}
+
+func RemoveMissingSharedRootOutputs(s SharedRootState, missingPaths []string) (SharedRootState, bool) {
+	if len(missingPaths) == 0 {
+		return s, false
+	}
+	missing := pathSet(missingPaths)
+	next := s
+	next.Outputs = make([]SharedRootOutputFile, 0, len(s.Outputs))
+	changed := false
+	for _, output := range s.Outputs {
+		if _, remove := missing[output.Path]; remove {
+			changed = true
+			continue
+		}
+		next.Outputs = append(next.Outputs, output)
+	}
+	return next, changed
+}
+
 func (s SharedRootState) OutputOwner(path string) (OwnerScope, bool) {
 	for _, output := range s.Outputs {
 		if output.Path == path {
@@ -264,3 +320,11 @@ func upsertOwner(owners []OwnerRecord, owner OwnerRecord) []OwnerRecord {
 	}
 	return append(next, owner)
 }
+
+func pathSet(paths []string) map[string]struct{} {
+	set := make(map[string]struct{}, len(paths))
+	for _, path := range paths {
+		set[path] = struct{}{}
+	}
+	return set
+}

internal/state/shared_root_test.go

@@ -132,6 +132,32 @@ func TestSharedRootOutputHelpers(t *testing.T) {
 	}
 }
 
+func TestRemoveMissingSharedRootOwnerOutputs(t *testing.T) {
+	state := validSharedRootState(t)
+	archive := CurrentOwnerScope("reports", "archive")
+	next, changed := RemoveMissingSharedRootOwnerOutputs(state, archive, []string{"report.md", "report.html"})
+	if !changed {
+		t.Fatal("RemoveMissingSharedRootOwnerOutputs() changed = false, want true")
+	}
+	if got, want := next.AllManagedOutputPaths(), []string{"report.html"}; strings.Join(got, ",") != strings.Join(want, ",") {
+		t.Fatalf("paths = %#v, want %#v", got, want)
+	}
+	if _, ok := next.OutputOwner("report.html"); !ok {
+		t.Fatal("report.html owner missing, want unrelated owner preserved")
+	}
+}
+
+func TestRemoveMissingSharedRootOutputs(t *testing.T) {
+	state := validSharedRootState(t)
+	next, changed := RemoveMissingSharedRootOutputs(state, []string{"report.md", "report.html"})
+	if !changed {
+		t.Fatal("RemoveMissingSharedRootOutputs() changed = false, want true")
+	}
+	if got := next.AllManagedOutputPaths(); len(got) != 0 {
+		t.Fatalf("paths = %#v, want none", got)
+	}
+}
+
 func TestSharedRootProjectAndMergeOwnerOutputs(t *testing.T) {
 	state := validSharedRootState(t)
 	archive := CurrentOwnerScope("reports", "archive")