distributor/docs/roadmap/public_bundle_package.md

# Roadmap: Public Bundle Manifest Package

## Purpose

Expose a small public Go package that producer applications can import to create
valid distributor source bundle manifests.

The package should encode the producer-side source bundle contract without
exposing distributor's publication, storage, transform, destination state,
notification, or config internals.

## Current Implementation Grounding

The current implementation keeps source bundle behavior in `internal/bundle`:

- `Manifest` and `ManifestFile` model `manifest.json`;
- `ParseManifest` parses JSON and RFC3339 `created` timestamps;
- `ValidateManifest` owns schema version, digest format, file path, duplicate,
  size, and bundle digest validation;
- `FileDigest`, `BundleDigest`, and `CanonicalFilePayload` define digest
  behavior;
- source validation rejects unsafe paths, reserved distributor metadata paths,
  non-file entries, size mismatches, SHA-256 mismatches, and bundle digest
  mismatches.

Producers cannot import `internal/bundle`, so Go producers currently need to
duplicate this contract or shell out to future CLI tooling.

## Goals

- Provide a stable producer-facing Go API for manifest creation and validation.
- Reuse the same source manifest, digest, path safety, and RFC3339 behavior used
  by distributor validation.
- Keep the public API intentionally small and producer-only.
- Include a safe bundle writer so producers can create complete local bundle
  directories without hand-rolling manifest-write and staging behavior.
- Make the future `distributor manifest create` command a thin wrapper over this
  package.
- Avoid exposing destination state, publish planning, storage backends,
  transforms, notifications, or config.

## Non-Goals

- Do not expose the distributor runner or publication workflow as public API.
- Do not expose storage backends or destination `.distributor.json` state.
- Do not add domain-specific manifest metadata.
- Do not require non-Go producers to use Go APIs.
- Do not implement latest paths, link generation, transforms, or notification
  behavior in this package.

## Package Boundary

Use `pkg/bundle` as the public package name.

The package should own only producer-side source bundle concerns:

- manifest model and schema version constant;
- file digest and bundle digest calculation;
- manifest building from producer files;
- manifest JSON load/write helpers;
- manifest and bundle validation;
- source path safety matching distributor validation.

Prefer options structs over long positional functions so future additive
behavior can be introduced without avoidable API churn.

## Initial Exported API

The initial `pkg/bundle` API is locked to the following exported constants,
types, and functions. Implementation should not rename, remove, or reshape
these public symbols during the initial implementation pass.

```go
const ManifestName = "manifest.json"
const SchemaVersion = 1

type Manifest struct {
    SchemaVersion int            `json:"schema_version"`
    ID            string         `json:"id"`
    Digest        string         `json:"digest"`
    Created       time.Time      `json:"created"`
    Files         []ManifestFile `json:"files"`
}

type ManifestFile struct {
    Path   string `json:"path"`
    SHA256 string `json:"sha256"`
    Size   int64  `json:"size"`
}

type BuildOptions struct {
    Root    string
    ID      string
    Created time.Time
    Files   []string
    Scan    bool
}

type WriteManifestOptions struct {
    Overwrite bool
}

type BundleFile struct {
    SourcePath string
    Path       string
}

type WriteBundleOptions struct {
    Root      string
    ID        string
    Created   time.Time
    Files     []BundleFile
    Overwrite bool
}

func ParseManifest(data []byte) (Manifest, error)
func MarshalManifest(manifest Manifest) ([]byte, error)
func LoadManifest(root string) (Manifest, error)
func WriteManifest(root string, manifest Manifest, opts WriteManifestOptions) error
func BuildManifest(opts BuildOptions) (Manifest, error)
func ValidateManifest(manifest Manifest) error
func ValidateBundle(root string, manifest Manifest) error
func WriteBundle(opts WriteBundleOptions) (Manifest, error)
func ValidateSourcePath(path string) error
func FileDigest(data []byte) string
func BundleDigest(files []ManifestFile) string
func CanonicalFilePayload(files []ManifestFile) string
```

## API Semantics

`BuildManifest` requires `Root`, `ID`, and exactly one file-selection mode:
explicit `Files` or `Scan: true`.

Explicit `Files` preserve caller order. `Scan: true` recursively scans `Root`,
includes regular files including dotfiles, excludes `manifest.json` and
`.distributor.json`, rejects symlinks, and sorts by slash-separated relative
path.

Zero `Created` values default to the current UTC time. All public path fields
use slash-separated bundle-relative paths.

`MarshalManifest` validates before marshaling and emits deterministic JSON with
fixed field order and a trailing newline.

`WriteManifest` writes `manifest.json`; it fails if the file exists unless
`WriteManifestOptions.Overwrite` is true, and it uses temp-and-rename
replacement where practical.

`ValidateManifest` checks manifest-only semantics. `ValidateBundle` checks the
supplied manifest against local files under `root`, including existence,
regular-file type, size, SHA-256, path safety, duplicates, and bundle digest.

`WriteBundle` copies existing local files from `BundleFile.SourcePath` into a
staged bundle at `BundleFile.Path`, writes a compliant manifest, validates the
staged bundle, and promotes it to `WriteBundleOptions.Root`.

`WriteBundleOptions.Overwrite` permits replacing an existing bundle root.
Replacement must build the new bundle completely before touching the existing
root, then use sibling temp and backup paths for best-effort promotion and
restore on failure.

The writer remains producer-side and filesystem-local. It must not expose
distributor storage backends or publication behavior.

## Manifest Compatibility

The package should treat the source manifest schema as a compatibility boundary:

- export the current schema version;
- preserve JSON field names exactly;
- use lowercase `sha256:<64 hex>` digests;
- use slash-separated relative paths in JSON;
- use RFC3339 timestamps;
- default a zero build or writer `Created` value to the current UTC time;
- preserve caller-provided file order for explicit file lists;
- produce deterministic ordering when scan-based building is selected;
- reject symlinks if distributor validation still rejects source symlinks.

Scan-based building belongs in v1 of the public package. Explicit file lists
should preserve caller order. Scan mode should sort by slash-separated relative
path and share the same filtering rules expected by future CLI manifest
creation.

The internal implementation may either move source-bundle core logic into
`pkg/bundle` and have internal packages consume it, or keep internal wrappers
around public core logic. The important invariant is that public package,
future CLI manifest creation, and distributor validation must not drift.

## Relationship To Other Roadmaps

`distributor manifest create` should call `pkg/bundle` rather than maintaining a
separate manifest builder.

Remote `validate` and `inspect` should continue using distributor's storage
abstraction and internal app wiring; they do not need public producer APIs.

HTML index mode, link generation, and latest path destinations operate after a
bundle has already entered distributor and should not affect this package.

## Testing Expectations

Suggested coverage:

- build a manifest from explicit files;
- build a manifest by scanning a local bundle root;
- preserve explicit file order;
- sort scan results deterministically by slash-separated relative path;
- compute per-file SHA-256 and size;
- compute the expected canonical bundle digest;
- write and load `manifest.json`;
- default zero `Created` to current UTC time while honoring explicit timestamps;
- validate generated manifests successfully;
- reject unsafe paths, missing files, non-regular files, and symlinks;
- emit slash-separated JSON paths;
- parse, marshal, load, and write manifests through the exact exported API;
- fail `WriteManifest` when `manifest.json` exists unless overwrite is enabled;
- emit deterministic manifest JSON with fixed field order and trailing newline;
- write a complete local bundle through the public writer;
- copy `BundleFile.SourcePath` content to the configured bundle-relative path;
- support `WriteBundleOptions.Overwrite` through staged replacement;
- avoid leaving a completed bundle path without a valid manifest when writer
  staging or promotion fails where practical;
- compile public examples under `go test` where practical;
- prove consistency with distributor validation fixtures.

## Documentation Updates After Implementation

- Add Go package documentation under `pkg/bundle`.
- Update `README.md` to mention Go producer support.
- Update `docs/operations.md` with a producer integration example.
- Cross-reference `distributor manifest create` once that CLI command exists.

Keep this roadmap under `docs/roadmap/` until implemented.

## Implementation Stages

`docs/roadmap/implementation.md` intentionally splits this roadmap across two
implementation prompts: package core/building first, then the local bundle
writer. The split keeps the public API extraction separate from producer-side
bundle assembly.

Core/building stage:

1. Move or wrap the existing source manifest model, digest logic, path
   validation, and RFC3339 handling so `pkg/bundle` and internal validation use
   one contract.
2. Implement the locked Stage 2 public API symbols:
   `ManifestName`, `SchemaVersion`, `Manifest`, `ManifestFile`,
   `BuildOptions`, `WriteManifestOptions`, `ParseManifest`,
   `MarshalManifest`, `LoadManifest`, `WriteManifest`, `BuildManifest`,
   `ValidateManifest`, `ValidateBundle`, `ValidateSourcePath`, `FileDigest`,
   `BundleDigest`, and `CanonicalFilePayload`.
3. Add explicit-list and scan-based manifest building APIs, including zero
   `Created` defaulting to current UTC time.
4. Update internal packages to consume the shared implementation without
   changing current validation behavior.

Writer stage:

1. Implement the locked Stage 3 public API symbols: `BundleFile`,
   `WriteBundleOptions`, and `WriteBundle`.
2. Add the local bundle writer with staged promotion, atomic filesystem
   operations where practical, and overwrite behavior through sibling temp and
   backup paths.
3. Add package documentation and producer-facing examples.

## Decisions

- A zero `Created` value defaults to the current UTC time. Producers may still
  provide explicit timestamps for reproducible or backfilled bundles.
- Scan-based manifest building is included in v1, behind explicit options.
  Explicit file lists preserve caller order; scan mode sorts deterministically.
- A local bundle writer is included in v1. It should be safe and producer-side,
  but it must not expose distributor publication or storage internals.
- The exported API names and signatures in `Initial Exported API` are
  normative for implementation.

## Future Work

- Broader producer workflow helpers, such as richer ignore rules or template
  scaffolding, can be considered after the first public package exists.
- Remote or storage-backed producer writers remain out of scope unless a future
  producer use case requires them.