distributor/docs/roadmap/public_bundle_package.md

Roadmap: Public Bundle Manifest Package

Purpose

Expose a small public Go package that producer applications can import to create valid distributor source bundle manifests.

The package should encode the producer-side source bundle contract without exposing distributor's publication, storage, transform, destination state, notification, or config internals.

Current Implementation Grounding

The current implementation keeps source bundle behavior in internal/bundle:

• Manifest and ManifestFile model manifest.json;
• ParseManifest parses JSON and RFC3339 created timestamps;
• ValidateManifest owns schema version, digest format, file path, duplicate, size, and bundle digest validation;
• FileDigest, BundleDigest, and CanonicalFilePayload define digest behavior;
• source validation rejects unsafe paths, reserved distributor metadata paths, non-file entries, size mismatches, SHA-256 mismatches, and bundle digest mismatches.

Producers cannot import internal/bundle, so Go producers currently need to duplicate this contract or shell out to future CLI tooling.

Goals

• Provide a stable producer-facing Go API for manifest creation and validation.
• Reuse the same source manifest, digest, path safety, and RFC3339 behavior used by distributor validation.
• Keep the public API intentionally small and producer-only.
• Include a safe bundle writer so producers can create complete local bundle directories without hand-rolling manifest-write and staging behavior.
• Make the future distributor manifest create command a thin wrapper over this package.
• Avoid exposing destination state, publish planning, storage backends, transforms, notifications, or config.

Non-Goals

• Do not expose the distributor runner or publication workflow as public API.
• Do not expose storage backends or destination .distributor.json state.
• Do not add domain-specific manifest metadata.
• Do not require non-Go producers to use Go APIs.
• Do not implement latest paths, link generation, transforms, or notification behavior in this package.

Package Boundary

Use pkg/bundle as the public package name.

The package should own only producer-side source bundle concerns:

• manifest model and schema version constant;
• file digest and bundle digest calculation;
• manifest building from producer files;
• manifest JSON load/write helpers;
• manifest and bundle validation;
• source path safety matching distributor validation.

Prefer options structs over long positional functions so future additive behavior can be introduced without avoidable API churn.

Conceptual shape:

type BuildOptions struct {
    Root    string
    ID      string
    Created time.Time // zero means current UTC time
    Files   []string
}

func BuildManifest(opts BuildOptions) (Manifest, error)
func WriteManifest(root string, manifest Manifest) error
func LoadManifest(root string) (Manifest, error)
func ValidateManifest(manifest Manifest) error
func ValidateBundle(root string, manifest Manifest) error

Exact API names can be refined during implementation, but the first public API should stay narrow.

The initial package should also include a local bundle writer API. The writer should remain producer-side and filesystem-local; it should not expose distributor storage backends or publication behavior. Its purpose is to let producers assemble files, write a compliant manifest, and promote the completed bundle safely.

Conceptual shape:

type WriterOptions struct {
    Root      string
    ID        string
    Created   time.Time // zero means current UTC time
    Overwrite bool
}

func WriteBundle(opts WriterOptions, files []InputFile) (Manifest, error)

The implementation may choose a different concrete API, but it should support temp-directory staging plus atomic rename where practical for the final bundle promotion. It should not hide partial-file write errors or leave a completed bundle path that lacks a valid manifest.json.

Manifest Compatibility

The package should treat the source manifest schema as a compatibility boundary:

• export the current schema version;
• preserve JSON field names exactly;
• use lowercase sha256:<64 hex> digests;
• use slash-separated relative paths in JSON;
• use RFC3339 timestamps;
• default a zero build or writer Created value to the current UTC time;
• preserve caller-provided file order for explicit file lists;
• produce deterministic ordering when scan-based building is selected;
• reject symlinks if distributor validation still rejects source symlinks.

Scan-based building belongs in v1 of the public package. Explicit file lists should preserve caller order. Scan mode should sort by slash-separated relative path and share the same filtering rules expected by future CLI manifest creation.

The internal implementation may either move source-bundle core logic into pkg/bundle and have internal packages consume it, or keep internal wrappers around public core logic. The important invariant is that public package, future CLI manifest creation, and distributor validation must not drift.

Relationship To Other Roadmaps

distributor manifest create should call pkg/bundle rather than maintaining a separate manifest builder.

Remote validate and inspect should continue using distributor's storage abstraction and internal app wiring; they do not need public producer APIs.

HTML index mode, link generation, and latest path destinations operate after a bundle has already entered distributor and should not affect this package.

Testing Expectations

Suggested coverage:

• build a manifest from explicit files;
• build a manifest by scanning a local bundle root;
• preserve explicit file order;
• sort scan results deterministically by slash-separated relative path;
• compute per-file SHA-256 and size;
• compute the expected canonical bundle digest;
• write and load manifest.json;
• default zero Created to current UTC time while honoring explicit timestamps;
• validate generated manifests successfully;
• reject unsafe paths, missing files, non-regular files, and symlinks;
• emit slash-separated JSON paths;
• write a complete local bundle through the public writer;
• avoid leaving a completed bundle path without a valid manifest when writer staging or promotion fails where practical;
• compile public examples under go test where practical;
• prove consistency with distributor validation fixtures.

Documentation Updates After Implementation

• Add Go package documentation under pkg/bundle.
• Update README.md to mention Go producer support.
• Update docs/operations.md with a producer integration example.
• Cross-reference distributor manifest create once that CLI command exists.

Keep this roadmap under docs/roadmap/ until implemented.

Implementation Stages

1. Move or wrap the existing source manifest model, digest logic, path validation, and RFC3339 handling so pkg/bundle and internal validation use one contract.
2. Add explicit-list and scan-based manifest building APIs, including zero Created defaulting to current UTC time.
3. Add load, write, and validation helpers for source manifest.json.
4. Add the local bundle writer with staging and atomic promotion where practical.
5. Update internal packages to consume the shared implementation without changing current validation behavior.
6. Add package documentation and producer-facing examples.

Decisions

• A zero Created value defaults to the current UTC time. Producers may still provide explicit timestamps for reproducible or backfilled bundles.
• Scan-based manifest building is included in v1, behind explicit options. Explicit file lists preserve caller order; scan mode sorts deterministically.
• A local bundle writer is included in v1. It should be safe and producer-side, but it must not expose distributor publication or storage internals.

Future Work

• Broader producer workflow helpers, such as richer ignore rules or template scaffolding, can be considered after the first public package exists.
• Remote or storage-backed producer writers remain out of scope unless a future producer use case requires them.