From 0382978af01f1f67529483d32e9882786f2407fe Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Mon, 1 Jun 2026 15:31:36 -0500 Subject: [PATCH] Fixed minor inconsistencies and ambiguities in the implementation roadmap --- docs/roadmap/cli_output_policy.md | 4 +- docs/roadmap/implementation.md | 10 +- docs/roadmap/public_bundle_package.md | 152 ++++++++++++++++++++------ 3 files changed, 125 insertions(+), 41 deletions(-) diff --git a/docs/roadmap/cli_output_policy.md b/docs/roadmap/cli_output_policy.md index e6cb5f8..02c05e7 100644 --- a/docs/roadmap/cli_output_policy.md +++ b/docs/roadmap/cli_output_policy.md @@ -61,8 +61,8 @@ Policy: - help and usage output remain text-only. - invalid `--format` values are usage errors. - `--format` is a per-command flag, not a root-global flag. -- commands that do not yet support JSON must fail validation if `--format json` - is accepted by their parser but not implemented. +- a command must not accept `--format json` unless it emits the shared JSON + envelope for that command. The first implementation should add JSON support for all current output-producing commands rather than leaving a mixed CLI where some commands diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index f789e7c..39756da 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -82,6 +82,8 @@ Implementation scope: - introduce `pkg/bundle` with the public source manifest model, schema version, digest logic, parsing, validation, explicit file-list building, scan-based building, and zero-`Created` defaulting to current UTC; +- implement the locked Stage 2 exported API symbols defined in + `docs/roadmap/public_bundle_package.md`; - update internal validation to consume the shared implementation without behavior drift; - keep destination state, publish planning, storage backends, transforms, @@ -111,8 +113,10 @@ Goal: implement the local bundle writer portion of Implementation scope: -- add producer-side local bundle writing with staging and atomic promotion where - practical; +- add producer-side local bundle writing with staging, atomic filesystem + operations where practical, and best-effort restore on overwrite failure; +- implement the locked Stage 3 exported API symbols defined in + `docs/roadmap/public_bundle_package.md`; - keep the writer filesystem-local and producer-focused; - do not expose distributor storage backends or publication behavior through the public package. @@ -124,7 +128,7 @@ Documentation updates after implementation: Tests: -- writer creates a complete valid local bundle; +- writer creates a complete valid local bundle through staged promotion; - writer output validates through distributor's normal validation path; - overwrite and failure behavior avoid leaving a completed bundle path without a valid manifest where practical. diff --git a/docs/roadmap/public_bundle_package.md b/docs/roadmap/public_bundle_package.md index 7d1f944..36b256e 100644 --- a/docs/roadmap/public_bundle_package.md +++ b/docs/roadmap/public_bundle_package.md @@ -64,49 +64,104 @@ The package should own only producer-side source bundle concerns: Prefer options structs over long positional functions so future additive behavior can be introduced without avoidable API churn. -Conceptual shape: +## Initial Exported API + +The initial `pkg/bundle` API is locked to the following exported constants, +types, and functions. Implementation should not rename, remove, or reshape +these public symbols during the initial implementation pass. ```go +const ManifestName = "manifest.json" +const SchemaVersion = 1 + +type Manifest struct { + SchemaVersion int `json:"schema_version"` + ID string `json:"id"` + Digest string `json:"digest"` + Created time.Time `json:"created"` + Files []ManifestFile `json:"files"` +} + +type ManifestFile struct { + Path string `json:"path"` + SHA256 string `json:"sha256"` + Size int64 `json:"size"` +} + type BuildOptions struct { Root string ID string - Created time.Time // zero means current UTC time + Created time.Time Files []string + Scan bool } -func BuildManifest(opts BuildOptions) (Manifest, error) -func WriteManifest(root string, manifest Manifest) error -func LoadManifest(root string) (Manifest, error) -func ValidateManifest(manifest Manifest) error -func ValidateBundle(root string, manifest Manifest) error -``` - -Exact API names can be refined during implementation, but the first public API -should stay narrow. - -The initial package should also include a local bundle writer API. The writer -should remain producer-side and filesystem-local; it should not expose -distributor storage backends or publication behavior. Its purpose is to let -producers assemble files, write a compliant manifest, and promote the completed -bundle safely. - -Conceptual shape: - -```go -type WriterOptions struct { - Root string - ID string - Created time.Time // zero means current UTC time +type WriteManifestOptions struct { Overwrite bool } -func WriteBundle(opts WriterOptions, files []InputFile) (Manifest, error) +type BundleFile struct { + SourcePath string + Path string +} + +type WriteBundleOptions struct { + Root string + ID string + Created time.Time + Files []BundleFile + Overwrite bool +} + +func ParseManifest(data []byte) (Manifest, error) +func MarshalManifest(manifest Manifest) ([]byte, error) +func LoadManifest(root string) (Manifest, error) +func WriteManifest(root string, manifest Manifest, opts WriteManifestOptions) error +func BuildManifest(opts BuildOptions) (Manifest, error) +func ValidateManifest(manifest Manifest) error +func ValidateBundle(root string, manifest Manifest) error +func WriteBundle(opts WriteBundleOptions) (Manifest, error) +func ValidateSourcePath(path string) error +func FileDigest(data []byte) string +func BundleDigest(files []ManifestFile) string +func CanonicalFilePayload(files []ManifestFile) string ``` -The implementation may choose a different concrete API, but it should support -temp-directory staging plus atomic rename where practical for the final bundle -promotion. It should not hide partial-file write errors or leave a completed -bundle path that lacks a valid `manifest.json`. +## API Semantics + +`BuildManifest` requires `Root`, `ID`, and exactly one file-selection mode: +explicit `Files` or `Scan: true`. + +Explicit `Files` preserve caller order. `Scan: true` recursively scans `Root`, +includes regular files including dotfiles, excludes `manifest.json` and +`.distributor.json`, rejects symlinks, and sorts by slash-separated relative +path. + +Zero `Created` values default to the current UTC time. All public path fields +use slash-separated bundle-relative paths. + +`MarshalManifest` validates before marshaling and emits deterministic JSON with +fixed field order and a trailing newline. + +`WriteManifest` writes `manifest.json`; it fails if the file exists unless +`WriteManifestOptions.Overwrite` is true, and it uses temp-and-rename +replacement where practical. + +`ValidateManifest` checks manifest-only semantics. `ValidateBundle` checks the +supplied manifest against local files under `root`, including existence, +regular-file type, size, SHA-256, path safety, duplicates, and bundle digest. + +`WriteBundle` copies existing local files from `BundleFile.SourcePath` into a +staged bundle at `BundleFile.Path`, writes a compliant manifest, validates the +staged bundle, and promotes it to `WriteBundleOptions.Root`. + +`WriteBundleOptions.Overwrite` permits replacing an existing bundle root. +Replacement must build the new bundle completely before touching the existing +root, then use sibling temp and backup paths for best-effort promotion and +restore on failure. + +The writer remains producer-side and filesystem-local. It must not expose +distributor storage backends or publication behavior. ## Manifest Compatibility @@ -158,7 +213,12 @@ Suggested coverage: - validate generated manifests successfully; - reject unsafe paths, missing files, non-regular files, and symlinks; - emit slash-separated JSON paths; +- parse, marshal, load, and write manifests through the exact exported API; +- fail `WriteManifest` when `manifest.json` exists unless overwrite is enabled; +- emit deterministic manifest JSON with fixed field order and trailing newline; - write a complete local bundle through the public writer; +- copy `BundleFile.SourcePath` content to the configured bundle-relative path; +- support `WriteBundleOptions.Overwrite` through staged replacement; - avoid leaving a completed bundle path without a valid manifest when writer staging or promotion fails where practical; - compile public examples under `go test` where practical; @@ -175,17 +235,35 @@ Keep this roadmap under `docs/roadmap/` until implemented. ## Implementation Stages +`docs/roadmap/implementation.md` intentionally splits this roadmap across two +implementation prompts: package core/building first, then the local bundle +writer. The split keeps the public API extraction separate from producer-side +bundle assembly. + +Core/building stage: + 1. Move or wrap the existing source manifest model, digest logic, path validation, and RFC3339 handling so `pkg/bundle` and internal validation use one contract. -2. Add explicit-list and scan-based manifest building APIs, including zero +2. Implement the locked Stage 2 public API symbols: + `ManifestName`, `SchemaVersion`, `Manifest`, `ManifestFile`, + `BuildOptions`, `WriteManifestOptions`, `ParseManifest`, + `MarshalManifest`, `LoadManifest`, `WriteManifest`, `BuildManifest`, + `ValidateManifest`, `ValidateBundle`, `ValidateSourcePath`, `FileDigest`, + `BundleDigest`, and `CanonicalFilePayload`. +3. Add explicit-list and scan-based manifest building APIs, including zero `Created` defaulting to current UTC time. -3. Add load, write, and validation helpers for source `manifest.json`. -4. Add the local bundle writer with staging and atomic promotion where - practical. -5. Update internal packages to consume the shared implementation without +4. Update internal packages to consume the shared implementation without changing current validation behavior. -6. Add package documentation and producer-facing examples. + +Writer stage: + +1. Implement the locked Stage 3 public API symbols: `BundleFile`, + `WriteBundleOptions`, and `WriteBundle`. +2. Add the local bundle writer with staged promotion, atomic filesystem + operations where practical, and overwrite behavior through sibling temp and + backup paths. +3. Add package documentation and producer-facing examples. ## Decisions @@ -195,6 +273,8 @@ Keep this roadmap under `docs/roadmap/` until implemented. Explicit file lists preserve caller order; scan mode sorts deterministically. - A local bundle writer is included in v1. It should be safe and producer-side, but it must not expose distributor publication or storage internals. +- The exported API names and signatures in `Initial Exported API` are + normative for implementation. ## Future Work