Update documentation to clarity bundle_id and idempotency_key usage and distinctions

docs/consumers/api.md

@@ -11,11 +11,13 @@ The upstream application needs these values from deployment or operator configur
 - distributor endpoint: the HTTP server base URL, such as `https://distributor.example.com`;
 - upload token: bearer token for exactly one configured `http_upload` pipeline;
 - generated files: regular local files to include in the source bundle;
-- bundle id: stable identifier for this producer output;
+- bundle id: stable identifier for the logical report stream or artifact;
-- idempotency key: stable key for retrying the same producer operation.
+- idempotency key: unique key for one producer run, reused only when retrying that same run.
 
 Do not put destination routing, public URLs, transform settings, or credentials in the source manifest. Those belong in the `distributor` pipeline configuration.
 
+The bundle id and idempotency key have different jobs. The bundle id tells `distributor` whether a new upload is a newer version of the same source; keep it stable across runs that should replace the same managed destination artifact. The idempotency key tells `distributor` whether an upload request is a retry; change it for each distinct producer run so new content is enqueued.
+
 ## Recommended Workflow
 
 Use `gitea.maximumdirect.net/eric/distributor/pkg/upload`.
@@ -53,7 +55,8 @@ func SubmitReport(reportPath, summaryPath string) error {
 		return fmt.Errorf("distributor endpoint and token are required")
 	}
 
-	reportID := "weather.hourly.brentwood.2026-06-07T15"
+	reportID := "weather.hourly.brentwood"
+	runID := time.Now().UTC().Format("20060102T150405.000000000Z")
 	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
 	defer cancel()
 
@@ -67,7 +70,7 @@ func SubmitReport(reportPath, summaryPath string) error {
 
 	result, err := client.UploadFiles(ctx, upload.UploadFilesOptions{
 		ID:             reportID,
-		IdempotencyKey: reportID,
+		IdempotencyKey: reportID + "." + runID,
 		Files: []bundle.BundleFile{
 			{SourcePath: reportPath, Path: "report.md"},
 			{SourcePath: summaryPath, Path: "summary.txt"},
@@ -88,8 +91,10 @@ func SubmitReport(reportPath, summaryPath string) error {
 
 ## Producer Responsibilities
 
-- Use a stable bundle id for the producer output, such as a report type plus logical timestamp.
+- Use a stable bundle id for the logical producer output that should replace the same destination artifact, such as `weather.hourly.brentwood`.
-- Use a stable idempotency key for cross-process retries of the same producer operation.
+- Do not include per-run timestamps, random values, or job ids in the bundle id unless each run should be treated as a different source.
+- Use an idempotency key that changes for every distinct producer run, such as `<bundle-id>.<run-id>`.
+- Reuse the same idempotency key only when retrying the exact same producer run with the same source manifest.
 - Map each generated file to a clean slash-separated bundle path, such as `report.md` or `assets/chart.png`.
 - Include only regular files. Symlinks, directories as files, devices, FIFOs, and sockets are rejected.
 - Keep file contents stable after upload inputs are selected. Bundle digests are calculated from file bytes.
@@ -99,9 +104,9 @@ Valid bundle paths are relative slash paths. They must not be empty, absolute, c
 
 ## Idempotency And Status
 
-`pkg/upload` sends `Idempotency-Key` on every upload. If the caller omits one, the package generates a random key for that call and reuses it for in-process retries. That is enough for transient network retry within one process.
+`pkg/upload` sends `Idempotency-Key` on every upload. If the caller omits one, the package generates a random key for that call and reuses it for in-process retries. That is enough for transient network retry within one process, but it does not give cross-process retry identity.
 
-For producer jobs that may retry after process restart, supply a stable key derived from the producer operation, such as the report id or job id. Reusing the same key with the same normalized source manifest returns the original accepted run. Reusing the same key with different source content returns a conflict.
+For producer jobs that may retry after process restart, supply a key derived from the producer run, such as `<bundle-id>.<run-id>`. Reusing the same key with the same normalized source manifest returns the original accepted run. Reusing the same key with different source content returns a conflict. Reusing one key across multiple distinct report generations prevents those generations from being treated as new uploads.
 
 `Status` polls `/runs/<run-id>` while the distributor server retains the in-memory status record. Status values are `accepted`, `queued`, `running`, `succeeded`, and `failed`. Completed records expire according to the server's `server.http.retention` setting, and server restart clears status and idempotency records.
 

docs/consumers/pkg-bundle.md

@@ -19,7 +19,7 @@ Use `WriteBundle` when producer-generated files live outside the final bundle ro
 ```go
 manifest, err := bundle.WriteBundle(bundle.WriteBundleOptions{
 	Root: "/var/spool/distributor/weather/hourly-2026-06-07T15",
-	ID:   "weather.hourly.brentwood.2026-06-07T15",
+	ID:   "weather.hourly.brentwood",
 	Files: []bundle.BundleFile{
 		{SourcePath: "/tmp/weather/report.md", Path: "report.md"},
 		{SourcePath: "/tmp/weather/summary.txt", Path: "summary.txt"},
@@ -41,7 +41,7 @@ Use `BuildManifest` and `WriteManifest` when files are already staged under the
 root := "/var/spool/distributor/weather/hourly-2026-06-07T15"
 manifest, err := bundle.BuildManifest(bundle.BuildOptions{
 	Root:  root,
-	ID:    "weather.hourly.brentwood.2026-06-07T15",
+	ID:    "weather.hourly.brentwood",
 	Files: []string{"report.md", "summary.txt"},
 })
 if err != nil {
@@ -72,6 +72,8 @@ Invalid paths include:
 
 Explicit file lists preserve caller order. File order is part of the bundle digest, so producers should choose it deliberately and keep it stable.
 
+The manifest `ID` is the logical source identity used by `distributor` destination comparison. Keep it stable for runs that should replace the same managed destination artifact. If every run uses a different manifest `ID`, `distributor` treats those runs as different sources and may report a destination conflict instead of replacing older output.
+
 ## Validation And Digest Helpers
 
 Use `ValidateBundle` before handing an existing local bundle to another process. It verifies manifest semantics, file existence, regular-file type, file size, per-file SHA-256 digests, and bundle digest.

docs/consumers/pkg-upload.md

@@ -40,8 +40,8 @@ Use `UploadFiles` when the producer has generated output files but has not assem
 
 ```go
 result, err := client.UploadFiles(ctx, upload.UploadFilesOptions{
-	ID:             "weather.hourly.brentwood.2026-06-07T15",
+	ID:             "weather.hourly.brentwood",
-	IdempotencyKey: "weather.hourly.brentwood.2026-06-07T15",
+	IdempotencyKey: "weather.hourly.brentwood.20260607T150000Z",
 	Files: []bundle.BundleFile{
 		{SourcePath: "/tmp/weather/report.md", Path: "report.md"},
 		{SourcePath: "/tmp/weather/summary.txt", Path: "summary.txt"},
@@ -62,7 +62,7 @@ Use `UploadBundle` when the producer already has a complete local bundle root co
 ```go
 result, err := client.UploadBundle(ctx, upload.UploadBundleOptions{
 	Root:           "/var/spool/weather/hourly-2026-06-07T15",
-	IdempotencyKey: "weather.hourly.brentwood.2026-06-07T15",
+	IdempotencyKey: "weather.hourly.brentwood.20260607T150000Z",
 })
 if err != nil {
 	return err
@@ -94,7 +94,9 @@ Status values are `accepted`, `queued`, `running`, `succeeded`, and `failed`. Co
 
 Every upload request includes `Idempotency-Key`.
 
-If `IdempotencyKey` is omitted, the client generates a random 128-bit lowercase hexadecimal key for that upload operation and reuses it for retries within the same call. For cross-process retry safety, producers should pass a stable key derived from the producer job or report id.
+If `IdempotencyKey` is omitted, the client generates a random 128-bit lowercase hexadecimal key for that upload operation and reuses it for retries within the same call. For cross-process retry safety, producers should pass a key derived from the producer run, such as `<bundle-id>.<run-id>`.
+
+Do not reuse the same idempotency key for multiple distinct report generations. Reuse it only when retrying the exact same run with the same source manifest. A repeated key with the same manifest returns the original accepted run instead of enqueueing another run; a repeated key with different content returns an idempotency conflict.
 
 The client retries only safe cases:
 

docs/integrations/http-upload.md

@@ -71,7 +71,7 @@ Retryable idempotency conflicts include:
 {"error":"upload idempotency key is already being processed","retryable":true}
 ```
 
-When `Idempotency-Key` is omitted, upload admission preserves the raw HTTP behavior: every valid accepted upload receives its own run id. When a key is supplied, the server records the accepted run after archive staging and source bundle validation succeed. Reusing the same key for the same authenticated pipeline and the same normalized source manifest returns the original `202 Accepted` response and does not enqueue another run. Reusing the same key for a different normalized source manifest returns `409 Conflict`.
+When `Idempotency-Key` is omitted, upload admission preserves the raw HTTP behavior: every valid accepted upload receives its own run id. When a key is supplied, the server records the accepted run after archive staging and source bundle validation succeed. Reusing the same key for the same authenticated pipeline and the same normalized source manifest returns the original `202 Accepted` response and does not enqueue another run. Reusing the same key for a different normalized source manifest returns `409 Conflict`. Producers should use a fresh key for each distinct producer run and reuse a key only for retries of that same run.
 
 ### `GET /runs/<run-id>`
 

docs/integrations/source-bundle.md

@@ -27,7 +27,7 @@ Current schema version: `1`.
 Required manifest fields:
 
 - `schema_version`: must be `1`.
-- `id`: non-empty bundle identifier.
+- `id`: non-empty bundle identifier. For replacement workflows, keep this stable for the logical source that should update the same managed destination artifact.
 - `digest`: lowercase `sha256:<64 hex>` digest of the ordered `files` list.
 - `created`: RFC3339 timestamp.
 - `files`: non-empty ordered list of file records.

docs/operations.md

@@ -138,7 +138,7 @@ curl -X POST http://127.0.0.1:8080/upload \
   --data-binary @bundle.tar.gz
 ```
 
-For safe producer retries, include an idempotency key that is stable for the producer operation:
+For safe producer retries, include an idempotency key that is stable for the same producer run and different for each distinct run:
 
 ```sh
 curl -X POST http://127.0.0.1:8080/upload \
@@ -150,7 +150,7 @@ curl -X POST http://127.0.0.1:8080/upload \
 
 Go producer applications can use `pkg/upload` instead of constructing archives and HTTP requests directly. See [Upstream Producer Integration](consumers/api.md) for the copyable producer implementation guide.
 
-The maintained example client uses the local upload server and reads the token from `DISTRIBUTOR_EXAMPLE_UPLOAD_TOKEN`. It generates an idempotency key by default; set `DISTRIBUTOR_EXAMPLE_UPLOAD_IDEMPOTENCY_KEY` when a retry must be stable across separate process runs.
+The maintained example client uses the local upload server and reads the token from `DISTRIBUTOR_EXAMPLE_UPLOAD_TOKEN`. It generates an idempotency key by default; set `DISTRIBUTOR_EXAMPLE_UPLOAD_IDEMPOTENCY_KEY` when retrying the same producer run across separate process runs.
 
 ```sh
 go run ./examples/upload-client