2.1 KiB
Ingestion Internals
Purpose
internal/ingest stages uploaded source bundle archives into local per-run directories. It does not authenticate requests, manage upload queues, publish destinations, or start an HTTP server.
Archive staging
ValidateContentType owns accepted upload content-type policy for archive
staging callers.
StageArchive accepts one upload body, content type, pipeline staging path, run id, and explicit size and file-count limits. It writes the request body to temporary storage while enforcing the configured upload size limit, extracts the archive into temporary local storage, validates the extracted source bundle, and then commits the validated bundle to:
<pipeline staging path>/<run id>
The returned StagedBundle.Root is a local filesystem path to the validated source bundle root.
Accepted archive formats
The package accepts only:
application/x-tarapplication/gzipapplication/x-gzip
Gzip uploads must contain a tar archive.
Extraction rules
Archive entry paths must be clean relative slash-separated paths. Extraction rejects absolute paths, path traversal, backslash paths, duplicate files, symlinks, hardlinks, devices, sockets, and other special entries.
The archive must contain exactly one root-level manifest.json. Nested manifests are rejected.
Regular files and directories are the only accepted tar entries. Regular file extraction enforces the explicit maximum extracted byte count and maximum file count supplied by the caller.
Bundle validation
After extraction, the package loads and validates the staged bundle through pkg/bundle. Manifest parsing, source path validation, file existence checks, regular-file checks, file sizes, file SHA-256 digests, and bundle digest validation use the existing source bundle contract.
Validation happens before the staged bundle is committed to its final per-run path.
Failure behavior
Failed staging removes temporary archive and extraction data created by the package. A failed call does not publish anything and does not leave a committed per-run bundle directory.