Compare commits
13 Commits
1fc282f796
...
v0.2.0
| Author | SHA1 | Date | |
|---|---|---|---|
| 9aaa1e9426 | |||
| bb276101d2 | |||
| 67dc07435d | |||
| 9684ffd37f | |||
| eba4d6dd56 | |||
| d5e3aa7a44 | |||
| c372a02357 | |||
| 07f1eb2148 | |||
| 4b6a0a3b74 | |||
| 42fb4aa82a | |||
| c4cfd3fc74 | |||
| 004283fc0a | |||
| 063a13b1c9 |
@@ -26,9 +26,22 @@ The runner:
|
||||
|
||||
Destination failures are collected while later destinations continue to run. Source open and source discovery failures stop the run because there are no valid bundles to fan out.
|
||||
|
||||
## Run implementation
|
||||
|
||||
`run.go` contains the public `Run` entrypoint and the main configuration orchestration path. Package-local run helpers are grouped by responsibility:
|
||||
|
||||
- `run_selection.go`: destination bundle selection, path mapping decisions, and fixed-path warnings;
|
||||
- `run_warnings.go`: secret and SSH warning data;
|
||||
- `run_output.go`: text plan lines, JSON action records, and output projections;
|
||||
- `run_summary.go`: summary counters and JSON summary records;
|
||||
- `run_failures.go`: destination failure aggregation and partial-result detection;
|
||||
- `run_notify.go`: notification event projection and action filtering.
|
||||
|
||||
These helpers remain in `internal/app` because command output, warning collection, destination failure aggregation, notifier handoff, and backend construction are app-owned orchestration concerns.
|
||||
|
||||
## Backend and transform wiring
|
||||
|
||||
The app-level backend factory registers local, SSH, and S3 backends for execution. S3 explicit credential references are resolved through the config environment resolver before adapter construction.
|
||||
The app-level backend factory registers local, SSH, and S3 backends for execution. Source and destination backend config is converted through a shared app-local open spec before adapter construction. S3 explicit credential references are resolved through the config environment resolver.
|
||||
|
||||
The app-level transform registry registers Markdown-to-HTML using `internal/transform/markdown`. Lower-level publish code receives a resolver and does not import concrete transform implementations.
|
||||
|
||||
|
||||
@@ -20,6 +20,8 @@ The source manifest requires:
|
||||
|
||||
Each file requires `path`, `sha256`, and `size`. Digests must use lowercase `sha256:<64 hex>` format. `created` must parse as RFC3339.
|
||||
|
||||
`pkg/bundle.ValidateDigest` is the canonical digest format validator for producer-facing and internal code. `internal/bundle.ValidateDigest` delegates to that public validator so source manifests and destination state use the same digest grammar.
|
||||
|
||||
## Validation
|
||||
|
||||
`pkg/bundle.ValidateManifest` owns normalized source manifest semantics: schema version, id, digest format, timestamp presence, file list presence, source path safety, duplicate file paths, reserved paths, file digest format, non-negative file sizes, and the top-level bundle digest.
|
||||
|
||||
27
docs/internal/link.md
Normal file
27
docs/internal/link.md
Normal file
@@ -0,0 +1,27 @@
|
||||
# Link URL Policy
|
||||
|
||||
## Purpose
|
||||
|
||||
`internal/link` defines shared validation for configured and persisted HTTP link URLs.
|
||||
|
||||
## Inputs and outputs
|
||||
|
||||
Input is a URL string. Output is either nil for an accepted URL or a concise validation error that callers wrap with field context.
|
||||
|
||||
## Validation behavior
|
||||
|
||||
Accepted URLs must parse successfully, use `http` or `https`, include a host, and omit query strings and fragments.
|
||||
|
||||
## Boundaries
|
||||
|
||||
This package validates URL shape only. It does not construct destination output URLs, choose primary URLs, infer public URLs from backend configuration, or read configuration files.
|
||||
|
||||
## Tests
|
||||
|
||||
Before changing link URL policy, inspect tests under `internal/link` and callers in `internal/config`, `internal/state`, and `internal/publish`.
|
||||
|
||||
## Invariants
|
||||
|
||||
- Configured `links.base_url`, persisted `links.primary_url`, persisted output `url`, and publish link planning use the same URL policy.
|
||||
- Callers own field-specific error context.
|
||||
- URL path construction remains in `internal/publish`.
|
||||
@@ -26,6 +26,8 @@ The package publishes source files and Markdown-to-HTML outputs. Markdown sideca
|
||||
|
||||
The package uses `internal/state` for destination comparison, `internal/storage` for IO, and the shared `internal/config` publish/transform policy helper for request validation. It resolves transforms through a narrow resolver supplied by the caller; concrete transform registration is owned by the app layer. It does not parse CLI flags, load config files, or choose which source bundles a destination receives.
|
||||
|
||||
The package owns projection from planned publish outputs to destination state output records and managed destination output paths. App JSON results and notification events keep their own schemas, but may use the publish output projection to avoid field-mapping drift.
|
||||
|
||||
The app layer computes the destination bundle path before planning. `preserve_relative` destinations pass the source-root-relative bundle path. `fixed` destinations pass an empty destination bundle path, which means the destination backend root, and pass only the newest selected source bundle for that destination.
|
||||
|
||||
When link config is present, publish planning builds per-output URLs from `links.base_url`, the destination bundle path, and each output path. `index.html` outputs use directory-style URLs. The primary URL is selected from planned outputs according to the destination primary policy.
|
||||
|
||||
@@ -26,6 +26,19 @@ Storage errors use typed categories such as not found, already exists, invalid p
|
||||
|
||||
Backends may wrap implementation-specific errors, but callers should receive storage errors where practical. Traversal can stop cleanly with `ErrStopWalk`.
|
||||
|
||||
## Traversal helpers
|
||||
|
||||
Backends own their traversal mechanics. The local adapter owns filesystem walking, the SSH adapter owns SFTP directory walking, and the S3 adapter owns object listing and pagination.
|
||||
|
||||
`internal/storage` owns the shared callback emission rules used by backends:
|
||||
|
||||
- context cancellation is checked before callback emission;
|
||||
- `WalkOptions.Limit` bounds the number of emitted entries;
|
||||
- `ErrStopWalk` stops traversal without becoming a caller-visible error;
|
||||
- callback errors are wrapped as storage walk errors.
|
||||
|
||||
`storage.HasAny(ctx, backend, prefix)` provides the shared destination-content check. It calls `Walk` with non-recursive, limit-one traversal and stops after the first emitted entry.
|
||||
|
||||
## Deletion
|
||||
|
||||
`DeleteManagedBundle` may delete listed managed outputs plus `.distributor.json`.
|
||||
|
||||
@@ -101,7 +101,7 @@ Do not edit `.distributor.json` by hand during normal operation. If it is missin
|
||||
|
||||
## Go Producer Bundles
|
||||
|
||||
Go producer applications can import `gitea.maximumdirect.net/eric/distributor/pkg/bundle` to create complete local source bundles with the same path, digest, timestamp, and validation rules used by `distributor`.
|
||||
Go producer applications can import `gitea.maximumdirect.net/eric/distributor/pkg/bundle` to create complete local source bundles with the same path, digest, timestamp, and validation rules used by `distributor`. The package also exposes digest helpers, including `ValidateDigest`, for producer code that needs to validate lowercase `sha256:<64 hex>` strings before writing manifests.
|
||||
|
||||
Minimal producer-side bundle creation:
|
||||
|
||||
|
||||
@@ -202,6 +202,7 @@ Use this current layout unless the project has a documented reason to differ:
|
||||
- `internal/config`: configuration structs, defaults, loading, precedence, and validation.
|
||||
- `internal/bundle`: storage-backed source bundle discovery and validation over the public manifest contract.
|
||||
- `internal/state`: `.distributor.json` parsing, validation, comparison, and output metadata.
|
||||
- `internal/link`: shared HTTP URL validation for configured and persisted link metadata.
|
||||
- `internal/storage`: backend interfaces, shared path/resource types, backend registry, and storage errors.
|
||||
- `internal/adapters/local`: local filesystem backend.
|
||||
- `internal/adapters/ssh`: SSH/SFTP backend.
|
||||
|
||||
@@ -161,31 +161,10 @@ func (b *Backend) Walk(ctx context.Context, prefix string, opts storage.WalkOpti
|
||||
}
|
||||
return b.translateError(storage.OpWalk, prefix, err)
|
||||
}
|
||||
visited := 0
|
||||
emit := func(entry storage.Entry) error {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
if opts.Limit > 0 && visited >= opts.Limit {
|
||||
return storage.ErrStopWalk
|
||||
}
|
||||
visited++
|
||||
if err := fn(entry); err != nil {
|
||||
if errors.Is(err, storage.ErrStopWalk) {
|
||||
return storage.ErrStopWalk
|
||||
}
|
||||
return storage.NewError(storage.OpWalk, backendName, entry.Path, storage.ErrUnknown, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
emitter := storage.NewWalkEmitter(ctx, backendName, opts, fn)
|
||||
|
||||
if !info.IsDir() {
|
||||
if err := emit(entryFromInfo(prefix, info)); errors.Is(err, storage.ErrStopWalk) {
|
||||
return nil
|
||||
} else if err != nil {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
return storage.FinishWalk(emitter.Emit(entryFromInfo(prefix, info)))
|
||||
}
|
||||
|
||||
walkErr := filepath.WalkDir(nativePrefix, func(nativePath string, dirEntry fs.DirEntry, err error) error {
|
||||
@@ -210,24 +189,13 @@ func (b *Backend) Walk(ctx context.Context, prefix string, opts storage.WalkOpti
|
||||
if err != nil {
|
||||
return b.translateError(storage.OpWalk, logicalPath, err)
|
||||
}
|
||||
return emit(entryFromInfo(logicalPath, info))
|
||||
return emitter.Emit(entryFromInfo(logicalPath, info))
|
||||
})
|
||||
if errors.Is(walkErr, storage.ErrStopWalk) {
|
||||
return nil
|
||||
}
|
||||
return walkErr
|
||||
return storage.FinishWalk(walkErr)
|
||||
}
|
||||
|
||||
func (b *Backend) HasAny(ctx context.Context, prefix string) (bool, error) {
|
||||
found := false
|
||||
err := b.Walk(ctx, prefix, storage.WalkOptions{Recursive: false, Limit: 1}, func(storage.Entry) error {
|
||||
found = true
|
||||
return storage.ErrStopWalk
|
||||
})
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
return found, nil
|
||||
return storage.HasAny(ctx, b, prefix)
|
||||
}
|
||||
|
||||
func (b *Backend) DeleteManagedBundle(ctx context.Context, bundlePath string, managedOutputPaths []string, opts storage.DeleteOptions) error {
|
||||
|
||||
@@ -183,55 +183,25 @@ func (b *Backend) Walk(ctx context.Context, logicalPrefix string, opts storage.W
|
||||
if err := storage.ValidatePrefix(logicalPrefix); err != nil {
|
||||
return err
|
||||
}
|
||||
visited := 0
|
||||
emit := func(entry storage.Entry) error {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
if opts.Limit > 0 && visited >= opts.Limit {
|
||||
return storage.ErrStopWalk
|
||||
}
|
||||
visited++
|
||||
if err := fn(entry); err != nil {
|
||||
if errors.Is(err, storage.ErrStopWalk) {
|
||||
return storage.ErrStopWalk
|
||||
}
|
||||
return storage.NewError(storage.OpWalk, BackendName, entry.Path, storage.ErrUnknown, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
emitter := storage.NewWalkEmitter(ctx, BackendName, opts, fn)
|
||||
if logicalPrefix != "" {
|
||||
entry, err := b.Stat(ctx, logicalPrefix)
|
||||
if err == nil {
|
||||
if err := emit(entry); errors.Is(err, storage.ErrStopWalk) {
|
||||
return nil
|
||||
} else if err != nil {
|
||||
return err
|
||||
if err := emitter.Emit(entry); err != nil {
|
||||
return storage.FinishWalk(err)
|
||||
}
|
||||
if opts.Limit > 0 && visited >= opts.Limit {
|
||||
if emitter.LimitReached() {
|
||||
return nil
|
||||
}
|
||||
} else if !storage.IsNotFound(err) {
|
||||
return err
|
||||
}
|
||||
}
|
||||
err := b.walkObjects(ctx, logicalPrefix, opts, emit)
|
||||
if errors.Is(err, storage.ErrStopWalk) {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
return storage.FinishWalk(b.walkObjects(ctx, logicalPrefix, opts, emitter.Emit))
|
||||
}
|
||||
|
||||
func (b *Backend) HasAny(ctx context.Context, logicalPrefix string) (bool, error) {
|
||||
found := false
|
||||
err := b.Walk(ctx, logicalPrefix, storage.WalkOptions{Recursive: false, Limit: 1}, func(storage.Entry) error {
|
||||
found = true
|
||||
return storage.ErrStopWalk
|
||||
})
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
return found, nil
|
||||
return storage.HasAny(ctx, b, logicalPrefix)
|
||||
}
|
||||
|
||||
func (b *Backend) DeleteManagedBundle(ctx context.Context, bundlePath string, managedOutputPaths []string, opts storage.DeleteOptions) error {
|
||||
|
||||
@@ -209,51 +209,17 @@ func (b *Backend) Walk(ctx context.Context, prefix string, opts storage.WalkOpti
|
||||
return b.translateError(storage.OpWalk, prefix, err)
|
||||
}
|
||||
|
||||
visited := 0
|
||||
emit := func(entry storage.Entry) error {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
if opts.Limit > 0 && visited >= opts.Limit {
|
||||
return storage.ErrStopWalk
|
||||
}
|
||||
visited++
|
||||
if err := fn(entry); err != nil {
|
||||
if errors.Is(err, storage.ErrStopWalk) {
|
||||
return storage.ErrStopWalk
|
||||
}
|
||||
return storage.NewError(storage.OpWalk, BackendName, entry.Path, storage.ErrUnknown, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
emitter := storage.NewWalkEmitter(ctx, BackendName, opts, fn)
|
||||
|
||||
if !info.IsDir() {
|
||||
if err := emit(entryFromInfo(prefix, info)); errors.Is(err, storage.ErrStopWalk) {
|
||||
return nil
|
||||
} else if err != nil {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
return storage.FinishWalk(emitter.Emit(entryFromInfo(prefix, info)))
|
||||
}
|
||||
|
||||
if err := b.walkDirectory(ctx, prefix, nativePrefix, opts, emit); errors.Is(err, storage.ErrStopWalk) {
|
||||
return nil
|
||||
} else if err != nil {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
return storage.FinishWalk(b.walkDirectory(ctx, prefix, nativePrefix, opts, emitter.Emit))
|
||||
}
|
||||
|
||||
func (b *Backend) HasAny(ctx context.Context, prefix string) (bool, error) {
|
||||
found := false
|
||||
err := b.Walk(ctx, prefix, storage.WalkOptions{Recursive: false, Limit: 1}, func(storage.Entry) error {
|
||||
found = true
|
||||
return storage.ErrStopWalk
|
||||
})
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
return found, nil
|
||||
return storage.HasAny(ctx, b, prefix)
|
||||
}
|
||||
|
||||
func (b *Backend) DeleteManagedBundle(ctx context.Context, bundlePath string, managedOutputPaths []string, opts storage.DeleteOptions) error {
|
||||
|
||||
@@ -37,6 +37,22 @@ type backendFactory struct {
|
||||
readOnlyKnownHosts bool
|
||||
}
|
||||
|
||||
type backendOpenSpec struct {
|
||||
role string
|
||||
backend string
|
||||
path string
|
||||
host string
|
||||
user string
|
||||
port int
|
||||
ssh config.SSH
|
||||
endpoint string
|
||||
bucket string
|
||||
prefix string
|
||||
region string
|
||||
forcePath *bool
|
||||
credentials config.Credentials
|
||||
}
|
||||
|
||||
func newBackendFactory() *backendFactory {
|
||||
return newBackendFactoryWithEnvironment(config.ProcessEnvironment())
|
||||
}
|
||||
@@ -91,25 +107,11 @@ func newBackendFactoryWithEnvironment(environment config.Environment) *backendFa
|
||||
}
|
||||
|
||||
func (f *backendFactory) openSource(ctx context.Context, source config.Backend) (storage.Backend, error) {
|
||||
if source.Backend != config.BackendLocal && source.Backend != config.BackendSSH && source.Backend != config.BackendS3 {
|
||||
return nil, fmt.Errorf("source backend %s is not implemented for execution", source.Backend)
|
||||
}
|
||||
openConfig, err := f.sourceOpenConfig(source)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return f.registry.Open(ctx, source.Backend, openConfig)
|
||||
return f.openBackend(ctx, backendOpenSpecFromSource(source))
|
||||
}
|
||||
|
||||
func (f *backendFactory) openDestination(ctx context.Context, destination config.Destination) (storage.Backend, error) {
|
||||
if destination.Backend != config.BackendLocal && destination.Backend != config.BackendSSH && destination.Backend != config.BackendS3 {
|
||||
return nil, fmt.Errorf("backend %s is not implemented for execution", destination.Backend)
|
||||
}
|
||||
openConfig, err := f.destinationOpenConfig(destination)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return f.registry.Open(ctx, destination.Backend, openConfig)
|
||||
return f.openBackend(ctx, backendOpenSpecFromDestination(destination))
|
||||
}
|
||||
|
||||
func (f *backendFactory) openLocalPath(ctx context.Context, path string) (storage.Backend, error) {
|
||||
@@ -120,40 +122,51 @@ func (f *backendFactory) resolveCredentials(creds config.Credentials) (config.Re
|
||||
return f.environment.ResolveCredentials(creds)
|
||||
}
|
||||
|
||||
func (f *backendFactory) sourceOpenConfig(source config.Backend) (storage.OpenConfig, error) {
|
||||
cfg := sourceOpenConfig(source)
|
||||
if source.Backend == config.BackendS3 {
|
||||
if err := f.addS3Config(cfg, source.Endpoint, source.Bucket, source.Prefix, source.Region, source.ForcePath, source.Creds); err != nil {
|
||||
func (f *backendFactory) openBackend(ctx context.Context, spec backendOpenSpec) (storage.Backend, error) {
|
||||
if !backendExecutable(spec.backend) {
|
||||
if spec.role == "source" {
|
||||
return nil, fmt.Errorf("source backend %s is not implemented for execution", spec.backend)
|
||||
}
|
||||
return nil, fmt.Errorf("backend %s is not implemented for execution", spec.backend)
|
||||
}
|
||||
openConfig, err := f.openConfig(spec)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return f.registry.Open(ctx, spec.backend, openConfig)
|
||||
}
|
||||
|
||||
func backendExecutable(name string) bool {
|
||||
return name == config.BackendLocal || name == config.BackendSSH || name == config.BackendS3
|
||||
}
|
||||
|
||||
func (f *backendFactory) openConfig(spec backendOpenSpec) (storage.OpenConfig, error) {
|
||||
cfg := storage.OpenConfig{storagePathKey: spec.path}
|
||||
switch spec.backend {
|
||||
case config.BackendSSH:
|
||||
cfg[sshHostKey] = spec.host
|
||||
cfg[sshUserKey] = spec.user
|
||||
cfg[sshPortKey] = strconv.Itoa(spec.port)
|
||||
cfg[sshKeyFileKey] = spec.ssh.KeyFile
|
||||
cfg[sshKnownHostsKey] = spec.ssh.KnownHosts
|
||||
cfg[sshHostKeyPolicyKey] = string(spec.ssh.HostKeyPolicy)
|
||||
cfg[sshReadOnlyHostsKey] = strconv.FormatBool(f.readOnlyKnownHosts)
|
||||
case config.BackendS3:
|
||||
if err := f.addS3Config(cfg, spec); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
if source.Backend == config.BackendSSH {
|
||||
cfg[sshReadOnlyHostsKey] = strconv.FormatBool(f.readOnlyKnownHosts)
|
||||
}
|
||||
return cfg, nil
|
||||
}
|
||||
|
||||
func (f *backendFactory) destinationOpenConfig(destination config.Destination) (storage.OpenConfig, error) {
|
||||
cfg := destinationOpenConfig(destination)
|
||||
if destination.Backend == config.BackendS3 {
|
||||
if err := f.addS3Config(cfg, destination.Endpoint, destination.Bucket, destination.Prefix, destination.Region, destination.ForcePath, destination.Creds); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
if destination.Backend == config.BackendSSH {
|
||||
cfg[sshReadOnlyHostsKey] = strconv.FormatBool(f.readOnlyKnownHosts)
|
||||
}
|
||||
return cfg, nil
|
||||
}
|
||||
|
||||
func (f *backendFactory) addS3Config(cfg storage.OpenConfig, endpoint, bucket, prefix, region string, forcePath *bool, creds config.Credentials) error {
|
||||
cfg[s3EndpointKey] = endpoint
|
||||
cfg[s3BucketKey] = bucket
|
||||
cfg[s3PrefixKey] = prefix
|
||||
cfg[s3RegionKey] = region
|
||||
cfg[s3ForcePathStyleKey] = strconv.FormatBool(config.ForcePathStyle(forcePath))
|
||||
if creds.AccessKeyIDEnv != "" || creds.SecretAccessKeyEnv != "" {
|
||||
resolved, err := f.resolveCredentials(creds)
|
||||
func (f *backendFactory) addS3Config(cfg storage.OpenConfig, spec backendOpenSpec) error {
|
||||
cfg[s3EndpointKey] = spec.endpoint
|
||||
cfg[s3BucketKey] = spec.bucket
|
||||
cfg[s3PrefixKey] = spec.prefix
|
||||
cfg[s3RegionKey] = spec.region
|
||||
cfg[s3ForcePathStyleKey] = strconv.FormatBool(config.ForcePathStyle(spec.forcePath))
|
||||
if spec.credentials.AccessKeyIDEnv != "" || spec.credentials.SecretAccessKeyEnv != "" {
|
||||
resolved, err := f.resolveCredentials(spec.credentials)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -163,30 +176,38 @@ func (f *backendFactory) addS3Config(cfg storage.OpenConfig, endpoint, bucket, p
|
||||
return nil
|
||||
}
|
||||
|
||||
func sourceOpenConfig(source config.Backend) storage.OpenConfig {
|
||||
cfg := storage.OpenConfig{storagePathKey: source.Path}
|
||||
if source.Backend == config.BackendSSH {
|
||||
cfg[sshHostKey] = source.Host
|
||||
cfg[sshUserKey] = source.User
|
||||
cfg[sshPortKey] = strconv.Itoa(source.Port)
|
||||
cfg[sshKeyFileKey] = source.SSH.KeyFile
|
||||
cfg[sshKnownHostsKey] = source.SSH.KnownHosts
|
||||
cfg[sshHostKeyPolicyKey] = string(source.SSH.HostKeyPolicy)
|
||||
cfg[sshReadOnlyHostsKey] = "false"
|
||||
func backendOpenSpecFromSource(source config.Backend) backendOpenSpec {
|
||||
return backendOpenSpec{
|
||||
role: "source",
|
||||
backend: source.Backend,
|
||||
path: source.Path,
|
||||
host: source.Host,
|
||||
user: source.User,
|
||||
port: source.Port,
|
||||
ssh: source.SSH,
|
||||
endpoint: source.Endpoint,
|
||||
bucket: source.Bucket,
|
||||
prefix: source.Prefix,
|
||||
region: source.Region,
|
||||
forcePath: source.ForcePath,
|
||||
credentials: source.Creds,
|
||||
}
|
||||
return cfg
|
||||
}
|
||||
|
||||
func destinationOpenConfig(destination config.Destination) storage.OpenConfig {
|
||||
cfg := storage.OpenConfig{storagePathKey: destination.Path}
|
||||
if destination.Backend == config.BackendSSH {
|
||||
cfg[sshHostKey] = destination.Host
|
||||
cfg[sshUserKey] = destination.User
|
||||
cfg[sshPortKey] = strconv.Itoa(destination.Port)
|
||||
cfg[sshKeyFileKey] = destination.SSH.KeyFile
|
||||
cfg[sshKnownHostsKey] = destination.SSH.KnownHosts
|
||||
cfg[sshHostKeyPolicyKey] = string(destination.SSH.HostKeyPolicy)
|
||||
cfg[sshReadOnlyHostsKey] = "false"
|
||||
func backendOpenSpecFromDestination(destination config.Destination) backendOpenSpec {
|
||||
return backendOpenSpec{
|
||||
role: "destination",
|
||||
backend: destination.Backend,
|
||||
path: destination.Path,
|
||||
host: destination.Host,
|
||||
user: destination.User,
|
||||
port: destination.Port,
|
||||
ssh: destination.SSH,
|
||||
endpoint: destination.Endpoint,
|
||||
bucket: destination.Bucket,
|
||||
prefix: destination.Prefix,
|
||||
region: destination.Region,
|
||||
forcePath: destination.ForcePath,
|
||||
credentials: destination.Creds,
|
||||
}
|
||||
return cfg
|
||||
}
|
||||
|
||||
@@ -1,13 +1,16 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"fmt"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/config"
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/storage"
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/storage/fake"
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/testutil"
|
||||
)
|
||||
|
||||
func TestBackendFactoryOpensLocalSource(t *testing.T) {
|
||||
@@ -275,7 +278,8 @@ func TestBackendFactoryResolvesCredentialsThroughEnvironment(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestBackendFactoryBuildsSSHSourceOpenConfig(t *testing.T) {
|
||||
cfg := sourceOpenConfig(config.Backend{
|
||||
factory := &backendFactory{environment: config.NewEnvironment(nil, nil)}
|
||||
cfg, err := factory.openConfig(backendOpenSpecFromSource(config.Backend{
|
||||
Backend: config.BackendSSH,
|
||||
Host: "source.example.com",
|
||||
User: "reports",
|
||||
@@ -286,7 +290,10 @@ func TestBackendFactoryBuildsSSHSourceOpenConfig(t *testing.T) {
|
||||
KnownHosts: "/home/reports/.ssh/known_hosts",
|
||||
HostKeyPolicy: config.HostKeyPolicyStrict,
|
||||
},
|
||||
})
|
||||
}))
|
||||
if err != nil {
|
||||
t.Fatalf("openConfig() error = %v", err)
|
||||
}
|
||||
|
||||
assertOpenConfig(t, cfg, map[string]string{
|
||||
storagePathKey: "/reports",
|
||||
@@ -300,7 +307,8 @@ func TestBackendFactoryBuildsSSHSourceOpenConfig(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestBackendFactoryBuildsSSHDestinationOpenConfig(t *testing.T) {
|
||||
cfg := destinationOpenConfig(config.Destination{
|
||||
factory := &backendFactory{environment: config.NewEnvironment(nil, nil)}
|
||||
cfg, err := factory.openConfig(backendOpenSpecFromDestination(config.Destination{
|
||||
Backend: config.BackendSSH,
|
||||
Host: "destination.example.com",
|
||||
User: "deploy",
|
||||
@@ -309,7 +317,10 @@ func TestBackendFactoryBuildsSSHDestinationOpenConfig(t *testing.T) {
|
||||
SSH: config.SSH{
|
||||
HostKeyPolicy: config.HostKeyPolicyAcceptNew,
|
||||
},
|
||||
})
|
||||
}))
|
||||
if err != nil {
|
||||
t.Fatalf("openConfig() error = %v", err)
|
||||
}
|
||||
|
||||
assertOpenConfig(t, cfg, map[string]string{
|
||||
storagePathKey: "/srv/archive",
|
||||
@@ -320,6 +331,257 @@ func TestBackendFactoryBuildsSSHDestinationOpenConfig(t *testing.T) {
|
||||
})
|
||||
}
|
||||
|
||||
func TestBackendFactoryBuildsEquivalentSourceAndDestinationOpenConfig(t *testing.T) {
|
||||
forcePathStyle := false
|
||||
tests := []struct {
|
||||
name string
|
||||
source config.Backend
|
||||
destination config.Destination
|
||||
}{
|
||||
{
|
||||
name: "local",
|
||||
source: config.Backend{Backend: config.BackendLocal, Path: "/reports"},
|
||||
destination: config.Destination{Backend: config.BackendLocal, Path: "/reports"},
|
||||
},
|
||||
{
|
||||
name: "ssh",
|
||||
source: config.Backend{
|
||||
Backend: config.BackendSSH,
|
||||
Host: "reports.example.com",
|
||||
User: "reports",
|
||||
Port: 2222,
|
||||
Path: "/reports",
|
||||
SSH: config.SSH{
|
||||
KeyFile: "/home/reports/.ssh/id_ed25519",
|
||||
KnownHosts: "/home/reports/.ssh/known_hosts",
|
||||
HostKeyPolicy: config.HostKeyPolicyStrict,
|
||||
},
|
||||
},
|
||||
destination: config.Destination{
|
||||
Backend: config.BackendSSH,
|
||||
Host: "reports.example.com",
|
||||
User: "reports",
|
||||
Port: 2222,
|
||||
Path: "/reports",
|
||||
SSH: config.SSH{
|
||||
KeyFile: "/home/reports/.ssh/id_ed25519",
|
||||
KnownHosts: "/home/reports/.ssh/known_hosts",
|
||||
HostKeyPolicy: config.HostKeyPolicyStrict,
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "s3",
|
||||
source: config.Backend{
|
||||
Backend: config.BackendS3,
|
||||
Endpoint: "https://s3.example.com",
|
||||
Bucket: "reports",
|
||||
Prefix: "archive",
|
||||
Region: "us-west-2",
|
||||
ForcePath: &forcePathStyle,
|
||||
},
|
||||
destination: config.Destination{
|
||||
Backend: config.BackendS3,
|
||||
Endpoint: "https://s3.example.com",
|
||||
Bucket: "reports",
|
||||
Prefix: "archive",
|
||||
Region: "us-west-2",
|
||||
ForcePath: &forcePathStyle,
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "s3 explicit credentials",
|
||||
source: config.Backend{
|
||||
Backend: config.BackendS3,
|
||||
Endpoint: "https://s3.example.com",
|
||||
Bucket: "reports",
|
||||
Region: config.DefaultS3Region,
|
||||
Creds: config.Credentials{
|
||||
AccessKeyIDEnv: "ACCESS_KEY_ID",
|
||||
SecretAccessKeyEnv: "SECRET_ACCESS_KEY",
|
||||
},
|
||||
},
|
||||
destination: config.Destination{
|
||||
Backend: config.BackendS3,
|
||||
Endpoint: "https://s3.example.com",
|
||||
Bucket: "reports",
|
||||
Region: config.DefaultS3Region,
|
||||
Creds: config.Credentials{
|
||||
AccessKeyIDEnv: "ACCESS_KEY_ID",
|
||||
SecretAccessKeyEnv: "SECRET_ACCESS_KEY",
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
factory := &backendFactory{
|
||||
environment: config.NewEnvironment(map[string]string{
|
||||
"ACCESS_KEY_ID": "secret-access",
|
||||
"SECRET_ACCESS_KEY": "secret-secret",
|
||||
}, func(string) (string, bool) { return "", false }),
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
sourceConfig, err := factory.openConfig(backendOpenSpecFromSource(tt.source))
|
||||
if err != nil {
|
||||
t.Fatalf("source openConfig() error = %v", err)
|
||||
}
|
||||
destinationConfig, err := factory.openConfig(backendOpenSpecFromDestination(tt.destination))
|
||||
if err != nil {
|
||||
t.Fatalf("destination openConfig() error = %v", err)
|
||||
}
|
||||
if !openConfigEqual(sourceConfig, destinationConfig) {
|
||||
t.Fatalf("source open config = %#v, destination open config = %#v, want equivalent", sourceConfig, destinationConfig)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestBackendFactoryBuildsEquivalentDryRunSSHOpenConfig(t *testing.T) {
|
||||
factory := &backendFactory{readOnlyKnownHosts: true}
|
||||
sourceConfig, err := factory.openConfig(backendOpenSpecFromSource(config.Backend{
|
||||
Backend: config.BackendSSH,
|
||||
Host: "reports.example.com",
|
||||
Port: 22,
|
||||
Path: "/reports",
|
||||
SSH: config.SSH{HostKeyPolicy: config.HostKeyPolicyAcceptNew},
|
||||
}))
|
||||
if err != nil {
|
||||
t.Fatalf("source openConfig() error = %v", err)
|
||||
}
|
||||
destinationConfig, err := factory.openConfig(backendOpenSpecFromDestination(config.Destination{
|
||||
Backend: config.BackendSSH,
|
||||
Host: "reports.example.com",
|
||||
Port: 22,
|
||||
Path: "/reports",
|
||||
SSH: config.SSH{HostKeyPolicy: config.HostKeyPolicyAcceptNew},
|
||||
}))
|
||||
if err != nil {
|
||||
t.Fatalf("destination openConfig() error = %v", err)
|
||||
}
|
||||
if !openConfigEqual(sourceConfig, destinationConfig) {
|
||||
t.Fatalf("source open config = %#v, destination open config = %#v, want equivalent", sourceConfig, destinationConfig)
|
||||
}
|
||||
if sourceConfig[sshReadOnlyHostsKey] != "true" {
|
||||
t.Fatalf("open config %s = %q, want true", sshReadOnlyHostsKey, sourceConfig[sshReadOnlyHostsKey])
|
||||
}
|
||||
}
|
||||
|
||||
func TestConfiguredSourceValidationAndRunUseEquivalentSourceOpenConfig(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
source config.Backend
|
||||
sourceKey string
|
||||
dest config.Destination
|
||||
destKey string
|
||||
wantFields map[string]string
|
||||
}{
|
||||
{
|
||||
name: "s3",
|
||||
source: config.Backend{
|
||||
Backend: config.BackendS3,
|
||||
Endpoint: "https://s3.example.com",
|
||||
Bucket: "source-bucket",
|
||||
Prefix: "source-prefix",
|
||||
Region: config.DefaultS3Region,
|
||||
},
|
||||
sourceKey: "s3:source-bucket",
|
||||
dest: config.Destination{
|
||||
ID: "archive",
|
||||
Backend: config.BackendS3,
|
||||
Endpoint: "https://s3.example.com",
|
||||
Bucket: "destination-bucket",
|
||||
Region: config.DefaultS3Region,
|
||||
},
|
||||
destKey: "s3:destination-bucket",
|
||||
wantFields: map[string]string{
|
||||
s3EndpointKey: "https://s3.example.com",
|
||||
s3BucketKey: "source-bucket",
|
||||
s3PrefixKey: "source-prefix",
|
||||
s3RegionKey: config.DefaultS3Region,
|
||||
s3ForcePathStyleKey: "true",
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "ssh",
|
||||
source: config.Backend{
|
||||
Backend: config.BackendSSH,
|
||||
Host: "ssh.example.com",
|
||||
User: "reports",
|
||||
Port: 2222,
|
||||
Path: "/source",
|
||||
SSH: config.SSH{
|
||||
KeyFile: "/home/reports/.ssh/id_ed25519",
|
||||
KnownHosts: "/home/reports/.ssh/known_hosts",
|
||||
HostKeyPolicy: config.HostKeyPolicyStrict,
|
||||
},
|
||||
},
|
||||
sourceKey: "ssh:/source",
|
||||
dest: config.Destination{
|
||||
ID: "archive",
|
||||
Backend: config.BackendSSH,
|
||||
Host: "ssh.example.com",
|
||||
Port: 2222,
|
||||
Path: "/destination",
|
||||
SSH: config.SSH{HostKeyPolicy: config.HostKeyPolicyStrict},
|
||||
},
|
||||
destKey: "ssh:/destination",
|
||||
wantFields: map[string]string{
|
||||
storagePathKey: "/source",
|
||||
sshHostKey: "ssh.example.com",
|
||||
sshUserKey: "reports",
|
||||
sshPortKey: "2222",
|
||||
sshKeyFileKey: "/home/reports/.ssh/id_ed25519",
|
||||
sshKnownHostsKey: "/home/reports/.ssh/known_hosts",
|
||||
sshHostKeyPolicyKey: "strict",
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
sourceBackend := fake.New()
|
||||
testutil.WriteFakeSourceBundle(t, sourceBackend, "", testutil.BundleOptions{ID: "reports.source"})
|
||||
destinationBackend := fake.New()
|
||||
var validateSourceConfig storage.OpenConfig
|
||||
var runSourceConfig storage.OpenConfig
|
||||
validateProvider := recordingBackendFactoryProvider(t, map[string]storage.Backend{
|
||||
tt.sourceKey: sourceBackend,
|
||||
tt.destKey: destinationBackend,
|
||||
}, func(cfg storage.OpenConfig) {
|
||||
validateSourceConfig = cfg
|
||||
})
|
||||
runProvider := recordingBackendFactoryProvider(t, map[string]storage.Backend{
|
||||
tt.sourceKey: sourceBackend,
|
||||
tt.destKey: destinationBackend,
|
||||
}, func(cfg storage.OpenConfig) {
|
||||
runSourceConfig = cfg
|
||||
})
|
||||
cfg := config.Config{Pipelines: []config.Pipeline{{
|
||||
ID: "reports",
|
||||
Source: tt.source,
|
||||
Destinations: []config.Destination{tt.dest},
|
||||
}}}
|
||||
config.ApplyDefaults(&cfg)
|
||||
|
||||
var validateOutput bytes.Buffer
|
||||
if err := validateConfigWithBackendFactory(context.Background(), cfg, ValidateOptions{
|
||||
PipelineID: "reports",
|
||||
Stdout: &validateOutput,
|
||||
}, validateProvider); err != nil {
|
||||
t.Fatalf("validateConfigWithBackendFactory() error = %v", err)
|
||||
}
|
||||
if err := runConfigWithBackendFactory(context.Background(), cfg, RunOptions{}, runProvider); err != nil {
|
||||
t.Fatalf("runConfigWithBackendFactory() error = %v", err)
|
||||
}
|
||||
if !openConfigEqual(validateSourceConfig, runSourceConfig) {
|
||||
t.Fatalf("validate source config = %#v, run source config = %#v, want equivalent", validateSourceConfig, runSourceConfig)
|
||||
}
|
||||
assertOpenConfig(t, runSourceConfig, tt.wantFields)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func assertOpenConfig(t *testing.T, got map[string]string, want map[string]string) {
|
||||
t.Helper()
|
||||
for key, wantValue := range want {
|
||||
@@ -328,3 +590,49 @@ func assertOpenConfig(t *testing.T, got map[string]string, want map[string]strin
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func openConfigEqual(left, right storage.OpenConfig) bool {
|
||||
if len(left) != len(right) {
|
||||
return false
|
||||
}
|
||||
for key, leftValue := range left {
|
||||
if right[key] != leftValue {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func recordingBackendFactoryProvider(t *testing.T, remoteBackends map[string]storage.Backend, recordSource func(storage.OpenConfig)) backendFactoryProvider {
|
||||
t.Helper()
|
||||
return func(environment config.Environment) *backendFactory {
|
||||
registry := storage.NewRegistry()
|
||||
if err := registry.Register(config.BackendS3, func(ctx context.Context, cfg storage.OpenConfig) (storage.Backend, error) {
|
||||
if cfg[s3BucketKey] == "source-bucket" {
|
||||
recordSource(cfg)
|
||||
}
|
||||
key := "s3:" + cfg[s3BucketKey]
|
||||
backend := remoteBackends[key]
|
||||
if backend == nil {
|
||||
return nil, fmt.Errorf("missing fake backend for %s", key)
|
||||
}
|
||||
return backend, nil
|
||||
}); err != nil {
|
||||
t.Fatalf("register s3 backend: %v", err)
|
||||
}
|
||||
if err := registry.Register(config.BackendSSH, func(ctx context.Context, cfg storage.OpenConfig) (storage.Backend, error) {
|
||||
if cfg[storagePathKey] == "/source" {
|
||||
recordSource(cfg)
|
||||
}
|
||||
key := "ssh:" + cfg[storagePathKey]
|
||||
backend := remoteBackends[key]
|
||||
if backend == nil {
|
||||
return nil, fmt.Errorf("missing fake backend for %s", key)
|
||||
}
|
||||
return backend, nil
|
||||
}); err != nil {
|
||||
t.Fatalf("register ssh backend: %v", err)
|
||||
}
|
||||
return &backendFactory{registry: registry, environment: environment}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2,11 +2,8 @@ package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/bundle"
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/config"
|
||||
@@ -260,431 +257,3 @@ func closeBackend(backend storage.Backend) {
|
||||
}
|
||||
_ = closeable.Close()
|
||||
}
|
||||
|
||||
func writePlanLine(w io.Writer, backend string, plan publish.Plan, planErr error) {
|
||||
if w == nil {
|
||||
return
|
||||
}
|
||||
if planErr != nil {
|
||||
destinationID := plan.DestinationID
|
||||
if destinationID == "" {
|
||||
destinationID = "unknown"
|
||||
}
|
||||
fmt.Fprintf(w, " - bundle=%s destination=%s backend=%s%s action=error reason=%q\n", storage.DisplayPath(plan.BundlePath), destinationID, backend, pathMappingSummary(plan), planErr.Error())
|
||||
return
|
||||
}
|
||||
fmt.Fprintf(w, " - bundle=%s destination=%s backend=%s%s action=%s outputs=%s reason=%q\n", storage.DisplayPath(plan.BundlePath), plan.DestinationID, backend, pathMappingSummary(plan), plan.Action, outputSummary(plan.Outputs), plan.Reason)
|
||||
}
|
||||
|
||||
func pathMappingSummary(plan publish.Plan) string {
|
||||
if plan.PathMapping != config.PathMappingFixed {
|
||||
return ""
|
||||
}
|
||||
return fmt.Sprintf(" path_mapping=fixed target=%s", storage.DisplayPath(plan.DestinationBundlePath))
|
||||
}
|
||||
|
||||
func writeErrorLine(w io.Writer, bundlePath, destinationID, backend string, err error) {
|
||||
if w == nil {
|
||||
return
|
||||
}
|
||||
fmt.Fprintf(w, " - bundle=%s destination=%s backend=%s action=error reason=%q\n", storage.DisplayPath(bundlePath), destinationID, backend, err.Error())
|
||||
}
|
||||
|
||||
func outputSummary(outputs []publish.Output) string {
|
||||
if len(outputs) == 0 {
|
||||
return "none"
|
||||
}
|
||||
paths := make([]string, 0, len(outputs))
|
||||
for _, output := range outputs {
|
||||
paths = append(paths, output.DestinationPath)
|
||||
}
|
||||
return strings.Join(paths, ",")
|
||||
}
|
||||
|
||||
type destinationBundleSelection struct {
|
||||
SourceBundle bundle.Bundle
|
||||
DestinationBundlePath string
|
||||
}
|
||||
|
||||
func selectDestinationBundles(destination config.Destination, bundles []bundle.Bundle) []destinationBundleSelection {
|
||||
if !isFixedPathDestination(destination) {
|
||||
selections := make([]destinationBundleSelection, 0, len(bundles))
|
||||
for _, sourceBundle := range bundles {
|
||||
selections = append(selections, destinationBundleSelection{
|
||||
SourceBundle: sourceBundle,
|
||||
DestinationBundlePath: sourceBundle.RootRelativePath,
|
||||
})
|
||||
}
|
||||
return selections
|
||||
}
|
||||
if len(bundles) == 0 {
|
||||
return nil
|
||||
}
|
||||
sourceBundle := newestBundle(bundles)
|
||||
return []destinationBundleSelection{{
|
||||
SourceBundle: sourceBundle,
|
||||
DestinationBundlePath: "",
|
||||
}}
|
||||
}
|
||||
|
||||
func newestBundle(bundles []bundle.Bundle) bundle.Bundle {
|
||||
if len(bundles) == 0 {
|
||||
return bundle.Bundle{}
|
||||
}
|
||||
sorted := append([]bundle.Bundle(nil), bundles...)
|
||||
sort.Slice(sorted, func(i, j int) bool {
|
||||
if sorted[i].Manifest.Created.Equal(sorted[j].Manifest.Created) {
|
||||
return sorted[i].RootRelativePath < sorted[j].RootRelativePath
|
||||
}
|
||||
return sorted[i].Manifest.Created.After(sorted[j].Manifest.Created)
|
||||
})
|
||||
return sorted[0]
|
||||
}
|
||||
|
||||
func isFixedPathDestination(destination config.Destination) bool {
|
||||
return destination.PathMap.Mode == config.PathMappingFixed
|
||||
}
|
||||
|
||||
func fixedPathSelectionWarning(pipelineID, destinationID string, selections []destinationBundleSelection, candidateCount int) OutputWarning {
|
||||
selected := "none"
|
||||
if len(selections) > 0 {
|
||||
selected = storage.DisplayPath(selections[0].SourceBundle.RootRelativePath)
|
||||
}
|
||||
return OutputWarning{Message: fmt.Sprintf("pipeline=%s destination=%s path_mapping=fixed candidates=%d selected_bundle=%s destination_bundle=.", pipelineID, destinationID, candidateCount, selected)}
|
||||
}
|
||||
|
||||
func isDestructiveFixedPathAction(action publish.Action) bool {
|
||||
return action == publish.ActionReplaceOlder || action == publish.ActionForceReplace
|
||||
}
|
||||
|
||||
func fixedPathReplacementWarning(plan publish.Plan) OutputWarning {
|
||||
return OutputWarning{Message: fmt.Sprintf("pipeline=%s destination=%s path_mapping=fixed action=%s replaces destination root for selected_bundle=%s", plan.PipelineID, plan.DestinationID, plan.Action, storage.DisplayPath(plan.BundlePath))}
|
||||
}
|
||||
|
||||
func destinationIDs(destinations []config.Destination) []string {
|
||||
ids := make([]string, 0, len(destinations))
|
||||
for _, destination := range destinations {
|
||||
ids = append(ids, destination.ID)
|
||||
}
|
||||
return ids
|
||||
}
|
||||
|
||||
func destinationSummary(destinations []config.Destination) string {
|
||||
if len(destinations) == 0 {
|
||||
return "none"
|
||||
}
|
||||
ids := make([]string, 0, len(destinations))
|
||||
for _, destination := range destinations {
|
||||
ids = append(ids, destination.ID)
|
||||
}
|
||||
return strings.Join(ids, ",")
|
||||
}
|
||||
|
||||
func writeSecretConflictWarnings(w io.Writer, conflicts []config.SecretConflict) error {
|
||||
return writeWarnings(w, secretConflictWarnings(conflicts))
|
||||
}
|
||||
|
||||
func secretConflictWarnings(conflicts []config.SecretConflict) []OutputWarning {
|
||||
warnings := make([]OutputWarning, 0, len(conflicts))
|
||||
for _, conflict := range conflicts {
|
||||
warnings = append(warnings, OutputWarning{
|
||||
Message: fmt.Sprintf("secret %s ignored because the real environment already has that variable", conflict.Name),
|
||||
})
|
||||
}
|
||||
return warnings
|
||||
}
|
||||
|
||||
func writeSSHWarnings(w io.Writer, pipeline config.Pipeline) error {
|
||||
return writeWarnings(w, sshWarnings(pipeline))
|
||||
}
|
||||
|
||||
func sshWarnings(pipeline config.Pipeline) []OutputWarning {
|
||||
var warnings []OutputWarning
|
||||
if pipeline.Source.Backend == config.BackendSSH && pipeline.Source.SSH.HostKeyPolicy == config.HostKeyPolicyOff {
|
||||
warnings = append(warnings, OutputWarning{
|
||||
Message: fmt.Sprintf("pipeline=%s source host_key_policy=off disables SSH host key checking", pipeline.ID),
|
||||
})
|
||||
}
|
||||
for _, destination := range pipeline.Destinations {
|
||||
if destination.Backend == config.BackendSSH && destination.SSH.HostKeyPolicy == config.HostKeyPolicyOff {
|
||||
warnings = append(warnings, OutputWarning{
|
||||
Message: fmt.Sprintf("pipeline=%s destination=%s host_key_policy=off disables SSH host key checking", pipeline.ID, destination.ID),
|
||||
})
|
||||
}
|
||||
}
|
||||
return warnings
|
||||
}
|
||||
|
||||
func writeWarnings(w io.Writer, warnings []OutputWarning) error {
|
||||
if w == nil {
|
||||
return nil
|
||||
}
|
||||
for _, warning := range warnings {
|
||||
if _, err := fmt.Fprintf(w, "Warning: %s\n", warning.Message); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func shouldNotify(action publish.Action) bool {
|
||||
return action == publish.ActionPublishNew || action == publish.ActionReplaceOlder || action == publish.ActionForceReplace
|
||||
}
|
||||
|
||||
func notifyEvent(plan publish.Plan) notify.Event {
|
||||
outputs := make([]notify.Output, 0, len(plan.Outputs))
|
||||
for _, output := range plan.Outputs {
|
||||
outputs = append(outputs, notify.Output{
|
||||
Path: output.DestinationPath,
|
||||
Kind: output.Kind,
|
||||
SourcePath: output.SourcePath,
|
||||
Transform: output.Transform,
|
||||
SHA256: output.SHA256,
|
||||
Size: output.Size,
|
||||
})
|
||||
}
|
||||
return notify.Event{
|
||||
PipelineID: plan.PipelineID,
|
||||
DestinationID: plan.DestinationID,
|
||||
BundleID: plan.BundleID,
|
||||
BundlePath: plan.BundlePath,
|
||||
Action: string(plan.Action),
|
||||
Outputs: outputs,
|
||||
}
|
||||
}
|
||||
|
||||
type runResult struct {
|
||||
DryRun bool `json:"dry_run"`
|
||||
Pipelines []runPipelineResult `json:"pipelines"`
|
||||
Actions []runActionResult `json:"actions"`
|
||||
Summary runSummaryResult `json:"summary"`
|
||||
}
|
||||
|
||||
type runPipelineResult struct {
|
||||
ID string `json:"id"`
|
||||
SourceBackend string `json:"source_backend"`
|
||||
BundleCount int `json:"bundle_count"`
|
||||
Destinations []string `json:"destinations"`
|
||||
}
|
||||
|
||||
type runActionResult struct {
|
||||
PipelineID string `json:"pipeline_id,omitempty"`
|
||||
DestinationID string `json:"destination_id"`
|
||||
Backend string `json:"backend"`
|
||||
BundleID string `json:"bundle_id,omitempty"`
|
||||
BundlePath string `json:"bundle_path"`
|
||||
DestinationPath string `json:"destination_path"`
|
||||
PathMapping string `json:"path_mapping,omitempty"`
|
||||
Action string `json:"action"`
|
||||
PrimaryURL string `json:"primary_url,omitempty"`
|
||||
Reason string `json:"reason,omitempty"`
|
||||
Outputs []runOutputResult `json:"outputs"`
|
||||
}
|
||||
|
||||
type runOutputResult struct {
|
||||
Path string `json:"path"`
|
||||
Kind string `json:"kind"`
|
||||
SourcePath string `json:"source_path,omitempty"`
|
||||
Transform string `json:"transform,omitempty"`
|
||||
URL string `json:"url,omitempty"`
|
||||
SHA256 string `json:"sha256"`
|
||||
Size int64 `json:"size"`
|
||||
}
|
||||
|
||||
func runActionFromPlan(backend string, plan publish.Plan, planErr error) runActionResult {
|
||||
if planErr != nil {
|
||||
destinationID := plan.DestinationID
|
||||
if destinationID == "" {
|
||||
destinationID = "unknown"
|
||||
}
|
||||
return runActionResult{
|
||||
PipelineID: plan.PipelineID,
|
||||
DestinationID: destinationID,
|
||||
Backend: backend,
|
||||
BundleID: plan.BundleID,
|
||||
BundlePath: storage.DisplayPath(plan.BundlePath),
|
||||
DestinationPath: storage.DisplayPath(plan.DestinationBundlePath),
|
||||
PathMapping: plan.PathMapping,
|
||||
Action: "error",
|
||||
PrimaryURL: plan.PrimaryURL,
|
||||
Reason: planErr.Error(),
|
||||
Outputs: []runOutputResult{},
|
||||
}
|
||||
}
|
||||
return runActionResult{
|
||||
PipelineID: plan.PipelineID,
|
||||
DestinationID: plan.DestinationID,
|
||||
Backend: backend,
|
||||
BundleID: plan.BundleID,
|
||||
BundlePath: storage.DisplayPath(plan.BundlePath),
|
||||
DestinationPath: storage.DisplayPath(plan.DestinationBundlePath),
|
||||
PathMapping: plan.PathMapping,
|
||||
Action: string(plan.Action),
|
||||
PrimaryURL: plan.PrimaryURL,
|
||||
Reason: plan.Reason,
|
||||
Outputs: runOutputsFromPlan(plan.Outputs),
|
||||
}
|
||||
}
|
||||
|
||||
func errorAction(pipelineID, destinationID, backend, bundlePath string, err error) runActionResult {
|
||||
return runActionResult{
|
||||
PipelineID: pipelineID,
|
||||
DestinationID: destinationID,
|
||||
Backend: backend,
|
||||
BundlePath: storage.DisplayPath(bundlePath),
|
||||
DestinationPath: storage.DisplayPath(bundlePath),
|
||||
Action: "error",
|
||||
Reason: err.Error(),
|
||||
Outputs: []runOutputResult{},
|
||||
}
|
||||
}
|
||||
|
||||
func runOutputsFromPlan(outputs []publish.Output) []runOutputResult {
|
||||
results := make([]runOutputResult, 0, len(outputs))
|
||||
for _, output := range outputs {
|
||||
results = append(results, runOutputResult{
|
||||
Path: output.DestinationPath,
|
||||
Kind: output.Kind,
|
||||
SourcePath: output.SourcePath,
|
||||
Transform: output.Transform,
|
||||
URL: output.URL,
|
||||
SHA256: output.SHA256,
|
||||
Size: output.Size,
|
||||
})
|
||||
}
|
||||
return results
|
||||
}
|
||||
|
||||
type runSummary struct {
|
||||
dryRun bool
|
||||
planned int
|
||||
publishNew int
|
||||
replaceOlder int
|
||||
forceReplace int
|
||||
skipped int
|
||||
failures int
|
||||
fixedPath int
|
||||
}
|
||||
|
||||
func (s *runSummary) recordPlan(action publish.Action) {
|
||||
s.planned++
|
||||
switch action {
|
||||
case publish.ActionPublishNew:
|
||||
s.publishNew++
|
||||
case publish.ActionReplaceOlder:
|
||||
s.replaceOlder++
|
||||
case publish.ActionForceReplace:
|
||||
s.forceReplace++
|
||||
case publish.ActionSkipSame, publish.ActionSkipDestinationNewer:
|
||||
s.skipped++
|
||||
}
|
||||
}
|
||||
|
||||
func (s *runSummary) recordFailure() {
|
||||
s.failures++
|
||||
}
|
||||
|
||||
func (s *runSummary) recordFixedPath() {
|
||||
s.fixedPath++
|
||||
}
|
||||
|
||||
func (s runSummary) Line() string {
|
||||
status := "ok"
|
||||
if s.failures > 0 {
|
||||
status = "failed"
|
||||
}
|
||||
return fmt.Sprintf("Final status: %s planned=%d publish_new=%d replace_older=%d force_replace=%d skipped=%d failed=%d dry_run=%t fixed_path=%d", status, s.planned, s.publishNew, s.replaceOlder, s.forceReplace, s.skipped, s.failures, s.dryRun, s.fixedPath)
|
||||
}
|
||||
|
||||
type runSummaryResult struct {
|
||||
Status string `json:"status"`
|
||||
Planned int `json:"planned"`
|
||||
PublishNew int `json:"publish_new"`
|
||||
ReplaceOlder int `json:"replace_older"`
|
||||
ForceReplace int `json:"force_replace"`
|
||||
Skipped int `json:"skipped"`
|
||||
Failed int `json:"failed"`
|
||||
DryRun bool `json:"dry_run"`
|
||||
FixedPath int `json:"fixed_path"`
|
||||
}
|
||||
|
||||
func (s runSummary) Result() runSummaryResult {
|
||||
status := "ok"
|
||||
if s.failures > 0 {
|
||||
status = "failed"
|
||||
}
|
||||
return runSummaryResult{
|
||||
Status: status,
|
||||
Planned: s.planned,
|
||||
PublishNew: s.publishNew,
|
||||
ReplaceOlder: s.replaceOlder,
|
||||
ForceReplace: s.forceReplace,
|
||||
Skipped: s.skipped,
|
||||
Failed: s.failures,
|
||||
DryRun: s.dryRun,
|
||||
FixedPath: s.fixedPath,
|
||||
}
|
||||
}
|
||||
|
||||
type runFailure struct {
|
||||
pipelineID string
|
||||
destinationID string
|
||||
backend string
|
||||
bundlePath string
|
||||
err error
|
||||
}
|
||||
|
||||
type runFailures struct {
|
||||
items []runFailure
|
||||
}
|
||||
|
||||
func (f *runFailures) add(pipelineID, destinationID, backend, bundlePath string, err error) {
|
||||
f.items = append(f.items, runFailure{
|
||||
pipelineID: pipelineID,
|
||||
destinationID: destinationID,
|
||||
backend: backend,
|
||||
bundlePath: bundlePath,
|
||||
err: err,
|
||||
})
|
||||
}
|
||||
|
||||
func (f runFailures) Error() string {
|
||||
if len(f.items) == 0 {
|
||||
return ""
|
||||
}
|
||||
parts := make([]string, 0, len(f.items))
|
||||
for _, item := range f.items {
|
||||
parts = append(parts, fmt.Sprintf("pipeline %s destination %s backend %s bundle %s: %v", item.pipelineID, item.destinationID, item.backend, item.bundlePath, item.err))
|
||||
}
|
||||
return "run failed: " + strings.Join(parts, "; ")
|
||||
}
|
||||
|
||||
func (f runFailures) outputErrors() []OutputError {
|
||||
if len(f.items) == 0 {
|
||||
return nil
|
||||
}
|
||||
errors := make([]OutputError, 0, len(f.items))
|
||||
for _, item := range f.items {
|
||||
errors = append(errors, OutputError{
|
||||
PipelineID: item.pipelineID,
|
||||
DestinationID: item.destinationID,
|
||||
Backend: item.backend,
|
||||
BundlePath: item.bundlePath,
|
||||
Message: item.err.Error(),
|
||||
})
|
||||
}
|
||||
return errors
|
||||
}
|
||||
|
||||
func IsPartialResultError(err error) bool {
|
||||
var failures runFailures
|
||||
return errors.As(err, &failures)
|
||||
}
|
||||
|
||||
func (f runFailures) Unwrap() error {
|
||||
errs := make([]error, 0, len(f.items))
|
||||
for _, item := range f.items {
|
||||
errs = append(errs, item.err)
|
||||
}
|
||||
return errors.Join(errs...)
|
||||
}
|
||||
|
||||
70
internal/app/run_failures.go
Normal file
70
internal/app/run_failures.go
Normal file
@@ -0,0 +1,70 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
type runFailure struct {
|
||||
pipelineID string
|
||||
destinationID string
|
||||
backend string
|
||||
bundlePath string
|
||||
err error
|
||||
}
|
||||
|
||||
type runFailures struct {
|
||||
items []runFailure
|
||||
}
|
||||
|
||||
func (f *runFailures) add(pipelineID, destinationID, backend, bundlePath string, err error) {
|
||||
f.items = append(f.items, runFailure{
|
||||
pipelineID: pipelineID,
|
||||
destinationID: destinationID,
|
||||
backend: backend,
|
||||
bundlePath: bundlePath,
|
||||
err: err,
|
||||
})
|
||||
}
|
||||
|
||||
func (f runFailures) Error() string {
|
||||
if len(f.items) == 0 {
|
||||
return ""
|
||||
}
|
||||
parts := make([]string, 0, len(f.items))
|
||||
for _, item := range f.items {
|
||||
parts = append(parts, fmt.Sprintf("pipeline %s destination %s backend %s bundle %s: %v", item.pipelineID, item.destinationID, item.backend, item.bundlePath, item.err))
|
||||
}
|
||||
return "run failed: " + strings.Join(parts, "; ")
|
||||
}
|
||||
|
||||
func (f runFailures) outputErrors() []OutputError {
|
||||
if len(f.items) == 0 {
|
||||
return nil
|
||||
}
|
||||
errors := make([]OutputError, 0, len(f.items))
|
||||
for _, item := range f.items {
|
||||
errors = append(errors, OutputError{
|
||||
PipelineID: item.pipelineID,
|
||||
DestinationID: item.destinationID,
|
||||
Backend: item.backend,
|
||||
BundlePath: item.bundlePath,
|
||||
Message: item.err.Error(),
|
||||
})
|
||||
}
|
||||
return errors
|
||||
}
|
||||
|
||||
func IsPartialResultError(err error) bool {
|
||||
var failures runFailures
|
||||
return errors.As(err, &failures)
|
||||
}
|
||||
|
||||
func (f runFailures) Unwrap() error {
|
||||
errs := make([]error, 0, len(f.items))
|
||||
for _, item := range f.items {
|
||||
errs = append(errs, item.err)
|
||||
}
|
||||
return errors.Join(errs...)
|
||||
}
|
||||
33
internal/app/run_notify.go
Normal file
33
internal/app/run_notify.go
Normal file
@@ -0,0 +1,33 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/notify"
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/publish"
|
||||
)
|
||||
|
||||
func shouldNotify(action publish.Action) bool {
|
||||
return action == publish.ActionPublishNew || action == publish.ActionReplaceOlder || action == publish.ActionForceReplace
|
||||
}
|
||||
|
||||
func notifyEvent(plan publish.Plan) notify.Event {
|
||||
outputs := make([]notify.Output, 0, len(plan.Outputs))
|
||||
for _, output := range plan.Outputs {
|
||||
stateOutput := output.StateOutputFile()
|
||||
outputs = append(outputs, notify.Output{
|
||||
Path: stateOutput.Path,
|
||||
Kind: stateOutput.Kind,
|
||||
SourcePath: stateOutput.SourcePath,
|
||||
Transform: stateOutput.Transform,
|
||||
SHA256: stateOutput.SHA256,
|
||||
Size: stateOutput.Size,
|
||||
})
|
||||
}
|
||||
return notify.Event{
|
||||
PipelineID: plan.PipelineID,
|
||||
DestinationID: plan.DestinationID,
|
||||
BundleID: plan.BundleID,
|
||||
BundlePath: plan.BundlePath,
|
||||
Action: string(plan.Action),
|
||||
Outputs: outputs,
|
||||
}
|
||||
}
|
||||
154
internal/app/run_output.go
Normal file
154
internal/app/run_output.go
Normal file
@@ -0,0 +1,154 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/config"
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/publish"
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/storage"
|
||||
)
|
||||
|
||||
func writePlanLine(w io.Writer, backend string, plan publish.Plan, planErr error) {
|
||||
if w == nil {
|
||||
return
|
||||
}
|
||||
if planErr != nil {
|
||||
destinationID := plan.DestinationID
|
||||
if destinationID == "" {
|
||||
destinationID = "unknown"
|
||||
}
|
||||
fmt.Fprintf(w, " - bundle=%s destination=%s backend=%s%s action=error reason=%q\n", storage.DisplayPath(plan.BundlePath), destinationID, backend, pathMappingSummary(plan), planErr.Error())
|
||||
return
|
||||
}
|
||||
fmt.Fprintf(w, " - bundle=%s destination=%s backend=%s%s action=%s outputs=%s reason=%q\n", storage.DisplayPath(plan.BundlePath), plan.DestinationID, backend, pathMappingSummary(plan), plan.Action, outputSummary(plan.Outputs), plan.Reason)
|
||||
}
|
||||
|
||||
func pathMappingSummary(plan publish.Plan) string {
|
||||
if plan.PathMapping != config.PathMappingFixed {
|
||||
return ""
|
||||
}
|
||||
return fmt.Sprintf(" path_mapping=fixed target=%s", storage.DisplayPath(plan.DestinationBundlePath))
|
||||
}
|
||||
|
||||
func writeErrorLine(w io.Writer, bundlePath, destinationID, backend string, err error) {
|
||||
if w == nil {
|
||||
return
|
||||
}
|
||||
fmt.Fprintf(w, " - bundle=%s destination=%s backend=%s action=error reason=%q\n", storage.DisplayPath(bundlePath), destinationID, backend, err.Error())
|
||||
}
|
||||
|
||||
func outputSummary(outputs []publish.Output) string {
|
||||
if len(outputs) == 0 {
|
||||
return "none"
|
||||
}
|
||||
paths := make([]string, 0, len(outputs))
|
||||
for _, output := range outputs {
|
||||
paths = append(paths, output.DestinationPath)
|
||||
}
|
||||
return strings.Join(paths, ",")
|
||||
}
|
||||
|
||||
type runResult struct {
|
||||
DryRun bool `json:"dry_run"`
|
||||
Pipelines []runPipelineResult `json:"pipelines"`
|
||||
Actions []runActionResult `json:"actions"`
|
||||
Summary runSummaryResult `json:"summary"`
|
||||
}
|
||||
|
||||
type runPipelineResult struct {
|
||||
ID string `json:"id"`
|
||||
SourceBackend string `json:"source_backend"`
|
||||
BundleCount int `json:"bundle_count"`
|
||||
Destinations []string `json:"destinations"`
|
||||
}
|
||||
|
||||
type runActionResult struct {
|
||||
PipelineID string `json:"pipeline_id,omitempty"`
|
||||
DestinationID string `json:"destination_id"`
|
||||
Backend string `json:"backend"`
|
||||
BundleID string `json:"bundle_id,omitempty"`
|
||||
BundlePath string `json:"bundle_path"`
|
||||
DestinationPath string `json:"destination_path"`
|
||||
PathMapping string `json:"path_mapping,omitempty"`
|
||||
Action string `json:"action"`
|
||||
PrimaryURL string `json:"primary_url,omitempty"`
|
||||
Reason string `json:"reason,omitempty"`
|
||||
Outputs []runOutputResult `json:"outputs"`
|
||||
}
|
||||
|
||||
type runOutputResult struct {
|
||||
Path string `json:"path"`
|
||||
Kind string `json:"kind"`
|
||||
SourcePath string `json:"source_path,omitempty"`
|
||||
Transform string `json:"transform,omitempty"`
|
||||
URL string `json:"url,omitempty"`
|
||||
SHA256 string `json:"sha256"`
|
||||
Size int64 `json:"size"`
|
||||
}
|
||||
|
||||
func runActionFromPlan(backend string, plan publish.Plan, planErr error) runActionResult {
|
||||
if planErr != nil {
|
||||
destinationID := plan.DestinationID
|
||||
if destinationID == "" {
|
||||
destinationID = "unknown"
|
||||
}
|
||||
return runActionResult{
|
||||
PipelineID: plan.PipelineID,
|
||||
DestinationID: destinationID,
|
||||
Backend: backend,
|
||||
BundleID: plan.BundleID,
|
||||
BundlePath: storage.DisplayPath(plan.BundlePath),
|
||||
DestinationPath: storage.DisplayPath(plan.DestinationBundlePath),
|
||||
PathMapping: plan.PathMapping,
|
||||
Action: "error",
|
||||
PrimaryURL: plan.PrimaryURL,
|
||||
Reason: planErr.Error(),
|
||||
Outputs: []runOutputResult{},
|
||||
}
|
||||
}
|
||||
return runActionResult{
|
||||
PipelineID: plan.PipelineID,
|
||||
DestinationID: plan.DestinationID,
|
||||
Backend: backend,
|
||||
BundleID: plan.BundleID,
|
||||
BundlePath: storage.DisplayPath(plan.BundlePath),
|
||||
DestinationPath: storage.DisplayPath(plan.DestinationBundlePath),
|
||||
PathMapping: plan.PathMapping,
|
||||
Action: string(plan.Action),
|
||||
PrimaryURL: plan.PrimaryURL,
|
||||
Reason: plan.Reason,
|
||||
Outputs: runOutputsFromPlan(plan.Outputs),
|
||||
}
|
||||
}
|
||||
|
||||
func errorAction(pipelineID, destinationID, backend, bundlePath string, err error) runActionResult {
|
||||
return runActionResult{
|
||||
PipelineID: pipelineID,
|
||||
DestinationID: destinationID,
|
||||
Backend: backend,
|
||||
BundlePath: storage.DisplayPath(bundlePath),
|
||||
DestinationPath: storage.DisplayPath(bundlePath),
|
||||
Action: "error",
|
||||
Reason: err.Error(),
|
||||
Outputs: []runOutputResult{},
|
||||
}
|
||||
}
|
||||
|
||||
func runOutputsFromPlan(outputs []publish.Output) []runOutputResult {
|
||||
results := make([]runOutputResult, 0, len(outputs))
|
||||
for _, output := range outputs {
|
||||
stateOutput := output.StateOutputFile()
|
||||
results = append(results, runOutputResult{
|
||||
Path: stateOutput.Path,
|
||||
Kind: stateOutput.Kind,
|
||||
SourcePath: stateOutput.SourcePath,
|
||||
Transform: stateOutput.Transform,
|
||||
URL: stateOutput.URL,
|
||||
SHA256: stateOutput.SHA256,
|
||||
Size: stateOutput.Size,
|
||||
})
|
||||
}
|
||||
return results
|
||||
}
|
||||
91
internal/app/run_selection.go
Normal file
91
internal/app/run_selection.go
Normal file
@@ -0,0 +1,91 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/bundle"
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/config"
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/publish"
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/storage"
|
||||
)
|
||||
|
||||
type destinationBundleSelection struct {
|
||||
SourceBundle bundle.Bundle
|
||||
DestinationBundlePath string
|
||||
}
|
||||
|
||||
func selectDestinationBundles(destination config.Destination, bundles []bundle.Bundle) []destinationBundleSelection {
|
||||
if !isFixedPathDestination(destination) {
|
||||
selections := make([]destinationBundleSelection, 0, len(bundles))
|
||||
for _, sourceBundle := range bundles {
|
||||
selections = append(selections, destinationBundleSelection{
|
||||
SourceBundle: sourceBundle,
|
||||
DestinationBundlePath: sourceBundle.RootRelativePath,
|
||||
})
|
||||
}
|
||||
return selections
|
||||
}
|
||||
if len(bundles) == 0 {
|
||||
return nil
|
||||
}
|
||||
sourceBundle := newestBundle(bundles)
|
||||
return []destinationBundleSelection{{
|
||||
SourceBundle: sourceBundle,
|
||||
DestinationBundlePath: "",
|
||||
}}
|
||||
}
|
||||
|
||||
func newestBundle(bundles []bundle.Bundle) bundle.Bundle {
|
||||
if len(bundles) == 0 {
|
||||
return bundle.Bundle{}
|
||||
}
|
||||
sorted := append([]bundle.Bundle(nil), bundles...)
|
||||
sort.Slice(sorted, func(i, j int) bool {
|
||||
if sorted[i].Manifest.Created.Equal(sorted[j].Manifest.Created) {
|
||||
return sorted[i].RootRelativePath < sorted[j].RootRelativePath
|
||||
}
|
||||
return sorted[i].Manifest.Created.After(sorted[j].Manifest.Created)
|
||||
})
|
||||
return sorted[0]
|
||||
}
|
||||
|
||||
func isFixedPathDestination(destination config.Destination) bool {
|
||||
return destination.PathMap.Mode == config.PathMappingFixed
|
||||
}
|
||||
|
||||
func fixedPathSelectionWarning(pipelineID, destinationID string, selections []destinationBundleSelection, candidateCount int) OutputWarning {
|
||||
selected := "none"
|
||||
if len(selections) > 0 {
|
||||
selected = storage.DisplayPath(selections[0].SourceBundle.RootRelativePath)
|
||||
}
|
||||
return OutputWarning{Message: fmt.Sprintf("pipeline=%s destination=%s path_mapping=fixed candidates=%d selected_bundle=%s destination_bundle=.", pipelineID, destinationID, candidateCount, selected)}
|
||||
}
|
||||
|
||||
func isDestructiveFixedPathAction(action publish.Action) bool {
|
||||
return action == publish.ActionReplaceOlder || action == publish.ActionForceReplace
|
||||
}
|
||||
|
||||
func fixedPathReplacementWarning(plan publish.Plan) OutputWarning {
|
||||
return OutputWarning{Message: fmt.Sprintf("pipeline=%s destination=%s path_mapping=fixed action=%s replaces destination root for selected_bundle=%s", plan.PipelineID, plan.DestinationID, plan.Action, storage.DisplayPath(plan.BundlePath))}
|
||||
}
|
||||
|
||||
func destinationIDs(destinations []config.Destination) []string {
|
||||
ids := make([]string, 0, len(destinations))
|
||||
for _, destination := range destinations {
|
||||
ids = append(ids, destination.ID)
|
||||
}
|
||||
return ids
|
||||
}
|
||||
|
||||
func destinationSummary(destinations []config.Destination) string {
|
||||
if len(destinations) == 0 {
|
||||
return "none"
|
||||
}
|
||||
ids := make([]string, 0, len(destinations))
|
||||
for _, destination := range destinations {
|
||||
ids = append(ids, destination.ID)
|
||||
}
|
||||
return strings.Join(ids, ",")
|
||||
}
|
||||
78
internal/app/run_summary.go
Normal file
78
internal/app/run_summary.go
Normal file
@@ -0,0 +1,78 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/publish"
|
||||
)
|
||||
|
||||
type runSummary struct {
|
||||
dryRun bool
|
||||
planned int
|
||||
publishNew int
|
||||
replaceOlder int
|
||||
forceReplace int
|
||||
skipped int
|
||||
failures int
|
||||
fixedPath int
|
||||
}
|
||||
|
||||
func (s *runSummary) recordPlan(action publish.Action) {
|
||||
s.planned++
|
||||
switch action {
|
||||
case publish.ActionPublishNew:
|
||||
s.publishNew++
|
||||
case publish.ActionReplaceOlder:
|
||||
s.replaceOlder++
|
||||
case publish.ActionForceReplace:
|
||||
s.forceReplace++
|
||||
case publish.ActionSkipSame, publish.ActionSkipDestinationNewer:
|
||||
s.skipped++
|
||||
}
|
||||
}
|
||||
|
||||
func (s *runSummary) recordFailure() {
|
||||
s.failures++
|
||||
}
|
||||
|
||||
func (s *runSummary) recordFixedPath() {
|
||||
s.fixedPath++
|
||||
}
|
||||
|
||||
func (s runSummary) Line() string {
|
||||
status := "ok"
|
||||
if s.failures > 0 {
|
||||
status = "failed"
|
||||
}
|
||||
return fmt.Sprintf("Final status: %s planned=%d publish_new=%d replace_older=%d force_replace=%d skipped=%d failed=%d dry_run=%t fixed_path=%d", status, s.planned, s.publishNew, s.replaceOlder, s.forceReplace, s.skipped, s.failures, s.dryRun, s.fixedPath)
|
||||
}
|
||||
|
||||
type runSummaryResult struct {
|
||||
Status string `json:"status"`
|
||||
Planned int `json:"planned"`
|
||||
PublishNew int `json:"publish_new"`
|
||||
ReplaceOlder int `json:"replace_older"`
|
||||
ForceReplace int `json:"force_replace"`
|
||||
Skipped int `json:"skipped"`
|
||||
Failed int `json:"failed"`
|
||||
DryRun bool `json:"dry_run"`
|
||||
FixedPath int `json:"fixed_path"`
|
||||
}
|
||||
|
||||
func (s runSummary) Result() runSummaryResult {
|
||||
status := "ok"
|
||||
if s.failures > 0 {
|
||||
status = "failed"
|
||||
}
|
||||
return runSummaryResult{
|
||||
Status: status,
|
||||
Planned: s.planned,
|
||||
PublishNew: s.publishNew,
|
||||
ReplaceOlder: s.replaceOlder,
|
||||
ForceReplace: s.forceReplace,
|
||||
Skipped: s.skipped,
|
||||
Failed: s.failures,
|
||||
DryRun: s.dryRun,
|
||||
FixedPath: s.fixedPath,
|
||||
}
|
||||
}
|
||||
@@ -159,9 +159,9 @@ pipelines:
|
||||
}
|
||||
}
|
||||
|
||||
func TestWriteSSHWarningsReportsInsecureHostKeyPolicy(t *testing.T) {
|
||||
func TestSSHWarningsReportInsecureHostKeyPolicy(t *testing.T) {
|
||||
var stdout bytes.Buffer
|
||||
err := writeSSHWarnings(&stdout, config.Pipeline{
|
||||
err := writeWarnings(&stdout, sshWarnings(config.Pipeline{
|
||||
ID: "reports",
|
||||
Source: config.Backend{
|
||||
Backend: config.BackendSSH,
|
||||
@@ -172,9 +172,9 @@ func TestWriteSSHWarningsReportsInsecureHostKeyPolicy(t *testing.T) {
|
||||
Backend: config.BackendSSH,
|
||||
SSH: config.SSH{HostKeyPolicy: config.HostKeyPolicyOff},
|
||||
}},
|
||||
})
|
||||
}))
|
||||
if err != nil {
|
||||
t.Fatalf("writeSSHWarnings() error = %v", err)
|
||||
t.Fatalf("writeWarnings() error = %v", err)
|
||||
}
|
||||
output := stdout.String()
|
||||
for _, want := range []string{
|
||||
@@ -200,8 +200,8 @@ func TestRunPublishesNewLocalBundle(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
assertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nSunny.\n")
|
||||
assertFile(t, filepath.Join(destinationRoot, "summary.txt"), "Summary\n")
|
||||
testutil.AssertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nSunny.\n")
|
||||
testutil.AssertFile(t, filepath.Join(destinationRoot, "summary.txt"), "Summary\n")
|
||||
if _, err := os.Stat(filepath.Join(destinationRoot, "manifest.json")); !os.IsNotExist(err) {
|
||||
t.Fatalf("destination manifest stat error = %v, want not exist", err)
|
||||
}
|
||||
@@ -225,11 +225,11 @@ func TestRunExplicitPreserveRelativePathMappingMatchesDefault(t *testing.T) {
|
||||
destinationRoot := t.TempDir()
|
||||
writeSourceBundle(t, sourceRoot, "daily/report", testBundleOptions{})
|
||||
|
||||
err := Run(context.Background(), RunOptions{ConfigPath: writeLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingPreserveRelative)})
|
||||
err := Run(context.Background(), RunOptions{ConfigPath: testutil.WriteLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingPreserveRelative)})
|
||||
if err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
assertFile(t, filepath.Join(destinationRoot, "daily", "report", "report.md"), "# Report\nSunny.\n")
|
||||
testutil.AssertFile(t, filepath.Join(destinationRoot, "daily", "report", "report.md"), "# Report\nSunny.\n")
|
||||
if _, err := os.Stat(filepath.Join(destinationRoot, "report.md")); !os.IsNotExist(err) {
|
||||
t.Fatalf("root report.md stat error = %v, want not exist", err)
|
||||
}
|
||||
@@ -241,7 +241,7 @@ func TestRunRecordsLinksForNestedBundlePath(t *testing.T) {
|
||||
writeSourceBundle(t, sourceRoot, "daily/brentwood", testBundleOptions{})
|
||||
|
||||
err := Run(context.Background(), RunOptions{
|
||||
ConfigPath: writeLocalConfigWithLinks(t, sourceRoot, destinationRoot, config.PathMappingPreserveRelative, "https://reports.example.com/archive", config.LinkPrimaryAuto, true, false, ""),
|
||||
ConfigPath: testutil.WriteLocalConfigWithLinks(t, sourceRoot, destinationRoot, config.PathMappingPreserveRelative, "https://reports.example.com/archive", config.LinkPrimaryAuto, true, false, ""),
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
@@ -266,7 +266,7 @@ func TestRunRecordsLinksForFixedIndexDestination(t *testing.T) {
|
||||
writeSourceBundle(t, sourceRoot, "newer", testBundleOptions{ID: "reports.newer", Created: testutil.DefaultCreated.Add(time.Hour)})
|
||||
|
||||
err := Run(context.Background(), RunOptions{
|
||||
ConfigPath: writeLocalConfigWithLinks(t, sourceRoot, destinationRoot, config.PathMappingFixed, "https://reports.example.com/latest", config.LinkPrimaryAuto, false, true, config.TransformModeIndex),
|
||||
ConfigPath: testutil.WriteLocalConfigWithLinks(t, sourceRoot, destinationRoot, config.PathMappingFixed, "https://reports.example.com/latest", config.LinkPrimaryAuto, false, true, config.TransformModeIndex),
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
@@ -303,11 +303,11 @@ func TestRunFixedPathPublishesNewestBundleAtDestinationRoot(t *testing.T) {
|
||||
},
|
||||
})
|
||||
|
||||
err := Run(context.Background(), RunOptions{ConfigPath: writeLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed)})
|
||||
err := Run(context.Background(), RunOptions{ConfigPath: testutil.WriteLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed)})
|
||||
if err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
assertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nNew.\n")
|
||||
testutil.AssertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nNew.\n")
|
||||
if _, err := os.Stat(filepath.Join(destinationRoot, "new", "report.md")); !os.IsNotExist(err) {
|
||||
t.Fatalf("nested new report stat error = %v, want not exist", err)
|
||||
}
|
||||
@@ -337,7 +337,7 @@ func TestRunFixedPathTieBreaksByBundlePath(t *testing.T) {
|
||||
},
|
||||
})
|
||||
|
||||
err := Run(context.Background(), RunOptions{ConfigPath: writeLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed)})
|
||||
err := Run(context.Background(), RunOptions{ConfigPath: testutil.WriteLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed)})
|
||||
if err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
@@ -355,7 +355,7 @@ func TestRunFixedPathDryRunReportsSelection(t *testing.T) {
|
||||
var stdout bytes.Buffer
|
||||
|
||||
err := Run(context.Background(), RunOptions{
|
||||
ConfigPath: writeLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed),
|
||||
ConfigPath: testutil.WriteLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed),
|
||||
DryRun: true,
|
||||
Stdout: &stdout,
|
||||
})
|
||||
@@ -391,7 +391,7 @@ func TestRunFixedPathDryRunWarnsForReplacement(t *testing.T) {
|
||||
{Path: "summary.txt", Data: "Old summary\n"},
|
||||
},
|
||||
})
|
||||
configPath := writeLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed)
|
||||
configPath := testutil.WriteLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed)
|
||||
if err := Run(context.Background(), RunOptions{ConfigPath: configPath}); err != nil {
|
||||
t.Fatalf("first Run() error = %v", err)
|
||||
}
|
||||
@@ -422,7 +422,7 @@ func TestRunFixedPathDryRunWarnsForReplacement(t *testing.T) {
|
||||
t.Fatalf("stdout = %q, want substring %q", output, want)
|
||||
}
|
||||
}
|
||||
assertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nOld.\n")
|
||||
testutil.AssertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nOld.\n")
|
||||
}
|
||||
|
||||
func TestRunFixedPathReplacesOlderManagedState(t *testing.T) {
|
||||
@@ -436,11 +436,11 @@ func TestRunFixedPathReplacesOlderManagedState(t *testing.T) {
|
||||
{Path: "summary.txt", Data: "Old summary\n"},
|
||||
},
|
||||
})
|
||||
configPath := writeLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed)
|
||||
configPath := testutil.WriteLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed)
|
||||
if err := Run(context.Background(), RunOptions{ConfigPath: configPath}); err != nil {
|
||||
t.Fatalf("first Run() error = %v", err)
|
||||
}
|
||||
assertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nOld.\n")
|
||||
testutil.AssertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nOld.\n")
|
||||
|
||||
writeSourceBundle(t, sourceRoot, "new", testBundleOptions{
|
||||
ID: "reports.new",
|
||||
@@ -454,7 +454,7 @@ func TestRunFixedPathReplacesOlderManagedState(t *testing.T) {
|
||||
if err := Run(context.Background(), RunOptions{ConfigPath: configPath}); err != nil {
|
||||
t.Fatalf("second Run() error = %v", err)
|
||||
}
|
||||
assertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nNew.\n")
|
||||
testutil.AssertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nNew.\n")
|
||||
destinationState := readStateFile(t, filepath.Join(destinationRoot, storage.StateFileName))
|
||||
if destinationState.Source.Manifest.ID != "reports.new" {
|
||||
t.Fatalf("state source id = %q, want reports.new", destinationState.Source.Manifest.ID)
|
||||
@@ -479,7 +479,7 @@ func TestRunFixedPathSkipsWhenDestinationStateIsNewer(t *testing.T) {
|
||||
|
||||
var stdout bytes.Buffer
|
||||
err := Run(context.Background(), RunOptions{
|
||||
ConfigPath: writeLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed),
|
||||
ConfigPath: testutil.WriteLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed),
|
||||
Stdout: &stdout,
|
||||
})
|
||||
if err != nil {
|
||||
@@ -488,7 +488,7 @@ func TestRunFixedPathSkipsWhenDestinationStateIsNewer(t *testing.T) {
|
||||
if !strings.Contains(stdout.String(), "action=skip_destination_newer") {
|
||||
t.Fatalf("stdout = %q, want skip_destination_newer", stdout.String())
|
||||
}
|
||||
assertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nExisting.\n")
|
||||
testutil.AssertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nExisting.\n")
|
||||
}
|
||||
|
||||
func TestRunFixedPathFailsUnmanagedWithoutForce(t *testing.T) {
|
||||
@@ -499,7 +499,7 @@ func TestRunFixedPathFailsUnmanagedWithoutForce(t *testing.T) {
|
||||
t.Fatalf("write unmanaged file: %v", err)
|
||||
}
|
||||
|
||||
err := Run(context.Background(), RunOptions{ConfigPath: writeLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed)})
|
||||
err := Run(context.Background(), RunOptions{ConfigPath: testutil.WriteLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed)})
|
||||
if err == nil || !strings.Contains(err.Error(), "fail_unmanaged") {
|
||||
t.Fatalf("Run() error = %v, want unmanaged failure", err)
|
||||
}
|
||||
@@ -521,14 +521,14 @@ func TestRunFixedPathForceReplacementStaysWithinDestinationRoot(t *testing.T) {
|
||||
writeSourceBundle(t, sourceRoot, "bundle", testBundleOptions{})
|
||||
|
||||
err := Run(context.Background(), RunOptions{
|
||||
ConfigPath: writeLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed),
|
||||
ConfigPath: testutil.WriteLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed),
|
||||
Force: true,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
assertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nSunny.\n")
|
||||
assertFile(t, filepath.Join(parent, "keep.txt"), "keep")
|
||||
testutil.AssertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nSunny.\n")
|
||||
testutil.AssertFile(t, filepath.Join(parent, "keep.txt"), "keep")
|
||||
if _, err := os.Stat(filepath.Join(destinationRoot, "unmanaged.txt")); !os.IsNotExist(err) {
|
||||
t.Fatalf("unmanaged stat error = %v, want removed", err)
|
||||
}
|
||||
@@ -583,12 +583,12 @@ func TestRunFixedPathRemoteBackendsUseBackendRoots(t *testing.T) {
|
||||
if err := runConfigWithBackendFactory(context.Background(), cfg, RunOptions{}, provider); err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
assertFakeFile(t, s3Destination, "report.md", "# Report\nNew.\n")
|
||||
assertFakeFile(t, s3Destination, "summary.txt", "New summary\n")
|
||||
assertFakeMissing(t, s3Destination, "new/report.md")
|
||||
assertFakeFile(t, sshDestination, "report.md", "# Report\nNew.\n")
|
||||
assertFakeFile(t, sshDestination, "summary.txt", "New summary\n")
|
||||
assertFakeMissing(t, sshDestination, "new/report.md")
|
||||
testutil.AssertFakeFile(t, s3Destination, "report.md", "# Report\nNew.\n")
|
||||
testutil.AssertFakeFile(t, s3Destination, "summary.txt", "New summary\n")
|
||||
testutil.AssertFakeMissing(t, s3Destination, "new/report.md")
|
||||
testutil.AssertFakeFile(t, sshDestination, "report.md", "# Report\nNew.\n")
|
||||
testutil.AssertFakeFile(t, sshDestination, "summary.txt", "New summary\n")
|
||||
testutil.AssertFakeMissing(t, sshDestination, "new/report.md")
|
||||
}
|
||||
|
||||
func TestRunNotifiesAfterPublication(t *testing.T) {
|
||||
@@ -622,6 +622,32 @@ func TestRunNotifiesAfterPublication(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunNotifiesGeneratedOutputMetadata(t *testing.T) {
|
||||
sourceRoot := t.TempDir()
|
||||
destinationRoot := t.TempDir()
|
||||
writeSourceBundle(t, sourceRoot, "", testBundleOptions{})
|
||||
notifier := &recordingNotifier{}
|
||||
|
||||
err := Run(context.Background(), RunOptions{
|
||||
ConfigPath: testutil.WriteLocalConfigWithPublishPolicy(t, sourceRoot, destinationRoot, false, true),
|
||||
Notifier: notifier,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
if got, want := len(notifier.events), 1; got != want {
|
||||
t.Fatalf("notification count = %d, want %d", got, want)
|
||||
}
|
||||
outputs := notifier.events[0].Outputs
|
||||
if got, want := len(outputs), 1; got != want {
|
||||
t.Fatalf("notification output count = %d, want %d", got, want)
|
||||
}
|
||||
output := outputs[0]
|
||||
if output.Path != "report.html" || output.Kind != state.OutputKindGenerated || output.SourcePath != "report.md" || output.Transform != "markdown_to_html" || output.SHA256 == "" || output.Size <= 0 {
|
||||
t.Fatalf("notification output = %#v", output)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunNotifiesAfterReplacement(t *testing.T) {
|
||||
sourceRoot := t.TempDir()
|
||||
destinationRoot := t.TempDir()
|
||||
@@ -649,6 +675,49 @@ func TestRunNotifiesAfterReplacement(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunJSONIncludesGeneratedOutputMetadata(t *testing.T) {
|
||||
sourceRoot := t.TempDir()
|
||||
destinationRoot := t.TempDir()
|
||||
writeSourceBundle(t, sourceRoot, "", testBundleOptions{})
|
||||
|
||||
var stdout bytes.Buffer
|
||||
err := Run(context.Background(), RunOptions{
|
||||
ConfigPath: testutil.WriteLocalConfigWithLinks(t, sourceRoot, destinationRoot, config.PathMappingFixed, "https://reports.example.com/latest", config.LinkPrimaryAuto, false, true, config.TransformModeIndex),
|
||||
DryRun: true,
|
||||
Stdout: &stdout,
|
||||
OutputFormat: OutputFormatJSON,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
result := decodeAppResult(t, stdout.String())
|
||||
actions, ok := result["actions"].([]any)
|
||||
if !ok || len(actions) != 1 {
|
||||
t.Fatalf("actions = %#v, want one action", result["actions"])
|
||||
}
|
||||
action, ok := actions[0].(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("action = %#v, want object", actions[0])
|
||||
}
|
||||
if action["primary_url"] != "https://reports.example.com/latest/" {
|
||||
t.Fatalf("action primary_url = %#v", action["primary_url"])
|
||||
}
|
||||
outputs, ok := action["outputs"].([]any)
|
||||
if !ok || len(outputs) != 1 {
|
||||
t.Fatalf("outputs = %#v, want one output", action["outputs"])
|
||||
}
|
||||
output, ok := outputs[0].(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("output = %#v, want object", outputs[0])
|
||||
}
|
||||
if output["path"] != "index.html" || output["kind"] != state.OutputKindGenerated || output["source_path"] != "report.md" || output["transform"] != "markdown_to_html" || output["url"] != "https://reports.example.com/latest/" {
|
||||
t.Fatalf("output = %#v, want generated index metadata", output)
|
||||
}
|
||||
if output["sha256"] == "" || output["size"] == nil {
|
||||
t.Fatalf("output = %#v, want digest and size", output)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunDoesNotNotifyForSkippedDestination(t *testing.T) {
|
||||
sourceRoot := t.TempDir()
|
||||
destinationRoot := t.TempDir()
|
||||
@@ -720,7 +789,7 @@ func TestRunContinuesAfterDestinationFailure(t *testing.T) {
|
||||
t.Fatalf("stdout = %q, want substring %q", output, want)
|
||||
}
|
||||
}
|
||||
assertFile(t, filepath.Join(secondDestination, "report.md"), "# Report\nSunny.\n")
|
||||
testutil.AssertFile(t, filepath.Join(secondDestination, "report.md"), "# Report\nSunny.\n")
|
||||
}
|
||||
|
||||
func TestRunPublishesHTMLOnly(t *testing.T) {
|
||||
@@ -728,11 +797,11 @@ func TestRunPublishesHTMLOnly(t *testing.T) {
|
||||
destinationRoot := t.TempDir()
|
||||
writeSourceBundle(t, sourceRoot, "", testBundleOptions{})
|
||||
|
||||
err := Run(context.Background(), RunOptions{ConfigPath: writeLocalConfigWithPolicy(t, sourceRoot, destinationRoot, false, true)})
|
||||
err := Run(context.Background(), RunOptions{ConfigPath: testutil.WriteLocalConfigWithPublishPolicy(t, sourceRoot, destinationRoot, false, true)})
|
||||
if err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
assertFileContains(t, filepath.Join(destinationRoot, "report.html"), "<h1>Report</h1>")
|
||||
testutil.AssertFileContains(t, filepath.Join(destinationRoot, "report.html"), "<h1>Report</h1>")
|
||||
if _, err := os.Stat(filepath.Join(destinationRoot, "report.md")); !os.IsNotExist(err) {
|
||||
t.Fatalf("report.md stat error = %v, want not exist", err)
|
||||
}
|
||||
@@ -753,11 +822,11 @@ func TestRunPublishesHTMLIndexWithExplicitInput(t *testing.T) {
|
||||
ExtraFiles: []testFile{{Path: "notes.md", Data: "# Notes\nHidden.\n"}},
|
||||
})
|
||||
|
||||
err := Run(context.Background(), RunOptions{ConfigPath: writeLocalConfigWithMarkdownTransform(t, sourceRoot, destinationRoot, false, true, config.TransformModeIndex, "report.md")})
|
||||
err := Run(context.Background(), RunOptions{ConfigPath: testutil.WriteLocalConfigWithMarkdownTransform(t, sourceRoot, destinationRoot, false, true, config.TransformModeIndex, "report.md")})
|
||||
if err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
assertFileContains(t, filepath.Join(destinationRoot, "index.html"), "<h1>Report</h1>")
|
||||
testutil.AssertFileContains(t, filepath.Join(destinationRoot, "index.html"), "<h1>Report</h1>")
|
||||
if _, err := os.Stat(filepath.Join(destinationRoot, "report.html")); !os.IsNotExist(err) {
|
||||
t.Fatalf("report.html stat error = %v, want not exist", err)
|
||||
}
|
||||
@@ -776,11 +845,11 @@ func TestRunPublishesHTMLIndexWithSingleMarkdownFallback(t *testing.T) {
|
||||
destinationRoot := t.TempDir()
|
||||
writeSourceBundle(t, sourceRoot, "", testBundleOptions{})
|
||||
|
||||
err := Run(context.Background(), RunOptions{ConfigPath: writeLocalConfigWithMarkdownTransform(t, sourceRoot, destinationRoot, false, true, config.TransformModeIndex, "")})
|
||||
err := Run(context.Background(), RunOptions{ConfigPath: testutil.WriteLocalConfigWithMarkdownTransform(t, sourceRoot, destinationRoot, false, true, config.TransformModeIndex, "")})
|
||||
if err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
assertFileContains(t, filepath.Join(destinationRoot, "index.html"), "<h1>Report</h1>")
|
||||
testutil.AssertFileContains(t, filepath.Join(destinationRoot, "index.html"), "<h1>Report</h1>")
|
||||
}
|
||||
|
||||
func TestRunFailsIndexModeWithAmbiguousMarkdownInput(t *testing.T) {
|
||||
@@ -790,7 +859,7 @@ func TestRunFailsIndexModeWithAmbiguousMarkdownInput(t *testing.T) {
|
||||
ExtraFiles: []testFile{{Path: "notes.md", Data: "# Notes\n"}},
|
||||
})
|
||||
|
||||
err := Run(context.Background(), RunOptions{ConfigPath: writeLocalConfigWithMarkdownTransform(t, sourceRoot, destinationRoot, false, true, config.TransformModeIndex, "")})
|
||||
err := Run(context.Background(), RunOptions{ConfigPath: testutil.WriteLocalConfigWithMarkdownTransform(t, sourceRoot, destinationRoot, false, true, config.TransformModeIndex, "")})
|
||||
if err == nil || !strings.Contains(err.Error(), "multiple markdown source files") {
|
||||
t.Fatalf("Run() error = %v, want ambiguous input error", err)
|
||||
}
|
||||
@@ -804,13 +873,13 @@ func TestRunPublishesSourceAndHTML(t *testing.T) {
|
||||
destinationRoot := t.TempDir()
|
||||
writeSourceBundle(t, sourceRoot, "", testBundleOptions{})
|
||||
|
||||
err := Run(context.Background(), RunOptions{ConfigPath: writeLocalConfigWithPolicy(t, sourceRoot, destinationRoot, true, true)})
|
||||
err := Run(context.Background(), RunOptions{ConfigPath: testutil.WriteLocalConfigWithPublishPolicy(t, sourceRoot, destinationRoot, true, true)})
|
||||
if err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
assertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nSunny.\n")
|
||||
assertFileContains(t, filepath.Join(destinationRoot, "report.html"), "<p>Sunny.</p>")
|
||||
assertFile(t, filepath.Join(destinationRoot, "summary.txt"), "Summary\n")
|
||||
testutil.AssertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nSunny.\n")
|
||||
testutil.AssertFileContains(t, filepath.Join(destinationRoot, "report.html"), "<p>Sunny.</p>")
|
||||
testutil.AssertFile(t, filepath.Join(destinationRoot, "summary.txt"), "Summary\n")
|
||||
destinationState := readStateFile(t, filepath.Join(destinationRoot, storage.StateFileName))
|
||||
if got, want := len(destinationState.Outputs), 3; got != want {
|
||||
t.Fatalf("state output count = %d, want %d", got, want)
|
||||
@@ -827,7 +896,7 @@ func TestRunDoesNotMutateSourceBundle(t *testing.T) {
|
||||
t.Fatalf("read source before: %v", err)
|
||||
}
|
||||
|
||||
err = Run(context.Background(), RunOptions{ConfigPath: writeLocalConfigWithPolicy(t, sourceRoot, destinationRoot, true, true)})
|
||||
err = Run(context.Background(), RunOptions{ConfigPath: testutil.WriteLocalConfigWithPublishPolicy(t, sourceRoot, destinationRoot, true, true)})
|
||||
if err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
@@ -845,7 +914,7 @@ func TestRunFailsOnOutputPathCollision(t *testing.T) {
|
||||
destinationRoot := t.TempDir()
|
||||
writeSourceBundle(t, sourceRoot, "", testBundleOptions{ExtraFiles: []testFile{{Path: "report.html", Data: "<p>source html</p>\n"}}})
|
||||
|
||||
err := Run(context.Background(), RunOptions{ConfigPath: writeLocalConfigWithPolicy(t, sourceRoot, destinationRoot, true, true)})
|
||||
err := Run(context.Background(), RunOptions{ConfigPath: testutil.WriteLocalConfigWithPublishPolicy(t, sourceRoot, destinationRoot, true, true)})
|
||||
if err == nil || !strings.Contains(err.Error(), "destination output path collision") {
|
||||
t.Fatalf("Run() error = %v, want collision", err)
|
||||
}
|
||||
@@ -861,7 +930,7 @@ func TestRunFailsOnIndexOutputPathCollision(t *testing.T) {
|
||||
ExtraFiles: []testFile{{Path: "index.html", Data: "<p>source index</p>\n"}},
|
||||
})
|
||||
|
||||
err := Run(context.Background(), RunOptions{ConfigPath: writeLocalConfigWithMarkdownTransform(t, sourceRoot, destinationRoot, true, true, config.TransformModeIndex, "report.md")})
|
||||
err := Run(context.Background(), RunOptions{ConfigPath: testutil.WriteLocalConfigWithMarkdownTransform(t, sourceRoot, destinationRoot, true, true, config.TransformModeIndex, "report.md")})
|
||||
if err == nil || !strings.Contains(err.Error(), "destination output path collision") {
|
||||
t.Fatalf("Run() error = %v, want collision", err)
|
||||
}
|
||||
@@ -877,7 +946,7 @@ func TestRunDryRunReportsGeneratedOutputs(t *testing.T) {
|
||||
|
||||
var stdout bytes.Buffer
|
||||
err := Run(context.Background(), RunOptions{
|
||||
ConfigPath: writeLocalConfigWithPolicy(t, sourceRoot, destinationRoot, false, true),
|
||||
ConfigPath: testutil.WriteLocalConfigWithPublishPolicy(t, sourceRoot, destinationRoot, false, true),
|
||||
DryRun: true,
|
||||
Stdout: &stdout,
|
||||
})
|
||||
@@ -896,7 +965,7 @@ func TestRunDryRunReportsIndexOutputWithoutWriting(t *testing.T) {
|
||||
|
||||
var stdout bytes.Buffer
|
||||
err := Run(context.Background(), RunOptions{
|
||||
ConfigPath: writeLocalConfigWithMarkdownTransform(t, sourceRoot, destinationRoot, false, true, config.TransformModeIndex, ""),
|
||||
ConfigPath: testutil.WriteLocalConfigWithMarkdownTransform(t, sourceRoot, destinationRoot, false, true, config.TransformModeIndex, ""),
|
||||
DryRun: true,
|
||||
Stdout: &stdout,
|
||||
})
|
||||
@@ -916,11 +985,11 @@ func TestRunSourceOnlyDoesNotWriteIndexOutput(t *testing.T) {
|
||||
destinationRoot := t.TempDir()
|
||||
writeSourceBundle(t, sourceRoot, "", testBundleOptions{})
|
||||
|
||||
err := Run(context.Background(), RunOptions{ConfigPath: writeLocalConfigWithMarkdownTransform(t, sourceRoot, destinationRoot, true, false, config.TransformModeIndex, "")})
|
||||
err := Run(context.Background(), RunOptions{ConfigPath: testutil.WriteLocalConfigWithMarkdownTransform(t, sourceRoot, destinationRoot, true, false, config.TransformModeIndex, "")})
|
||||
if err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
assertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nSunny.\n")
|
||||
testutil.AssertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nSunny.\n")
|
||||
if _, err := os.Stat(filepath.Join(destinationRoot, "index.html")); !os.IsNotExist(err) {
|
||||
t.Fatalf("index.html stat error = %v, want not exist", err)
|
||||
}
|
||||
@@ -936,11 +1005,11 @@ func TestRunReplacesHTMLIndexOutput(t *testing.T) {
|
||||
{Path: "summary.txt", Data: "Summary\n"},
|
||||
},
|
||||
})
|
||||
configPath := writeLocalConfigWithMarkdownTransform(t, sourceRoot, destinationRoot, false, true, config.TransformModeIndex, "")
|
||||
configPath := testutil.WriteLocalConfigWithMarkdownTransform(t, sourceRoot, destinationRoot, false, true, config.TransformModeIndex, "")
|
||||
if err := Run(context.Background(), RunOptions{ConfigPath: configPath}); err != nil {
|
||||
t.Fatalf("first Run() error = %v", err)
|
||||
}
|
||||
assertFileContains(t, filepath.Join(destinationRoot, "index.html"), "<p>Old.</p>")
|
||||
testutil.AssertFileContains(t, filepath.Join(destinationRoot, "index.html"), "<p>Old.</p>")
|
||||
|
||||
writeSourceBundle(t, sourceRoot, "", testBundleOptions{
|
||||
Created: testutil.DefaultCreated.Add(time.Hour),
|
||||
@@ -953,7 +1022,7 @@ func TestRunReplacesHTMLIndexOutput(t *testing.T) {
|
||||
if err := Run(context.Background(), RunOptions{ConfigPath: configPath}); err != nil {
|
||||
t.Fatalf("second Run() error = %v", err)
|
||||
}
|
||||
assertFileContains(t, filepath.Join(destinationRoot, "index.html"), "<p>New.</p>")
|
||||
testutil.AssertFileContains(t, filepath.Join(destinationRoot, "index.html"), "<p>New.</p>")
|
||||
}
|
||||
|
||||
func TestRunSkipsWhenDestinationStateMatches(t *testing.T) {
|
||||
@@ -994,7 +1063,7 @@ func TestRunReplacesOlderDestination(t *testing.T) {
|
||||
if !strings.Contains(stdout.String(), "action=replace_older") {
|
||||
t.Fatalf("stdout = %q, want replace_older", stdout.String())
|
||||
}
|
||||
assertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nSunny.\n")
|
||||
testutil.AssertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nSunny.\n")
|
||||
}
|
||||
|
||||
func TestRunSkipsNewerDestination(t *testing.T) {
|
||||
@@ -1016,7 +1085,7 @@ func TestRunSkipsNewerDestination(t *testing.T) {
|
||||
if !strings.Contains(stdout.String(), "action=skip_destination_newer") {
|
||||
t.Fatalf("stdout = %q, want skip_destination_newer", stdout.String())
|
||||
}
|
||||
assertFile(t, filepath.Join(destinationRoot, "report.md"), "newer\n")
|
||||
testutil.AssertFile(t, filepath.Join(destinationRoot, "report.md"), "newer\n")
|
||||
}
|
||||
|
||||
func TestRunFailsOnConflict(t *testing.T) {
|
||||
@@ -1069,7 +1138,7 @@ func TestRunForceReplacesUnmanagedDestination(t *testing.T) {
|
||||
if _, err := os.Stat(filepath.Join(destinationRoot, "unmanaged.txt")); !os.IsNotExist(err) {
|
||||
t.Fatalf("unmanaged file stat error = %v, want not exist", err)
|
||||
}
|
||||
assertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nSunny.\n")
|
||||
testutil.AssertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nSunny.\n")
|
||||
}
|
||||
|
||||
func TestRunFansOutToLocalDestinations(t *testing.T) {
|
||||
@@ -1082,8 +1151,8 @@ func TestRunFansOutToLocalDestinations(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
assertFile(t, filepath.Join(firstDestination, "daily", "report.md"), "# Report\nSunny.\n")
|
||||
assertFile(t, filepath.Join(secondDestination, "daily", "summary.txt"), "Summary\n")
|
||||
testutil.AssertFile(t, filepath.Join(firstDestination, "daily", "report.md"), "# Report\nSunny.\n")
|
||||
testutil.AssertFile(t, filepath.Join(secondDestination, "daily", "summary.txt"), "Summary\n")
|
||||
}
|
||||
|
||||
func TestRunFansOutWithDifferentPublishPolicies(t *testing.T) {
|
||||
@@ -1092,16 +1161,16 @@ func TestRunFansOutWithDifferentPublishPolicies(t *testing.T) {
|
||||
htmlDestination := t.TempDir()
|
||||
writeSourceBundle(t, sourceRoot, "", testBundleOptions{})
|
||||
|
||||
err := Run(context.Background(), RunOptions{ConfigPath: writeMixedPolicyFanoutConfig(t, sourceRoot, archiveDestination, htmlDestination)})
|
||||
err := Run(context.Background(), RunOptions{ConfigPath: testutil.WriteMixedPolicyFanoutLocalConfig(t, sourceRoot, archiveDestination, htmlDestination)})
|
||||
if err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
assertFile(t, filepath.Join(archiveDestination, "report.md"), "# Report\nSunny.\n")
|
||||
assertFile(t, filepath.Join(archiveDestination, "summary.txt"), "Summary\n")
|
||||
testutil.AssertFile(t, filepath.Join(archiveDestination, "report.md"), "# Report\nSunny.\n")
|
||||
testutil.AssertFile(t, filepath.Join(archiveDestination, "summary.txt"), "Summary\n")
|
||||
if _, err := os.Stat(filepath.Join(archiveDestination, "report.html")); !os.IsNotExist(err) {
|
||||
t.Fatalf("archive report.html stat error = %v, want not exist", err)
|
||||
}
|
||||
assertFileContains(t, filepath.Join(htmlDestination, "report.html"), "<h1>Report</h1>")
|
||||
testutil.AssertFileContains(t, filepath.Join(htmlDestination, "report.html"), "<h1>Report</h1>")
|
||||
if _, err := os.Stat(filepath.Join(htmlDestination, "report.md")); !os.IsNotExist(err) {
|
||||
t.Fatalf("html report.md stat error = %v, want not exist", err)
|
||||
}
|
||||
@@ -1159,10 +1228,10 @@ func TestRunExercisesRemoteBackendShapesThroughCommonPath(t *testing.T) {
|
||||
if err := runConfigWithBackendFactory(context.Background(), cfg, RunOptions{Stdout: &publishOutput}, provider); err != nil {
|
||||
t.Fatalf("publish error = %v", err)
|
||||
}
|
||||
assertFakeFile(t, s3Destination, "report.md", "# Report\nSunny.\n")
|
||||
assertFakeFile(t, sshDestination, "summary.txt", "Summary\n")
|
||||
assertFile(t, filepath.Join(s3ToLocalDestination, "report.md"), "# Report\nSunny.\n")
|
||||
assertFile(t, filepath.Join(sshToLocalDestination, "summary.txt"), "Summary\n")
|
||||
testutil.AssertFakeFile(t, s3Destination, "report.md", "# Report\nSunny.\n")
|
||||
testutil.AssertFakeFile(t, sshDestination, "summary.txt", "Summary\n")
|
||||
testutil.AssertFile(t, filepath.Join(s3ToLocalDestination, "report.md"), "# Report\nSunny.\n")
|
||||
testutil.AssertFile(t, filepath.Join(sshToLocalDestination, "summary.txt"), "Summary\n")
|
||||
|
||||
var repeatOutput bytes.Buffer
|
||||
if err := runConfigWithBackendFactory(context.Background(), cfg, RunOptions{Stdout: &repeatOutput}, provider); err != nil {
|
||||
@@ -1178,10 +1247,10 @@ func TestRunForceReplacementStaysWithinRemoteBundlePaths(t *testing.T) {
|
||||
writeSourceBundle(t, localSourceRoot, "bundle", testBundleOptions{})
|
||||
s3Destination := fake.New()
|
||||
sshDestination := fake.New()
|
||||
mustWriteFake(t, s3Destination, "bundle/old.txt", "old")
|
||||
mustWriteFake(t, s3Destination, "bundle-sibling/keep.txt", "keep")
|
||||
mustWriteFake(t, sshDestination, "bundle/old.txt", "old")
|
||||
mustWriteFake(t, sshDestination, "bundle-sibling/keep.txt", "keep")
|
||||
testutil.WriteFakeFile(t, s3Destination, "bundle/old.txt", "old")
|
||||
testutil.WriteFakeFile(t, s3Destination, "bundle-sibling/keep.txt", "keep")
|
||||
testutil.WriteFakeFile(t, sshDestination, "bundle/old.txt", "old")
|
||||
testutil.WriteFakeFile(t, sshDestination, "bundle-sibling/keep.txt", "keep")
|
||||
cfg := config.Config{Pipelines: []config.Pipeline{{
|
||||
ID: "reports",
|
||||
Source: config.Backend{Backend: config.BackendLocal, Path: localSourceRoot},
|
||||
@@ -1209,12 +1278,12 @@ func TestRunForceReplacementStaysWithinRemoteBundlePaths(t *testing.T) {
|
||||
if err := runConfigWithBackendFactory(context.Background(), cfg, RunOptions{Force: true}, provider); err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
assertFakeFile(t, s3Destination, "bundle/report.md", "# Report\nSunny.\n")
|
||||
assertFakeMissing(t, s3Destination, "bundle/old.txt")
|
||||
assertFakeFile(t, s3Destination, "bundle-sibling/keep.txt", "keep")
|
||||
assertFakeFile(t, sshDestination, "bundle/report.md", "# Report\nSunny.\n")
|
||||
assertFakeMissing(t, sshDestination, "bundle/old.txt")
|
||||
assertFakeFile(t, sshDestination, "bundle-sibling/keep.txt", "keep")
|
||||
testutil.AssertFakeFile(t, s3Destination, "bundle/report.md", "# Report\nSunny.\n")
|
||||
testutil.AssertFakeMissing(t, s3Destination, "bundle/old.txt")
|
||||
testutil.AssertFakeFile(t, s3Destination, "bundle-sibling/keep.txt", "keep")
|
||||
testutil.AssertFakeFile(t, sshDestination, "bundle/report.md", "# Report\nSunny.\n")
|
||||
testutil.AssertFakeMissing(t, sshDestination, "bundle/old.txt")
|
||||
testutil.AssertFakeFile(t, sshDestination, "bundle-sibling/keep.txt", "keep")
|
||||
}
|
||||
|
||||
func TestRunDryRunDoesNotWrite(t *testing.T) {
|
||||
@@ -1272,141 +1341,11 @@ func writeLocalConfig(t *testing.T, sourceRoot, destinationRoot string) string {
|
||||
return testutil.WriteMinimalLocalConfig(t, sourceRoot, destinationRoot)
|
||||
}
|
||||
|
||||
func writeLocalConfigWithPolicy(t *testing.T, sourceRoot, destinationRoot string, publishSource, publishHTML bool) string {
|
||||
t.Helper()
|
||||
transformConfig := ""
|
||||
if publishHTML {
|
||||
transformConfig = `
|
||||
transform:
|
||||
markdown_to_html:
|
||||
enabled: true
|
||||
mode: sidecar`
|
||||
}
|
||||
return writeConfigFile(t, `
|
||||
pipelines:
|
||||
- id: reports
|
||||
source:
|
||||
backend: local
|
||||
path: `+sourceRoot+`
|
||||
destinations:
|
||||
- id: archive
|
||||
backend: local
|
||||
path: `+destinationRoot+`
|
||||
publish:
|
||||
source: `+fmt.Sprintf("%t", publishSource)+`
|
||||
html: `+fmt.Sprintf("%t", publishHTML)+transformConfig+`
|
||||
`)
|
||||
}
|
||||
|
||||
func writeLocalConfigWithPathMapping(t *testing.T, sourceRoot, destinationRoot, mode string) string {
|
||||
t.Helper()
|
||||
return writeConfigFile(t, `
|
||||
pipelines:
|
||||
- id: reports
|
||||
source:
|
||||
backend: local
|
||||
path: `+sourceRoot+`
|
||||
destinations:
|
||||
- id: archive
|
||||
backend: local
|
||||
path: `+destinationRoot+`
|
||||
path_mapping:
|
||||
mode: `+mode+`
|
||||
`)
|
||||
}
|
||||
|
||||
func writeLocalConfigWithLinks(t *testing.T, sourceRoot, destinationRoot, pathMapping, baseURL, primary string, publishSource, publishHTML bool, transformMode string) string {
|
||||
t.Helper()
|
||||
transformConfig := ""
|
||||
if publishHTML {
|
||||
transformConfig = `
|
||||
transform:
|
||||
markdown_to_html:
|
||||
enabled: true
|
||||
mode: ` + transformMode
|
||||
}
|
||||
return writeConfigFile(t, `
|
||||
pipelines:
|
||||
- id: reports
|
||||
source:
|
||||
backend: local
|
||||
path: `+sourceRoot+`
|
||||
destinations:
|
||||
- id: archive
|
||||
backend: local
|
||||
path: `+destinationRoot+`
|
||||
path_mapping:
|
||||
mode: `+pathMapping+`
|
||||
links:
|
||||
base_url: `+baseURL+`
|
||||
primary: `+primary+`
|
||||
publish:
|
||||
source: `+fmt.Sprintf("%t", publishSource)+`
|
||||
html: `+fmt.Sprintf("%t", publishHTML)+transformConfig+`
|
||||
`)
|
||||
}
|
||||
|
||||
func writeLocalConfigWithMarkdownTransform(t *testing.T, sourceRoot, destinationRoot string, publishSource, publishHTML bool, mode, input string) string {
|
||||
t.Helper()
|
||||
enabled := publishHTML
|
||||
inputConfig := ""
|
||||
if input != "" {
|
||||
inputConfig = `
|
||||
input: ` + input
|
||||
}
|
||||
return writeConfigFile(t, `
|
||||
pipelines:
|
||||
- id: reports
|
||||
source:
|
||||
backend: local
|
||||
path: `+sourceRoot+`
|
||||
destinations:
|
||||
- id: archive
|
||||
backend: local
|
||||
path: `+destinationRoot+`
|
||||
publish:
|
||||
source: `+fmt.Sprintf("%t", publishSource)+`
|
||||
html: `+fmt.Sprintf("%t", publishHTML)+`
|
||||
transform:
|
||||
markdown_to_html:
|
||||
enabled: `+fmt.Sprintf("%t", enabled)+`
|
||||
mode: `+mode+inputConfig+`
|
||||
`)
|
||||
}
|
||||
|
||||
func writeFanoutConfig(t *testing.T, sourceRoot, firstDestination, secondDestination string) string {
|
||||
t.Helper()
|
||||
return testutil.WriteFanoutLocalConfig(t, sourceRoot, firstDestination, secondDestination)
|
||||
}
|
||||
|
||||
func writeMixedPolicyFanoutConfig(t *testing.T, sourceRoot, archiveDestination, htmlDestination string) string {
|
||||
t.Helper()
|
||||
return writeConfigFile(t, `
|
||||
pipelines:
|
||||
- id: reports
|
||||
source:
|
||||
backend: local
|
||||
path: `+sourceRoot+`
|
||||
destinations:
|
||||
- id: archive
|
||||
backend: local
|
||||
path: `+archiveDestination+`
|
||||
publish:
|
||||
source: true
|
||||
html: false
|
||||
- id: html
|
||||
backend: local
|
||||
path: `+htmlDestination+`
|
||||
publish:
|
||||
source: false
|
||||
html: true
|
||||
transform:
|
||||
markdown_to_html:
|
||||
enabled: true
|
||||
mode: sidecar
|
||||
`)
|
||||
}
|
||||
|
||||
func writeConfigFile(t *testing.T, body string) string {
|
||||
t.Helper()
|
||||
path := filepath.Join(t.TempDir(), "config.yml")
|
||||
@@ -1434,53 +1373,6 @@ func outputsByPath(outputs []state.OutputFile) map[string]state.OutputFile {
|
||||
return byPath
|
||||
}
|
||||
|
||||
func assertFile(t *testing.T, path, want string) {
|
||||
t.Helper()
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatalf("read file %s: %v", path, err)
|
||||
}
|
||||
if got := string(data); got != want {
|
||||
t.Fatalf("%s = %q, want %q", path, got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func assertFileContains(t *testing.T, path, want string) {
|
||||
t.Helper()
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatalf("read file %s: %v", path, err)
|
||||
}
|
||||
if !strings.Contains(string(data), want) {
|
||||
t.Fatalf("%s = %q, want substring %q", path, data, want)
|
||||
}
|
||||
}
|
||||
|
||||
func assertFakeFile(t *testing.T, backend *fake.Backend, path, want string) {
|
||||
t.Helper()
|
||||
data, err := backend.ReadFile(context.Background(), path)
|
||||
if err != nil {
|
||||
t.Fatalf("read fake file %s: %v", path, err)
|
||||
}
|
||||
if got := string(data); got != want {
|
||||
t.Fatalf("%s = %q, want %q", path, got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func assertFakeMissing(t *testing.T, backend *fake.Backend, path string) {
|
||||
t.Helper()
|
||||
if _, err := backend.Stat(context.Background(), path); !storage.IsNotFound(err) {
|
||||
t.Fatalf("fake file %s stat error = %v, want not found", path, err)
|
||||
}
|
||||
}
|
||||
|
||||
func mustWriteFake(t *testing.T, backend *fake.Backend, path, data string) {
|
||||
t.Helper()
|
||||
if _, err := backend.WriteFile(context.Background(), path, []byte(data), storage.WriteOptions{}); err != nil {
|
||||
t.Fatalf("write fake file %s: %v", path, err)
|
||||
}
|
||||
}
|
||||
|
||||
func crossBackendConfig(localSourceRoot, s3ToLocalDestination, sshToLocalDestination string) config.Config {
|
||||
cfg := config.Config{
|
||||
Pipelines: []config.Pipeline{
|
||||
|
||||
47
internal/app/run_warnings.go
Normal file
47
internal/app/run_warnings.go
Normal file
@@ -0,0 +1,47 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/config"
|
||||
)
|
||||
|
||||
func secretConflictWarnings(conflicts []config.SecretConflict) []OutputWarning {
|
||||
warnings := make([]OutputWarning, 0, len(conflicts))
|
||||
for _, conflict := range conflicts {
|
||||
warnings = append(warnings, OutputWarning{
|
||||
Message: fmt.Sprintf("secret %s ignored because the real environment already has that variable", conflict.Name),
|
||||
})
|
||||
}
|
||||
return warnings
|
||||
}
|
||||
|
||||
func sshWarnings(pipeline config.Pipeline) []OutputWarning {
|
||||
var warnings []OutputWarning
|
||||
if pipeline.Source.Backend == config.BackendSSH && pipeline.Source.SSH.HostKeyPolicy == config.HostKeyPolicyOff {
|
||||
warnings = append(warnings, OutputWarning{
|
||||
Message: fmt.Sprintf("pipeline=%s source host_key_policy=off disables SSH host key checking", pipeline.ID),
|
||||
})
|
||||
}
|
||||
for _, destination := range pipeline.Destinations {
|
||||
if destination.Backend == config.BackendSSH && destination.SSH.HostKeyPolicy == config.HostKeyPolicyOff {
|
||||
warnings = append(warnings, OutputWarning{
|
||||
Message: fmt.Sprintf("pipeline=%s destination=%s host_key_policy=off disables SSH host key checking", pipeline.ID, destination.ID),
|
||||
})
|
||||
}
|
||||
}
|
||||
return warnings
|
||||
}
|
||||
|
||||
func writeWarnings(w io.Writer, warnings []OutputWarning) error {
|
||||
if w == nil {
|
||||
return nil
|
||||
}
|
||||
for _, warning := range warnings {
|
||||
if _, err := fmt.Fprintf(w, "Warning: %s\n", warning.Message); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -1,19 +1,9 @@
|
||||
package bundle
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"regexp"
|
||||
|
||||
publicbundle "gitea.maximumdirect.net/eric/distributor/pkg/bundle"
|
||||
)
|
||||
|
||||
var digestPattern = regexp.MustCompile(`^sha256:[0-9a-f]{64}$`)
|
||||
import publicbundle "gitea.maximumdirect.net/eric/distributor/pkg/bundle"
|
||||
|
||||
func ValidateDigest(value string) error {
|
||||
if !digestPattern.MatchString(value) {
|
||||
return fmt.Errorf("must be lowercase sha256:<64 hex>")
|
||||
}
|
||||
return nil
|
||||
return publicbundle.ValidateDigest(value)
|
||||
}
|
||||
|
||||
func FileDigest(data []byte) string {
|
||||
|
||||
@@ -2,7 +2,6 @@ package cli
|
||||
|
||||
import (
|
||||
"context"
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
|
||||
@@ -14,33 +13,17 @@ func inspectCommand(ctx context.Context, args []string, stdout, stderr io.Writer
|
||||
printInspectHelp(stdout)
|
||||
return exitOK
|
||||
}
|
||||
flags := flag.NewFlagSet("inspect", flag.ContinueOnError)
|
||||
flags.SetOutput(stderr)
|
||||
configPath := flags.String("config", "", "path to config file")
|
||||
pipelineID := flags.String("pipeline", "", "pipeline id")
|
||||
bundlePath := flags.String("bundle", "", "source-root-relative bundle path")
|
||||
formatFlag := addFormatFlag(flags)
|
||||
if err := flags.Parse(args); err != nil {
|
||||
return exitUsage
|
||||
}
|
||||
format, ok := parseOutputFormat(stderr, "inspect", *formatFlag)
|
||||
parsed, ok := parseSourceDiagnosticArgs(stderr, "inspect", args)
|
||||
if !ok {
|
||||
return exitUsage
|
||||
}
|
||||
path, ok := parseOptionalPathArg(stderr, "inspect", flags.Args())
|
||||
if !ok {
|
||||
return exitUsage
|
||||
}
|
||||
if !validateInspectModeOK(stderr, "inspect", path, *configPath, *pipelineID, *bundlePath) {
|
||||
return exitUsage
|
||||
}
|
||||
if err := app.Inspect(ctx, app.InspectOptions{
|
||||
Path: path,
|
||||
ConfigPath: *configPath,
|
||||
PipelineID: *pipelineID,
|
||||
BundlePath: *bundlePath,
|
||||
Path: parsed.Path,
|
||||
ConfigPath: parsed.ConfigPath,
|
||||
PipelineID: parsed.PipelineID,
|
||||
BundlePath: parsed.BundlePath,
|
||||
Stdout: stdout,
|
||||
OutputFormat: format,
|
||||
OutputFormat: parsed.OutputFormat,
|
||||
}); err != nil {
|
||||
return fail(stderr, err)
|
||||
}
|
||||
|
||||
@@ -217,12 +217,24 @@ func TestExecuteValidateArgs(t *testing.T) {
|
||||
wantCode: exitUsage,
|
||||
wantStderr: "requires --config",
|
||||
},
|
||||
{
|
||||
name: "bundle without config",
|
||||
args: []string{"validate", "--bundle", "daily"},
|
||||
wantCode: exitUsage,
|
||||
wantStderr: "requires --config",
|
||||
},
|
||||
{
|
||||
name: "config without pipeline",
|
||||
args: []string{"validate", "--config", "config.yml"},
|
||||
wantCode: exitUsage,
|
||||
wantStderr: "requires --pipeline",
|
||||
},
|
||||
{
|
||||
name: "invalid format",
|
||||
args: []string{"validate", "--format", "xml", validPath},
|
||||
wantCode: exitUsage,
|
||||
wantStderr: "format must be text or json",
|
||||
},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
@@ -343,12 +355,24 @@ func TestExecuteInspectArgs(t *testing.T) {
|
||||
wantCode: exitUsage,
|
||||
wantStderr: "requires --config",
|
||||
},
|
||||
{
|
||||
name: "bundle without config",
|
||||
args: []string{"inspect", "--bundle", "daily"},
|
||||
wantCode: exitUsage,
|
||||
wantStderr: "requires --config",
|
||||
},
|
||||
{
|
||||
name: "config without pipeline",
|
||||
args: []string{"inspect", "--config", "config.yml"},
|
||||
wantCode: exitUsage,
|
||||
wantStderr: "requires --pipeline",
|
||||
},
|
||||
{
|
||||
name: "invalid format",
|
||||
args: []string{"inspect", "--format", "xml", validPath},
|
||||
wantCode: exitUsage,
|
||||
wantStderr: "format must be text or json",
|
||||
},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
|
||||
@@ -1,10 +1,51 @@
|
||||
package cli
|
||||
|
||||
import (
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/app"
|
||||
)
|
||||
|
||||
type sourceDiagnosticArgs struct {
|
||||
Path string
|
||||
ConfigPath string
|
||||
PipelineID string
|
||||
BundlePath string
|
||||
OutputFormat app.OutputFormat
|
||||
}
|
||||
|
||||
func parseSourceDiagnosticArgs(stderr io.Writer, command string, args []string) (sourceDiagnosticArgs, bool) {
|
||||
flags := flag.NewFlagSet(command, flag.ContinueOnError)
|
||||
flags.SetOutput(stderr)
|
||||
configPath := flags.String("config", "", "path to config file")
|
||||
pipelineID := flags.String("pipeline", "", "pipeline id")
|
||||
bundlePath := flags.String("bundle", "", "source-root-relative bundle path")
|
||||
formatFlag := addFormatFlag(flags)
|
||||
if err := flags.Parse(args); err != nil {
|
||||
return sourceDiagnosticArgs{}, false
|
||||
}
|
||||
format, ok := parseOutputFormat(stderr, command, *formatFlag)
|
||||
if !ok {
|
||||
return sourceDiagnosticArgs{}, false
|
||||
}
|
||||
path, ok := parseOptionalPathArg(stderr, command, flags.Args())
|
||||
if !ok {
|
||||
return sourceDiagnosticArgs{}, false
|
||||
}
|
||||
if !validateInspectModeOK(stderr, command, path, *configPath, *pipelineID, *bundlePath) {
|
||||
return sourceDiagnosticArgs{}, false
|
||||
}
|
||||
return sourceDiagnosticArgs{
|
||||
Path: path,
|
||||
ConfigPath: *configPath,
|
||||
PipelineID: *pipelineID,
|
||||
BundlePath: *bundlePath,
|
||||
OutputFormat: format,
|
||||
}, true
|
||||
}
|
||||
|
||||
func validateInspectModeOK(stderr io.Writer, command, path, configPath, pipelineID, bundlePath string) bool {
|
||||
configMode := configPath != "" || pipelineID != "" || bundlePath != ""
|
||||
if !configMode {
|
||||
|
||||
@@ -2,7 +2,6 @@ package cli
|
||||
|
||||
import (
|
||||
"context"
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
|
||||
@@ -14,33 +13,17 @@ func validateCommand(ctx context.Context, args []string, stdout, stderr io.Write
|
||||
printValidateHelp(stdout)
|
||||
return exitOK
|
||||
}
|
||||
flags := flag.NewFlagSet("validate", flag.ContinueOnError)
|
||||
flags.SetOutput(stderr)
|
||||
configPath := flags.String("config", "", "path to config file")
|
||||
pipelineID := flags.String("pipeline", "", "pipeline id")
|
||||
bundlePath := flags.String("bundle", "", "source-root-relative bundle path")
|
||||
formatFlag := addFormatFlag(flags)
|
||||
if err := flags.Parse(args); err != nil {
|
||||
return exitUsage
|
||||
}
|
||||
format, ok := parseOutputFormat(stderr, "validate", *formatFlag)
|
||||
parsed, ok := parseSourceDiagnosticArgs(stderr, "validate", args)
|
||||
if !ok {
|
||||
return exitUsage
|
||||
}
|
||||
path, ok := parseOptionalPathArg(stderr, "validate", flags.Args())
|
||||
if !ok {
|
||||
return exitUsage
|
||||
}
|
||||
if !validateInspectModeOK(stderr, "validate", path, *configPath, *pipelineID, *bundlePath) {
|
||||
return exitUsage
|
||||
}
|
||||
if err := app.Validate(ctx, app.ValidateOptions{
|
||||
Path: path,
|
||||
ConfigPath: *configPath,
|
||||
PipelineID: *pipelineID,
|
||||
BundlePath: *bundlePath,
|
||||
Path: parsed.Path,
|
||||
ConfigPath: parsed.ConfigPath,
|
||||
PipelineID: parsed.PipelineID,
|
||||
BundlePath: parsed.BundlePath,
|
||||
Stdout: stdout,
|
||||
OutputFormat: format,
|
||||
OutputFormat: parsed.OutputFormat,
|
||||
}); err != nil {
|
||||
return fail(stderr, err)
|
||||
}
|
||||
|
||||
@@ -2,9 +2,10 @@ package config
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net/url"
|
||||
"regexp"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/link"
|
||||
)
|
||||
|
||||
var idPattern = regexp.MustCompile(`^[a-zA-Z0-9][a-zA-Z0-9._-]*$`)
|
||||
@@ -187,7 +188,7 @@ func validateLinks(errs ValidationErrors, context string, links *Links) Validati
|
||||
}
|
||||
if links.BaseURL == "" {
|
||||
errs = append(errs, context+".base_url is required")
|
||||
} else if err := validateLinkBaseURL(links.BaseURL); err != nil {
|
||||
} else if err := link.ValidateHTTPURL(links.BaseURL); err != nil {
|
||||
errs = append(errs, context+".base_url "+err.Error())
|
||||
}
|
||||
switch links.Primary {
|
||||
@@ -198,26 +199,6 @@ func validateLinks(errs ValidationErrors, context string, links *Links) Validati
|
||||
return errs
|
||||
}
|
||||
|
||||
func validateLinkBaseURL(value string) error {
|
||||
parsed, err := url.Parse(value)
|
||||
if err != nil {
|
||||
return fmt.Errorf("must be a valid URL")
|
||||
}
|
||||
if parsed.Scheme != "http" && parsed.Scheme != "https" {
|
||||
return fmt.Errorf("must use http or https")
|
||||
}
|
||||
if parsed.Host == "" {
|
||||
return fmt.Errorf("must include a host")
|
||||
}
|
||||
if parsed.RawQuery != "" {
|
||||
return fmt.Errorf("must not include a query string")
|
||||
}
|
||||
if parsed.Fragment != "" {
|
||||
return fmt.Errorf("must not include a fragment")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func validateTransferPolicy(errs ValidationErrors, context string, policy TransferPolicy) ValidationErrors {
|
||||
if policy.OnDestinationSame != TransferActionSkip && policy.OnDestinationSame != TransferActionFail {
|
||||
errs = append(errs, context+".on_destination_same must be skip or fail")
|
||||
|
||||
@@ -1,6 +1,9 @@
|
||||
package config
|
||||
|
||||
import "testing"
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestValidatePublishTransformPolicy(t *testing.T) {
|
||||
tests := publishTransformPolicyCases()
|
||||
@@ -145,6 +148,28 @@ func TestValidateLinks(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateLinksReportsFieldContext(t *testing.T) {
|
||||
cfg := Config{Pipelines: []Pipeline{{
|
||||
ID: "reports",
|
||||
Source: Backend{Backend: BackendLocal, Path: "/source"},
|
||||
Destinations: []Destination{{
|
||||
ID: "web",
|
||||
Backend: BackendLocal,
|
||||
Path: "/destination",
|
||||
Links: &Links{BaseURL: "https://reports.example.com/archive?preview=1", Primary: LinkPrimaryAuto},
|
||||
}},
|
||||
}}}
|
||||
ApplyDefaults(&cfg)
|
||||
err := Validate(cfg)
|
||||
if err == nil {
|
||||
t.Fatal("Validate() error = nil, want error")
|
||||
}
|
||||
want := "pipelines[0].destinations[0].links.base_url must not include a query string"
|
||||
if !strings.Contains(err.Error(), want) {
|
||||
t.Fatalf("Validate() error = %q, want %q", err, want)
|
||||
}
|
||||
}
|
||||
|
||||
type publishTransformPolicyCase struct {
|
||||
name string
|
||||
publish PublishPolicy
|
||||
|
||||
26
internal/link/url.go
Normal file
26
internal/link/url.go
Normal file
@@ -0,0 +1,26 @@
|
||||
package link
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net/url"
|
||||
)
|
||||
|
||||
func ValidateHTTPURL(value string) error {
|
||||
parsed, err := url.Parse(value)
|
||||
if err != nil {
|
||||
return fmt.Errorf("must be a valid URL")
|
||||
}
|
||||
if parsed.Scheme != "http" && parsed.Scheme != "https" {
|
||||
return fmt.Errorf("must use http or https")
|
||||
}
|
||||
if parsed.Host == "" {
|
||||
return fmt.Errorf("must include a host")
|
||||
}
|
||||
if parsed.RawQuery != "" {
|
||||
return fmt.Errorf("must not include a query string")
|
||||
}
|
||||
if parsed.Fragment != "" {
|
||||
return fmt.Errorf("must not include a fragment")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
39
internal/link/url_test.go
Normal file
39
internal/link/url_test.go
Normal file
@@ -0,0 +1,39 @@
|
||||
package link
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestValidateHTTPURL(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
value string
|
||||
wantErr string
|
||||
}{
|
||||
{name: "http", value: "http://reports.example.com/archive"},
|
||||
{name: "https", value: "https://reports.example.com/archive"},
|
||||
{name: "missing host", value: "https:///archive", wantErr: "must include a host"},
|
||||
{name: "unsupported scheme", value: "ftp://reports.example.com/archive", wantErr: "must use http or https"},
|
||||
{name: "query string", value: "https://reports.example.com/archive?preview=1", wantErr: "must not include a query string"},
|
||||
{name: "fragment", value: "https://reports.example.com/archive#top", wantErr: "must not include a fragment"},
|
||||
{name: "parse failure", value: "http://[::1", wantErr: "must be a valid URL"},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
err := ValidateHTTPURL(tt.value)
|
||||
if tt.wantErr == "" {
|
||||
if err != nil {
|
||||
t.Fatalf("ValidateHTTPURL() error = %v", err)
|
||||
}
|
||||
return
|
||||
}
|
||||
if err == nil {
|
||||
t.Fatal("ValidateHTTPURL() error = nil, want error")
|
||||
}
|
||||
if !strings.Contains(err.Error(), tt.wantErr) {
|
||||
t.Fatalf("ValidateHTTPURL() error = %q, want %q", err, tt.wantErr)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -23,7 +23,7 @@ func Execute(ctx context.Context, req Request, plan Plan) error {
|
||||
if plan.ExistingState == nil {
|
||||
return fmt.Errorf("replace requires existing destination state")
|
||||
}
|
||||
if err := req.DestinationBackend.DeleteManagedBundle(ctx, req.DestinationBundlePath, existingManagedOutputPaths(*plan.ExistingState), storage.DeleteOptions{IgnoreMissing: true, PruneEmptyDirs: true}); err != nil {
|
||||
if err := req.DestinationBackend.DeleteManagedBundle(ctx, req.DestinationBundlePath, stateOutputManagedPaths(plan.ExistingState.Outputs), storage.DeleteOptions{IgnoreMissing: true, PruneEmptyDirs: true}); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := ensureDestinationEmpty(ctx, req.DestinationBackend, req.DestinationBundlePath); err != nil {
|
||||
@@ -41,7 +41,7 @@ func Execute(ctx context.Context, req Request, plan Plan) error {
|
||||
|
||||
writtenOutputs := make([]Output, 0, len(plan.Outputs))
|
||||
cleanup := func() {
|
||||
_ = req.DestinationBackend.DeleteManagedBundle(ctx, req.DestinationBundlePath, managedOutputPaths(writtenOutputs), storage.DeleteOptions{IgnoreMissing: true, PruneEmptyDirs: true})
|
||||
_ = req.DestinationBackend.DeleteManagedBundle(ctx, req.DestinationBundlePath, ManagedOutputPaths(writtenOutputs), storage.DeleteOptions{IgnoreMissing: true, PruneEmptyDirs: true})
|
||||
}
|
||||
for _, output := range plan.Outputs {
|
||||
destinationPath, err := storage.Join(req.DestinationBundlePath, output.DestinationPath)
|
||||
@@ -76,7 +76,7 @@ func Execute(ctx context.Context, req Request, plan Plan) error {
|
||||
DestinationID: req.DestinationID,
|
||||
PublishedAt: time.Now().UTC(),
|
||||
Source: state.SourceState{Manifest: req.SourceBundle.Manifest},
|
||||
Outputs: stateOutputs(plan.Outputs),
|
||||
Outputs: StateOutputFiles(plan.Outputs),
|
||||
}
|
||||
if plan.PrimaryURL != "" {
|
||||
destinationState.Links = &state.LinkState{PrimaryURL: plan.PrimaryURL}
|
||||
@@ -102,11 +102,3 @@ func Execute(ctx context.Context, req Request, plan Plan) error {
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func existingManagedOutputPaths(destinationState state.DistributorState) []string {
|
||||
paths := make([]string, 0, len(destinationState.Outputs))
|
||||
for _, output := range destinationState.Outputs {
|
||||
paths = append(paths, output.Path)
|
||||
}
|
||||
return paths
|
||||
}
|
||||
|
||||
@@ -2,13 +2,11 @@ package publish
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/bundle"
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/config"
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/storage"
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/storage/fake"
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/testutil"
|
||||
)
|
||||
@@ -25,7 +23,7 @@ func TestBuildPlansForcedReplacementOnlyWhenExplicit(t *testing.T) {
|
||||
name: "unmanaged content",
|
||||
prepare: func(t *testing.T, backend *fake.Backend, source bundle.Manifest) {
|
||||
t.Helper()
|
||||
writeFakeFile(t, backend, "bundle/old.txt", "old")
|
||||
testutil.WriteFakeFile(t, backend, "bundle/old.txt", "old")
|
||||
},
|
||||
transfer: defaultTransfer(),
|
||||
wantReason: "fail_unmanaged",
|
||||
@@ -37,7 +35,7 @@ func TestBuildPlansForcedReplacementOnlyWhenExplicit(t *testing.T) {
|
||||
t.Helper()
|
||||
conflict := source
|
||||
conflict.ID = "other.source"
|
||||
writeFakeDestinationState(t, backend, "bundle", conflict, testutil.DestinationStateOptions{})
|
||||
testutil.WriteFakeDestinationState(t, backend, "bundle", conflict, testutil.DestinationStateOptions{})
|
||||
},
|
||||
transfer: conflictReplaceTransfer(),
|
||||
wantReason: "requires --force",
|
||||
@@ -48,7 +46,7 @@ func TestBuildPlansForcedReplacementOnlyWhenExplicit(t *testing.T) {
|
||||
prepare: func(t *testing.T, backend *fake.Backend, source bundle.Manifest) {
|
||||
t.Helper()
|
||||
conflict := testutil.ValidManifest(testutil.BundleOptions{Files: []testutil.SourceFile{{Path: "report.md", Data: "# Different\n"}}})
|
||||
writeFakeDestinationState(t, backend, "bundle", conflict, testutil.DestinationStateOptions{})
|
||||
testutil.WriteFakeDestinationState(t, backend, "bundle", conflict, testutil.DestinationStateOptions{})
|
||||
},
|
||||
transfer: conflictReplaceTransfer(),
|
||||
wantReason: "requires --force",
|
||||
@@ -58,7 +56,7 @@ func TestBuildPlansForcedReplacementOnlyWhenExplicit(t *testing.T) {
|
||||
name: "pipeline mismatch",
|
||||
prepare: func(t *testing.T, backend *fake.Backend, source bundle.Manifest) {
|
||||
t.Helper()
|
||||
writeFakeDestinationState(t, backend, "bundle", source, testutil.DestinationStateOptions{PipelineID: "other-pipeline"})
|
||||
testutil.WriteFakeDestinationState(t, backend, "bundle", source, testutil.DestinationStateOptions{PipelineID: "other-pipeline"})
|
||||
},
|
||||
transfer: conflictReplaceTransfer(),
|
||||
wantReason: "requires --force",
|
||||
@@ -68,7 +66,7 @@ func TestBuildPlansForcedReplacementOnlyWhenExplicit(t *testing.T) {
|
||||
name: "destination mismatch",
|
||||
prepare: func(t *testing.T, backend *fake.Backend, source bundle.Manifest) {
|
||||
t.Helper()
|
||||
writeFakeDestinationState(t, backend, "bundle", source, testutil.DestinationStateOptions{DestinationID: "other-destination"})
|
||||
testutil.WriteFakeDestinationState(t, backend, "bundle", source, testutil.DestinationStateOptions{DestinationID: "other-destination"})
|
||||
},
|
||||
transfer: conflictReplaceTransfer(),
|
||||
wantReason: "requires --force",
|
||||
@@ -80,7 +78,7 @@ func TestBuildPlansForcedReplacementOnlyWhenExplicit(t *testing.T) {
|
||||
t.Helper()
|
||||
newer := source
|
||||
newer.Created = newer.Created.AddDate(0, 0, 1)
|
||||
writeFakeDestinationState(t, backend, "bundle", newer, testutil.DestinationStateOptions{})
|
||||
testutil.WriteFakeDestinationState(t, backend, "bundle", newer, testutil.DestinationStateOptions{})
|
||||
},
|
||||
transfer: newerReplaceTransfer(),
|
||||
wantReason: "requires --force",
|
||||
@@ -121,7 +119,7 @@ func TestBuildRequiresConflictPolicyForStateConflicts(t *testing.T) {
|
||||
destinationBackend := fake.New()
|
||||
conflict := sourceBundle.Manifest
|
||||
conflict.ID = "other.source"
|
||||
writeFakeDestinationState(t, destinationBackend, "bundle", conflict, testutil.DestinationStateOptions{})
|
||||
testutil.WriteFakeDestinationState(t, destinationBackend, "bundle", conflict, testutil.DestinationStateOptions{})
|
||||
|
||||
req := forceRequest(sourceBackend, destinationBackend, sourceBundle, defaultTransfer())
|
||||
req.Force = true
|
||||
@@ -135,10 +133,10 @@ func TestExecuteForcedReplacementDeletesOnlyBundlePath(t *testing.T) {
|
||||
sourceBackend := fake.New()
|
||||
sourceBundle := testutil.WriteFakeSourceBundle(t, sourceBackend, "bundle", testutil.BundleOptions{})
|
||||
destinationBackend := fake.New()
|
||||
writeFakeFile(t, destinationBackend, "bundle/old.txt", "old")
|
||||
writeFakeFile(t, destinationBackend, "bundle/nested/old.txt", "old")
|
||||
writeFakeFile(t, destinationBackend, "bundle-sibling/keep.txt", "keep")
|
||||
writeFakeFile(t, destinationBackend, "outside.txt", "outside")
|
||||
testutil.WriteFakeFile(t, destinationBackend, "bundle/old.txt", "old")
|
||||
testutil.WriteFakeFile(t, destinationBackend, "bundle/nested/old.txt", "old")
|
||||
testutil.WriteFakeFile(t, destinationBackend, "bundle-sibling/keep.txt", "keep")
|
||||
testutil.WriteFakeFile(t, destinationBackend, "outside.txt", "outside")
|
||||
|
||||
req := forceRequest(sourceBackend, destinationBackend, sourceBundle, defaultTransfer())
|
||||
req.Force = true
|
||||
@@ -152,11 +150,11 @@ func TestExecuteForcedReplacementDeletesOnlyBundlePath(t *testing.T) {
|
||||
if err := Execute(context.Background(), req, plan); err != nil {
|
||||
t.Fatalf("Execute() error = %v", err)
|
||||
}
|
||||
assertFakeFile(t, destinationBackend, "bundle/report.md", "# Report\nSunny.\n")
|
||||
assertFakeMissing(t, destinationBackend, "bundle/old.txt")
|
||||
assertFakeMissing(t, destinationBackend, "bundle/nested/old.txt")
|
||||
assertFakeFile(t, destinationBackend, "bundle-sibling/keep.txt", "keep")
|
||||
assertFakeFile(t, destinationBackend, "outside.txt", "outside")
|
||||
testutil.AssertFakeFile(t, destinationBackend, "bundle/report.md", "# Report\nSunny.\n")
|
||||
testutil.AssertFakeMissing(t, destinationBackend, "bundle/old.txt")
|
||||
testutil.AssertFakeMissing(t, destinationBackend, "bundle/nested/old.txt")
|
||||
testutil.AssertFakeFile(t, destinationBackend, "bundle-sibling/keep.txt", "keep")
|
||||
testutil.AssertFakeFile(t, destinationBackend, "outside.txt", "outside")
|
||||
}
|
||||
|
||||
func forceRequest(sourceBackend, destinationBackend *fake.Backend, sourceBundle bundle.Bundle, transfer config.TransferPolicy) Request {
|
||||
@@ -193,49 +191,3 @@ func newerReplaceTransfer() config.TransferPolicy {
|
||||
transfer.OnDestinationNewer = config.TransferActionReplace
|
||||
return transfer
|
||||
}
|
||||
|
||||
func writeFakeDestinationState(t *testing.T, backend *fake.Backend, relative string, manifest bundle.Manifest, opts testutil.DestinationStateOptions) {
|
||||
t.Helper()
|
||||
destinationState := testutil.DestinationState(manifest, opts)
|
||||
data, err := json.MarshalIndent(destinationState, "", " ")
|
||||
if err != nil {
|
||||
t.Fatalf("marshal destination state: %v", err)
|
||||
}
|
||||
statePath, err := storage.StatePath(relative)
|
||||
if err != nil {
|
||||
t.Fatalf("state path: %v", err)
|
||||
}
|
||||
writeFakeFile(t, backend, statePath, string(append(data, '\n')))
|
||||
for _, output := range destinationState.Outputs {
|
||||
path, err := storage.Join(relative, output.Path)
|
||||
if err != nil {
|
||||
t.Fatalf("join output path: %v", err)
|
||||
}
|
||||
writeFakeFile(t, backend, path, "old")
|
||||
}
|
||||
}
|
||||
|
||||
func writeFakeFile(t *testing.T, backend *fake.Backend, path, data string) {
|
||||
t.Helper()
|
||||
if _, err := backend.WriteFile(context.Background(), path, []byte(data), storage.WriteOptions{}); err != nil {
|
||||
t.Fatalf("write fake file %s: %v", path, err)
|
||||
}
|
||||
}
|
||||
|
||||
func assertFakeFile(t *testing.T, backend *fake.Backend, path, want string) {
|
||||
t.Helper()
|
||||
data, err := backend.ReadFile(context.Background(), path)
|
||||
if err != nil {
|
||||
t.Fatalf("read fake file %s: %v", path, err)
|
||||
}
|
||||
if got := string(data); got != want {
|
||||
t.Fatalf("fake file %s = %q, want %q", path, got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func assertFakeMissing(t *testing.T, backend *fake.Backend, path string) {
|
||||
t.Helper()
|
||||
if _, err := backend.Stat(context.Background(), path); !storage.IsNotFound(err) {
|
||||
t.Fatalf("fake file %s stat error = %v, want not found", path, err)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -6,6 +6,7 @@ import (
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/config"
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/link"
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/state"
|
||||
)
|
||||
|
||||
@@ -13,6 +14,9 @@ func PlanLinks(req Request, outputs []Output) ([]Output, string, error) {
|
||||
if req.Links == nil {
|
||||
return outputs, "", nil
|
||||
}
|
||||
if err := link.ValidateHTTPURL(req.Links.BaseURL); err != nil {
|
||||
return nil, "", fmt.Errorf("link base URL: %w", err)
|
||||
}
|
||||
linked := make([]Output, 0, len(outputs))
|
||||
for _, output := range outputs {
|
||||
outputURL, err := OutputURL(req.Links.BaseURL, req.DestinationBundlePath, output.DestinationPath)
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package publish
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/config"
|
||||
@@ -147,3 +148,22 @@ func TestPlanLinksLeavesOutputsUnchangedWithoutConfig(t *testing.T) {
|
||||
t.Fatalf("output URL = %q, want empty", linked[0].URL)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPlanLinksValidatesBaseURLBeforePlanning(t *testing.T) {
|
||||
_, _, err := PlanLinks(Request{
|
||||
DestinationBundlePath: "daily",
|
||||
Links: &config.Links{
|
||||
BaseURL: "https://reports.example.com/archive?preview=1",
|
||||
Primary: config.LinkPrimaryAuto,
|
||||
},
|
||||
}, []Output{{
|
||||
DestinationPath: "report.md",
|
||||
Kind: state.OutputKindSource,
|
||||
}})
|
||||
if err == nil {
|
||||
t.Fatal("PlanLinks() error = nil, want error")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "link base URL: must not include a query string") {
|
||||
t.Fatalf("PlanLinks() error = %q, want link base URL context", err)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -97,26 +97,42 @@ func rejectOutputCollisions(outputs []Output) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func stateOutputs(outputs []Output) []state.OutputFile {
|
||||
func (o Output) StateOutputFile() state.OutputFile {
|
||||
return state.OutputFile{
|
||||
Path: o.DestinationPath,
|
||||
Kind: o.Kind,
|
||||
SourcePath: o.SourcePath,
|
||||
Transform: o.Transform,
|
||||
URL: o.URL,
|
||||
SHA256: o.SHA256,
|
||||
Size: o.Size,
|
||||
}
|
||||
}
|
||||
|
||||
func (o Output) ManagedPath() string {
|
||||
return o.DestinationPath
|
||||
}
|
||||
|
||||
func StateOutputFiles(outputs []Output) []state.OutputFile {
|
||||
files := make([]state.OutputFile, 0, len(outputs))
|
||||
for _, output := range outputs {
|
||||
files = append(files, state.OutputFile{
|
||||
Path: output.DestinationPath,
|
||||
Kind: output.Kind,
|
||||
SourcePath: output.SourcePath,
|
||||
Transform: output.Transform,
|
||||
URL: output.URL,
|
||||
SHA256: output.SHA256,
|
||||
Size: output.Size,
|
||||
})
|
||||
files = append(files, output.StateOutputFile())
|
||||
}
|
||||
return files
|
||||
}
|
||||
|
||||
func managedOutputPaths(outputs []Output) []string {
|
||||
func ManagedOutputPaths(outputs []Output) []string {
|
||||
paths := make([]string, 0, len(outputs))
|
||||
for _, output := range outputs {
|
||||
paths = append(paths, output.DestinationPath)
|
||||
paths = append(paths, output.ManagedPath())
|
||||
}
|
||||
return paths
|
||||
}
|
||||
|
||||
func stateOutputManagedPaths(outputs []state.OutputFile) []string {
|
||||
paths := make([]string, 0, len(outputs))
|
||||
for _, output := range outputs {
|
||||
paths = append(paths, output.Path)
|
||||
}
|
||||
return paths
|
||||
}
|
||||
|
||||
@@ -6,11 +6,58 @@ import (
|
||||
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/bundle"
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/config"
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/state"
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/storage/fake"
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/testutil"
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/transform"
|
||||
)
|
||||
|
||||
func TestOutputStateProjection(t *testing.T) {
|
||||
sourceOutput := Output{
|
||||
SourcePath: "report.md",
|
||||
DestinationPath: "report.md",
|
||||
Kind: state.OutputKindSource,
|
||||
URL: "https://reports.example.com/report.md",
|
||||
SHA256: "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
Size: 123,
|
||||
}
|
||||
sourceState := sourceOutput.StateOutputFile()
|
||||
if sourceState.Path != "report.md" || sourceState.Kind != state.OutputKindSource || sourceState.SourcePath != "report.md" || sourceState.URL != sourceOutput.URL || sourceState.SHA256 != sourceOutput.SHA256 || sourceState.Size != sourceOutput.Size {
|
||||
t.Fatalf("source state output = %#v", sourceState)
|
||||
}
|
||||
|
||||
generatedOutput := Output{
|
||||
SourcePath: "report.md",
|
||||
DestinationPath: "report.html",
|
||||
Kind: state.OutputKindGenerated,
|
||||
Transform: transform.MarkdownToHTML,
|
||||
URL: "https://reports.example.com/report.html",
|
||||
SHA256: "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
Size: 456,
|
||||
}
|
||||
generatedState := generatedOutput.StateOutputFile()
|
||||
if generatedState.Path != "report.html" || generatedState.Kind != state.OutputKindGenerated || generatedState.SourcePath != "report.md" || generatedState.Transform != transform.MarkdownToHTML || generatedState.URL != generatedOutput.URL || generatedState.SHA256 != generatedOutput.SHA256 || generatedState.Size != generatedOutput.Size {
|
||||
t.Fatalf("generated state output = %#v", generatedState)
|
||||
}
|
||||
}
|
||||
|
||||
func TestOutputSliceProjections(t *testing.T) {
|
||||
outputs := []Output{
|
||||
{SourcePath: "report.md", DestinationPath: "report.md", Kind: state.OutputKindSource, SHA256: "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", Size: 1},
|
||||
{SourcePath: "report.md", DestinationPath: "report.html", Kind: state.OutputKindGenerated, Transform: transform.MarkdownToHTML, URL: "https://reports.example.com/report.html", SHA256: "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", Size: 2},
|
||||
}
|
||||
|
||||
stateOutputs := StateOutputFiles(outputs)
|
||||
if len(stateOutputs) != 2 || stateOutputs[1].Path != "report.html" || stateOutputs[1].Transform != transform.MarkdownToHTML || stateOutputs[1].URL != outputs[1].URL {
|
||||
t.Fatalf("state outputs = %#v", stateOutputs)
|
||||
}
|
||||
|
||||
paths := ManagedOutputPaths(outputs)
|
||||
if len(paths) != 2 || paths[0] != "report.md" || paths[1] != "report.html" {
|
||||
t.Fatalf("managed paths = %#v", paths)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPlanOutputsRejectsCollision(t *testing.T) {
|
||||
sourceBackend := fake.New()
|
||||
sourceBundle := testutil.WriteFakeSourceBundle(t, sourceBackend, "", testutil.BundleOptions{
|
||||
|
||||
@@ -179,6 +179,23 @@ func TestParseRejectsInvalidOutputMetadata(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateReportsURLFieldContext(t *testing.T) {
|
||||
source := validManifest(t)
|
||||
state := *withState(t, source, func(s *DistributorState) {
|
||||
s.Links = &LinkState{PrimaryURL: "https://reports.example.com/archive#top"}
|
||||
})
|
||||
err := Validate(state)
|
||||
assertStateErrorContains(t, err, "state links.primary_url")
|
||||
assertStateErrorContains(t, err, "must not include a fragment")
|
||||
|
||||
state = *withState(t, source, func(s *DistributorState) {
|
||||
s.Outputs[0].URL = "https://reports.example.com/archive?preview=1"
|
||||
})
|
||||
err = Validate(state)
|
||||
assertStateErrorContains(t, err, "state outputs[0].url")
|
||||
assertStateErrorContains(t, err, "must not include a query string")
|
||||
}
|
||||
|
||||
func TestParseRejectsMalformedPublishedTimestamp(t *testing.T) {
|
||||
body := strings.Replace(validStateJSON(t), `"published_at": "2026-05-30T11:12:00Z"`, `"published_at": "May 30"`, 1)
|
||||
_, err := Parse([]byte(body))
|
||||
|
||||
@@ -2,9 +2,9 @@ package state
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net/url"
|
||||
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/bundle"
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/link"
|
||||
"gitea.maximumdirect.net/eric/distributor/internal/storage"
|
||||
)
|
||||
|
||||
@@ -30,7 +30,7 @@ func Validate(s DistributorState) error {
|
||||
return fmt.Errorf("state source.manifest: %w", err)
|
||||
}
|
||||
if s.Links != nil && s.Links.PrimaryURL != "" {
|
||||
if err := validateStateURL(s.Links.PrimaryURL); err != nil {
|
||||
if err := link.ValidateHTTPURL(s.Links.PrimaryURL); err != nil {
|
||||
return fmt.Errorf("state links.primary_url: %w", err)
|
||||
}
|
||||
}
|
||||
@@ -70,7 +70,7 @@ func validateOutput(index int, output OutputFile) error {
|
||||
return fmt.Errorf("state outputs[%d].transform is required for generated output", index)
|
||||
}
|
||||
if output.URL != "" {
|
||||
if err := validateStateURL(output.URL); err != nil {
|
||||
if err := link.ValidateHTTPURL(output.URL); err != nil {
|
||||
return fmt.Errorf("state outputs[%d].url: %w", index, err)
|
||||
}
|
||||
}
|
||||
@@ -82,23 +82,3 @@ func validateOutput(index int, output OutputFile) error {
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func validateStateURL(value string) error {
|
||||
parsed, err := url.Parse(value)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if parsed.Scheme != "http" && parsed.Scheme != "https" {
|
||||
return fmt.Errorf("must use http or https")
|
||||
}
|
||||
if parsed.Host == "" {
|
||||
return fmt.Errorf("must include a host")
|
||||
}
|
||||
if parsed.RawQuery != "" {
|
||||
return fmt.Errorf("must not include a query string")
|
||||
}
|
||||
if parsed.Fragment != "" {
|
||||
return fmt.Errorf("must not include a fragment")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -3,7 +3,6 @@ package fake
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"io"
|
||||
"sort"
|
||||
"strings"
|
||||
@@ -133,14 +132,14 @@ func (b *Backend) Walk(ctx context.Context, prefix string, opts storage.WalkOpti
|
||||
if err := storage.ValidatePrefix(prefix); err != nil {
|
||||
return err
|
||||
}
|
||||
emitter := storage.NewWalkEmitter(ctx, backendName, opts, fn)
|
||||
if entry, err := b.Stat(ctx, prefix); err == nil && entry.Type != storage.EntryTypeDirectory {
|
||||
return emit(ctx, entry, opts, fn)
|
||||
return storage.FinishWalk(emitter.Emit(entry))
|
||||
} else if err != nil && !storage.IsNotFound(err) {
|
||||
return err
|
||||
}
|
||||
|
||||
entries := b.entries()
|
||||
visited := 0
|
||||
for _, entry := range entries {
|
||||
if entry.Path == "" || !entryBelow(prefix, entry.Path) {
|
||||
continue
|
||||
@@ -148,33 +147,15 @@ func (b *Backend) Walk(ctx context.Context, prefix string, opts storage.WalkOpti
|
||||
if !opts.Recursive && !isImmediateChild(prefix, entry.Path) {
|
||||
continue
|
||||
}
|
||||
if opts.Limit > 0 && visited >= opts.Limit {
|
||||
return nil
|
||||
}
|
||||
visited++
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := fn(entry); err != nil {
|
||||
if errors.Is(err, storage.ErrStopWalk) {
|
||||
return nil
|
||||
}
|
||||
return storage.NewError(storage.OpWalk, backendName, entry.Path, storage.ErrUnknown, err)
|
||||
if err := emitter.Emit(entry); err != nil {
|
||||
return storage.FinishWalk(err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (b *Backend) HasAny(ctx context.Context, prefix string) (bool, error) {
|
||||
found := false
|
||||
err := b.Walk(ctx, prefix, storage.WalkOptions{Recursive: false, Limit: 1}, func(storage.Entry) error {
|
||||
found = true
|
||||
return storage.ErrStopWalk
|
||||
})
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
return found, nil
|
||||
return storage.HasAny(ctx, b, prefix)
|
||||
}
|
||||
|
||||
func (b *Backend) DeleteManagedBundle(ctx context.Context, bundlePath string, managedOutputPaths []string, opts storage.DeleteOptions) error {
|
||||
@@ -302,19 +283,6 @@ func (b *Backend) entries() []storage.Entry {
|
||||
return entries
|
||||
}
|
||||
|
||||
func emit(ctx context.Context, entry storage.Entry, opts storage.WalkOptions, fn storage.WalkFunc) error {
|
||||
if opts.Limit > 0 && opts.Limit < 1 {
|
||||
return nil
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := fn(entry); err != nil && !errors.Is(err, storage.ErrStopWalk) {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func entryBelow(prefix, path string) bool {
|
||||
if prefix == "" {
|
||||
return path != ""
|
||||
|
||||
63
internal/storage/walk.go
Normal file
63
internal/storage/walk.go
Normal file
@@ -0,0 +1,63 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
)
|
||||
|
||||
type WalkEmitter struct {
|
||||
ctx context.Context
|
||||
backend string
|
||||
opts WalkOptions
|
||||
fn WalkFunc
|
||||
count int
|
||||
}
|
||||
|
||||
func NewWalkEmitter(ctx context.Context, backend string, opts WalkOptions, fn WalkFunc) *WalkEmitter {
|
||||
return &WalkEmitter{
|
||||
ctx: ctx,
|
||||
backend: backend,
|
||||
opts: opts,
|
||||
fn: fn,
|
||||
}
|
||||
}
|
||||
|
||||
func (e *WalkEmitter) Emit(entry Entry) error {
|
||||
if err := e.ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
if e.opts.Limit > 0 && e.count >= e.opts.Limit {
|
||||
return ErrStopWalk
|
||||
}
|
||||
e.count++
|
||||
if err := e.fn(entry); err != nil {
|
||||
if errors.Is(err, ErrStopWalk) {
|
||||
return ErrStopWalk
|
||||
}
|
||||
return NewError(OpWalk, e.backend, entry.Path, ErrUnknown, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (e *WalkEmitter) LimitReached() bool {
|
||||
return e.opts.Limit > 0 && e.count >= e.opts.Limit
|
||||
}
|
||||
|
||||
func FinishWalk(err error) error {
|
||||
if errors.Is(err, ErrStopWalk) {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func HasAny(ctx context.Context, backend Backend, prefix string) (bool, error) {
|
||||
found := false
|
||||
err := backend.Walk(ctx, prefix, WalkOptions{Recursive: false, Limit: 1}, func(Entry) error {
|
||||
found = true
|
||||
return ErrStopWalk
|
||||
})
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
return found, nil
|
||||
}
|
||||
107
internal/storage/walk_test.go
Normal file
107
internal/storage/walk_test.go
Normal file
@@ -0,0 +1,107 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestWalkEmitterHonorsLimit(t *testing.T) {
|
||||
emitter := NewWalkEmitter(context.Background(), "test", WalkOptions{Limit: 2}, func(Entry) error {
|
||||
return nil
|
||||
})
|
||||
|
||||
if err := emitter.Emit(Entry{Path: "one"}); err != nil {
|
||||
t.Fatalf("first Emit() error = %v", err)
|
||||
}
|
||||
if err := emitter.Emit(Entry{Path: "two"}); err != nil {
|
||||
t.Fatalf("second Emit() error = %v", err)
|
||||
}
|
||||
if err := emitter.Emit(Entry{Path: "three"}); !errors.Is(err, ErrStopWalk) {
|
||||
t.Fatalf("third Emit() error = %v, want ErrStopWalk", err)
|
||||
}
|
||||
if !emitter.LimitReached() {
|
||||
t.Fatal("LimitReached() = false, want true")
|
||||
}
|
||||
}
|
||||
|
||||
func TestWalkEmitterStopsWithoutError(t *testing.T) {
|
||||
emitter := NewWalkEmitter(context.Background(), "test", WalkOptions{}, func(Entry) error {
|
||||
return ErrStopWalk
|
||||
})
|
||||
|
||||
err := emitter.Emit(Entry{Path: "one"})
|
||||
if !errors.Is(err, ErrStopWalk) {
|
||||
t.Fatalf("Emit() error = %v, want ErrStopWalk", err)
|
||||
}
|
||||
if err := FinishWalk(err); err != nil {
|
||||
t.Fatalf("FinishWalk() error = %v, want nil", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWalkEmitterWrapsCallbackErrors(t *testing.T) {
|
||||
callbackErr := errors.New("callback failed")
|
||||
emitter := NewWalkEmitter(context.Background(), "test", WalkOptions{}, func(Entry) error {
|
||||
return callbackErr
|
||||
})
|
||||
|
||||
err := emitter.Emit(Entry{Path: "one"})
|
||||
if !errors.Is(err, callbackErr) {
|
||||
t.Fatalf("Emit() error = %v, want callback error", err)
|
||||
}
|
||||
var storageErr *Error
|
||||
if !errors.As(err, &storageErr) {
|
||||
t.Fatalf("Emit() error type = %T, want *Error", err)
|
||||
}
|
||||
if storageErr.Op != OpWalk || storageErr.Backend != "test" || storageErr.Path != "one" || storageErr.Kind != ErrUnknown {
|
||||
t.Fatalf("wrapped error = %#v", storageErr)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWalkEmitterHonorsContextCancellation(t *testing.T) {
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
cancel()
|
||||
called := false
|
||||
emitter := NewWalkEmitter(ctx, "test", WalkOptions{}, func(Entry) error {
|
||||
called = true
|
||||
return nil
|
||||
})
|
||||
|
||||
err := emitter.Emit(Entry{Path: "one"})
|
||||
if !errors.Is(err, context.Canceled) {
|
||||
t.Fatalf("Emit() error = %v, want context.Canceled", err)
|
||||
}
|
||||
if called {
|
||||
t.Fatal("callback was called after context cancellation")
|
||||
}
|
||||
}
|
||||
|
||||
func TestHasAnyUsesNonRecursiveLimitOneWalk(t *testing.T) {
|
||||
backend := &recordingBackend{}
|
||||
|
||||
found, err := HasAny(context.Background(), backend, "bundle")
|
||||
if err != nil {
|
||||
t.Fatalf("HasAny() error = %v", err)
|
||||
}
|
||||
if !found {
|
||||
t.Fatal("HasAny() = false, want true")
|
||||
}
|
||||
if backend.prefix != "bundle" {
|
||||
t.Fatalf("walk prefix = %q, want bundle", backend.prefix)
|
||||
}
|
||||
if backend.opts != (WalkOptions{Recursive: false, Limit: 1}) {
|
||||
t.Fatalf("walk options = %#v, want non-recursive limit one", backend.opts)
|
||||
}
|
||||
}
|
||||
|
||||
type recordingBackend struct {
|
||||
Backend
|
||||
prefix string
|
||||
opts WalkOptions
|
||||
}
|
||||
|
||||
func (b *recordingBackend) Walk(_ context.Context, prefix string, opts WalkOptions, fn WalkFunc) error {
|
||||
b.prefix = prefix
|
||||
b.opts = opts
|
||||
return FinishWalk(fn(Entry{Path: "bundle/file.txt", Type: EntryTypeFile}))
|
||||
}
|
||||
@@ -3,6 +3,7 @@ package testutil
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
@@ -106,6 +107,53 @@ func WriteFakeSourceBundle(t testing.TB, backend *fake.Backend, relative string,
|
||||
return bundle.Bundle{RootRelativePath: relative, Manifest: manifest}
|
||||
}
|
||||
|
||||
func WriteFakeFile(t testing.TB, backend *fake.Backend, path, data string) {
|
||||
t.Helper()
|
||||
if _, err := backend.WriteFile(context.Background(), path, []byte(data), storage.WriteOptions{}); err != nil {
|
||||
t.Fatalf("write fake file %s: %v", path, err)
|
||||
}
|
||||
}
|
||||
|
||||
func AssertFakeFile(t testing.TB, backend *fake.Backend, path, want string) {
|
||||
t.Helper()
|
||||
data, err := backend.ReadFile(context.Background(), path)
|
||||
if err != nil {
|
||||
t.Fatalf("read fake file %s: %v", path, err)
|
||||
}
|
||||
if got := string(data); got != want {
|
||||
t.Fatalf("fake file %s = %q, want %q", path, got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func AssertFakeMissing(t testing.TB, backend *fake.Backend, path string) {
|
||||
t.Helper()
|
||||
if _, err := backend.Stat(context.Background(), path); !storage.IsNotFound(err) {
|
||||
t.Fatalf("fake file %s stat error = %v, want not found", path, err)
|
||||
}
|
||||
}
|
||||
|
||||
func WriteFakeDestinationState(t testing.TB, backend *fake.Backend, relative string, manifest bundle.Manifest, opts DestinationStateOptions) state.DistributorState {
|
||||
t.Helper()
|
||||
destinationState := DestinationState(manifest, opts)
|
||||
data, err := json.MarshalIndent(destinationState, "", " ")
|
||||
if err != nil {
|
||||
t.Fatalf("marshal destination state: %v", err)
|
||||
}
|
||||
statePath, err := storage.StatePath(relative)
|
||||
if err != nil {
|
||||
t.Fatalf("state path: %v", err)
|
||||
}
|
||||
WriteFakeFile(t, backend, statePath, string(append(data, '\n')))
|
||||
for _, output := range destinationState.Outputs {
|
||||
path, err := storage.Join(relative, output.Path)
|
||||
if err != nil {
|
||||
t.Fatalf("join output path: %v", err)
|
||||
}
|
||||
WriteFakeFile(t, backend, path, "old")
|
||||
}
|
||||
return destinationState
|
||||
}
|
||||
|
||||
func WriteMinimalLocalConfig(t testing.TB, sourceRoot, destinationRoot string) string {
|
||||
t.Helper()
|
||||
return writeConfigFile(t, `
|
||||
@@ -139,6 +187,136 @@ pipelines:
|
||||
`)
|
||||
}
|
||||
|
||||
func WriteLocalConfigWithPublishPolicy(t testing.TB, sourceRoot, destinationRoot string, publishSource, publishHTML bool) string {
|
||||
t.Helper()
|
||||
transformConfig := ""
|
||||
if publishHTML {
|
||||
transformConfig = `
|
||||
transform:
|
||||
markdown_to_html:
|
||||
enabled: true
|
||||
mode: sidecar`
|
||||
}
|
||||
return writeConfigFile(t, `
|
||||
pipelines:
|
||||
- id: reports
|
||||
source:
|
||||
backend: local
|
||||
path: `+sourceRoot+`
|
||||
destinations:
|
||||
- id: archive
|
||||
backend: local
|
||||
path: `+destinationRoot+`
|
||||
publish:
|
||||
source: `+fmt.Sprintf("%t", publishSource)+`
|
||||
html: `+fmt.Sprintf("%t", publishHTML)+transformConfig+`
|
||||
`)
|
||||
}
|
||||
|
||||
func WriteLocalConfigWithPathMapping(t testing.TB, sourceRoot, destinationRoot, mode string) string {
|
||||
t.Helper()
|
||||
return writeConfigFile(t, `
|
||||
pipelines:
|
||||
- id: reports
|
||||
source:
|
||||
backend: local
|
||||
path: `+sourceRoot+`
|
||||
destinations:
|
||||
- id: archive
|
||||
backend: local
|
||||
path: `+destinationRoot+`
|
||||
path_mapping:
|
||||
mode: `+mode+`
|
||||
`)
|
||||
}
|
||||
|
||||
func WriteLocalConfigWithLinks(t testing.TB, sourceRoot, destinationRoot, pathMapping, baseURL, primary string, publishSource, publishHTML bool, transformMode string) string {
|
||||
t.Helper()
|
||||
transformConfig := ""
|
||||
if publishHTML {
|
||||
transformConfig = `
|
||||
transform:
|
||||
markdown_to_html:
|
||||
enabled: true
|
||||
mode: ` + transformMode
|
||||
}
|
||||
return writeConfigFile(t, `
|
||||
pipelines:
|
||||
- id: reports
|
||||
source:
|
||||
backend: local
|
||||
path: `+sourceRoot+`
|
||||
destinations:
|
||||
- id: archive
|
||||
backend: local
|
||||
path: `+destinationRoot+`
|
||||
path_mapping:
|
||||
mode: `+pathMapping+`
|
||||
links:
|
||||
base_url: `+baseURL+`
|
||||
primary: `+primary+`
|
||||
publish:
|
||||
source: `+fmt.Sprintf("%t", publishSource)+`
|
||||
html: `+fmt.Sprintf("%t", publishHTML)+transformConfig+`
|
||||
`)
|
||||
}
|
||||
|
||||
func WriteLocalConfigWithMarkdownTransform(t testing.TB, sourceRoot, destinationRoot string, publishSource, publishHTML bool, mode, input string) string {
|
||||
t.Helper()
|
||||
enabled := publishHTML
|
||||
inputConfig := ""
|
||||
if input != "" {
|
||||
inputConfig = `
|
||||
input: ` + input
|
||||
}
|
||||
return writeConfigFile(t, `
|
||||
pipelines:
|
||||
- id: reports
|
||||
source:
|
||||
backend: local
|
||||
path: `+sourceRoot+`
|
||||
destinations:
|
||||
- id: archive
|
||||
backend: local
|
||||
path: `+destinationRoot+`
|
||||
publish:
|
||||
source: `+fmt.Sprintf("%t", publishSource)+`
|
||||
html: `+fmt.Sprintf("%t", publishHTML)+`
|
||||
transform:
|
||||
markdown_to_html:
|
||||
enabled: `+fmt.Sprintf("%t", enabled)+`
|
||||
mode: `+mode+inputConfig+`
|
||||
`)
|
||||
}
|
||||
|
||||
func WriteMixedPolicyFanoutLocalConfig(t testing.TB, sourceRoot, archiveDestination, htmlDestination string) string {
|
||||
t.Helper()
|
||||
return writeConfigFile(t, `
|
||||
pipelines:
|
||||
- id: reports
|
||||
source:
|
||||
backend: local
|
||||
path: `+sourceRoot+`
|
||||
destinations:
|
||||
- id: archive
|
||||
backend: local
|
||||
path: `+archiveDestination+`
|
||||
publish:
|
||||
source: true
|
||||
html: false
|
||||
- id: html
|
||||
backend: local
|
||||
path: `+htmlDestination+`
|
||||
publish:
|
||||
source: false
|
||||
html: true
|
||||
transform:
|
||||
markdown_to_html:
|
||||
enabled: true
|
||||
mode: sidecar
|
||||
`)
|
||||
}
|
||||
|
||||
func WriteDestinationState(t testing.TB, root, relative string, manifest bundle.Manifest, opts DestinationStateOptions) state.DistributorState {
|
||||
t.Helper()
|
||||
bundleRoot := filepath.Join(root, filepath.FromSlash(relative))
|
||||
@@ -175,6 +353,28 @@ func ReadDestinationState(t testing.TB, path string) state.DistributorState {
|
||||
return destinationState
|
||||
}
|
||||
|
||||
func AssertFile(t testing.TB, path, want string) {
|
||||
t.Helper()
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatalf("read file %s: %v", path, err)
|
||||
}
|
||||
if got := string(data); got != want {
|
||||
t.Fatalf("%s = %q, want %q", path, got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func AssertFileContains(t testing.TB, path, want string) {
|
||||
t.Helper()
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatalf("read file %s: %v", path, err)
|
||||
}
|
||||
if !strings.Contains(string(data), want) {
|
||||
t.Fatalf("%s = %q, want substring %q", path, data, want)
|
||||
}
|
||||
}
|
||||
|
||||
func sourceFiles(opts BundleOptions) []SourceFile {
|
||||
files := opts.Files
|
||||
if files == nil {
|
||||
|
||||
@@ -11,7 +11,8 @@ import (
|
||||
|
||||
var digestPattern = regexp.MustCompile(`^sha256:[0-9a-f]{64}$`)
|
||||
|
||||
func validateDigest(value string) error {
|
||||
// ValidateDigest reports whether value uses the lowercase sha256:<64 hex> form.
|
||||
func ValidateDigest(value string) error {
|
||||
if !digestPattern.MatchString(value) {
|
||||
return fmt.Errorf("must be lowercase sha256:<64 hex>")
|
||||
}
|
||||
|
||||
58
pkg/bundle/digest_test.go
Normal file
58
pkg/bundle/digest_test.go
Normal file
@@ -0,0 +1,58 @@
|
||||
package bundle
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestValidateDigest(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
value string
|
||||
wantErr bool
|
||||
}{
|
||||
{
|
||||
name: "valid",
|
||||
value: "sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
|
||||
},
|
||||
{
|
||||
name: "uppercase hex",
|
||||
value: "sha256:0123456789ABCDEF0123456789abcdef0123456789abcdef0123456789abcdef",
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "missing prefix",
|
||||
value: "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "wrong algorithm",
|
||||
value: "sha512:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "short hex",
|
||||
value: "sha256:0123456789abcdef",
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "long hex",
|
||||
value: "sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0",
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "non hex",
|
||||
value: "sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdeg",
|
||||
wantErr: true,
|
||||
},
|
||||
}
|
||||
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
err := ValidateDigest(test.value)
|
||||
if test.wantErr && err == nil {
|
||||
t.Fatal("ValidateDigest() error = nil, want error")
|
||||
}
|
||||
if !test.wantErr && err != nil {
|
||||
t.Fatalf("ValidateDigest() error = %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -13,7 +13,7 @@ func ValidateManifest(manifest Manifest) error {
|
||||
if manifest.ID == "" {
|
||||
return fmt.Errorf("id is required")
|
||||
}
|
||||
if err := validateDigest(manifest.Digest); err != nil {
|
||||
if err := ValidateDigest(manifest.Digest); err != nil {
|
||||
return fmt.Errorf("digest: %w", err)
|
||||
}
|
||||
if manifest.Created.IsZero() {
|
||||
@@ -27,7 +27,7 @@ func ValidateManifest(manifest Manifest) error {
|
||||
if err := ValidateSourcePath(file.Path); err != nil {
|
||||
return fmt.Errorf("files[%d].path: %w", index, err)
|
||||
}
|
||||
if err := validateDigest(file.SHA256); err != nil {
|
||||
if err := ValidateDigest(file.SHA256); err != nil {
|
||||
return fmt.Errorf("files[%d].sha256: %w", index, err)
|
||||
}
|
||||
if file.Size < 0 {
|
||||
|
||||
Reference in New Issue
Block a user