208 lines
9.2 KiB
Markdown
208 lines
9.2 KiB
Markdown
# Development Policy
|
|
|
|
This document defines the day-to-day development workflow for `distributor`.
|
|
Use it with `docs/policy/architecture.md` and `docs/policy/documentation.md`.
|
|
|
|
## Repository Layout
|
|
|
|
- `cmd/distributor`: executable entrypoint only.
|
|
- `internal/app`: top-level use cases for `run`, `validate`, and `inspect`.
|
|
- `internal/cli`: standard-library command parsing, flags, help text, and command wiring.
|
|
- `internal/config`: YAML configuration structs, loading, defaults, and validation.
|
|
- `internal/bundle`: source bundle discovery, manifest parsing, digest calculation, and validation.
|
|
- `internal/state`: destination `.distributor.json` parsing, validation, and comparison.
|
|
- `internal/storage`: backend interface, registry, logical path rules, typed errors, and shared storage helpers.
|
|
- `internal/adapters/local`: local filesystem backend.
|
|
- `internal/adapters/ssh`: SSH/SFTP backend.
|
|
- `internal/adapters/s3`: S3-compatible object storage backend.
|
|
- `internal/storage/fake`: in-memory backend for tests.
|
|
- `internal/publish`: destination inspection, output planning, reconciliation, execution, and managed cleanup.
|
|
- `internal/transform`: transform interface and registry.
|
|
- `internal/transform/markdown`: Markdown-to-HTML sidecar transform.
|
|
- `internal/notify`: notification interface and current no-op notifier.
|
|
- `internal/testutil`: shared test fixtures. Production code must not import this package.
|
|
- `docs`: current user, operator, policy, internal, and roadmap documentation.
|
|
- `examples`: copyable example configs and source bundles.
|
|
|
|
Do not create new top-level package families such as `pkg`, `internal/stage`,
|
|
`internal/modules`, or service-specific adapter directories unless the
|
|
architecture policy or a current roadmap explicitly calls for them.
|
|
|
|
## Common Commands
|
|
|
|
Run the full test suite:
|
|
|
|
```bash
|
|
go test ./...
|
|
```
|
|
|
|
Run targeted packages while developing:
|
|
|
|
```bash
|
|
go test ./internal/config
|
|
go test ./internal/cli ./internal/app
|
|
go test ./internal/publish ./internal/state
|
|
go test ./internal/transform/markdown
|
|
```
|
|
|
|
Run the CLI against an example config:
|
|
|
|
```bash
|
|
go run ./cmd/distributor run --config examples/local-publish.yml --dry-run
|
|
```
|
|
|
|
Validate or inspect a local source bundle:
|
|
|
|
```bash
|
|
go run ./cmd/distributor validate examples/source-bundle
|
|
go run ./cmd/distributor inspect examples/source-bundle
|
|
```
|
|
|
|
If Go cache permissions fail in a restricted environment, use workspace-safe
|
|
temporary caches:
|
|
|
|
```bash
|
|
GOCACHE=/private/tmp/distributor-gocache GOMODCACHE=/private/tmp/distributor-gomodcache go test ./...
|
|
```
|
|
|
|
## Coding Conventions
|
|
|
|
- Keep the application small, explicit, and dependency-light.
|
|
- Prefer package-local helpers over broad abstractions until behavior is shared by multiple packages.
|
|
- Keep CLI parsing in `internal/cli`; business decisions belong in `internal/app`, `internal/bundle`, `internal/publish`, `internal/state`, and related core packages.
|
|
- Keep adapter packages thin. Backend-specific filesystem or service behavior belongs in adapters; bundle, state, transform, and publish policy belongs outside adapters.
|
|
- Preserve public CLI behavior, config semantics, manifest schema, destination state schema, and local MVP behavior unless the current task explicitly changes them.
|
|
- Use `storage.DisplayPath`, `storage.StateFileName`, `storage.StatePath`, and `storage.ManagedBundleTargets` instead of duplicating those conventions.
|
|
- Use `bundle.ValidateManifest` for normalized source manifest semantics, including embedded source manifests in destination state.
|
|
- Use `config.ValidatePublishTransformPolicy` for publish and transform policy combinations.
|
|
- Do not import concrete transform implementations from `internal/publish`; app-level wiring owns transform registration.
|
|
- Do not import `internal/testutil` from production code.
|
|
|
|
## Dependency Policy
|
|
|
|
The project currently depends on:
|
|
|
|
- `gopkg.in/yaml.v3` for YAML configuration loading.
|
|
- `github.com/yuin/goldmark` for Markdown rendering.
|
|
- `golang.org/x/crypto/ssh`, `golang.org/x/crypto/ssh/agent`, and `golang.org/x/crypto/ssh/knownhosts` for native SSH support.
|
|
- `github.com/pkg/sftp` for native SFTP support.
|
|
- `github.com/aws/aws-sdk-go-v2/...` packages for S3-compatible storage support.
|
|
|
|
Add external dependencies only when they materially improve correctness,
|
|
security, interoperability, or implementation complexity. Avoid dependencies
|
|
for small conveniences. Do not let dependency-specific types leak across
|
|
internal package boundaries unless that dependency is the explicit package
|
|
contract.
|
|
|
|
## Configuration Changes
|
|
|
|
When adding or changing configuration:
|
|
|
|
1. Update `internal/config/config.go` structs and YAML tags.
|
|
2. Add defaults in `internal/config/defaults.go` only for built-in defaults.
|
|
3. Add validation in `internal/config/validate.go` with clear field context.
|
|
4. Update config load and validation tests.
|
|
5. Update `docs/config.md` in the same change if current user-visible config behavior changes.
|
|
6. Update examples only with configs that are valid and executable for implemented behavior.
|
|
|
|
Config validation may accept fields for backends that are not executable yet,
|
|
but user-facing docs and examples must clearly state execution support. At the
|
|
time of this policy, local, SSH, and S3 backends are executable.
|
|
|
|
Credential-consuming code must use the config-owned environment resolver for
|
|
explicit credential environment variable references. Do not call `os.Getenv`
|
|
directly for backend credentials, because `secrets.directory` values are
|
|
intentionally available through the resolver without mutating the process
|
|
environment.
|
|
|
|
## CLI Changes
|
|
|
|
The CLI is hand-written with the Go standard library. Do not introduce a CLI
|
|
framework without a documented reason.
|
|
|
|
When adding or changing commands or flags:
|
|
|
|
1. Keep parsing and help text in `internal/cli`.
|
|
2. Keep command work in `internal/app` or a lower-level package.
|
|
3. Add or update CLI tests in `internal/cli`.
|
|
4. Update `docs/cli.md` if syntax, flags, output expectations, or workflows change.
|
|
|
|
`validate` and `inspect` are local path commands. `run` loads configured
|
|
pipelines and currently executes local, SSH, and S3 backends.
|
|
|
|
## Storage Backends
|
|
|
|
Storage behavior is defined by `internal/storage.Backend` and shared path rules
|
|
in `internal/storage`.
|
|
|
|
When adding a backend:
|
|
|
|
1. Implement the storage interface in an adapter package.
|
|
2. Translate backend-specific errors into storage errors where practical.
|
|
3. Keep bundle comparison, transform, routing, and replacement policy out of the adapter.
|
|
4. Register runtime construction through app-level backend factory wiring.
|
|
5. Add focused adapter tests and app-level wiring tests.
|
|
6. Update user docs, operations docs, examples, and internal docs only for behavior that is actually implemented.
|
|
|
|
Do not document future backend execution as available until the corresponding
|
|
adapter package and app wiring exist.
|
|
|
|
## Transforms
|
|
|
|
Transforms use `internal/transform` interfaces and registry wiring.
|
|
|
|
When adding or changing a transform:
|
|
|
|
1. Keep the transform implementation in its own package under `internal/transform`.
|
|
2. Register default runtime transforms from `internal/app`.
|
|
3. Keep `internal/publish` dependent only on the transform interface or resolver.
|
|
4. Record deterministic output metadata: path, source path, transform name, digest, and size.
|
|
5. Add transform tests and app or publish tests for wiring and policy behavior.
|
|
6. Update `docs/internal/transform.md` and any relevant integration docs for implemented behavior.
|
|
|
|
## Tests
|
|
|
|
Test close to the behavior being changed:
|
|
|
|
- Use package tests for parsing, validation, comparison, planning, and adapter behavior.
|
|
- Use `internal/app` and `internal/cli` tests for user-facing workflows.
|
|
- Use `internal/testutil` for shared valid fixtures only; keep edge cases near the package under test.
|
|
- Run `go test ./...` after cross-package changes or documentation/example changes tied to tests.
|
|
|
|
Live integration tests must be opt-in and skipped during normal `go test ./...`
|
|
unless their required environment variables are set. Test-only environment
|
|
variables must use this prefix shape:
|
|
|
|
```text
|
|
DISTRIBUTOR_TEST_<BACKEND>_*
|
|
```
|
|
|
|
Examples include `DISTRIBUTOR_TEST_SSH_HOST` and
|
|
`DISTRIBUTOR_TEST_S3_ENDPOINT`. Do not use production credential variable names
|
|
for test-only controls.
|
|
|
|
## Examples
|
|
|
|
Examples under `examples/` must be valid, maintained, and free of secrets.
|
|
They should be copyable for implemented behavior. Do not leave examples that
|
|
look runnable but require unsupported backend execution.
|
|
|
|
When changing examples:
|
|
|
|
1. Keep paths relative to the repository where practical.
|
|
2. Prefer local examples until remote backend support exists.
|
|
3. Run `go test ./internal/config` because config tests load examples.
|
|
4. Update README, CLI, or config docs if links or recommended workflows change.
|
|
|
|
## Documentation
|
|
|
|
Follow `docs/policy/documentation.md`.
|
|
|
|
- Document implemented behavior outside `docs/roadmap/`.
|
|
- Keep future, planned, or aspirational behavior under `docs/roadmap/`.
|
|
- Keep `docs/config.md` canonical for user-facing config reference.
|
|
- Keep `docs/cli.md` canonical for command syntax and workflows.
|
|
- Keep `docs/operations.md` canonical for operational and recovery behavior.
|
|
- Keep `docs/internal/` focused on implemented package contracts.
|
|
- Update docs in the same change as behavior when public behavior, config, CLI, examples, or internal contracts change.
|