refactor!: introduce generic processors registry and remove normalize registry adapter

- add new `processors` package with canonical `Processor` interface
- add `processors.Registry` with Register/Build/BuildChain factory model
- switch `pipeline.Pipeline` to `[]processors.Processor`
- replace `normalize.Registry` + registry adapter with direct `normalize.Processor`
- remove `normalize/registry.go`
- update root docs to position normalize as one optional processing stage
- add tests for processors registry, normalize processor behavior, and pipeline flow

BREAKING CHANGE:
- `pipeline.Processor` removed; use `processors.Processor`
- `normalize.Registry` and old normalize processor adapter APIs removed
- downstream daemons must update processor wiring to new `processors.Registry`
  and `normalize.NewProcessor(...)`
This commit is contained in:
2026-03-16 13:14:24 -05:00
parent 6c5f95ad26
commit 96039f6530
12 changed files with 543 additions and 162 deletions

View File

@@ -15,13 +15,12 @@ lifecycle, domain schemas, and domain-specific validation in your daemon.
## Conceptual pipeline
Collect -> Normalize (optional) -> Policy -> Route -> Emit
Collect -> Process (optional stages, including normalize) -> Route -> Emit
| Stage | Package(s) |
|---|---|
| Collect | `sources`, `scheduler` |
| Normalize | `normalize` (optional in `pipeline`) |
| Policy | `pipeline` |
| Process | `pipeline`, `processors`, `normalize` (optional stage) |
| Route | `dispatch` |
| Emit | `sinks` |
| Configure | `config` |
@@ -82,10 +81,15 @@ Runs one goroutine per source job:
Optional processing chain between collection and dispatch.
Processors can transform, drop, or reject events.
### `processors`
Defines the generic processor interface and a named-driver registry used by
daemons to build ordered processor chains.
### `normalize`
Optional normalization package (already implemented). Typical use: sources emit raw
payload events, then normalize to canonical schemas in a pipeline stage.
Concrete normalization processor implementation. Typical use: sources emit raw
payload events, then a normalize stage maps them to canonical schemas.
### `dispatch`