Update documentation for SPC outlook support
All checks were successful
ci/woodpecker/push/build-image Pipeline was successful
All checks were successful
ci/woodpecker/push/build-image Pipeline was successful
This commit is contained in:
@@ -64,16 +64,20 @@ and before each interval tick. If no jitter is configured in code, feedkit uses
|
||||
Poll failures are logged and do not stop the daemon. A failed poll emits no
|
||||
events for that source until a subsequent poll succeeds.
|
||||
|
||||
## Conditional HTTP Fetches
|
||||
## Unchanged Upstream Content
|
||||
|
||||
All current sources use feedkit's HTTP polling helper. By default,
|
||||
`params.conditional` is `true`, so the helper keeps ETag and Last-Modified
|
||||
validators in memory for each source instance.
|
||||
Most sources use feedkit's single-document HTTP polling helper. By default,
|
||||
`params.conditional` is `true` for those sources, so the helper keeps ETag and
|
||||
Last-Modified validators in memory for each source instance.
|
||||
|
||||
If the upstream returns `304 Not Modified`, the source emits no events for that
|
||||
poll. Validator state is in memory only; restarting the process starts with no
|
||||
cached validators.
|
||||
|
||||
The SPC convective outlook source polls multiple required documents as one
|
||||
bundle. It emits no event when a later complete bundle has the same fetched
|
||||
document bodies as the previous successful poll.
|
||||
|
||||
## Processing And Dedupe
|
||||
|
||||
Every event passes through normalization first and dedupe second.
|
||||
@@ -128,6 +132,7 @@ table in that same transaction. See the
|
||||
|
||||
- scheduler goroutines and timers;
|
||||
- HTTP conditional request validators;
|
||||
- source-local unchanged-content state;
|
||||
- event channel buffers;
|
||||
- per-sink fanout queues;
|
||||
- the dedupe ID set.
|
||||
|
||||
Reference in New Issue
Block a user