Problem
The sink enqueues individual log entries into a single buffered channel and spawns a goroutine per flush. This makes throughput sensitive to burst patterns and offers limited visibility into queue depth or backpressure.
Proposed optimization
Introduce a bounded work queue + worker pool that batches by size/time with explicit queue depth metrics. This provides stable throughput under load and makes backpressure behavior observable.
Notes / references
Acceptance criteria
- Sink uses a bounded queue with worker pool and batcher.
- Metrics expose queue depth and batch flush latency.
- Throughput remains stable under bursty event streams.
Problem
The sink enqueues individual log entries into a single buffered channel and spawns a goroutine per flush. This makes throughput sensitive to burst patterns and offers limited visibility into queue depth or backpressure.
Proposed optimization
Introduce a bounded work queue + worker pool that batches by size/time with explicit queue depth metrics. This provides stable throughput under load and makes backpressure behavior observable.
Notes / references
Acceptance criteria