FairGate
Open source · MIT · beta
A TCP ingestion gateway in Go that keeps one noisy producer from starving the rest, and makes every accepted event durable before acknowledging it.
When many producers share one ingestion path, a single high-rate producer can take over buffers and bandwidth, and anything held only in memory is lost if the process crashes.
internal/sched
DRR scheduler
The scheduler visits queues in rounds. Each backlogged producer earns quantum × weight credit and is served one event per unit of credit. A producer with an empty queue has its credit reset, so it can’t bank credit and burst ahead later.
Under contention, shares follow weights.
- AShare 0%
- BShare 0%
- CShare 0%
Press play or step to run the scheduler.
- Go
- ClickHouse
- Prometheus
- Grafana
- Docker Compose
D1
Group commit instead of one fsync per write
Concurrent writers are batched into a shared fsync, so durability is paid for once per batch rather than once per event.
D2
Torn records are only tolerated at the tail
Recovery replays segments in order and rejects a torn record anywhere except the final segment, the only place an interrupted write can legitimately appear.
D3
Shipping is decoupled from scheduling
The shipper reads durable records straight from the WAL, checkpoints what it has delivered, and reclaims segments once they are fully shipped.