System Architecture & High-Scale Design

System Overview

---

100x Scale Strategy & First Bottlenecks

At 100x volume (100,000 spans/second):

1. First Bottleneck: In-memory ring buffer exhaustion in the single FastAPI worker.
   • Mitigation: Introduce Apache Kafka or Redis Streams as an external durable buffer between collectors and storage writers.
2. Second Bottleneck: Database insert latency.
   • Mitigation: Increase batch size from 5,000 to 50,000 spans, flushing every 5 seconds, utilizing ClickHouse partitioned MergeTree tables.
3. Third Bottleneck: Tail-sampling state memory.
   • Mitigation: Partition tail-sampling nodes using consistent hashing on trace_id.