Introducing Glommio, a thread-per-core crate for Rust and Linux
Thread-per-core architecture can significantly improve performance and reduce cloud costs by avoiding lock contention and expensive context switches. However, adopting it directly can reduce developer productivity because it requires new programming patterns and careful data ownership. Datadog developed Glommio, a Rust framework intended to make thread-per-core applications easier to build and maintain. ## Why Traditional Threading Has Limits - Applications commonly use multiple threads to perform independent tasks in parallel. - Shared data requires locks, which introduce contention and waiting. - Thread context switches can cost around five microseconds—potentially more than modern storage I/O operations using technologies such as `io_uring`. - Asynchronous programming reduces blocking, but many runtimes still rely on thread pools or separate worker threads for operations such as file I/O. ## How Thread-per-Core Works - Each CPU core runs a single application thread, often pinned to that core. - Because the operating system does not move the thread between cores, ordinary thread context switches are eliminated. - Hardware interrupts and auxiliary tasks can still interrupt execution. - For maximum performance, operators may reserve certain CPUs for interrupts and system services rather than application work. ## Sharding Data Across Cores - Thread-per-core applications depend on sharding: each thread owns a distinct subset of the data or requests. - Examples include assigning Kafka partitions or database key ranges to individual threads. - Requests assigned to one thread execute there to completion unless the code explicitly yields. - This ownership model prevents multiple threads from handling the same request or data simultaneously. ## Eliminating Locks - Since one thread processes a shard at a time, operations on that shard are naturally serialized. - A conventional threaded cache requires locks because multiple threads may update the same data concurrently. - Sharding reduces contention by dividing a large cache into smaller sections, but locks may still be needed if the operating system switches between threads. - With thread-per-core, updates to keys in the same shard occur sequentially, so an update can complete without acquiring a lock. ## Glommio and Existing Precedents - Thread-per-core is not a new concept; the author previously worked with Seastar, a C++ framework used by ScyllaDB. - Datadog’s Glommio brings the model to Rust while aiming to make its programming challenges more manageable. - The framework is motivated by the need to preserve developer productivity while achieving the efficiency gains of thread-per-core systems. Thread-per-core is most suitable for highly parallel, high-throughput workloads with naturally shardable data. Its performance benefits depend on disciplined data ownership and cooperative execution, while frameworks such as Glommio can reduce the complexity of adopting the model.
Read original(opens in new tab)