Stop Answering the Same Question Twice: Interval-Aware Caching for Druid at Netflix Scale
Netflix’s Druid deployment now exceeds 10 trillion rows and can ingest 15 million events per second, but repetitive dashboard queries became a scaling problem. Its new experimental caching layer handles rolling time windows by reusing settled historical results and querying Druid only for recent, changing data. Netflix accepts up to five seconds of additional staleness in exchange for substantially lower query load. ## The Scaling Problem - A dashboard with 26 charts can issue 64 queries per load. - Viewed by 30 people and refreshed every 10 seconds, that becomes roughly 192 queries per second. - Druid’s full-result cache misses whenever a rolling time interval changes. - Druid avoids caching realtime segments to preserve result correctness and determinism. - Per-segment caching reduces historical scans but still requires brokers to gather and merge results for every request. - Adding hardware to handle this redundant workload would be prohibitively expensive. ## Caching Only the Unsettled Data - In a three-hour query, most data is already stable; only the newest minutes are likely to change. - The cache stores previously returned historical portions and sends Druid only the uncached interval. - This approach is designed for time-grouped queries such as timeseries and groupBy queries. ## Deliberate Staleness - The cache can make the newest data up to five seconds stale. - This is acceptable because dashboards typically refresh every 10–30 seconds. - Netflix’s pipeline already has up to roughly five seconds of latency at P90. - Many queries also intentionally end at `now-1m` or `now-5s` to avoid unstable, newly arriving data. ## Exponential TTLs - Cache lifetimes increase with the age of each data point because older data is less likely to change. - Data under two minutes old has a minimum TTL of five seconds. - After that, TTL doubles for each additional minute: - 10 seconds at two minutes old - 20 seconds at three minutes - 40 seconds at four minutes - TTLs are capped at one hour. - Fresh data is refreshed frequently to account for late-arriving events, while older data remains cached longer. ## Time-Based Bucketing - A single cache entry per query and interval would still miss whenever a rolling window shifted. - Netflix instead uses a map-of-maps: - The outer key is a hash of the query excluding its time interval. - Inner keys represent timestamps bucketed by query granularity or at least one minute. - Big-endian timestamp encoding preserves chronological order for efficient range scans. - A three-hour query at one-minute granularity becomes 180 independently cached buckets. - When the window moves, most buckets can be reused and only the newly exposed range must be fetched. ## Router-Integrated Cache Service - The cache currently operates as an external service behind the Druid Router. - Cacheable requests are intercepted transparently: - Fully cached requests are answered directly. - Partially cached requests are narrowed to the missing interval and sent to Druid. - Metadata queries and queries without time-based grouping bypass the cache. - The proxy can be enabled or disabled without changing clients. - Netflix views this as an interim design while exploring deeper integration with Druid. ## Query Identification and Lookup - Incoming queries are parsed to extract their interval, granularity, and structure. - A SHA-256 hash is generated from the query’s logical contents, including datasource, filters, aggregations, and relevant context properties, while excluding the time interval. - The cache looks for buckets within the requested range. - Lookup requires cached buckets to be contiguous from the beginning of the requested interval; the provided article text ends while explaining the handling of expired or missing buckets. Netflix’s approach is best suited to frequently repeated rolling-window dashboards where a small, slightly stale tail is acceptable. Segmenting results by time and assigning age-based TTLs allows the system to preserve freshness where it matters while eliminating most redundant Druid work.
Read original(opens in new tab)
