Making fetch happen: Building a general-purpose query and render scheduler
Datadog rebuilt its dashboard scheduler to improve responsiveness while distributing network and rendering work more efficiently. The legacy system helped, but had grown into a complex set of roughly 20 interdependent heuristics that were difficult to maintain and poorly separated query scheduling from rendering. A simpler, general-purpose approach reduced request spikes, improved fetching performance, and created a foundation for browser-aware task scheduling. ## Limitations of the Original Scheduler - A periodic updater determined when widgets should request fresh data based on factors such as time range and browser focus. - Query tasks for visible widgets ran immediately; offscreen queries were delayed using heuristics such as pending-query counts and historical fetch durations. - Render tasks similarly prioritized visible widgets and delayed offscreen work. - The system improved performance over an unscheduled baseline by reducing main-thread work and flattening query traffic. - Over time, it accumulated around 20 parameters and interlinked rules. - Query and render concerns were mixed together: - Queries could be delayed because too many render tasks were pending. - Renders could be delayed based on data size even when browser resources were available. - The dashboard-specific implementation could not easily be reused across Datadog’s increasingly generalized widget framework. ## A General-Purpose Scheduling Strategy - Datadog separated query scheduling from render scheduling so each could be developed, tested, and rolled out independently. - The team evaluated existing heuristics across dashboards of different sizes and under different browser conditions. - Several rules were removed without harming performance: - Unfocused or occluded tabs did not need special delays because the periodic updater and browser already throttle them. - The redesign aimed to preserve two goals: - Keep query execution distributed over time. - Prioritize widgets visible to the user. - The new system was progressively deployed, first to dashboards and then to the shared data-fetching framework used across Datadog. ## Simpler Query Scheduling The new query algorithm uses a small set of straightforward rules: - Fetches for visible widgets run immediately. - Non-visible queries are ranked and executed in fixed time windows, subject to a task limit. - Query execution pauses when the number of pending fetches becomes too high. - The chosen configuration uses: - A 2,000-millisecond time window. - A maximum of 10 tasks per window. - FIFO-style ranking for offscreen queries, favoring earlier requests. - The scheduler uses only about six parameters instead of the legacy system’s roughly 20. - The simplified algorithm produced a better task distribution than the old scheduler. - “429 Too many requests” errors dropped significantly, reducing retries and helping data arrive sooner. ## Browser-Aware Render Scheduling - The old render scheduler did not account for the browser’s available CPU and memory resources. - Datadog adopted the Browser Scheduling API to create prioritized tasks that the browser can schedule natively. - Tasks can receive priorities such as: - `user-blocking` - `user-visible` - `background` - A `TaskController` assigns a priority signal to scheduled work. - Priorities can later be changed for all tasks controlled by the same controller, and tasks can be aborted. - The API was supported in Chromium and Firefox Nightly, with a polyfill for other browsers. Datadog’s experience suggests that performance schedulers benefit from simple, independently testable rules: prioritize visible work, smooth network activity, and let the browser manage expensive rendering when possible.
Read original(opens in new tab)