Making fetch happen: Building a general-purpose query and render scheduler (opens in new tab)
Datadog rebuilt its dashboard scheduler to improve responsiveness while distributing network and rendering work more efficiently. The legacy system helped, but had grown into a complex set of roughly 20 interdependent heuristics that were difficult to maintain and poorly separated query scheduling from rendering. A simpler, general-purpose approach reduced request spikes, improved fetching performance, and created a foundation for browser-aware task scheduling. ## Limitations of the Original Scheduler - A periodic updater determined when widgets should request fresh data based on factors such as time range and browser focus. - Query tasks for visible widgets ran immediately; offscreen queries were delayed using heuristics such as pending-query counts and historical fetch durations. - Render tasks similarly prioritized visible widgets and delayed offscreen work. - The system improved performance over an unscheduled baseline by reducing main-thread work and flattening query traffic. - Over time, it accumulated around 20 parameters and interlinked rules. - Query and render concerns were mixed together: - Queries could be delayed because too many render tasks were pending. - Renders could be delayed based on data size even when browser resources were available. - The dashboard-specific implementation could not easily be reused across Datadog’s increasingly generalized widget framework. ## A General-Purpose Scheduling Strategy - Datadog separated query scheduling from render scheduling so each could be developed, tested, and rolled out independently. - The team evaluated existing heuristics across dashboards of different sizes and under different browser conditions. - Several rules were removed without harming performance: - Unfocused or occluded tabs did not need special delays because the periodic updater and browser already throttle them. - The redesign aimed to preserve two goals: - Keep query execution distributed over time. - Prioritize widgets visible to the user. - The new system was progressively deployed, first to dashboards and then to the shared data-fetching framework used across Datadog. ## Simpler Query Scheduling The new query algorithm uses a small set of straightforward rules: - Fetches for visible widgets run immediately. - Non-visible queries are ranked and executed in fixed time windows, subject to a task limit. - Query execution pauses when the number of pending fetches becomes too high. - The chosen configuration uses: - A 2,000-millisecond time window. - A maximum of 10 tasks per window. - FIFO-style ranking for offscreen queries, favoring earlier requests. - The scheduler uses only about six parameters instead of the legacy system’s roughly 20. - The simplified algorithm produced a better task distribution than the old scheduler. - “429 Too many requests” errors dropped significantly, reducing retries and helping data arrive sooner. ## Browser-Aware Render Scheduling - The old render scheduler did not account for the browser’s available CPU and memory resources. - Datadog adopted the Browser Scheduling API to create prioritized tasks that the browser can schedule natively. - Tasks can receive priorities such as: - `user-blocking` - `user-visible` - `background` - A `TaskController` assigns a priority signal to scheduled work. - Priorities can later be changed for all tasks controlled by the same controller, and tasks can be aborted. - The API was supported in Chromium and Firefox Nightly, with a polyfill for other browsers. Datadog’s experience suggests that performance schedulers benefit from simple, independently testable rules: prioritize visible work, smooth network activity, and let the browser manage expensive rendering when possible.