Performance Optimization

discordFeb 26, 20252 min readCurated summary

Discord Update: December 19, 2024 Changelog

Discord’s December 19, 2024 changelog recaps the year’s most useful performance and reliability improvements through a festive poem. The updates focused on making mobile apps more stable and responsive, speeding up servers, GIFs, chat, and APIs, and improving image quality and storage usage. Discord concludes by encouraging users to join voice, play games, or Go Live, while promoting Nitro gifting. ## Mobile Stability and Performance - iOS crash fixes reduced the overall crash rate by **84%**. - Android’s chat renderer reduced slow frames by up to **60%** and lowered chat-list memory usage by about **12%**. - The Android Expression Picker—covering emojis, GIFs, and stickers—saw up to **50% fewer dropped frames** and approximately **7.5% lower memory usage**. - Foldable-device support and multitasking were significantly improved. - Mobile server lists were virtualized so only visible servers remain loaded, improving scrolling performance. - Server switching on Android and iOS became more than **30% faster**. - Mobile GIF Picker loading times improved by up to **80%**. ## Backend and Media Improvements - Infrastructure changes, including moving to Google Cloud C3 instances and removing an Nginx layer, reduced p90+ API latency by roughly **25%**. - Discord’s media proxy now preserves ICC color-profile data when resizing images into WebP, improving color accuracy across platforms. - iOS data-storage changes reduced disk usage substantially; the most affected 1% of users went from approximately **5.7 GB to 1.7 GB**, with some users recovering up to **4 GB**. ## Seasonal Extras - The recap is presented as a “Fixmas” poem summarizing fixes from Discord’s 2024 Patch Notes. - Users who gift Nitro during the promotion also receive an Avatar Decoration. Together, these changes show Discord’s emphasis on incremental improvements that make everyday communication faster, more reliable, and less demanding on mobile devices.

Read original

Performance Optimization Android iOS API+4

discordFeb 26, 2025Original article

Discord Update: September 26, 2024 Changelog (opens in new tab)

Discord’s September 2024 update centers on transforming the platform into a more interactive entertainment hub while significantly hardening its security infrastructure. By centralizing third-party integrations through a new App Launcher and implementing end-to-end encryption for audio and video, the platform aims to balance expanded developer functionality with robust user privacy. ### The App Launcher and Interactive Activities * The newly launched App Launcher is now available across desktop and mobile, allowing users to search, browse curated collections, and add thousands of apps directly to their accounts for use in chats and voice calls. * Four new Activities have been integrated: *Arena Kingdoms* for cross-server battles, *Echo Chess* for daily puzzles, the Viking-themed *Battletabs*, and the social-focused *Magic Circle*. * New image-editing capabilities allow users to hover over chat images to access the App Launcher for quick modifications, such as adding captions or using Viggle’s “Animate” command to generate motion from static photos. * The developer ecosystem has been expanded to allow third parties to build, launch, and monetize their own Activities, with options to opt-in to platform-wide discovery via the launcher. ### Security and Privacy Enhancements * End-to-End Encryption (E2EE) is being introduced for all audio and video communication, including DMs, Group DMs, voice channels, and Go Live streams, ensuring that stream data is accessible only to participants. * Support for Passkeys has been implemented, allowing users to replace traditional passwords with biometric authentication such as Face ID or Touch ID. * Passkey technology remains localized to the user's device, ensuring that Discord does not have access to sensitive biometric data. ### Platform Performance and Community Resources * Discord’s engineering team reported a significant performance milestone, reducing iOS application crashes by 84%. * The "Discord Dojo" initiative has launched to provide educational content, including videos and blogs focused on message formatting and advanced keybinds for power users. * A new partnership with *Street Fighter 6* introduces themed shop items and a specific Quest that rewards users with a "Battle Field" decoration for their profiles. To maintain the highest level of account safety, users should consider migrating to Passkeys and verifying the encryption status during their next voice or video call. For those looking to increase engagement within their servers, the App Launcher provides a low-friction way to introduce collaborative games and media tools directly into existing conversations.

Performance Optimization iOS End To End Encryption Passkeys+3

datadogNov 1, 2024Original article

How we built a Ruby library that saves 50% in testing time | Datadog (opens in new tab)

Lengthy CI pipelines and flaky tests often hinder developer productivity by causing unnecessary wait times and costly infrastructure usage. To address this, Datadog developed a Ruby test impact analysis library that dynamically maps tests to specific source files, allowing the CI runner to skip tests unrelated to the latest code changes. By moving beyond standard coverage tools and utilizing low-level Ruby VM interpreter events, this solution significantly reduces testing time while maintaining high performance and correctness. ## The Strategy of Test Impact Analysis * Lengthy CI pipelines (often exceeding 20 minutes) increase the likelihood of intermittent "flaky" failures that are unrelated to current code changes. * While parallelization can reduce time, it increases cloud computing costs and does not mitigate the flakiness of irrelevant tests. * Test impact analysis generates a dynamic map between each test and the source files executed during its run; if a commit doesn't touch those files, the test is safely skipped. * Success depends on three pillars: correctness (never skipping a necessary test), performance (low overhead), and seamlessness (no required code changes for the user). ## Limitations of Standard Coverage Tools * Ruby’s built-in `Coverage` module (enhanced in version 3.1 with `resume`/`suspend` methods) proved incompatible with existing total code coverage tools like `simplecov`. * Initial prototypes using the `Coverage` module showed a performance overhead of 300%, making the test suite four times slower. * The `TracePoint` API was also evaluated as an alternative to spy on code execution via the `line` event, but it still produced a significant median overhead of 200% to 400%. * Benchmarks were conducted using the `rubocop` test suite—a "hard mode" scenario with 20,000+ tests—to ensure the tool could handle high-sensitivity environments. ## Implementing a Custom C Extension * To bypass the limitations of high-level APIs, developers utilized Ruby’s C extension capabilities to hook directly into the Virtual Machine. * The library uses `rb_add_event_hook2` and `rb_thread_add_event_hook` to subscribe to the `RUBY_EVENT_LINE` event at the interpreter level. * The implementation involves a C-based `dd_cov_start` function that triggers when a test begins and a `dd_cov_stop` function to collect the results. * During execution, the tool uses `rb_sourcefile()` to identify the current file and stores it in a Ruby hash only if the file is located within the project’s root directory. For engineering teams struggling with bloated CI pipelines, adopting test impact analysis is a highly effective way to optimize resources. By utilizing tools like Datadog’s Intelligent Test Runner, which leverages low-level VM events for minimal overhead, teams can cut their testing time in half without sacrificing the reliability of their master branch.

Performance Optimization Datadog CI/CD Ruby On Rails+4

datadogNov 1, 20243 min readCurated summary

How we built a Ruby library that saves 50% in testing time

Datadog built a Ruby test impact analysis library to reduce CI time and avoid rerunning unrelated, flaky tests. The approach maps each test to the source files it executes, then runs only tests affected by a commit. Existing Ruby coverage APIs were too slow or incompatible with standard coverage tools, so Datadog developed a faster solution using Ruby VM interpreter events. ## The CI Problem - Large test suites often take 20 minutes or more and may fail because of unrelated flaky tests. - Parallel execution reduces runtime but increases cloud costs and does not eliminate flakiness. - Selective testing can reduce: - Pipeline duration - Cloud resource usage - Exposure to unrelated flaky tests - Test impact analysis determines which source files each test executes and compares them with files changed in the latest Git commit. ## Requirements for Test Impact Analysis Datadog’s library needed to provide: - **Correctness:** Never skip a test that could detect a regression. - **Performance:** Add minimal overhead because impact data must be collected on every commit and branch. - **Seamlessness:** Require no user code changes and avoid changing test behavior or interfering with existing tooling. ## Limitations of Ruby Coverage APIs - Ruby’s built-in `Coverage` module can collect per-test coverage using `resume` and `suspend`, introduced in Ruby 3.1. - A prototype using Coverage had two major problems: - It conflicted with tools such as SimpleCov that collect total code coverage. - It added up to 300% overhead, making the test suite roughly four times slower. - Datadog then tried Ruby’s `TracePoint` API, subscribing to the `line` VM event. - TracePoint avoided interference with SimpleCov and provided the required data, but still introduced roughly 200% overhead, reaching 400% in some cases. ## A Custom Coverage Tool Using Ruby VM Events - Datadog examined Ruby’s internals, including: - `coverage.c` - `rb_coverage_resume` - `rb_resume_coverages` - `rb_add_event_hook2` - Ruby’s C extension API supports registering callbacks for `RUBY_EVENT_LINE`, allowing a custom native implementation. - The proof of concept: - Registers a line-event hook for the current thread. - Records the source file for executed lines. - Ignores files outside the project root. - Removes the hook when collection stops. - Returns and resets the collected coverage data for the next test. - Implementing collection closer to the VM was intended to preserve correctness while substantially reducing the overhead of per-test impact tracking. Datadog’s experience shows that selective testing is a promising way to make CI faster and more reliable, but practical test impact analysis requires a low-level implementation. Standard coverage and tracing APIs provide useful functionality but can impose unacceptable performance costs, making a purpose-built native VM-event collector a better fit.

Read original

Performance Optimization Datadog CI/CD Cloud Computing+4

figmaMay 21, 20242 min readCurated summary

Speeding Up File Load Times, One Page At A Time | Figma Blog

Figma improved file-loading performance by dynamically loading only the page a user opens instead of the entire file. This approach reflects user-perceived complexity, reduces memory usage, and avoids making small pages wait for unrelated content. For the slowest 5% of loads, the change reduced load times by 33%. ## Loading Content According to User Needs - Figma files can contain dozens of pages, hundreds of frames, components, styles, variables, and prototype screens. - Usage data showed that users often treat a file as an entire project but typically visit only a subset of its pages in one session. - Loading everything up front made a small page unnecessarily depend on the size of the whole file. - Dynamic loading lets Figma display the selected page first and fetch additional content only when needed. ## Cross-Page Read Dependencies - A Figma file is modeled as a tree of nodes, with nodes representing interactable layers and their properties. - Nodes can reference content located on other pages, creating cross-page dependencies. - An instance points to its backing component, which may be on another page; the component must be loaded before the instance can render correctly. - Styles and variables also create dependencies: - A fill style requires loading its corresponding style node. - A variable-based font size requires the variable node so the client can resolve the raw value. ## QueryGraph and Earlier Dynamic Loading - Figma had already developed dynamic loading for view-only files and prototypes. - Its QueryGraph framework stores dependency relationships as an in-memory graph. - The multiplayer system uses this graph to determine which parts of a file should be sent to connected clients. - Previous loading strategies included: - **Page-based canvas loading:** Load the selected page and its required dependencies, then fetch other pages on demand. - **Frame-based prototype loading:** Load the current prototype screen and preload reachable frames within a limited number of transitions. ## Performance Impact - The goal is for load times to trend downward even as files become larger and more feature-rich. - Dynamic loading improves both initial responsiveness and memory consumption. - The largest benefits appear in worst-case loads, with a reported 33% reduction for the slowest 5% of page loads. Figma’s approach demonstrates that large collaborative documents should be loaded according to the user’s immediate context, while dependency tracking ensures referenced content remains available when required.

Read original

Performance Optimization Figma Dependency Graph Dynamic Loading+1

datadogMay 20, 20243 min readCurated summary

.NET Continuous Profiler: Memory usage

Datadog’s .NET memory profiler helps identify excessive garbage collection, allocation hotspots, and objects that remain in memory after collection. It combines CLR events, operating-system thread metrics, sampled allocation data, stack traces, and weak handles to provide production-friendly memory insights. The approach favors low overhead, though some capabilities depend on the .NET version. ## Measuring Garbage Collector CPU Impact - The profiler uses CLR events to monitor garbage collection phases. - In server GC mode, the CLR creates two high-priority threads per heap/core to process collections in parallel. - Since .NET 5, these threads are named `.NET Server GC` and `.NET BGC`. - At each profile export, the profiler retrieves these threads’ CPU usage from the operating system. - It records the result as a sample with a native stack containing a `Garbage Collector` frame. - This uses a pull model: the exporter periodically requests the CPU measurement because no suitable event or dedicated profiler thread exists. - Before .NET 5, GC thread CPU usage could not reliably be identified because `GCCreateConcurrentThread` did not include thread IDs. ## Sampling Allocations - Per-allocation callbacks such as `ICorProfilerCallback::ObjectAllocated` provide detailed data but significantly slow allocation fast paths. - `GCSampledObjectAllocation` and `ObjectsAllocatedByClass` reduce some costs but do not provide call stacks for individual allocation sites. - Datadog instead listens to `AllocationTick`, emitted for roughly every 100 KB allocated. - Each event includes: - The object’s `ClassID` and type information. - The allocation address. - The object size and total allocation size since the previous tick. - The allocation kind: SOH (`0`), LOH (`1`), or POH (`2`). - Generic type names are reconstructed through the .NET profiling API. - Because allocation events are synchronous, the current thread is responsible for the allocation; the profiler walks that thread’s stack to capture the allocation call site. - This produces sampled allocation data for each heap category without imposing the cost of observing every allocation. ## Tracking Objects That Survive Garbage Collection - An allocation address alone cannot track an object indefinitely because compacting garbage collections can move objects. - Datadog uses weak handles, created through `GCHandle.Alloc`, which move with objects and do not keep them alive. - The profiler added this functionality through the .NET 7 `ICorProfilerInfo13` API and its `LiveObjectsProvider`. - For every sampled allocation, it creates a weak handle and records the object’s creation time. - After each garbage collection: - Handles for unreachable objects are removed and destroyed. - Handles for surviving objects remain and are included in the next profile. - This lets users inspect representative objects that persist after collection and investigate potential memory leaks. ## Practical Recommendation Use allocated-memory profiles to find endpoints and types responsible for excessive allocation, then examine surviving-object samples for retention or leak investigations. GC CPU data is especially useful for diagnosing applications whose high CPU usage is driven by frequent or expensive garbage collections.

Read original

Performance Optimization Datadog Dotnet Garbage Collection+4

microsoftMay 14, 20243 min readCurated summary

Copy-on-Write performance and debugging

Dev Drive’s ReFS-based copy-on-write (CoW) linking can significantly improve build performance, though results vary by repository structure. The largest gains occur when builds repeatedly copy assemblies or generate microservice layouts; C++-heavy builds generally benefit less. The post also explains how to inspect CoW links, use performance tools safely, and repair leaked ReFS references. ## Build Performance Results - Testing compared NTFS and Dev Drive on the same Dev Box VM. - Many repositories achieved build-time reductions of 10% or more, with a maximum observed improvement of 43%. - The strongest benefits appeared in: - C# repositories with deep project-to-project dependencies, where MSBuild repeatedly copies assemblies. - Builds that copy many files to construct microservice output layouts. - C++ repositories generally saw smaller improvements because: - MSBuild copies output files less frequently. - MSVC produces fewer, larger files, reducing the impact of lower file-I/O overhead. - Repositories with long chains of large dependent projects benefited less, since serial build stages limited the effect of faster I/O. - Tests used clean source and output directories, separated package restore and tests from build measurements, and ran at least five iterations while excluding the first cold-cache run. - The tests used the `Microsoft.Build.CopyOnWrite` SDK and, where relevant, an updated `Microsoft.Build.Artifacts` SDK. CoW-in-Win32 was not yet available during testing. ## Identifying CoW Links - CoW links, also called block clones, allow multiple files to reference the same physical disk blocks. - `fsutil file queryExtentsAndRefCounts <file>` displays the file’s extents and reference counts. - A reference count such as `Ref: 0x4` indicates that the underlying blocks are shared by four cloned files. - Each cloned file also requires a small amount of metadata storage, typically one additional cluster. ## Using ProcMon on Dev Drive - Dev Drive restricts file-system filter drivers through an allow-list. - To use ProcMon: - Check the current filter list with `fsutil devdrv query`. - Add ProcMon’s current filter driver, such as `ProcMon24`, using `fsutil devdrv setfiltersallowed`. - Dismount the Dev Drive for the change to take effect. - ProcMon’s filter is attached only while ProcMon is running, so it can generally remain on the allow-list. ## Using Microsoft Performance Recorder - Microsoft Performance Recorder requires the `FileInfo` filter driver. - Add `FileInfo` to the Dev Drive filter allow-list and dismount the volume before recording. - Remove `FileInfo` afterward because it remains attached whenever the filter is allowed and can reduce Dev Drive performance. ## Repairing Leaked CoW References - ReFS limits a data block to 8,176 clones. - Excessive or orphaned references can cause errors such as: - `MaxCloneFileLinksExceededException` - `ERROR_BLOCK_TOO_MANY_REFERENCES` (347) - `STATUS_BLOCK_TOO_MANY_REFERENCES` (`0xC000048C`) - The issue is uncommon but can occur after prolonged CoW-heavy builds, particularly with prerelease implementations. - Run `refsutil leak <drive> /s <scratch-file>` from an elevated console to scan and repair dangling references. - Add `/d` to detect leaks without fixing them. - Large volumes may contain billions of leaked references, and the repair process can take considerable time. Dev Drive and CoW linking are most worthwhile for build systems dominated by repeated file copying, especially large C# and microservice-oriented repositories. Teams should also configure diagnostic filter drivers carefully and periodically use `refsutil` if clone-reference errors appear.

Read original

Performance Optimization Dev Drive Copy On Write Msbuild+3

datadogApr 18, 20243 min readCurated summary

How we brought Datadog's data visualization to iOS: A focus on performance

Datadog built DogGraphs, a native SwiftUI graphing library, to support complex data visualizations across its iOS app and widgets. Because existing libraries did not meet its needs and the app supported iOS 14, the team had to optimize SwiftUI rendering without newer APIs such as `Canvas`. By combining careful API design, profiling, and a better understanding of SwiftUI’s update model, DogGraphs became a reusable framework used across multiple Datadog products. ## Building DogGraphs for Complex Visualizations - DogGraphs began with the Service Catalog and was designed to support additional Datadog products. - It needed to provide: - Native Swift and SwiftUI rendering - iOS 14 compatibility - Flexible, easy-to-use APIs - Datadog’s default visual style and behavior - Fast rendering for a responsive user experience - The library now powers visualizations in logs, services, dashboards, Bits AI, and mobile widgets. - It supports increasingly diverse graph types as new products integrate with the mobile application. ## A Declarative, Type-Safe API - DogGraphs uses Swift features such as result builders to describe complex graph configurations declaratively, in a style similar to SwiftUI. - Graph definitions can be generated dynamically from server-provided dashboard or widget configurations. - Compile-time type checking prevents invalid combinations, such as stacking incompatible Bar and Line graphs. - Progressive disclosure provides sensible Datadog defaults while still allowing customization when necessary. ## Profiling SwiftUI Performance Datadog’s visualizations can involve metrics, logs, traces, multiple aggregation strategies, arithmetic operations, axes, labels, scales, and color configuration. Query responses are preprocessed by a shared internal service so that formatting and visual behavior remain consistent across platforms. To optimize rendering, the team focused on two primary measurements: - **SwiftUI view body evaluations** - Excessive body evaluations can degrade performance, especially when many views are involved. - Expensive computation should be moved outside view bodies. - `_printChanges()` can reveal why a view is being reevaluated, though it is a private API unsuitable for production use. - **Time Profiler** - Instruments helps identify slow function calls and locate expensive work in the rendering pipeline. Important profiling scenarios included: - Initial graph rendering - Updates caused by window changes, tooltip selection, or layer visibility changes - Device rotation and light/dark mode changes - Interactions with unrelated views such as scroll views, toggles, and buttons ## Understanding SwiftUI’s Update Model The team used Apple’s “Demystify SwiftUI” session to build a mental model for how SwiftUI determines when views should update. - **Identity:** How SwiftUI determines whether an element is the same as, or different from, a previous element. - **Lifetime:** How SwiftUI tracks a view and its associated data over time. - **Dependencies:** How SwiftUI determines which changes require an interface update. - **Diffing:** SwiftUI compares view values to determine what changed, although the exact diffing mechanism is undocumented. Understanding these concepts helps developers explain unexpected view updates and identify the sources of rendering bottlenecks. ## Practical Recommendation For complex SwiftUI components, measure real interaction scenarios rather than relying on assumptions. Track body evaluations and expensive function calls, keep costly work out of `body`, and design APIs that provide efficient defaults while preserving type safety and flexibility.

Read original

Performance Optimization Data Visualization Swiftui Mobile App Development+3

datadogApr 17, 20233 min readCurated summary

Making fetch happen: Building a general-purpose query and render scheduler

Datadog rebuilt its dashboard scheduler to improve responsiveness while distributing network and rendering work more efficiently. The legacy system helped, but had grown into a complex set of roughly 20 interdependent heuristics that were difficult to maintain and poorly separated query scheduling from rendering. A simpler, general-purpose approach reduced request spikes, improved fetching performance, and created a foundation for browser-aware task scheduling. ## Limitations of the Original Scheduler - A periodic updater determined when widgets should request fresh data based on factors such as time range and browser focus. - Query tasks for visible widgets ran immediately; offscreen queries were delayed using heuristics such as pending-query counts and historical fetch durations. - Render tasks similarly prioritized visible widgets and delayed offscreen work. - The system improved performance over an unscheduled baseline by reducing main-thread work and flattening query traffic. - Over time, it accumulated around 20 parameters and interlinked rules. - Query and render concerns were mixed together: - Queries could be delayed because too many render tasks were pending. - Renders could be delayed based on data size even when browser resources were available. - The dashboard-specific implementation could not easily be reused across Datadog’s increasingly generalized widget framework. ## A General-Purpose Scheduling Strategy - Datadog separated query scheduling from render scheduling so each could be developed, tested, and rolled out independently. - The team evaluated existing heuristics across dashboards of different sizes and under different browser conditions. - Several rules were removed without harming performance: - Unfocused or occluded tabs did not need special delays because the periodic updater and browser already throttle them. - The redesign aimed to preserve two goals: - Keep query execution distributed over time. - Prioritize widgets visible to the user. - The new system was progressively deployed, first to dashboards and then to the shared data-fetching framework used across Datadog. ## Simpler Query Scheduling The new query algorithm uses a small set of straightforward rules: - Fetches for visible widgets run immediately. - Non-visible queries are ranked and executed in fixed time windows, subject to a task limit. - Query execution pauses when the number of pending fetches becomes too high. - The chosen configuration uses: - A 2,000-millisecond time window. - A maximum of 10 tasks per window. - FIFO-style ranking for offscreen queries, favoring earlier requests. - The scheduler uses only about six parameters instead of the legacy system’s roughly 20. - The simplified algorithm produced a better task distribution than the old scheduler. - “429 Too many requests” errors dropped significantly, reducing retries and helping data arrive sooner. ## Browser-Aware Render Scheduling - The old render scheduler did not account for the browser’s available CPU and memory resources. - Datadog adopted the Browser Scheduling API to create prioritized tasks that the browser can schedule natively. - Tasks can receive priorities such as: - `user-blocking` - `user-visible` - `background` - A `TaskController` assigns a priority signal to scheduled work. - Priorities can later be changed for all tasks controlled by the same controller, and tasks can be aborted. - The API was supported in Chromium and Firefox Nightly, with a polyfill for other browsers. Datadog’s experience suggests that performance schedulers benefit from simple, independently testable rules: prioritize visible work, smooth network activity, and let the browser manage expensive rendering when possible.

Read original

Performance Optimization Database Design Query Scheduling Browser Scheduling Api+4

datadogJan 31, 20232 min readCurated summary

Performance improvements in the Datadog Agent metrics pipeline

The Datadog Agent needed to process more metrics without increasing CPU usage. Profiling showed that generating unique metric contexts—especially sorting and deduplicating tags—was a major bottleneck. Datadog improved throughput through specialized sorting paths, faster hashing, and a more efficient context-storage design. ## Identifying the Bottleneck - Datadog uses Go’s CPU and memory profiling tools to optimize the Agent’s metrics pipeline. - Profiles were captured while Agents processed large volumes of DogStatsD metrics, ensuring the results reflected real workload pressure. - Flamegraphs showed that `addSample` and `trackContext` consumed the most CPU. - Sorting-related functions, including `util.SortUniqInPlace` and `sort`, were significant contributors to that cost. ## How Metric Contexts Work - Each received metric is assigned a metric context that uniquely identifies it in an in-memory hash table. - The context must incorporate: - The metric name - Tags included in the DogStatsD message - Container-generated tags - The context is computed as a hash, so it must be fast while minimizing collisions. - Tags must be consistently ordered so the same metric always produces the same context. - The original implementation sorted tags and removed duplicates, making sorting a recurring CPU expense. ## Specialized Sorting - Performance varied according to the number of tags attached to a metric. - Datadog introduced specialized sorting paths based on tag count. - This allowed common cases to use more efficient algorithms while retaining correct ordering and deduplication. ## Faster Hashing and Map Access - Micro-benchmarks compared hash functions according to speed and uniqueness. - Murmur3 performed best for Datadog’s requirements. - Datadog also changed metric contexts from 128-bit to 64-bit hashes. - A 64-bit hash still provided sufficient collision resistance for the use case and enabled Go runtime optimizations: - `runtime.mapassign_fast64` - `runtime.mapaccess2_fast64` - These optimized map operations improved both context storage and metric sampling performance. ## Redesigning the Algorithm - Sorting served two purposes: producing an ordered tag list and helping deduplicate tags. - Because sorting was the largest bottleneck, Datadog began exploring a design that could address these responsibilities more efficiently rather than relying on a single general-purpose sort. The practical lesson is to profile under realistic load, optimize the hottest paths, and combine targeted specialization, benchmark-driven implementation choices, and data-structure redesign to increase throughput without adding CPU capacity.

Read original

Performance Optimization Go Cpu Profiling Datadog Agent+3

figmaNov 20, 20193 min readCurated summary

Figma’s infrastructure: What goes into powering a web-based design tool | Figma Blog

Figma’s infrastructure team is focused on making a web-based design tool feel as fast and reliable as a desktop application. Its cloud-based, collaborative model creates demanding networking, storage, and scaling challenges, especially as organizations grow. The company is therefore evolving from a simple but operationally intensive backend toward a more mature, horizontally scalable architecture. ## Figma’s Collaborative Design Model - Figma stores designs in the cloud, giving each file a unique URL and making it a shared source of truth. - Users can collaborate, comment, prototype, and support developer handoff without exporting files or managing separate versions. - Its multiplayer functionality allows multiple people to view and edit files simultaneously. - Files may contain complex shapes and large images, creating significant data-transfer demands. - Users accustomed to desktop design tools expect fast interaction despite Figma running in a browser. ## Limits of the Original Infrastructure - Figma’s early backend favored simple strategies that worked well for smaller teams. - The application initially loaded all shared components available to a user when opening a file. - This approach became increasingly inefficient as organizations accumulated nearly 10,000 shared design elements. - Growth among large customers and the broader user base exposed the limits of the original design. ## Reducing Interaction Latency - Figma still preloads more data than users immediately need, increasing backend load and slowing startup. - Fixing this requires more than backend optimization: the client-server interaction model must be redesigned. - The client should request information incrementally, only when it is needed. - This change also requires coordination with product teams because existing user experiences depend on preloaded data. ## Building a Horizontally Scalable Database - At the time of the article, Figma relied on a single powerful AWS database instance. - The simple architecture reflected the company’s preference for the KISS principle and had supported substantial growth. - As usage increased, the database approached its capacity limits. - Figma planned to replace it with a database layer capable of scaling horizontally, a difficult transition because nearly every system depends on the database. ## Improving International Performance - More than 80% of Figma’s weekly active users were located outside the United States. - Their latency was affected by the round-trip distance to Figma’s Oregon datacenter. - Figma planned to move selected infrastructure components closer to users around the world. - The initial step was to deploy strategically placed remote proxies globally. Figma’s broader infrastructure strategy is to preserve the simplicity that helped it move quickly while introducing selective caching, better data loading, geographic distribution, and horizontal scaling where growth demands it.

Read original

Performance Optimization AWS Real Time Collaboration Distributed Systems+2

figmaAug 12, 20182 min readCurated summary

File loading, dragging & zooming is up to 3x faster | Figma Blog

Figma argues that performance is central to making a design tool feel like an extension of the user’s mind. A months-long effort to restructure its renderer and fix WebAssembly issues produced major gains, including up to 3× faster loading, zooming, and dragging in dense files. The improvements were measured not only by average speed, but also by interaction smoothness and frame-time consistency. ## Performance as a Core Product Requirement - Delays undermine the feeling of direct manipulation, much like a hammer lagging behind the user’s hand. - Figma continuously profiles real user documents to identify targeted optimizations. - Larger organizations increasingly use complex files with deeply nested components and many permutations, making performance more challenging. - The latest work focused on restructuring the document renderer and resolving WebAssembly bugs. ## File Loading: Up to 3× Faster - A large Microsoft Fluent Design document improved from approximately 29 seconds to under 8 seconds. - WebAssembly optimizations and other renderer changes reduced computational overhead. - WebAssembly support was enabled across: - Figma’s desktop app - Chrome - Firefox - Safari - macOS and Windows - The improvements were especially valuable for dense design-system files containing deeply nested components. ## Smoother Zooming and Dragging - Zooming and dragging are continuous interactions where responsiveness matters more than total operation duration. - Figma reduced visible “hitches” in these interactions, with improvements of up to 3×. - Dense files containing many bitmap images benefited substantially. - Figma also worked directly with customers such as N3TWORK to diagnose performance issues and test the new renderer. - Users reported immediately noticeable improvements in component publishing and file loading. ## Measuring Smoothness with Frame Time - A 500 ms operation can feel very different depending on whether it provides continuous visual feedback or freezes until completion. - Figma tracks two metrics: - **Average frame time:** Indicates overall choppiness or low frame rate. - **Maximum frame time:** Reveals occasional long pauses or “hitches.” - High maximum frame times feel like sudden interruptions, while high average frame times make motion consistently choppy. - Monitoring both metrics gives a more complete view of interaction quality than measuring total operation time or average frame rate alone. - Although a steady 60 frames per second is the ideal, tracking these metrics over time helps Figma evaluate progress toward that goal. Figma’s work demonstrates that performance optimization should focus on both raw speed and perceptual smoothness. For interactive tools, renderer architecture, WebAssembly execution, and detailed frame-time measurement are all essential to making complex documents feel responsive.

Read original

Performance Optimization Figma Webassembly Rendering+1

datadogApr 24, 20183 min readCurated summary

Using Datadog APM to improve the performance of Homebrew

Andrew Robert McBurney describes using Datadog APM to diagnose and optimize Homebrew’s slow `brew linkage` command. Instrumentation identified `LinkageChecker#check_dylibs` as the main bottleneck, and replacing repeated dynamic-library processing with persistent caching reduced execution time from 11.5 seconds to 182 milliseconds for 106 packages. A later implementation used Ruby’s built-in PStore instead of SQLite3 to avoid an additional gem dependency. ## Finding the Bottleneck with APM - Homebrew is widely used at Datadog, so improving its performance provides broad benefits. - The `brew linkage` command checks the library links of installed formulas and can identify when a reinstall is needed. - The target was to scan roughly 50 packages, including large packages such as Boost, in under five seconds. - The author instrumented Homebrew with Datadog’s Ruby `ddtrace` gem. - Flame graphs showed that most execution time was spent in `LinkageChecker#check_dylibs`. ## Why Multithreading Was Not Effective - The author tested Ruby threads as a way to process libraries concurrently. - Ruby’s Global Interpreter Lock limited the achievable parallelism. - Threading failed to meet the required performance target, so a different approach was needed. ## SQLite3-Based Caching - The expensive library-processing results were stored in an on-disk SQLite database. - A `linkage` table recorded: - Formula names and library paths - Linkage categories such as `system_dylibs`, `broken_dylibs`, `undeclared_deps`, and `brewed_dylibs` - Optional labels for selected linkage types - A uniqueness constraint on `(name, path, type, label)` prevented duplicate cache entries. - Homebrew could insert and retrieve linkage data using SQL queries. ## Performance Improvements - Without caching, processing 106 packages took 11.5 seconds. - Boost alone required about 1.01 seconds for dynamic-library checks. - With caching enabled: - The full command completed in 182 milliseconds. - Boost’s check took approximately 1.38 milliseconds. - The cached implementation significantly exceeded the original five-second performance requirement. ## Moving to PStore - After submitting the SQLite3 implementation for review, Homebrew maintainers recommended Ruby’s PStore. - PStore provides file-based persistence built around Ruby’s `Hash` data structure. - Its main advantage is avoiding a third-party SQLite3 gem dependency while preserving the benefits of caching. The central lesson is that profiling should guide optimization: rather than adding ineffective threading, the author located the true bottleneck and achieved a dramatic speedup through persistent caching. For similar command-line performance problems, instrument the complete execution path first, then choose the simplest cache or storage mechanism that satisfies both speed and dependency constraints.

Read original

Performance Optimization Datadog Application Performance Monitoring Caching+4