Data Visualization

16 posts

cloudflare2 min readCurated summary

Total eclipse of the Internet: traffic impacts in Iceland, Spain, and Portugal

The August 12, 2025 total solar eclipse caused a measurable, temporary decline in Internet activity across Europe. Cloudflare Radar data shows that HTTP traffic dropped most sharply when the eclipse reached maximum obscuration, especially in countries along the path of totality. Traffic generally returned to normal within minutes as people resumed using their devices. ## Traffic Drops Matched Eclipse Timing - Cloudflare analyzed HTTP requests in five-minute intervals across affected countries. - Traffic reductions aligned closely with each location’s moment of maximum eclipse. - The strongest declines occurred in Iceland, Ireland, the UK, France, Spain, and Portugal. - Countries with only shallow partial eclipses, including Sweden, Denmark, Poland, and Switzerland, saw little or no decline. - Regions experiencing deep eclipses recorded traffic drops of roughly 15% to 30%. - Traffic typically rebounded shortly after maximum obscuration. ## Eclipse Depth Predicted Internet Activity - Researchers compared each country’s peak solar obscuration with its average traffic change during the surrounding 15-minute window. - The results showed a clear downward relationship: greater obscuration generally produced larger traffic declines. - Local factors such as population density, cloud cover, and time of day caused some variation, but the precise timing supported the eclipse as the primary cause. - Solar obscuration was calculated geometrically using the apparent sizes and positions of the sun and moon, measuring how much of the sun’s disk was covered every five minutes. ## Iceland, Spain, and Portugal Saw the Largest Declines - Country-level traffic changes ranged from a 9.3% increase to a 46.7% decrease. - Iceland, Spain, and Portugal experienced the most dramatic reductions. - Norway and Sweden saw slight increases above normal levels. - Denmark experienced the smallest overall change, while Poland quickly returned to baseline. - Eclipse-day traffic was compared with the median traffic from the three previous Wednesdays, using matching times of day to reduce the effect of unusual weekly patterns. ## Physical Events Reshape Digital Behavior - The findings show that Internet traffic reflects where people direct their attention. - The eclipse reduced online activity because people temporarily stopped using their devices to observe it, not because of technical network problems. - Traffic normalized quickly afterward, demonstrating how a shared real-world event can create a continent-wide but short-lived shift in digital behavior. - Cloudflare Radar can be used to study similar changes during major global events.

Read original(opens in new tab)
kakao3 min readCurated summary

The Calendar We Designed for KakaoTalk Booking

KakaoTalk Reservation built a time-block calendar to help sellers understand inventory and bookings more easily than with a card list. The main challenge was arranging bookings with different start times and durations so that they remain readable and use space efficiently. The solution combines sorting rules, graph-based layout calculation, DFS, and additional expansion logic for edge cases. ## Product Requirements - The calendar is designed for the seller-facing reservation management center. - Each time slot can contain up to 10 bookings. - A booking can last from one to six hours. - Booking blocks should be displayed as clearly as possible without leaving unnecessary gaps. - The input data provides booking start and end times, while the frontend must calculate the visual layout. ## Booking Placement Rules - **Sort by earliest start time** - Earlier bookings are placed first to match the seller’s natural workflow. - This also supports the typical visual scanning order from the upper-left toward the lower-right. - **For bookings starting at the same time, sort by longest duration** - Longer bookings can block shorter bookings from expanding. - Placing them first gives them enough space and allows later bookings to occupy the remaining areas more effectively. ## Graph-Based Expansion - The bookings are modeled as nodes in a graph. - Each node stores relationships with preceding and following overlapping bookings. - A depth-first search calculates: - Each node’s depth, representing its horizontal position. - The maximum distance to the final booking in its connected path. - These values are used to calculate: - `left`: the node’s horizontal starting position. - `width`: how far the booking can expand across available space. - Nodes at the far-left edge of the graph are processed first, allowing the bookings to fill the calendar while respecting overlaps. ## Handling Layout Exceptions - The initial graph and DFS calculation did not always fill all available space. - Problems occurred when: - Multiple root nodes existed. - An upper root node had a longer path than a lower root node. - Connected nodes were constrained by earlier width calculations. - The implementation searches for unused gaps between neighboring nodes. - When multiple gaps exist, connected nodes are expanded by the smallest available amount needed to close the gaps. - A gap is detected when the next node’s `left` position is greater than the current node’s `left + width`. ## Lessons from the Implementation - A calendar that appears visually simple can require substantial algorithmic design. - Frontend developers are responsible not only for rendering data, but also for deciding how that data should be presented to users. - The calendar is treated as an evolving implementation that will be refined as new bugs, data patterns, and better algorithms are discovered. The practical approach is to begin with clear sorting rules, represent overlapping bookings as a graph, use DFS to determine layout constraints, and add targeted post-processing for unused space and edge cases.

Read original(opens in new tab)
google3 min readCurated summary

Improving the academic workflow: Introducing two AI agents for better figures and peer review

AI is being positioned as an active participant in academic research, not merely a tool for drafting text. The post introduces PaperVizAgent, which creates publication-ready figures, and ScholarPeer, which produces literature-grounded peer reviews. Both use multi-agent workflows and iterative verification to reduce researchers’ administrative burden while improving visual quality and review rigor. ## PaperVizAgent: Generating Publication-Ready Figures - PaperVizAgent converts manuscript text and a detailed figure caption into academic illustrations. - It uses five specialized agents: - **Retriever:** Finds relevant literature and reference figures. - **Planner:** Organizes the technical content. - **Stylist:** Develops appropriate visual and aesthetic guidelines. - **Visualizer:** Produces images or executable Python code for statistical plots. - **Critic:** Checks the result against the source text and requests revisions. - The critic-driven refinement loop is designed to ensure that figures are both technically faithful and visually clear. - Inputs typically include: - The manuscript’s method or technical sections. - A communicative-intent description explaining what the figure should convey. ### Evaluation Results - PaperVizAgent was compared with direct prompting, few-shot prompting, GPT-Image-1.5, Nano-Banana-Pro, and Paper2Any. - Figures were scored from 0 to 100 on: - Faithfulness - Conciseness - Readability - Aesthetics - It achieved an overall score of **60.2**, exceeding the human baseline of **50.0** and outperforming the evaluated automated systems. - Its strongest results were in conciseness and aesthetics, while its statistical plots reached human-competitive quality. ## ScholarPeer: Automating Rigorous Peer Review - ScholarPeer is a search-enabled, context-aware multi-agent system designed to emulate the workflow of a senior academic reviewer. - Rather than treating review as simple text generation, it combines literature retrieval, adversarial checking, and technical verification. - Its main components include: - A **sub-domain historian** that builds a current domain narrative from literature. - A **baseline scout** that searches for overlooked datasets, methods, and comparisons. - A **multi-aspect Q&A engine** that tests novelty and technical claims. - A **review generator** that follows conference-specific review guidelines. - The resulting review includes a summary, strengths, weaknesses, and questions for the authors. ### Evaluation Results - ScholarPeer was evaluated on public datasets against fine-tuned models and other agentic reviewing systems. - Its active web-search and verification process produced highly critical reviews grounded in existing research. - Side-by-side evaluations showed strong win rates against competing automated reviewers. - The system also narrowed the gap between AI-generated reviews and human reviews in terms of realism, diversity, and alignment with expert judgments. ## Implications for Academic Research - The two agents address separate bottlenecks in the publication process: - PaperVizAgent improves technical communication through better figures. - ScholarPeer helps scale peer review amid growing submission volumes and reviewer fatigue. - Their multi-agent designs suggest that specialized agents, coordinated through retrieval and iterative critique, may be more effective than a single general-purpose language model. - The systems are intended to support researchers rather than replace scientific judgment. Researchers could use PaperVizAgent for early figure prototyping and ScholarPeer for preliminary, literature-informed critique, while retaining human oversight for final scientific and editorial decisions.

Read original(opens in new tab)
gitlabOriginal article

Track vulnerability remediation with the updated GitLab Security Dashboard (opens in new tab)

The updated GitLab Security Dashboard addresses the challenge of vulnerability overload by shifting the focus from simple detection to contextual remediation and risk management. By providing integrated trend tracking and sophisticated risk scoring, the platform enables security and development teams to prioritize high-risk projects and measure the actual progress of their security programs. This update transforms raw security data into actionable insights that are tracked directly within the existing DevSecOps workflow. ## Transitioning from Detection to Remediation Context * Consolidates vulnerability data into a single view that spans across projects, groups, and entire business units to eliminate data silos. * Introduced initial time-based tracking in version 18.6, with version 18.9 adding expanded filters for severity, status, scanner type, and project. * Provides visualizations for remediation velocity and vulnerability age distribution, moving beyond static raw counts to show how quickly threats are being addressed. ## Data-Driven Prioritization with Risk Scoring * Utilizes a dynamic risk score calculated from multiple factors, including vulnerability age and repository security postures. * Integrates external threat intelligence such as the Exploit Prediction Scoring System (EPSS) and Known Exploited Vulnerability (KEV) scores to identify the most critical threats. * Allows teams to monitor risk scores over time to pinpoint specific areas of the infrastructure that require additional resources or immediate intervention. ## Strategic Impact for Security and Development Teams * Enables security leaders to prove program effectiveness to executives by showing downward trends in Common Weakness Enumeration (CWE) types and shrinking backlogs. * Streamlines the developer experience by highlighting critical vulnerabilities within active projects, removing the need for external spreadsheets or manual reporting tools. * Identifies specific teams or departments that may require additional remediation training based on their ability to meet company security policies. Organizations should leverage these updated dashboard features to transition from manual, reactive security tracking to an automated, risk-based posture. By integrating EPSS and KEV data into daily workflows, teams can ensure they are solving the most dangerous vulnerabilities first while maintaining a clear, measurable record of their security improvements.

discordOriginal article

Your Discord Checkpoint is Rolling Out! Celebrate What You Did in 2025 (opens in new tab)

Discord has introduced "Discord Checkpoint," the platform’s first comprehensive year-end recap designed to provide users with a personalized summary of their 2025 activity. By analyzing data such as message counts and voice call duration, the feature offers a nostalgic overview of a user's digital footprint and social interactions over the past year. This initiative marks a shift toward data-driven user engagement, rewarding active community members with exclusive digital collectibles based on their usage patterns. **Accessing the Activity Recap** * The feature is rolling out globally over several days and requires users to be on the latest version of the Discord application. * Desktop users can find their recap by clicking the flag icon located in the top-right corner of the interface. * Mobile users can access the experience via a Checkpoint banner located within the "You" tab at the bottom-right of the screen. * Visibility is contingent upon having "Use data to personalize my Discord experience" enabled in privacy settings and meeting a minimum activity threshold. **Key Metrics and Personal Statistics** * The recap calculates the total volume of messages sent and the cumulative time spent in voice channels throughout the year. * Users receive a breakdown of their most-frequented servers and their most-used emojis. * The system identifies a "top contact," highlighting the individual user with whom the account owner interacted the most. **Personalized Rewards and Social Integration** * Upon completion of the recap, users are assigned one of ten distinct "Checkpoint cards" that categorize their year. * Each card unlocks a corresponding Avatar Decoration that remains available to use until January 15, 2026. * The feature includes a direct sharing toggle that allows users to post a summary card into text channels, though the data remains private by default if the user chooses not to share. To ensure you can view your 2025 Checkpoint before it expires, confirm that your privacy settings allow for data personalization and that your client is fully updated. If the Checkpoint does not appear, you may need to increase your platform activity for future recaps or check the Help Center for specific troubleshooting regarding data permissions.

line4 min readCurated summary

Scaling to Infinity: LY Corporation’s

LY Corporation’s observability team evolved its time-series database to handle rapidly growing infrastructure and Kubernetes workloads. After outgrowing MySQL and OpenTSDB, the team built an engine optimized for high-cardinality metrics, low-latency queries, and seamless API compatibility. Its architecture now combines in-memory, Cassandra, and S3-compatible storage, enabling cost-efficient scaling while supporting trillions of daily metrics. ## Why Time-Series Storage Matters - Metrics record system state as timestamped numerical values. - They support dashboards, threshold-based alerts, and predictive analysis using tools such as ARIMA and Prophet. - Even a small metric record can consume about 280 bytes when timestamps, values, and tags are included. - One CPU metric collected every 15 seconds requires roughly 562 MiB per server annually; across 1,000 servers, this grows to about 548 GiB before adding memory, disk, and network metrics. - High-cardinality cloud environments make both storage cost and query latency critical operational concerns. ## Moving Beyond MySQL and OpenTSDB - MySQL initially became inadequate as the organization moved from SOA to MSA: - Write load increased sharply. - Storage costs and capacity requirements grew. - Query latency worsened for large datasets. - Rigid schemas could not easily represent changing cloud resources. - MySQL sharding provided temporary relief but could not support high-resolution metrics collected at intervals under one minute. - OpenTSDB, introduced in 2016 on Apache HBase, improved write performance but had important limitations: - Tag growth harmed UID-table lookup performance. - Metadata was restricted to a narrow character set. - Large queries required cache warm-up procedures. - These constraints led to the development of an internal database beginning in 2018. ## Building the Internal Time-Series Database - The 2019 engine was designed around: - Flexible protocol support independent of a particular agent. - Linear scalability without downtime. - Low-latency processing of high-resolution metrics. - Strong availability during failures. - Inspired by Meta’s Gorilla research, the team used access patterns in which most queries target recent data. - Frequently accessed metrics were kept in an in-memory database, while colder data was stored in Apache Cassandra. - The new engine enabled metric volumes to grow by more than 200 billion records annually while preserving existing APIs. - Users benefited from the new backend without migration work or code changes. ## Scaling for Kubernetes Workloads - Kubernetes introduced rapidly changing pods, dynamically allocated volumes, and much higher metric churn. - Both major storage layers encountered scaling problems: - IMDB initially required adding identical hardware, limiting expansion options. - Cassandra rebalancing could take tens of hours because of its data volume. - The team improved IMDB with weighted load balancing so nodes with different capacities could be used effectively. - Storage was divided into tiers: - Recent 14-day data remained in Cassandra for high-performance access. - Older data was moved to S3-compatible storage. - This reduced Cassandra dependency, lowered costs, simplified operations, and enabled more flexible hardware and Kubernetes-based deployment. ## Writing and Reading Through S3 - The write path separates data processing from long-term storage: - A Dumper reads metric slots from IMDB. - It converts them into internally defined sub-blocks. - A Block Dumper combines sub-blocks into blocks and writes them to S3. - A Storage Gateway reads the blocks for queries and caches them on local disks. - Disk caching initially caused excessive page-cache use and rapid memory exhaustion. - Direct I/O was considered but withdrawn after the cloud storage team warned that it consumed too much shared bandwidth. - Through cross-team collaboration, the team adopted a B+ tree-based cache that made better use of the kernel page cache without overloading infrastructure. ## Future Direction: From Storage to Intelligence - The team aims to move beyond recording metrics toward prediction and AI-assisted operations. - Achieving this requires consolidating time-series data currently scattered across internal systems. - A key requirement is to perform this integration without imposing migration work or breaking changes on users. - The broader goal is an observability platform that turns unified metrics into predictive and intelligent operational capabilities. The main recommendation is to design time-series platforms around real access patterns, tier storage according to data age, and preserve compatibility while evolving the backend. At extreme scale, careful storage architecture and collaboration across infrastructure teams are as important as raw database performance.

Read original(opens in new tab)
naverOriginal article

Naver TV (opens in new tab)

This session from NAVER ENGINEERING DAY 2025 explores the implementation of visual data tools to interpret complex user behavior within Naver’s Integrated Search. By transforming raw quantitative click logs into intuitive heatmaps and histograms, the development team provides a clearer understanding of how users navigate and consume content. This approach serves as a critical bridge for stakeholders to find actionable evidence for service improvements that are often obscured by traditional data analysis. ### Visualizing User Intent through Heatmaps and Histograms * Click logs from Naver Integrated Search are converted into heatmaps to pinpoint exactly where users are focusing their attention and making their "first clicks." * Histograms are utilized alongside heatmaps to provide a temporal and frequency-based perspective on user interactions, making it easier to identify patterns in data consumption. * The visualization system aims to help developers and designers who struggle with raw quantitative data to gain an immediate, intuitive grasp of user experience (UX) performance. ### Handling Dynamic Data in Real-Time Search Services * The system is designed to respond to the "real-time evolution" of Naver Search, where content and UI layouts change frequently based on trends and algorithms. * The FE Infrastructure team shared technical know-how on collecting and processing client-side logs to ensure data accuracy even as the search interface evolves. * Significant trial and error were involved in developing a visualization framework that remains consistent and reliable across diverse search result types and user devices. ### Practical Application for Service Improvement * By using heatmaps as a primary diagnostic tool, teams can move beyond speculative design and base UI/UX updates on concrete visual evidence of user friction or engagement. * The technology allows for the identification of "dead zones" or overlooked features that may require repositioning or removal to streamline the search experience. * Integrating these visual tools into the development workflow enables faster feedback loops between data analysis and front-end implementation. For organizations managing high-traffic web platforms, moving from raw data tables to visual behavior mapping is essential for understanding the nuance of user interaction. Implementing a robust heatmap and histogram system allows teams to validate product hypotheses quickly and ensures that service updates are driven by actual user behavior rather than just aggregate metrics.

googleOriginal article

DS-STAR: A state-of-the-art versatile data science agent (opens in new tab)

DS-STAR is an advanced autonomous data science agent developed to handle the complexity and heterogeneity of real-world data tasks, ranging from statistical analysis to visualization. By integrating a specialized file analysis module with an iterative planning and verification loop, the system can interpret unstructured data and refine its reasoning steps dynamically based on execution feedback. This architecture allows DS-STAR to achieve state-of-the-art performance on major industry benchmarks, effectively bridging the gap between natural language queries and executable, verified code. ## Comprehensive Data File Analysis The framework addresses a major limitation of current agents—the over-reliance on structured CSV files—by implementing a dedicated analysis stage for diverse data formats. * The system automatically scans a directory to extract context from heterogeneous formats, including JSON, unstructured text, and markdown files. * A Python-based analysis script generates a textual summary of the data structure and content, which serves as the foundational context for the planning phase. * This module ensures the agent can navigate complex, multi-file environments where critical information is often spread across non-relational sources. ## Iterative Planning and Verification Architecture DS-STAR utilizes a sophisticated loop involving four specialized roles to mimic the workflow of a human expert conducting sequential analysis. * **Planner and Coder:** A Planner agent establishes high-level objectives, which a Coder agent سپس translates into executable Python scripts. * **LLM-based Verification:** A Verifier agent acts as a judge, assessing whether the generated code and its output are sufficient to solve the problem or if the reasoning is flawed. * **Dynamic Routing:** If the Verifier identifies gaps, a Router agent guides the refinement process by adding new steps or correcting errors, allowing the cycle to repeat for up to 10 rounds. * **Intermediate Review:** The agent reviews intermediate results before proceeding to the next step, similar to how data scientists use interactive environments like Google Colab. ## Benchmarking and State-of-the-Art Performance The effectiveness of the DS-STAR framework was validated through rigorous testing against existing agents like AutoGen and DA-Agent. * The agent secured the top rank on the public DABStep leaderboard, raising accuracy from 41.0% to 45.2% compared to previous best-performing models. * Performance gains were consistent across other benchmarks, including KramaBench (39.8% to 44.7%) and DA-Code (37.0% to 38.5%). * DS-STAR showed a significant advantage in "hard" tasks—those requiring the synthesis of information from multiple, varied data sources—demonstrating its superior versatility in complex environments. By automating the time-intensive tasks of data wrangling and verification, DS-STAR provides a robust template for the next generation of AI assistants. Organizations looking to scale their data science capabilities should consider adopting iterative agentic workflows that prioritize multi-format data understanding and self-correcting execution loops.

coupangOriginal article

Coupang Rocket Delivery’s spatial index-based delivery management system (opens in new tab)

Coupang’s Rocket Delivery system recently transitioned from a text-based postal code infrastructure to a sophisticated spatial index-based management system to handle increasing delivery density. By adopting Uber’s H3 hexagonal grid system, the engineering team enabled the visualization and precise segmentation of delivery areas that were previously too large for a single driver to manage. This move has transformed the delivery process into an intuitive, map-centric operation that allows for data-driven optimization and real-time area modifications. ### Limitations of Text-Based Postal Codes * While postal codes provided a government-standardized starting point, they became inefficient as delivery volumes grew from double to triple digits per code. * The lack of spatial data meant that segmenting a single postal code into smaller units, such as individual apartment complexes or buildings, required manual input from local experts familiar with the terrain. * Relying on text strings prevented the system from providing intuitive visual feedback or automated metrics for optimizing delivery routes. ### Adopting H3 for Geospatial Indexing * The team evaluated different spatial indexing systems, specifically comparing Google’s S2 (square-based) and Uber’s H3 (hexagon-based) frameworks. * H3 was chosen because hexagons provide a constant distance between the center of a cell and all six of its neighbors, which simplifies the modeling of movement and coverage. * The hexagonal structure minimizes "edge effect" distortions compared to squares or triangles, making it more accurate for calculating delivery radius and area density. ### Technical Redesign and Implementation * The system utilizes H3’s hierarchical indexing, allowing the platform to store delivery data at various resolutions to balance granularity with computational performance. * Delivery zones were converted from standard polygons into "hexagonized" groups, enabling the system to treat complex geographical shapes as sets of standardized cell IDs. * This transition allowed for the creation of a visual interface where camp leaders can modify delivery boundaries directly on a map, with changes reflected instantly across the logistics chain. By shifting to a spatial index, Coupang has decoupled its logistics logic from rigid administrative boundaries like postal codes. This technical foundation allows for more agile resource distribution and provides the scalability needed to handle the continued growth of high-density urban deliveries.

datadog1 min readCurated summary

How we built the Datadog heatmap to visualize distributions over time at arbitrary scale | Datadog

The supplied content does not include the blog post itself; it mainly contains Datadog’s navigation menu and a promotional link announcing its Gartner recognition. The only identifiable article reference is a post about building Datadog’s heatmap for visualizing distributions over time at arbitrary scale, but its body text is missing. ## Available Content - Datadog is promoting its recognition as a Leader in the Gartner® Magic Quadrant™ for Observability Platforms. - The navigation lists Datadog products across: - Infrastructure and application monitoring - Logs, databases, and data observability - Security - Digital experience monitoring - Software delivery - Incident and service management - AI and platform capabilities - The referenced engineering post appears to discuss: - A heatmap visualization - Distributions over time - Scaling to arbitrary data volumes A meaningful technical summary would require the actual article text, which is not present in the supplied content.

Read original(opens in new tab)
datadog3 min readCurated summary

How we built the Datadog heatmap to visualize distributions over time at arbitrary scale

Datadog uses DDSketch-powered distribution metrics and heatmaps to reveal performance patterns that percentile lines can hide. Heatmaps preserve the full shape of latency distributions across hosts and time, making distinct behavioral modes, seasonality, and outliers visible. The visualization is designed to remain scalable and readable even with hundreds of trillions of underlying datapoints. ## Why Unaggregated Distributions Matter - Line graphs reduce billions of events to a single value, such as p50, p99, or max. - Multiple percentile lines provide more context, but the selected percentiles remain arbitrary and can obscure important behavior. - Aggregated percentile changes may suggest that all requests are slowing when only one subset of traffic is changing. - Heatmaps expose separate “modes”—distinct groups of measurements with different behavior. - For example, periodic latency spikes may come from a low-latency benchmarking service rather than from a general degradation in the endpoint. - Filtering out an identified mode can reveal other patterns, such as daily seasonality in the remaining traffic. ## Building Heatmaps with DDSketch - DDSketch sacrifices a small amount of precision to represent extremely large numbers of observations efficiently. - Datadog sends histogram bins and counts to the frontend instead of transmitting every individual datapoint. - Limiting the number of bins keeps the payload size constant as traffic volume grows. - Counts use `float32`, supporting values up to approximately `3 × 10^38` per bin—far beyond practical monitoring volumes. - This allows heatmaps to represent massive datasets, including hundreds of trillions of datapoints. ## Preserving Resolution and Avoiding Aliasing - Heatmap requests contain time buckets, distribution bins, and counts. - Since bucket boundaries are shared across a request, Datadog stores those boundaries only once. - Boundaries must be explicit because distributions may use logarithmic rather than linear scales. - Time buckets need to align with the source data intervals. - Misaligned intervals create aliasing artifacts: for example, grouping 10-second data into 7-second buckets produces repeating count patterns such as `[1, 1, 2, 1, 1, 2, …]`. - Careful discretization preserves the resolution available in the original DDSketch data. ## Designing the Color Scale - The default palette begins with light blue, consistent with other single-series Datadog visualizations. - It transitions toward purple to match Datadog’s visual identity. - The scale avoids lingering on red, which can imply negative alerts, and ends in orange for the hottest values. - Color choices must communicate both the volume and structure of the distribution. ## Maintaining Dynamic Range - A few high-count bins can dominate a linear color scale, leaving most of the heatmap visually indistinguishable. - This is especially problematic for power-law distributions with a dense central mode and a long tail. - A linear scale may clearly show the main mode around 20 ms while hiding a smaller mode near 1 second. - Human brightness perception is nonlinear, approximately following a power law described by Stevens’ law. - Applying nonlinear color interpolation improves the visibility of meaningful differences across both dense regions and long tails. - This helps preserve distribution details that would otherwise be lost when the color range is dominated by outliers or highly concentrated buckets. Datadog’s heatmap approach combines DDSketch compression, aligned high-resolution buckets, and perceptually informed color scaling. For systems where averages or a handful of percentiles conceal important subpopulations, distribution heatmaps provide a more reliable way to investigate performance at scale.

Read original(opens in new tab)
datadog3 min readCurated summary

How we brought Datadog's data visualization to iOS: A focus on performance

Datadog built DogGraphs, a native SwiftUI graphing library, to support complex data visualizations across its iOS app and widgets. Because existing libraries did not meet its needs and the app supported iOS 14, the team had to optimize SwiftUI rendering without newer APIs such as `Canvas`. By combining careful API design, profiling, and a better understanding of SwiftUI’s update model, DogGraphs became a reusable framework used across multiple Datadog products. ## Building DogGraphs for Complex Visualizations - DogGraphs began with the Service Catalog and was designed to support additional Datadog products. - It needed to provide: - Native Swift and SwiftUI rendering - iOS 14 compatibility - Flexible, easy-to-use APIs - Datadog’s default visual style and behavior - Fast rendering for a responsive user experience - The library now powers visualizations in logs, services, dashboards, Bits AI, and mobile widgets. - It supports increasingly diverse graph types as new products integrate with the mobile application. ## A Declarative, Type-Safe API - DogGraphs uses Swift features such as result builders to describe complex graph configurations declaratively, in a style similar to SwiftUI. - Graph definitions can be generated dynamically from server-provided dashboard or widget configurations. - Compile-time type checking prevents invalid combinations, such as stacking incompatible Bar and Line graphs. - Progressive disclosure provides sensible Datadog defaults while still allowing customization when necessary. ## Profiling SwiftUI Performance Datadog’s visualizations can involve metrics, logs, traces, multiple aggregation strategies, arithmetic operations, axes, labels, scales, and color configuration. Query responses are preprocessed by a shared internal service so that formatting and visual behavior remain consistent across platforms. To optimize rendering, the team focused on two primary measurements: - **SwiftUI view body evaluations** - Excessive body evaluations can degrade performance, especially when many views are involved. - Expensive computation should be moved outside view bodies. - `_printChanges()` can reveal why a view is being reevaluated, though it is a private API unsuitable for production use. - **Time Profiler** - Instruments helps identify slow function calls and locate expensive work in the rendering pipeline. Important profiling scenarios included: - Initial graph rendering - Updates caused by window changes, tooltip selection, or layer visibility changes - Device rotation and light/dark mode changes - Interactions with unrelated views such as scroll views, toggles, and buttons ## Understanding SwiftUI’s Update Model The team used Apple’s “Demystify SwiftUI” session to build a mental model for how SwiftUI determines when views should update. - **Identity:** How SwiftUI determines whether an element is the same as, or different from, a previous element. - **Lifetime:** How SwiftUI tracks a view and its associated data over time. - **Dependencies:** How SwiftUI determines which changes require an interface update. - **Diffing:** SwiftUI compares view values to determine what changed, although the exact diffing mechanism is undocumented. Understanding these concepts helps developers explain unexpected view updates and identify the sources of rendering bottlenecks. ## Practical Recommendation For complex SwiftUI components, measure real interaction scenarios rather than relying on assumptions. Track body evaluations and expensive function calls, keep costly work out of `body`, and design APIs that provide efficient defaults while preserving type safety and flexibility.

Read original(opens in new tab)
coupangOriginal article

Coupang Rocket Delivery: A (opens in new tab)

Coupang transitioned its Rocket Delivery management from a text-based zip code system to a spatial index-based system using Uber’s H3 library. This shift addresses the limitations of zip codes, which became too coarse for high-density delivery areas, by enabling precise, map-based visualization and manipulation of delivery zones. By adopting a hexagonal grid-based approach, Coupang has improved operational flexibility and its ability to handle complex urban delivery environments. ### The Limitations of Zip Code Systems * Zip codes originally served as the base unit for Rocket Delivery, but as delivery volumes scaled, individual codes became too large for a single driver to manage. * Sub-dividing these areas (e.g., splitting a zip code into specific apartment complexes or even individual buildings) required the manual expertise of senior managers because text-based addresses lack inherent spatial intelligence. * The previous reliance on text made it difficult to visualize delivery boundaries or reassign areas quickly in response to changes in order volume. ### Implementing H3 for Geospatial Indexing * To modernize the system, Coupang adopted H3, a hexagonal hierarchical geospatial indexing system that converts geographic coordinates into unique cell identifiers. * Hexagons were selected over square grids because they provide uniform distances between the center of a cell and all its neighbors, which minimizes distortion in distance-based calculations. * The system uses H3’s hierarchical structure to manage different levels of detail, allowing the platform to aggregate small hexagonal units into larger, custom-defined delivery polygons. ### Technical Challenges in System Redesign * A primary engineering hurdle was selecting the optimal grid resolution to ensure cells were small enough to capture individual building footprints without creating excessive data overhead. * The team developed algorithms to transform groups of hexagonal indices into filled polygons, enabling camp managers to "draw" and modify delivery zones directly on a digital map. * By basing the system on spatial coordinates rather than administrative text, the platform can dynamically adjust to urban changes, such as the construction of new high-rises or the demolition of old structures. Transitioning from text-based addressing to hexagonal indexing allows logistics platforms to move beyond the constraints of administrative boundaries. For high-density urban delivery services, adopting a spatial-first infrastructure like H3 is a necessary step to ensure scalability and operational precision.

figma3 min readCurated summary

Inside Maker Week: more than a hackathon | Figma Blog

Maker Week is Figma’s twice-yearly, company-wide alternative to a traditional hackathon. It gives employees across disciplines, teams, and time zones dedicated time to pursue creative projects without requiring technical expertise. The program has produced hundreds of ideas since 2018, many of which became product features, community tools, or cultural initiatives. ## A Maker Culture Open to Everyone - Figma defines “maker” broadly—not just as an artist, engineer, or craftsperson, but as anyone expressing creativity in an authentic way. - The company reinforces this culture during weekly All Hands meetings, where new employees explain how they are makers. - This inclusive definition allows people who see themselves as curators, organizers, educators, or collaborators to participate. ## Maker Week as a Cross-Functional Festival - Maker Week happens twice a year and pauses normal day-to-day work for a week of experimentation and connection. - Unlike many hackathons, which can favor engineering, product, and design teams, Maker Week invites employees from every function. - Projects have included: - Recruiter-created employee baseball cards for candidates. - An intern-built “Name That Figmate” game. - A chemistry lesson taught through baking by data scientists. - The format encourages cross-pollination, shared ownership, and a better understanding of how different teams contribute to Figma. ## From Experiments to Shipped Work - Figma has held five Maker Weeks since 2018, generating hundreds of projects. - Some projects have become externally released products or features, including: - Auto Layout. - Interactive components. - Figma plugins. - The company suggests that FigJam itself may represent the culmination of multiple Maker Week experiments. - Other projects, such as *Figma in Quarantine: The Musical*, have offered glimpses into Figma’s internal culture. ## Featured Maker Week Projects ### Empathy-Building Cards - Research, sales enablement, and brand design employees created cards combining user personas with different characteristics and situations. - The cards help Figmates better understand users’ perspectives and contextualize the challenges they face. - The project was shared with the broader Figma community. ### Figma in 3D - An engineer and product support employee collaborated to create an extension that projects layers off the canvas. - Users can orbit around the design with a mouse, making Figma feel more like a 3D mapping application. - The project explored new possibilities for spatial software. ### User Collaboration Data Visualization - A data scientist and enterprise sales employee built a Mode-based map showing collaboration between users in different countries within the same domain. - The visualization helps organizations understand how teams collaborate across regions and time zones. Maker Week demonstrates that innovation does not have to come only from formal product teams. Figma’s approach recommends creating structured time for broad participation, experimentation, and cross-functional collaboration—while allowing the most promising ideas to evolve into real products or cultural initiatives.

Read original(opens in new tab)
figma2 min readCurated summary

Adding it all up: The math behind designing your career | Figma | Figma Blog

A career is less a predetermined trajectory than a collection of experiments, interests, and opportunities whose meaning becomes clearer in hindsight. Kylie Poppen argues that because work occupies roughly 90,000 hours of our lives, we should actively examine the patterns shaping our choices rather than assume a single “destiny.” Mathematical metaphors—scatterplots and Venn diagrams—offer tools for making more intentional career decisions. ## Careers as Scatterplots - People often describe careers as trend lines, connecting past experiences into a coherent narrative. - In reality, careers are lived as scatterplots: a series of moves between interests, jobs, and opportunities. - The next point cannot reliably be predicted from the current one; clarity usually comes only afterward. - Regularly reflecting on experiences can help reveal a more accurate career trajectory over time. - Work deserves careful consideration because it consumes about one-third of waking life and affects financial stability, purpose, frustration, pride, and identity. ## Finding Meaning Through Overlap - Venn diagrams illustrate how combining different interests can reveal unexpected possibilities and innovations. - Career choices do not need to force a decision between passion and practicality; meaningful work can emerge where hobbies, skills, and interests intersect. - Poppen’s childhood interest in designing posters with CorelDRAW, combined with her passions for storytelling, technology, and people, eventually pointed toward product design. - Her path was not direct: she explored recruiting, marketing, engineering, and project management before gaining the confidence to pursue design. - Mentorship and constructive feedback helped her see design as a craft developed through practice rather than an innate destiny. - Mapping personal interests can uncover overlooked career options and provide confidence to pursue a desired direction. A practical approach is to treat your career as an evolving set of data points, then look for recurring patterns and intersections among your interests, abilities, and experiences. This can lead to more fulfilling choices without requiring certainty about the final destination.

Read original(opens in new tab)