github·Jun 19, 2026·3 min readCurated summary

How we built an internal data analytics agent

Qubot is GitHub’s internal, Copilot-powered analytics agent, designed to make warehouse data accessible without requiring an analyst. Employees ask natural-language questions through Slack, VS Code, or the Copilot CLI, while Qubot uses curated context and selects between Kusto and Trino to produce answers. GitHub’s experience shows that structured context is central to improving both accuracy and response speed. ## The Problem Qubot Addresses - Product teams often struggle to identify the right data model, grain, filters, and query. - Limited analyst availability leaves many teams to investigate telemetry independently. - Qubot targets exploratory questions rather than replacing dashboards or formal reporting. - It enables employees to investigate unfamiliar datasets with minimal setup and maintenance. ## Multiple Ways to Ask Questions - **Slack:** Users ask questions in a shared channel, receive answers in threads, and collaborate on follow-up questions. - Each result is saved as a Markdown report in a pull request, allowing users to refine queries or reuse them in dashboards. - **VS Code and Copilot CLI:** Qubot is installed as a plugin and operates alongside other agents, skills, and tools. - Offering both zero-configuration and developer-focused interfaces increased adoption among different user groups. ## A Federated Context Layer Qubot’s context is tailored to GitHub’s warehouse layers: - **Bronze:** Product teams provide telemetry schemas and metadata for raw events. - **Silver:** Data teams maintain query examples, usage guidance, and required filters for conformed data. - **Gold:** Dataset owners contribute business rules and metric definitions. - ETL pipelines add derived metadata and other signals automatically. - Context is fetched at runtime through the GitHub MCP Server. A dedicated context agent standardizes contributions from different teams. It ingests Markdown documentation and repository references, then organizes and normalizes them into a structure that Qubot can use effectively. ## Evaluation Before Deployment Every change to Qubot’s instructions or context layer is tested through an offline evaluation framework. - Test cases contain prompts, expected answers, ground-truth SQL, domains, and difficulty levels. - Automated orchestration launches multiple parallel agent trials using `gh agent-task create`. - Results are saved as JSON and aggregated by completion rate, accuracy, and duration. - Teams compare configurations and detect regressions before releasing changes. ## Choosing Between Kusto and Trino Qubot connects to both query engines through MCP servers: - **Kusto** is used for fast exploration of recent event data. - **Trino** handles complex joins and deeper historical analysis. - Qubot defaults to Kusto and switches to Trino when the question requires more advanced or historical querying. - This hides query-engine selection from users. ## Adoption and Lessons Learned - Hundreds of GitHub employees have run thousands of queries through Qubot. - Routine questions in analytics support channels declined as teams gained more autonomy. - The agent helped employees who previously avoided the warehouse access data for decision-making. - GitHub found that well-structured, carefully maintained context improved accuracy and made Qubot approximately three times faster at finding the right answer. The main recommendation is to treat analytics context as a maintained engineering asset. A capable agent depends not only on model intelligence, but also on accurate metadata, documented business rules, evaluation-driven iteration, and interfaces that fit users’ existing workflows.

Read original(opens in new tab)

Data AnalyticsAI AgentsModel Context ProtocolSlack+4

toss·Jun 16, 2026·4 min readCurated summary

User Segmentation for Understanding 28 Million MAU, TUES

Toss developed TUES (Toss User Engagement Segment) to analyze its 28 million monthly active users from a platform-wide perspective. It groups users by their service-use patterns, enabling Toss to understand user motivations, design segment-specific strategies, and explain changes in company-wide metrics. TUES V2 improves on the original by capturing usage depth, multi-service behavior, and engagement with individual service categories. ## Platform-Wide User Segmentation - Service-specific segments such as “users of Service A” are not mutually exclusive or collectively exhaustive because users may use multiple services. - TUES groups users with similar patterns across Toss’s entire service ecosystem. - It helps identify: - Which services users primarily use - How engaged they are with the app - Which user groups may be suitable for particular growth or marketing strategies ## How TUES V1 Worked - Toss calculated each user’s service-use rate per app open. - For example, a user who opened the app 60 times and used Toss Pay during 20 of those sessions had a 33% usage rate. - Users with similar service-usage distributions were grouped using K-Means clustering. - The raw clusters were interpreted and renamed to make them more useful for product and strategy teams. - V1 included: - **Highly engaged users:** Users who regularly use several services - **Service-oriented users:** Users primarily focused on Toss Bank, Toss Securities, inquiry services, benefits, transfers, or other services - **Simple visitors:** Users who open the app but rarely use its services ## How Toss Uses TUES - **Transition strategy:** Teams can plan how to move users from simple visits to service-oriented engagement and eventually to highly engaged usage. - **Product growth:** Product teams can quickly identify which user segments use their service most and combine that insight with transition strategies. - **Behavior analysis:** TUES reveals when users change segments, begin churning, or return after inactivity. - **Top-line metric analysis:** When MAU changes, Toss can identify which user segments moved and which services likely caused the change. - **Targeted marketing:** Marketers use TUES segments for campaigns such as push notifications. The segments are also available in Toss’s internal marketing tool, TUBA. ## Limitations of TUES V1 After roughly two years of use, Toss identified several weaknesses: - V1 measured only the probability of using a service during an app open, not the number of times it was used. - Users who engaged with a service once and users who used it ten times could appear equivalent. - It could not show engagement with secondary service categories. - K-Means is a hard-clustering method, so each user belonged to only one segment despite often using multiple services. - New major services, including Toss Shopping, App in Toss, and Toss Pay, were grouped into a generic “ETC” category. ## TUES V2 Improvements - **Usage-depth measurement:** V2 uses the number of service interactions per app open as a feature, capturing the intensity of engagement. - **Soft clustering:** Instead of assigning each user to one segment, V2 calculates each user’s degree of association with multiple segments and selectively uses those results. - **Three-layer structure:** Users are described through: 1. Overall app engagement 2. Primary service orientation 3. Engagement with each individual service category - The layers are built sequentially, making it clearer why a user belongs to a segment and what action may be appropriate next. ## New Strategic Capabilities in V2 - Teams can identify which service-category engagement should increase first to move users from a semi-engaged segment to a highly engaged one. - Individual service teams, or silos, can quantitatively connect actions that increase service engagement with company-wide segment and performance changes. - Products can more clearly compare the engagement profiles of users who do and do not use their services. - Cross-activation strategies now have a more precise starting point based on service-level engagement. ## Future Development Toss plans to combine TUES with additional analytical frameworks to: - Create faster and more detailed transition strategies using concepts such as service similarity. - Build strategic user maps based on user profiles and service-use patterns. - Quantify segment-specific value by combining TUES with frameworks such as MTVi. TUES demonstrates how platform-level segmentation can make a growing MAU base easier to understand and act upon. By combining overall engagement, primary service use, and service-level depth, TUES helps Toss develop more targeted growth strategies and connect individual product actions to broader company outcomes.

Read original(opens in new tab)

Data AnalyticsMachine LearningSoft ClusteringUser Segmentation+4

aws·Mar 13, 2026·3 min readCurated summary

Twenty years of Amazon S3 and building what’s next | Amazon Web Services

Amazon S3 began in 2006 as a simple web service for storing and retrieving objects, but its emphasis on security, durability, availability, performance, and elasticity enabled it to become foundational infrastructure. Over two decades, it scaled from roughly one petabyte to hundreds of exabytes while preserving API compatibility, reducing prices, and expanding beyond object storage. Amazon’s long-term vision is for S3 to serve as a universal foundation for data, analytics, and AI workloads. ## The Original S3 Philosophy - S3 introduced two basic operations: - `PUT` to store an object - `GET` to retrieve it - The service abstracted away complex infrastructure so developers could focus on applications. - Its five enduring design principles are: - **Security:** Data is protected by default. - **Durability:** Designed for 11 nines of durability, with a lossless operating model. - **Availability:** Failure is assumed and handled throughout the system. - **Performance:** Storage capacity can grow without degrading performance. - **Elasticity:** Capacity expands and contracts automatically. ## From One Petabyte to Hundreds of Exabytes - At launch, S3 had approximately: - One petabyte of capacity - 400 storage nodes across 15 racks and three data centers - 15 Gbps of bandwidth - A maximum object size of 5 GB - A price of $0.15 per GB - Today, S3: - Stores more than 500 trillion objects. - Serves over 200 million requests per second. - Operates across 123 Availability Zones in 39 AWS Regions. - Supports objects up to 50 TB—10,000 times larger than the original limit. - Storage prices have fallen by roughly 85%, to slightly above 2 cents per GB. - S3 Intelligent-Tiering has saved customers more than $6 billion in storage costs. - The S3 API has become an industry standard, with many other storage systems offering compatible interfaces. ## Backward Compatibility and Long-Term Reliability - Code written against S3 in 2006 still works without modification. - AWS has repeatedly replaced disks, storage systems, and request-processing code while preserving access to older data. - This compatibility reflects S3’s goal of remaining infrastructure that “just works” despite continuous internal change. ## Engineering for Durability and Scale - Microservices continuously inspect every byte across the fleet. - Auditor services detect degradation and automatically trigger repair and re-replication. - Automated formal methods mathematically verify correctness in areas such as: - The index subsystem - Cross-Region replication - Access policies - AWS has progressively rewritten performance-critical components in Rust over the past eight years. - Rust improves performance while preventing memory-safety bugs and other classes of errors at compile time. - S3 follows the principle that scale should improve the service: larger, more distributed workloads become increasingly decorrelated, improving reliability for all customers. ## S3 as a Foundation for Data and AI Amazon’s future vision is for customers to store data once in S3 and work with it directly, avoiding costly copies and specialized systems. - **S3 Tables** provides managed Apache Iceberg tables with automated maintenance to improve query performance and reduce storage costs. - **S3 Vectors** supports semantic search and retrieval-augmented generation, with up to 2 billion vectors per index and sub-100 ms query latency. - Within five months of launch, customers created over 250,000 indexes, ingested more than 40 billion vectors, and executed over 1 billion queries. - **S3 Metadata** enables centralized, faster data discovery without recursively listing large buckets. These additions extend S3 from inexpensive object storage into a broader platform for analytics, search, and AI while retaining its scale and cost advantages.

Read original(opens in new tab)

Data AnalyticsDatabase DesignRustAPI+4

google·Mar 12, 2026·3 min readCurated summary

Protecting cities with AI-driven flash flood forecasting

Google Research is expanding Flood Hub with urban flash flood forecasts that can provide up to 24 hours’ warning. The system addresses the lack of historical flood observations by using Gemini to extract verified events from public news reports, creating the Groundsource dataset for model training. Its global, lower-resolution approach aims to extend useful warnings to regions that lack expensive sensors and forecasting infrastructure, particularly in the Global South. ## The Need for Earlier Flash Flood Warnings - Flash floods cause roughly 85% of flood-related deaths worldwide and kill more than 5,000 people annually. - They often develop within six hours of intense rainfall, making rapid warnings essential. - Even 12 hours of warning can reduce flood damage by about 60%. - Early warning coverage remains highly unequal: fewer than half of developing countries have access to multi-hazard warning systems. - Flood Hub previously focused mainly on slower-moving riverine floods, covering more than 2 billion people across 150 countries. ## The Data Problem: “Invisible” Floods - River flood models can rely on stream gauges that record water levels and flow. - Flash floods may occur far from gauges, especially in cities where rainfall, impermeable surfaces, drainage, and terrain interact unpredictably. - Building detailed physical simulations globally would be computationally expensive. - Historical, precisely located flash flood records are also scarce, preventing conventional supervised machine learning. - Google’s Groundsource method uses Gemini to analyze public news reports, verify flood locations and times, and assemble a historical flash flood dataset. ## Scaling from Local Systems to Global Coverage - Local flash flood systems can be highly accurate using rain sensors, radar, water-level monitors, and flow measurements. - These systems are expensive to deploy and require location-specific calibration and engineering expertise. - Broader systems such as WMO’s FFGS, ERIC, and the U.S. NWS warning system depend on high-resolution maps, radar forecasts, and skilled hydrologists. - Those resources are often unavailable in the Global South. - Google’s model instead uses globally available products, including NASA IMERG, NOAA CPC, ECMWF’s IFS HRES forecasts, and Google DeepMind’s medium-range weather model. - Forecasts currently operate at a 20-by-20-kilometer resolution, constrained by the resolution of global data sources. ## The Urban Flash Flood Model - The model estimates whether a flash flood is likely in a given area during the next 24 hours. - It uses a recurrent neural network with a long short-term memory (LSTM) component to process meteorological time series. - Inputs also include static geographic and human-environment factors: - Urbanization density - Topography - Soil absorption rates - The initial rollout targets urban regions, where news coverage is denser and most of the world’s population lives. - It currently predicts impacts in areas with population densities above 100 people per square kilometer. ## Evaluation and Reported Performance - Precision was measured against the Groundsource dataset, but raw precision likely understates actual performance because some genuine floods are never reported. - A manual review of 100 alerts per continent found that many apparent false positives were confirmed flood events. - Recall was also evaluated against major floods recorded by the Global Disaster Awareness and Coordination System (GDACS). - Results indicate comparable precision and recall in regions such as South America and Southeast Asia and in wealthier countries with better instrumentation. The approach demonstrates how AI and unstructured public information can help provide scalable flash flood warnings where conventional monitoring infrastructure is limited. Its current urban focus and 20-kilometer resolution make it a broad early-warning tool rather than a replacement for highly localized sensor networks.

Read original(opens in new tab)

Data AnalyticsAIMachine LearningFlood Forecasting+4

discord·Feb 18, 2026Original article

Your Discord Checkpoint is Rolling Out! Celebrate What You Did in 2025 (opens in new tab)

Discord has introduced "Discord Checkpoint," the platform’s first comprehensive year-end recap designed to provide users with a personalized summary of their 2025 activity. By analyzing data such as message counts and voice call duration, the feature offers a nostalgic overview of a user's digital footprint and social interactions over the past year. This initiative marks a shift toward data-driven user engagement, rewarding active community members with exclusive digital collectibles based on their usage patterns. **Accessing the Activity Recap** * The feature is rolling out globally over several days and requires users to be on the latest version of the Discord application. * Desktop users can find their recap by clicking the flag icon located in the top-right corner of the interface. * Mobile users can access the experience via a Checkpoint banner located within the "You" tab at the bottom-right of the screen. * Visibility is contingent upon having "Use data to personalize my Discord experience" enabled in privacy settings and meeting a minimum activity threshold. **Key Metrics and Personal Statistics** * The recap calculates the total volume of messages sent and the cumulative time spent in voice channels throughout the year. * Users receive a breakdown of their most-frequented servers and their most-used emojis. * The system identifies a "top contact," highlighting the individual user with whom the account owner interacted the most. **Personalized Rewards and Social Integration** * Upon completion of the recap, users are assigned one of ten distinct "Checkpoint cards" that categorize their year. * Each card unlocks a corresponding Avatar Decoration that remains available to use until January 15, 2026. * The feature includes a direct sharing toggle that allows users to post a summary card into text channels, though the data remains private by default if the user chooses not to share. To ensure you can view your 2025 Checkpoint before it expires, confirm that your privacy settings allow for data personalization and that your client is fully updated. If the Checkpoint does not appear, you may need to increase your platform activity for future recaps or check the Help Center for specific troubleshooting regarding data permissions.

Data AnalyticsUser ExperienceDiscordData Visualization+3

aws·Dec 2, 2025Original article

Amazon CloudWatch introduces unified data management and analytics for operations, security, and compliance (opens in new tab)

Amazon CloudWatch has evolved into a unified platform for managing operational, security, and compliance log data, significantly reducing the need for redundant data stores and complex ETL pipelines. By standardizing ingestion through industry-standard formats like OCSF and OpenTelemetry, the service enables seamless cross-source analytics while lowering operational overhead and storage costs. This update allows organizations to move away from fragmented data silos toward a centralized, Iceberg-compatible architecture for deeper technical and business insights. **Data Ingestion and Schema Normalization** * Automatically collects AWS-vended logs across accounts and regions via AWS Organizations, including CloudTrail, VPC Flow Logs, WAF access logs, and Route 53 resolver logs. * Includes pre-built connectors for a wide range of third-party sources, such as endpoint security (CrowdStrike, SentinelOne), identity providers (Okta, Entra ID), and network security (Zscaler, Palo Alto Networks). * Utilizes managed Open Cybersecurity Schema Framework (OCSF) and OpenTelemetry (OTel) conversion to ensure data consistency across disparate sources. * Provides built-in processors, such as Grok for custom parsing and field-level operations, to transform and manipulate strings during the ingestion phase. **Unified Architecture and Cost Optimization** * Consolidates log management into a single service with built-in governance, eliminating the need to store and maintain duplicate copies of data across different tools. * Introduces Apache Iceberg-compatible access via Amazon S3 Tables, allowing data to be queried in place by external tools. * Removes the requirement for complex ETL pipelines by providing a unified data store that is accessible to Amazon Athena, Amazon SageMaker Unified Studio, and other Iceberg-compatible analytics engines. **Advanced Analytics and Discovery Tools** * Supports multiple query interfaces, allowing users to interact with logs using natural language, SQL, LogsQL, or PPL (Piped Processing Language). * The new "Facets" interface enables intuitive filtering by application, account, region, and log type, featuring intelligent parameter inference for cross-account queries. * Enables the correlation of operational logs with business data from third-party tools like ServiceNow CMDB or GitHub to provide a more comprehensive view of organizational health. Organizations should leverage these unified management features to consolidate their security and operational monitoring into a single source of truth. By adopting OCSF normalization and the new S3 Tables integration, teams can reduce the technical debt associated with managing multiple log silos while improving their ability to run cross-functional analytics.

Data AnalyticsLog ManagementAmazon S3Apache Iceberg+4

toss·Nov 19, 2025Original article

Beyond LTV: MTV (opens in new tab)

Toss has developed MTVi (Mid-term Value - incremental) to quantify the financial impact of specific services within its platform, moving beyond the limitations of traditional LifeTime Value (LTV). By focusing on the incremental value generated over a one-year period, the metric allows the company to justify services that may lose money individually but drive significant ecosystem-wide growth. This framework provides a data-driven standard for prioritizing features and setting marketing budgets based on actual financial contributions. ### Limitations of Traditional LTV * **Time Horizon Mismatch:** Traditional LTV projects value over 3 to 5 years, which is too slow for Toss’s rapid iteration cycles and fails to reflect the immediate impact of service improvements. * **Investment Recovery Gaps:** Standard LTV models often benchmark marketing costs (CAC) against long-term projections, making it difficult to evaluate the efficiency of short-term experiments. * **Lack of Incrementality:** LTV measures average user value but cannot isolate the specific "extra" value created by a single service, making it impossible to distinguish between a service's impact and natural user growth. ### Defining MTVi and DID Methodology * **Incremental Focus:** MTVi is defined as the net financial value generated over one year specifically because a user experienced a new service, rather than just the average revenue of a user. * **Quasi-Experimental Design:** Since A/B testing every service combination is impossible, Toss uses the Difference-in-Difference (DID) method to compare "Newly Activated Users" (NAU) against "Never" users. * **Segment-Based Analysis:** To prevent bias—such as highly active users naturally gravitating toward more services—Toss segments users by age and historical activity (e.g., app open frequency) to ensure "apples-to-apples" comparisons within identical cohorts. ### Organizational Impact and Strategy * **Unified Decision Metric:** MTVi provides a "common language" for different product teams (silos), allowing them to compare the value of disparate services—like pedometers versus remittances—on a single financial scale. * **Efficiency Benchmarking:** The metric establishes a hard ceiling for investment; for example, Customer Acquisition Cost (CAC) is strictly managed so it does not exceed the calculated MTVi. * **Platform-Wide Valuation:** By calculating both direct revenue and indirect spillover effects, Toss can prove the financial viability of "loss-leader" services that provide user benefits but increase overall app engagement and cross-service usage. For organizations operating complex multi-service platforms, adopting an incremental value metric like MTVi is essential for moving beyond isolated P&L statements. Data teams should prioritize quasi-experimental methods like DID and rigorous user segmentation to accurately map how individual features influence the broader financial health of the ecosystem.

Data AnalyticsCausal InferenceDifference In DifferenceUnit Economics+4

figma·Feb 10, 2025·2 min readCurated summary

Making Metrics Matter | Design Systems 104 | Figma Blog

Design systems create measurable business value, not just reusable components and documentation. Figma argues that tracking adoption, usage, consistency, and time savings helps teams prove that value and identify where systems need improvement. The evidence is significant: designers with access to a design system completed tasks 34% faster, while some companies reported time savings of up to 50%. ## Measuring the Value of Design Systems - Figma’s research found that design-system users completed tasks **34% faster** than those without access. - For a team of seven designers, this efficiency is comparable to adding **3.5 designers’ worth of capacity each week**. - Participants also felt more confident because the system helped ensure their work aligned with the broader product. - Vanguard reported design updates happening **50% faster** with effective systems. ## Adoption and Usage as Business Signals - Creating components and documentation is only the starting point; measuring actual adoption reveals whether the system is delivering value. - Headspace reported: - **20–30% time savings** on straightforward tasks. - Up to **50% savings** on complex projects through tokens and variables. - Swiggy cut feature rollout time in half after implementing stronger tracking. - These results connect design-system usage to improved consistency, efficiency, and scalability. ## Reading the Signals Organizations should choose metrics that reflect their goals, including: - **Library and component usage:** Track which components, variables, and styles are frequently used. - High usage identifies the system’s most valuable “workhorses.” - Low usage can reveal elements that need improvement, replacement, or deprecation. - Tracking time saved through component reuse provides a concrete benefit that can help secure stakeholder support. A practical approach is to measure not only what a design system contains, but how consistently and effectively teams use it.

Read original(opens in new tab)

Data AnalyticsFigmaDesign SystemsDesign Tokens+2

figma·Feb 2, 2023·3 min readCurated summary

How Pinterest’s design systems team measures adoption | Figma Blog

Pinterest’s Gestalt design systems team created a “design adoption” metric to understand how widely its components are used during the design phase, not just after implementation. Code-based adoption metrics were limited to web components and often lagged behind design activity, while raw Figma instance counts lacked context. Using Figma’s REST API, the team built FigStats to measure Gestalt usage relative to all content in Pinterest design files. ## Why Code Adoption Wasn’t Enough - Gestalt’s existing adoption metric tracked component usage in code. - Code provided concrete data about which components shipped and whether teams modified them. - However, it had two limitations: - It only covered web components, not Gestalt’s iOS and Android components. - It took time for newly designed components to reach production code. - Measuring adoption in Figma provided earlier insight into whether designers knew about and used Gestalt components across all platforms. ## Defining a More Meaningful Adoption Metric - Figma’s built-in library analytics reported: - Component instances - Component insertions - Usage by team - These were useful counts but did not indicate whether usage was significant. - For example, 10 Gestalt components in a 1,000-node design represent only 1% adoption, even though the raw usage count is nonzero. - Pinterest therefore defined adoption relatively: Gestalt usage compared with the total content in a design file or page. - This approach helped distinguish isolated component use from meaningful reliance on the design system. ## Building FigStats with the Figma REST API - The Gestalt team used Figma’s REST API to inspect design files and identify layers originating from Gestalt libraries. - FigStats aggregated this information into a dashboard for visualizing component usage. - The dashboard enabled the team to explore adoption across Pinterest’s design work rather than relying only on manually selected examples. - Measuring actual file content also helped reveal whether designers were using complete Gestalt components or recreating and modifying patterns themselves. ## Using Adoption Data to Guide the Design System - Design adoption became a way to evaluate the value and reach of Gestalt. - Higher usage could demonstrate that investment in corresponding engineering components was justified. - Low adoption could indicate that: - Designers were unaware of an existing component. - The component did not meet their needs. - Documentation or discoverability needed improvement. - A component required redesign or better cross-platform support. - Because design precedes implementation, Figma usage could serve as an earlier signal than production code adoption. Figma’s native analytics provide a useful starting point, but meaningful adoption measurement requires context. Teams should compare design-system usage with the total design surface, use the data to identify gaps, and treat Figma adoption as a complementary metric to code adoption.

Read original(opens in new tab)

Data AnalyticsFigmaDesign SystemsRest Api+2

figma·Oct 12, 2022·2 min readCurated summary

How we use data | Figma Blog

Figma uses data in two broad ways: functional data to provide its service and analytics data to improve the product. The company emphasizes collecting only the information necessary to operate Figma, while using aggregated usage insights and experiments to guide product decisions. Examples include improving file sharing and identifying performance problems in the iOS app. ## Functional Data - Functional data supports core account and product operations. - Figma collects a relatively small amount of information at signup: - Email address - Name - Role - This information enables usernames, password-reset messages, file creation, and collaboration. - Figma generally does not require sensitive information such as identity documents or verification. - Payment information for paid plans is collected and processed by Stripe. ## Analytics Data - Analytics data describes how users access and use Figma. - It helps teams understand: - Which features users adopt - Which features they ignore - Where users encounter difficulties - How usage varies across platforms - Data scientists analyze these signals alongside user research, product intuition, and direct feedback. ## Improving Features Through Experiments - A/B testing is a central part of Figma’s product-development process. - Experiments test hypotheses and measure how proposed changes affect user behavior. - In one study, Figma examined its file-sharing modal, where users invite collaborators, manage permissions, and publish work. - Research showed that: - Only 20% of users opened the share modal during their first month. - Only half of those users successfully shared a file. - Figma simplified the interface and moved secondary functions into separate tabs. - The change produced: - A 2% increase in users sending invitations - A 2% increase in users invited to each file - No observed decline in users publishing work to Figma Community ## Identifying Performance Issues - Figma uses data to monitor application performance across platforms and prioritize improvements. - After launching the beta iOS app, the data team analyzed crashes by platform, scenario, and timing. - The analysis found that prototypes were a major source of iOS crashes. - About 25% of prototype crashes occurred within the first 10 seconds of loading. Figma’s approach combines minimal functional data collection with analytics, experimentation, and performance monitoring. The practical goal is to use data selectively to improve usability, reliability, and collaboration while limiting the amount of sensitive information required from users.

Read original(opens in new tab)

Data AnalyticsiOSData ScienceA B Testing+2