Metrics | Techlist.io

kakaoNov 19, 2025Original article

How the POPM Course Became a (opens in new tab)

Kakao developed its internal POPM (Product Owner/Product Manager) training program by treating the curriculum itself as an evolving product rather than a static lecture series. By applying agile methodologies such as data-driven prioritization and iterative versioning, the program successfully moved from a generic pilot to a structured framework that aligns teams through a shared language of problem-solving. This approach demonstrates that internal capability building is most effective when managed with the same rigor and experimentation used in software development. ## Strategic Motivation for POPM Training * Addressed the inherent ambiguity of the PO/PM role, where non-visible tasks often make it difficult for practitioners to define their own growth or impact. * Sought to resolve the disconnect between strategic problem definition (PO) and tactical execution (PM) within Kakao’s teams. * Prioritized the creation of a "common language" to allow cross-functional team members to define problems, analyze metrics, and design experiments under a unified structure. ## Iterative Design and Versioning * The program transitioned through multiple "versions," starting with an 8-session pilot that covered the entire lifecycle from bottleneck exploration to execution review. * Based on participant feedback regarding high fatigue and low efficiency in long presentations, the curriculum was condensed into 5 core modules: Strategy, Metrics, Experiment, Design, and Execution. * The instructional design shifted from "delivering information" to "designing a rhythm," utilizing a "one slide, one question, one example" rule to maintain engagement. ## Data-Driven Program Refinement * Applied a "Product Metaphor" to education by calculating "Opportunity Scores" using a matrix of Importance vs. Satisfaction for each session. * Identified "Data/Metrics" as the highest priority for redesign because it scored high in importance but low in satisfaction, indicating a structural gap in the teaching method. * Refined the "features" of the training by redesigning worksheets to focus on execution routines and converting mandatory practice tasks into selective, flexible modules. ## Structural Insights for Organizational Growth * Focused on accumulating "structure" rather than just training individuals, ensuring that even as participants change, the framework for defining problems remains consistent within the organization. * Designed practice sessions to function as "thinking structures" rather than "answer-seeking" exercises, encouraging teams to bring their training insights directly into actual team meetings. * Prioritized scalability and simplicity in the curriculum to ensure the structure can be adopted across different departments with varying product needs. To build effective internal capabilities, organizations should treat training as a product that requires constant maintenance and versioning. Instead of focusing on one-off lectures, leaders should design structural "rhythms" and feedback loops that allow the curriculum to evolve based on the actual pain points of the practitioners.

Metrics Product Design Product Management Data Analysis+3

datadogJun 24, 20251 min readCurated summary

How we built a real-time, client-side noise suppression library without server dependencies | Datadog

Datadog’s page announces that the company was named a Leader in Gartner’s 2026 Magic Quadrant for Observability Platforms. However, the provided content contains only the page header, navigation links, and product categories—not the blog post itself—so its technical argument and supporting details cannot be reliably summarized. ## Available Content - Announcement: - Datadog was named a Leader in the Gartner® Magic Quadrant™ for Observability Platforms. - The page links to a Gartner-related resource. - Product areas listed: - Infrastructure and application monitoring - Logs, databases, and data observability - Security and cloud security - Digital experience monitoring - CI/CD and software delivery - Service management - AI and observability tools - The URL references a “noise suppression library,” but no corresponding article text was included. Please provide the full blog post content for a substantive summary.

Read original

Metrics Kubernetes Datadog Observability+3

figmaFeb 10, 20252 min readCurated summary

Making Metrics Matter | Design Systems 104 | Figma Blog

Design systems create measurable business value, not just reusable components and documentation. Figma argues that tracking adoption, usage, consistency, and time savings helps teams prove that value and identify where systems need improvement. The evidence is significant: designers with access to a design system completed tasks 34% faster, while some companies reported time savings of up to 50%. ## Measuring the Value of Design Systems - Figma’s research found that design-system users completed tasks **34% faster** than those without access. - For a team of seven designers, this efficiency is comparable to adding **3.5 designers’ worth of capacity each week**. - Participants also felt more confident because the system helped ensure their work aligned with the broader product. - Vanguard reported design updates happening **50% faster** with effective systems. ## Adoption and Usage as Business Signals - Creating components and documentation is only the starting point; measuring actual adoption reveals whether the system is delivering value. - Headspace reported: - **20–30% time savings** on straightforward tasks. - Up to **50% savings** on complex projects through tokens and variables. - Swiggy cut feature rollout time in half after implementing stronger tracking. - These results connect design-system usage to improved consistency, efficiency, and scalability. ## Reading the Signals Organizations should choose metrics that reflect their goals, including: - **Library and component usage:** Track which components, variables, and styles are frequently used. - High usage identifies the system’s most valuable “workhorses.” - Low usage can reveal elements that need improvement, replacement, or deprecation. - Tracking time saved through component reuse provides a concrete benefit that can help secure stakeholder support. A practical approach is to measure not only what a design system contains, but how consistently and effectively teams use it.

Read original

Metrics Figma Design Systems Design Tokens+2

datadogJan 29, 20251 min readCurated summary

Husky: Efficient compaction at Datadog scale | Datadog

The provided content does not include the blog post itself. It contains Datadog’s navigation links and a reference to an article at `/blog/engineering/husky-storage-compaction/`, but no substantive text about Husky or storage compaction. ## Available Information - The article appears to be an engineering post about **Husky storage compaction**. - The surrounding site navigation lists Datadog products across infrastructure, applications, data, logs, security, and AI. - No technical details, arguments, implementation choices, or conclusions from the article are present. Please provide the article’s body text or a complete excerpt for an accurate summary.

Read original

Metrics Kubernetes Datadog Observability+2

datadogMay 1, 20241 min readCurated summary

How we built the Datadog heatmap to visualize distributions over time at arbitrary scale | Datadog

The supplied content does not include the blog post itself; it mainly contains Datadog’s navigation menu and a promotional link announcing its Gartner recognition. The only identifiable article reference is a post about building Datadog’s heatmap for visualizing distributions over time at arbitrary scale, but its body text is missing. ## Available Content - Datadog is promoting its recognition as a Leader in the Gartner® Magic Quadrant™ for Observability Platforms. - The navigation lists Datadog products across: - Infrastructure and application monitoring - Logs, databases, and data observability - Security - Digital experience monitoring - Software delivery - Incident and service management - AI and platform capabilities - The referenced engineering post appears to discuss: - A heatmap visualization - Distributions over time - Scaling to arbitrary data volumes A meaningful technical summary would require the actual article text, which is not present in the supplied content.

Read original

Metrics Datadog Observability Data Visualization+1

datadogMay 17, 20223 min readCurated summary

Introducing Husky, Datadog's third-generation event store

Datadog built Husky, a new event-storage system, after its original log architecture struggled with multi-tenant reliability, rapid platform growth, and evolving product requirements. The post explains how Datadog moved from metrics-oriented storage to event storage, introduced custom sharding and routing, and eventually recognized the need for a more flexible system. Husky emerged from these lessons about isolation, scalability, and retaining high-cardinality event data. ## From Metrics to Logs - Metrics systems store pre-aggregated tuples such as `<timeseries_id, timestamp, float64>`. - Aggregation makes metrics extremely efficient: millions of events in a second can become one compact datapoint, often requiring less than two bytes with delta-of-delta encoding. - This model is poorly suited to logs because logs must preserve individual events and their full context. - Metrics typically favor long-lived, low-cardinality dimensions such as: - Datacenter - Service - Pod name - Short-lived, high-cardinality fields such as transaction IDs and packet IDs are usually pre-aggregated or omitted. - Logs instead need to support: - Multi-kilobyte events - High-cardinality values such as UUIDs and stack traces - Arbitrary aggregations performed at query time ## Limitations of the Initial Logs System - Datadog’s first Logs architecture initially worked well but became vulnerable in a multi-tenant environment. - A single unhealthy or overloaded node could degrade service for every tenant in the cluster. - Scaling overloaded clusters could worsen the situation because nodes began streaming data to one another while already handling excessive read and write workloads. - Diagnosing and mitigating these cascading failures was difficult. ## Separating Storage from Clustering Datadog’s second architecture retained the same single-node storage engine but moved clustering responsibilities into dedicated services. - Storage nodes no longer knew about one another and behaved like independent one-node clusters. - Failures were isolated to the tenants assigned to a particular shard instead of spreading across the entire cluster. - A Shard Router: - Read events from Kafka - Reorganized them into shard-based Kafka partitions - Dynamically assigned tenants to an appropriate number of shards based on their recent five-minute data volume - Each shard was consumed by two storage-node replicas for redundancy. - A custom query engine tracked tenant-to-shard assignments, queried the relevant replicas, merged partial aggregates, and produced final results. ## Growth of the Event Platform - The new architecture substantially improved reliability and reduced operational burden. - Datadog expanded the platform beyond Logs to support products including: - Network Performance Monitoring - Real User Monitoring - Continuous Profiler - These products generated structured, multi-kilobyte events with storage and indexing requirements similar to logs. - As usage grew, new problems appeared: - A tenant producing a sudden burst of events could degrade query performance for other tenants sharing its shard. - Product teams requested longer retention for important but infrequently queried data, while still requiring it to remain immediately queryable. - The existing architecture was increasingly difficult to adapt to these isolation, scalability, and retention requirements, motivating the development of Husky. Datadog’s progression shows that event storage cannot simply reuse metrics-oriented designs. Systems must preserve event-level context, isolate tenants from one another, and support changing retention and query requirements as products and workloads evolve.

Read original

Metrics Database Design Datadog Apache Kafka+4

datadogSep 23, 20194 min readCurated summary

Computing accurate percentiles with DDSketch | Datadog

Datadog’s post explains why accurately computing percentiles is difficult when monitoring large-scale, distributed systems. Traditional approaches either require retaining every observation or sacrifice accuracy through fixed-size summaries, especially for long-tailed data such as request latency. DDSketch addresses this by providing mergeable percentile estimates with a guaranteed relative-error bound and memory usage that remains effectively constant. ## Why Percentiles Matter - Averages can hide slow requests and do not describe the tail of a distribution. - Percentiles such as p95, p99, and p99.9 are more useful for measuring latency and reliability. - Monitoring systems must calculate these values from enormous numbers of observations across many hosts and services. - Storing every measurement is too expensive, while calculating percentiles independently on each machine and averaging the results is mathematically incorrect. ## Limitations of Common Approaches - Exact percentile calculation requires sorting or retaining all values, which is impractical for high-volume metrics. - Histograms use predefined buckets, making their accuracy dependent on bucket boundaries. - Fixed-width buckets are inefficient for distributions spanning several orders of magnitude: - Small values may require fine-grained buckets. - Large values may require a huge number of buckets. - Many quantile sketches optimize for rank accuracy, but a small rank error can still produce a large value error in heavy-tailed distributions. - Summaries must also be mergeable so that data collected from multiple agents can be combined without losing their accuracy guarantees. ## DDSketch’s Logarithmic Mapping - DDSketch groups values into logarithmically spaced bins rather than equally sized intervals. - Values close together near zero receive finer absolute resolution, while larger values receive wider buckets. - Each value is mapped to a key based on its logarithm: - Positive and negative values are handled separately. - Zero and values sufficiently close to zero use a dedicated zero bucket. - A representative value is chosen for each bucket, typically using the bucket’s geometric center. - Because adjacent buckets have a fixed ratio, the estimated value is bounded by a predictable relative error rather than a fixed absolute error. ## Relative-Error Guarantees - DDSketch is configured with a target relative accuracy, such as 1%. - Its logarithmic base is selected so that the returned quantile is within that relative-error bound of the true value. - Relative error is particularly appropriate for latency data: - An error of a few milliseconds matters greatly for a 10 ms request. - The same absolute error is much less significant for a 10-second request. - The sketch preserves accuracy across a wide range of values without requiring a proportional increase in the number of buckets. ## Distributed Aggregation and Memory Use - DDSketches can be merged by adding the bucket counts from separate sketches. - This allows agents, hosts, containers, and regional services to aggregate measurements into a global percentile. - Merging does not require access to the original observations. - The sketch stores counts rather than individual values, substantially reducing memory and network costs. - Datadog also describes bounded-memory variants that collapse older or less significant bins when necessary, allowing sketches to maintain a fixed storage limit while retaining useful tail information. ## Practical Trade-offs - Higher accuracy requires more buckets and therefore more memory. - Lower accuracy reduces resource usage but produces wider estimates. - The choice of relative accuracy should reflect the metric’s operational needs rather than defaulting to the smallest possible error. - Implementations must account for negative values, zeros, very small values, and values outside the normal range. - Accurate percentile reporting depends not only on the sketch algorithm but also on correct aggregation and consistent configuration across producers. DDSketch is therefore a practical choice for observability systems that need scalable, mergeable, and predictable percentile calculations. Its logarithmic buckets and relative-error guarantees make it especially well suited to latency and other long-tailed measurements where fixed-width histograms or rank-based approximations can be misleading.

Read original

Metrics Kubernetes Datadog Observability+3

datadogJun 8, 20172 min readCurated summary

Protobuf parsing in Python

Protocol Buffers provides a compact, efficient binary format for structured data, making it suitable for APIs and inter-machine communication. The post introduces Protobuf through a Python metrics example and explains how to serialize and deserialize messages. It also shows how to stream multiple messages by prefixing each with its length, since Protobuf messages are not inherently self-delimiting. ## Protocol Buffers Basics - A `.proto` file defines the structure of a message. - The example `Metric` message contains: - A name - A type - A floating-point value - Repeated string tags - The `protoc` compiler generates language-specific code, such as Python’s `metric_pb2.py`. - Python can serialize a message with `SerializeToString()` and restore it with `ParseFromString()`. ## Streaming Multiple Messages - A single Protobuf message can be parsed directly, but consecutive messages need delimiters. - Protobuf does not automatically indicate where one message ends and the next begins. - The recommended approach is to prepend each serialized message with its byte length. - The length is encoded as a Varint, which uses fewer bytes for smaller integers. - This mirrors Java’s `writeDelimitedTo` and `parseDelimitedFrom` behavior and is also how the kube-state-metrics API chains messages. ## Varints and Python Implementation - Python’s Protobuf library does not provide public convenience methods for delimited messages. - The implementation uses internal helpers: - `_VarintBytes` to encode message lengths - `_DecodeVarint32` to read them - Serialization writes the length followed by the message bytes. - Deserialization reads the length, extracts the corresponding byte range, and parses it as a `Metric`. - The example loads the entire stream into memory, though a production implementation could process data incrementally. For APIs that exchange sequences of structured records, length-prefixed Protobuf messages offer an efficient and interoperable alternative to plain-text formats. Teams should account for message framing explicitly and use generated code plus appropriate streaming logic when handling multiple messages.

Read original

Metrics Kubernetes Datadog Python+4

datadogJan 23, 20172 min readCurated summary

The trouble with mounting

Datadog found that some agents stopped reporting all metrics because they became stuck in an unkillable state during disk checks. The root cause was `os.statvfs`, whose glibc implementation can hang while inspecting NFS mounts configured with hard-mount behavior. Since agents run in unpredictable customer environments, Datadog isolated the call in a separate thread and allowed the main process to continue after a timeout. ## Detecting the Hang - Customers reported gaps across every metric, indicating that the agent—not an individual check—had stopped functioning. - Logs showed the agent sometimes hung without producing an error. - A watchdog failed to terminate it because the process was stuck in an unkillable system call. - Developer-mode timing data identified `os.statvfs` as the consistently slow operation. ## How NFS Causes Unkillable Processes - `os.statvfs` calls the Linux `statvfs` function through CPython and glibc. - `statvfs` can hang when examining a remote directory mounted through NFS. - NFS hard mounts retry indefinitely and do not time out system calls. - Soft mounts eventually return an error, while the `intr` option allows interruption of the calling process. - Hard mounts may be appropriate when reads and writes must eventually succeed, but they are risky with unreliable NFS connections because they are the default in many configurations. ## The `/proc/mounts` Complication - Glibc’s `statvfs` implementation checks each directory listed in `/proc/mounts` until it finds the requested mount. - Consequently, a disconnected NFS mount can block `statvfs` even when the agent is checking a different filesystem. - This made changing NFS mount options impractical as a universal fix because Datadog cannot control customers’ system configurations. ## Datadog’s Workaround - The agent now runs `statvfs` on a separate thread. - If the call exceeds a timeout, the main agent thread continues operating. - This approach avoids total metric loss across heterogeneous environments. - The trade-off is a modest increase in memory usage on systems with hard-mounted NFS volumes. The practical lesson is to treat filesystem statistics as potentially blocking operations, especially in environments with NFS. Isolating such calls behind timeouts provides more reliable monitoring than assuming system calls will always return promptly.

Read original

Metrics Database Design Datadog Python+3

datadogOct 27, 2016Original article

Cheering on coworkers: Building culture with Datadog dashboards | Datadog (opens in new tab)

Datadog engineers developed a real-time tracking dashboard to monitor a colleague’s progress during an 850km, six-day ultra-marathon challenge. By scraping public race statistics and piping the data into their monitoring platform, the team created a centralized visualization tool to provide remote support and office-wide engagement. ### Data Extraction and Parsing The team needed to harvest race data that was only available as plain HTML on the event’s official website. * A crawler was built using the Python `Requests` library to automate the retrieval of the webpage's source code. * The team utilized `BeautifulSoup` to parse the HTML and isolate specific data points, such as the runner's current ranking and total distance covered. ### Ingesting Metrics with StatsD Once the data was structured, it was converted into telemetry using the Datadog agent and the `statsd` Python library. * The script utilized `dog.gauge` to emit three primary metrics: `runner.distance`, `runner.ranking`, and `runner.elapsed_time`. * Each metric was assigned a "name" tag corresponding to the runner, allowing the team to filter data and compare participants within the Datadog interface. * The data was updated periodically to ensure the dashboard reflected the most current race standings. ### Dashboard Visualization and Results The final phase involved synthesizing the metrics into a high-visibility dashboard displayed in the company’s New York and Paris offices. * The dashboard combined technical performance graphs with multimedia elements, including live video feeds and GIFs, to create an interactive cheering station. * The system successfully tracked the athlete's 47km lead in real-time, providing the team with immediate updates on his physical progress and elapsed time over the 144-hour event. This project demonstrates how standard observability tools can be repurposed for creative "life-graphing" applications. By combining simple web scraping with metric ingestion, engineers can quickly build custom monitoring solutions for any public data source.

Metrics Datadog Python Beautifulsoup+3