techlist.io
KREN

Datadog

190 posts

www.datadoghq.com/blog/engineering

Filter by tag

Datadog110Kubernetes85Observability83Application Performance Monitoring64Database Design61Log Management56Infrastructure Monitoring52Cloud Security38Synthetic Monitoring16PostgreSQL15Python14Apache Kafka13+505
datadog3 min readCurated summary

Scaling support with Vagrant and Terraform

Datadog’s Solutions Team uses reproducible virtual environments to investigate customer issues across diverse operating systems, kernels, and integrations. Vagrant simplifies local VM creation, while provisioning scripts eliminate repeated installation and configuration work. Terraform extends the same approach to shared AWS environments, enabling teams to provision, preserve, and collaborate on sandboxes quickly. ## Reproducing Customer Environments with Vagrant - Containers are useful, but virtual machines are better when reproducing specific operating systems, kernels, orchestrators, or complex infrastructure. - Vagrant provides a simple workflow: - `vagrant init` - `vagrant up` - `vagrant ssh` - The main challenge is not creating a VM, but installing and configuring the technologies needed to match a customer’s environment. - With more than 200 integrations, engineers cannot be experts in every technology they may need to troubleshoot. ## Standardizing Setup with Provisioning Scripts - Vagrant provisioning supports tools such as Chef, Puppet, Ansible, and ordinary shell scripts. - Datadog stores reusable reproduction environments in a shared GitHub repository. - Each sandbox includes: - A `Vagrantfile` - A `setup.sh` provisioning script - A `data` directory for configuration files and supporting scripts - A `README.md` with usage information - Engineer-specific values, such as hostnames and tags, are kept in a local `.sandbox.conf.sh` file. - Once a sandbox exists, an engineer can run `vagrant up` and begin reproducing the customer issue within minutes. - The directory hierarchy organizes sandboxes by operating system, version or provider, and technology—for example, Ubuntu Xenial with Kafka. ## Sharing Remote Environments with Terraform - Terraform provides similar infrastructure management for remote cloud instances, including AWS EC2. - The team reuses the same `setup.sh` and `data` files for both Vagrant and Terraform, avoiding duplicate configuration work. - Each sandbox adds a `.tf` file that: - Creates an EC2 instance - Copies required data files - Executes the provisioning script remotely - A shared Terraform module handles common infrastructure tasks, while a `tf.example` file helps engineers create new configurations. - This preserves the same repository structure and workflow while extending sandboxes from local VMs to remote environments. ## Benefits for Team Collaboration - Remote sandboxes can remain available without consuming engineers’ local RAM. - Proper network security allows teammates to access and share environments. - Engineers can reproduce previously configured integrations during live customer interactions. - Investigations can continue across time zones, allowing teams to hand off urgent issues without rebuilding the environment. The overall recommendation is to treat reproduction environments as reusable infrastructure: encode installation and configuration steps once, store them in version control, and use Vagrant for local testing and Terraform for persistent, shared cloud sandboxes.

Read original(opens in new tab)
PostgreSQLDocker & ContainersTerraformVirtual Machines+4
datadog3 min readCurated summary

Improving cloud security visibility with ChatOps

Datadog built a largely serverless AWS security monitoring pipeline to detect suspicious API activity across more than 15 AWS accounts. Rather than process every CloudTrail event equally, it categorizes actions as log, notify, or alert, reducing false positives while preserving broad visibility. The system centralizes events, batches activity, and uses automated workflows to verify engineers’ actions or escalate potential compromises. ## The Security Monitoring Challenge - Datadog manages extensive AWS infrastructure across multiple accounts and nearly 200 geographically distributed engineers. - Every console or CLI operation generates an AWS API call, creating a high-volume CloudTrail data stream. - The monitoring system must detect malicious activity while also catching accidental exposure caused by configuration mistakes. - Processing every event manually would require an expensive, constantly staffed Security Operations Center. ## Prioritizing Relevant API Calls - Datadog maintains a focused list of security-relevant AWS API calls and assigns each to one of three categories: - **Log:** Lower-risk events retained for investigation, such as `CreateGroup` or `UpdateUser`. - **Notify:** Events that require the initiating engineer to confirm their identity and intent, such as `CreateUser` or `PutUserPolicy`. - **Alert:** Rare, dangerous, or clearly misconfigured actions sent directly to the security team. - A representative alert is `AuthorizeSecurityGroupIngress` with `0.0.0.0/0`, which exposes an EC2 security group to the entire Internet. - User verification reduces false positives and helps identify compromised AWS credentials. ## Cross-Account Event Pipeline - CloudTrail records API activity in each AWS account. - CloudWatch Event Rules filter for the selected API calls and publish matching events to SNS. - SNS forwards events across accounts to an SQS queue in a dedicated security AWS account. - Centralization is necessary because CloudWatch could not directly send events cross-account to SQS. - The queue also supports batching, which is important when Terraform generates many AWS changes in a short period. - A CloudWatch rule triggers a Lambda function every two minutes to drain the SQS queue and forward events to the security orchestration layer. ## Automated Decision-Making with Komand - Datadog uses Komand, a security orchestration and automation platform, to construct workflows from built-in and custom plugins. - A custom decision plugin evaluates: - The calling user - The event’s age - Request parameters and their content - Other contextual details - Based on the analysis, the workflow silently logs the event, notifies the engineer, or pages the security team through PagerDuty. ## Engineer Verification and Escalation - For notification-level events, the engineer receives an interactive Slack message containing API call details. - Confirming the action triggers a Duo push for second-factor identity verification. - If the engineer denies the action or fails to respond promptly, the workflow alerts the security team. - Komand coordinates the Slack, Duo, PagerDuty, and custom integration logic in one centralized workflow. ## Visibility and Continuous Improvement - Every workflow execution is logged and sent to Elasticsearch. - The resulting data helps Datadog visualize security events, measure detection effectiveness, identify behavioral trends, and improve alerting. - The pipeline is designed to provide actionable security intelligence without overwhelming engineers or security personnel. Datadog’s approach combines selective event filtering, cross-account centralization, batching, and automated identity verification. Organizations facing similar AWS-scale monitoring challenges can use the same principles to reduce alert fatigue while maintaining strong detection and response capabilities.

Read original(opens in new tab)
AWSSlackTerraformSecurity Monitoring+4
datadog2 min readCurated summary

Improving cloud security visibility with ChatOps | Datadog

Datadog announces that it was named a Leader in the 2026 Gartner® Magic Quadrant™ for Observability Platforms. The provided content does not include the blog post’s body or Gartner’s evaluation details; it mainly contains Datadog’s navigation menu and product links. ## Announcement - Datadog highlights its recognition as a Leader in Gartner’s 2026 Observability Platforms Magic Quadrant. - The linked resource appears to be a Gartner-related announcement rather than a technical deep dive. ## Datadog’s Product Scope The navigation reflects a broad observability and operations platform covering: - Infrastructure monitoring, metrics, containers, Kubernetes, networks, serverless systems, and cloud costs - Application performance monitoring, profiling, dynamic instrumentation, and agent observability - Database, data-stream, jobs, and quality monitoring - Log management, sensitive-data scanning, audit trails, and observability pipelines - Security capabilities including cloud security, SIEM, workload protection, code security, and vulnerability management - Digital experience tools such as real-user monitoring, session replay, synthetic monitoring, and error tracking - Software delivery, CI visibility, testing, feature flags, and code coverage - Incident response, service catalogs, SLOs, workflow automation, and case management - AI agents, GPU monitoring, AI integrations, and investigation tools The supplied excerpt does not provide enough information to summarize Gartner’s criteria, Datadog’s strengths or weaknesses, or the report’s comparative findings.

Read original(opens in new tab)
KubernetesDatadogObservabilityCI/CD+4
datadog3 min readCurated summary

Piecewise regression: When one line simply isn’t enough

Piecewise regression models a timeseries with multiple linear segments when one line is insufficient. Datadog’s approach automatically detects both breakpoints and the number of segments, while avoiding a brute-force search of all possible partitions. It starts with an intentionally overfit model and greedily merges neighboring segments until the increase in error indicates that further merging would lose important structure. ## Objectives - **Automated breakpoint detection** - The algorithm identifies where one linear trend changes into another. - This is necessary for running hundreds of regressions per second without manual input. - **Automated segment-count selection** - The number of segments is not specified in advance. - The method must distinguish between data best represented by one line and data requiring several. - **No continuity requirement** - Adjacent regression lines do not need to meet at their shared breakpoint. - This allows the model to represent discontinuous changes in the data. ## Challenges - **Large search space** - A timeseries can be partitioned in exponentially many ways. - Although dynamic programming is more efficient than brute force, it remains too slow for Datadog’s performance requirements. - A greedy heuristic is used to eliminate large portions of the search space quickly. - **Balancing fit and simplicity** - More segments generally reduce the sum of squared errors. - Using one segment per point could produce nearly zero error but would provide little useful information for interpolation or extrapolation. - The goal is therefore to find the fewest segments that model the data accurately. ## Greedy Merging Algorithm - Begin with approximately **n/2 segments** for a timeseries containing *n* observations. - Fit each segment using ordinary least squares regression. - Repeatedly examine every pair of neighboring segments: - Calculate the increase in total squared error if the pair were merged. - Merge the pair producing the smallest error increase. - Continue merging until only one segment remains. - Record the segmentation state immediately before a merge appears to go too far. - If no merge triggers the stopping rule, select one large segment; otherwise, return the last recorded segmentation. ## Stopping Criteria - A merge becomes a potential stopping point when its increase in total squared error exceeds that of every earlier merge. - To avoid stopping prematurely on data that is fundamentally linear, the increase must also be less than **3% of the total error from a single-line regression**. - The 3% threshold is heuristic but was found to work well in practice. - For data generated from one noisy linear trend, error increases gradually as segments are merged, so no merge qualifies as an adequate stopping point and the algorithm ultimately selects one segment. The method provides a practical compromise between exhaustive optimization and model quality: greedy merging makes automated regression fast, while the error-based stopping rule limits overfitting and preserves meaningful changes in trend.

Read original(opens in new tab)
Data ModelingDynamic ProgrammingPiecewise RegressionTimeseries Data+4
datadog3 min readCurated summary

Piecewise regression: When one line simply isn’t enough | Datadog

Piecewise regression offers a practical way to model time series whose trends change over time, something a single straight-line regression cannot represent well. The technique divides data into segments and fits a separate regression line to each one, allowing systems such as observability platforms to detect trend shifts more accurately. Its usefulness depends on selecting meaningful breakpoints without overfitting noise. ## Why a Single Regression Line Falls Short - Ordinary linear regression assumes one constant relationship between time and the measured value. - Real-world operational metrics often contain: - Sudden changes in growth rate - Traffic or usage shifts - Deployment-related behavior changes - Periods of increase followed by stabilization or decline - A single line averages these different behaviors, producing inaccurate trend estimates and potentially misleading forecasts. ## How Piecewise Regression Works - The time series is divided into multiple regions by one or more breakpoints. - Each region receives its own regression equation, such as: - Before the breakpoint: one intercept and slope - After the breakpoint: a different intercept and slope - The fitting process searches for the breakpoint that minimizes the combined prediction error across all segments. - Models may require the lines to connect at the breakpoint, preventing unrealistic discontinuities, or allow independent segments when abrupt jumps are meaningful. ## Finding Useful Breakpoints - Candidate breakpoints are evaluated by comparing the residual error produced by different segmentations. - A breakpoint is valuable when it significantly improves the fit rather than merely explaining random fluctuations. - More segments can capture complex behavior, but they also increase the risk of overfitting. - Practical implementations therefore need safeguards such as minimum segment sizes, limits on the number of breakpoints, and validation against noisy data. ## Applications in Observability - Piecewise models can improve the interpretation of infrastructure and application metrics. - They are particularly useful for identifying: - Changes in request volume - Altered resource-consumption patterns - Performance regressions - Long-term growth phases - Recovery or stabilization after an incident - By distinguishing genuine trend changes from normal variation, the method can support better anomaly detection and forecasting. ## Limitations and Tradeoffs - Noisy or sparse data can make breakpoint selection unstable. - A model with too many segments may describe historical noise instead of general behavior. - Sudden outliers can distort regression parameters unless they are handled separately. - Piecewise regression captures trend changes, but it does not automatically explain their causes; engineers still need deployment, traffic, and infrastructure context. Piecewise regression is therefore best treated as a lightweight, interpretable tool for detecting changes in metric behavior. It provides more realistic trend modeling than a single regression line while remaining simpler and easier to operate than highly complex forecasting models.

Read original(opens in new tab)
KubernetesDatadogObservabilityApplication Performance Monitoring+3
datadog3 min readCurated summary

Hackathon project: Viewing Datadog metrics in Minecraft

Datadog engineers used a two-day hackathon to display real-time Datadog metrics inside Minecraft. They connected Minecraft’s Python API with Datadog’s metrics API, then built configurable, live-updating graphs and monitor indicators in the game world. The project demonstrated that even an unconventional visualization environment can be practical to prototype with familiar tools. ## Controlling Minecraft with Python - The team used Raspberry Juice and a Minecraft Pi Edition server to expose Minecraft controls. - They ran the setup on laptops for better performance and faster development. - The `py3minepi` library enabled Python code to create, remove, and query blocks. - Creating a block required only a connection to the server and a call such as `mc.setBlock(...)`. ## Retrieving Datadog Metrics - The Datadog Python library provided access to the Metrics API. - The prototype authenticated with an API key and application key. - It queried recent data, such as average system CPU idle time over the previous five minutes. - The Minecraft and Datadog components were then combined so metric values could be rendered as blocks and structures. - Monitor status indicators changed between green and red depending on whether an alert was active. ## YAML-Based Dashboard Configuration - The team moved dashboard definitions out of Python code into YAML files. - Configuration specified: - Graph position, size, and orientation - Visual properties such as colors, transparency, and borders - Datadog queries and time ranges - Monitor IDs and display locations - This allowed complete dashboards containing multiple graphs and monitor indicators to be updated in real time. ## Handling Minecraft’s Persistence - Minecraft blocks remain in the world after being created, while metric graphs change constantly. - Early experiments left behind random cubes that made the world difficult to navigate. - The team implemented “vacuum” functions to remove everything generated by the visualization code before redrawing it. ## Rendering and Performance Challenges - Without browser technologies such as JavaScript and CSS, graphs had to be reduced to rows of data and represented with Minecraft blocks. - Large graphs could overwhelm the data pipeline. - Caching was added to reduce bandwidth usage and avoid repeatedly requesting the same data. - The performance concerns mirrored Datadog’s everyday engineering work, where caching and efficient data handling are essential. ## Hackathon Experience - The first four hours focused on configuring the environment and connecting the systems. - The remaining time was spent experimenting with building, viewing, destroying, and rebuilding metric displays. - The project’s main value was creative exploration rather than production monitoring. The prototype shows how quickly APIs can be combined to create unusual monitoring interfaces. While Minecraft is not intended to replace conventional dashboards, the project is a playful demonstration of real-time data visualization and rapid experimentation.

Read original(opens in new tab)
Database DesignDatadogPythonAPI+3
datadog1 min readCurated summary

Hackathon project: Viewing Datadog metrics in Minecraft | Datadog

The provided content does not include the blog post itself. It contains Datadog’s navigation menu and a link titled “Hackathon Project: Viewing Datadog Metrics in Minecraft,” but no article text describing the project, implementation, or conclusions. ## Available Information - The linked post appears to cover a hackathon project involving: - Datadog metrics - Minecraft - A likely visualization or integration between the two - The surrounding content is primarily Datadog’s product navigation, covering: - Infrastructure and application monitoring - Logs, security, and digital experience - Software delivery and service management - AI products and platform capabilities ## Conclusion The article body is missing from the supplied material, so a reliable technical summary cannot be produced. Please provide the post’s full text or relevant excerpt.

Read original(opens in new tab)
KubernetesDatadogObservabilityApplication Performance Monitoring+3
datadog1 min readCurated summary

Protobuf parsing in Python | Datadog

The provided content does not include the blog post itself. It contains Datadog’s site navigation and a link whose URL suggests the article concerns Protobuf parsing in Python, but no technical claims, explanations, or conclusions are available to summarize. ## Available Information - The linked article appears to be: - **“Protobuf parsing in Python”** - Located on Datadog’s engineering blog. - The rest of the content is primarily Datadog product navigation, covering: - Infrastructure and application monitoring - Logs, security, digital experience, CI, and AI products - A separate banner promotes Datadog’s recognition as a Leader in the Gartner Magic Quadrant for Observability Platforms. Please provide the article’s body text or a complete page extract for a substantive summary.

Read original(opens in new tab)
PythonSerializationProtobufParsing
datadog2 min readCurated summary

Protobuf parsing in Python

Protocol Buffers provides a compact, efficient binary format for structured data, making it suitable for APIs and inter-machine communication. The post introduces Protobuf through a Python metrics example and explains how to serialize and deserialize messages. It also shows how to stream multiple messages by prefixing each with its length, since Protobuf messages are not inherently self-delimiting. ## Protocol Buffers Basics - A `.proto` file defines the structure of a message. - The example `Metric` message contains: - A name - A type - A floating-point value - Repeated string tags - The `protoc` compiler generates language-specific code, such as Python’s `metric_pb2.py`. - Python can serialize a message with `SerializeToString()` and restore it with `ParseFromString()`. ## Streaming Multiple Messages - A single Protobuf message can be parsed directly, but consecutive messages need delimiters. - Protobuf does not automatically indicate where one message ends and the next begins. - The recommended approach is to prepend each serialized message with its byte length. - The length is encoded as a Varint, which uses fewer bytes for smaller integers. - This mirrors Java’s `writeDelimitedTo` and `parseDelimitedFrom` behavior and is also how the kube-state-metrics API chains messages. ## Varints and Python Implementation - Python’s Protobuf library does not provide public convenience methods for delimited messages. - The implementation uses internal helpers: - `_VarintBytes` to encode message lengths - `_DecodeVarint32` to read them - Serialization writes the length followed by the message bytes. - Deserialization reads the length, extracts the corresponding byte range, and parses it as a `Metric`. - The example loads the entire stream into memory, though a production implementation could process data incrementally. For APIs that exchange sequences of structured records, length-prefixed Protobuf messages offer an efficient and interoperable alternative to plain-text formats. Teams should account for message framing explicitly and use generated code plus appropriate streaming logic when handling multiple messages.

Read original(opens in new tab)
KubernetesDatadogPythonMetrics+4
datadog2 min readCurated summary

Being a solutions engineer at Datadog

Solutions Engineers at Datadog combine customer support, technical investigation, and product improvement. They troubleshoot issues across configuration, dashboards, metrics, and alerting while collaborating with customers and engineers. The role also offers structured opportunities to build engineering skills and shape a career toward engineering, product management, or sales engineering. ## Role and Responsibilities - Serve as the technical point of contact for Datadog customers. - Handle tickets involving: - Agent and integration configuration - Dashboard and metric visualization - Alerting issues and bugs - Investigate customer needs, inspect source code, fix bugs when necessary, and coordinate with engineering teams. - Provide short-term workarounds and identify potential product enhancements. - Communicate with customers through chat and technical calls. ## Learning and Collaboration - New Solutions Engineers rapidly develop skills in technologies such as Linux, Ruby, and SQL. - The role requires multitasking while continuously learning new technical concepts. - Team members share knowledge through Datadog documentation and collaborative projects. - Customer feedback provides direct insight into how the product could evolve. ## Engineering Experience and Career Growth - During two-week “embedding” sprints, Solutions Engineers join engineering teams and work on projects alongside developers. - Embedding provides hands-on experience with: - Datadog’s internal technologies - Engineering and design challenges - Architectural decisions - Cross-team collaboration - Once fully trained, employees can pursue side projects such as improving demo environments or automating internal processes. - Career paths can be tailored toward engineering, product management, or sales engineering. Datadog presents the Solutions Engineer role as a strong fit for people who enjoy customer interaction, technical problem-solving, collaboration, and continuous learning. The company was hiring for these positions in New York, Paris, and remote locations.

Read original(opens in new tab)
DatadogLinuxSqlRuby+3
datadog2 min readCurated summary

Being a solutions engineer at Datadog | Datadog

Datadog’s page announces that the company has been named a Leader in the 2026 Gartner® Magic Quadrant™ for Observability Platforms. The provided content primarily consists of navigation links rather than the blog article itself, so it does not include Gartner’s evaluation details or Datadog’s specific strengths. It does, however, show the broad range of products Datadog positions as part of its observability platform. ## Gartner Recognition - Datadog highlights its designation as a Leader in Gartner’s 2026 Magic Quadrant for Observability Platforms. - No supporting analysis, ranking details, or Gartner assessment criteria are included in the provided text. ## Datadog’s Platform Scope - **Infrastructure:** Infrastructure and container monitoring, metrics, Kubernetes autoscaling, network monitoring, serverless, cloud cost, GPU, and storage management. - **Applications and data:** Application Performance Monitoring, profiling, dynamic instrumentation, database monitoring, data-stream monitoring, and data quality. - **Logs and security:** Log management, observability pipelines, sensitive-data scanning, cloud security, SIEM, workload protection, and code security. - **Digital experience:** Browser and mobile RUM, session replay, synthetic monitoring, product analytics, experiments, and error tracking. - **Software delivery and service management:** CI visibility, test optimization, code coverage, feature flags, incident response, SLOs, workflow automation, and internal developer portals. - **AI capabilities:** Agent observability, AI integrations, GPU monitoring, Bits AI agents, investigation tools, and an MCP server. Datadog’s positioning is therefore centered on providing a unified platform spanning infrastructure, applications, logs, security, user experience, software delivery, and AI. A complete assessment would require the missing body of the article or Gartner report details.

Read original(opens in new tab)
KubernetesDatadogObservabilityApplication Performance Monitoring+3
datadog2 min readCurated summary

The trouble with mounting

Datadog found that some agents stopped reporting all metrics because they became stuck in an unkillable state during disk checks. The root cause was `os.statvfs`, whose glibc implementation can hang while inspecting NFS mounts configured with hard-mount behavior. Since agents run in unpredictable customer environments, Datadog isolated the call in a separate thread and allowed the main process to continue after a timeout. ## Detecting the Hang - Customers reported gaps across every metric, indicating that the agent—not an individual check—had stopped functioning. - Logs showed the agent sometimes hung without producing an error. - A watchdog failed to terminate it because the process was stuck in an unkillable system call. - Developer-mode timing data identified `os.statvfs` as the consistently slow operation. ## How NFS Causes Unkillable Processes - `os.statvfs` calls the Linux `statvfs` function through CPython and glibc. - `statvfs` can hang when examining a remote directory mounted through NFS. - NFS hard mounts retry indefinitely and do not time out system calls. - Soft mounts eventually return an error, while the `intr` option allows interruption of the calling process. - Hard mounts may be appropriate when reads and writes must eventually succeed, but they are risky with unreliable NFS connections because they are the default in many configurations. ## The `/proc/mounts` Complication - Glibc’s `statvfs` implementation checks each directory listed in `/proc/mounts` until it finds the requested mount. - Consequently, a disconnected NFS mount can block `statvfs` even when the agent is checking a different filesystem. - This made changing NFS mount options impractical as a universal fix because Datadog cannot control customers’ system configurations. ## Datadog’s Workaround - The agent now runs `statvfs` on a separate thread. - If the call exceeds a timeout, the main agent thread continues operating. - This approach avoids total metric loss across heterogeneous environments. - The trade-off is a modest increase in memory usage on systems with hard-mounted NFS volumes. The practical lesson is to treat filesystem statistics as potentially blocking operations, especially in environments with NFS. Isolating such calls behind timeouts provides more reliable monitoring than assuming system calls will always return promptly.

Read original(opens in new tab)
Database DesignDatadogPythonMetrics+3
datadog2 min readCurated summary

The trouble with mounting | Datadog

The post explains why filesystem mounting becomes surprisingly complex in containerized Linux environments. Datadog’s Agent needs to inspect host filesystems from inside a container, but mount namespaces, bind mounts, and propagation rules can make the host’s view incomplete or inconsistent. The conclusion is that reliable mounting requires understanding namespace boundaries and deliberately configuring mount propagation rather than treating mounts as ordinary directory mappings. ## Linux Mount Namespaces - Each process can have its own mount namespace, isolating its view of mounted filesystems. - A container therefore sees a filesystem tree that may differ substantially from the host’s tree. - Bind-mounting a directory into a container does not necessarily expose mounts created beneath that directory. - This is particularly problematic for paths such as `/proc`, `/sys`, `/var/lib/docker`, and other locations containing nested mounts. ## The Problem with Bind Mounts - A bind mount initially exposes only the directory tree visible at the time it is created. - Later mounts beneath the source directory may not appear in the container. - Recursive bind mounts can copy nested mounts, but they introduce their own behavior and compatibility concerns. - Mounts can also be shared, private, or “slave,” determining whether mount and unmount events propagate between namespaces. ## Mount Propagation - Linux mount propagation controls how changes in one namespace are reflected in another. - Shared mounts propagate events in both directions, while slave mounts receive changes without sending them back. - Private mounts isolate changes completely. - Choosing the wrong propagation mode can cause the Agent to miss newly mounted filesystems or, worse, allow container-side changes to affect the host. ## Datadog’s Engineering Challenge - Datadog needs host-level visibility while keeping the monitoring container isolated and safe. - The Agent must account for mounts that appear after startup, including dynamically created container and volume mounts. - Correct behavior depends on both the container runtime configuration and the host’s existing mount topology. - A robust implementation must inspect mount metadata and handle namespace and propagation details explicitly. The practical recommendation is to treat mounting as a namespace and event-propagation problem, not merely a path-sharing mechanism. Systems that need host visibility should use carefully selected recursive mounts and propagation modes, validate the resulting mount tree, and test behavior when mounts are created or removed dynamically.

Read original(opens in new tab)
KubernetesDatadogObservabilityApplication Performance Monitoring+4
datadog2 min readCurated summary

Engineering spotlight: Marie-Laure Bardonnet

Datadog’s Notebooks feature demonstrates that interns can make substantial contributions when given meaningful ownership and effective mentorship. Marie-Laure Bardonnet progressed from small bug fixes to prototyping the feature, gaining hands-on experience with React, Redux, and Redux Saga. Her experience ultimately led to continued part-time work and a full-time role at Datadog. ## From Bug Fixes to Feature Ownership - Marie-Laure initially handled small issues, such as fixing a dashboard favorite-star interaction. - Gradually more complex tasks helped her learn Datadog’s codebase and application architecture. - Rather than limiting her to routine maintenance, Datadog assigned her a major project earlier than expected. ## Building Datadog Notebooks - Notebooks let users save graphs from a specific point in time alongside text and other contextual information. - The feature is designed to preserve and share organizational knowledge, helping teams respond more quickly. - Marie-Laure built most of the prototype during her seven-month internship. - The project introduced her to: - React for the frontend - Redux for state management - Redux Saga for handling side effects ## The Role of Mentorship - Team lead Ivan DiLernia gave Marie-Laure substantial autonomy while remaining available for difficult architectural decisions. - He encouraged her to investigate ideas independently, then collaborated with her when problems required deeper discussion. - Marie-Laure identified this balance between independence and guidance as one of the most valuable parts of the internship. ## Lasting Impact - The internship changed how Marie-Laure viewed her academic coursework, helping her distinguish practical engineering skills from more theoretical material. - After returning to France, she continued working part-time on Notebooks and other web-platform projects. - She later completed her studies and accepted a full-time position at Datadog. Datadog’s experience suggests that internships are most effective when they combine gradual onboarding, meaningful technical ownership, and thoughtful mentorship rather than restricting interns to low-impact tasks.

Read original(opens in new tab)
Database DesignDatadogReactWeb Development+3
datadog1 min readCurated summary

Engineering spotlight: Marie-Laure Bardonnet | Datadog

The provided text contains Datadog’s navigation and promotional banner announcing its recognition as a Leader in the 2026 Gartner® Magic Quadrant™ for Observability Platforms. It does not include the actual blog post about Marie-Laure Bardonnet, so its argument, technical details, and conclusion cannot be reliably summarized. ## Available Content - Datadog promotes its observability platform and Gartner recognition. - The navigation lists products covering: - Infrastructure and application monitoring - Logs, databases, and data observability - Security and cloud protection - Digital experience monitoring - CI/CD and software delivery - Incident and service management - AI-powered observability and investigation - The page URL suggests the article is an “Engineering Spotlight” featuring Marie-Laure Bardonnet. Please provide the article body or a complete extraction of the page for a substantive summary.

Read original(opens in new tab)
KubernetesDatadogObservabilityApplication Performance Monitoring+2