59 posts
Anthonin Bonnefoy At Datadog, we track the life cycle of millions of ephemeral hosts that report telemetry data to our platform. When a host stops emitting data, we eventually need to clean it up to avoid bloating our metadata store. To detect inactive hosts, the Datadog team th…
Christoph Hamsen Kylian Serrania Christophe Tafani-Dereeper Cloud Security Researcher and Advocate At Datadog, our commitment to open source means operating transparently and accepting that our repositories will be probed by adversaries. A few months ago, we shared our approach…
Reilly Wood I work on Datadog's official MCP (Model Context Protocol) server, our first observability interface designed specifically for customers' AI agents. Our first version was a thin wrapper around existing APIs, the kind of thing you can build in a weekend. It worked well…
Guillaume Fournier eBPF has opened up new capabilities for observability, networking, and security. But when you run it in production across thousands of environments and kernel versions, things get complicated fast. At Datadog, we've spent the past 5 years building and operatin…
Yoann Ghigoff Jonathan Ribas Sylvain Afchain Sylvain Baubeau Guillaume Fournier File integrity monitoring (FIM) helps teams detect unauthorized changes to sensitive files and is a critical part of any security posture. Yet building an FIM system that works reliably across modern…
Sanketh Balakrishna Andrew Zhang At Datadog, we operate thousands of services that rely on consistent, low-latency data access. Moving data between diverse systems—quickly and reliably—is essential, but complex. Each application may have its own requirements for data freshness,…
Laura de Vesine Rob Thomas Maciej Kowalewski In March 2023, Datadog experienced a rare, widespread incident that left large parts of our infrastructure only partially functional, but from a customer’s perspective, our platform looked completely down. This square-wave failure pat…
Sami Tabet At Datadog, we process more than 100 trillion events and billions of queries every day—across logs, traces, network data, and more. To support that scale, we built Husky, our third-generation event store. We detailed its architecture in a series of posts on exactly-on…
Yevgeniy Miretskiy Sesh Nalla Arun Parthiban Alp Keles At Datadog, cost-aware engineering is more than a principle; it’s a performance challenge at scale. We've published how we saved $17 million by rethinking our infrastructure, and we’ve built Cloud Cost Management to help cus…
Kai Zong Khor William Yu Many Datadog products offer a live view of their telemetry, allowing you to access your data in near real time from across your infrastructure. Live views improve responsiveness, but they also introduce strict requirements on data ingestion latency and s…
Khayyam Guliyev Duarte Nunes Ming Chen Justin Jaffray As Datadog continues to scale, the volume, complexity, and cardinality of the metrics we ingest and store steadily grow by orders of magnitude. This growth pushes the boundaries of our core timeseries database—the internal sy…
Nayef Ghattas When Go 1.24 was released in early 2025, we were eager to roll it out across our services. The headline feature—the new Swiss Tables map implementation—promised reduced CPU and memory overhead. Our story begins while the new version was being rolled out internally.…
Nayef Ghattas In Part 1: How we tracked down a Go 1.24 memory regression across hundreds of pods, we shared how upgrading to Go 1.24 introduced a subtle runtime regression that increased physical memory usage (RSS) across Datadog services. We worked with the Go community to iden…
Jason Thomas Senior Staff Engineer Brandon West Rosa Trieu A few years ago, we had to scramble to find a quiet place for meetings to avoid the inevitable interruptions—leaf blowers, barking dogs, and kids shouting in the background. Today, thanks to the pervasiveness of both rem…
Gabriel Reid Package delivery services like UPS, FedEx, and the postal service have a tough job. They contend with a never-ending stream of packages to be delivered, with expectations of prompt and reliable delivery. In building the Datadog Log Forwarding feature, we had to cont…