Docker & Containers | Techlist.io

gitlabFeb 26, 20262 min readCurated summary

GitLab Duo Agent Platform with Claude accelerates development

GitLab Duo Agent Platform integrates external AI models such as Anthropic’s Claude and OpenAI’s Codex directly into GitLab workflows. Instead of operating as isolated coding assistants, these agents use project context and organizational standards to handle multi-step development tasks. The result is faster delivery, more consistent quality, and less manual work across the software development lifecycle. ## From an Idea to a Working Application - An agent can use an issue’s title and detailed requirements as the foundation for a complete application. - It analyzes project context and related assets, then generates: - Backend Java classes - Frontend HTML, CSS, and JavaScript - Business logic and UI components - Build configuration - The agent creates a merge request containing the implementation for developers to test and refine through natural-language interaction. ## Automated Code Review - Developers can mention the external agent in a merge request to request a review. - The review can cover: - Code strengths and critical issues - Medium- and low-priority improvements - Security risks - Testing gaps and code metrics - Recommendations and an approval status - This provides consistent review coverage while allowing senior developers to focus on architecture and complex decisions. ## Pipeline and Container Image Creation - When a project lacks CI/CD configuration, the agent can generate the required pipeline. - It creates a Dockerfile with a suitable base image for the project’s Java version. - The pipeline can: - Build the application - Build a Docker image - Push the image to GitLab’s container registry - The resulting workflow runs automatically through build, image creation, and deployment stages. ## Broader Impact on Development - External agents remain within GitLab, reducing context switching between development tools. - They can follow project-specific coding standards and understand broader repository context. - Teams can automate work from initial requirements through implementation, review, and deployment. - Developers spend less time on repetitive tasks while maintaining stronger consistency and quality. GitLab presents Duo Agent Platform as a way to turn external AI models into integrated development collaborators. Teams can use it to accelerate coding, automate reviews, and create deployment pipelines while keeping humans focused on validation, architecture, and innovation.

Read original

Docker & Containers Machine Learning CI/CD Code Review+4

gitlabFeb 25, 20262 min readCurated summary

GitLab metrics and registry features help reduce CI/CD bottlenecks

GitLab’s two new beta features target common CI/CD bottlenecks without requiring additional third-party tools. CI/CD Job Performance Metrics provides job-level visibility into duration and failures, while Container Virtual Registry centralizes pulls from multiple registries through a cached GitLab endpoint. Together, they help platform teams identify pipeline problems faster and simplify container management. ## CI/CD Job Performance Metrics - Available in GitLab Premium and Ultimate. - Limited beta on GitLab.com; available on Self-Managed and Dedicated with ClickHouse configured. - Adds a job-focused panel to **Analyze > CI/CD analytics**. - Shows, for the previous 30 days by default: - Median (P50) and worst-case (P95) job duration - Failure rate - Job name and pipeline stage - Supports sorting, searching, and pagination to identify slow or unreliable jobs. - GitLab plans to add stage-level aggregation for build, test, and deploy bottlenecks. ## Container Virtual Registry - Available in GitLab Premium and Ultimate; API-ready in GitLab 18.9. - Provides one GitLab endpoint for pulling images from multiple upstream registries. - Supports registries such as Docker Hub, Harbor, Quay, and other sources using long-lived token authentication. - Uses pull-through caching to: - Reduce repeated downloads and bandwidth costs - Improve availability and reliability - Centralize authentication and registry configuration - Currently configured through the API, with UI management in development. - Cloud registries requiring IAM authentication, including Amazon ECR, Google Artifact Registry, and Azure Container Registry, may be supported later. ## Beta Access and Feedback - GitLab.com users can request access through their customer success manager or the feature’s feedback issue. - Self-managed users can enable the feature flag and configure the virtual registry through the API. - GitLab is seeking feedback to guide future improvements to both features. These betas are worth evaluating if your team needs better visibility into pipeline performance or manages images across several registries. The metrics feature can replace custom dashboards, while the virtual registry can reduce registry-related configuration and operational overhead.

Read original

Docker & Containers CI/CD Gitlab DevOps+4

datadogFeb 18, 20263 min readCurated summary

How we reduced the size of our Agent Go binaries by up to 77%

The Datadog Agent’s Linux artifact grew from 428 MiB in version 7.16.0 to 1.22 GiB in 7.60.0, creating problems for serverless, IoT, and containerized environments. Rather than remove features, Datadog reduced Go binary sizes by up to 77% between versions 7.60.0 and 7.68.0. The effort combined dependency analysis, targeted code refactoring, and renewed use of Go linker optimizations. ## Why the Agent Became So Large - The Agent supports many operating systems, architectures, distributions, and deployment environments. - Its codebase contains hundreds of dependencies, including cloud SDKs, container runtimes, and security tools. - Build tags and dependency injection determine which features are included in each binary. - The compressed Linux amd64 Debian package grew from 126 MiB to 265 MiB. - Its uncompressed size increased from 428 MiB to 1,248 MiB—a 192% increase over five years. - Go binaries represented a substantial portion of that growth and became the primary optimization target. ## How Go Selects Dependencies - Go compiles required packages individually before the linker combines them into a binary. - Files are included only when they: - Are not test files ending in `_test.go` - Match the current operating system, architecture, and build tags - Satisfy other constraints such as CGO settings, compiler version, or architecture features - Starting from the main package, Go transitively includes imported packages and the runtime required by every Go binary. - Unnecessary dependencies can be excluded by: - Adding a build tag to the file that imports them - Moving dependency-using symbols into a separate package imported only by relevant binaries ## Analyzing Imports and Dependencies - `go list` reveals all packages used for a specific OS, architecture, and set of build tags. - `goda` generates dependency graphs, including indirect imports. - `goda` can also show only the paths leading to a particular target package using its `reach` function. - These tools account for `GOOS`, `GOARCH`, and build constraints, making them useful for examining platform-specific builds. ## Why Package Lists Are Not Enough - A package’s presence does not directly indicate its binary size impact. - The linker removes symbols that are not reachable from the program’s entry points. - The same package can therefore contribute different amounts of code depending on how it is used. - Importing a package can still have significant side effects: - `init` functions execute. - Global variables are initialized. - These behaviors may force otherwise unnecessary symbols to remain in the binary. - Certain uses of reflection can also limit linker optimizations. - Datadog used `go-size-analyzer` to measure the contribution of individual dependencies more accurately than import graphs alone. ## Overall Optimization Strategy - Datadog systematically audited dependencies rather than removing product capabilities. - The work focused on restructuring imports, isolating optional functionality, and restoring linker optimizations that had been disabled or undermined over time. - The resulting improvements brought artifact sizes close to levels from roughly five years earlier. - Some compiler and linker behaviors uncovered during the effort led to improvements benefiting other large Go projects, including Kubernetes. The practical lesson is to treat binary size as an ongoing dependency and architecture concern: analyze actual symbol reachability, isolate optional features behind build constraints or packages, and verify each build variant independently.

Read original

Docker & Containers Kubernetes Go Dependency Injection+4

gitlabJan 14, 2026Original article

Getting started with GitLab Duo Agentic Chat (opens in new tab)

GitLab Duo Agentic Chat marks a shift from traditional Q&A chatbots to autonomous AI collaboration partners integrated directly into the software development lifecycle. By leveraging specialized agents and context-aware large language models, the platform enables developers to automate complex tasks like code refactoring, security remediation, and issue triaging. This system serves as a centralized interface across both the GitLab Web UI and IDEs to streamline workflows from initial planning to production deployment. ## Capabilities of Agentic AI * **Autonomous Actions:** The system can move beyond simple chat by creating files, modifying existing code, and opening merge requests on behalf of the user. * **Deep Context Integration:** Agents have access to the full GitLab ecosystem, including issues, epics, Git commits, CI/CD pipelines, and security scans. * **Extensibility:** Through the Model Context Protocol (MCP), the chat can integrate with external services to expand its functional scope. * **Information Retrieval:** Users can query project architecture or use GitLab Query Language (GLQL) to pull specific project analytics and insights. ## Model and Agent Customization * **Flexible Model Selection:** Users and administrators can choose from different LLMs based on task requirements, with configuration available at both the group and individual user levels. * **Specialized Agents:** The platform features dedicated agents for specific roles, such as the **Planner Agent** for product management and the **Security Analyst Agent** for vulnerability management. * **Contextual Switching:** In IDEs, users can switch between agents via a dropdown menu, while the Web UI allows for agent selection when starting new chat sessions. ## Specialized Workflow Use Cases * **Project Planning:** The Planner Agent can break down epics into smaller tasks, list high-priority bugs, and generate technical requirements for new features. * **Security Remediation:** Security-focused agents can explain vulnerabilities in simple terms, identify false positives in scans, and suggest specific code fixes for SQL injection or XSS risks. * **Troubleshooting and Debugging:** The system can analyze CI/CD pipeline logs to identify why a build failed and suggest optimizations for job performance. * **Legacy Modernization:** Specific prompts can guide the AI to refactor code to follow SOLID principles or create migration plans for modernizing legacy languages like COBOL to Java or Python. ## Access and Integration * **Interface Options:** The chat is accessible via a collapsible sidebar in the Web UI and through dedicated plugins in popular IDEs. * **Future Development:** While currently limited to UI and IDE interfaces, a GitLab Duo CLI is in development to bring agentic capabilities to the terminal. To get the most out of GitLab Duo Agentic Chat, it is recommended to transition between specialized agents as you move through different project phases. Using the Security Analyst for code reviews and the Planner for backlog grooming ensures that the underlying models are optimized for the specific metadata and constraints of those tasks.

Docker & Containers Database Design AI Agents CI/CD+4

datadogAug 12, 20253 min readCurated summary

Scaling down to speed up: How we improved efficiency of live process metrics by 100x

Datadog redesigned its real-time Processes and Containers pipeline to avoid collecting high-frequency metrics that users never see. By limiting 2-second collection to hosts actively viewed and using standard 10-second data for sorting, the company reduced real-time traffic by over 100x, cut infrastructure costs by 98%, and lowered Agent resource usage. The approach also improved scalability without sacrificing the live investigation experience. ## Original Real-Time Collection Model - Datadog Agents normally collect process and container metrics every 10 seconds. - When a user opened a live Processes or Containers view, all hosts in that tenant switched to 2-second collection. - This supported near-real-time monitoring similar to `htop`, but across distributed infrastructure. - As tenants grew, the pipeline had to process millions of processes per second, even though users typically viewed only around 50 processes or containers. - Live sorting required keeping all tenant data in memory on a single server, limiting horizontal scaling and forcing vertical scaling. ## Refocusing on User-Visible Data - Most collected metrics were never displayed to users. - Datadog determined that real-time collection only needed to be enabled for hosts running the processes or containers currently in view—up to roughly 50 hosts per user. - Internal telemetry suggested this could reduce traffic by more than 100x. - This required tracking active host subscriptions and updating them as users navigated the product. - Because sorting occurred every 10 seconds, it did not need 2-second data. Datadog switched live views to use the existing 10-second metrics, aligning live and historical sorting logic. ## Host Subscription Filtering - A proof of concept added host subscriptions to the live data servers. - Servers filtered Kafka payloads and discarded data for hosts without active subscriptions. - This immediately reduced: - Memory usage by 85% - CPU usage by 33% - The improvement came from storing fewer live metrics and processing fewer incoming payloads. - The prototype confirmed that filtering preserved product behavior while simplifying sorting. ## Moving Filtering Earlier in the Pipeline - Late filtering improved live data servers but still left unnecessary work for the rest of the system and customer-side Datadog Agents. - Datadog therefore planned to propagate subscription state to the intake service. - Live data servers publish users’ active host sets over Kafka once per second. - The intake service consumes this information and decides which hosts should activate 2-second process and container metric collection. - This allows real-time collection to be restricted to hosts users are actively investigating while maintaining responsive live views. Datadog’s redesign demonstrates that real-time systems scale more effectively when they prioritize data users can actually see. Filtering at intake, limiting high-frequency collection to subscribed hosts, and reusing standard-resolution data for sorting provide a simpler and more economical architecture without eliminating live functionality.

Read original

Docker & Containers Database Design Datadog Apache Kafka+4

figmaAug 7, 20243 min readCurated summary

How We Migrated onto K8s in Less Than 12 months | Figma Blog

Figma migrated most of its core services from AWS ECS to Kubernetes in under 12 months because ECS was increasingly limiting its platform ambitions. Kubernetes offered better support for stateful workloads, Helm-based software, autoscaling, service networking, and the broader CNCF ecosystem. The migration was considered worthwhile because Figma had relatively few core services and had already containerized its workloads, making the transition more manageable. ## Figma’s Existing Compute Platform - By early 2023, Figma was already running all services in containers on Amazon ECS. - ECS had enabled rapid adoption of containerized workloads, but Figma’s growing infrastructure team began evaluating a more capable long-term platform. - Figma is not organized around thousands of microservices: - A small set of powerful core services provides modularization and traffic isolation. - New product capabilities are usually added to existing services rather than creating new ones. - This limited service count made a Kubernetes migration more practical. ## Limitations of ECS - ECS lacked Kubernetes primitives needed for complex workloads. - Running `etcd` on ECS required fragile custom startup code to manage cluster membership because ECS does not provide StatefulSets or persistent pod identity. - Kubernetes StatefulSets provide stable identities and stateful networking for systems such as `etcd`. - ECS did not natively support deploying groups of services packaged as Helm charts. - Open-source tools such as Temporal would require manual conversion into Terraform configurations. - This increased installation and maintenance effort. - ECS also made routine infrastructure operations more cumbersome. - For example, safely removing a malfunctioning EC2 instance was difficult. - EKS can cordon a node and move its pods elsewhere while respecting graceful shutdown behavior. ## Access to the CNCF Ecosystem - Kubernetes would give Figma access to a larger ecosystem of open-source cloud-native tools. - Autoscaling was a major motivation: - Figma was provisioning services for peak demand, wasting resources during lower-traffic periods. - Kubernetes tooling such as KEDA supports scaling based on CPU, SQS queue length, and custom Datadog metrics. - Figma expected to adopt a service mesh eventually. - Existing AWS load balancer routing created operational drawbacks: - Network Load Balancers could take several minutes to register or remove targets. - This slowed emergency deployments and increased incident remediation time. - Envoy offered more customization than AWS load balancers, including custom filters for shedding load during incidents. - Figma had already deployed standalone Envoy machines for a major service and saw Kubernetes ecosystems such as Istio as a path toward fleet-wide service-mesh adoption. Figma’s experience suggests that Kubernetes was justified not simply as a replacement for ECS, but as a foundation for more capable operations and broader platform tooling. Organizations considering a similar move should first assess their workload complexity, existing container maturity, and whether Kubernetes capabilities will materially reduce infrastructure work.

Read original

Docker & Containers Kubernetes Terraform Helm+4

figmaOct 23, 20232 min readCurated summary

Server-side Sandboxing: An Introduction | Figma Blog

Server-side sandboxing helps contain the damage caused by vulnerabilities in software that processes untrusted user input. This is especially important for image processing, parsing, compression, and thumbnailing libraries often written in memory-unsafe languages, as demonstrated by ImageTragick. Figma argues that sandboxing complements—rather than replaces—secure coding by limiting a compromised workload’s access to data, services, and infrastructure. ## Why Server-Side Sandboxing Matters - Modern SaaS applications must process user-generated content using complex libraries. - Many of these libraries are written in C or C++, which are vulnerable to memory-corruption bugs. - ImageTragick showed how a vulnerability in ImageMagick could enable remote code execution when processing user-supplied images. - Preventing every vulnerability through rewrites, memory-safe languages, or program analysis is expensive and imperfect. - Sandboxing provides defense in depth by containing failures when vulnerabilities are exploited. ## Figma’s Server-Side Risk - Figma uses server-side components such as RenderServer, a C++ version of the editor, along with third-party libraries for graphical data. - Malicious input processed directly inside production infrastructure could allow an attacker to: - Access data belonging to other jobs - Make requests to internal production services - Move laterally through the environment - Compromise additional systems - Sandboxing reduces the external interfaces and resources available to potentially compromised workloads. ## Common Sandboxing Approaches - The article introduces three major sandboxing primitives: - **Virtual machines (VMs):** Isolate workloads through a hypervisor and separate guest operating systems. - **Containers:** Isolate workloads using operating-system-level mechanisms and container engines. - **Seccomp:** Restricts the system calls a program is permitted to make. - Each approach involves trade-offs in security properties, operational complexity, performance, and suitability for different workloads. - The article’s broader goal is to help teams compare these options and select an appropriate combination of isolation techniques. ## Choosing an Appropriate Strategy - Sandboxing technologies have historically been expensive, immature, or difficult to operate at scale. - Recent improvements have made virtualization, containment, and workload isolation more practical for a wider range of security teams. - Teams should evaluate sandboxing based on their workload’s risk, required interfaces, resource needs, and acceptable operational trade-offs. Teams should treat sandboxing as a practical layer of defense around risky processing workloads, rather than relying solely on preventing vulnerabilities.

Read original

Docker & Containers Webassembly C++Application Security+3

figmaOct 23, 20232 min readCurated summary

Server-side sandboxing: Containers and seccomp | Figma Blog

Containers and seccomp provide lightweight alternatives to virtual machines for isolating untrusted server-side workloads. Containers rely on OS-level features such as namespaces, cgroups, privilege dropping, and mandatory access controls, while seccomp restricts which system calls a process can invoke. Figma’s conclusion is that containers are not secure by default: effective sandboxing depends on the runtime, kernel, configuration, and broader infrastructure design. ## Container Isolation and Its Attack Surface - Container escapes depend on three main components: - The container runtime implementation - Operating-system primitives and interfaces exposed to the runtime - Runtime configuration - On Linux, Docker commonly uses the `runC` runtime alongside: - Namespaces and cgroups - Privilege dropping - Seccomp - SELinux or AppArmor - Vulnerabilities in the kernel or runtime, as well as configuration mistakes, can allow malicious workloads to modify host files or execute host-level code. - Unlike many VM solutions, containers place more responsibility on operators to configure isolation correctly. ## Risks from Compromised Containers - Simply running untrusted code inside a container does not guarantee safety. - Container settings should be strengthened to prevent host takeover. - The surrounding architecture should limit what a compromised container can access. - A safer design may use containers with: - No mounted network devices - No credentials - No access to unrelated data - Containers can be placed in an isolated network, with orchestration systems passing inputs and collecting outputs through controlled channels. ## Seccomp as an Additional Boundary - Seccomp, or “secure computing mode,” restricts the system calls available to a process. - This can reduce the kernel attack surface available to malicious code running inside a container. - Seccomp works alongside container mechanisms rather than replacing them; isolation depends on combining syscall restrictions with carefully configured namespaces, privileges, access controls, and infrastructure. ## Figma’s Evaluation Criteria Figma assesses sandboxing technologies using two questions: - Can a malicious workload escape its container and affect the host? - If it cannot escape, can it misuse the container’s permissions to reach other systems or cause harm? The practical recommendation is to treat containers as configurable security primitives, not automatically secure sandboxes. Use restrictive seccomp and container configurations, minimize credentials and connectivity, and design the surrounding system so that a compromised workload has limited impact.

Read original

Docker & Containers Seccomp Cgroups Container Sandboxing+4

datadogMay 24, 2023Original article

2023-03-08 incident: A deep dive into the platform-level impact | Datadog (opens in new tab)

The March 2023 Datadog outage was triggered by a simultaneous, global failure across multiple cloud providers and regions, caused by an unexpected interaction between a systemd security patch and Ubuntu 22.04’s default networking behavior. While Datadog typically employs rigorous, staged rollouts for infrastructure changes, the automated nature of OS-level security updates bypassed these controls. The incident highlights the hidden risks in system-level defaults and the potential for "unattended upgrades" to create synchronized failures across supposedly isolated environments. ## The systemd-networkd Routing Change * In December 2020, systemd version 248 introduced a change where `systemd-networkd` flushes all IP routing rules it does not recognize upon startup. * Version 249 introduced the `ManageForeignRoutingPolicyRules` setting, which defaults to "yes," confirming this management behavior for any rules not explicitly defined in systemd configuration files. * These changes were backported to earlier versions (v247 and v248) but were notably absent from v245, the version used in Ubuntu 20.04. ## Dormant Risks in the Ubuntu 22.04 Migration * Datadog began migrating its fleet from Ubuntu 20.04 to 22.04 in late 2022, eventually reaching 90% coverage across its infrastructure. * Ubuntu 22.04 utilizes systemd v249, meaning the majority of the fleet was susceptible to the routing rule flushing behavior. * The risk remained dormant during the initial rollout because `systemd-networkd` typically only starts during the initial boot sequence when no complex routing rules have been established yet. ## The Trigger: Unattended Upgrades and the CVE Patch * On March 7, 2023, a security patch for a systemd CVE was released to the Ubuntu security repositories. * Datadog’s fleet used the Ubuntu default configuration for `unattended-upgrades`, which automatically installs security-labeled patches once a day, typically between 06:00 and 07:00 UTC. * The installation of the patch forced a restart of the `systemd-networkd` service on active, running nodes. * Upon restarting, the service identified existing IP routing rules (crucial for container networking) as "foreign" and deleted them, effectively severing network connectivity for the nodes. ## Failure of Regional Isolation * Because the security patch was released globally and the automated upgrade window was synchronized across regions, the failure occurred nearly simultaneously worldwide. * This automation bypassed Datadog’s standard practice of "baking" changes in staging and experimental clusters for weeks before proceeding to production. * Nodes on the older Ubuntu 20.04 (systemd v245) were unaffected by the patch, as that version of systemd does not flush IP rules upon a service restart. To mitigate similar risks, infrastructure teams should consider explicitly disabling the management of foreign routing rules in systemd-networkd configuration when using third-party networking plugins. Furthermore, while automated security patching is a best practice, organizations must balance the speed of patching with the need for controlled, staged rollouts to prevent global configuration drift or synchronized failures.

Docker & Containers Kubernetes Systemd Ubuntu+4

datadogMar 25, 2022Original article

Using the Dirty Pipe vulnerability to break out from containers | Datadog (opens in new tab)

The Dirty Pipe vulnerability (CVE-2022-0847) is a critical Linux kernel flaw that allows unprivileged processes to write data to any file they can read, effectively bypassing standard write permissions. This primitive is particularly dangerous in containerized environments like Kubernetes, where it can be leveraged to overwrite the host’s container runtime binary. By exploiting how the kernel manages page caches, an attacker can achieve a full container breakout and gain administrative privileges on the underlying host. ## Container Runtimes and the OCI Specification * Kubernetes utilizes the Container Runtime Interface (CRI) to manage containers via high-level runtimes like containerd or CRI-O. * These high-level runtimes rely on low-level Open Container Interface (OCI) runtimes, most commonly runC, to handle the heavy lifting of namespaces and control groups. * Isolation is achieved by runC setting up a restricted environment before executing the user-supplied entrypoint via the `execve` system call. ## Evolution of runC Vulnerabilities * A historical vulnerability, CVE-2019-5736, previously allowed escapes by overwriting the host’s runC binary through the `/proc/self/exe` file descriptor. * To mitigate this, runC was updated to either clone the binary before execution or mount the host's runC binary as read-only inside the container. * While the read-only mount improved performance through kernel cache page sharing, it created a target for the Dirty Pipe vulnerability, which specifically targets the kernel page cache. ## The Dirty Pipe Exploitation Primitive * Dirty Pipe allows an attacker to overwrite any file they can read, including read-only files, by manipulating the kernel's internal pipe-buffer structures. * The exploit targets the page cache, meaning the overwrite is non-persistent and resides only in memory; the original file on disk remains unchanged. * In a container escape scenario, the attacker waits for a runC process to start (triggered by actions like `kubectl exec`) and targets the file descriptor at `/proc/<runC-pid>/exe`. ## Proof-of-Concept Escape Walkthrough * The attack begins with a standard, unprivileged pod running a malicious script that monitors the system for new runC processes. * Once a `kubectl exec` command is issued by an administrator, the script identifies the runC PID and applies the Dirty Pipe exploit to the associated executable. * The exploit overwrites the runC binary in the kernel page cache with a malicious ELF binary. * Because the host kernel is executing this hijacked binary with root privileges to manage the container, the attacker’s malicious code (e.g., a reverse shell or administrative command) runs with full host-level authority. To protect against this attack vector, it is essential to patch the Linux kernel to a version that includes the fix for CVE-2022-0847 and ensure that container nodes are running updated distributions.

Docker & Containers Kubernetes Linux Container Runtime+4

datadogFeb 10, 20213 min readCurated summary

Engineering spotlight: Maël Nison

Maël Nison’s journey from learning DarkBASIC on La Réunion and in Toulouse to becoming Yarn’s principal maintainer illustrates how curiosity, open source, and a focus on solving practical problems can shape a career. His early experimentation with games, websites, forums, and content-management systems developed into a lasting interest in improving developer workflows. That path eventually led through EPITECH, startups, Facebook, and Datadog, while giving him broad experience across both software and community leadership. ## Early Programming on La Réunion and in Toulouse - Maël grew up on the remote Indian Ocean island of La Réunion, where he had little access to computers. - After moving to Toulouse, he discovered a school programming club and began creating games with DarkBASIC. - DarkBASIC simplified 2D and 3D Windows game development through built-in libraries, tutorials, and DirectX support. - Seeing code immediately produce something on screen made programming feel logical and compelling to him. - By high school, he was building PHP websites, working with SQL, and experimenting with multiple languages and platforms. ## Discovering Open Source and Workflow Automation - In the early 2000s, distributing software was much harder because platforms such as GitHub did not yet exist. - Maël shared source archives through online forums, reflecting the informal nature of early open source communities. - His interest in forum software led to work on content-management systems. - He focused on reducing repetitive administrative workflows, such as allowing users to edit content directly instead of navigating through multiple administration pages. - This pattern—identifying a problem, building a solution, and sharing it with others—became a central theme in his career. ## Education at EPITECH - Maël attended EPITECH in Paris, an institution centered on practical technical education and self-directed learning. - The school emphasized peer assessment and hands-on projects rather than traditional, theory-heavy instruction. - He also spent a year abroad in Québec. - During his final year, he combined his studies with his first full-time job, gaining professional experience before graduation. ## Joining Facebook and Yarn - In 2017, after several years in startups, Maël moved from France to London seeking opportunities at larger organizations. - He joined Facebook without specifically intending to work on a package manager. - Facebook’s onboarding “boot camp” identified his skills and connected him with the emerging Yarn project. - He welcomed the opportunity to work on open source during his regular working hours. - What began as a few pull requests became a multi-year role as a major maintainer and leader of the project. ## Yarn’s Technical and Community Evolution - Yarn was rewritten in TypeScript and re-architected into a more modular system. - It evolved from an internal Facebook tool into a genuinely community-driven open source project. - Maël’s responsibilities expanded far beyond coding: - Product management and roadmap planning - Team leadership and infrastructure - Customer support and community work - Web design, evangelism, and outreach - Defining the project’s broader vision - Although he left Facebook for Datadog in 2019, he continued leading Yarn while taking on new challenges at Datadog. Maël’s experience suggests that careers can grow from small, self-directed experiments into major technical leadership opportunities. Developers can follow a similar path by solving concrete problems, sharing their work openly, and being willing to take on the technical, organizational, and community responsibilities that accompany successful projects.

Read original

Docker & Containers Database Design Open Source TypeScript+4

datadogAug 23, 20173 min readCurated summary

Scaling support with Vagrant and Terraform

Datadog’s Solutions Team uses reproducible virtual environments to investigate customer issues across diverse operating systems, kernels, and integrations. Vagrant simplifies local VM creation, while provisioning scripts eliminate repeated installation and configuration work. Terraform extends the same approach to shared AWS environments, enabling teams to provision, preserve, and collaborate on sandboxes quickly. ## Reproducing Customer Environments with Vagrant - Containers are useful, but virtual machines are better when reproducing specific operating systems, kernels, orchestrators, or complex infrastructure. - Vagrant provides a simple workflow: - `vagrant init` - `vagrant up` - `vagrant ssh` - The main challenge is not creating a VM, but installing and configuring the technologies needed to match a customer’s environment. - With more than 200 integrations, engineers cannot be experts in every technology they may need to troubleshoot. ## Standardizing Setup with Provisioning Scripts - Vagrant provisioning supports tools such as Chef, Puppet, Ansible, and ordinary shell scripts. - Datadog stores reusable reproduction environments in a shared GitHub repository. - Each sandbox includes: - A `Vagrantfile` - A `setup.sh` provisioning script - A `data` directory for configuration files and supporting scripts - A `README.md` with usage information - Engineer-specific values, such as hostnames and tags, are kept in a local `.sandbox.conf.sh` file. - Once a sandbox exists, an engineer can run `vagrant up` and begin reproducing the customer issue within minutes. - The directory hierarchy organizes sandboxes by operating system, version or provider, and technology—for example, Ubuntu Xenial with Kafka. ## Sharing Remote Environments with Terraform - Terraform provides similar infrastructure management for remote cloud instances, including AWS EC2. - The team reuses the same `setup.sh` and `data` files for both Vagrant and Terraform, avoiding duplicate configuration work. - Each sandbox adds a `.tf` file that: - Creates an EC2 instance - Copies required data files - Executes the provisioning script remotely - A shared Terraform module handles common infrastructure tasks, while a `tf.example` file helps engineers create new configurations. - This preserves the same repository structure and workflow while extending sandboxes from local VMs to remote environments. ## Benefits for Team Collaboration - Remote sandboxes can remain available without consuming engineers’ local RAM. - Proper network security allows teammates to access and share environments. - Engineers can reproduce previously configured integrations during live customer interactions. - Investigations can continue across time zones, allowing teams to hand off urgent issues without rebuilding the environment. The overall recommendation is to treat reproduction environments as reusable infrastructure: encode installation and configuration steps once, store them in version control, and use Vagrant for local testing and Terraform for persistent, shared cloud sandboxes.

Read original

Docker & Containers PostgreSQL Terraform Virtual Machines+4