Ai Infrastructure | Techlist.io

spotifyMar 12, 20263 min readCurated summary

Let’s Talk Agentic Development: Spotify x Anthropic Live | Spotify Engineering

Agentic development is reshaping software engineering at Spotify and Anthropic, from how developers write code to how organizations manage delivery. The discussion highlights Claude-powered agents, enterprise-scale context management, and the need to rethink testing, review, and accountability. The speakers conclude that agents will soon handle more of the full software lifecycle, including maintenance and deletion. ## The Opus 4.5 Inflection Point - Spotify observed a sharp increase in agent-driven development after Opus 4.5 went online on November 25, 2025. - Engineers increasingly shifted from working primarily in IDEs to using terminals and agent-based workflows. - The change was presented as a practical transformation in daily engineering work, not merely an experimental trend. ## Honk: Spotify’s Background Coding Agent - Spotify employees can invoke Honk by mentioning it in Slack. - Honk evolved from deterministic code migrations into a Slack-native agent capable of complex migrations across thousands of repositories. - Teams can discuss a problem in Slack and ask Honk to investigate or implement a solution directly. - Spotify is continuing to explore how background coding agents can operate at larger scale. ## Context Engineering and Control - Scaling agents across many repositories requires consistent, reproducible configuration. - Anthropic recommends well-structured `CLAUDE.md` files and reusable skills that describe engineering roles, domains, and expected workflows. - The emphasis is on simple, standardized context rather than overly complex orchestration. - Both companies are still identifying gaps in how agents receive context and how their actions are coordinated across enterprise systems. ## Testing, Reviews, and Accountability - Agent-generated code can be produced faster than humans can review it, creating new bottlenecks. - Organizations must reconsider testing, governance, and approval processes as output volume increases. - Accountability should remain tied to the outcome, regardless of whether code was produced by a human or an agent. - The discussion frames agent adoption as an organizational change, not just a tooling upgrade. ## The Next Stage of Agentic Development - The current phase has focused largely on code creation; the next phase will expand into maintenance, deletion, and other less popular but essential engineering work. - Spotify is evolving Backstage from a human-oriented developer portal into an agent-first platform. - MCP connections are expected to replace more manual developer workflows. - Anthropic’s internal “ant-fooding” practice continues to generate product ideas from employees using its own tools, including Claude Code and Cowork. Organizations adopting agentic development should start with reliable feedback loops, standardized context, and clear human accountability. The most significant gains will come when agents are integrated across the entire software lifecycle rather than used only for writing new code.

Read original

Ai Infrastructure Slack Claude Context Engineering+4

metaFeb 9, 20263 min readCurated summary

Building Prometheus: How Backend Aggregation Enables Gigawatt-Scale AI Clusters

Meta uses Backend Aggregation (BAG) as a high-capacity Ethernet super-spine to connect tens of thousands of GPUs across data centers and regions. In the Prometheus AI cluster, BAG links regional networks and Meta’s backbone while bridging two L2 fabric technologies: Disaggregated Schedule Fabric (DSF) and Non-Scheduled Fabric (NSF). Its modular hardware, resilient topologies, and advanced routing are designed to deliver reliable, petabit-scale connectivity for a gigawatt-scale AI system. ## Backend Aggregation’s Role - BAG interconnects multiple spine fabrics across data centers and regions. - It aggregates regional networks and connects them to Meta’s backbone. - Inter-BAG capacity can reach 16–48 petabits per second per regional pair. - Prometheus will span multiple buildings and connect tens of thousands of GPUs. ## Regional BAG Connectivity - BAG layers are distributed regionally to serve groups of L2 fabrics while respecting distance, latency, and buffer constraints. - Two connection topologies are used: - **Planar topology:** One-to-one connections between BAG switches in different regions; simpler to manage but creates more concentrated failure domains. - **Spread topology:** Links are distributed across switches and planes, improving path diversity and resilience. - The choice depends on site size and available fiber. ## Connecting DSF and NSF Fabrics - Meta’s L2 networks use both: - **Disaggregated Schedule Fabric (DSF)** - **Non-Scheduled Fabric (NSF)** - DSF zones across multiple buildings connect to BAG through backend edge pods. - NSF connects to BAG planes through matching Spine Training Switches. - Oversubscription is carefully managed: - L2-to-BAG oversubscription is typically around 4.5:1. - One NSF example has an effective ratio of 4.98:1. - BAG-to-BAG ratios vary by region and link capacity. ## Hardware and Routing - BAG uses modular chassis with Jericho3 ASIC line cards. - Each line card supports up to 432 800G ports. - Larger central-hub chassis support many spoke connections and long-distance links. - eBGP with link-bandwidth attributes enables Unequal Cost Multipath (UCMP), improving load balancing and failure recovery. - BAG-to-BAG links use MACsec for network security. ## Resilience and Failure Management - The design includes detailed port striping, IP addressing, and failure-domain analysis. - Failures are evaluated at the BAG, data-hall, and power-distribution levels. - Mitigation techniques include: - Draining affected BAG planes - Conditional route aggregation - Reducing blackholing risks during failures ## Managing Long-Distance Links - Distributed BAG architecture keeps L2-to-edge distances short, which benefits shallow-buffer NSF switches. - Longer BAG-to-BAG connections require deep-buffer switches. - These buffers provide headroom for lossless congestion-control mechanisms such as Priority Flow Control (PFC). ## Broader Impact BAG provides the networking foundation for Prometheus and future AI clusters. By combining regional aggregation, high-density hardware, resilient connection topologies, and fabric interoperability, Meta can scale AI infrastructure across multiple data centers while maintaining bandwidth, reliability, and operational flexibility.

Read original

Ai Infrastructure Backend Aggregation Gpu Networking Ethernet+4

kakaoJan 15, 2026Original article

Kanana-2 Development Story ( (opens in new tab)

Kakao has introduced Kanana-2, a series of language models utilizing a Mixture of Experts (MoE) architecture to achieve high intelligence while maintaining low inference costs. To support the stable pre-training of their largest 155B parameter model, the team implemented advanced technical stacks including the Muon optimizer and MuonClip to prevent training instabilities. These developments reflect a strategic focus on balancing large-scale performance with "high-efficiency, low-cost" engineering. ### MoE Architecture and Scaling Strategy * Kanana-2 models, such as the 32B version, activate only 3B parameters during inference to maximize computational efficiency without sacrificing the intelligence of a larger model. * The team is currently training a massive 155B parameter version (Kanana-2-155b-a17b) using FP8 training infrastructure, MuonClip, and Hyperparameter Transfer to ensure stable convergence. * Custom-developed MoE kernels were integrated to reduce memory usage and increase training speed, resulting in a highly stable Loss Curve even during constant learning rate phases. ### A Controlled Testbed for Mid- and Post-Training * The Kanana-2-30b-a3b-base-2601 model was intentionally released without synthetic reasoning data to serve as a "clean" base for research. * This model allows researchers to investigate phenomena like "Reasoning Trace Distribution Mismatch" and "Spurious Rewards" by providing a baseline unaffected by post-training interventions. * By offering a high-quality Korean base model, Kakao aims to support the local AI community in conducting more rigorous experiments on mathematical and logical reasoning. ### Optimization with Muon and Polar Express * Kakao shifted from the industry-standard AdamW optimizer to Muon, which updates parameters by orthogonalizing gradients rather than performing element-wise updates. * To achieve more accurate orthogonalization, they implemented the Polar Express iterative algorithm instead of the standard Newton-Schulz method, aiming to reduce noise in weight updates during the latter stages of large-scale training. * The optimization process also involved detailed adjustments to RMSNorm parameterization and learning rate (LR) management to ensure the model scales effectively. ### Training Stability via MuonClip * To address potential "logit explosion" in large-scale models, the team utilized MuonClip, a technique that clips attention logits to maintain stability. * Because standard Flash Attention stores Max Logit values only on-chip, the team modified the Flash Attention kernels to extract and return these values for monitoring and clipping purposes. * Stress tests conducted with high learning rates proved that MuonClip prevents training divergence and maintains performance levels even when the model is pushed to its limits. The development of Kanana-2 demonstrates that scaling to hundreds of billions of parameters requires more than just data; it necessitates deep architectural optimizations and custom kernel engineering. For organizations looking to train large-scale MoE models, adopting sophisticated orthogonalization optimizers and logit clipping mechanisms is highly recommended to ensure predictable and stable model convergence.

Ai Infrastructure Large Language Models Mixture Of Experts Pre Training+3

stripeDec 18, 20254 min readCurated summary

Stripe Atlas startups in 2025: Year in review

In 2025, early-stage startups launched faster, reached revenue sooner, and sold internationally from the beginning. Stripe Atlas data shows that these gains occurred despite a smaller share of startups raising funding, suggesting improved infrastructure and execution—not just venture capital—are driving growth. Founders are also increasingly building AI companies, particularly AI-agent businesses, although the post ends before fully explaining that shift. ## A More Global Startup Ecosystem - Atlas incorporations represented 169 countries in 2025, up from 158 in 2024. - European incorporations grew 48%, especially in the UK, France, and Germany, as founders sought access to US capital markets. - While 56% of Atlas startups are US-based, founding teams are increasingly distributed: - 24% of teams with multiple founders span more than one country. - This is a 79% increase since 2017. - Common pairings include Canada–US, UK–US, and India–US. - Distributed teams form both through prior in-person relationships and entirely online professional connections. ## International Sales from Launch - The typical startup sold to customers in two countries during its first six months in 2025, compared with one country in previous years. - Startups at the 90th percentile reached 15 countries, up from 12 in 2024. - Examples include: - Rork, which reached 69 countries in its first month and generated $100,000 in five days. - Zeabur, which served developers in 46 countries. - Payment infrastructure, compliance tools, cloud services, translation APIs, and globally distributed founders have reduced the barriers to international expansion. - Selling globally is increasingly a default launch strategy rather than a post-product-market-fit phase. ## Revenue Arrives Faster - The share of Atlas startups gaining a first paying customer within 30 days rose from 8% in 2020 to 20% in 2025. - Among startups that began accepting payments within three months, median time to first payment fell from 38 to 34 days. - Atlas’s 2025 payment changes allowed founders to accept payments immediately after incorporation, avoiding lengthy EIN delays for non-US founders. - Median first-six-month revenue increased 39% year over year, indicating that faster monetization reflects stronger product shipping and customer acquisition as well as better infrastructure. ## More Startups Reach Significant Revenue - The number of startups reaching $100,000 in their first six months rose 56% from 2024. - These companies reached that milestone in 108 days, compared with 121 days previously. - The average startup acquired 242 customers in its first six months, more than 50% above the prior year. - Growth was strongest among top performers: - 10th-percentile startups generated 18% more revenue than comparable 2024 companies. - 90th-percentile startups generated 52% more. - The overall market improved, but the gap between breakout companies and average performers widened. ## AI Becomes a Dominant Startup Focus - The share of Atlas founders identifying their companies as AI startups grew from 15% in 2023 to 33% in 2024 and 42% in 2025. - AI adoption also expanded among LLCs, rising from 5% in 2023 to 22% in 2025. - This growth occurred alongside weaker early fundraising: - Only 2.2% of Atlas startups at least six months old raised funding within three months of incorporation, down from 3.1% in 2024. - Pre-seed deal volume was largely unchanged even as Delaware C-corp formations increased. - The supplied article indicates that founders are shifting attention toward AI agents rather than AI infrastructure or copilots, but the provided text ends before detailing that trend. ## Practical Conclusion Modern payments, cloud infrastructure, compliance tools, and developer platforms are enabling founders to launch, monetize, and expand globally within weeks. Early startups increasingly need to assume international reach from day one while focusing on rapid customer validation and efficient execution, particularly as funding becomes less automatic.

Read original

Ai Infrastructure AI Agents Cloud Infrastructure Fintech+4

metaNov 21, 2025Original article

Zoomer: Powering AI Performance at Meta's Scale Through Intelligent Debugging and Optimization (opens in new tab)

Zoomer is Meta’s centralized, automated platform designed to solve performance bottlenecks and GPU underutilization across its massive AI training and inference infrastructure. By integrating deep analytics with scalable data collection, the tool has become the internal standard for optimizing workloads ranging from Llama 3 training to large-scale ads recommendation engines. Ultimately, Zoomer enables significant energy savings and hardware efficiency gains, allowing Meta to accelerate model iteration and increase throughput across its global fleet of GPUs. ### The Three-Layered Architecture * **Infrastructure and Platform Layer:** This foundation utilizes Meta’s Manifold blob storage for trace data and employs fault-tolerant processing pipelines to manage massive trace files across thousands of hosts. * **Analytics and Insights Engine:** This layer performs deep analysis using specialized tools such as Kineto for GPU traces, NVIDIA DCGM for hardware metrics, and StrobeLight for CPU profiling. It automatically detects performance anti-patterns and provides actionable optimization recommendations. * **Visualization and User Interface Layer:** The presentation layer transforms complex data into interactive timelines and heat maps. It integrates with Perfetto for kernel-level inspection and provides drill-down dashboards that highlight outliers across distributed GPU deployments. ### Automated Profiling and Data Capture * **Trigger Mechanisms:** To ensure data accuracy, Zoomer automatically triggers profiling for training workloads during stable states (typically around iteration 550) to avoid startup noise, while inference workloads use on-demand or benchmark-integrated triggers. * **Comprehensive Metrics:** The platform simultaneously collects GPU SM utilization, Tensor Core usage, memory bandwidth, and power consumption via DCGM. * **System-Level Telemetry:** Beyond the GPU, Zoomer captures host-level data including CPU utilization, storage access patterns, and network I/O through dyno telemetry. * **Distributed Communication:** For large-scale training, the tool analyzes NCCL collective operations and inter-node communication patterns to identify stragglers and network bottlenecks. ### Inference and Training Optimization * **Inference Performance:** Zoomer tracks request/response latency, GPU memory allocation patterns, and Thrift request-level profiling to identify bottlenecks in serving user requests at scale. * **Workflow Acceleration:** By correlating application-level annotations—such as forward/backward passes and optimizer steps—with hardware performance, developers can pinpoint exactly which part of a model's execution is inefficient. * **Operational Impact:** These insights have led to significant improvements in Queries Per Second (QPS) for recommendation models and reduced training times for generative AI features by eliminating resource waste. For organizations managing large-scale AI clusters, the Zoomer model suggests that the key to efficiency is moving away from manual, reactive debugging toward an "always-on" automated profiling system. Correlating high-level software phases with low-level hardware telemetry is essential for maximizing the return on investment for expensive GPU resources and maintaining rapid iteration cycles.

Ai Infrastructure Distributed Training Gpu Optimization Performance Profiling+4

googleNov 3, 2025Original article

Exploring a space-based, scalable AI infrastructure system design (opens in new tab)

Project Suncatcher is a Google moonshot initiative aimed at scaling machine learning infrastructure by deploying solar-powered satellite constellations equipped with Tensor Processing Units (TPUs). By leveraging the nearly continuous energy of the sun in specific orbits and utilizing high-bandwidth free-space optical links, the project seeks to bypass the resource constraints of terrestrial data centers. Early research suggests that a modular, tightly clustered satellite design can achieve the necessary compute density and communication speeds required for modern AI workloads. ### Data-Center Bandwidth via Optical Links * To match terrestrial performance, inter-satellite links must support tens of terabits per second using multi-channel dense wavelength-division multiplexing (DWDM) and spatial multiplexing. * The system addresses signal power loss (the link budget) by maintaining satellites in extremely close proximity—kilometers or less—compared to traditional long-range satellite deployments. * Initial bench-scale demonstrations have successfully achieved 800 Gbps each-way transmission (1.6 Tbps total) using a single transceiver pair, validating the feasibility of high-speed optical networking. ### Orbital Mechanics of Compact Constellations * The proposed system utilizes a sun-synchronous low-earth orbit (LEO) at an altitude of approximately 650 km to maximize solar exposure and minimize the weight of onboard batteries. * Researchers use Hill-Clohessy-Wiltshire equations and JAX-based differentiable models to manage the complex gravitational perturbations and atmospheric drag affecting satellites flying in tight 100–200m formations. * Simulations of 81-satellite clusters indicate that only modest station-keeping maneuvers are required to maintain stable, "free-fall" trajectories within the orbital plane. ### Hardware Resilience in Space Environments * The project specifically tests Google’s Trillium (v6e) Cloud TPUs to determine if terrestrial AI accelerators can survive the radiation found in LEO. * Hardware is subjected to 67MeV proton beams to analyze the impact of Total Ionizing Dose (TID) and Single Event Effects (SEEs) on processing reliability. * Preliminary testing indicates promising results for the radiation tolerance of high-performance accelerators, suggesting that standard TPU architectures may be viable for orbital deployment with minimal modification. While still in the research and development phase, Project Suncatcher suggests that the future of massive AI scaling may involve shifting infrastructure away from terrestrial limits and toward modular, energy-rich orbital environments. Organizations should monitor the progress of free-space optical communication and radiation-hardened accelerators as these technologies will be the primary gatekeepers for space-based computation.

Ai Infrastructure AI Machine Learning Jax+5

dropboxAug 27, 20252 min readCurated summary

Hack Week 2025: How these engineers liquid-cooled a GPU server

Dropbox engineers used Hack Week 2025 to build a liquid-cooling system for GPU servers, anticipating the rising heat and power demands of AI workloads. Their prototype, assembled from radiators, fans, a pump, reservoir, tubing, manifolds, and sensors, reduced temperatures by 20–30°C during stress tests compared with air cooling. The project also enabled quieter operation and may help Dropbox use less data-center space and energy as GPU requirements increase. ## Building a Custom Liquid-Cooling System - The team built the system from scratch after failing to source a complete commercial setup in time. - It replicated key data-center cooling components: - Radiators and fans for heat dissipation - A pump and reservoir to circulate coolant - Tubing and manifolds - Sensors to verify flow and monitor performance - Engineers tested the cooling loop before connecting expensive GPUs, then integrated it with a GPU server. ## Thermal and Noise Improvements - Liquid cooling lowered CPU and GPU temperatures by approximately 20–30°C during demanding torture tests. - Because the liquid loop handled the primary heat-generating components, the team could: - Remove some fans - Run others at lower speeds - Reduce noise and potentially power consumption - Remaining airflow was directed toward lower-heat components such as DIMMs and network cards. - The team considered using an airflow baffle to cool those components more precisely. ## Preparing for AI-Era Infrastructure - High-end GPUs increasingly consume more power and generate more heat, potentially making air cooling insufficient. - Liquid cooling could allow Dropbox to: - Fit more powerful servers into existing data-center footprints - Avoid spreading servers across additional space - Reduce cooling energy and operating costs - Although vendors do not yet universally require liquid cooling for top-tier GPUs, the engineers expect that requirement to become more common. - Dropbox’s growing focus on AI workloads provided additional motivation to investigate the technology early. ## Hack Week as an Experimentation Platform - The project received funding and organizational support from Dropbox’s infrastructure teams. - Hack Week gave engineers time to explore a long-term infrastructure problem outside their normal work. - The team’s Learn Fast award recognized the project’s emphasis on experimentation and rapid learning. - Working in person helped the engineers exchange ideas, troubleshoot quickly, and collaborate with colleagues across the company. Dropbox plans to expand testing with additional liquid-cooling labs in multiple data centers. The prototype is an early step toward infrastructure capable of supporting increasingly powerful, GPU-intensive AI systems.

Read original

Ai Infrastructure Gpu Computing Liquid Cooling Thermal Management+3