Machine Learning | Techlist.io

metaDec 19, 2025Original article

DrP: Meta's Root Cause Analysis Platform at Scale (opens in new tab)

DrP is Meta’s programmatic root cause analysis (RCA) platform designed to automate incident investigations and reduce the burden of manual on-call tasks. By codifying investigation playbooks into executable "analyzers," the platform significantly lowers the mean time to resolve (MTTR) by 20% to 80% for over 300 teams. This systematic approach replaces outdated manual scripts with a scalable backend that executes 50,000 automated analyses daily, providing immediate context when alerts fire. ## Architecture and Core Components * **Expressive SDK:** Provides a framework for engineers to codify investigation workflows into "analyzers," utilizing a rich library of helper functions and machine learning algorithms. * **Built-in Analysis Tools:** The platform includes native support for anomaly detection, event isolation, time-series correlation, and dimension analysis to identify specific problem areas. * **Scalable Backend:** A multi-tenant execution environment manages a worker pool that handles thousands of requests securely and asynchronously. * **Workflow Integration:** DrP is integrated directly into Meta’s internal alerting and incident management systems, allowing for automatic triggering without human intervention. ## Authoring and Verification Workflow * **Template Bootstrapping:** Engineers use the SDK to generate boilerplate code that captures required input parameters and context in a type-safe manner. * **Analyzer Chaining:** The system allows for seamless dependency analysis by passing context between different analyzers, enabling investigations to span multiple interconnected services. * **Automated Backtesting:** Before deployment, analyzers undergo automated backtesting integrated into the code review process to ensure accuracy and performance. * **Decision Tree Logic:** Investigation steps are modeled as decision trees within the code, allowing the analyzer to follow different paths based on the data it retrieves. ## Execution and Post-Processing * **Trigger-based Analysis:** When an alert is activated, the backend automatically queues the relevant analyzer, ensuring findings are available as soon as an engineer begins triaging. * **Automated Mitigation:** A post-processing system can take direct action based on investigation results, such as creating tasks or submitting pull requests to resolve identified issues. * **DrP Insights:** This system periodically reviews historical analysis outputs to identify and rank the top causes of alerts, helping teams prioritize long-term reliability fixes. * **Alert Annotation:** Results are presented in both human-readable text and machine-readable formats, directly annotating the incident logs for the on-call responder. ## Practical Conclusion Organizations managing large-scale distributed systems should transition from static markdown playbooks to executable investigation code. By implementing a programmatic RCA framework like DrP, teams can scale their troubleshooting expertise and significantly reduce "on-call fatigue" by automating the repetitive triage steps that typically consume the first hour of an incident.

Machine Learning AI Observability Automation+4

woowahanDec 19, 2025Original article

Delivering the Future: Global Hackathon (opens in new tab)

The Global Hackathon 2025 served as a massive collaborative initiative to unite over 270 technical employees from seven global entities under DeliveryHero’s umbrella, including Woowa Brothers. By leveraging the community-building expertise of the Woowahan DevRel team, the event successfully bridged geographical and technical gaps to foster innovation in "Delivering the Future." The hackathon concluded with high-level recognition from global leadership and a strategic partnership with Google Cloud, demonstrating the power of synchronized global technical synergy. ## Strategic Planning and Global Coordination * The event adopted a hybrid "Base Camp" model, where participants worked from their local entity offices while staying connected through 24-hour live streaming and centralized online channels. * Organizers meticulously navigated the logistical hurdles of spanning 70 countries, including coordinating across vastly different time zones and respecting local public holidays and vacation seasons. * Efficiency was maintained through a decentralized communication strategy, using entity-specific meetings and comprehensive guidebooks rather than frequent global meetings to prevent "meeting fatigue" across time zones. ## Technical Infrastructure and Regulatory Compliance * To accommodate diverse technical preferences, the infrastructure had to support various stacks, including AWS, Google Cloud Platform (GCP), and specific machine learning models. * The central organization team addressed complex regulatory challenges, ensuring all sandbox environments complied with strict global security standards and GDPR (EU General Data Protection Regulation). * A strategic partnership with Google Cloud provided a standardized Google AI-based environment, enabling teams to experiment rapidly with mature tools and cloud-native services. ## Local Operations and Cross-Entity Collaboration * Physical office spaces were transformed into immersive hackathon hubs to maintain the high-intensity atmosphere characteristic of offline coding marathons. * The event encouraged "office sharing" between entities located in the same city and even supported travel for members to join different regional base camps, fostering a truly global networking culture. * Local supporters used standardized checklists and operational frameworks to ensure a consistent experience for participants, whether they were in Seoul, Berlin, or Dubai. Building a successful global technical event requires a delicate balance between centralized infrastructure and local autonomy. For organizations operating across multiple regions, investing in shared technical sandboxes and robust communication frameworks is essential for turning fragmented local talent into a unified global innovation engine.

Machine Learning AI Security Google Cloud+4

dropboxDec 18, 20253 min readCurated summary

Inside the feature store powering real-time AI in Dropbox Dash

Dropbox Dash’s ranking system depends on a hybrid feature store that can combine real-time user behavior with large-scale historical data. Because Dropbox operates across on-premises and cloud environments, and because each query can trigger thousands of feature lookups, off-the-shelf systems could not meet its latency, scale, and integration requirements. The resulting architecture uses Feast for orchestration, Spark for computation, Dynovault for low-latency storage, and a custom Go serving layer, achieving roughly 25–35 ms p95 latency while keeping features fresh. ## Goals and Requirements - Dash ranks documents, images, and conversations using behavioral, contextual, and real-time signals. - A single query can fan out into thousands of feature lookups across many candidate files. - The feature store needed to: - Support sub-100 ms search latency. - Reflect user actions within seconds or minutes. - Bridge Dropbox’s on-premises services and Spark-based cloud infrastructure. - Handle both streaming-style updates and batch computations. - Let engineers develop features without managing serving and orchestration details. ## Choosing a Hybrid Architecture - Dropbox evaluated Feast, Hopsworks, Featureform, Feathr, Databricks, and Tecton. - Feast was selected because: - It separates feature definitions from infrastructure concerns. - Engineers can focus on PySpark transformations. - Its modular adapter system supports existing Dropbox infrastructure. - Feast’s DynamoDB adapter enabled integration with Dynovault, Dropbox’s DynamoDB-compatible storage system. - The architecture combines: - Feast for orchestration and serving APIs. - Spark jobs for feature computation and ingestion. - Cloud storage for offline indexing and data management. - Dynovault for online, low-latency lookups. - A custom Go service replacing Feast’s Python online serving path. - Dynovault is colocated with inference workloads and provides approximately 20 ms client-side latency. - Monitoring covers job failures, feature freshness, and data lineage. ## Replacing Python with Go for Low Latency - The initial Feast-based Python service struggled under heavy concurrency. - CPU-bound JSON parsing and Python’s Global Interpreter Lock became bottlenecks. - A multi-process design helped temporarily but introduced coordination overhead. - The serving layer was rewritten in Go using: - Lightweight goroutines. - Shared memory. - Faster JSON parsing. - The Go service now handles thousands of requests per second. - It adds only about 5–10 ms beyond Dynovault latency and achieves roughly 25–35 ms p95 latency. ## Keeping Features Fresh - Fresh signals are essential for ranking quality; actions such as opening a document should influence subsequent searches quickly. - Fully real-time computation is impractical for features requiring large joins, aggregations, and historical context. - Dropbox therefore built a three-part ingestion strategy. - Batch ingestion handles complex, high-volume transformations using a medallion architecture. - Intelligent change detection updates only modified records rather than rewriting every feature. - This reduced online-store writes from hundreds of millions to fewer than one million per run and significantly shortened update time. ## Practical Takeaway The system demonstrates that a feature store does not need to be entirely off-the-shelf or entirely real-time. Combining a modular framework with custom serving, colocated storage, batch optimization, and freshness monitoring allowed Dropbox to meet demanding latency and scale requirements while keeping feature development manageable.

Read original

Machine Learning Go Feature Store Dynamodb+4

googleDec 11, 2025Original article

Spotlight on innovation: Google-sponsored Data Science for Health Ideathon across Africa (opens in new tab)

Google Research, in partnership with several pan-African machine learning communities, recently concluded the Africa-wide Data Science for Health Ideathon to address regional medical challenges. By providing access to specialized open-source health models and technical mentorship, the initiative empowered local researchers to develop tailored solutions for issues ranging from maternal health to oncology. The event demonstrated that localized innovation, supported by high-performance AI foundations, can effectively bridge healthcare gaps in resource-constrained environments. ## Collaborative Framework and Objectives * The Ideathon was launched at the 2025 Deep Learning Indaba in Kigali, Rwanda, in collaboration with SisonkeBiotik, Ro’ya, and DS-I Africa. * The primary goal was to foster capacity building within the African AI community, moving beyond theoretical research toward the execution of practical healthcare tools. * Participants received hands-on training on Google’s specialized health models and were supported with Google Cloud Vertex AI compute credits and mentorship from global experts. * Submissions were evaluated based on their innovation, technical feasibility, and contextual relevance to African health systems. ## Technical Foundations and Google Health Models * Developers focused on a suite of open health AI models, including MedGemma for clinical reasoning, TxGemma for therapeutics, and MedSigLIP for medical vision-language tasks. * The competition utilized a two-phase journey: an initial "Idea Development" stage where teams defined clinical problems and outlined AI approaches, followed by a "Prototype & Pitch" phase. * Technical implementations frequently involved advanced techniques such as Retrieval-Augmented Generation (RAG) to ensure alignment with local medical protocols and WHO guidelines. * Fine-tuning methods, specifically Low-Rank Adaptation (LoRA), were utilized by teams to specialize large-scale models like MedGemma-27B-IT for niche datasets. ## Innovative Solutions for Regional Health * **Dawa Health:** This first-place winner developed an AI-powered cervical cancer screening tool that uses MedSigLIP to identify abnormalities in colposcopy images uploaded via WhatsApp, combined with Gemini RAG for clinical guidance. * **Solver (CerviScreen AI):** This team built a web application for automated cervical-cytology screening by fine-tuning MedGemma-27B-IT on the CRIC dataset to assist cytopathologists with annotated images. * **Mkunga:** A maternal health call center that adapts MedGemma and Gemini to provide advice in Swahili using Speech-to-Text (STT) and Text-to-Speech (TTS) technologies. * **HexAI (DermaDetect):** Recognized for the best proof-of-concept, this offline-first mobile app allows community health workers to triage skin conditions using on-device versions of MedSigLIP, specifically designed for low-connectivity areas. The success of the Ideathon underscores the importance of "local solutions for local priorities." By making sophisticated models like MedGemma and MedSigLIP openly available, the technical barrier to entry is lowered, allowing African developers to build high-impact, culturally and linguistically relevant medical tools. For organizations looking to implement AI in global health, this model of providing foundational tools and cloud resources to local experts remains a highly effective strategy for sustainable innovation.

Machine Learning AI Generative AI RAG+5

woowahanDec 11, 2025Original article

Enhancing the “Frequently Bought (opens in new tab)

Baedal Minjok (Baemin) has significantly improved its cart recommendation system by transitioning from a basic Item2Vec model to a sophisticated two-stage architecture that combines graph-based embeddings with Transformer sequence modeling. This evolution addresses the "substitutability bias" and lack of sequential context found in previous methods, allowing the system to understand the specific intent behind a user's shopping journey. By moving beyond simple item similarity, the new model effectively identifies cross-selling opportunities that align with the logical flow of a customer's purchase behavior. ### Limitations of the Item2Vec Approach * **Substitutability Bias:** The original Item2Vec model, based on the Skip-gram architecture, tended to map items from the same category into similar vector spaces. This resulted in recommending alternative brands of the same product (e.g., suggesting another brand of milk) rather than complementary goods (e.g., cereal or bread). * **Loss of Sequential Context:** Because Item2Vec treats a basket of goods as a "bag of words," it ignores the order in which items are added. This prevents the model from distinguishing between different user intents, such as a user starting with meat to grill versus a user starting with ingredients for a stew. * **Failure in Cross-Selling:** The primary goal of cart recommendations is to encourage cross-selling, but the reliance on embedding similarity alone limited the diversity of suggestions, often trapping users within a single product category. ### Stage 1: Graph-Based Product and Category Embeddings * **Node2Vec Implementation:** To combat data sparsity and the "long-tail" problem where many items have low purchase frequency, the team utilized Node2Vec. This method uses random walks to generate sequences that help the model learn structural relationships even when direct transaction data is thin. * **Heterogeneous Graph Construction:** The graph consists of both "Item Nodes" and "Category Nodes." Connecting items to their respective categories allows the system to generate initial vectors for new or low-volume products that lack sufficient historical purchase data. * **Association Rule Weighting:** Rather than using simple co-occurrence counts for edge weights, the team applied Association Rules. This ensures that weights reflect the actual strength of the complementary relationship, preventing popular "mega-hit" items from dominating all recommendation results. ### Stage 2: Transformer-Based Sequence Recommendation * **Capturing Purchase Context:** The second stage employs a Transformer model to analyze the sequence of items currently in the user's cart. This architecture is specifically designed to understand how the meaning of an item changes based on what preceded it. * **Next Item Prediction:** Using the pre-trained embeddings from Stage 1 as inputs, the Transformer predicts the most likely "next item" a user will add. This allows the system to provide dynamic recommendations that evolve as the user continues to shop. * **Integration of Category Data:** By feeding both item-level and category-level embeddings into the Transformer, the model maintains a high level of accuracy even when a user interacts with niche products, as the category context provides a fallback for the recommendation logic. ### Practical Conclusion For production-scale recommendation systems, relying solely on item similarity often leads to redundant suggestions that do not drive incremental sales. By decoupling the learning of structural relationships (via graphs) from the learning of temporal intent (via Transformers), engineers can build a system that is robust against data sparsity while remaining highly sensitive to the immediate context of a user's session. This two-stage approach is recommended for e-commerce environments where cross-category discovery is a key business metric.

Machine Learning AI Deep Learning Transformer+5

awsDec 8, 2025Original article

AWS Weekly Roundup: AWS re:Invent keynote recap, on-demand videos, and more (December 8, 2025) (opens in new tab)

The December 8, 2025, AWS Weekly Roundup recaps the major themes from AWS re:Invent, signaling a significant industry transition from AI assistants to autonomous AI agents. While technical innovation in infrastructure remains a priority, the event underscored that developers remain at the heart of the AWS mission, empowered by new tools to automate complex tasks using natural language. This shift represents a "renaissance" in cloud computing, where purpose-built infrastructure is now designed to support the non-deterministic nature of agentic workloads. ## Community Recognition and the Now Go Build Award * Raphael Francis Quisumbing (Rafi) from the Philippines was honored with the Now Go Build Award, presented by Werner Vogels. * A veteran of the ecosystem, Quisumbing has served as an AWS Hero since 2015 and has co-led the AWS User Group Philippines for over a decade. * The recognition emphasizes AWS's continued focus on community dedication and the role of individual builders in empowering regional developer ecosystems. ## The Evolution from AI Assistants to Agents * AWS CEO Matt Garman identified AI agents as the next major inflection point for the industry, moving beyond simple chat interfaces to systems that perform tasks and automate workflows. * Dr. Swami Sivasubramanian highlighted a paradigm shift where natural language serves as the primary interface for describing complex goals. * These agents are designed to autonomously generate plans, write necessary code, and call various tools to execute complete solutions without constant human intervention. * AWS is prioritizing the development of production-ready infrastructure that is secure and scalable specifically to handle the "non-deterministic" behavior of these AI agents. ## Core Infrastructure and the Developer Renaissance * Despite the focus on AI, AWS reaffirmed that its core mission remains the "freedom to invent," keeping developers central to its 20-year strategy. * Leaders Peter DeSantis and Dave Brown reinforced that foundational attributes—security, availability, and performance—remain the non-negotiable pillars of the AWS cloud. * The integration of AI agents is framed as a way to finally realize material business returns on AI investments by moving from experimental use cases to automated business logic. To maximize the value of these updates, organizations should begin evaluating how to transition from simple LLM implementations to agentic frameworks that can execute end-to-end business processes. Reviewing the on-demand keynote sessions from re:Invent 2025 is recommended for technical teams looking to implement the latest secure, agent-ready infrastructure.

Machine Learning AI AI Agents Generative AI+3

pinterestDec 8, 20254 min readCurated summary

How Pinterest Built a Real‑Time Radar for Violative Content using AI

Pinterest built an AI-assisted prevalence measurement system to estimate how often users actually see policy-violating content, rather than relying only on user reports. The system samples daily impressions, uses production risk scores to improve efficiency, labels content with a multimodal LLM, and applies statistical reweighting to preserve unbiased estimates. This enables daily, segmented monitoring with substantially lower cost and latency than human-only review. ## Why Prevalence Matters - User reports miss important harms because: - Some sensitive issues, such as self-harm, are under-reported. - Users seeking harmful content may not report it. - Rare policy categories provide too few reports for reliable trend detection. - Human review of reports is expensive and slow. - Prevalence measures exposure: the share of total views directed to violating content. - This helps Pinterest identify under-reported harms, evaluate interventions, and detect changes earlier. - Human-only prevalence studies were previously conducted only about every six months and required multiple reviewers plus adjudication. ## What Pinterest Measures - Daily prevalence is calculated as: - **Views of content violating a policy ÷ total views** - For example, 10 violating views in a sample of 100,000 produces an estimated prevalence of 0.01%. - Results include 95% confidence intervals to communicate statistical precision. - Metrics can be segmented by: - Policy area, such as Adult Content, Self-harm, or Graphic Violence - Sub-policy, such as nudity versus explicit sexual content - Surface, including Homefeed, Search, and Related Pins - Content age, geography, and user-age groups where relevant ## Risk-Aware, Unbiased Sampling - Pinterest samples from the daily user-impressions stream. - Production enforcement risk scores are used to prioritize likely high-risk and high-exposure content, but they are not treated as labels or eligibility rules. - Missing scores are replaced with the day’s median so that new content remains eligible. - Weighted reservoir sampling approximates probability-proportional-to-size sampling, considering impressions and risk scores. - Inverse-probability weighting removes the bias introduced by risk-based sampling, ensuring estimates represent impressions rather than model thresholds. - Pinterest uses Hansen–Hurwitz ratio estimators for sampling with replacement and Horvitz–Thompson ratio estimators for sampling without replacement. - Pure random sampling is also available for validation studies. ## LLM-Based Labeling - A multimodal LLM analyzes sampled content using both images and text. - Prompts are reviewed by policy subject-matter experts and can return structured label hierarchies such as `safe`, `not_safe`, and `unsure`. - Each decision records: - The label and brief rationale - Policy version - Prompt and model identifiers - Token usage and run cost - Human validation is performed on strategically selected samples to identify edge cases and AI blind spots. - The LLM is tested against human-reviewed gold sets before launch and periodically afterward to detect drift. - The workflow is reportedly 15 times faster and far cheaper than human-only labeling while maintaining comparable decision quality and statistical governance. ## Production System and Monitoring - Inputs include entity-by-day engagement data such as impressions, clicks, hides, and reports, alongside current production risk scores. - The system stores prevalence estimates, sampling weights, labels, diagnostics, and lineage for audits. - Dashboards display: - Daily prevalence and 95% confidence intervals - Confidence-interval width and effective sample size - Sample positive rate - Risk-score distributions - Prompt, model, taxonomy, and metric versions - Teams can pivot results by policy, sub-policy, and surface. - Validation samples and run-health information help monitor both statistical quality and operational reliability. Pinterest’s approach combines probability sampling, inverse-probability estimation, and continuously calibrated multimodal AI labeling to create a daily radar for harmful exposure. The practical recommendation is to use AI to scale measurement, but retain rigorous sampling, human validation, confidence intervals, and full model and policy lineage so that faster estimates remain trustworthy.

Read original

Machine Learning Large Language Models Multimodal AI Trust And Safety+3

pinterestDec 5, 20253 min readCurated summary

Improving Quality of Recommended Content through Pinner Surveys

Pinterest uses Pinner surveys to measure visual quality and incorporate user preferences into recommendation systems, rather than optimizing solely for engagement. The company surveyed 5,000 Pins, trained a lightweight neural network to predict average perceived quality, and applied the resulting model across Homefeed, Related Pins, and Search. This approach aims to reduce clickbait and promote content that supports positive, long-term user experiences. ## Why Engagement Alone Is Insufficient - High engagement does not necessarily indicate high-quality content; optimizing for clicks can promote clickbait or harmful material. - Pinterest defines quality as content that feels good, inspires further exploration, and encourages fulfilling long-term engagement. - Direct user feedback helps recommendation systems prioritize content that Pinners actually value. - The work supports Pinterest’s Inspired Internet Pledge principles, especially listening to users and tuning the platform for wellbeing. ## Collecting Pinner Quality Ratings - Pinners rated images from 1 to 5 in response to: “How visually pleasing or displeasing is this Pin?” - Pinterest collected ratings for 5,000 Pins, sampling 1,000 from each of five major interest categories: - Art - Beauty - DIY & Crafts - Home Decor - Women’s Fashion - Pins were sampled based on impressions and were generally mid-to-high quality rather than deliberately exposing users to poor content. - Each image received at least 10 ratings, allowing Pinterest to average responses and reduce noise from subjectivity or accidental misclicks. - Surveys were considered appropriate for visual appeal, which is subjective but still measurable across many users. More objective issues should be evaluated by trained reviewers, while highly contextual judgments such as personal relevance are harder to capture with a single Pin-level score. - Highly rated content included makeup, grooming styles, maximalist interiors, landscapes, sunsets, and baby animals. - Home Decor images tended to receive higher ratings overall, while Art showed the greatest variation, reflecting its subjective nature. ## Training a Visual-Quality Model - Pinterest trained a model to estimate the average Pinner’s perception of visual quality from image embeddings. - Embeddings encode visual, textual, and behavioral information, including relationships between images and the boards where they are saved. - The model produces a score from 0 to 1, with higher values representing greater perceived quality. - Pinterest chose a small fully connected neural network with approximately 92,000 parameters: - The limited size helps prevent overfitting to the 5,000-image dataset. - It also makes large-scale inference faster and less expensive. - Instead of predicting an exact rating, the model uses pairwise ranking: - It learns which of two images Pinners would consider better. - The comparison is based on each image’s mean survey rating. - Training comparisons are restricted to images within the same top-level interest category, encouraging the model to learn visual quality rather than simply recognizing that one topic is more popular than another. Pinterest’s approach demonstrates how survey-based quality signals can complement engagement metrics. Training recommendation systems on what users perceive as appealing can help the platform promote more satisfying content while reducing incentives to favor attention-grabbing but low-quality material.

Read original

Machine Learning Recommendation Systems Embeddings Neural Networks+3

awsDec 3, 2025Original article

Amazon Bedrock adds reinforcement ﬁne-tuning simplifying how developers build smarter, more accurate AI models (opens in new tab)

Amazon Bedrock has introduced reinforcement fine-tuning, a new model customization capability that allows developers to build more accurate and cost-effective AI models using feedback-driven training. By moving away from the requirement for massive labeled datasets in favor of reward signals, the platform enables average accuracy gains of 66% while automating the complex infrastructure typically associated with advanced machine learning. This approach allows organizations to optimize smaller, faster models for specific business needs without sacrificing performance or incurring the high costs of larger model variants. **Challenges of Traditional Model Customization** * Traditional fine-tuning often requires massive, high-quality labeled datasets and expensive human annotation, which can be a significant barrier for many organizations. * Developers previously had to choose between settle for generic "out-of-the-box" results or managing the high costs and complexity of large-scale infrastructure. * The high barrier to entry for advanced reinforcement learning techniques often required specialized ML expertise that many development teams lack. **Mechanics of Reinforcement Fine-Tuning** * The system uses an iterative feedback loop where models improve based on reward signals that judge the quality of responses against specific business requirements. * Reinforcement Learning with Verifiable Rewards (RLVR) utilizes rule-based graders to provide objective feedback for tasks such as mathematics or code generation. * Reinforcement Learning from AI Feedback (RLAIF) uses AI-driven evaluations to help models understand preference and quality without manual human intervention. * The workflow can be powered by existing API logs within Amazon Bedrock or by uploading training datasets, eliminating the need for complex infrastructure setup. **Performance and Security Advantages** * The technique achieves an average accuracy improvement of 66% over base models, enabling smaller models to perform at the level of much larger alternatives. * Current support includes the Amazon Nova 2 Lite model, which helps developers optimize for both speed and price-to-performance. * All training data and customization processes remain within the secure AWS environment, ensuring that proprietary data is protected and compliant with organizational security standards. Developers should consider reinforcement fine-tuning as a primary strategy for optimizing smaller models like Amazon Nova 2 Lite to achieve high-tier performance at a lower cost. This capability is particularly recommended for specialized tasks like reasoning and coding where objective reward functions can be used to rapidly iterate and improve model accuracy.

Machine Learning AI Generative AI Amazon Bedrock+5

awsDec 3, 2025Original article

New serverless customization in Amazon SageMaker AI accelerates model fine-tuning (opens in new tab)

Amazon SageMaker AI has introduced a new serverless customization capability designed to accelerate the fine-tuning of popular models like Llama, DeepSeek, and Amazon Nova. By automating resource provisioning and providing an intuitive interface for advanced reinforcement learning techniques, this feature reduces the model customization lifecycle from months to days. This end-to-end workflow allows developers to focus on model performance rather than infrastructure management, from initial training through to final deployment. **Automated Infrastructure and Model Support** * The service provides a serverless environment where SageMaker AI automatically selects and provisions compute resources based on the specific model architecture and dataset size. * Supported models include a broad range of high-performance options such as Amazon Nova, DeepSeek, GPT-OSS, Meta Llama, and Qwen. * The feature is accessible directly through the Amazon SageMaker Studio interface, allowing users to manage their entire model catalog in one location. **Advanced Customization and Reinforcement Learning** * Users can choose from several fine-tuning techniques, including traditional Supervised Fine-Tuning (SFT) and more advanced methods. * The platform supports modern optimization techniques such as Direct Preference Optimization (DPO), Reinforcement Learning from Verifiable Rewards (RLVR), and Reinforcement Learning from AI Feedback (RLAIF). * To simplify the process, SageMaker AI provides recommended defaults for hyperparameters like batch size, learning rate, and epochs based on the selected tuning technique. **Experiment Tracking and Security** * The workflow introduces a serverless MLflow application, enabling seamless experiment tracking and performance monitoring without additional setup. * Advanced configuration options allow for fine-grained control over network encryption and storage volume encryption to ensure data security. * The "Continue customization" feature allows for iterative tuning, where users can adjust hyperparameters or apply different techniques to an existing customized model. **Evaluation and Deployment Flexibility** * Built-in evaluation tools allow developers to compare the performance of their customized models against the original base models to verify improvements. * Once a model is finalized, it can be deployed with a few clicks to either Amazon SageMaker or Amazon Bedrock. * A centralized "My Models" dashboard tracks all custom iterations, providing detailed logs and status updates for every training and evaluation job. This serverless approach is highly recommended for teams that need to adapt large language models to specific domains quickly without the operational overhead of managing GPU clusters. By utilizing the integrated evaluation and multi-platform deployment options, organizations can transition from experimentation to production-ready AI more efficiently.

Machine Learning AI Large Language Models Generative AI+5

awsDec 3, 2025Original article

Introducing checkpointless and elastic training on Amazon SageMaker HyperPod (opens in new tab)

Amazon SageMaker HyperPod has introduced checkpointless and elastic training features to accelerate AI model development by minimizing infrastructure-related downtime. These advancements replace traditional, slow checkpoint-restart cycles with peer-to-peer state recovery and enable training workloads to scale dynamically based on available compute capacity. By decoupling training progress from static hardware configurations, organizations can significantly reduce model time-to-market while maximizing cluster utilization. **Checkpointless Training and Rapid State Recovery** * Replaces the traditional five-stage recovery process—including job termination, network setup, and checkpoint retrieval—which can often take up to an hour on self-managed clusters. * Utilizes peer-to-peer state replication and in-process recovery to allow healthy nodes to restore the model state instantly without restarting the entire job. * Incorporates technical optimizations such as collective communications initialization and memory-mapped data loading to enable efficient data caching. * Reduces recovery downtime by over 80% based on internal studies of clusters with up to 2,000 GPUs, and was a core technology used in the development of Amazon Nova models. **Elastic Training and Automated Cluster Scaling** * Allows AI workloads to automatically expand to use idle cluster capacity as it becomes available and contract when resources are needed for higher-priority tasks. * Reduces the need for manual intervention, saving hours of engineering time previously spent reconfiguring training jobs to match fluctuating compute availability. * Optimizes total cost of ownership by ensuring that training momentum continues even as inference volumes peak and pull resources away from the training pool. * Orchestrates these transitions seamlessly through the HyperPod training operator, ensuring that model development is not disrupted by infrastructure changes. For teams managing large-scale AI workloads, adopting these features can reclaim significant development time and lower operational costs by preventing idle cluster periods. Organizations scaling to thousands of accelerators should prioritize checkpointless training to mitigate the impact of hardware faults and maintain continuous training momentum.

Machine Learning AI AWS Cloud Computing+5

googleDec 2, 2025Original article

From Waveforms to Wisdom: The New Benchmark for Auditory Intelligence (opens in new tab)

Google Research has introduced the Massive Sound Embedding Benchmark (MSEB) to unify the fragmented landscape of machine sound intelligence. By standardizing the evaluation of eight core auditory capabilities across diverse datasets, the framework reveals that current sound representations are far from universal and have significant performance "headroom" for improvement. Ultimately, MSEB provides an open-source platform to drive the development of general-purpose sound embeddings for next-generation multimodal AI. ### Diverse Datasets for Real-World Scenarios The benchmark utilizes a curated collection of high-quality, accessible datasets designed to reflect global diversity and complex acoustic environments. * **Simple Voice Questions (SVQ):** A foundational dataset featuring 177,352 short spoken queries across 17 languages and 26 locales, recorded in varying conditions like traffic and media noise. * **Speech-MASSIVE:** Used for multilingual spoken language understanding and intent classification. * **FSD50K:** A large-scale dataset for environmental sound event recognition containing 200 classes based on the AudioSet Ontology. * **BirdSet:** A massive-scale benchmark specifically for avian bioacoustics and complex soundscape recordings. ### Eight Core Auditory Capabilities MSEB is structured around "super-tasks" that represent the essential functions an intelligent auditory system must perform within a multimodal context. * **Retrieval and Reasoning:** These tasks simulate voice search and the ability of an assistant to find precise answers within documents based on spoken questions. * **Classification and Transcription:** Standard perception tasks that categorize sounds by environment or intent and convert audio signals into verbatim text. * **Segmentation and Clustering:** These involve identifying and localizing salient terms with precise timestamps and grouping sound samples by shared attributes without predefined labels. * **Reranking and Reconstruction:** Advanced tasks that reorder ambiguous text hypotheses to match spoken queries and test embedding quality by regenerating original audio waveforms. ### Unified Evaluation and Performance Goals The framework is designed to move beyond fragmented research by providing a consistent structure for evaluating different model architectures. * **Model Agnostic:** The open framework allows for the evaluation of uni-modal, cascade, and end-to-end multimodal embedding models. * **Objective Baselines:** By establishing clear performance goals, the benchmark highlights specific research opportunities where current state-of-the-art models fall short of their potential. * **Multimodal Integration:** Every task assumes sound is the critical input but incorporates other modalities, such as text context, to better simulate real-world AI interactions. By providing a comprehensive roadmap for auditory intelligence, MSEB encourages the community to move toward universal sound embeddings. Researchers can contribute to this evolving standard by accessing the open-source GitHub repository and utilizing the newly released datasets on Hugging Face to benchmark their own models.

Machine Learning AI Multimodal AI Benchmarking+4

tossDec 1, 2025Original article

Toss Next ML Challenge (opens in new tab)

Toss recently hosted the "Toss Next ML Challenge," a large-scale competition focused on predicting advertisement Click-Through Rates (CTR) using real-world, anonymized data from the Toss app. By tasking over 2,600 participants with developing high-performance models under real-time serving constraints, the event successfully identified innovative technical approaches to feature engineering and model ensembling. ### Designing a Real-World CTR Prediction Task * The competition required participants to predict the probability of a user clicking a display ad based on a dataset of 10.7 million training samples. * Data included anonymized features such as age, gender, ad inventory IDs, and historical user behavior. * A primary technical requirement was "real-time navigability," meaning models had to be optimized for fast inference to function within a live service environment. ### Overcoming Anonymization with Sequence Engineering * To maintain data privacy while allowing external access, Toss provided anonymized features in a single flattened table, which limited the ability of participants to perform traditional data joins. * A complex, raw "Sequence" feature was intentionally left unprocessed to serve as a differentiator for high-performing teams. * Top-tier participants demonstrated extreme persistence by deriving up to 37 unique variables from this single sequence, including transition probabilities, unique token counts, and sequence lengths. ### Winning Strategies and Technical Trends * All of the top 30 teams utilized Boosting Tree-based models (such as XGBoost or LightGBM), while Deep Learning was used only by a subset of participants. * One standout solution utilized a massive ensemble of 260 different models, providing a fresh perspective on the limits of ensemble learning for predictive accuracy. * Performance was largely driven by the ability to extract meaningful signals from anonymized data through rigorous cross-validation and creative feature interactions. The results of the Toss Next ML Challenge suggest that even in the absence of domain-specific context due to anonymization, meticulous feature engineering and robust tree-based architectures remains the gold standard for tabular data. For ML engineers, the competition underscores that the key to production-ready models lies in balancing complex ensembling with the strict latency requirements of real-time serving.

Machine Learning AI Feature Engineering Xgboost+4

dropboxNov 26, 20253 min readCurated summary

Building the future: highlights from Dropbox’s 2025 summer intern class

Dropbox’s 2025 intern program brought together 43 interns from 27 universities for 12 weeks of mentorship, technical work, and community-building. The 28 engineering interns contributed to systems spanning AI, search, storage, data infrastructure, and developer tools, with many projects supporting Dropbox Dash. Their work demonstrates how targeted refactoring, automation, and intelligent infrastructure can improve reliability, reduce costs, and expand product capabilities. ## A Diverse, Mentorship-Focused Internship Program - Interns received more than 6,000 hours of one-on-one mentorship. - The cohort included students from institutions in the United States, Canada, Poland, and Ireland. - Programming included Virtual First events, employee resource group activities, and an in-person Emerging Talent Summit. - Projects were aligned with Dropbox’s production systems and company goals rather than being isolated experiments. ## Infrastructure and Reliability Improvements - **Filesystem Data:** Rhea Rai redesigned Dropbox’s file history tracking system, emphasizing strongly tested code and simpler metadata infrastructure while reducing operational costs. - **Storage Core:** Albert Joon Sung reduced Magic Pocket write latency during disk restarts by adding storage-health caching and routing writes away from degraded volumes. - **Metrics:** Yonatan Ginsburg developed adaptive anomaly detection for Vortex2, accounting for changing patterns and seasonality to reduce alert fatigue and improve incident response. - **Analytics Platform:** Sanjith Udupa built recommendations for optimizing Databricks queries and ETL pipelines, and documented a plan to migrate a 500 TB mobile-events dataset to liquid clustering. ## AI, Search, and Dropbox Dash - **ML Platform:** Ben Juntilla created AI Sentinel, which gives engineers real-time visibility into machine-learning deployment health and improves confidence in model releases. - **Connector Platform:** Eddie Ormseth built tools that provide access to fresher Dash persistence data and additional third-party metadata without requiring connector teams to reprocess data. - **Retrieval Platform:** Rishi Peddakama expanded unified search to more than 20 languages by integrating language detection into indexing and retrieval. - **Find & Discover:** Francesca Venditti created in-context document previews for Dash, including PDF viewing and links to AI-powered follow-up chat. - **Conversational AI:** Alan Zhu developed a modular web-automation agent and connected it to Dropbox APIs for actions such as searching for and uploading files. ## Developer Automation - Ahmed Ibrahim built an AI-assisted code migration tool on Dropbox’s internal migration platform. - Developers can run migrations on selected folders, configure them through a CLI or automated workflow, and receive pull requests automatically when jobs succeed. - The tool enabled two major migrations during the internship and illustrates how automation can reduce repetitive engineering work. ## Broader Impact The interns’ projects improved system performance, operational visibility, multilingual accessibility, data freshness, and developer productivity. Together, they supported Dropbox’s move toward AI-first products while reinforcing the company’s emphasis on trustworthy, maintainable, and efficient engineering. For students interested in production-scale software, AI, and infrastructure, Dropbox presents its internship program and open roles as opportunities to contribute directly to products such as Dropbox Dash.

Read original

Machine Learning AI Distributed Systems Anomaly Detection+4

googleNov 20, 2025Original article

Reducing EV range anxiety: How a simple AI model predicts port availability (opens in new tab)

Google Research has developed a lightweight AI model designed to predict the probability of EV charging port availability at specific future intervals, directly addressing the "range anxiety" experienced by electric vehicle drivers. By co-designing the model with deployment infrastructure, researchers found that a simple linear regression approach outperformed more complex architectures like neural networks and decision trees. The resulting system effectively predicts availability changes during high-turnover periods, providing more reliable navigation and planning data than traditional "no-change" assumptions. ### Model Architecture and Feature Selection * The development team prioritized a minimal feature set to ensure low-latency deployment and high speed in real-world navigational applications. * After testing various architectures, a straightforward linear regression model was selected for its robustness and superior performance in this specific predictive task. * The model was trained using real-time availability data from diverse geographical regions, specifically California and Germany, with an emphasis on larger charging stations that reflect high-traffic usage patterns. ### Temporal Feature Weights and Occupancy Trends * The model uses the hour of the day as a primary feature, treating each hour as an independent variable to capture specific daily cycles. * Learned numerical "weights" dictate the predicted rate of occupancy change: positive weights indicate ports are becoming occupied (e.g., during morning rush), while negative weights indicate ports are being freed up (e.g., during evening hours). * The system is designed to only deviate from the current occupancy state when the change rate is statistically significant or when a station's large size amplifies the likelihood of a status change. ### Performance Benchmarking and Validation * The model was evaluated against a "Keep Current State" baseline, which assumes future availability will be identical to the present status—a difficult baseline to beat since port status remains unchanged roughly 90% of the time over 30-minute windows. * Accuracy was measured using Mean Squared Error (MSE) and Mean Absolute Error (MAE) over 30-minute and 60-minute time horizons across 100 randomly selected stations. * Testing confirmed that the linear regression model provides its greatest value during infrequent but critical moments of high turnover, successfully identifying when a station is likely to become full or available. The success of this model demonstrates that sophisticated deep learning is not always the optimal solution for infrastructure challenges. By combining intuitive real-world logic—such as driver schedules and station capacity—with simple machine learning techniques, developers can create highly efficient tools that significantly improve the EV user experience without requiring massive computational overhead.

Machine Learning AI Neural Networks Predictive Modeling+4