techlist.io
KREN

Machine Learning

149 posts

stripe3 min readCurated summary

Expanding Stripe Radar to protect more of your business

Stripe has significantly expanded Radar from card fraud prevention into a broader, AI-powered risk platform. It now protects transactions across global payment methods, supports off-Stripe fraud signals and custom models, detects newer abuses such as multi-account and pay-as-you-go fraud, and helps platforms assess merchant risk. Stripe’s goal is to let businesses intervene earlier and with greater precision while reducing false positives and operational losses. ## Global Payment Coverage and Custom Fraud Models - Radar now protects transactions across supported payment methods, including: - Bank debits - BNPL - Crypto - Digital wallets - Real-time payments - Cash vouchers - Fraud signals such as IP addresses and device fingerprints can now protect transactions across payment methods and businesses on the Stripe network. - Stripe reported a 71% reduction in suspected fraud over five months for businesses using Affirm, Cash App, Klarna, and PayPal. - New multiprocessor signals predict: - Whether a transaction may trigger an early fraud warning - Whether it is likely to result in a fraudulent dispute - Businesses can use these predictions to refund transactions early, gather evidence, or adjust dispute strategies. - Custom fraud models allow businesses to provide proprietary signals such as: - Product catalog information - Loyalty status - Behavioral data - Structured metadata - Early adopters detected at least 15% more fraud without increasing false positives. ## Defending Against New Fraud Types ### Multi-Account Abuse - Fraudsters create multiple accounts to reuse promotions or distribute stolen-card activity. - More than one in six AI-company sign-ups on Stripe are associated with multi-account abuse. - Radar evaluates accounts in real time using network-wide signals such as device fingerprints, IP addresses, and email domains. - ElevenLabs reportedly blocks around 2,000 abusive users per day from its free tier. ### Pay-As-You-Go Abuse - Customers can consume substantial resources and intentionally avoid paying when billed later. - Radar predicts nonpayment risk as usage accumulates. - Businesses can respond by requiring top-ups, suspending service, or applying other controls before billing. ### Malicious Bot Payments - Radar assigns a bot score to Stripe Checkout payments. - Businesses can distinguish legitimate automated agents from malicious bots. - The score can support controls against: - Inventory hoarding - Promotional abuse - Purchase-limit bypasses - High-velocity automated orders ## Platform and Merchant Risk Management - Platforms receive 0–100 fraud scores for businesses and transactions. - AI-powered explanations, notes, account history, and account-level metrics help risk teams investigate merchants. - New merchant-risk signals include: - **Fraudulent website signal:** Detects suspicious pricing, AI-generated copy, misspelled domains, and other website red flags. - **Fraudulent merchant signal:** Uses business information, bank details, transaction activity, and disputes to identify risky accounts. - **Merchant delinquency risk signal:** Predicts whether a merchant’s negative balance is likely to persist for at least 60 days. - Platforms can use these signals to automate verification, trigger reviews, pause payments or payouts, reject accounts, establish reserves, adjust payout schedules, or request additional identity verification. Stripe’s expanded Radar offering is designed to move fraud prevention earlier in the customer and merchant lifecycle. Businesses and platforms should combine these network-wide signals with their own risk tolerance and workflows to block abuse proactively while minimizing unnecessary friction.

Read original(opens in new tab)
Machine LearningFraud DetectionPayment ProcessingBot Detection+4
google2 min readCurated summary

Empirical Research Assistance (ERA): From Nature publication to catalyzing Computational Discovery

Empirical Research Assistance (ERA) is a Google AI system designed to help scientists develop expert-level computational models. Using Gemini, it searches literature, generates and evaluates code, and explores thousands of possible solutions through tree search. A Nature paper reports strong performance across scientific benchmarks, while new applications suggest ERA can accelerate research in health, climate, energy, and economics. ## How ERA Supports Scientific Coding - ERA starts with a scientific problem and a success metric. - It searches relevant research, combines methods, writes code, and iteratively tests and improves solutions. - Its tree-search process evaluates thousands of alternatives to optimize the resulting model. - Benchmarks in genomics, public health, satellite imagery, neuroscience, time-series forecasting, and mathematics showed expert-level performance. ## Applications to Open Scientific Problems - **Epidemiological forecasting** - Predicted U.S. hospital admissions up to four weeks ahead for flu, COVID-19, and RSV. - Forecasts ranked at or near the top of CDC leaderboards. - The techniques can potentially be adapted to other countries and diseases. - **California water-supply forecasting** - Produced seasonal runoff predictions for snow-fed river basins. - Delivered more accurate early forecasts than California’s official Bulletin 120 outlook. - Improved predictions could support water management and agriculture. - **Atmospheric carbon dioxide monitoring** - Combined geostationary weather-satellite data with other inputs to estimate CO₂ concentrations every 10 minutes across broad areas. - Captured urban emissions, plant-driven daytime absorption, and other atmospheric cycles. - Provides higher spatial and temporal coverage than measurements from satellites such as Orbiting Carbon Observatory-2. - **Solar-energy design** - Combined ERA with Google Antigravity to optimize three-dimensional solar-panel geometries. - Identified a 500-triangle volumetric fan design that could capture scattered radiation without backward shading. - **Retail forecasting** - Used economic indicators, Google Trends, historical patterns, and consumer sentiment. - Matched or exceeded commercial consensus forecasts and the Chicago Fed’s monthly retail forecast. ## Computational Discovery - Google is gradually opening access to Computational Discovery through a trusted tester program in Google Labs. - The system combines ERA with AlphaEvolve to support computational scientific investigation. - It complements other Gemini for Science experiments: - **Hypothesis Generation**, built with AI Co-Scientist, supports developing scientific hypotheses. - **Literature Insights** supports research and literature analysis. ERA’s demonstrated value lies in automating the labor-intensive cycle of designing, testing, and refining scientific software. Its expanding applications indicate that AI-assisted computational research could broaden access to advanced modeling while helping experts investigate complex scientific problems more quickly.

Read original(opens in new tab)
Machine LearningAIGeminiTime Series Forecasting+4
spotify3 min readCurated summary

When Can LLMs Replace Humans in A/B Tests? | Spotify Engineering

LLMs can approximate human outcomes in A/B tests only when strong assumptions hold; unlike randomized user experiments, those assumptions are not guaranteed by design. In the Upworthy headline dataset, raw GPT-4o-mini predictions recovered just 39% of the human treatment effect, but appropriate calibration and repeated sampling substantially improved accuracy. However, the assumptions are hardest to justify for genuinely new products or interventions—the cases where replacing human tests would be most valuable. ## Raw LLM Predictions Underestimate Treatment Effects - Researchers used the Upworthy Research Archive, containing click-through rates from thousands of headline experiments. - GPT-4o-mini predicted click-through rates for treatment and control headlines. - Treating those predictions as human outcomes recovered only 39% of the observed human treatment effect. - The error was systematic rather than random: LLM predictions attenuated effects toward zero. - This could lead companies to underestimate product improvements and make poor shipping decisions. ## Conditions for Valid LLM Surrogates Two assumptions are required: - **Surrogacy:** LLM predictions must capture everything about a treatment that affects human behavior. Once predictions and relevant pre-treatment characteristics are accounted for, treatment assignment should provide no additional information about the human outcome. - **Comparability:** The relationship between LLM predictions and human behavior—the calibration function—must remain stable between historical experiments and the new experiment. - If either condition fails, more LLM samples will not solve the problem; the procedure estimates an effect on the model rather than the effect on users. ## Calibration Methods Matter - Linear calibration with ordinary least squares failed a falsification test, producing estimates 3.8 standard errors away from the human benchmark. - Random forests and gradient-boosted trees performed better because they could model nonlinear relationships between LLM predictions and human click behavior. - Repeatedly sampling the LLM and averaging its outputs reduces temperature-driven noise, lowering bias toward zero and reducing variance. ## Limits for New Interventions - Historical data can partially test surrogacy and comparability, but neither assumption can be verified for a treatment never previously tested. - Trust declines as a new treatment becomes more different from past experiments. - The Upworthy setting is unusually favorable: binary outcomes, text-only treatments, similar headline variants, and an LLM trained on extensive textual data. - These assumptions are much less plausible for changes to layouts, recommendation algorithms, pricing, or entirely new product concepts. Human A/B tests remain essential for genuine product innovation. LLM-based testing is most defensible for incremental changes that resemble well-understood historical treatments, with human experiments used to validate the approach and calibrate its predictions.

Read original(opens in new tab)
Machine LearningLarge Language ModelsAb TestingCausal Inference+4
github1 min readCurated summary

From latency to instant: Modernizing GitHub Issues navigation performance

Alexander is a senior software engineer on GitHub’s Issues team, where he focuses on making developer workflows feel fast and seamless. His background spans computer graphics, machine learning, and geospatial software, giving him a broad technical perspective. ### Professional Role - Works on the GitHub Issues team. - Focuses on improving everyday developer workflows. - Enjoys finding creative ways to make interactions feel instant. ### Technical Background - Computer graphics - Machine learning - Geospatial software The provided content is a short professional biography rather than a technical blog post, so it does not include a specific argument, technical explanation, or conclusion.

Read original(opens in new tab)
Machine LearningSoftware EngineeringGithub IssuesComputer Graphics+1
meta1 min readCurated summary

Reel Friends: Building Social Discovery that Scales to Billions

Friend Bubbles may look like a simple Reels feature, but building it required substantial engineering work. The feature surfaces Reels that friends have watched or reacted to, relying on an evolving machine-learning model and platform-specific behavior. Meta engineers explain that a key, unexpected discovery ultimately helped make the experience work. ### What Friend Bubbles Does - Highlights Reels that a user’s friends have watched or reacted to. - Connects social activity with Reels recommendations in a more visible way. ### Engineering Challenges - The team had to evolve the machine-learning model powering the feature. - iOS and Android users exhibited different behaviors, requiring the team to account for platform-specific usage patterns. - The feature’s apparent simplicity concealed complex recommendation and product-engineering challenges. ### Podcast Discussion - Meta Tech Podcast host Pascal Hartig speaks with Facebook Reels engineers Subasree and Joseph. - They discuss the model’s development, differences between mobile platforms, and the surprising insight that helped the feature succeed. - The episode is available through Meta’s podcast channels and services including Spotify, Apple Podcasts, and Pocket Casts. The episode illustrates why seemingly straightforward social features can demand deep experimentation, modeling, and cross-platform engineering.

Read original(opens in new tab)
Machine LearningAndroidiOSSoftware Engineering+1
meta3 min readCurated summary

Migrating Data Ingestion Systems at Meta Scale

Meta rebuilt its hyperscale MySQL data ingestion system to improve reliability, efficiency, and data-langing latency. The migration moved workloads from customer-owned pipelines to a simpler, self-managed warehouse service and ultimately transitioned 100% of jobs. Success depended on staged validation, continuous data comparison, and fast rollback mechanisms. ## Why Meta Migrated - The system incrementally moved several petabytes of social graph data from MySQL into Meta’s data warehouse each day. - This data supports analytics, reporting, machine learning, and product development. - The legacy architecture became increasingly unstable as data-landing requirements grew stricter. - Customer-owned pipelines worked at smaller scales but became difficult to manage reliably at hyperscale. ## Migration Success Criteria Each job had to meet defined requirements before advancing: - **Data correctness:** Old and new systems had matching row counts and checksums. - **Landing latency:** The new system performed at least as well as the legacy system. - **Resource usage:** Compute and storage consumption did not regress. - **Critical-table requirements:** Additional criteria were agreed upon with dependent teams. ## Three-Phase Migration Lifecycle ### Shadow Phase - New-system shadow jobs ran against the same production sources as existing jobs. - Their output was written to separate shadow tables. - Row counts and checksums were continuously compared with production data. - Compute and storage requirements were measured before production rollout. - Once validated in pre-production, shadow jobs were tested in production. ### Reverse Shadow Phase - The new system began writing to the production table. - The legacy system continued running, but wrote to a shadow table. - This preserved continuous comparison between both systems. - If discrepancies appeared, Meta could quickly restore the old system without rebuilding its configuration. ### Migration Cleanup - Both systems continued to be monitored for mismatches. - After validation, the legacy shadow job was removed. - The new system became the sole production pipeline. ## Data Quality and Debugging Tooling - Meta built tooling to compare corresponding table partitions from the two systems. - Comparisons included row counts, checksums, and example rows responsible for mismatches. - Mismatch records and debugging details were logged to Scuba for real-time analysis. - Hourly queries helped engineers identify root causes and determine whether issues were already known. - The same tooling remains part of post-migration release validation. ## Rollout and Rollback Controls - Both systems used change data capture (CDC), with internal full-dump and delta tables feeding customer-facing target tables. - Because CDC builds new data from previously landed data, an existing defect could propagate after migration. - Meta therefore emphasized: - Detecting problems before they reached data consumers. - Stopping further propagation quickly during rollback. - The reverse-shadow design provided early quality signals and preserved a ready-to-use legacy pipeline for rapid recovery. Meta’s migration demonstrates that large-scale infrastructure changes are safest when treated as controlled, observable lifecycle transitions rather than one-time cutovers. Parallel execution, automated data validation, explicit resource checks, and reversible rollouts enabled the company to migrate the entire workload while protecting downstream consumers.

Read original(opens in new tab)
Machine LearningMysqlData IngestionChange Data Capture+3
netflix3 min readCurated summary

Democratizing Machine Learning at Netflix: Building the Model Lifecycle Graph

Netflix’s growing use of machine learning across personalization, Studio, payments, advertising, and other domains has created a fragmented ecosystem of tools and metadata. The Metadata Service (MDS) addresses this problem by building a Model Lifecycle Graph that connects models, features, pipelines, experiments, datasets, and ownership information. Its goal is to make ML assets discoverable, understandable, and reusable across organizational boundaries. ## A Fragmented Machine Learning Landscape - Netflix ML has expanded from personalization into areas such as: - Studio production and post-production - Fraud detection and payment optimization - Advertising and real-time targeting - Each domain uses different technologies, metrics, and organizational structures. - Valuable assets often remain isolated in specialized systems. - For example, Studio-generated content embeddings could support: - Contextual ad matching - Episodic merchandising - Recommendations based on tone, topic, or mood - Practitioners struggle to answer basic questions because relevant information is split across: - Model registries - Pipeline orchestrators - Experimentation platforms - Feature stores - Dataset systems - This fragmentation makes discovery, lineage tracking, impact analysis, and ownership difficult. ## The Challenge of Connecting ML Infrastructure - MDS must unify metadata from many independent systems, including: - Pipeline execution and transformation data - Model versions, artifacts, deployments, and staleness - A/B test configurations - Feature definitions and usage - Dataset creation and discovery - User, team, and organization information - These systems use different identifiers, formats, and conceptual models. - The core challenge is transforming heterogeneous metadata into a common entity model and connected graph—not merely creating a consolidated user interface. ## The Model Lifecycle Graph - Netflix’s Metadata Service indexes ML-related assets and materializes relationships between them. - It supports real-time metadata ingestion and cross-domain questions such as: - Which experiments use a particular model? - Which models depend on a feature? - What data sources feed a model? - Who owns each part of the workflow? - The graph is intended to make every ML asset discoverable and reusable regardless of its originating team or business domain. ## Core Concepts and Vocabulary - **Component:** Any uniquely addressable object identified by an AIP URI, such as: - `aip://model/registry/ranking-v5` - `aip://user/identity/alice` - `aip://pipeline/orchestrator/weekly-training` - **Entity:** A component enriched with properties such as name, description, creation date, and ownership. - **Entity type:** A group of entities sharing the same data shape and required properties. - **Domain:** An abstract interface for a category of ML assets, such as Models or Pipelines. - **Provider:** A concrete backend implementation of a domain, such as Netflix’s internal model registry. - Separating domains from providers allows multiple systems to implement the same interface without changing how consumers interact with MDS. - URI-based addressing gives services a consistent way to reference assets and resolve them to connected metadata. ## From Events to a Queryable Graph - MDS receives metadata events through Kafka and AWS SNS/SQS. - Source systems emit lightweight events containing an event type and resource identifier. - For example, a model registry might emit a `model_instance_created` event with the new instance’s ID. - This keeps event producers simple while allowing MDS to enrich events, construct entities, and infer relationships such as connections between models and A/B tests. The Model Lifecycle Graph provides Netflix with a common layer for connecting previously isolated ML systems. By standardizing identifiers, entities, domains, and providers, MDS can support cross-domain discovery, lineage, impact analysis, and collaboration at scale.

Read original(opens in new tab)
Machine LearningApache KafkaFeature StoreGraph Database+3
netflix3 min readCurated summary

State of Routing in Model Serving

Netflix’s centralized ML serving platform provides a single, domain-independent API for model inference across personalized experiences and other use cases. Rather than exposing individual scoring functions, Netflix packages feature computation, preprocessing, inference, and postprocessing into self-contained model workflows. The core routing challenge is directing each request to the correct model version and serving cluster while keeping client services independent from model changes and infrastructure topology. ## Models as End-to-End Workflows - Netflix distinguishes **model serving** from traditional model inference: - Inference typically means `infer(features) -> score`. - Serving includes preprocessing, feature computation, optional trained components, and postprocessing. - Example workflows include: - Ranking titles for a personalized Continue Watching row using user, country, and device context. - Predicting payment fraud using user, country, and transaction details. - Models declare the facts they need, while the serving platform retrieves those facts from other microservices. - During offline training, Netflix’s ML fact store provides snapshots for bulk feature computation. - Calling services provide standard request context and domain-specific inputs, while the platform handles feature generation, model selection, and execution. ## Platform Design Principles - **Model innovation without client changes** - Client applications integrate with the platform once. - Model versions, A/B tests, additional experimental data, logging, and model selection remain hidden behind the platform API. - **Clients decoupled from model sharding** - Models run across multiple serving cluster shards, each with its own Virtual IP address. - Shard assignments can change based on traffic, SLAs, model architecture, and resource availability. - Clients should not need to track these VIP changes. - **Flexible traffic routing** - Routing must support A/B allocations, gradual traffic shifts, new model versions, new VIPs, and client-specific overrides. - Safe lifecycle management requires support for shadow deployments, canaries, rollbacks, and migrations. ## Switchboard: Context-Aware Routing - Generic API gateways and service-mesh proxies did not satisfy Netflix’s requirements. - Netflix needed: - Native integration with its experimentation platform. - gRPC support. - Routing based on rich, domain-specific request context. - Model-specific rollout and migration controls. - Netflix built **Switchboard**, a custom proxy layer handling more than one million requests per second. - Switchboard is the mandatory entry point for clients and: - Routes requests to the appropriate model based on request context. - Applies configured context enrichment before invoking the model. - Hides model locations and infrastructure changes from client services. ## Objective Abstraction - Every request must provide an **Objective**, an enumeration defined by the serving platform. - The excerpt introduces Objectives as a central abstraction for identifying the business purpose of a serving request, but the supplied text ends before describing its full roles. Netflix’s approach is to centralize routing, experimentation, and model execution behind one stable API. This allows client applications to evolve independently while researchers can iterate on models and safely manage large-scale production rollouts.

Read original(opens in new tab)
Machine LearningDatabase DesignPostgreSQLMicroservices+4
grammarly2 min readCurated summary

Educator of the Year

Grammarly’s inaugural Educator of the Year Award honors teachers nominated directly by their students. The first winner, Dr. Humberto López Castillo of the University of Central Florida, is recognized for teaching precise, accessible communication and applying it to public health, technology, and community engagement. His approach combines audience-aware writing, responsible AI use, and hands-on research. ## Student-Led Recognition - Students nominate educators through short videos describing their impact on academic and professional development. - UCF student Vardhan Avaradi nominated Dr. López Castillo for encouraging students to make their language “precise yet accessible.” - López Castillo is a pediatrician, public health researcher, translator, and four-language polyglot from Panama. - His teaching emphasizes collaboration and the connection between individual health and broader communities. ## Communicating With Different Audiences - Students translate complex public health topics for audiences outside academia. - Assignments have included: - Storybooks about mosquitoes for kindergarteners - Monopoly-style games about living with HIV - Rap songs explaining tuberculosis - Podcasts that personalize epidemiology - His medical experience informs this approach: communication must change depending on whether the audience is a child, parent, or professional researcher. ## AI Requires Human Judgment - López Castillo permits students to use AI for drafting but expects them to verify and critically evaluate its output. - When AI-generated citations referenced nonexistent research, he treated the error as a lesson rather than a punishment. - He compares AI to a calculator: useful and powerful, but dependent on the judgment of the person using it. - He and Vardhan are developing a machine learning project using the NIH All of Us dataset, which contains nearly one million de-identified health records. - Their research explores using AI to classify populations and predict health risks. ## Preparing Students for Broader Impact - Students leave with stronger writing, critical-thinking, collaboration, and communication skills. - López Castillo’s teaching focuses not just on adopting new tools, but on using them responsibly and communicating with purpose. - His students learn to reach people beyond academic audiences while keeping human needs at the center of technology and research. The post’s central recommendation is to pair emerging technologies with critical thinking, audience awareness, and a strong sense of social responsibility.

Read original(opens in new tab)
Machine LearningAIDatabase DesignPublic Health+3
gitlab2 min readCurated summary

GitLab and Anthropic: Governed AI for enterprise development

GitLab is expanding its integration with Anthropic Claude to provide enterprise teams with more capable AI inside a governed software development platform. Claude supports GitLab Duo Agent Platform features such as code generation, review, agentic chat, and vulnerability resolution. The central argument is that organizations should not have to trade advanced AI capabilities for security, compliance, and auditability. ## Governed AI across the SDLC - Claude-generated changes follow GitLab’s existing merge request process, approval rules, security scans, and audit trails. - AI agents do not bypass controls; their actions remain attributable, reviewable, and subject to policy enforcement. - This governance becomes increasingly important as agents autonomously plan, code, test, secure, and deploy software. - GitLab positions built-in governance as a core architectural differentiator rather than an added feature. ## Flexible enterprise deployment - Claude is available in GitLab through: - Google Cloud Vertex AI - Amazon Bedrock - Organizations can use existing cloud contracts, governance frameworks, and data-residency arrangements. - GitLab’s availability in the Claude Marketplace lets customers purchase GitLab Credits and apply them toward Anthropic spending commitments. - These options simplify procurement and consolidate AI spending. ## Supporting an agentic development model - GitLab is selecting model partners based on reasoning ability, reliability, and safety. - The platform is designed to maintain visibility into what AI agents do, when they act, and how their changes are tracked. - As agents take on more complex engineering tasks, GitLab argues that strong models must be paired with equally strong governance. ## Implications for customers - Existing GitLab Duo users gain deeper Claude-powered assistance without changing their established governance processes. - Organizations evaluating AI development platforms can access advanced models while retaining enterprise control. - GitLab presents the integration as a way to accelerate development without compromising compliance or oversight. The practical recommendation is to evaluate AI platforms not only by model capability, but also by how well they integrate governance, auditability, cloud deployment options, and existing enterprise workflows.

Read original(opens in new tab)
Machine LearningGitlabDevSecOpsSoftware Development+4
gitlab3 min readCurated summary

Give your AI agent direct GitLab access with glab CLI

The post argues that connecting AI agents to GitLab through the `glab` CLI gives them reliable, current, structured project data instead of forcing them to rely on copied UI content or stale assumptions. Through Model Context Protocol (MCP), agents can inspect issues, merge requests, discussions, and pipelines, then take actions such as commenting or resolving review threads. This reduces friction and enables faster code review and issue triage. ## Why AI Agents Need Direct GitLab Access - Without direct access, agents may: - Hallucinate issue or merge request details. - Rely on outdated training data. - Require developers to manually copy information from GitLab. - `glab` lets agents fetch live project data, act on it, and report results. - The approach supports tools such as GitLab Duo, Claude, Cursor, and other AI assistants. ## Connecting an Agent Through MCP - Model Context Protocol allows AI tools to discover and use external capabilities at runtime. - Start the `glab` MCP server with: ```bash glab mcp serve ``` - Once configured, an agent can answer questions such as: - “What’s the status of my open merge requests?” - “Are there failing pipelines on `main`?” - When used through MCP, `glab` automatically adds `--output json` where supported, giving agents clean, structured responses. - Interactive commands are excluded from MCP so agents do not hang waiting for terminal input. - The implementation uses the official MCP SDK for compatibility with protocol changes. ## Using AI for Merge Request Review - Agents can inspect unresolved review feedback with: ```bash glab mr view 2677 --comments --unresolved --output json ``` - The response includes: - Merge request metadata and description. - Labels and author information. - Unresolved discussions and reviewer comments. - Whether blocking discussions remain unresolved. - The agent can turn this data into a prioritized list of required fixes and suggested changes. ## Resolving Review Discussions Programmatically - Agents can list discussions in structured form: ```bash glab mr note list 456 --output json ``` - After verifying that feedback has been addressed, they can resolve a discussion: ```bash glab mr note resolve 456 3107030349 ``` - Discussions can be reopened when further review is needed: ```bash glab mr note reopen 456 3107030349 ``` - Discussion IDs are available in the GitLab UI and API, so no additional lookup is required. ## Feeding AI Better GitLab Context Without MCP - Even without an MCP server, developers can use `glab` to provide agents with richer, more accurate data. - Instead of pasting a short summary such as issue counts, milestones, and labels, command output can provide structured issue, merge request, or pipeline details. - This gives the agent more context for triage and debugging while avoiding manual browser-based copying. ## Practical Recommendation Use `glab` as the structured interface between GitLab and AI agents. MCP is the most capable option for agents that need to query and modify GitLab directly, while ordinary JSON-producing `glab` commands are a useful fallback for supplying accurate context manually.

Read original(opens in new tab)
Machine LearningModel Context ProtocolCode ReviewJson+3
toss5 min readCurated summary

Introducing Toss Place's Data Bot 'PANDA': How every team member works like a data expert

PANDA, short for Place Analytics & Data, is Toss Place’s AI data-analysis assistant, designed to let employees retrieve and interpret approved data without waiting for analysts. It was created after the team found that 70% of data requests involved simple metric lookups rather than complex analysis. The project’s main conclusion is that reliable AI analytics depends less on prompting alone and more on standardized data, business definitions, controlled table selection, and iterative validation. ## Why Toss Place Built PANDA - Employees previously relied on analysts to search dashboards, write SQL, or manually investigate data requests. - PANDA provides self-service access within each employee’s security permissions. - It reduces routine extraction work for analysts, allowing them to focus on deeper analysis. - The goal is to establish a stronger culture of “data democracy,” where employees can access and use data immediately. ## Challenges with a Simple AI Chatbot Early experiments showed that asking an AI model to search all company data produced unreliable and expensive results: - Referencing thousands of tables and internal documents consumed excessive tokens. - The model sometimes selected different tables for identical questions, producing inconsistent answers. - It often misunderstood business definitions. For example, “active stores” could mean stores with completed installations or stores that had processed payments. - Inefficient SQL caused unnecessary Snowflake data scans and higher warehouse costs. ## Standardized Data Marts as a Single Source of Truth Toss Place collaborated across its Data Analysis and Data Platform teams to establish reliable standard data marts. - Core concepts, such as store information, were consolidated into standardized tables. - Naming conventions made table and column purposes easier for both people and AI to understand: - Tables follow `{mart_type}_{domain}_{subject}`, such as `fact_device_error_log`. - Columns follow `{prefix}_{entity}_{attribute}_{suffix}`, such as `is_merchant_active`. - Table and column descriptions were documented comprehensively. - The standardization effort reduced ambiguity by ensuring the same business concepts were represented consistently. ## Connecting Business Language to Data Data structures alone could not answer questions about terms such as “installed store” or “store category.” - Domain-specific terms and metric definitions were documented. - These business definitions were linked to the relevant standard data marts. - Data analysts helped reconcile differing interpretations and establish shared organizational definitions. - This gave PANDA the context needed to apply the correct business logic. ## Scoring and Ranking for Reliable Table Selection PANDA limits its search to well-managed tables and uses dbt tags to import selected metadata into a Manifest file. - Tables are ranked using: - **Similarity score:** Based on relationships between the question and table, including table-name matches and description relevance. - **Hierarchy weight:** Reflecting the reliability of the data layer. - The final score is calculated as: `similarity score × hierarchy weight` - Weights are assigned as follows: - Company-wide SSOT metrics: ×4 - Validated standard marts: ×3 - Domain analysis marts: ×2 - Raw bronze data and logs: ×1 - This improves accuracy, consistency, and trustworthiness while reducing unnecessary warehouse exploration. ## Agentic Loop for Querying and Validation Rather than expecting a correct answer in one attempt, PANDA uses an agentic loop. - It selects appropriate tools based on the question. - It explores tables, generates and executes queries, and reviews the results. - If the result appears inaccurate, it can inspect the schema again, modify the query, and retry. - If necessary, it asks the user for clarification. - This approach allows PANDA to handle exceptions dynamically instead of relying only on predefined rules. ## Answers Designed for Practical Use PANDA structures responses so users can understand and apply the results: - **Result:** The requested data or metric. - **Query criteria:** The period, filters, and aggregation method used. - **Insight:** An interpretation that can support practical decisions. This makes PANDA more than a number-retrieval chatbot; it also exposes part of the reasoning process normally provided by a data analyst. ## Adoption and User Response PANDA quickly became part of everyday work at Toss Place. - One-third of employees used it on its first day. - Half of the organization had tried it within a week. - More than 4,000 messages were exchanged during that period. - Current adoption is approximately 70%. - Employees reported feeling more comfortable asking small questions and using data while away from their desks. - Users particularly valued receiving insights alongside raw figures. - Unexpectedly, developers and even data professionals used PANDA actively, suggesting that its answers achieved a meaningful level of trust. ## Future Development PANDA was developed and launched in just one month, but the team plans further improvements. - Increase data coverage to more than 90%. - Raise answer accuracy above 97%. - Use real user questions, follow-up behavior, and abandonment patterns to identify unmet needs. - Expand beyond basic data retrieval to reduce more of the data team’s workload. PANDA’s central lesson is that effective enterprise AI does not require the most complicated technology. It requires solving a real business pain point with trustworthy data foundations, clear definitions, and a workflow that users can rely on.

Read original(opens in new tab)
Machine LearningAIDatabase DesignSql+4
gitlab3 min readCurated summary

GitLab AI Hackathon 2026: Meet the winners

Nearly 7,000 developers participated in GitLab’s 2026 AI Hackathon, creating more than 600 agents and workflows for the GitLab Duo Agent Platform. The winning projects focused on practical software delivery challenges—including organizational knowledge loss, security, compliance, migrations, and sustainability—rather than simple chatbot interactions. The results suggest that agentic AI is becoming most valuable when integrated directly into development workflows and given richer project context. ## Hackathon Scope and Evaluation - The hackathon ran from February 9 to March 25, 2026, on Devpost. - Google Cloud and Anthropic co-sponsored the event, contributing judges, prizes, and cloud resources. - Nineteen judges evaluated projects on: - Technical execution - Design - Potential impact - Quality of the idea - Total prizes reached $65,000. ## Grand Prize: LORE - LORE, or Living Organizational Record Engine, addresses the loss of institutional knowledge when engineers leave. - It combines: - Eight specialized agents - A router that directs questions to the appropriate agent - Protections against circular loops in its knowledge graph - A visual dashboard - Carbon tracking - Its command-line tool includes 43 tests, leading judges to describe it as a polished product rather than a typical hackathon prototype. ## Google Cloud and Anthropic Winners - **Gitdefender**, the Google Cloud Grand Prize winner, detects security issues during code review, writes fixes, and opens the review automatically. - **Aegis**, the Google Cloud Runner Up, explains the reasoning behind its AI decisions and is deployed on Google Cloud. - **GraphDev**, the Anthropic Grand Prize winner, maps code relationships and shows how systems evolve, helping developers understand the impact of changes. - **DocSync**, the Anthropic Runner Up, uses Detector, Writer, and Reviewer agents to update documentation. It opens a review when confident and creates an issue for human review when uncertain. ## Category Winners - **Time-Traveler**, winner for technical achievement, creates a safe copy of a production environment and runs database migrations against it using five connected agents, PostgreSQL, real data, and Google Cloud deployment. - **RedAgent**, the most impactful project, verifies AI-generated security findings before developers act on them, addressing distrust in automated reports. - **Launch Control**, recognized for ease of use, combined polished user experience, strong infrastructure, and sustainability considerations. ## Sustainability-Focused Projects - Five projects received sustainability prizes or bonuses as the organizers highlighted the growing energy cost of CI/CD systems and large language models. - **GreenPipe** analyzes CI/CD pipelines and generates carbon-footprint reports. - Sustainable Design bonuses recognized projects including: - **BugFlow**, which generated 10 fixes from one bug report in 20 minutes - **DELTA Cyber Reasoning**, an automated fuzz-testing tool - **CarbonLint**, which applies code analysis to energy consumption - **TFGuardian**, which includes carbon-footprint analysis - One project reduced monthly costs from $556 to $18, representing a reported 96% carbon reduction. ## Honorable Mentions - **SecurityMonkey** tests security scanners by injecting known vulnerabilities. - **stregent** enables CI/CD investigation and fixes through WhatsApp. - **Compliance Sentinel** evaluates merge requests for compliance risk and blocks critical violations. - **Carbon Tracker** measures the carbon footprint of individual pipeline jobs and suggests improvements. - **RepoWarden** captures the rationale behind code, not only its behavior. - **MR Compliance Auditor** maps merge-request evidence to SOC 2 controls and displays compliance scores in real time. ## What Comes Next The projects operated within a single GitLab project, but many teams supplemented their agents with local knowledge graphs to understand code relationships and dependencies. GitLab plans to build on this approach in future hackathons by providing agents with richer context. GitLab’s hackathon demonstrates that the strongest AI agents are workflow-integrated tools that can investigate, make decisions, execute changes, and involve humans when needed. Developers can explore the 600-plus projects in the gallery or build their own agents on the GitLab Duo Agent Platform.

Read original(opens in new tab)
Machine LearningDatabase DesignCI/CDGitlab+4
spotify3 min readCurated summary

Better Experiments with LLM Evals — A funnel, not a fork | Spotify Engineering

LLM evaluations and A/B experiments serve different purposes and should work as a funnel rather than alternatives. Evals verify whether a change meets qualitative standards and filter out weak ideas, while experiments validate whether it improves real user and business outcomes. Their greatest value comes from a feedback loop that continuously calibrates eval scores against online results. ## Evals as Verification, Experiments as Validation - LLM judges can assess relevance, coherence, tone, intent alignment, and other qualities at scale. - Evals identify promising candidates before they consume experiment capacity. - Experiments determine whether users actually respond better to the change. - An eval can uncover unexpected failure patterns, guide product fixes, and later verify that violations have decreased. - It cannot establish whether those fixes improve long-term outcomes such as engagement, trust, or retention. ## The Importance of Guardrails - Evals typically measure selected quality dimensions, not every possible business risk. - Spotify reports that roughly 42% of launched experiments are rolled back because of regressions in secondary metrics such as crashes, session length, or retention. - Online experiments expose production effects and monitor guardrail metrics that offline evals may miss. - Although only about 12% of Spotify’s A/B tests produce a shipped positive result, approximately 64% generate useful learning. ## Calibrating Evals Against Outcomes - Evals are proxies, so their scores must be validated against the outcomes they are intended to predict. - LLM judges introduce an additional calibration layer on top of conventional metrics such as precision, recall, and ranking scores. - Judges can reward superficial patterns or miss improvements that matter in real-world use. For example, Qodo’s coding evals did not detect improvements in Anthropic’s Opus 4.5 on longer tasks. - Both judges and quantitative metrics can drift, making offline-to-online comparison essential. - Continuous calibration clarifies when evals are reliable verification tools and whether they may eventually become strong enough to support validation. ## Closing the Evaluation Loop - Run evals early and frequently to select the strongest treatments. - Use experiments to confirm user impact, assess business outcomes, and detect unintended regressions. - Apply the same LLM evals to A/B test data to compare qualitative scores with actual user results. - A mismatch between eval scores and experiment outcomes is valuable diagnostic information: - Improved scores and improved outcomes validate the judge. - Improved scores without better outcomes show that the judge is measuring something that may not drive value. - Use lightweight experiments for iteration and learning, and more rigorous tests for launch decisions. The practical recommendation is to treat LLM evals as an upstream filter and a downstream calibration tool—not a replacement for experimentation. This funnel improves test efficiency while making both the judges and product decisions more trustworthy over time.

Read original(opens in new tab)
Machine LearningA B TestingLlm EvalsOnline Experiments+3
google2 min readCurated summary

It's all about the angle: Your photos, re-composed

Google’s new Auto frame feature in Google Photos re-composes photos by changing the apparent camera viewpoint after capture. It combines 3D scene reconstruction with generative AI to adjust framing, reduce wide-angle distortion, and generate background areas that were outside the original frame. The result is a more natural perspective while preserving the visible content and subjects’ identities. ## Why Traditional Editing Falls Short - Cropping and zooming cannot change the original camera perspective or parallax. - They also cannot reveal areas hidden outside the captured frame. - This is especially limiting for portraits, selfies, and “almost perfect” photos with poor framing or lens distortion. ## Two-Stage 3D-Aware Editing - **3D scene and camera estimation** - A machine-learning model creates a 3D point map from the 2D image using monocular depth estimation. - Each pixel is associated with an estimated 3D surface point. - The system estimates the original focal length and reconstructs human faces and bodies carefully to reduce identity-distorting artifacts. - Semantic analysis detects subjects’ face positions and 3D orientations to determine better camera parameters. - **Generative completion and retouching** - Classical 3D rendering produces an image from the modified camera position, orientation, and focal length. - Moving the virtual camera exposes “holes” where the original image contained no information. - A latent diffusion model fills these missing regions and corrects the rendered result. - Classifier guidance with regional scaling helps preserve original content while allowing generation in newly revealed areas. ## Automatic Framing and Distortion Correction - The system automatically selects camera parameters for improved portrait composition. - It can adjust both camera pose and intrinsic properties such as focal length. - For wide-angle selfies, it detects perspective distortion that enlarges features close to the lens. - By effectively moving the virtual camera backward and adjusting the optics, it produces more natural facial proportions. ## Availability in Google Photos - The technology is now integrated into the Auto frame feature. - It applies to eligible photos containing people. - Users receive the re-composed image as a second Auto frame candidate. - The enhancement requires only a single action and automatically changes the viewpoint while retaining the original scene. The feature demonstrates how combining geometric 3D reconstruction with controlled generative inpainting can perform edits that conventional cropping and retouching cannot. For portraits and selfies, Auto frame offers a practical way to improve composition and perspective after the moment has passed.

Read original(opens in new tab)
Machine LearningGenerative AI3d RenderingImage Editing+4