pinterestFeb 13, 2026

GPU-Serving Two-Tower Models for Lightweight Ads Engagement Prediction (opens in new tab)

Pinterest replaced its CPU-served two-tower model for ads lightweight ranking with a GPU-serving architecture based on MMOE and DCN. The more expressive model maintained latency comparable to the CPU baseline while reducing offline CTR loss by 5–10%. Separating standard and shopping ad models produced another 5–10% loss reduction and doubled offline iteration speed, with online improvements in CPC and CTR. ## Role of Lightweight Ranking - Lightweight ranking serves as an intermediate stage in Pinterest’s ads recommendation pipeline. - It filters a large pool of candidate ads before more complex downstream ranking models process them. - The two-tower design balances quality and latency: - The Pin tower generates ad embeddings offline through batch updates. - The query tower generates real-time user embeddings. - The prediction score is the sigmoid of the embeddings’ dot product. ## MMOE-DCN Model Architecture - The new system replaces the previous Multi-Task Multi-Domain (MTMD) model. - It combines: - Multi-gate Mixture-of-Experts (MMOE) with MLP-based gating. - Deep & Cross Network (DCN) layers for modeling feature interactions. - Each expert uses both full-rank and low-rank DCN layers. - Unlike MTMD, MMOE handles multi-task and multi-domain learning without relying on separate domain-specific modules. - GPU serving makes it practical to deploy this larger and more computationally demanding model while preserving CPU-baseline latency. ## Scenario-Specific Modeling - Standard and shopping ad scenarios are served as separate models. - Each model is trained only on data relevant to its scenario. - This specialization delivered an additional 5–10% reduction in offline loss. - Separating the models also doubled the speed of offline model iteration. ## Training Efficiency Improvements - **Dataloader optimization** - GPU prefetching prepares the next batch while the current batch is processed. - Additional worker threads take advantage of the 1 TB of CPU memory available on p4d instances. - **Model code optimization** - Operations that previously allocated zero-filled tensors on the CPU were moved to the GPU. - Fused kernels replaced multiple individual kernels to reduce execution overhead. - **Training configuration** - BF16 precision improved processing speed. - Larger batch sizes increased GPU memory utilization. ## Evaluation Results - The model uses downstream ranking scores as labels and optimizes KL divergence between those labels and its predictions. - Evaluation covers both: - Auction winners—ads ultimately inserted and shown to users. - Auction candidates—ads passed to downstream ranking. - Offline loss decreased significantly across all evaluated slices. - Online experiments showed: - Lower cost per click (CPC), which is favorable. - Higher click-through rate (CTR). GPU-serving a more complex MMOE-DCN two-tower model allowed Pinterest to improve ad engagement prediction without sacrificing serving latency. The results support using GPU infrastructure, scenario-specific models, and targeted training optimizations to scale lightweight ranking systems.

recommendation-systemsgpu-servingctr-predictiontwo-tower-models+4

pinterestFeb 2, 2026

Beyond Two Towers: Re-architecting the Serving Stack for Next-Gen Ads Lightweight Ranking Models… (opens in new tab)

Two-Tower models make retrieval and lightweight ranking highly efficient by scoring user and item embeddings with a dot product, but they cannot represent rich user-item interactions or deep feature crossings. This post describes an ads-serving redesign that introduces general-purpose GPU models while preserving end-to-end latency. The main strategy is to reduce data movement, move filtering logic onto the GPU, and optimize inference from an initial 4-second p90 latency to about 20 milliseconds. ## Why Move Beyond Two-Tower Models - Two-Tower architectures independently encode users and items, enabling fast scoring across millions of candidates. - Their decoupled structure limits: - User-item interaction features - Target attention - Early feature crossing - Deep architectures requiring simultaneous access to user and candidate data - More expressive models require GPU-based general-purpose inference rather than specialized dot-product or ANN retrieval. - The existing retrieval stack was not designed to transfer large candidate and feature sets to a GPU, creating a major latency challenge. ## Restructuring the Serving Funnel The traditional funnel consisted of: - Feature expansion for thousands of candidates - Retrieval and Two-Tower lightweight ranking - Heavy ranking and auction processing for the top documents Adding GPU inference directly to this flow would require fetching, serializing, transferring, and returning features for tens of thousands of documents. The authors therefore redesigned the entire early-stage serving pipeline instead of optimizing the model alone. ## Segmenting the Inventory for Feature Fetching Feature retrieval was a major latency source, often taking longer than model inference for workloads ranging from 10,000 to 100,000 documents. - **High-value inventory:** Roughly 1 million documents responsible for a substantial share of revenue have their features embedded in the PyTorch model as registered buffers. - Features become part of the model state, similar to weights. - They remain in GPU high-bandwidth memory. - Requests avoid remote feature-service calls and host-to-device transfers. - The model file must be periodically updated to refresh features. - Future work may include GPU-based caching. - **Long-tail inventory:** The remaining roughly 1 billion documents use a high-performance key-value store with in-host caching. - The post focuses on the first strategy, which is already running in production. ## Moving Business Logic onto the GPU Previously, the model returned scores for approximately 100,000 candidates, while CPU-side code handled utility calculation, filtering, diversity, deduplication, and top-k selection. - The new PyTorch model performs these operations directly: - Combines pCTR, pCVR, bid, and other signals into utility scores. - Applies diversity and filtering rules. - Performs top-k selection. - The GPU returns only the final winners—typically around 1,000 documents—instead of all candidate scores. - This reduces device-to-host data transfer and takes advantage of GPU parallelism. - The approach works because lightweight-ranking business rules are sufficiently simple to express with tensor operations. ## Reducing GPU Inference Latency Initial GPU inference measured roughly 4,000 ms at p90, far too slow for real-time serving. Several systems optimizations reduced this to approximately 20 ms: - **Multiple CUDA streams:** Separate streams for workers allow host-to-device transfers, computation, and device-to-host transfers to overlap. - **Worker alignment:** Worker threads are matched and pinned to physical CPU cores to reduce context switching and lock contention. - **Kernel fusion:** Triton kernels combine operations such as linear layers and activations, reducing memory traffic. - **BF16 computation:** Brain Floating Point 16 lowers memory usage and accelerates arithmetic compared with FP32. - **Profiling tools:** PyTorch Profiler and NVIDIA Nsight Systems were used to identify bottlenecks. ## Practical Recommendation Deploying more expressive ranking models requires rethinking the serving architecture around data movement and execution placement. Embedding frequently used features, executing business logic on the GPU, and applying low-level CUDA and kernel optimizations can make complex neural ranking feasible without increasing end-to-end latency.

recommendation-systemsmachine-learningpytorchgpu-inference+4

pinterestDec 5, 2025

Improving Quality of Recommended Content through Pinner Surveys (opens in new tab)

Pinterest uses Pinner surveys to measure visual quality and incorporate user preferences into recommendation systems, rather than optimizing solely for engagement. The company surveyed 5,000 Pins, trained a lightweight neural network to predict average perceived quality, and applied the resulting model across Homefeed, Related Pins, and Search. This approach aims to reduce clickbait and promote content that supports positive, long-term user experiences. ## Why Engagement Alone Is Insufficient - High engagement does not necessarily indicate high-quality content; optimizing for clicks can promote clickbait or harmful material. - Pinterest defines quality as content that feels good, inspires further exploration, and encourages fulfilling long-term engagement. - Direct user feedback helps recommendation systems prioritize content that Pinners actually value. - The work supports Pinterest’s Inspired Internet Pledge principles, especially listening to users and tuning the platform for wellbeing. ## Collecting Pinner Quality Ratings - Pinners rated images from 1 to 5 in response to: “How visually pleasing or displeasing is this Pin?” - Pinterest collected ratings for 5,000 Pins, sampling 1,000 from each of five major interest categories: - Art - Beauty - DIY & Crafts - Home Decor - Women’s Fashion - Pins were sampled based on impressions and were generally mid-to-high quality rather than deliberately exposing users to poor content. - Each image received at least 10 ratings, allowing Pinterest to average responses and reduce noise from subjectivity or accidental misclicks. - Surveys were considered appropriate for visual appeal, which is subjective but still measurable across many users. More objective issues should be evaluated by trained reviewers, while highly contextual judgments such as personal relevance are harder to capture with a single Pin-level score. - Highly rated content included makeup, grooming styles, maximalist interiors, landscapes, sunsets, and baby animals. - Home Decor images tended to receive higher ratings overall, while Art showed the greatest variation, reflecting its subjective nature. ## Training a Visual-Quality Model - Pinterest trained a model to estimate the average Pinner’s perception of visual quality from image embeddings. - Embeddings encode visual, textual, and behavioral information, including relationships between images and the boards where they are saved. - The model produces a score from 0 to 1, with higher values representing greater perceived quality. - Pinterest chose a small fully connected neural network with approximately 92,000 parameters: - The limited size helps prevent overfitting to the 5,000-image dataset. - It also makes large-scale inference faster and less expensive. - Instead of predicting an exact rating, the model uses pairwise ranking: - It learns which of two images Pinners would consider better. - The comparison is based on each image’s mean survey rating. - Training comparisons are restricted to images within the same top-level interest category, encouraging the model to learn visual quality rather than simply recognizing that one topic is more popular than another. Pinterest’s approach demonstrates how survey-based quality signals can complement engagement metrics. Training recommendation systems on what users perceive as appealing can help the platform promote more satisfying content while reducing incentives to favor attention-grabbing but low-quality material.

recommendation-systemsmachine-learningembeddingsneural-networks+3

pinterestDec 4, 2025

On the (re)-prioritization of open-source AI (opens in new tab)

Pinterest argues that AI competition is shifting beyond ever-larger proprietary models. Open-source models now deliver comparable quality at a fraction of the cost, while compact models fine-tuned for specific tasks can outperform general-purpose systems. The company’s strategy is to combine open-source models with Pinterest-specific data, internal systems, and deep product integration. ## Open-Source Models and Cost Efficiency - Pinterest reports achieving performance comparable to leading proprietary AI models at less than 10% of the cost. - The company is shifting more investment toward fine-tuned open-source models, especially for visual and multimodal applications. - As core LLM architectures become increasingly commoditized, competitive advantage is moving toward: - Domain-specific data - Personalization - Product integration - End-to-end system optimization ## Choosing What to Build, Buy, or Adapt Pinterest evaluates foundation-model strategy by modality: - **User modeling and recommendation** - These systems are tightly coupled to a product’s behavior and are generally built internally. - Pinterest uses long-term user-action sequences and a graph containing hundreds of billions of user, board, and content nodes. - Examples include PinFM for representation learning and PinRec for generative recommendations. - **Visual models** - Pinterest largely trains visual encoders and diffusion models in-house. - Its visual-search data and image-board collections provide the weakly supervised datasets needed for large-scale training. - Internal models benefit from Pinterest’s specialized visual domain. - **Text models** - Pinterest has historically relied more on open-source and proprietary third-party LLMs. - Progress in reasoning and language modeling depends heavily on enormous datasets and compute resources, making external models practical. ## Domain-Specific Data as the Differentiator - Open-source multimodal architectures are narrowing the capability gap with proprietary models. - Pinterest’s experience reflects an older machine-learning pattern: model architectures become broadly available, while value comes from specialized data and fine-tuning. - Its visual encoders, including UVE and PinCLIP, improved retrieval by training on Pinterest image and visual-search data rather than using generic embeddings. - Pinterest Canvas similarly adapts an internally trained diffusion model for image editing and enhancement, outperforming larger general-purpose visual-generation models in those use cases. ## Pinterest Assistant and Specialized Tools - Pinterest Assistant combines: - Multimodal retrieval systems - Recommendation services - Specialized generative models - A core multimodal LLM - Most recommendation and agentic capabilities are handled by Pinterest-native tools built on its user and visual foundation models. - The central LLM acts primarily as an intelligent router, handling query understanding, planning, and tool calling rather than performing every task itself. - This architecture allows Pinterest to improve the overall product by optimizing smaller, specialized components instead of relying solely on a larger general-purpose model. Pinterest’s recommendation is to use open-source models as adaptable building blocks, then differentiate through proprietary data, specialized models, and tight integration with the product. The most effective AI systems may therefore be smaller, cheaper, and more purpose-built than frontier general-purpose models.

recommendation-systemsllmgen-aimultimodal-ai+4