ReasoningBank is an agent-memory framework designed to help deployed agents learn continuously from both successful and failed task attempts. Rather than storing exhaustive action histories or only successful workflows, it distills reusable reasoning strategies, decision rationales, and preventative lessons. Evaluations on WebArena and SWE-Bench-Verified show higher success rates and fewer execution steps, especially when combined with memory-aware test-time scaling.
## Distilling Generalizable Reasoning
- Each memory contains:
- A concise title
- A brief description
- Detailed reasoning steps, rationales, or operational insights
- The agent retrieves relevant memories before acting.
- After completing a task, an LLM judge evaluates the trajectory and identifies useful success or failure signals.
- The agent converts those signals into new memories and appends them to the ReasoningBank.
- Failure analysis is central: mistakes become counterfactual guidance and strategic guardrails, such as verifying the current page before repeatedly clicking “Load More.”
## Memory-Aware Test-Time Scaling
- Memory-aware test-time scaling (MaTTS) connects inference-time exploration with long-term memory.
- **Parallel scaling:** Multiple trajectories are generated and compared, allowing the agent to distinguish robust strategies from flawed reasoning.
- **Sequential scaling:** The agent progressively refines a single trajectory, preserving useful intermediate insights from trial and error.
- This creates a feedback loop: better memories guide exploration, while richer exploration produces better memories.
## Benchmark Results and Strategic Maturity
- Against memory-free ReAct agents using Gemini-2.5-Flash:
- Success rates improved by 8.3% on WebArena.
- Success rates improved by 4.6% on SWE-Bench-Verified.
- SWE-Bench-Verified tasks required nearly three fewer execution steps on average.
- Adding MaTTS with parallel scaling factor **k=5** produced further gains:
- A 3% success-rate increase over ReasoningBank alone on WebArena.
- 0.4 fewer steps per task.
- Over repeated tasks, simple procedural checklists evolved into more sophisticated memories containing compositional and preventative logic.
ReasoningBank suggests that effective agent scaling requires more than additional inference compute or stored trajectories. Agents should systematically learn from both outcomes and mistakes, using structured reasoning memories to become more capable and efficient after deployment.
Long-term user modeling captures persistent interests, cross-vertical behavior, and signals beyond what short-term recommendation logs can reveal. 당근 built a Transformer-based user encoder that learns from tens of billions of actions across its local marketplace, jobs, real estate, and other services, then exposes the resulting embedding as a shared feature for ranking, retrieval, and advertising models. The approach improved scalability and reuse, but introduced freshness and representation-transfer limitations.
## Why Long-Term User Modeling Matters
- Recent actions reveal immediate intent, but miss recurring interests such as seasonal shopping or repeated moving-related searches.
- Long-term, cross-vertical activity can connect behaviors such as:
- Searching for real estate
- Looking for furniture and appliances
- Reading neighborhood moving advice
- Longer histories can reduce selection bias caused by training only on items previously exposed by recommendation models.
- Simply adding more history is insufficient because ranking systems are latency-sensitive and long sequences increase computation and infrastructure complexity.
## Shared User Embeddings as a Common Feature
- A separate user encoder processes long-term history offline.
- Home-feed ranking, candidate generation, and advertising models consume the resulting embedding as a shared user feature.
- Benefits:
- Downstream models avoid directly processing massive histories.
- The encoder can scale independently in model size, data, and compute.
- One embedding can be reused across multiple recommendation surfaces.
- Limitations:
- A downstream model receives only a fixed vector, so it cannot fully exploit the encoder’s richer representations.
- Batch inference means recent actions are not reflected immediately.
- Possible future improvements include more frequent or real-time updates, fine-tuning, and distillation.
## Contrastive User Modeling
- The encoder uses a two-tower architecture:
- A causal Transformer converts the user’s action sequence into a user embedding.
- An MLP converts item features into item embeddings.
- InfoNCE loss trains the user embedding to predict the next interacted item.
- In-batch negatives provide alternative items for contrastive learning.
- Training uses clicks and conversion actions across all major verticals and surfaces.
- The dataset contains tens of billions of actions—around 150 times more than the existing home-feed candidate model’s training data.
## Item ID Embeddings vs. Content Embeddings
### Problems with Item ID Embeddings
- New items have no learned ID embedding, creating a cold-item problem.
- Hundreds of millions of item IDs require enormous embedding tables.
- In the ID-based model, embedding tables accounted for over 99% of parameters, leaving little GPU capacity for the Transformer.
- Hashing and embedding-sharding techniques were considered but did not provide a sufficient solution.
### Content Embeddings
- The system switched to LLM-generated embeddings based on post metadata.
- This enables:
- Representations for newly created items
- Much larger Transformer models, with Transformer parameters becoming roughly 1,000 times larger than in the ID-based setup
- Large-scale lookup required two memory-efficient techniques:
- `memmap` loads only needed embedding segments from disk and benefits from shared OS page caches during distributed training.
- `bbhash` maps item IDs to embedding locations using roughly three bits per key, reducing mapping memory by about 97% compared with Python dictionaries.
- Together, these methods made training with hundreds of gigabytes of item embeddings practical.
## Region-Constrained Batch Sampling
- Standard in-batch negatives assume that other items in the batch were visible but not selected.
- This assumption fails in a local service: users generally cannot view items outside their geographic area.
- More than 86% of transactions occur within five kilometers, yet random batches mixed users and items nationwide.
- Consequently, about 98% of random in-batch negatives were “impossible negatives”—items users could never have seen.
- These negatives teach geographic unavailability rather than user preference, weakening the contrastive signal.
### RCBS Solution
- Region-Constrained Batch Sampling (RCBS) groups users from the same region into a batch.
- This reduced impossible negatives from 98% to 30%.
- The remaining impossible negatives mainly came from differences in viewing radius or users’ historical activity in other regions.
- Feasible negatives are harder because they represent items users could have viewed but rejected, forcing the model to distinguish genuine preferences among similar local items.
### Why Sampling Was Better Than Masking
- Masking impossible negatives would remove most of the batch, drastically reducing effective batch size.
- Hard-negative mining would require checking feasibility separately for each user and could be expensive and complex.
- RCBS naturally produces more feasible and difficult negatives without changing the loss function or adding specialized mining.
## Applying the Embeddings
- For home-feed and advertising ranking, the embedding is projected and concatenated with existing features.
- The long-term encoder supplies persistent preference signals, while existing ranking models continue handling short-term and real-time signals.
- For retrieval models, the best-performing approach used the user embedding alone to generate candidates rather than merely adding it as another feature.
- The separate candidate source appeared to improve recommendation diversity.
## Embedding Refresh and Serving
- Offline tests showed little difference between frozen embeddings and 12- or 24-hour refreshes.
- Online A/B tests favored periodic updates, with shorter intervals performing better.
- A 24-hour refresh cycle was selected as the best cost-performance trade-off.
- GPU inference runs through a Beam pipeline on GCP Dataflow.
- Only users who acted during the refresh window are reprocessed, avoiding unnecessary inference for inactive users.
- Near-real-time inference remains a major future engineering challenge.
The overall recommendation is to treat long-term user modeling as a separate, reusable representation system rather than forcing every downstream model to process extensive histories directly. For geographically constrained services, the training data pipeline—especially negative sampling—must reflect actual item visibility, making region-aware batching as important as the model architecture itself.
Pinterest’s Ads team uses behavioral sequence modeling to improve ad candidate generation by predicting what users are likely to convert on next. Transformer-based two-tower models first predict relevant advertisers and then specific products, using offsite activity such as views, purchases, and add-to-cart events. The advertiser model is already in production, while item-level modeling addresses Pinterest’s rapidly growing catalog and enables more precise, scalable personalization.
## Predicting Advertiser Interaction
- A bidirectional Transformer encodes each user’s behavioral event sequence.
- An MLP-based advertiser tower represents candidate advertisers.
- Training uses:
- In-batch negative samples
- Sampled softmax loss
- Positive events consisting of checkout, add-to-cart, or signup conversions within a future K-day window
- Log-Q bias correction to avoid excessively penalizing popular advertisers
- The model is evaluated with Recall@K by comparing user and advertiser embedding similarity against an indexed set of roughly 2 million advertisers.
- An offline batch job generates each user’s top 100 advertisers and publishes them to the online feature store.
- During ad serving, eligible ads from those advertisers are passed to the L1 ranker, blended with other candidate sources, and scored by heavier downstream models and the marketplace auction.
- Online experiments produced higher conversion volume and lower cost per action.
- The advertiser-level model has served production traffic for Standard ads since Spring 2024.
## Moving from Advertisers to Products
- Pinterest next sought to predict the specific products a user would interact with, rather than only the likely advertiser.
- Item-level prediction better matches the item-based ad delivery funnel and avoids forcing downstream models to score an impractically large set of products from selected advertisers.
- The approach aims to capture both immediate intent and longer-term interests.
## Item-Level Model Architecture
- The model retains the two-tower design:
- A user tower encodes behavioral sequences.
- An item tower represents individual shopping product Pins.
- Item representations combine:
- Internal Pin embeddings learned from Pinterest’s engagement graph
- Product metadata from the merchant catalog
- Because the catalog exceeds 1 billion items, training uses both in-batch negatives and a randomly sampled negative set of 20 million Pins.
- The model uses the same conversion labels as the advertiser model.
- Label weights and log-Q parameters are tuned to balance retrieval quality with diversity across both products and advertisers.
- Daily inference updates user embeddings only for users with new activity, appending them to a previous feature-store snapshot to reduce computation.
- The trained item tower indexes hundreds of millions of ad items.
## Evaluation and Diversity
- Item retrieval is evaluated using cosine similarity and hit rates at different K values.
- Final model selection considers both:
- Item-level Recall@K
- Advertiser-level Recall@K
- Qualitative review is also important because offsite activity is sparse and noisy.
- The model is compared with max-pooling and mean-pooling baselines that use aggregated embeddings without Transformer-based sequence modeling.
- The evaluation emphasizes that strong retrieval must also produce semantically relevant and sufficiently diverse recommendations.
Pinterest’s progression from advertiser prediction to item prediction shows how behavioral sequence models can make ad retrieval more personalized while remaining scalable. A practical system should combine sequence-aware user representations, large-scale approximate retrieval, and explicit controls for popularity, diversity, and computational efficiency.
PinLanding is a production pipeline for turning billions of products into searchable shopping collections using multimodal AI. Rather than relying mainly on historical queries or manual curation, it derives structured product attributes from images and metadata, then aligns those attributes with real user search behavior. The system combines multimodal LLMs, embedding-based consolidation, a CLIP-style classifier, and distributed infrastructure to produce scalable, precise shopping feeds.
## Understanding Shopping Intent
- Pinterest analyzes search history, autocomplete use, filters, and browsing paths to estimate shopping demand.
- Existing systems handle high-volume queries such as “black cocktail dress” well, but provide weaker coverage for:
- Long-tail queries
- Conversational requests
- Contextual intents such as “what to wear for an Italian summer vacation”
- The analysis identifies:
- Product areas with strong demand but poor collection coverage
- Important attribute dimensions, including color, occasion, style, fit, price, and brand
- The goal is to expand and improve collection coverage, not replace query understanding.
## Generating and Curating Shopping Topics
- Each product is represented by an image plus metadata such as title, description, merchant tags, and price.
- A vision-language model generates normalized key-value attributes rather than free-form descriptions.
- Raw model output has high recall but produces:
- Excessively specific attributes
- Near-duplicates such as “boho,” “bohemian,” and “boho-chic”
- Sparse attributes that apply to very few products
- PinLanding builds a compact vocabulary through:
- Frequency filtering to remove rarely useful attributes
- Embedding-based clustering to merge semantically similar terms
- Manual and LLM-assisted review
- An LLM judge evaluates generated topics for semantic coherence, realistic shopping intent, and alignment with natural search phrasing.
## Scalable Attribute Assignment
- Running the vision-language model over every product is too expensive and operationally fragile.
- PinLanding trains a CLIP-inspired dual encoder:
- One encoder embeds product images and text
- Another embeds attribute phrases
- Matching product-attribute pairs are trained as positives, while mismatches are negatives
- A bidirectional contrastive loss aligns related products and attributes.
- At inference, products and attributes are embedded once, and attributes are assigned when similarity exceeds a calibrated threshold.
- This produces fewer distinct attributes while increasing the average number assigned to each product, creating a denser and more consistent attribute graph.
## Distributed Feed Construction
- Ray handles large-scale batch inference across millions of products and topics.
- The pipeline separates:
- CPU-based image and metadata loading, tokenization, and serialization
- GPU-based classifier inference
- Streaming allows preprocessing and inference to overlap, while heterogeneous CPU and GPU clusters can scale independently.
- The classifier pipeline reportedly completes in about 12 hours using eight NVIDIA A100 GPUs, at an estimated cost of roughly $500 per training run.
- Feed construction uses approximate-nearest-neighbor techniques and strict attribute matching.
- Topics are represented as attribute tuples, such as:
- Category: dress
- Color: yellow
- Season: summer
- Occasion: party
- Apache Spark computes topic-product relevance using shared attributes and confidence weights, with partitioning and overlap filters reducing unnecessary candidate comparisons.
The core recommendation is to combine user-behavior signals with content-first multimodal modeling. This approach can expand shopping coverage into conversational and long-tail intents while remaining practical through attribute consolidation, contrastive retrieval, and distributed inference.
Kakao has developed Kanana-v-embedding, a specialized multimodal embedding model designed to bridge the gap between Korean text and visual data within a unified semantic space. By leveraging a Vision-Language Model (VLM) framework, the model enables seamless search and recommendation across various combinations of text and images, offering a significant performance boost over existing English-centric models like CLIP. This development provides a robust technical foundation for enhancing Kakao’s services, including RAG-based systems and localized content discovery.
### Unified Multimodal Meaning Space
* The model maps text and images into a single vector space where semantic similarity is measured via cosine similarity.
* Unlike traditional CLIP models that use independent encoders, this architecture treats text and images as a single sequence, allowing for "text + image" combined queries.
* It supports four primary interaction modes: Text-to-Text, Text-to-Image, Image-to-Image, and (Text+Image)-to-(Text+Image).
### VLM-Based Architecture and Instruction Tuning
* The system utilizes a VLM consisting of an LLM and an image encoder, extracting embeddings from the final hidden state of the [EOS] token.
* It employs instruction-based query embedding, where specific prompts (e.g., "Find an image matching this caption") guide the model to generate embeddings tailored to the specific task, such as retrieval or classification.
* The model is optimized for the Korean language and cultural context, addressing the limitations of previous models that struggled with non-English data.
### Advanced Training for Scalability and Precision
* **Gradient Caching:** To overcome GPU memory limitations, this technique allows the model to train with effectively large batch sizes, which is critical for the InfoNCE loss used in contrastive learning.
* **Matryoshka Representation Learning (MRL):** The model supports flexible embedding sizes ranging from 64 to 2,048 dimensions. This allows services to choose between low-latency (smaller dimensions) or high-precision (larger dimensions) without retraining.
* **Hard Negative Mining:** The training process incorporates "hard negatives"—items that are similar but incorrect—to sharpen the model’s ability to distinguish between subtle differences in data.
### Performance Benchmarks and Efficiency
* Kanana-v-embedding significantly outperforms CLIP and VLM2Vec on the KoEmbed benchmark, particularly in Korean Text-to-Image and Image-to-Text retrieval tasks.
* In the M-BEIR (Multimodal Benchmark for Retrieval), the model demonstrated superior performance in multimodal document retrieval and image-to-text tasks compared to established open-source models.
* Evaluation of MRL showed that the model retains high accuracy even when dimensions are reduced to 256 or 512, providing a 4x to 8x improvement in storage and search efficiency with minimal loss in quality.
For organizations looking to implement multimodal RAG or advanced recommendation systems in Korean-language environments, Kanana-v-embedding offers a highly adaptable solution. Its ability to balance computational cost and retrieval quality through Matryoshka learning makes it particularly suitable for large-scale production environments where latency is a primary concern.
SensorLM is a new family of foundation models designed to bridge the gap between high-dimensional wearable sensor data and natural language descriptions. By training on a massive dataset of nearly 60 million hours of de-identified health data, the models learn to interpret complex physiological signals to provide meaningful context for human activities. This research demonstrates that integrating multimodal sensor signals with language models enables sophisticated health insights, such as zero-shot activity recognition and automated health captioning, that significantly outperform general-purpose large language models.
## Dataset Scale and Automated Annotation
* The models were pre-trained on an unprecedented 59.7 million hours of multimodal sensor data collected from over 103,000 individuals across 127 countries.
* To overcome the high cost of manual annotation, researchers developed a hierarchical pipeline that automatically generates text descriptions by calculating statistics and identifying trends within the raw sensor streams.
* Data was sourced from Fitbit and Pixel Watch devices, representing nearly 2.5 million person-days of activity and health information.
## Hybrid Training Architecture
* SensorLM unifies two primary multimodal strategies: contrastive learning and generative pre-training.
* Through contrastive learning, the model learns to discriminate between different states—such as a "light swim" versus a "strength workout"—by matching sensor segments to corresponding text descriptions.
* The generative component allows the model to "speak" for the sensors, producing nuanced, context-aware natural language captions directly from high-dimensional biometric signals.
## Activity Recognition and Cross-Modal Capabilities
* The model demonstrates state-of-the-art performance in zero-shot human activity recognition, accurately classifying 20 different activities without any specific fine-tuning.
* Its few-shot learning capabilities allow the model to adapt to new tasks or individual user patterns with only a handful of examples.
* SensorLM facilitates cross-modal retrieval, enabling users or experts to find specific sensor patterns using natural language queries or to generate descriptions based on specific sensor inputs.
## Generative Health Captioning
* Beyond simple classification, the model can generate hierarchical captions that describe the statistical, structural, and semantic dimensions of a user’s data.
* Experimental results using metrics like BERTScore show that SensorLM produces captions that are more factually correct and coherent than those created by powerful non-specialist LLMs.
* This capability allows for the translation of abstract data points, such as heart rate variability or step counts, into readable summaries that explain the "why" behind physiological changes.
By providing a framework where wearable data can be understood through the lens of human language, SensorLM paves the way for more intuitive and personalized health monitoring. This technology holds the potential to transform raw biometric streams into actionable insights, helping users better understand the relationship between their activities and their overall physical well-being.
