techlist.io
KR EN

personalization

3 posts

meta

Adapting the Facebook Reels RecSys AI Model Based on User Feedback - Engineering at Meta (opens in new tab)

Meta has enhanced the Facebook Reels recommendation engine by shifting focus from traditional engagement signals, like watch time and likes, to direct user feedback. By implementing the User True Interest Survey (UTIS) model, the system now prioritizes content that aligns with genuine user preferences rather than just short-term interactions. This shift has resulted in significant improvements in recommendation relevance, high-quality content delivery, and long-term user retention. **Limitations of Engagement-Based Metrics** * Traditional signals like "likes" and "watch time" are often noisy and may not reflect a user’s actual long-term interests. * Models optimized solely for engagement tend to favor short-term value over the long-term utility of the product. * Internal research found that previous heuristic-based interest models only achieved 48.3% precision in identifying what users truly care about. * Effective interest matching requires understanding nuanced factors such as production style, mood, audio, and motivation, which implicit signals often miss. **The User True Interest Survey (UTIS) Model** * Meta collects direct feedback via randomized, single-question surveys asking users to rate video interest on a 1–5 scale. * The raw survey data is binarized to denoise responses and weighted to correct for sampling and nonresponse bias. * The UTIS model functions as a lightweight "alignment model layer" built on top of the main multi-task ranking system. * The architecture uses existing model predictions as input features, supplemented by engineered features that capture content attributes and user behavior. **Integration into the Ranking Funnel** * **Late Stage Ranking (LSR):** The UTIS score is used as an additional input feature in the final value formula, allowing the system to boost high-interest videos and demote low-interest ones. * **Early Stage Ranking (Retrieval):** The model aggregates survey data to reconstruct user interest profiles, helping the system source more relevant candidates during the initial retrieval phase. * **Knowledge Distillation:** Large sequence-based retrieval models are aligned using UTIS predictions as labels through distillation objectives. **Performance and Impact** * The deployment of UTIS has led to a measurable increase in the delivery of niche, high-quality content. * Generic, popularity-based recommendations that often lack depth have been reduced. * Meta observed robust improvements across core metrics, including higher follow rates, more shares, and increased user retention. * The system now offers better interpretability, allowing engineers to understand which specific factors contribute to a user’s sense of "interest match." To continue improving the Reels ecosystem, Meta is focusing on doubling down on personalization by tackling challenges related to sparse data and sampling bias while exploring more advanced AI architectures to further diversify recommendations.

personalizationaimachine-learningrecommendation-systems+5
line

Introducing a New A/B (opens in new tab)

LY Corporation has developed an advanced A/B testing system that moves beyond simple random assignment to support dynamic user segmentation. By integrating a dedicated targeting system with a high-performance experiment assigner, the platform allows for precise experiments tailored to specific user characteristics and behaviors. This architecture enables data-driven decisions that are more relevant to localized or specialized user groups rather than relying on broad averages. ## Limitations of Traditional A/B Testing * General A/B test systems typically rely on random assignment, such as applying a hash function to a user ID (`hash(id) % 2`), which is simple and cost-effective. * While random assignment reduces selection bias, it is insufficient for hypotheses that only apply to specific cohorts, such as "iOS users living in Osaka." * Advanced systems solve this by shifting from general testing across an entire user base to personalized testing for specific segments. ## Architecture of the Targeting System * The system processes massive datasets including user information, mobile device data, and application activity stored in HDFS. * Apache Spark is used to execute complex conditional operations—such as unions, intersections, and subtractions—to refine user segments. * Segment data is written to Object Storage and then cached in Redis using a `{user_id}-{segment_id}` key format to ensure low-latency lookups during live requests. ## A/B Test Management and Assignment * The system utilizes "Central Dogma" as a configuration repository where operators and administrators define experiment parameters. * A Test Group Assigner orchestrates the process: when a client makes a request, the assigner retrieves experiment info and checks the user's segment membership in Redis. * Once a user is assigned to a specific group (e.g., Test Group 1), the system serves the corresponding content and logs the event to a data store for dashboard visualization and analysis. ## Strategic Use Cases and Future Plans * **Content Recommendation:** Testing different Machine Learning models to see which performs better for a specific user demographic. * **Targeted Incentives:** Limiting shopping discount experiments to "light users," as coupons may not significantly change the behavior of "heavy users." * **Onboarding Optimization:** Restricting UI tests to new users only, ensuring that existing users' experiences remain uninterrupted. * **Platform Expansion:** Future goals include building a unified admin interface for the entire lifecycle of an experiment and expanding the system to cover all services within LY Corporation. For organizations looking to optimize user experience, transitioning from random assignment to dynamic segmentation is essential for high-precision product development. Ensuring that segment data is cached in a high-performance store like Redis is critical to maintaining low latency when serving experimental variations in real-time.

personalizationdata-architectureapache-sparkredis+4
naver

[DAN25] (opens in new tab)

Naver recently released the full video archives from its DAN25 conference, highlighting the company’s strategic roadmap for AI agents, Sovereign AI, and digital transformation. The sessions showcase how Naver is moving beyond general AI applications to implement specialized, real-time systems that integrate large language models (LLMs) directly into core services like search, commerce, and content. By open-sourcing these technical insights, Naver demonstrates its progress in building a cohesive AI ecosystem capable of handling massive scale and complex user intent. ### Naver PersonA and LLM-Based User Memory * The "PersonA" project focuses on building a "user memory" by treating fragmented logs across various Naver services as indirect conversations with the user. * By leveraging LLM reasoning, the system transitions from simple data tracking to a sophisticated AI agent that offers context-aware, real-time suggestions. * Technical hurdles addressed include the stable implementation of real-time log reflection for a massive user base and the selection of optimal LLM architectures for personalized inference. ### Trend Analysis and Search-Optimized Models * The Place Trend Analysis system utilizes ranking algorithms to distinguish between temporary surges and sustained popularity, providing a balanced view of "hot places." * LLMs and text mining are employed to move beyond raw data, extracting specific keywords that explain the underlying reasons for a location's trending status. * To improve search quality, Naver developed search-specific LLMs that outperform general models by using specialized data "recipes" and integrating traditional information retrieval with features like "AI briefing" and "AuthGR" for higher reliability. ### Unified Recommendation and Real-Time CRM * Naver Webtoon and Series replaced fragmented recommendation and CRM (Customer Relationship Management) models with a single, unified framework to ensure data consistency. * The architecture shifted from batch-based processing to a real-time, API-based serving system to reduce management complexity and improve the immediacy of personalized user experiences. * This transition focuses on maintaining a seamless UX by synchronizing different ML models under a unified serving logic. ### Scalable Log Pipelines and Infrastructure Stability * The "Logiss" pipeline manages up to tens of billions of logs daily, utilizing a Storm and Kafka environment to ensure high availability and performance. * Engineers implemented a multi-topology approach to allow for seamless, non-disruptive deployments even under heavy loads. * Intelligent features such as "peak-shaving" (distributing peak traffic to off-peak hours), priority-based processing during failures, and efficient data sampling help balance cost, performance, and stability. These sessions provide a practical blueprint for organizations aiming to scale LLM-driven services while maintaining infrastructure integrity. For developers and system architects, Naver’s transition toward unified ML frameworks and specialized, real-time data pipelines offers a proven model for moving AI from experimental phases into high-traffic production environments.

personalizationaillmmachine-learning+4