How We’re Building Scam Alert on WhatsApp With End-to-End Encryption and Verifiability Guarantees (opens in new tab)
WhatsApp’s optional Scam Alert uses an on-device machine-learning model to identify likely scam messages without sending message content to WhatsApp, Meta, or third parties. The system is designed to preserve end-to-end encryption, give users control over warnings and reporting, and make its model and privacy safeguards independently reviewable. It is being introduced gradually in Beta while security researchers test its implementation. ## Design Principles - **On-device only:** The model and messages it analyzes remain on the user’s device. - **No automatic reporting:** WhatsApp receives message content or scam-detection information only if the user explicitly reports a chat. - **User control:** Users can enable or disable Scam Alert and decide how to respond to warnings. - Recent advances in mobile machine learning make it practical to run a small, reviewable text-classification model locally. ## How Scam Alert Works - After activation, the device downloads the model and analyzes incoming messages from non-contacts. - Classification is based on conversational structure, language signals, and patterns found in previously reported scam conversations. - When a message appears suspicious, the user sees a private warning visible only to them. - The user can: - Block the sender - Report the chat - Continue the conversation - Mark the chat as trusted - Trusted chats no longer receive Scam Alert warnings. Users may optionally share the last five received messages from a trusted chat to help improve accuracy. ## Foundational Safeguards - **Privacy-preserving analytics:** Only anonymous, aggregate warning and user-action counts are collected. - **Confidential computing:** Metrics are processed inside confidential virtual machines using trusted execution environments. - **No targeted model delivery:** WhatsApp cannot send a specific model to an individual user. - **Public transparency:** Every model version, including experimental versions, is recorded in a public transparency ledger before deployment. - **Verifiable behavior:** Model weights are published so researchers can confirm that the model is designed specifically to detect scams. ## Privacy-Preserving Analytics WhatsApp wants to measure whether Scam Alert catches scams accurately without collecting message content. The system therefore limits telemetry to two categories: - **Warning counts:** Approximate aggregate counts of how often the model displays warnings, helping measure detection rates and identify regressions. - **User action counts:** Aggregate counts of whether users trust, block, or report after receiving warnings, helping estimate false-positive rates. These metrics are protected using differential privacy, which adds carefully calibrated noise so that the presence or absence of one person’s data has negligible impact on the aggregate results. ## Confidential Federated Analytics - Devices aggregate local events before transmitting them; raw signals never leave the device. - Metrics contain no device identifiers, use coarse time intervals, and are sent at randomized times. - Data is encrypted between the device and the trusted execution environment. - Devices verify the environment’s software through hardware-backed attestations and a third-party record of approved binaries. - The confidential environment prevents WhatsApp, Meta, relays, and other intermediaries from accessing individual measurements. WhatsApp’s approach aims to provide scam detection without weakening message privacy: processing remains local, reporting remains user initiated, and system performance is measured only through minimized, privacy-protected aggregates. The feature is currently best viewed as an early Beta system whose effectiveness and security will depend on continued public review and bug-bounty testing.