meta·Aug 12, 2026·3 min readCurated summary

How We’re Building Scam Alert on WhatsApp With End-to-End Encryption and Verifiability Guarantees

WhatsApp’s optional Scam Alert uses an on-device machine-learning model to identify likely scam messages without sending message content to WhatsApp, Meta, or third parties. The system is designed to preserve end-to-end encryption, give users control over warnings and reporting, and make its model and privacy safeguards independently reviewable. It is being introduced gradually in Beta while security researchers test its implementation. ## Design Principles - **On-device only:** The model and messages it analyzes remain on the user’s device. - **No automatic reporting:** WhatsApp receives message content or scam-detection information only if the user explicitly reports a chat. - **User control:** Users can enable or disable Scam Alert and decide how to respond to warnings. - Recent advances in mobile machine learning make it practical to run a small, reviewable text-classification model locally. ## How Scam Alert Works - After activation, the device downloads the model and analyzes incoming messages from non-contacts. - Classification is based on conversational structure, language signals, and patterns found in previously reported scam conversations. - When a message appears suspicious, the user sees a private warning visible only to them. - The user can: - Block the sender - Report the chat - Continue the conversation - Mark the chat as trusted - Trusted chats no longer receive Scam Alert warnings. Users may optionally share the last five received messages from a trusted chat to help improve accuracy. ## Foundational Safeguards - **Privacy-preserving analytics:** Only anonymous, aggregate warning and user-action counts are collected. - **Confidential computing:** Metrics are processed inside confidential virtual machines using trusted execution environments. - **No targeted model delivery:** WhatsApp cannot send a specific model to an individual user. - **Public transparency:** Every model version, including experimental versions, is recorded in a public transparency ledger before deployment. - **Verifiable behavior:** Model weights are published so researchers can confirm that the model is designed specifically to detect scams. ## Privacy-Preserving Analytics WhatsApp wants to measure whether Scam Alert catches scams accurately without collecting message content. The system therefore limits telemetry to two categories: - **Warning counts:** Approximate aggregate counts of how often the model displays warnings, helping measure detection rates and identify regressions. - **User action counts:** Aggregate counts of whether users trust, block, or report after receiving warnings, helping estimate false-positive rates. These metrics are protected using differential privacy, which adds carefully calibrated noise so that the presence or absence of one person’s data has negligible impact on the aggregate results. ## Confidential Federated Analytics - Devices aggregate local events before transmitting them; raw signals never leave the device. - Metrics contain no device identifiers, use coarse time intervals, and are sent at randomized times. - Data is encrypted between the device and the trusted execution environment. - Devices verify the environment’s software through hardware-backed attestations and a third-party record of approved binaries. - The confidential environment prevents WhatsApp, Meta, relays, and other intermediaries from accessing individual measurements. WhatsApp’s approach aims to provide scam detection without weakening message privacy: processing remains local, reporting remains user initiated, and system performance is measured only through minimized, privacy-protected aggregates. The feature is currently best viewed as an early Beta system whose effectiveness and security will depend on continued public review and bug-bounty testing.

Read original(opens in new tab)

Federated AnalyticsDifferential PrivacyEnd To End EncryptionConfidential Computing+3

google·May 27, 2026·3 min readCurated summary

Private analytics via zero-trust aggregation

Google presents a private analytics architecture combining one-shot cryptographic secure aggregation with trusted execution environments (TEEs). The design follows a zero-trust model: cryptography prevents individual data from being reconstructed, while TEE attestation verifies that approved, publicly auditable code is running. It aims to provide useful population-level insights without exposing raw user data, even if hardware protections are compromised. ## Why Private Analytics Matters - On-device AI protects user information, but developers still need to measure performance across millions of devices. - Aggregated analytics can reveal: - Model drift, such as difficulty handling new language or slang. - Hidden biases across regions, environments, or demographic conditions. - Real-world error rates and user reactions to model outputs. - Google already uses federated analytics in products including Pixel Recorder and Gboard. ## Limitations of Existing Protections - **TEEs** isolate sensitive computation from compromised operating systems and hypervisors. - Hardware attestation produces a cryptographic fingerprint proving which firmware and software are running. - However, TEEs remain vulnerable to newly discovered side-channel attacks. - Traditional cryptographic aggregation offers mathematical privacy guarantees, but many protocols require devices to stay online through several interactive rounds. - Extended multiround participation limits practical deployment at large scale. ## Combining Encryption and Isolation - The new system allows devices to submit data in a single message. - This removes the need for devices to remain connected for multiple protocol rounds. - Data is encrypted before leaving the device and is never reconstructed in server memory, including inside a TEE. - Unencrypted information is processed off-device only after it has been aggregated and anonymized. - TEE attestation provides verifiable evidence that the intended secure aggregation implementation is running correctly. - The cryptographic and hardware layers provide defense in depth: failure of one layer does not automatically expose individual data. ## One-Shot Lattice-Based Aggregation - The protocol uses lattice-based cryptography. - Ciphertexts can be combined so that their underlying data—and encryption keys—are aggregated together. - A resulting decryption key can reveal only the aggregate, not individual contributions. - Small client committees hold decryption hints and help unlock the aggregated result. - Differential privacy noise is added to further protect the aggregate. - Decryption authority is distributed across multiple parties, preventing any single party from accessing encrypted data. ## Application to Android SafetyCore - SafetyCore provides privacy-preserving, on-device safety features for Android 9 and later. - Private analytics can help measure which threats are detected and identify areas where detection needs improvement. - Google is working with the Android SafetyCore team to apply this system while preserving the confidentiality of individual users’ data. ## Practical Takeaway The approach pairs the scalability of one-shot cryptographic aggregation with the verifiability of TEEs. It is designed for private, large-scale analytics where neither individual data nor trust in a single hardware or software component is required.

Read original(opens in new tab)

Federated AnalyticsZero TrustDifferential PrivacyOn Device Ai+3

google·Oct 29, 2025Original article

Toward provably private insights into AI use (opens in new tab)

Google Research has introduced Provably Private Insights (PPI), a framework designed to analyze generative AI usage patterns while providing mathematical guarantees of user privacy. By integrating Large Language Models (LLMs) with differential privacy and trusted execution environments (TEEs), the system enables developers to derive aggregate trends from unstructured data without exposing individual user content. This approach ensures that server-side processing remains limited to privacy-preserving computations that are fully auditable by external parties. ### The Role of LLMs in Structured Summarization The system employs "data expert" LLMs to transform unstructured generative AI data into actionable, structured insights. * The framework utilizes open-source Gemma 3 models to perform specific analysis tasks, such as classifying transcripts into topics or identifying user frustration levels. * This "structured summarization" occurs entirely within a TEE, ensuring that the model processes raw data in an environment inaccessible to human operators or external processes. * Developers can update LLM prompts frequently to answer new research questions without compromising the underlying privacy architecture. ### Confidential Federated Analytics (CFA) Infrastructure The PPI system is built upon Confidential Federated Analytics, a technique that isolates data through hardware-based security and cryptographic verification. * User devices encrypt data and define specific authorized processing steps before uploading it to the server. * A TEE-hosted key management service only releases decryption keys to processing steps that match public, open-source code signatures. * System integrity is verified using Rekor, a public, tamper-resistant transparency log that allows external parties to confirm that the code running in the TEE is exactly what was published. ### Anonymization via Differential Privacy Once the LLM extracts features from the data, the system applies differential privacy (DP) to ensure that the final output does not reveal information about any specific individual. * The extracted categories are aggregated into histograms, with DP noise added to the final counts to prevent the identification of single users. * Because the privacy guarantee is applied at the aggregation stage, the system remains secure even if a developer uses a prompt specifically designed to isolate a single user's data. * All aggregation algorithms are open-source and reproducibly buildable, allowing for end-to-end verifiability of the privacy claims. By open-sourcing the PPI stack through the Google Parfait project and deploying it in applications like Pixel Recorder, this framework establishes a new standard for transparent data analysis. Developers should look to integrate similar TEE-based federated analytics to balance the need for product insights with the necessity of provable, hardware-backed user privacy.

Federated AnalyticsAILarge Language ModelsGenerative AI+5