Writing Help: WhatsApp's AI writing help promises maximum data protection

WhatsApp has released an AI-supported writing aid called "Writing Help". To counteract its reputation as a "data octopus", the parent company Meta wants to secure the processing of private messages with an exceptionally high level of technical effort. However, two security audits published in parallel show that the "Private Processing" architecture developed for this purpose initially had significant and sometimes serious vulnerabilities.

With "Writing Help", WhatsApp wants to help its users formulate messages in future. Such a function inevitably raises the question of how the content of private end-to-end encrypted chats can be processed without jeopardizing confidentiality.

Meta's answer to this is the "Private Processing" architecture that has been under development for some time. According to the company, this is technically designed to ensure that no one – not even Meta employees – can access the processed messages. According to Meta, the system is based on several pillars:

• Anonymized requests: before a request reaches Meta's servers, the identity of the user is concealed by the Oblivious HTTP protocol (OHTTP) via an external relay service.
• Isolated processing: The actual AI analysis takes place in a shielded Trusted Execution Environment (TEE).
• Stateless processing: The messages are only stored in volatile memory for the duration of processing and then deleted again immediately.

Audits revealed vulnerabilities

To prove the security of this complex system, Meta commissioned the NCC Group (PDF) and Trail of Bits (PDF) to carry out independent audits. The reports, which list a total of 49 vulnerabilities (21 from NCC, 28 from Trail of Bits), have been published and paint a clear picture: the approach is ambitious, but was anything but secure at the time of the audits.

Both security teams independently identified several critical design flaws which, according to Meta, have since been fixed:

• Risk of de-anonymization: both NCC and Trail of Bits found that the key configurations required for anonymization (OHTTP) were initially delivered directly from Meta servers to clients. This would have enabled Meta to specifically remove anonymity and assign requests to individual users.
• Lack of "freshness guarantee": Both audits criticized the fact that there were no mechanisms in place to check that the TEE software – for the Trusted Execution Environment – was up to date. An attacker could have continued to use a software version that was once certified as secure but later found to be vulnerable indefinitely in order to attack users.
• Lack of hardware binding: Trail of Bits also found that the attestation was not bound to specific CPUs controlled by Meta. An attacker could have used any SEV-SNP CPU compromised elsewhere to spoof the system.

NCC Group also found unnecessary network interfaces in the protected VMs that a compromised host server could have used to exfiltrate data. Trail of Bits also discovered the possibility of code injection through environment variables and an initial complete lack of security checks on the NVIDIA GPUs used for AI calculations.

Trust remains a basic requirement

According to the reports, Meta fixed most of the critical vulnerabilities before launching the feature. Both audits come to a similar conclusion: despite the sophisticated technology, the security of "Private Processing" ultimately depends on assumptions of trust. Users must trust that Meta is not colluding with its infrastructure partners (such as Fastly and Cloudflare) or that the non-open source parts of the system and the images delivered as binaries do not contain hidden backdoors.

Both reports strongly recommended that Meta work towards full reproducibility of builds from open source code to enable true, independent public review. Meta has been running its bug bounty program since 2011.

(mack)