Uncontrolled access to on-device data could undermine message privacy, says Whittaker
Life
Meredith Whittaker, the head of the Signal Technology Foundation, has expressed concern about the potential risks of AI agents to user privacy at a recent conference in the US. She believes we face a real threat because of the significant control these systems exert and their access to vast amounts of data.
Whittaker described AI agents as “magic bots” capable of thinking in multiple steps and performing tasks for users, essentially disabling their own cognitive processes.
An example often cited is an AI agent’s ability to search for concerts, buy tickets and even send friends messages about the event. However, Whittaker said she was concerned about the data these agents require at various stages of this process, potentially giving them access to sensitive information that users would prefer to keep private.
She explained that agents need access to users’ browsers, credit card information for ticket purchases, calendars, contact lists and even the ability to open Signal to send messages. This implies a wide range of permissions, similar to ‘root access,’ giving the agent control over numerous system databases.
Whittaker argued that an AI agent powerful enough would likely process data outside the device and send information back and forth to cloud servers. This raises serious security and privacy concerns, and could compromise the boundary between the application and operating system layers. Ultimately, she said, this uncontrolled access could undermine message privacy.
Whittaker’s concerns are shared by other prominent figures in the field of AI. Yoshua Bengio, a renowned AI researcher, reiterated similar warnings in January at the World Economic Forum in Davos. He emphasised that the development of AI agents could lead to the disaster scenarios often associated with artificial general intelligence, machines that can reason just like humans.
Bengio stresses the need for ongoing research into safe AI development while recognising and understanding the inherent risks. He called for investment in technologies designed to prevent the creation of systems that could eventually pose a threat to humanity.
Business AM
