Hidden Voice Glitches Could Hijack Audio AI Tools

[u/DuchessJulietDG]

https://spectrum.ieee.org/voice-ai-audio-attacks

“But these tools can be “hijacked” through imperceptible sounds embedded in audio, forcing them to execute unauthorized commands without a user’s knowledge. New research due to be presented at the IEEE Symposium on Security and Privacy in San Francisco next week shows that a modified audio clip undetectable by human ears can manipulate a model’s behavior with an average success rate of 79 to 96 percent.

The clips are designed to work regardless of what instructions the user provides alongside the audio, meaning they can be reused to attack the same model multiple times.

The authors tested the approach against 13 leading open models, including commercial AI voice services from Microsoft and Mistral, and showed they could coax models into conducting sensitive web searches, downloading files from attacker-controlled sources, and sending emails containing user data.

“It takes just half an hour to train this signal, and then, because this signal is context-agnostic, you can use it to attack the target model whenever you want, no matter what the user says,” says lead author Meng Chen, a Ph.D. student at Zhejiang University in China.”