Imagine this scenario: your algorithm has pulled up a background YouTube video, or maybe a podcast. Unbeknownst to you, hackers have embedded inaudible sounds in it, designed to hijack your smart speaker or phone’s AI assistant — meaning the cybercriminals can now access your private photos, bank accounts, or any other personal information you’ve hooked up to your AI system.
It sounds like an also-ran episode of “Black Mirror,” but it’s exactly what researchers have shown is possible in new research being presented this week at the IEEE Symposium on Security and Privacy.
Basically, a team of researchers in China and Singapore found that they can construct “adversarial audio,” completely undetectable to the human ear, that tricks voice AI models into doing things they shouldn’t. Then it’s a breeze to hide it in innocent-sounding audio — a song, a movie, or anything else that unsuspecting targets might play in the background — and lay in wait for users to accidentally compromise their digital lives.
“It takes just half an hour to train this signal, and then, because this signal is context-agnostic, you can use it to attack the target model whenever you want, no matter what the user says,” lead author Meng Chen, a PhD candidate at China’s Zhejiang University, told IEEE Spectrum of the work. “These single-point defenses struggle to resist our attack because we found it’s very hard for these models to distinguish the normal user intent and our adversary attack.”
One catch, at least for now: the technique required the hackers to have access to the full weights of the AI model they’re targeting, meaning they were only able to attack open source models. But because many commercial AI systems are built on open source models, that meant that their exploit was effective against mainstream products by Microsoft and Mistral.
Mistral didn’t respond to IEEE‘s request for comment, but Microsoft issued a statement that should probably give anyone pause before connecting any important information whatsoever to one of the company’s voice AI models.
“We appreciate the researchers’ work to advance understanding of this type of technique,” it read. “This study evaluates model resilience through controlled, direct interactions with the model itself, which helps inform our approach to building model resiliency. In practice, AI models are often integrated into user applications, and we offer developers tools and guidance they can use to implement additional layers of protection that help safeguard users.”
More on AI: Researchers Alarmed by AI That Can Self-Replicate Into Another Machine
Source link
Disclaimer
We strive to uphold the highest ethical standards in all of our reporting and coverage. We blogs.grocliq.com want to be transparent with our readers about any potential conflicts of interest that may arise in our work. It’s possible that some of the investors we feature may have connections to other businesses, including competitors or companies we write about. However, we want to assure our readers that this will not have any impact on the integrity or impartiality of our reporting. We are committed to delivering accurate, unbiased news and information to our audience, and we will continue to uphold our ethics and principles in all of our work. Thank you for your trust and support.
Website Upgradation is going on for any glitch kindly connect at [email protected]
