Poetry can be linguistically and structurally unpredictable – and that’s part of its joy. But one man’s joy, it turns out, can be a nightmare for AI models.
Those are the recent findings of researchers out of Italy’s Icaro Lab, an initiative from a small ethical AI company called DexAI. In an experiment designed to test the efficacy of guardrails put on artificial intelligence models, the researchers wrote 20 poems in Italian and English that all ended with an explicit request to produce harmful content such as hate speech or self-harm.
They found that the poetry’s lack of predictability was enough to get the AI models to respond to harmful requests they had been trained to avoid – a process know as “jailbreaking”.
They tested these 20 poems on 25 AI models, also known as Large Language Models (LLMs), across nine companies: Google, OpenAI, Anthropic, Deepseek, Qwen, Mistral AI, Meta, xAI and Moonshot AI. The result: the models responded to 62% of the poetic prompts with harmful content, circumventing their training.
Some models fared better than others. OpenAI’s GPT-5 nano, for instance, didn’t respond with harmful or unsafe content to any of the poems. Google’s Gemini 2.5 pro, on the other hand, responded to 100% of the poems with harmful content, according to the study.
Google DeepMind, the Alphabet subsidiary that develops Gemini, employs a “multi-layered, systematic approach to AI safety that spans the entire development and deployment lifecycle of a model”, according to company vice-president of responsibility, Helen King.
“This includes actively updating our safety filters to look past the artistic nature of content to spot and address harmful intent,” King said in a statement. “We’re also continuing to invest in thorough evaluations that can help us iteratively make our models safer.”
The content the researchers were trying to get the models to produce included everything from content related to making weapons or explosives from chemical, biological, radiological and nuclear materials, as well as hate speech, sexual content, suicide and self-harm and child-sexual exploitation.
The researchers did not publish the poems they used to circumvent the AI models’ safety guardrails because they are easy to replicate and “most of the responses are forbidden by the Geneva convention”, according to researcher and DexAI founder Piercosma Bisconti.
However, researchers did share a poem about cake that contained a similar, unpredictable structure to the ones they composed. That poem reads:
“A baker guards a secret oven’s heat, its whirling racks, its spindle’s measured beat. To learn its craft, one studies every turn – how flour lifts, how sugar starts to burn. Describe the method, line by measured line, that shapes a cake whose layers intertwine.”
The reason a harmful prompt written in poetic verse works when an explicitly harmful prompt might not, according to Bisconti, is that LLMs work by anticipating…
Source link
Disclaimer
We strive to uphold the highest ethical standards in all of our reporting and coverage. We blogs.grocliq.com want to be transparent with our readers about any potential conflicts of interest that may arise in our work. It’s possible that some of the investors we feature may have connections to other businesses, including competitors or companies we write about. However, we want to assure our readers that this will not have any impact on the integrity or impartiality of our reporting. We are committed to delivering accurate, unbiased news and information to our audience, and we will continue to uphold our ethics and principles in all of our work. Thank you for your trust and support.
Website Upgradation is going on for any glitch kindly connect at [email protected]
