Cloudflare launched a feature that converts HTML pages to markdown when AI systems request it. Sites on its network can now serve lighter content to bots without building separate pages.
The feature, called Markdown for Agents, works through HTTP content negotiation. An AI crawler sends a request with Accept: text/markdown in the header. Cloudflare intercepts it, fetches the original HTML from the origin server, converts it to markdown, and delivers the result.
The launch arrives days after Google’s John Mueller called the idea of serving markdown to AI bots “a stupid idea” and questioned whether bots can even parse markdown links properly.
What’s New
Cloudflare described the feature as treating AI agents as “first-class citizens” alongside human visitors. The company used its own blog post as an example. The HTML version consumed 16,180 tokens while the markdown conversion used 3,150 tokens.
“Feeding raw HTML to an AI is like paying by the word to read packaging instead of the letter inside,” the company wrote.
The conversion happens at Cloudflare’s edge network, not at the origin server. Websites enable it per zone through the dashboard, and it’s available in beta at no additional cost for Pro, Business, and Enterprise plan customers, plus SSL for SaaS customers.
Cloudflare noted that some AI coding tools already send the Accept: text/markdown header. The company named Claude Code and OpenCode as examples.
Each converted response includes an x-markdown-tokens header that estimates the token count of the markdown version. Developers can use this to manage context windows or plan chunking strategies.
Content-Signal Defaults
Converted responses include a Content-Signal header set to ai-train=yes, search=yes, ai-input=yes by default, signaling the content can be used for AI training, search use, and AI input (including agentic use). Whether a given bot honors those signals depends on the bot operator. Cloudflare said the feature will offer custom Content-Signal policies in the future.
The Content Signals framework, which Cloudflare announced during Birthday Week 2025, lets site owners set preferences for how their content gets used. Enabling markdown conversion also applies a default usage signal, not just a format change.
How This Differs From What Mueller Criticized
Mueller was criticizing a different practice. Some site owners build separate markdown pages and serve them to AI user agents through middleware. Mueller raised concerns about cloaking and broken linking, and questioned whether bots could even parse markdown properly.
Cloudflare’s feature uses a different mechanism. Instead of detecting user agents and serving alternate pages, it relies on content negotiation. The same URL serves different representations based on what the client requests in the header.
Mueller’s comments addressed user-agent-based serving, not content negotiation. In a Reddit thread about Cloudflare’s feature, Mueller responded with the…
Source link
Disclaimer
We strive to uphold the highest ethical standards in all of our reporting and coverage. We blogs.grocliq.com want to be transparent with our readers about any potential conflicts of interest that may arise in our work. It’s possible that some of the investors we feature may have connections to other businesses, including competitors or companies we write about. However, we want to assure our readers that this will not have any impact on the integrity or impartiality of our reporting. We are committed to delivering accurate, unbiased news and information to our audience, and we will continue to uphold our ethics and principles in all of our work. Thank you for your trust and support.
Website Upgradation is going on for any glitch kindly connect at [email protected]
