On a recent Search Off the Record podcast, hosts John Mueller and Martin Splitt pushed back on the idea promoted by AI SEOs that stripped-down, content-only versions are a better way to optimize for AI Search. They made the case that all the things AI SEOs want to remove are actually useful for ranking.
Non-Content Parts Of Web Pages Matter
The TL;DR of this part is that HTML is for browsers to render into a visible page for humans, as well as for screen readers to read.
Martin Splitt begins the discussion by explaining why plain HTML appears not to be the ideal way to provide content to AI agents and LLMs. The idea is that, in addition to content, there’s a lot of other code in the HTML that is irrelevant for an LLM or AI agent that may be visiting a site for the content.
The appeal of markdown, then, is that it can provide the content in a manner that breaks free of all the HTML that’s meant to make a web page visible for humans or readable by a screen reader.
Splitt explains:
“And I think that’s also why people think it’s good for LLMs, because you have less stuff, less tokens. And if you look at an HTML file without a browser rendering it, if you just look at the plain HTML in a text editor, basically, then it’s hard to read the content, because there’s so much cruft, so much stuff in it. There’s all these HTML tags and all this maybe even inline styles and all that kind of stuff.”
He also praises markdown for the ability to still communicate the essence of the content:
“But if a Markdown render fails and you look at the Markdown file in a text editor, it still is structured and readable. Like a link is the word of the link text, like the anchor text, and then in square brackets and then in normal brackets. It’s probably what I would do if text was all I had available.
If I was writing an email without the possibility to actually link things, I would probably mark up some sort of link text and then put some sort of way to say, like, and this is where you need to go to actually see that.
And I think this minimalism is probably what makes people think, yeah, this is great for a machine that needs to understand this content, unlike HTML.”
Converting HTML To Text Is Trivial
Mueller and Splitt noted that despite how complex HTML looks, crawling and making sense of it is trivial and very easy to do. The selling point about using markdown for LLMs, that it simplifies crawling and indexing content, completely breaks down at this point.
John Mueller explains:
“I think the big thing is that the web with HTML and everything has been around for really long time, longer than Markdown. And all of the crawlers out there, have practiced with HTML. And converting HTML into text is trivial. There are lots of libraries out there that can do that for you. So if you think about what an average web crawler might look for or might need to find on a page to be able to understand it, then probably that’s just HTML.”
Markdown Fails…
Source link
Disclaimer
We strive to uphold the highest ethical standards in all of our reporting and coverage. We blogs.grocliq.com want to be transparent with our readers about any potential conflicts of interest that may arise in our work. It’s possible that some of the investors we feature may have connections to other businesses, including competitors or companies we write about. However, we want to assure our readers that this will not have any impact on the integrity or impartiality of our reporting. We are committed to delivering accurate, unbiased news and information to our audience, and we will continue to uphold our ethics and principles in all of our work. Thank you for your trust and support.
Website Upgradation is going on for any glitch kindly connect at [email protected]
