Google adds llms.txt check to Chrome Lighthouse

Google’s new Lighthouse “Agentic Browsing” audits now check for the presence of an llms.txt file. The new experimental Lighthouse documentation frames llms.txt as a discoverability and efficiency signal for AI agents, not a traditional crawling directive.

• The audits are part of Chrome’s emerging “Agentic Browsing” category, which evaluates whether sites are structured for machine interaction.
• This document comes less than a week after Google published new guidance on optimizing for AI search features like AI Overviews and AI Mode, in which it said you don’t need llms.txt files in a mythbusting section of its new guide on optimizing for generative AI features.

What Lighthouse now checks. Lighthouse’s Agentic Browsing category evaluates “how well your site is constructed for machine interaction” using deterministic audits, according to Google’s documentation. Among the checks:

• WebMCP integration.
• Accessibility tree integrity.
• Layout stability through CLS.
• Presence of an llms.txt file.

Lighthouse checks for “the presence of a machine-readable summary at the domain root.” Google also explained why the file matters for agents:

“Without llms.txt, agents may spend more time crawling the site to understand its high-level structure and primary content.”

The audit category doesn’t produce a traditional Lighthouse score (0-100). Instead, Google surfaces a fractional pass ratio along with pass/fail checks tied to agentic readiness signals.

The tension. The new Lighthouse documentation doesn’t directly conflict with Google’s advice on optimizing your website for generative AI features because these audits focus on AI agents and browser tools, not Google Search rankings. Still, seeing llms.txt mentioned in Chrome’s own readiness checks may cause some SEOs to rethink earlier doubts about the file.

Agentic engine optimization. The Lighthouse audits also align with ideas Google Cloud AI engineering director Addy Osmani outlined in April around Agentic Engine Optimization. Osmani said AI agents with limited context windows may cut off long pages or miss important information buried too deep in content. Among his recommendations:

• Cleaner semantic structure.
• Token-efficient content.
• Markdown delivery.
• llms.txt discovery layers.
• Capability signaling files like AGENTS.md.

SEO vs. llms.txt. Here’s exactly what Google recommends in Mythbusting generative AI search: what you don’t need to do:

• LLMS.txt files and other “special” markup: You don’t need to create new machine readable files, AI text files, markup, or Markdown to appear in generative AI search. Note that Google may discover, crawl, and index many kinds of files in addition to HTML on a website: this doesn’t mean that the file is treated in a special way.

Here’s what Google’s John Mueller said about Google using llms.txt, in response to Lily Ray asking him on Bluesky…