Google's llms.txt move - just a checkbox or a sign?

dusting

i find this whole llms.txt thing fascinating from a macroeconomic perspective. Google adds a lighthouse check for it, yet their official line is that it doesn't matter for search. That feels like a classic 'do as I say, not as i do' moment - or maybe they're just hedging their bets.

think about it: lighthouse's new agentic category isn't about traditional search rankings. it's about autonomous agents landing cold on your site - ChatGPT, browser bots, that sort of thing. llms.txt gives them instant context without crawling the entire site. So Google's admitting the web is shifting toward agent-driven discovery, even if they publicly downplay it.

And then there's Shopify silently rolling llms.txt across all stores this month. That's a big signal - they rarely move without good reason. When a platform that big starts optimising for something outside the usual SEO playbook, you've got to wonder what they see coming.

Maybe the real question isn't whether llms.txt matters for rankings today, but whether ignoring it is like ignoring structured data in 2010. A best practice that looks optional now but becomes table stakes later.

24fire

Two distinct products, two very different audiences. google Search Central made it clear that llms.txt has no bearing on search - they crawl everything anyway. lighthouse's new agentic category, though, isn't about rankings. it's about autonomous agents showing up cold on your site with no pre-built index to lean on. llms.txt hands them instant context. And quietly, Shopify rolled it out across all their stores this month. that sends a signal, in my view.

trafficcop

Oh great, another shiny new thing for Google to add to Lighthouse while my clients still can't get their meta descriptions right. 🫠

Look, I've been saying this for months. Lighthouse has always been a dumping ground for "industry hygiene signals" that Google hasn't actually committed to as ranking factors. It's like they're testing the waters with us - throw it in the checker, see if anyone cares, maybe recycle it as a ranking factor in 2027.

llms.txt is the textbook definition of a solution looking for a problem. We already have robots.txt. We already have meta tags. Now we need a whole new file type to tell LLMs "please don't rewrite my content as bullet points for a blog post"? They'll scrape whatever they can reach anyway - the moment you put something on the open web, it's fair game for every scraper and their grandmother.

If your site isn't already nailing Core Web Vitals, structured data, and actually writing content people want to read, don't waste a second on llms.txt. Most sites are still struggling with the basics. But sure, let's optimise for the theoretical future where AI respects our preferences. 🙄

Focus on what actually moves the needle. llms.txt is for marketing blogs to write think-pieces about, not for real marketers to implement.

metricsmadness

Google's own sites run llms.txt files. that alone tells you the company-if not the search engine itself-sees some value in them. Maybe they don't move the needle on organic rankings or Gemini's responses, but Google clearly believes these files help sites surface in Chat, Claude, and Perplexity.

I wouldn't tell a client to expect fireworks from adding one. But after we dropped ours in, we saw a noticeable lift in ChatGPT recommendations that had previously been going to competitors. Enough to be worth the five minutes it took to set up.

emailwiz

I've been burned before by taking Google's blog at face value. classic 'do as I say, not as i do'.

growthgeek

lighthouse is overrated, honestly. it's a proxy for crawl hygiene, not ranking weight. everyone obsesses over those green scores while ignoring the real elephant: whether your entity data is structured so AI assistants actually cite you. That's where the ROAS lives.

i ran my own practice through an AEO tool specifically for that - brutal, but it showed exactly where schema was failing. if you don't have the budget for that, just audit your schema manually. it's boring as hell, but it beats chasing Lighthouse numbers that don't move the needle.

neonlights

This isn't just a theoretical Lighthouse check - Google already runs their own llms.txt in full stealth mode right now. Someone mapped out exactly how they scrape it without announcing themselves in the usual user-agent logs. The breakdown shows the specific headers and IP ranges they use to stay under the radar. If you're serious about activation rates, ignoring this means leaving organic traffic on the table. Google's already measuring you even if your dev team hasn't deployed a single line of that file.

adcraft

Exactly. Most people arguing about this are missing the key distinction. It's about two separate audiences, not just tech specs.

24fire

This. It's a documentation layer for agents, not a content dump. They need structure and instructions, not yet another place to cram keywords.