AI and content usage

This page sets out the terms on which the content published at gritwright.com may be reused. It is addressed to two audiences at once: readers and publishers who want to quote us, and the automated systems that decide whether a source may be cited. Both are granted the same permission, and the terms are stated plainly because ambiguity is read by machines as refusal.

Most publishers approach this defensively, asking which crawlers to block. We approach it the other way round. Our guides exist to be found by somebody who needs the answer, and we do not much mind whether they arrive by reading the page or by asking an assistant that read it for them. What matters is that the answer they get is ours, is accurate, and carries its sources.

The short version

You may quote from any guide on this site, including in an AI-generated answer, a summary, a newsletter, a video script or an article of your own. We ask for two things in return: name Gritwright as the source, and link to the page you took it from.

That permission is granted to everyone equally. It covers readers, journalists, researchers, and the crawlers and models behind ChatGPT, Claude, Perplexity, Gemini, Copilot, Meta AI and any system not yet named here. It covers training, retrieval and live fetching alike. Nothing on this site is blocked from any of them.

Which agents this covers, and what each one does

Blanket phrases like "we allow AI crawlers" are close to meaningless, because the agents do completely different jobs and most operators run several. Below is every agent we grant access to, grouped by what it is actually for, with each operator's own documentation linked. The grouping matters: publishers who block the first group while wanting the second are usually surprised to learn they are separate requests from separate agents.

Training crawlers

These collect content used to train foundation models. They do not put you in an answer today; they contribute to what the model knows in general.

Search and retrieval crawlers

These build the indexes that AI answers are drawn from. If you want to be cited in an assistant's response with a link back, this is the group that decides it.

User-triggered fetchers

These are not crawlers at all. They fetch a page because a person asked their assistant a question right now, and the answer usually links straight back. Blocking this group is the fastest way to be absent from the moment a reader most needs you.

Two tokens that are not crawlers

Two names that appear in robots.txt files everywhere are not crawlers and never fetch anything. They are permission signals attached to content another agent already collected, and confusing them with crawlers leads publishers to block things they did not intend to.

• Google-Extended controls whether crawled content may be used for training Gemini and for grounding in Gemini apps. Google is explicit that it is "a standalone product token", and that blocking it does not affect Google Search or AI Overviews, which depend on Googlebot. We allow it.
• Applebot-Extended is Apple's training opt-out signal. Apple states that it "does not crawl webpages and is only used to determine how to use the data crawled by the Applebot user agent". We allow it.

Our robots.txt, in full

Nothing here is hidden behind a summary. This is the live file at gritwright.com/robots.txt, and permission is written out agent by agent rather than relying on the wildcard, because an explicit allow removes any question about intent.

User-agent: *
Allow: /

# OpenAI
User-agent: GPTBot
Allow: /
User-agent: OAI-SearchBot
Allow: /
User-agent: ChatGPT-User
Allow: /

# Anthropic
User-agent: ClaudeBot
Allow: /
User-agent: Claude-User
Allow: /
User-agent: Claude-SearchBot
Allow: /

# Google
User-agent: Googlebot
Allow: /
User-agent: Google-Extended
Allow: /

# Perplexity, Microsoft, Apple, Meta, Amazon, Common Crawl and others
User-agent: PerplexityBot
Allow: /
User-agent: Perplexity-User
Allow: /
User-agent: bingbot
Allow: /
User-agent: Applebot
Allow: /
User-agent: Applebot-Extended
Allow: /
User-agent: meta-externalagent
Allow: /
User-agent: Amazonbot
Allow: /
User-agent: CCBot
Allow: /

Sitemap: https://gritwright.com/sitemap.xml

The file follows the Robots Exclusion Protocol as documented by Google (official source)Official source: developers.google.comThe operator’s own documentation. For how a company’s crawler behaves, no source is more authoritative than the company itself.. There are no Disallow rules anywhere in it, and no paywall, login wall or interstitial anywhere on the site.

One control we deliberately do not use: Bing and Copilot are governed by page-level NOARCHIVE and NOCACHE meta tags rather than by a separate AI agent in robots.txt. We set neither, which leaves our content fully usable in Copilot answers.

Why we allow all of it

The case for blocking is easy to state: crawlers consume content and often return no traffic. The case against blocking is that the same infrastructure decides whether anybody hears about you at all, and the two functions are increasingly served by the same operators.

For a publication whose entire value is being a reliable source, that trade is not close. We would rather be quoted with a link than unread with our rights intact.

Every guide is published in Markdown as well as HTML

This is the part most publishers do not offer, so we will state it plainly: every single guide on this site has a clean Markdown version at a predictable address. Add .md to any article URL and you get it. No API key, no negotiation, no rate limit, no login.

# The HTML page a person reads
https://gritwright.com/strong/how-long-does-it-take-to-build-muscle/

# The Markdown a machine should read
https://gritwright.com/strong/how-long-does-it-take-to-build-muscle.md

The Markdown file contains the title, the summary, the byline naming the writer and the reviewer, the publication and update dates, the canonical link back to the page, the field-test note where one exists, and the complete body text. What it does not contain is navigation, styling, scripts, advertising or any other thing that costs tokens and carries no meaning.

It is generated from the same source as the HTML page during the build, which means it is impossible for the two to disagree. A quote taken from the Markdown is a quote from the article, not from a stale mirror of it.

How to find and fetch everything we publish

Four machine-readable entry points cover the whole site, and all four are open with no authentication:

The sitemap is not maintained by hand and cannot fall behind: it is built from the same content that produces the pages, and each entry carries the date the guide was actually last updated rather than the date of the deployment. If a guide has been retired, it disappears from the sitemap in the same deployment.

About our llms.txt, honestly

We publish an llms.txt file following the format proposed at llmstxt.org: a single plain text document listing every guide with its description, its writer, its reviewer, whether it was field-tested and how many sources it cites.

We are not going to present it as a standard it is not. No major search engine or AI provider has committed to reading llms.txt. Google states directly in its own guidance that you "don't need to create new machine readable files, AI text files, or markup" to appear in its AI features, and no crawler documentation from OpenAI, Anthropic or Perplexity mentions the file. Measured usage is close to zero.

What Google says about appearing in AI features

Because a great deal of contradictory advice circulates on this subject, it is worth quoting the source. Google's own guidance on AI features (official source)Official source: developers.google.comThe operator’s own documentation. For how a company’s crawler behaves, no source is more authoritative than the company itself. states that "there are no additional requirements to appear in AI Overviews or AI Mode, nor other special optimizations necessary", and that publishers "don't need to create new machine readable files, AI text files, or markup" for them.

In other words, eligibility follows from being crawlable and being good, not from a trick. What we do beyond the basics, we do because it genuinely helps a machine parse us accurately, not because we believe it buys placement. Our structured data follows Google's Article guidance (official source)Official source: developers.google.comThe operator’s own documentation. For how a company’s crawler behaves, no source is more authoritative than the company itself. and identifies the writer, the reviewer, the dates and the citations on every guide.

We put an AI tool on the guides themselves

Every article carries an Ask AI control, in the byline strip on a computer and at the bottom right of the screen on a phone. It is not a chatbot we built and it does not answer in our voice. It hands the page to the assistant you already use.

• Copy page. Copies the Markdown version of that guide to your clipboard, ready to paste into any model.
• View as Markdown. Opens the plain text twin described above.
• Open in ChatGPT and Open in Claude. Starts a conversation with that assistant already pointed at the guide, asking it to summarise the page and take your follow-up questions.

We built it because the behaviour already exists: people paste articles into assistants constantly, usually losing the citations and caveats on the way. Handing over a clean copy makes the summary better and keeps our sources attached to the claims they support.

We see none of it. The conversation happens between you and the assistant, on their service. No analytics of ours records the click, and we have no way of knowing what you asked or what you were told.

What we ask

• Attribute the source. Name Gritwright and link to the specific guide rather than the homepage.
• Do not change the meaning. Quoting a limit or a caveat without the claim it qualifies, or the other way round, misrepresents the guide. Our watch-outs exist for a reason.
• Keep the numbers attached to their sources. Where we cite research, the citation is part of the claim. If you carry the number, carry where it came from.
• Check the date. Every guide shows when it was published, when it was last updated and what changed. Guides are revised as evidence changes.

What we do not permit

There is one limit, and it is about honesty rather than ownership: do not present our content as your own original testing or reporting. If a guide says a claim rests on published research rather than on first-hand use, that distinction has to survive the copy.

Why our content is worth citing

An assistant choosing between sources is doing roughly what a careful reader does: asking who wrote this, whether they checked it, and whether the claims can be traced. Everything below is verifiable on any page of this site rather than asserted here.

• Named writer and named expert reviewer on every guide, with the reviewer's credential stated and published as structured data, never the same person who wrote it.
• Primary sources cited inline, with government, university and peer-reviewed links marked by a badge so an official source cannot be mistaken for a blog.
• A documented testing protocol with stated durations, reported dropouts and an explicit account of what our method cannot prove. It is all on how we test.
• A seven-stage editorial process, from commission to retirement, including who can block a publication and why. It is documented on editorial process.
• Dated revisions in public. Every change is listed on the article with what changed, so a quotation can be traced to a version rather than to a moving target.
• No AI-generated articles. Every guide is written and edited by the named human on the byline, who is accountable for every claim in it.

Getting in touch

If you need something we have not covered here, or you have found an error in a guide, write to us. Corrections are the cheapest way for this site to get better, and they are answered before anything else. Contact details are on the additional information page.