User-agent: * Allow: / # Search and answer-retrieval bots, named explicitly. These fetch pages so answers can cite them. User-agent: Googlebot Allow: / User-agent: Bingbot Allow: / User-agent: Applebot Allow: / User-agent: DuckAssistBot Allow: / User-agent: OAI-SearchBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: PerplexityBot Allow: / User-agent: Perplexity-User Allow: / User-agent: Claude-SearchBot Allow: / User-agent: Claude-User Allow: / User-agent: MistralAI-Index Allow: / User-agent: MistralAI-User Allow: / User-agent: Meta-ExternalFetcher Allow: / # Link-preview fetchers (the share image and title when a link is pasted into a chat). User-agent: facebookexternalhit Allow: / User-agent: LinkedInBot Allow: / User-agent: Twitterbot Allow: / # Crawlers that collect content for model training. Allowed by default. Opting out is # Griffin's call: to opt out, change Allow to Disallow under any of these. User-agent: GPTBot Allow: / User-agent: ClaudeBot Allow: / User-agent: Google-Extended Allow: / User-agent: Applebot-Extended Allow: / User-agent: MistralAI-Training Allow: / User-agent: Meta-ExternalAgent Allow: / User-agent: Amazonbot Allow: / User-agent: CCBot Allow: / Sitemap: https://griffinadelmann.com/sitemap-index.xml