User-agent: * Allow: / Disallow: /admin/ Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: /wp-content/plugins/ Disallow: /wp-content/cache/ # AI crawlers blocked from training User-agent: GPTBot Disallow: / User-agent: anthropic-ai Disallow: / User-agent: ClaudeBot Disallow: / User-agent: CCBot Disallow: / # Allow Google AI Overviews / SearchGPT (they cite source) User-agent: Google-Extended Allow: / User-agent: OAI-SearchBot Allow: / Sitemap: https://soltia.net/sitemap.xml Sitemap: https://soltia.net/blog/sitemap_index.xml