User-agent: ClaudeBot Allow: / Crawl-delay: 10 seconds User-agent: ShapBot Disallow: / User-agent: Go-http-client Disallow: / User-agent: DotBot Disallow: / User-agent: python-requests Disallow: / User-agent: Scrapy Disallow: / User-agent: slopsearch-linux-crawler Disallow: / User-agent: WIndexBot Disallow: / User-agent: AIWebIndex Disallow: /