User-agent: Googlebot Allow: / User-agent: Googlebot-Image Allow: / User-agent: Googlebot-News Allow: / User-agent: Googlebot-Video Allow: / User-agent: Googlebot-Mobile Allow: / User-agent: AdsBot-Google Allow: / User-agent: AdsBot-Google-Mobile Allow: / User-agent: Mediapartners-Google Allow: / User-agent: Storebot-Google Allow: / User-agent: Google-InspectionTool Allow: / User-agent: GoogleOther Allow: / User-agent: Bingbot Allow: / User-agent: Twitterbot Allow: / User-agent: facebookexternalhit Allow: / # ---- Explicitly blocked: AI training scrapers (no referral traffic) ---- User-agent: GPTBot Disallow: / User-agent: ClaudeBot Disallow: / User-agent: anthropic-ai Disallow: / User-agent: CCBot Disallow: / User-agent: Google-Extended Disallow: / User-agent: Applebot-Extended Disallow: / User-agent: Amazonbot Disallow: / User-agent: Bytespider Disallow: / User-agent: Meta-ExternalAgent Disallow: / User-agent: FacebookBot Disallow: / # ---- Allowed: answer-engine bots that can send referral traffic ---- # ChatGPT-User, OAI-SearchBot, Claude-Web, PerplexityBot fetch pages # on-demand to cite in AI answers — leaving them allowed preserves # visibility in those products. User-agent: Google-Extended Disallow: / User-agent: Applebot-Extended Disallow: / User-agent: Amazonbot Disallow: / User-agent: Bytespider Disallow: / User-agent: Meta-ExternalAgent Disallow: / User-agent: FacebookBot Disallow: / # ---- Explicitly blocked: SEO / scraping crawlers ---- User-agent: AhrefsBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: DotBot Disallow: / User-agent: PetalBot Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: BLEXBot Disallow: / User-agent: SeznamBot Disallow: / User-agent: serpstatbot Disallow: / # ---- Everything else: allowed, but stay out of admin + honeypot ---- User-agent: * Allow: / Disallow: /host/ Disallow: /trap/ Disallow: /blocked Sitemap: https://lakeconroeescape.com/sitemap.xml