# robots.txt for www.wondrfly.com # # Served by settings/routes.js -> GET /robots.txt (reads this file). # Keep in sync with wondrfly_web/src/robots.txt, which Angular ships as a static # asset — whichever origin answers /robots.txt must return the same rules. # # NOTE: Cloudflare's managed robots.txt feature currently REPLACES this file at the # edge, so none of the rules below are live. Disable the managed override (or switch # it to append mode) in the Cloudflare dashboard for these to take effect. User-agent: * Allow: / # Admin, API and internal/experimental surfaces — no crawl value, and /api/ would # otherwise expose JSON endpoints to the index. Disallow: /admin/ Disallow: /api/ Disallow: /category/ Disallow: /blogs-latest/ Disallow: /blogs-latest Disallow: /search-latest Disallow: /welcome Disallow: /discover-landing # Sitemap index. Both paths serve the same file (settings/routes.js registers # /sitemap_index.xml and /sitemaps/*), but only the canonical one is advertised — # listing both invites duplicate discovery. Sitemap: https://www.wondrfly.com/sitemaps/sitemap_index.xml