2 Commits

Author	SHA1	Message	Date
Jack Levy	1afe8601ed	chore: expand robots.txt — block AI scrapers and SEO bots ... Add blocks for AI training crawlers (GPTBot, CCBot, Bytespider, anthropic-ai, Google-Extended, PerplexityBot, YouBot, cohere-ai), SEO tool bots (AhrefsBot, SemrushBot, DotBot, MJ12bot, BLEXBot), and /_next/ static chunks. Add Crawl-delay: 10 for well-behaved bots. Authored by: Jack Levy	2026-03-15 21:49:05 -04:00
Jack Levy	989419665e	chore: add robots.txt, update docs with mobile layout fix ... - Add frontend/public/robots.txt: allow public pages, block auth/user-private routes and /api/ - ROADMAP.md: add mobile layout fix to UX & Polish shipped items - TROUBLESHOOTING.md: add section on Android Chrome viewport overflow root cause and fix Authored by: Jack Levy	2026-03-15 21:36:46 -04:00