# DesignCourse robots.txt. AI-crawler profile: OPEN (KIT_05 ยง6). Search engines and AI search agents are allowed. User-agent: * Allow: / Disallow: /api/ Disallow: /search # Bulk scrapers (all profiles); also blocked by a Cloudflare WAF rule User-agent: Bytespider User-agent: img2dataset User-agent: Diffbot User-agent: Omgilibot Disallow: / Sitemap: https://designcourse.in/sitemap-index.xml