# Crawlers (Google, Bing, etc.) # -------------------------------------------------------------------------- User-agent: * Crawl-delay: 10 Disallow: /archive/ Disallow: /captcha/ Disallow: /cgi-bin/ Disallow: /db/ Disallow: /jquery/ Disallow: /jquery-plugins/ Disallow: /logs/ Disallow: /stats/