User-agent: * Disallow: / User-agent: moget Disallow: / User-agent: Googlebot Disallow: / User-Agent: Googlebot-Image Disallow: / User-agent: InfoSeek Sidewinder Disallow: / User-agent: Slurp Disallow: / User-agent: mogimogi Disallow: / User-agent: indexpert Disallow: / User-agent: ZyBorg Disallow: / User-agent: nabot Disallow: / User-agent: Openbot Disallow: / User-agent: aruyo Disallow: / User-agent: fast Disallow: / User-agent: Scooter Disallow: / User-agent: ia_archiver Disallow: / User-agent: Girafabot Disallow: / User-agent: Ask Jeeves Disallow: / User-agent: ArchitextSpider Disallow: / User-agent: girafa Disallow: / User-agent: DiaGem Disallow: / User-agent: SlySearch Disallow: / User-agent: Steeler Disallow: / User-agent: TurnitinBot Disallow: / User-agent: sitecheck.internetseer.com Disallow: / User-agent: http://www.almaden.ibm.com/cs/crawler Disallow: / User-Agent: http://www.almaden.ibm.com/cs/crawler [c01] Disallow: / User-agent: grub-client Disallow: / User-agent: MicrosoftPrototypeCrawler(please report obnoxious behavior to newbiecrawler@hotmail.com) Disallow: / User-agent: QuepasaCreep v0.9.13 Disallow: / User-Agent: NPBot Disallow: / User-Agent: Gaisbot/3.0+(robot@gais.cs.ccu.edu.tw;+http://gais.cs.ccu.edu.tw/robot.php) Disallow: / User-Agent: Hatena Antenna/(version string) Disallow: / User-Agent: msnbot Disallow: / User-agent: Nutch Disallow: / User-agent: NutchOrg Disallow: / User-agent: searchpreview Disallow: / User-Agent: Zao Disallow: / User-Agent: Baiduspider Disallow: / User-agent: baiduspider Disallow: / User-Agent : Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) Disallow: / User-agent: psbot Disallow: / User-Agent: ndl-japan Disallow: /