# robots.txt # Evil non-search engine robot that requests bad URLs and pummels server User-agent: http://www.almaden.ibm.com/cs/crawler Disallow: / ### Paul blocked all spiders on 2/16/2009 User-agent: * Disallow: / ### crawl-delay sets the crawler connection interval to this many seconds ### http://help.yahoo.com/l/us/yahoo/search/webcrawler/slurp-03.html Crawl-delay: 60 ###Disallow: /admin ###Disallow: /editor ###Disallow: /logs/ ###Disallow: /docs/ ###Disallow: /rate/ ###Disallow: /emailto.php ###Disallow: /mod/comments/