# Thought Equity Robots.txt file # enquiries to webmaster(at)thoughtequity.com # AllTheWeb #User-agent: fast-webcrawler #Crawl-Delay: 10 # AltaVista #User-agent: scooter #Crawl-Delay: 10 # Googlebot #User-agent: googlebot #Crawl-Delay: 10 # Looksmart #User-agent: wisenutbot #Crawl-Delay: 10 #User-agent: zyborg #Crawl-Delay: 10 # MSN #User-agent: msnbot #Crawl-Delay: 10 # Yahoo Vertical Crawler #User-agent: yahoo-verticalcrawler #Crawl-Delay: 10 # Yahoo Slurp #User-agent: Slurp #Crawl-delay: 15 #Disallow: #Disallow: /video/ ## Twiceler Slurp #User-agent: twiceler #Disallow: / ## For Pre-Release Slurp User-agent: * Disallow: /