## ## Peterslaw - robots.txt ## The following are infinitely deep trees User-agent: * Disallow: /cgi-bin Disallow: /cms/events Disallow: /cms/latest ## Keep general spiders off the intranet but allow ours User-agent: IntranetSearch Disallow: User-agent: * Disallow: /site/home/intranet/