# exclude help system from robots User-agent: * Disallow: /manual/ /gif/ /error/ /intranet/ /template/ /cdb/ /htdig/ Disallow: /images/ /webalizer/ /logs/ /test/ Disallow: /intern/ Disallow: /cms/ Disallow: /*.pdf$ User-agent: Googlebot Disallow: /*.pdf$ Disallow: /manual/ /gif/ /error/ /intranet/ /template/ /cdb/ /htdig/ Disallow: /images/ /webalizer/ /logs/ /test/ Disallow: /intern/ Disallow: /cms/ # Wozu das?: # but allow htdig to index our doc-tree # User-agent: susedig # Disallow: