# exclude help system from robots User-agent: * Disallow: / #Disallow: /manual/ #Disallow: /doc/ #Disallow: /gif/ # but allow htdig to index our doc-tree User-agent: susedig Disallow: # disallow stress test user-agent: stress-agent Disallow: / User-agent: MelodyCatcher Disallow: Disallow: /florian/Media/Karaoke Disallow: /doc/ User-agent: Browsershots Disallow: