# This is a file retrieved by webwalkers a.k.a. spiders that # conform to a defacto standard. # See # # This file should be uploaded to the root of your site # # The webmaster for this site is: Robert Bruce Rodger # # Format is: # User-agent: * # Disallow: /images/ # # # all user agents User-agent: * Disallow: /images/ Disallow: /test/ # you can add as many disallow lines as you wish under the # user agent line and these # would apply to all user agents. # If you discover that a specific spider is causing a # problem on your site you can add a line just to deal with that spider.