|
If at all possible make sure you also lock the directory with an empty index.html in the least and .htaccess file at the most.
Posting a directory in robot.txt leads "people" to look inside that directory to see what's there.
If you have no link to a directory and the directory that "this" directory is under is locked by a index.html or .htaccess file the robots will never find it unless you tell them where it is.
Aram
|