Proper robot.txt, keep SE's out

cldnails · 10-10-2007, 07:36 AM

I'm looking to block all search engines, or any that listen to robots.txt, from a subdomain. Could anyone point me in the right direction to create a proper robots.txt?

I've done some searching and found some conflicting reports. Some say that using the simple disallow agent it will block from the entire server, which is not what I want to do. I simply want to keep them out of a subdomain.

Johan007 · 10-10-2007, 08:34 AM

Search Engines consider a sub domain as an entirely new website. All the information is here Webmaster Help Center: Block or remove your entire website using a robots.txt file

cldnails · 10-10-2007, 08:37 AM

Thanks for the link, I was just a little confused and didn't want to take a chance on barring google from all my other sites on my server. But, if I place that robots.txt in just the folder of the subdomain I should be ok...right?

zagu060807 · 10-10-2007, 09:56 AM

Quote:

Originally Posted by cldnails

I'm looking to block all search engines, or any that listen to robots.txt, from a subdomain. Could anyone point me in the right direction to create a proper robots.txt?

I've done some searching and found some conflicting reports. Some say that using the simple disallow agent it will block from the entire server, which is not what I want to do. I simply want to keep them out of a subdomain.

Create it using notepad, then upload it in your root folder. Then specify a folder to restrict...