Robots.txt

[DimlerJ]

Does anyone know if the absence of robots.txt influence Google in a bad way? I don't have any page I wouldn't want Google to index, so is it a bad decision not to create a robots.txt empty file?

User-agent: *
Disallow:

[Costin Trifan]

I don't think that will be a bad decision because spiders when are reaching your site they are looking for robots.txt first. I believe that that robots.txt is considered as beeing their map of your site.

I also believe that robots.txt file should exist on your site only if you want to restrict some robots to visit some pages. Otherwise, there's no need for it.

[paladin2]

You should use the robots.txt if you have rss feeds, from a content page. The generated rss is the duplicate content of that particular page.

[cityba]

Originally Posted by DimlerJ

Does anyone know if the absence of robots.txt influence Google in a bad way? I don't have any page I wouldn't want Google to index, so is it a bad decision not to create a robots.txt empty file?

User-agent: *
Disallow:

All robot every directory.Good robots.txt

[sniperhiga]

Originally Posted by paladin2

You should use the robots.txt if you have rss feeds, from a content page. The generated rss is the duplicate content of that particular page.

Good point. Can you tell me code so i can copy and paste...