I want to prevent search engines from visiting a page (/foo/) on my site. What I mean is I don't want them even visiting it.
Right now I have this in my robots.txt file:
Quote:
User-agent: *
Disallow: /foo/
|
Will this accomplish what I want, or will they still visit it and follow links on it (just not include it in their index)?