Using ROBOT Meta Text

Buskerdoo · 02-26-2004, 08:14 AM

Is it worth using and which way is best:

Is this
<META NAME="robots" CONTENT="index,follow">

better than
<META NAME="ROBOTS" CONTENT="ALL">

or
META NAME="robots" CONTENT="index, follow,all">

John Scott · 02-26-2004, 08:16 AM

No. Unless you tell a spider not to crawl, it will crawl, so those are a waste of b/w.

Buskerdoo · 02-26-2004, 08:20 AM

Thanks.

Thought I'd heard that, but there is a lot of conflicting info.

Canz · 02-29-2004, 08:39 PM

I have to disagree, as far as google is concerned anyway. A few months ago www.canz.biz had fallen out of the search engines, I couldn't work out why, I hadn't changed anything I could think of.

I did searches in Google for specific phases from actual pages in quotation marks, the pages just weren't being found.

Anyway, reading around I read about robot.txt and decided to try it while I was looking for another solution, life happened, and 10 days later I got back to the problem, I noticed where checking keyword phases on my statistics that phases that could only be found on pages that were showing were now working.

Now we are showing up in all sorts of searches that we didn't before, and thats the reason I think.

Lynny