#1 spot with robots.txt disallowing

John Scott · 10-22-2003, 08:03 AM

http://www.google.com/search?q=forum+images

http://www.forumimages.com/robots.txt

Just an FYI thing showing the power of anchor text for those who still don't understand it.

Gede · 10-22-2003, 08:07 AM

But its looks shit, such an empty description...

And is it usefull?

John Scott · 10-22-2003, 08:44 AM

That's Daz's site. He doesn't want search engine traffic. I am not suggesting that people should exclude robots, but it does show how much weight Google puts on anchor text.

Kinda makes the "seo copywriters" look silly.

PhilC · 10-22-2003, 08:56 AM

Summat funny there. How long has the robots.txt disallow all been in place?

It could be that Google spidered the site when indexing was allowed, then the disallow went up and the stuff has been removed from some parts of Google's system but not from other parts. I can't see Google displaying results in the serps for pages that have been disallowed. I reckon it's a transitional state.

John_Poger · 10-22-2003, 08:59 AM

wow. That's a bit strange.

John Scott · 10-22-2003, 09:04 AM

Quote:

Summat funny there. How long has the robots.txt disallow all been in place?

Phil, Google has never seen that page, as far as I know. Robots.txt has been in place for over 6 months.

I was just looking for an example of a case I heard about. Couldn't find it right off, but I'll look some more.

It was a case of a website linking to a non-existent page from every page of it's site. The page never existed, but maintained top ranking for the search term.

Like, if I linked all my sites to internet-marketing-research.net/this-page-does-not-exist.php using the link text "PhilCraven7", it might just rank top for that search term, even while being a 404.

PhilC · 10-22-2003, 09:13 AM

Wanna try it? It's Google bombing (which still works) but without a page on the other end.

I was basing the idea that Google had spidered the site on the deep URLs that are listed. I was assuming that nobody would link to those deep URLs, but that's not right - websites do link to topics.

John Scott · 10-22-2003, 09:19 AM

I'm going to try to find that example... Damn I should bookmark those.

Bubo · 10-22-2003, 09:21 AM

Why not just make your own example? Your recent skirmish with Rackshack has shown how quickly you can get a #1 ranking. Do the same thing with a non-competitive test phrase and let's see what happens in a week.

John Scott · 10-22-2003, 09:27 AM

Just for those who don't know, EV1 Servers kinda got on my bad side, so I put up a page and got the #1 position for "EV1 Servers" in two days. It dropped one spot today, but I'll get it back.

I only have so much pagerank, and don't like to waste it on non-existent pages.