Google can find content that is unlinked

janeth · 03-09-2004, 11:49 AM

http://www.webproworld.com/viewtopic.php?p=80884

The bot, says Craig, can find content that's unlinked. That's right, the Google bot can find single pages dangling unlinked in space. He didn't explain how this happens.

alpha3m · 03-09-2004, 12:00 PM

interesting.. very very interesting...

kwvarga · 03-09-2004, 12:06 PM

*cough* google bar.

janeth · 03-09-2004, 12:07 PM

Ok, I did not think about the Google bar.

edbarton · 03-09-2004, 12:46 PM

It seems to spider my sites that aren't links. It might do it via looking at the directory although I am not sure why.

spidersam · 03-09-2004, 01:35 PM

Interesting. Maybe it is the Googlebar spying.

I, Brian · 03-10-2004, 04:25 AM

Quote:

Originally Posted by janeth

http://www.webproworld.com/viewtopic.php?p=80884

The bot, says Craig, can find content that's unlinked. That's right, the Google bot can find single pages dangling unlinked in space. He didn't explain how this happens.

I'd like to see an example.

We've discussed this issue before, to some degree, here at IMR. Generally, it is still down to links, from somewhere, anywhere.

I haven't yet seen evidence of the toolbar actually forwarding URLs to index either.

Orphan pages can be pretty common, and seem to take around 2-3 months to be removed from the Google index.

But...I'll set up a page on SEO lab to test this later. Remind me if I forget.

sem4u · 03-10-2004, 05:14 AM

Probably spidering from whois data.

compar · 03-10-2004, 05:43 AM

I'm not sure we are all talking about the same thing.

I think we are talking about individual web pages that for one reason or the other are not linked internally to a site, and of course have no IBLs. This frequently happens when a new page is created and rather than erase the old page it is just left in the folder with the rest of the site.

There is no way using the whois record that Google could find these pages. I think the whois comment would only apply to entire sites with no IBLs and I don't think that is what is being talked about here.

So how would Google find these orphan or abandoned pages. The only way I can see then doing it is if they has psuedo ftp access to your web folders. That would allow them to find every page in the folder without using hypertext.

I guess my other question is why would they want to do this? If a page has been abandoned, and no longer in use, what connection can it have to the site's relevance? And relevance is still supposed to be Google's holy grail?

Andy · 03-10-2004, 06:44 AM

The other thing to bear in mind would be that the Access logs for many Sites are held in public folders. Therefore any time you visit this "Isolated page" and then move on to another site you could be providing Google (or any other spider) a link to the page in the form of a refferer URL in the logs.

This kind of thing can easily be overlooked and provides one possible explanation of how pages such as these could be spidered. I would have to see much more evidence to believe this theory. Especially as I, Brian points out there is no evidence of the Toolbar passing pages to the index.

I tell you what - It'll be a scary thought if Google has learned the skills of the Psychic

Andy