How long to spider complete site

losangel27 · 12-24-2004, 04:17 AM

right now, the search engines only show my home page... but i have 100's of pages under that... typically, how long does it take before the search engines index those pages?

jg_v7n · 12-24-2004, 05:42 AM

If they are linked correctly they probably "the search engines" have most of your pages spidered but just aren't showing them, depending on the engine it could be anything between a few days and a few months, but even then there will be fluctuations in the number of indexed pages.

Try not to worry about it and simply work on ensureing that your pages and all linked up nicely, and are well maintained - don't have any broken links!

crazyhorse · 12-25-2004, 03:25 AM

Check your log files

pcarlow · 12-27-2004, 08:13 AM

The more incoming backlinks you have, the quicker your site will be spidered and indexed.

bcombs · 12-27-2004, 01:13 PM

A sitemap that is linked from the homepage will also help. Googlebot prefers pages that are within two clicks of the homepage. Assuming the links are crawlable, it'll get them all without it, but it generally takes longer.

Brian