Google Crawl How Often?

intruth · 11-20-2006, 04:57 PM

It seems like my site gets crawled by google every 3-4 days but yahoo and msn crawl it daily...

How often does Google crawl your site?

trichnosis · 11-20-2006, 05:30 PM

that depents on your backlink . if you have strong backlinks , google may crawl your site daily

uranus · 11-20-2006, 07:23 PM

On some of my directories, daily others two to three days

TheBrain · 11-20-2006, 07:42 PM

My situation is very weird.
I have several sites and most of them are visited once in a while by Googlebot, but there is one, which I don't want indexed, where Google is there all day and night, ignoring robots.txt, metatags and all the unimaginable tricks I'm using to kick it off.

Julie · 11-20-2006, 07:52 PM

There are always Google spiders on my sites. Yahoo and MSN visit daily. Ask visits every couple of days.

Webfu · 11-20-2006, 11:19 PM

Yes some of our pages ger crawled through everyday. I've also noticed some pages that don't get linked to much might get crawled maybe once a month.

vistadivine.com · 11-21-2006, 01:24 AM

Well it depends upon how many quality links you have more links means a better crawlage.

chillfire · 11-21-2006, 01:36 AM

My clients site (visalogic.net) is being hit;
google - 200
baidu - 2
msn - 30
slurp - 30

all figs a re approx, I have a live dashboard I wrote for the site so these are avg of today (8am) and all of yesterday.
Generally whenever MSN goes through the roof it means they are getting ready to launch something, the weeks before live.com was launched in beta their spiders were really doing the rounds and were hitting us more then google.

backlinks, backlinks and more backlinks (the closer the content is between the pages the better for link scoring!?)

Does anyone else have detailed spider activity?

Dotster · 11-21-2006, 01:44 AM

I think that it depends on your NEW backlink. That's all.

overdriveelectronics · 11-21-2006, 09:19 AM

Yep, it all based on backlinks, the more you have the more times the bots will come to crawl your site. But just because they crawl doesnt mean that they will index!

SaltLampKing · 11-21-2006, 02:53 PM

We only get crawled about once a week.

hhheng · 11-22-2006, 02:01 AM

Googlebot keeps visiting my website everyday, while yahoo and msn visit it every week.

For some keywords, google has much more results than yahoo, while for others, yahoo has more than google.

Very funny and confused!

vistadivine.com · 11-22-2006, 02:11 AM

Well all the search engines may have a different point of view while viewing the same page so it is possible.

edpudol · 11-22-2006, 03:59 AM

my guess is that googel will visit your site depending on how many back link you have and how high their PR. Also your site PR have something to do with it also.

vistadivine.com · 11-22-2006, 04:37 AM

Quote:

Originally Posted by edpudol

my guess is that googel will visit your site depending on how many back link you have and how high their PR. Also your site PR have something to do with it also.

Yes exactly.

webguru · 11-22-2006, 11:20 AM

Every 24 hours. If you have new content on site it will get crawled
everyday.

mazter · 11-22-2006, 06:35 PM

You can see actual 'crawl rate' from Google Webmaster tools.
https://www.google.com/webmasters/tools/

According to that tool, at least 150 pages are crawled daily.

TE75 · 11-25-2006, 08:48 AM

Some of my sites get crawled at least once a day, others may get crawled every 3 - 4 days. I have noticed that once I submit a sitemap, my time between crawls usually increases after a week or so.

Diddy1 · 11-25-2006, 02:14 PM

Quote:

Originally Posted by intruth

It seems like my site gets crawled by google every 3-4 days but yahoo and msn crawl it daily...

How often does Google crawl your site?

If you have google sitemaps there is a new feature that shows you how much google has crawled your website. And you can also tell it how many times you want to crawl your site daily. Just click on the website name and and on the "Diagnostic" tab look all the way down and you should see a crawl rate section.
It has three options faster normal and slow. But you can't choose faster unless they deem your website needs it. It also shows you googlebot's activity on your site in the last 90 days.

Thank You

TE75 · 11-25-2006, 03:04 PM

The googlebot crawl frequency tag I think works like most of the google web products...it's merely a suggestion. Even if you have a site google will allow to be crawled more than once a day doesn't mean they actually will.