Do Google's bots crawl proxy servers?

livePR.ezer.com · #1 (**permalink**) 01-22-2009, 02:50 AM

Do Google's bots crawl our sites directly or G bots have proxy setting?

Really like to know.

Thanks

Regards,

livePR.ezer.com · #2 (**permalink**) 01-24-2009, 08:35 AM

Ok, try another way to say
For example US AT&T may have big data servers to catch caches as a proxy for its customers.
The question is : will Google sometimes crawl your site pages from AT&T server instead of crawl your site pages directly?

Thanks,

Regards,

**juliensimon** · #3 (**permalink**) 01-24-2009, 11:26 AM

I believe it's from your site directly.

yenerich · #4 (**permalink**) 01-25-2009, 07:11 AM

No, they wont crawl the content of proxy server pages showing 3rd party pages. Or at least will consider it duplicates.

jhoanna · #5 (**permalink**) 01-26-2009, 02:02 PM

I'm not so sure about the answer on your question, but I hope this one does it: http://www.mattcutts.com/blog/crawl-caching-proxy/

livePR.ezer.com · #6 (**permalink**) 01-31-2009, 04:26 AM

Quote:

Originally Posted by jhoanna

I'm not so sure about the answer on your question, but I hope this one does it: http://www.mattcutts.com/blog/crawl-caching-proxy/

nice info you linked

................
IMO
not only ISPs have proxy servers I think Google has its own proxy servers too
that is how they can compare web sites very fast