How do they do that?

[Angel69]

I've been meaning to ask this for some time now.

I look at my log files and check to see what pages the 3 main spiders look at. I understand when they do a full scan or when you can see them drilling down from the root or other page.

Most often it is pages here and pages there sort of at random.

Does anyone have an idea of how they decide what pages they want to see?

I know that Google does not know what you click after a search. Did a packet sniff.
Maybe all the URLs of the first page?
Maybe from the Googlebar (if installed) after a search?