Uncovering The Mysteries Of The Goolgle Crawl – Part One

by Payo Hernandez | 28th October 2009

To many online business owners, the way Google conducts its indexing of the web through spiders and robots is quite a mysterious subject.

While it’s certainly true that there’s a lot of science and math behind Google algorithms all you actually need is to understand are the core basics in order to make well judged business decisions regarding Google and your website.

Firstly, Google does 2 main types of crawls. We have a ‘deep crawl’ which is done once a month on average. This is where Google goes through websites in their entirety indexing pages. Now this takes a long time even for Google to get through so they cannot simply rely on this type of crawl. The internet changes so quickly that Google would find itself quickly out of date.

Performing a deep crawl takes up to one week for Google to complete but of course it must index all of that information which takes longer again. By the time the full process is complete it can take 6 weeks or more for a new site to be found and indexed if it’s lucky!

For this reason we also have what’s known ‘fresh crawls’ which rather than going deeply into websites it skims the surface quickly picking up new recently added information and updates.

These fresh crawls are automated robots which follow links from site to site and are running more frequently than deep crawls, in many ways they are responsible for keeping Google current with what’s on the web.

For this reason it’s widely known that you are much more likely to get indexed by Google by having some new backlinks to your site than by simply submitting your URL and relying on Google’s deep crawl to find you.

Realistically it could take months for a new site and domain appear on Google’s database, something which could be achieve in a single day with backlinks. The exact frequency of fresh crawls are secret however by staying alert it is possible to notice when they have occurred.

Looking to find the best deals on fishing rod cases, then visit www.fishingrodcases.org to find the best advice on building your own fishing rod case for you.

Related Blogs

Leave a Reply

Name (Required)

Email (Required - will not be published)


Message (Required)