Pro-SEO

We will have seen an increase in Slurp activity on our websites in the last few days/weeks as yahoo gives their new spider bot a run out along side the old one before completely retiring the old Slurp bot.

It has taken some time for yahoo to ensure their new bot functions as expected which has meant increased Yahoo spidering and an increase in Slurp activity.

David Simpson and Priyank Garg have been discussing the new Spider bot on the Yahoo! Blog

In addition to crawling the Internet faster, our new crawler is more efficient at visiting websites. As a result, site owners will notice as much as a 25% reduction in the number of requests and bandwidth consumed by the crawler.

Now, with the new crawler in full production, we have turned off the old crawler and site owners should see a much lower crawl load without a loss in content coverage.

With this change of behavior in the crawler, you may see some shuffling of the pages that are included in the index and some changes in ranking as well.

As with any changes search engines make to the way they spider/index websites we may start seeing some changes to SERPS in the yahoo index, So webmasters should start monitoring where they appear in the Yahoo search, Especially for search terms they previously ranked well for.

The expected efficiency of the new spider bot should be welcomed when webmasters start seeing a reduced bandwidth bill. Though the effect it has on indexing time and SERPS may put a dampener on those savings.

This entry was posted on Friday, July 28, 2006 at 7/28/2006. You can skip to the end and leave a response.