All 1 entries tagged Engines

No other Warwick Blogs use the tag Engines on entries | View entries tagged Engines at Technorati | There are no images tagged Engines on this blog

July 03, 2008

Yahoo slurp– waste of bandwidth

Looking at the stats for a couple of websites I administer I have noticed that Yahoo Slurp uses an unfeasibly large amount of bandwidth compared to all the other crawlers despite actually providing very few visitors.

In the worst case for one site last month (www.tinangelrecords.co.uk) slurp used 1000 times the bandwidth of google, yet produced only 1/20th of the number of visits, it was also responsible for almost 1/3 of the total site bandwidth. Fortunately at the moment bandwidth isn't a problem, however if it should become one then the first thing I will do is just block  slurp  as the number of lost visitors doesn't justify the effort required to work out how to simply limit the amount of robot traffic generated by it.

Robots/Spiders visitors (Top 25)   -   Full list   -   Last visit
12 different robots* Hits Bandwidth Last visit
Yahoo Slurp 1003 429.59 MB 30 Jun 2008 - 23:10
MSNBot 223 782.52 KB 30 Jun 2008 - 23:54
Unknown robot (identified by 'spider') 166 444.09 KB 30 Jun 2008 - 23:10
Googlebot 113 411.59 KB 30 Jun 2008 - 10:43
Ask 80 777.73 KB 30 Jun 2008 - 03:50
Unknown robot (identified by 'robot') 61 4.20 MB 30 Jun 2008 - 20:46
MSNBot-media 60 518.56 KB 30 Jun 2008 - 03:35
Unknown robot (identified by 'bot/' or 'bot-') 17 75.99 KB 28 Jun 2008 - 23:25
Alexa (IA Archiver) 11 79.67 KB 24 Jun 2008 - 13:52
Unknown robot (identified by 'crawl') 11 39.39 KB 30 Jun 2008 - 03:00
Voila 6 16.48 KB 26 Jun 2008 - 03:12
Google Sitemaps 1 263 Bytes 25 Jun 2008 - 08:54
Connect to site from
Origin Pages Percent Hits Percent
Direct address / Bookmarks 915 68.6 % 1044 64.5 %
Links from a NewsGroup
Links from an Internet Search Engine - Full list
- Google 216 217
- Google (Images) 35 61
- Yahoo! 13 13
- Windows Live 13 13
- MSN Search 2 2
- Tiscali 1 1
- AOL 1 1
- Ask 1 1
- Sogou 1 1
- Google (cache) 0 4
283 21.2 % 314 19.4





August 2014

Mo Tu We Th Fr Sa Su
Jul |  Today  |
            1 2 3
4 5 6 7 8 9 10
11 12 13 14 15 16 17
18 19 20 21 22 23 24
25 26 27 28 29 30 31

Search this blog

Tags

Galleries

Most recent comments

  • the email may have got lost? did you recieve a bounce back? by prestige car hire on this entry
  • When I'm supreme ruler of the Universe the use or display of Christmas themed items before December … by Mike Willis on this entry
  • It's all about taking a deep breath and just going with it. At least they're not playing carols yet.… by Mathew Mannion on this entry
  • I wouldnt have bought it. Advent is the period leading up to Christmas. Therefore I never buy anythi… by on this entry
  • Oh God no. :( by on this entry

Blog archive

Loading…
RSS2.0 Atom
Not signed in
Sign in

Powered by BlogBuilder
© MMXIV