Home | About Me | Photos | Writing | Research | Scratchpad | Projects

Web robot statistics

Here is a list of web robots that have crawled my pages.

Robot name Hits
Googlebot/2.1 (+http://www.googlebot.com/bot.html) 24460
msnbot/0.11 (+http://search.msn.com/msnbot.htm) 2935
Mozilla/2.0 (compatible; Ask Jeeves/Teoma) 2526
HenryTheMiragoRobot (http://www.miragorobot.com/scripts/mrinfo.asp) 1723
ia_archiver 1013
Mozilla/4.0 compatible ZyBorg/1.0 Dead Link Checker (; http://www.WISEnutbot.com) 865
Mozilla/4.0 (compatible;) 783
NaverBot-1.0 (NHN Corp. / +82-2-3011-1954 / ) 691
Mozilla/3.01 (compatible;) 672
Pompos/1.3 http://dir.com/pompos.html 650
LinkWalker 582
larbin_2.6.3 381
Mozilla/3.0 (compatible) 298
Mozilla/4.0 compatible ZyBorg/1.0 (; http://www.WISEnutbot.com) 288
W3C_Validator/1.305.2.12 libwww-perl/5.64 277
Mozilla/4.0 (compatible; Netcraft Web Server Survey) 252
Mozilla/4.0 (compatible; Netcraft Web Server Survey) 252
Mozilla/5.0 (Slurp/cat; ; http://www.inktomi.com/slurp.html) 240
IQSearch 215
Baiduspider+(+http://www.baidu.com/search/spider.htm) 188
Tutorial Crawler 1.4 (http://www.tutorgig.com/crawler) 166
Mozilla/4.0 134
psbot/0.1 (+http://www.picsearch.com/bot.html) 130
appie 1.1 (www.walhello.com) 123
Gigabot/1.0 116
Zao/0.2 (http://www.kototoi.org/zao/) 105
http://www.almaden.ibm.com/cs/crawler [wf134] 99
Mozilla/5.0 96
http://www.almaden.ibm.com/cs/crawler [c01] 95
Mozilla/3.0 (compatible; Indy Library) 94
FAST-WebCrawler/3.8 (crawler at trd dot overture dot com; http://www.alltheweb.com/help/webmaster/crawler) 87
CrawlConvera0.1 76
Plucker/Py-1.4 76
Googlebot/Test (+http://www.googlebot.com/bot.html) 68
NutchCVS/0.03-dev (Nutch; http://www.nutch.org/docs/en/bot.html; ) 63
NutchCVS/0.03-dev (Nutch; http://www.nutch.org/docs/en/bot.html; ) 63
egothor/3.0a (+http://www.xdefine.org/robot.html) 62
http://www.almaden.ibm.com/cs/crawler [wf85] 61
Java/1.4.1_05 59
MSIE 6.0 57
MSIE 6.0 57
Mediapartners-Google/2.1 (+http://www.googlebot.com/bot.html) 51
NG/2.0 51
Scooter/3.3_SF 49
QuepasaCreep ( ) 47
Program Shareware 1.0.0 47
Mozilla/4.0 (compatible; Arachmo) 46
http://www.almaden.ibm.com/cs/crawler [wf163] 43
Mozilla/4.0 (compatible; BorderManager 3.0) 39
Scooter/3.3.vscooter 37
Java/1.4.2_03 36
iSiloX/3.35 Windows/32 28
webcollage/1.107 26
webcollage/1.102 26
NPBot (http://www.nameprotect.com/botinfo.html) 23
Teleport Pro/1.28 22
AnswerBus (http://www.answerbus.com/) 21
webcollage/1.103 21
webcollage/1.103 21
w3m/0.4.1 20
TranSGeniKBot http://www.tsgk.net 19
Cowbot-0.1.1 (NHN Corp. / +82-2-3011-1954 / ) 18
DA 5.3 18
Java/1.4.2_01 18
Microsoft URL Control - 6.00.8169 18
http://www.sygol.com 18
EasyDL/3.04 http://keywen.com/Encyclopedia/Bot 16
iSiloX/4.0 Windows/32 15
SIE-C55/10 UP.Browser/5.0.3.3 (GUI) (Google WAP Proxy/1.0) 14
WebCopier v4.0 12
larbin_2.6.3 11
Oracle Ultra Search 11
libwww-perl/5.53 10
RPT-HTTPClient/0.3-3E 10
sherlock_spider 9
QuepasaCreep v0.9.14 8
fmII URL validator/1.0 7
Generic 7
Mozilla/4.0 (compatible; Vagabondo/2.2; webcrawler at wise-guys dot nl; http://webagent.wise-guys.nl/) 7
Dumbot(version 0.1 beta - dumbfind.com) 7
HTTP agent 6
PlantyNet_WebRobot_V1.9 6
libwww-perl/5.64 6
Bobby/4.0.1 RPT-HTTPClient/0.3-3E 6
holy_browser 6
Stumbler v2.0 6
Download Ninja 2.0 6
MovableType/2.64 5
Tcl http client package 2.3 5
Java/1.4.1 5
MSIE5.5 5
Wotbox/alpha0.5.1 (; http://www.wotbox.com) Java/1.4.1_02 4
Nutscrape/1.0 (CP/M; 8-bit) 4
Schmozilla/v9.14 Platinum 4
Under the Rainbow 2.2 4
Mozilla/2.0 (compatible; MS FrontPage 5.0) 3
WebCopier v3.6 3
Popdexter/1.0 (http://www.popdex.com/) 3
ASPseek/1.2.10 3
DA 5.0 3
WebCopier v3.5 3
WebCopier v3.4 3
Mozilla(IE Compatible) 3
DA 7.0 3
Thames Research of Manchester 3
Butch__2.1.1 3
Mozilla/4.61 [de]C-CCK-MCD (OS/2; U) 2
WebZIP/5.0 (http://www.spidersoft.com) 2
Mozilla/4.0 Compatible 2
larbin_2.6.3 2
RealDownload/4.0.0.40 2
Space Bison/0.02 [fu] (Win67; X; SK) 2
Mozilla/4.0 (compatible) 2
Nokia6600/1.0 (3.42.1) SymbianOS/7.0s Series60/2.0 Profile/MIDP-2.0 Configuration/CLDC-1.0 (Google WAP Proxy/1.0) 2
Web Downloader/5.8 2
webcollage/1.104 2
Cisco ACNS Content Acquirer 2
SpaceBison/0.01 [fu] (Win67; X; ShonenKnife) 2
MovableType/2.63 1
Mozilla/5.0 (compatible; Vagabondo/2.1; webcrawler at wise-guys dot nl; http://webagent.wise-guys.nl/) 1
CerberianDrtrs/Version-3.0-Build-37 1
FlickBot 2.0 RPT-HTTPClient/0.3-3 1
WWWOFFLE/2.7 1
Mozilla/5.0 () Reese 1
Fetcher/0.95 1
Mozilla/4.0 (compatible; Cerberian Drtrs Version-3.0-Build-40) 1
HericomBot 1.0 1
JavaBee 1
BravoBrian SpiderEngine MarcoPolo 1
HLoader 1
Mozilla/4.0 (compatible; MSIE 4.00;) 1
WebZIP/4.21 (http://www.spidersoft.com) 1
linko/0.1 libwww-perl/5.65 1
larbin_2.6.3 1
Wolfenstein 3D for AmigaOS4.0/G4-1GHz (True AMIGA(one)) 1
Mozilla/4.0 (Compatible; BNSpider Beta 1.0) 1
fhkdfqhsrtrrxffanssTcTgfgfck 1
Mozilla/2.0 (compatible; Elaine/3.0) 1
Mozilla/2.0 (compatible; Elaine/3.0) 1
WebDownloader for X 2.4.1 1
EARTHCOM.info/1.3test [www.earthcom.info] 1
JPluck/2.1.2 (Java 1.4.2_02; Windows XP) 1
n4p_bot 1
falcon 1
lwp-request/2.06 1
htdig/3.1.5 () 1
PDPscape/1.1 (Multics; 18-bit) 1
NOYDB 1
Hi There! 1
xnmn5muffjjgg5lunsvmclvowcfrw5uo jxdn 1
qlhmhPsel Pbgl3huh w 1
IlTrovatore-Setaccio/1.2 (+http://www.iltrovatore.it/aiuto/faq.html) 1
Microsoft URL Control - 6.00.8877 1
DataAgent 1
Yahoo-VerticalCrawler-FormerWebCrawler/3.9 crawler at trd dot overture dot com; http://www.alltheweb.com/help/webmaster/crawler 1
k6txjhfgo vcgfhcqdmScfh6uSeirkhf6ir 1
HTMLDOC v1.8.23 1
mynftNkptuk9pnssuffwmrj 1
Sundoh Search 9.0 1
aaorbwaowhpsamjohchqrjmiiuhbdwvw 1
DoCoMo/1.0/P504iS/c10/TB 1
Mozilla/3.0 (compatible; Annots 1.0) 1
rywcyniqxavcsoofqwmb 1
Netscape 1
DoCoMo/1.0/D503iS/c10 1
Jakarta Commons-HttpClient/2.0final 1
OMS (OpenMetaSeek) - http://openseek.floschd.org 1
BlogBot/1.2 1
iexplore.exe 1
NCSA Beta 1 (http://vias.ncsa.uiuc.edu/viasarchivinginformation.html) 1
Indexer1035 1
sitescooper/3.1.3 (http://sitescooper.org) libwww-perl/5.64 1
Jigsaw/2.2.0 W3C_CSS_Validator_JFouffa/2.0 0