What is User-Agent?
When a user accesses a web site (which is hosted by a web server) via a browser, that browser sends its own ID along with the other information in the headers. This ID is shown in the User-Agent string in the Headers. In Asp.Net, you can see this information in the following variable (in the Headers member of the Request object associated with the Page):
Page.Request.Headers
For example, if you are going to a web server with Internet Explorer 7.0 on Windows Vista, you will see the following string:
Listing 1. Example User-Agent from ASP.NET
User-Agent=Mozilla%2f4.0+(compatible%3b+MSIE+7.0%3b+ Windows+NT+6.0%3b+sbcydsl+3.12%3b+YComp+5.0.0.0%3b+SLCC1%3b+ .NET+CLR+2.0.50727%3b+Media+Center+PC+5.0%3b+.NET+CLR+3.0.04506%3b+ .NET+CLR+1.1.4322%3b+Tablet+PC+2.0)
The same thing can be see more prettied-up from the UserAgent member of the Request object on a given Page:
Page.Request.UserAgent
The actual prettied-up version for the above IE 7.0 browser is:
Listing 2. Prettied up example User-Agent from ASP.NET
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; sbcydsl 3.12; YComp 5.0.0.0; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.0.04506; .NET CLR 1.1.4322; Tablet PC 2.0)
Following are some important User-Agent strings that can be used in your testing with tools like Fiddler. For example, you can place these user-agent strings in the request for a particular page on your web server and see how the web server reacts (i.e. does it fail for some reason or other; what type of content would it send back, etc.)
These browsers collectively cover more than 98% of all the browser-based web server access (numbers are of April, 2007). In the section below, you will find User-Agent strings for search engines.
Browser User-Agent Strings
The following are real User-Agent strings for various browsers. However, some individual pieces (like .Net CLR version, date of release, etc.) would be different on different computers and installations. Several examples are shown here to facilitate any testing (for example, from a tool like Fiddler).
Internet Explorer (~78% market share)
Listing 3. Internet Explorer User-Agent Strings
IE 7.x
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; sbcydsl 3.12; YComp 5.0.0.0; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.0.04506; .NET CLR 1.1.4322; Tablet PC 2.0)
IE 6.x
Mozilla/4.0 (compatible; MSIE 6.01; Windows NT 5.0)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; InfoPath.1)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; Win64; x64; SV1)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312461)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312462)
IE on Windows Mobile (e.g. Palm Treo 750)
Palm750/v0000 Mozilla/4.0 (compatible; MSIE 4.01; Windows CE; PPC; 240x320) UP.Link/6.3.1.17.0
FireFox (~15% market share)
Listing 4. FireFox User-Agent Strings
FireFox 2.x (on Windows Vista)
Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3
Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4
FireFox 1.5.x (on Windows XP)
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4
FireFox 1.x (on Linux)
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.12) Gecko/20050922 Fedora/1.0.7-1.1.fc4 Firefox/1.0.7
Safari (~5% market share)
Listing 5. Safari User-Agent Strings
Safari 2.x
Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en) AppleWebKit/419 (KHTML, like Gecko) Safari/419.3
Mozilla/5.0 (Macintosh; U; Intel Mac OS X; de-de) AppleWebKit/523.10.3 (KHTML, like Gecko) Version/3.0.4 Safari/523.10
Safari on Windows
Mozilla/5.0 (Windows; U; Windows NT 6.0; en) AppleWebKit/522.11.3 (KHTML, like Gecko) Version/3.0 Safari/522.11.3
iPhone
Mozilla/5.0 (iPhone; U; CPU like Mac OS X; en) AppleWebKit/420+ (KHTML, like Gecko) Version/3.0 Mobile/1A543a Safari/419.3
Opera
Listing 6. Opera User-Agent Strings
Opera 9.x
Opera/9.21 (Windows NT 6.0; U; en)
Konqueror
Listing 7. Konqueror User-Agent Strings
Mozilla/5.0 (compatible; Konqueror/3.0-rc2; i686 Linux; 20020506)
Mozilla/5.0 (compatible; Konqueror/3.1-rc6; i686 Linux; 20020701)
Mozilla/5.0 (compatible; Konqueror/3.1; i686 Linux; 20020922)
Following browsers have less than 1% market share (as of April, 2007): Netscape, Opera, General Mozilla, Konqueror.
Search Engine User-Agent Strings
Google
Listing 7. Google User-Agent Strings
Googlebot
Mozilla/5.0 (compatible; Googlebot/2.1; http://www.google.com/bot.html)
AdSense User-Agent
Mediapartners-Google
Mediapartners-Google/2.1
Yahoo!
Listing 8. Yahoo! User-Agent Strings
Slurp (Yahoo! Spider)
Yahoo! Slurp/Site Explorer
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
MSN
Listing 9. MSN User-Agent Strings
H2. MSNbot
msnbot/1.0 ( http://search.msn.com/msnbot.htm)
Ask (Jeeves)
Listing 10. Ask User-Agent Strings
Teoma
Mozilla/2.0 (compatible; Ask Jeeves/Teoma; http://sp.ask.com/docs/about/tech_crawling.html)
Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://about.ask.com/en/docs/about/webmasters.shtml)
Lynx
Listing 11. Lynx User-Agent Strings
Lynx/2.8.4rel.1 libwww-FM/2.14 SSL-MM/1.4.1 OpenSSL/0.9.6b
Miscellaneous
Several miscellaneous search spiders visit the sites on the web. There are also search spiders without a user-agent name -- they simply have an IP Address (these spiders should probably be seen with a little bit of suspicion). Not all of this are bad user agents, some of them are simply not well-known.
Listing 12. MiscellaneousUser-Agent Strings
Whois
SurveyBot/2.3 (Whois Source)
Pingdom
Pingdom GIGRIB (http://www.pingdom.com)
bot (?)
bot/1.0 (bot; http://; bot@bot.bot)
Panscient
panscient.com
boitho.com-dc/0.82 ( http://www.boitho.com/dcbot.html )
boitho.com-dc/0.85 ( http://www.boitho.com/dcbot.html )
boitho.com-dc/0.86 ( http://www.boitho.com/dcbot.html )
curl/7.15.4 (i386-portbld-freebsd6.0) libcurl/7.15.4 OpenSSL/0.9.7e zlib/1.2.2 c-ares/1.3.1
ia_archiver
Java/1.6.0_02
Missigua Locator 1.9
Mozilla/5.0 (MrCarlito-0.1 http://www.mrcarlito.com/spider.html)
Nokia6680/1.0 ((4.04.07) SymbianOS/8.0 Series60/2.6 Profile/MIDP-2.0 Configuration/CLDC-1.1 (for mobile crawler) )
WWW-Mechanize/1.12