Finding out the user-agents that are accessing your web server is really not a big deal. Just look in the web server logs, you will find the user-agent strings along with all the other gory details. You can simply export these log files to Microsoft Excel and do a sort on the User-Agent column, you will see all the different browsers and search spiders that came to your web server.
However, typically, web server archives each day’s log into a separate file. To get a complete idea over a period of several days, you have to mess with a whole different set of files. There are plenty of other options, of course, to find out this information -- there are tons of log analyzers and analytics tools that can be used.
Saving User-Agent in Database
Following is a quick method that stores the unique User-Agent strings in a database table. Steps to do this simple logging:
- Create a table, say, UserAgent in your database. Add just one column, say, UserAgent, to that table. Make it a primary key. In SQL Server, size of the primary key can not be more than 900 chars, so set this column length to, say, 500 chars.
- Call the LogUserAgent method from your entry page, say, default.aspx. You can keep the call to that method for a week or a month (as needed). As the time goes, you will see only the unique user-agent strings in that table.
/// <summary>
/// Logs the User Agent.
/// </summary>
private void LogUserAgent()
{
// Get the connection string from web.config
string connString = ConfigurationManager.ConnectionStrings["MyDB"].ToString();
// Make up an INSERT string; Escape the single quotes.
Regex rgxRep = new Regex("’");
string userAgent = rgxRep.Replace(Request.UserAgent, @"’’");
string insertString = "INSERT INTO UserAgent(UserAgent) VALUES (’" + userAgent + "’)";
// Open a database connection
using (SqlConnection sqlConn = new SqlConnection(connString))
{
sqlConn.Open();
// Issue the insert command. Ignore the failure (happens from
// trying to insert a duplicate User-Agent string.
using (SqlCommand sqlCmd = new SqlCommand(insertString, sqlConn))
{
try { sqlCmd.ExecuteNonQuery(); }
catch { }
}
}
}
Examples of User-Agent Entries
You will see entries from browsers and search spiders. Some examples:
Browsers (IE, FireFox, Safari)
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; sbcydsl 3.12; YComp 5.0.0.0; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.0.04506; .NET CLR 1.1.4322; Tablet PC 2.0)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; InfoPath.1)
Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4
Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en) AppleWebKit/419 (KHTML, like Gecko) Safari/419.3
Search Spiders (e.g. Google)
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mediapartners-Google/2.1
