What is Fiddler?

Fiddler is an impressive HTTP/S debugging tool from Microsoft that can be used to simulate various user agents (i.e. browsers like Internet Explorer, FireFox, Safari, etc.) going against your web server without actually installing those browsers on the computer.

Now, to make it clear, this tool doesn’t display the UI for that particular user agent; so, you wouldn’t see how a particular page from your website looks and feels and behaves inside a particular browser. Fiddler also comes with extensions – which includes Web View Extension. This tab in the Fiddler shows you how the page would be rendered inside the Internet Explorer (see the difference: a particular browser vs. IE).

However, you can send a particular user agent string to your server and see what your server returns.

The issue and test scenario

At the minimum, your web server should respond without an error to all the legitimate user agent strings (major browsers, major search engine spiders, etc.) when a legitimate page is requested.

Use the following steps to test (details follow):

• Use the Request Builder to input a URL and a User Agent
• Check the Results pane for the result code
• Check the tabs in Session Inspector to see what came from the server
• Optionally, check the Performance Statistics tab to see how many bytes were sent and how many bytes were returned

The Request Builder

Using the Request Builder tab in Fiddler, enter your http query and the user agent you wish to test it for. The example values could be:

In the above case, we are testing how our web server (and the web framework ASP.NET 2.0) reacts to this particular user agent. In this case, the user agent is the Googlebot.

Figure 1: Fiddler Request Builder

You need a whole bunch of real-world user-agent strings (both browsers and search engines) for the above test. You can get them from your web server logs. You can also get some popular user-agent strings from the following article:

Important User Agent Strings

What results should we expect?

If everything is working fine, you will see the HTTP OK result code 200. If the result code is 500, you should worry about it – basically the server is erroring out for that particular user agent string. Or, the error could be for a particular page, where you will see the error code for all the user agents.

Figure 2: Results from the HTTP Request

There were some issues with the pages redirected with RewritePath on ASP.NET 2.0. The specific solution for that problem is discussed in the following article:

Search Engine Issues with RewritePath

If you have widgets, etc. on your computer that are making HTTP calls (in the figure above requests are being made to msn.com), the Fiddler will continue to capture that traffic as well. In general, you don’t to have to worry about them. If these entries bother you, you can simply select them and delete them (Delete is on the right-click menu).

Inspecting the data that came back

You can use the Session Inspector tab to see what the server sent to this particular request from this particular user agent as shown in the figure below.

Figure 3: Inspecting the returned data

The bottom portion of the Session Inspector has several tabs to facilitate viewing the returned data in different ways. Some notes on this:

Raw Tab

Here you can see all the data returned in its raw form, including the HTTP headers. This is where you realize ASP.NET (or whatever the framework you are using) does a whole lot for you. For example:

• The __VIEWSTATE hidden input contains a gigantic string to facilitate the state management.
• Various scripts written by you and generated by the framework
• The HTML generated by the framework for the menus, master pages, etc.
• The HTML for your page, of course

The tabs here are self-explanatory. Two tabs – Syntax View and Web View --come from adding extensions, which are discussed below.

In addition, there is a Performance Statistics tab that you can use in conjunction with the Session Inspector tab to see if you can cut down on the number bytes you are sending for this page.

The Extensions

In addition to the basic Fiddler, you can also install extensions. Two extensions I found useful for Fiddler are Syntax View and Web View. You can download these extensions from the Fiddler page.

Syntax View

As the name suggests, the code is colorized depending on the syntax. Makes it nice to read what came back from the server.

Figure 4: Nicely formatted code with Fiddler’s Syntax View

Web View

After installing the WebView extension, you see how the downloaded page would look like when drawn with the IE rendering engine.

Figure 5. The web page UI from the WebView tab

Where to get Fiddler?

If you haven’t already downloaded Fiddler, you can get the latest version of Fiddler here:

http://www.fiddler2.com/Fiddler2/