Recently I had interesting experience with web applications that deserves a blog post. We decided to run broken link checker on huge ASP.NET site on staging server (Win 2003 & IIS6). This could also act as a small load test.

So far so good but I've noticed two main reports:

  • WebResource.axd with one specific parameter reported Error 404.0 - File not found
  • some requests to images (.gif) reported Error 403.1 - Forbidden

These errors are logged on IIS log. 

This is quite strange because when I try to run them in the browser manually everything is fine. The hardware seems not very loaded.

 

While investigating it further (and getting a clue from IIS.NET Forums) I got that IIS is actually trying to executed .gif files (See IIS Status codes). Probably same happens to WebResource.axd...

Then I went on link checker side and noted that it makes "HEAD" request (point 9.4) in order to reduce traffic.

 

I wrote this little tool to check my assumption.

1: using System;
2: using System.Net;
3: using System.Threading;
4:  
5: namespace UrlChecker {
6:     class Program {
7:         static string URL_TO_CHECK2 = "http://localhost/WebResource.axd?d=long-param-here";
8:         static string URL_TO_CHECK = "http://localhost/Photo/1116.gif";
9:         static int LOOPS_NUM = 10;
10:         static int THREADS_NUM = 10;
11:         static int REQUEST_TIMEOUT = 10000;
12:         static string REQUEST_METHOD = "HEAD"; // or "GET"
13:  
14:  
15:         static ManualResetEvent sync = null;
16:         static void Main(string[] args) {
17:             sync = new ManualResetEvent(false);
18:             Thread[] threads = new Thread[THREADS_NUM];
19:             for (int i = 0; i < THREADS_NUM; i++) {
20:                 threads[i] = new Thread(new ParameterizedThreadStart(CheckLink));
21:                 threads[i].Start(i);
22:             }
23:             sync.Set();
24:             Console.Write("Press 'Enter' to exit!");
25:             Console.ReadLine();
26:         }
27:  
28:         static void CheckLink(object ThreadId) {
29:             sync.WaitOne();
30:  
31:             for (int i = 0; i < LOOPS_NUM; i++) {
32:                 string statusCode;
33:                 try {
34:                     HttpWebRequest rq = (HttpWebRequest)WebRequest.Create(URL_TO_CHECK);
35:                     rq.Method = REQUEST_METHOD;
36:                     rq.Timeout = REQUEST_TIMEOUT;
37:                     HttpWebResponse resp = (HttpWebResponse)rq.GetResponse();
38:                     statusCode = resp.StatusCode.ToString();
39:                 } catch (WebException ex) {
40:                     statusCode = string.Format("Exception: {0}", ex.Message);
41:                 }
42:                 Console.WriteLine("{0} - '{1}'", ThreadId, statusCode);
43:             }
44:             Console.WriteLine("Thread {0} finished.", ThreadId);
45:         }
46:     }
47: }

And this helped me to confirm it..   Note on line 12 is set HEAD request and all requests fails with the messages shown above (404 and 403.1 accordingly). When GET request is used everything is OK. Luckily the link checker supported GET request on failed although I had some difficulties setting it.