Andornot Consulting
Friday, November 20, 2009 10:21 AM

IE6 gzip bug solved using IIS7's URL Rewrite Module

by Ted Jardine

“IE6, that shambling wreck that clings with nightmare strength to its living death.”
Peter Tyrrell, Literary genius nonpareil

When Internet Explorer 6 was first released way back in 2001 it quickly became the predominant browser. It was much better than everything else out there. But now NINE years later, working with IE6 is truly one of the most frustrating things a Web developer has to deal with. In 2009, it has countless issues: problems with transparent PNGs, flakey caching, esoteric rendering, lack of support for modern Web standards, and the list goes on and on.

robot-jonny-ie6
Image courtesy of RobotJonny

This post is about yet another IE6 hair-puller: gzip compression. Compressing HTML, CSS, javascript, etc. allows content to be scrunched up so that it’s served up way faster while reducing bandwidth costs. It’s a great way to increase the performance of your site…except when IE6 is involved that is: IE6 does not correctly handle data that has been compressed using the gzip algorithm. This would be frustrating to deal with, but nevertheless acceptable, except for the fact that IE6 problems with compressed content only pop up sometimes, on certain machines, and intermittently. There doesn’t seem to be any rhyme or reason to it. The worst kind of bugs are the ones you cannot faithfully replicate.

With IIS6 we use Port80 Software’s excellent httpZip module to compress site content (compression is available natively in IIS6, but it’s a pain to administrate, and doesn’t work well with older browsers). One of httpZip’s big selling features is that it accurately deals with browser idiosyncrasies; therefore, until now we haven’t had to worry about IE6 and compression.

However, we have recently started transitioning more sites from being hosted on Windows Server 2003 (with IIS6) over to Windows Server 2008 (IIS7) and Windows Server 2008 R2 (IIS7.5).  With IIS7, gzip compression can be easily and correctly implemented right out of the box. But we’ve discovered that in certain scenarios with static and dynamic compression enabled, IIS7 doesn’t safely handle IE6’s idiot tendencies concerning gzip compression. The immediate solution is to turn off dynamic compression altogether, and this seems to solve the problem, but that doesn’t sit right with me: do yet another regression for IE6 users that punishes non-IE6 users.

Therefore, on to saving the world from IE6, one workaround at a time! Seb Duggan presents a solution involving ISAPI_Rewrite which essentially sniffs out whether or not IE 5 or 6 is doing the request and turns off compression for CSS and JS files if that is the case. Sounds good (although he does a sniff for pre-SP2 IE6 only, which I believe is erroneous). However, we already use IIS7’s URL Rewrite Module and I would prefer to avoid bringing another cook into the kitchen. There’s a couple of obstacles to getting this working however:

  1. The current version of the URL Rewrite Module v1.1 does not support rewriting server variables.
  2. The IIS7 URL Rewrite Module has very different syntax.

So how did I do it?

  • Download and install the URL Rewrite Module 2.0 RC (Release Candidate) or wait until the RTM is released (don’t know when). This version supports rewriting request server variables so we can fake IIS7 into thinking the requesting browser does not support gzip encoding (because for all intents and purposes, it DOESN’T – at least for CSS and javascript files that is).
  • Brute force hacking and documentation reading resulted in the following (the handy “Import Rules”, which can be used to translate Apache mod_rewrite rules, wasn’t up to the task). The important thing to remember is that each dash is replaced with an underscore and each server variable is made all caps and prefixed with “HTTP_”. Oh, and looking at the RFC specs, “0” is valid for setting the encoding to nothing (“.*” didn’t work for some reason).
    <system.webServer>
        …
    <rewrite>
            <rules>
                <rule name="IE56 Do not gzip js and css" stopProcessing="false">
                    <match url="\.(css|js)" /> <!— Match all .css and .js requests -->
                    <conditions>
                        <add input="{HTTP_USER_AGENT}" pattern="MSIE\ [56]" />
    <!— Where the User Agent includes MSIE 5 or MSIE 6 -->
                    </conditions>
                    <action type="None" />
    <!— Don’t do any redirects, rewrites, etc. -->
                    <serverVariables>
                        <set name="HTTP_ACCEPT_ENCODING" value="0" />
    <!—Make it so the request’s Accept-Encoding variable is
    set to nothing instead of gzip,decompress -->
                    </serverVariables>
                </rule>
            </rules>
        </rewrite>

    </system.webServer>
  • Explicitly allow the Accept-Encoding server variable to be overwritten by specifying the following in the applicationHost.config file (required for security reasons). If you don’t do this part, you’ll get an "HTTP/1.1 500 URL Rewrite Module Error".

    <rewrite>
        <allowedServerVariables>
            <add name="HTTP_ACCEPT_ENCODING" />
        </allowedServerVariables>
         …
    </rewrite>
  • You can implement the above on a site-by-site basis or server-wide (rules in site’s Web.config or location specific in applicationHost.config or site-wide in applicationHost.config).
  • ie6-gzip
    Fiddler showing IE6 with compression off for .js and .css files (while still compressing all other applicable files such as .htm).

    ie8-gzip
    Fiddler showing IE8 with compression on for all applicable resource types.

    The usual caveats apply to implementing the above: we don’t guarantee any of it. If it kills your cat, I know nothing. If you lose your job because of it, it’s your fault, not mine. On the other hand, if it helps you out or if you figure out a refined implementation, please let us know.

Friday, October 02, 2009 3:21 PM

Handling 404 errors with ASP.NET

by Ted Jardine

As mentioned at the end of my previous post on handling errors with ASP.NET, handling "404 Not Found" errors are particularly problematic (if you haven't read it yet, please do so). And looking around, the vast majority of information out there on it is not complete, misinformed, or flat-out wrong (but I greatly appreciate all efforts!). And I would argue that this is because ASP.NET implementation of 404 error handling is flat-out-wrong. So with my super hero cape on, here I come to wobbly save the day!

The typical ASP.NET way to handle 404 errors is to put something like the following in your Web.config:

<customErrors mode="On" defaultRedirect="~/error.aspx">
    <error statusCode="404" redirect="~/page-not-found.aspx" />
</customErrors>

Make a page-not-found.aspx page and voila! Ya got 'er dun! If you're a little more on the ball, you'll realise that while this configuration works for end users (gives them a pretty page to look at hopefully clearly explaining that you can't find what they're looking for), it's bad for SEO (search engine optimization) because it sends back a 302 temporary redirect to your 404 page which in turn sends back a "200 OK" message. In other words, "Yeehah! No problems here as you've found what you're looking for! Index away!"

So, bright developer that you are, you add in some applicable status code into your 404 page thinking that should take care of it:

protected override void OnLoad(System.EventArgs e)
{
    Response.TrySkipIisCustomErrors = true; // For IIS 7 Integrated Pipeline - see previous post
    Response.Status = "404 Not Found";
    Response.StatusCode = 404;
    base.OnLoad(e);
}

Well, fire up Fiddler and you'll discover that you're still getting a 302 temporary redirect to your 404 page. So you fire up your error handling code and for 404s, you Server.Transfer to your 404 page just like all your other error transfers take place! But no, bafflingly enough, even running through a debug session to ensure you're properly catching your 404, ASP.NET still insists on 302'ing your precious response (although at least now your 404 page is sending back the proper "404 Not Found" error status). So go out there and google everywhere and try every suggestion (just a sampling) and then breathe a prayer of thanksgiving for me and my super-duper super hero cape, because this is how you make your way to the 404 handling bliss (caveat: thoroughly tested only on IIS 7):

First, make something like the following in Application_Error in your global.asax (or better yet, a custom error module):

protected void Application_Error(Object sender, EventArgs e)
{
    Exception exception = Server.GetLastError();
    if (exception is HttpUnhandledException)
    {
        if (exception.InnerException == null)
        {
            Server.Transfer(ERROR_PAGE_LOCATION, false);
            return;
        }
        exception = exception.InnerException;
    }

    if (exception is HttpException)
    {
        if (((HttpException)exception).GetHttpCode() == 404)
        {

            // Log if wished.
            Server.ClearError();
            Server.Transfer(NOT_FOUND_PAGE_LOCATION, false);
            return;
        }
    }

    if (Context != null && Context.IsCustomErrorEnabled)
        Server.Transfer(ERROR_PAGE_LOCATION, false);
    else
        Log.Error("Unhandled Exception trapped in Global.asax", exception);
}

Second, put something like the following in your 404 page (or even better, add it to your above custom error module):

protected override void OnLoad(System.EventArgs e)
{
    Response.TrySkipIisCustomErrors = true; // For IIS 7 Integrated Pipeline - see previous post
    Response.Status = "404 Not Found";
    Response.StatusCode = 404;
    base.OnLoad(e);
}

And lastly, put whatever you want in your customErrors section in your Web.config for 404 - remove the 404 references if you want. The only time it would get used is if you keep the confusion level down by reading that section in your error module to find out what your 404 page is.

And now, you don't get any nasty 302 redirects. You get a blissfully pure and pretty 404.

Note: you can have everything in place as above, but if you forget to Server.ClearErrror(), it'll all be for naught as you'll still get 302 redirected (thanks to http://stackoverflow.com/questions/667053/best-way-to-implement-a-404-in-asp-net for finally helping me get this one nailed).

UPDATE: If for some reason you cannot change your code, a good option to pursue is flipping a switch in your applicationHost.config file that passes your response through without IIS hijacking it. This is the sledgehammer approach, but could be applicable for your situation. For example, in order to allow Umbraco's alternative status code responses to work, this is the only way to get them to avoid being hijacked by IIS (i.e. NotFoundHandlers, booting screens, etc.).

<location path="Site Description">
    <system.webServer>
        <httpErrors existingResponse="PassThrough" />
    </system.webServer>
</location>

Thanks to Fabian Heussser's comment on Rick Strahl's post that helped with this.

Friday, October 02, 2009 2:36 PM

Errors: Sending the Right Message

by Ted Jardine

I've encountered many twists and turns in my journey towards handling application errors with aplomb. You have to make sure:

  1. Users are happy (by providing informative error pages) - well, as happy as they can be under the circumstances.
  2. Developers who need to fix the problem are happy (extensive logging and notification to help re-enact, locate, and then fix the bug so it doesn't happen again).
  3. Search bots are happy (by sending them the right message).

For example, when a user requests a resource that can't be found you want to help the user out with at least a pretty page that explains what has happened. Most users don't have a clue what a "404 error" is and could care less. And almost just as important, when a search bot encounters an error it needs to receive a proper error code back or else it gets all uppity on you and knocks down the page's ranking. In other words, when a page can't be found:

  1. Redirect to a helpful "Sorry, we can't find the resource you're looking for" page that hopefully looks somewhat decent and gives the user some options on how to proceed (perhaps like our "smart" page not found functionality on our soon-to-be-launched www.andornot.com).  
  2. Log the 404 so that it can be dealt with (if applicable).
  3. Set the proper 404 error code for SEO purposes.

However, sometimes getting the three above objectives dealt with in one fell swoop is problematic. For example, if you just redirect after having an error occur (say a "500 Internal Server Error") to a pretty page, users will be happy, but search bots will receive something like the following:

Cache-Control: private
Content-Type: text/html; charset=utf-8
Content-Encoding: gzip
Vary: Accept-Encoding
Server: Microsoft-IIS/7.5
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET
Date: Fri, 02 Oct 2009 18:30:47 GMT
Content-Length: 7719

200 OK

200 OK for something that really is not okay. It should be "500 Internal Server Error"! Erk. So how about we code in something like:

Response.StatusCode = 500;
Response.StatusDescription = "500 Internal Server Error";

But then things start to get interesting. In IIS 6 all appears to look good, but when you're using MasterPages, Internet Explorer 6 (in a rare case of actually being helpful) shows that what's being sent back is wacked as it wraps the custom error page with the default IIS error page. Gee, now two out of three of the above objectives are out the window. Suffice it to say, the result is grotesque, and I can't imagine what search bots think of it!

Moving on to IIS 7 and its Integrated Pipeline (doesn't apply to Classic Pipeline mode), things get even more interesting as it ignores your custom stuff altogether and sends back the applicable default IIS error page. What the? Rick Strahl has an excellent post addressing this issue. Essentially, the error is still trapped by ASP.NET (so it's logged, redirected, and whatever else you ask it to do), but then it is ultimately handled by IIS which sees the error code and hijacks the entire response and returns its ugly default error page (and who designed those anyway? Even IIS 6's were better).

So where do we go from here? In IIS 6, the problem only occurs when using MasterPages in your error page. Over simplifying, the page's MasterPage renders later on in the page lifecycle and resets the error code to 200 OK. And it just gets messier from there (see http://stackoverflow.com/questions/347281/asp-net-custom-404-returning-200-ok-instead-of-404-not-found for more info). So reset it back again further along in the page's lifecycle!

protected override void Render(HtmlTextWriter writer)
{
    base.Render(writer);
    Response.StatusCode = 500;
    Response.StatusDescription = "500 Internal Server Error";
}

But what about IIS 7? If you look further along in Rick Strahl's post mentioned above, there's a new flag in ASP.NET 3.5 called Response.TrySkipIisCustomErrors. As Rick says, "In a nutshell this property when set to true at any point in the request prevents IIS from injecting its custom error pages. This flag is new in ASP.NET 3.5 - so if you're running 2.0 only you're out of luck."

    Response.TrySkipIisCustomErrors = true;
    Response.StatusCode = 500;
    Response.StatusDescription = "500 Internal Server Error";

And Shazam!

HTTP/1.1 500 500 Internal Server Error
Cache-Control: private
Content-Type: text/html; charset=utf-8
Server: Microsoft-IIS/7.0
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET
Date: Fri, 02 Oct 2009 20:02:48 GMT
Content-Length: 6879

Error handled and logged, developers notified, user somewhat mollified, and Google/Bing happy. How's that for making the best of a bad situation?

For more information, see some of the following:

Of course, there's a ton of caveats in the above. However, one that is definitely worth addressing is how 404 handling is extra-special-different in ASP.NET. Very extra-special-different. And badly very different. But we'll save that for the next post.

UPDATE: If for some reason you cannot change your code, a good option to pursue is flipping a switch in your applicationHost.config file that passes your response through without IIS hijacking it. This is the sledgehammer approach, but could be applicable for your situation. For example, in order to allow Umbraco's alternative status code responses to work, this is the only way to get them to avoid being hijacked by IIS (i.e. NotFoundHandlers, booting screens, etc.).

<location path="Site Description">
    <system.webServer>
        <httpErrors existingResponse="PassThrough" />
    </system.webServer>
</location>

Thanks to Fabian Heussser's comment on Rick Strahl's post that helped with this.

Tuesday, August 25, 2009 11:39 AM

Windows 7: blissfully tabbing to complete path in Explorer

by Ted Jardine

This'll be a short post, but there are many things I love about Windows 7, and many things I really like, but I've just discovered something that you might not have heard about yet: in Windows Explorer, you can now tab to complete the path in the address bar. For example, in the path below, I can type "Of" and then hit tab and it will automatically complete "Office" followed by the next backward slash, ready for me to continue on down the path.

This might not seem like a big deal, but see how much time it saves navigating to "C:\Windows\Microsoft.NET\Framework64\v2.0.50727\Temporary ASP.NET Files". And yes, that's if you haven't already set up Executor/Launchy/Enso (note: does not work on x64)/Etc. to have a shortcut already to that particular location. However, it's even better than those launchy applications in that it's a) in Explorer already, b) it allows you to sometimes tab, sometimes type, sometimes arrow down to a selection, and c) hitting enter opens the current directory, but then you can go right back up with alt-d to the address bar and continue on again. If you use any of the above launchy-type applications, you'll know what I'm talking about. If you're not yet using one of them, hello?!? (and yes, while the new-in-Vista start button is an improvement, it's still not near as feature complete).

Another handy thing that I needed a third party app for before is the now-native "Copy as path" option for files (although you need to shift-f10 to open menu, or shift-left click to get it to show):

>Copy as path

Wednesday, May 20, 2009 9:18 AM

TypeMock unit testing

by Ted Jardine
I like free! As Michal Talaga in turn quotes TypeMock's promotion page:
Unit Testing ASP.NET? ASP.NET unit testing has never been this easy. Typemock is launching a new product for ASP.NET developers – the ASP.NET Bundle - and for the launch will be giving out FREE licenses to bloggers and their readers. The ASP.NET Bundle is the ultimate ASP.NET unit testing solution, and offers both Typemock Isolator, a unit test tool and Ivonna, the Isolator add-on for ASP.NET unit testing, for a bargain price. Typemock Isolator is a leading .NET unit testing tool (C# and VB.NET) for many ‘hard to test’ technologies such as SharePoint, ASP.NET, MVC, WCF, WPF, Silverlight and more. Note that for unit testing Silverlight there is an open source Isolator add-on called SilverUnit. The first 60 bloggers who will blog this text in their blog and tell us about it, will get a Free Isolator ASP.NET Bundle license (Typemock Isolator + Ivonna). If you post this in an ASP.NET dedicated blog, you'll get a license automatically (even if more than 60 submit) during the first week of this announcement. Also 8 bloggers will get an additional 2 licenses (each) to give away to their readers / friends. Go ahead, click the following link for more information on how to get your free license.
Yep, free sounds good. I've not had a chance to try out TypeMock, so this will be a good opportunity.