Skip to the content Back to Top

The Quick

Tell ASP.NET never to use %UnicodeValue notation when URL encoding by putting the following appSetting in web.config:

<add key="aspnet:DontUsePercentUUrlEncoding" value="true" />


The Slow

Sometimes, such as when calling NameValueCollection.ToString(), that value gets url encoded for you. However, the url encoding in .NET defaults to a %Unicode notation, which, according to MSDN's own warning now attached to the obsolete-as-of-4.5 HttpUtility.UrlEncodeUnicode() method, "produces non-standards-compliant output and has interoperability issues." Therefore, even if you are targeting .NET Framework 4.5 in your project, NameValueCollection.ToString() will still use that obsolete method, and you will get %u00XX style encoding in your URLs.

Telltale signs of "interoperability issues" include the word Français being encoded Fran%u00e7ais, which then blows up the search engine you lovingly built that runs on Java and Apache Solr.

A little over a week ago, a security vulnerability was found in all versions of ASP.NET (Microsoft’s web application framework) that could potentially allow private information disclosure. The vulnerability stems from a cryptographic weakness, specifically involving improper error handling during encryption padding verification. Attacks based on this weakness could allow a hacker to decrypt sniffed cookies or forge authentication tickets, among other attacks.

Microsoft has released an out-of-cycle emergency fix to address this security vulnerability. We strongly urge you or your IT staff to go to http://www.microsoft.com/technet/security/bulletin/MS10-070.mspx to download and install all applicable patches on your Microsoft Windows-based Web servers that may be affected by this vulnerability. Additionally, the fix will be rolled out via Windows Updates in the upcoming week or so.

This vulnerability could affect any web sites using ASP.NET, i.e. recent versions of Andornot’s Starter Kits and Inmagic WebPublisher PRO or Inmagic Genie.  It does not impact desktop only installs or clients using the default Inmagic web interfaces.

If you have any further questions regarding applications we have designed for you, please do not hesitate to contact us.

If you've read and followed up on my previous posts about handling errors, you might have found yourself pulling out your hair when you discovered that your error handling went south again when you changed your target ASP.NET framework from 2.0 to 3.5/4.0 (applies to WebForms only of course). So here I am to save your day again (and yes, I wasted an entire Friday evening figuring this one out - good times).

In a nutshell, upgrade to 3.5 and your awesomely handled 500 and 404 errors start going back to 302 and 200 redirects the ol' fashioned (and completely idiotic) ASP.NET way of doing things. Of course, if you are like me, you'd waste an entire evening on it. But fortunately, you're not so you won't.

There's two issues/solutions:

  1. As of ASP.NET 3.5 there's now a redirectMode="ResponseRewrite" setting for your customErrors setting in your Web.config (yeah!). Put it in and you're halfway there (the default is the aforementioned moronic "ResponseRedirect"). Now your:
  2.             Context.Response.Clear();
                Context.Response.TrySkipIisCustomErrors = true;
                Context.Response.StatusCode = statusCode;
                Context.Response.Status = status;
               
                Context.Server.Transfer(errorPage, false);
    is back firing on all cylinders again.
  3. Except that your 404s are fine now, but your 500 errors return your error page twice in the body of your error page! Genius that I am, I went back to my experiences with IIS 6 and wrapped error pages and figured out that IIS 7/7.5 was sending back "its own" error page and then my server transfer was getting tacked on as well. Again, this does not effect 404s - I'm guessing because we clear the error (see previous post). So remove the Server.Transfer for non-404s and we're golden again.

In summary, for IIS 7/7.5 and .NET 3.5/4.0, use the following as a starting point for your Global.asax.cs or, better yet, an error module (for good measure figure out how it could be all tested):

        private readonly static ILog Log = LogManager.GetLogger(typeof(Global));
        private string _errorPageLocation;
        private string _error404PageLocation;


        protected void Application_Start(object sender, EventArgs e)
        {
            // Wire up log4net database connection string if desired.


            if (Log.IsInfoEnabled)
                Log.Info("Log4Net initialized - Ignore.");
        }


        protected void Application_Error(object sender, EventArgs e)
        {
            InitConfigurationFields();


            Exception exception = Context.Server.GetLastError();

            if (exception is HttpUnhandledException)
                if (exception.InnerException != null)
                    exception = exception.InnerException;


            if (exception is HttpException)
            {
                var ex = (HttpException)exception;
                var statusCode = ex.GetHttpCode();

                if (statusCode == 404)
                {
                    ServerTransfer(_error404PageLocation,
"404 Not Found", statusCode);
                    return;
                }
            }


            Log.Error("Unhandled error caught in error module.", exception);

            //Todo: Any way to get the correct status statement for a specifc code?
//Hard-coding all to "500 Internal Server Error" here.
            ServerTransfer(_errorPageLocation, "500 Internal Server Error", 500);
        }


        /// <summary>
        /// Gets configuration fields for database connection (if applicable),
/// and customError redirects.
        /// </summary>
        private void InitConfigurationFields()
        {
            var section = WebConfigurationManager.GetSection("system.web/customErrors");

            if (section != null && section is CustomErrorsSection)
            {
                var customErrorsSection = (CustomErrorsSection)section;
                _errorPageLocation = customErrorsSection.DefaultRedirect;

                var error404 = customErrorsSection.Errors["404"];

                if (error404 != null)
                    _error404PageLocation = error404.Redirect;
            }


            if (string.IsNullOrEmpty(_errorPageLocation))
                throw new NullReferenceException(
"customErrors DefaultRedirect must be specified in Web.config.
...For i.e. ~/error.aspx"
);


            if (string.IsNullOrEmpty(_error404PageLocation))
                throw new NullReferenceException(
"customErrors 404 statusCode redirect must be specified
...in Web.config. For i.e. ~/page-not-found.aspx"
);
        }


        private void ServerTransfer(string errorPage, string status, int statusCode)
        {
            // If customErrors is off, just let ASP.NET default happen.
            if (Context == null || !Context.IsCustomErrorEnabled)
                return;


            // Want the error around so that we can provide a little more
// descriptive message to end user in error page.
            // However, for 404 errors (only!), not clearing the error causes IIS7 to
// still hijack the process and the custom 404 error page.
            if (statusCode == 404)
                Server.ClearError();

            Context.Response.Clear();
            Context.Response.TrySkipIisCustomErrors = true;
            Context.Response.StatusCode = statusCode;
            Context.Response.Status = status;

            // For .NET 3.5, doing a Server.Transfer combined with customErrors
// redirectMode="ResponseRewrite" returns the error page body *twice*.
            if (statusCode == 404)
                Context.Server.Transfer(errorPage, false);
        }

As mentioned at the end of my previous post on handling errors with ASP.NET, handling "404 Not Found" errors are particularly problematic (if you haven't read it yet, please do so). And looking around, the vast majority of information out there on it is not complete, misinformed, or flat-out wrong (but I greatly appreciate all efforts!). And I would argue that this is because ASP.NET implementation of 404 error handling is flat-out-wrong. So with my super hero cape on, here I come to wobbly save the day!

The typical ASP.NET way to handle 404 errors is to put something like the following in your Web.config:

<customErrors mode="On" defaultRedirect="~/error.aspx">
    <error statusCode="404" redirect="~/page-not-found.aspx" />
</customErrors>

Make a page-not-found.aspx page and voila! Ya got 'er dun! If you're a little more on the ball, you'll realise that while this configuration works for end users (gives them a pretty page to look at hopefully clearly explaining that you can't find what they're looking for), it's bad for SEO (search engine optimization) because it sends back a 302 temporary redirect to your 404 page which in turn sends back a "200 OK" message. In other words, "Yeehah! No problems here as you've found what you're looking for! Index away!"

So, bright developer that you are, you add in some applicable status code into your 404 page thinking that should take care of it:

protected override void OnLoad(System.EventArgs e)
{
    Response.TrySkipIisCustomErrors = true; // For IIS 7 Integrated Pipeline - see previous post
    Response.Status = "404 Not Found";
    Response.StatusCode = 404;
    base.OnLoad(e);
}

Well, fire up Fiddler and you'll discover that you're still getting a 302 temporary redirect to your 404 page. So you fire up your error handling code and for 404s, you Server.Transfer to your 404 page just like all your other error transfers take place! But no, bafflingly enough, even running through a debug session to ensure you're properly catching your 404, ASP.NET still insists on 302'ing your precious response (although at least now your 404 page is sending back the proper "404 Not Found" error status). So go out there and google everywhere and try every suggestion (just a sampling) and then breathe a prayer of thanksgiving for me and my super-duper super hero cape, because this is how you make your way to the 404 handling bliss (caveat: thoroughly tested only on IIS 7):

First, make something like the following in Application_Error in your global.asax (or better yet, a custom error module):

protected void Application_Error(Object sender, EventArgs e)
{
    Exception exception = Server.GetLastError();
    if (exception is HttpUnhandledException)
    {
        if (exception.InnerException == null)
        {
            Server.Transfer(ERROR_PAGE_LOCATION, false);
            return;
        }
        exception = exception.InnerException;
    }

    if (exception is HttpException)
    {
        if (((HttpException)exception).GetHttpCode() == 404)
        {

            // Log if wished.
            Server.ClearError();
            Server.Transfer(NOT_FOUND_PAGE_LOCATION, false);
            return;
        }
    }

    if (Context != null && Context.IsCustomErrorEnabled)
        Server.Transfer(ERROR_PAGE_LOCATION, false);
    else
        Log.Error("Unhandled Exception trapped in Global.asax", exception);
}

Second, put something like the following in your 404 page (or even better, add it to your above custom error module):

protected override void OnLoad(System.EventArgs e)
{
    Response.TrySkipIisCustomErrors = true; // For IIS 7 Integrated Pipeline - see previous post
    Response.Status = "404 Not Found";
    Response.StatusCode = 404;
    base.OnLoad(e);
}

And lastly, put whatever you want in your customErrors section in your Web.config for 404 - remove the 404 references if you want. The only time it would get used is if you keep the confusion level down by reading that section in your error module to find out what your 404 page is.

And now, you don't get any nasty 302 redirects. You get a blissfully pure and pretty 404.

Note: you can have everything in place as above, but if you forget to Server.ClearErrror(), it'll all be for naught as you'll still get 302 redirected (thanks to http://stackoverflow.com/questions/667053/best-way-to-implement-a-404-in-asp-net for finally helping me get this one nailed).

UPDATE: If for some reason you cannot change your code, a good option to pursue is flipping a switch in your applicationHost.config file that passes your response through without IIS hijacking it. This is the sledgehammer approach, but could be applicable for your situation. For example, in order to allow Umbraco's alternative status code responses to work, this is the only way to get them to avoid being hijacked by IIS (i.e. NotFoundHandlers, booting screens, etc.).

<location path="Site Description">
    <system.webServer>
        <httpErrors existingResponse="PassThrough" />
    </system.webServer>
</location>

Thanks to Fabian Heussser's comment on Rick Strahl's post that helped with this.

I've encountered many twists and turns in my journey towards handling application errors with aplomb. You have to make sure:

  1. Users are happy (by providing informative error pages) - well, as happy as they can be under the circumstances.
  2. Developers who need to fix the problem are happy (extensive logging and notification to help re-enact, locate, and then fix the bug so it doesn't happen again).
  3. Search bots are happy (by sending them the right message).

For example, when a user requests a resource that can't be found you want to help the user out with at least a pretty page that explains what has happened. Most users don't have a clue what a "404 error" is and could care less. And almost just as important, when a search bot encounters an error it needs to receive a proper error code back or else it gets all uppity on you and knocks down the page's ranking. In other words, when a page can't be found:

  1. Redirect to a helpful "Sorry, we can't find the resource you're looking for" page that hopefully looks somewhat decent and gives the user some options on how to proceed (perhaps like our "smart" page not found functionality on our soon-to-be-launched www.andornot.com).  
  2. Log the 404 so that it can be dealt with (if applicable).
  3. Set the proper 404 error code for SEO purposes.

However, sometimes getting the three above objectives dealt with in one fell swoop is problematic. For example, if you just redirect after having an error occur (say a "500 Internal Server Error") to a pretty page, users will be happy, but search bots will receive something like the following:

Cache-Control: private
Content-Type: text/html; charset=utf-8
Content-Encoding: gzip
Vary: Accept-Encoding
Server: Microsoft-IIS/7.5
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET
Date: Fri, 02 Oct 2009 18:30:47 GMT
Content-Length: 7719

200 OK

200 OK for something that really is not okay. It should be "500 Internal Server Error"! Erk. So how about we code in something like:

Response.StatusCode = 500;
Response.StatusDescription = "500 Internal Server Error";

But then things start to get interesting. In IIS 6 all appears to look good, but when you're using MasterPages, Internet Explorer 6 (in a rare case of actually being helpful) shows that what's being sent back is wacked as it wraps the custom error page with the default IIS error page. Gee, now two out of three of the above objectives are out the window. Suffice it to say, the result is grotesque, and I can't imagine what search bots think of it!

Moving on to IIS 7 and its Integrated Pipeline (doesn't apply to Classic Pipeline mode), things get even more interesting as it ignores your custom stuff altogether and sends back the applicable default IIS error page. What the? Rick Strahl has an excellent post addressing this issue. Essentially, the error is still trapped by ASP.NET (so it's logged, redirected, and whatever else you ask it to do), but then it is ultimately handled by IIS which sees the error code and hijacks the entire response and returns its ugly default error page (and who designed those anyway? Even IIS 6's were better).

So where do we go from here? In IIS 6, the problem only occurs when using MasterPages in your error page. Over simplifying, the page's MasterPage renders later on in the page lifecycle and resets the error code to 200 OK. And it just gets messier from there (see http://stackoverflow.com/questions/347281/asp-net-custom-404-returning-200-ok-instead-of-404-not-found for more info). So reset it back again further along in the page's lifecycle!

protected override void Render(HtmlTextWriter writer)
{
    base.Render(writer);
    Response.StatusCode = 500;
    Response.StatusDescription = "500 Internal Server Error";
}

But what about IIS 7? If you look further along in Rick Strahl's post mentioned above, there's a new flag in ASP.NET 3.5 called Response.TrySkipIisCustomErrors. As Rick says, "In a nutshell this property when set to true at any point in the request prevents IIS from injecting its custom error pages. This flag is new in ASP.NET 3.5 - so if you're running 2.0 only you're out of luck."

    Response.TrySkipIisCustomErrors = true;
    Response.StatusCode = 500;
    Response.StatusDescription = "500 Internal Server Error";

And Shazam!

HTTP/1.1 500 500 Internal Server Error
Cache-Control: private
Content-Type: text/html; charset=utf-8
Server: Microsoft-IIS/7.0
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET
Date: Fri, 02 Oct 2009 20:02:48 GMT
Content-Length: 6879

Error handled and logged, developers notified, user somewhat mollified, and Google/Bing happy. How's that for making the best of a bad situation?

For more information, see some of the following:

Of course, there's a ton of caveats in the above. However, one that is definitely worth addressing is how 404 handling is extra-special-different in ASP.NET. Very extra-special-different. And badly very different. But we'll save that for the next post.

UPDATE: If for some reason you cannot change your code, a good option to pursue is flipping a switch in your applicationHost.config file that passes your response through without IIS hijacking it. This is the sledgehammer approach, but could be applicable for your situation. For example, in order to allow Umbraco's alternative status code responses to work, this is the only way to get them to avoid being hijacked by IIS (i.e. NotFoundHandlers, booting screens, etc.).

<location path="Site Description">
    <system.webServer>
        <httpErrors existingResponse="PassThrough" />
    </system.webServer>
</location>

Thanks to Fabian Heussser's comment on Rick Strahl's post that helped with this.

Categories

Let Us Help You!

We're Librarians - We Love to Help People