Some posts in NG as well as a customer case reminded me of that infrequent issue that may happen when a user visit a web site or attempts to use a web-based application hosted on IIS.
HTTP requests/responses contain two parts: the header and the body. The header contains most of time technical information exchanged between the client and the server, the body contains user-oriented information like the content of a webpage or a file to download, for example. The error message ere above is therefore generated by the server because the request sent by the client contains a header that is simply too large compared to what the server expects. But what can make a header section so large? It all depends on how you browser was configured (and the underlying OS as well in some case), but most of time, the culprits of larger header are cookies (header: Cookie) and authentication information (Header: Authorization).
When this error is experienced on an Internet site, there is not much you can do except cleaning up your cookies and hoping it will work afterwards while.
When this error shows up in an intranet environment when web servers are running IIS and possibly SharePoint, SQL Server Reporting Services or Exchange with OWA on top of it, this is caused by a combination of multiple factors which are the following:
– The web server (or web site) was configured to use Integrated Windows Authentication and Kerberos in particular
– The client is able to authenticate using Kerberos (the client system is member of an AD forest, the user too)
– The size of the user’s security token is large. This can be caused by: the user being member of many AD groups (hundreds of groups), the user’s object in AD contains SID history information as consequence of a domain migration/consolidation, the groups the user is member of is also affected by SID history, just like the user.
IIS is configured by default
– The client sends Kerberos-based authentication AND authorization information. Unlike NTLM and Basic which only send authentication information, therefore smaller, Kerberos includes information such as group membership and SID history information in the request’s header.
-> Depending on the user’s group membership and SID history information, some users may be affected, others not…
Hopefully, there are many solution to work around this problem but all of them have their trade-offs:
Use IP address in the URL instead of host names
Since Kerberos solely works with host names, using UIP addresses will automatically force negotiation to NTLM instead. Of course, this does not only degrades security but it also involves hard coding IP’s, which is rarely practically and also means that this can work if you only host one and only one web site on your IIS, finally, if your application uses “delegation” of credentials, which is a Kerberos feature, it will not work anymore (I am thinking about SQL SRS in particular in this case, or even Exchange OWA when used in FE-BE scenario’s)
Configure IIS to use NTLM only
As part of the Integrated windows Authentication setup, you can simply configure IIS to use NTLM only, the following MS KB article will show you how to do so: http://support.microsoft.com/kb/215383. Although this configuration is less impacting that the first one, you’re still left with lower security as well an compatibility issue with Kerberos is required by your application.
Configure IIS to accept larger headers
You can do so by configuring IIS in registry. It is important to note that this configuration will apply to all web sites running on the system running IIS because this settings is used at kernel-component level (http.sys), it will therefore impact all your applications on that server. This MS KB article explains all those settings http://support.microsoft.com/kb/820129, the one that is to change being MaxFieldLength. Be very careful doing so because this will increase the memory used by the system (kernel memory) to handle requests. On a “busy” (read: getting a lot of requests, not fewer large requests) 32-bit system, this can exhaust kernel memory. If the boot.ini switch /3GB is used (possibility combined to /USERVA), the situation can get worse since less memory is available to the kernel. On a 64-bit system, this configuration is harmless, even if the application is running in 32-bit mode, since this is handled in kernel mode. Of course, take care of what your application is doing with those headers too…
Plan a crash-diet for your AD users
By reducing (read optimizing) their group membership and removing SID history information from both user’s and group’s AD object attribute. Though this solution will be profitable in all scenario and not only web authentication (faster logon, less memory usage on application servers, Exchange mailbox servers…), never implement without a careful impact assessment. This MS KB article explains how to automate this task: http://support.microsoft.com/kb/295758
Now for some background information:
– Since that error is generated by the HTTP.SYS driver, is will not be logged in the W3C of whatever files but in HTTPERR.LOG. To track who is affected, simply look at the client’ip in the log. Of course, since authentication information cannot be handled, user’s identity remains unknown.
– To estimate the token size for general purposes, no only for this issue in particular, you can use this tool provided by MS: http://www.microsoft.com/downloads/details.aspx?FamilyID=4A303FA5-CF20-43FB-9483-0F0B0DAE265C&displaylang=en
– The way IIS handles headers and therefore authorization information is one thing, the way Windows system, in general, handle large token is another. Make sure you keep you IIS and windows configuration consistent so that authentication is successful end-to-end. Read this MS paper for more information: http://www.microsoft.com/downloads/details.aspx?familyid=22DD9251-0781-42E6-9346-89D577A3E74A&displaylang=en
– A good network monitor (I like to call it http://www.wireshark.org/) and well as a troubleshooting tool like MS WFetch (http://www.microsoft.com/downloads/details.aspx?FamilyID=b134a806-d50e-4664-8348-da5c17129210&DisplayLang=en) will help the troubleshooting task. As usual, MS LogParser is your friend when it comes to analyzing IIS log files, look at this forum for help parsing HTTPERR.LOG http://forums.iis.net/1141.aspx or keep on reading this blog for more LogParser scripts.
– For simplicity’s sake, I use the work “Kerberos” when talking about authentication protocol between client and web server. The actual protocol is SPNEGO or “Negotiate”, which is a wrapper for multiple authentication protocols. Refer to Wikipedia for the details: http://en.wikipedia.org/wiki/SPNEGO