Suddenly some apparently well performing IIS servers recently started reporting this error regularly. some of them were also running SharePoint or OWA. all of them configured to use Integrated Windows Authentication (IWA) as authentication mechanism.
The problem with IIS worker process is that it can have so many explanation depending on the code executed that you can easily waste a week until you find a reasonable explanation. In this case, all servers were affected, regardless of the application they run. So my first idea was “they might be under attack”. But that was not the case: performance counters related to the worker process did not give any sign of that, this was confirmed by the IIS logs. Next”usual suspect”, a patch recently installed: bingo, that was it. Here are the details:
- The Security Update implementing “Extended Protection” for authentication in IIS (KB973917) was just deployed on all servers
- All impacted servers are running Windows Server 2003 Service Pack 2
- One or multiple application served by that application pool/worker process have IWA enabled
- After intensive file version analysis, it appeared that numerous IIS-related files (EXE, DLL’s…) were with a version prior SP2
Due to the inconsistency of IIS files in combination with that extra hot fix, the worker process keeps crashing –> root cause found!
Now how to fix it:
- Perform an inventory of currently installed post-SP2 fixes. I personally do it in a very straightforward way using psinfo but I am sure you’ll find plenty of methods to do it the way you like
- Reinstall the Service Pack 2
- Redeploy post-SP2 hot fixes, see step 1
- Check installed IIS File versions
- If file versions are OK, Install KB973917
- MS KB: Description of the update that implements Extended Protection for Authentication in Internet Information Services (IIS)
- MS KB: Internet Information Services 6.0 may not function correctly after installing KB973917
- MS Security Research & Defense Blog: Extended Protection for Authentication
- MS TechNet: Microsoft Security Advisory (974926) Credential Relaying Attacks on Integrated Windows Authentication
- MS IIS Support Team Blog: Sample Script to Verify IIS File Versions for KB973917. [UPDATE] Here is my PowerShell Translation [UPDATE 2] It seems like the scripts (both versions) do not behave as expected on a 64-bit server. I am currently investigating…
Note: Make sure you pay attention to the process exit code which is always 0xffffffff. If you see another code, it might of course have another cause.