WSUS SUP causes high CPU and clients fail updates scan

NOTE: The configuration suggestions I mention in this post won’t fix the underlying issue. Depending on the size of your environment they may be enough to get things working for you again. Microsoft is currently working on releasing a hotfix that I have tested and found to resolve this problem

 

Microsoft have released the WSUS server hotfix, details here: https://blogs.technet.microsoft.com/configurationmgr/2017/08/18/high-cpuhigh-memory-in-wsus-following-update-tuesdays/

NOTE2: It turns out there is a new issue from the August 2017 updates that “clears” the update history on a computer that will trigger a full client scan again. This will also cause high load on your WSUS server, although for slightly different reasons, however the suggestions here and the coming updates will help to resolve the load issue from that problem as well.

Microsoft have updated the August cumulative updates to resolve this issue, details here: https://support.microsoft.com/en-us/help/4039396/windows-10-update-kb4039396

 

NOTE3: Microsoft has now published some additional official guidance here: https://blogs.technet.microsoft.com/askcore/2017/08/18/high-cpuhigh-memory-in-wsus-following-update-tuesdays/

This issue is one I first encountered on only a couple of our WSUS servers (2 or 3 of 15 servers) last year in November 2016 after the new cumulative update process was introduced for patching. At first I assumed it to be a failure on my part to do more regular cleanup, or a result of the recent upgrade to ConfigMgr 1610, or an “end of year” rush of activity on the network. This isn’t unusual for the environment I currently manage (Education with approx. 370,000+ devices)

At first I looked at server bottlenecks (we run everything in VMWare) and even SQL DB corruptions. I tried doing WSUS resets, even recreating the database (this is a last resort in a large environment). I then thought maybe it was a Server 2012 WSUS issue as we had other Server 2012 related cases open with Microsoft. To test I rebuilt one server as 2012R2, but the problems persisted. Given it was only happening on a couple of server I assumed it was an issue with those servers in particular and didn’t suspect a larger issue.

Over the Christmas holidays things went quite, so there was nothing more I could do until school returned the following February.

Then everything basically exploded.

The first patch cycle we ran saw the WSUS server rocket to 100% CPU and stay there. Nothing I did could stop this reoccurring. I found ways to bring things under control for a few hours at a time. Endpoint definitions started falling behind because clients couldn’t scan for updates. Then it started happening on a couple more of the servers. At this point I conceded defeat and called in Microsoft. Unfortunately it was another 6 months before they finally identified it was a “function” of WSUS causing the grief and not the configuration or size of our environment.

The Problem

The most obvious symptoms will be clients failing to scan for updates and the WSUS server CPU (w3wp.exe) going very high. Some clients get through, many will fail. The main cause will be Windows 10 clients and the way WSUS has to process the Cumulative Updates.

Configuration Manager 1511 adventures

I am starting work on now upgrading have upgraded our ConfigMgr2012 R2 hierarchies to the new “Configuration Manager with no more version numbers” build 1511. There are roughly 120+ site servers in the central hierarchy (across 2 CAS hierarchies), so hopefully things go well. For the most part things went quite well, no major issues at all.

I’ll keep this page to note any special gotchas that are outside the various step-by-step guides already out there, and also some easy quick link references.

How to update ConfigMgr Package and Application source paths

When you need to change the location that you have been saving all your package source files, you will then need to update all the existing packages and applications to point to that new location.

(Note: See here to read what happens when you do update all your source locations)

This is fairly simple for Packages, but is a bit more complicated for Applications

Microsoft Updates deployment with Configuration Manager – Braindump

This post is going to be more of a “brain-dump” post of thoughts and ideas around the process of using ConfigMgr to deploy Microsoft updates in your environment. There are already numerous “guides” on the net already, so why?

This may or may not be a long read. It will just be my thoughts at this stage, with later posts (possibly) going into details of the pros and cons of different approaches and issues encountered. All feedback or experiences you would like to share are most welcome, and I’ll incorporate new ideas and points to consider into the post

Handy Powershell commands – Hyper-V

A collection of Powershell commands that I’ve come up with. Will add over time

Configuration Manager Typos

A light hearted post that I will update over time as I find new examples of typos and error in ConfigMgr log files.

(note: changed to be typos in ConfigMgr in general now)

Exchange 2013 warnings Event ID 10010, 6002, 6027, 1035

I’ve started to get a *lot* of warnings being logged lately with EventID 10010, 1035, 6027 and 6002. The specific cause I don’t know yet, but they have only appeared recently after I have been “fixing” my server after some serious disk issues. It’s hard to say though because there have been so many that my event logs have been clearing out the older entries so I need to increase their size to capture them all. While the server appears to be running fine, these warnings are appearing repeatedly now.

I don’t know if they are because of further issues since the crashes, or have started because of the way I fixed something else. I know I’m not the only one, a lot of other people have posted with the same messages, so perhaps it’s specific to Server 2012, or a recent OS update that has been applied.

Further investigation required…

UPDATE: I’ve seen some reports that this has been fixed in CU1. I haven’t installed it yet, so can’t confirm at this time.
http://technet.microsoft.com/en-us/library/jj983803(v=exchg.150).aspx