WSUS Delete Obsolete Updates

NOTE: Usual warnings apply. Do a backup before making any changes. If you are unsure about anything in the post then ask or look for more information or help before attempting it.

Over time WSUS will accumulate update metadata that can create performance issues for clients. In large environments this can be quite an issue.

There is a script Microsoft often provides during Premier Support calls to cleanup this update metadata, however there are a few issues:

• The query can take a *really* long time to run if there are a lot of updates to cleanup. In some cases it can take *days*
• You need to stop all the WSUS services while it runs
• If it fails for whatever reason, it will have to start all over because it doesn’t commit the changes until it completes successfully
• While it runs, the TEMPDB and Transaction logs will grow quite significantly until the data is committed
• It gives no useful information on progress

There is a TechNet article (This is essential reading and has LOTS of important stuff) and a Forum Post where an improved version was written that gave progress of the cleanup, however it didn’t address the temp/transaction growth issues or the time issues. To this end I have applied my very rudimentary SQL scripting skills.

WSUS SUP causes high CPU and clients fail updates scan

NOTE: The configuration suggestions I mention in this post won’t fix the underlying issue. Depending on the size of your environment they may be enough to get things working for you again. Microsoft is currently working on releasing a hotfix that I have tested and found to resolve this problem

 

Microsoft have released the WSUS server hotfix, details here: https://blogs.technet.microsoft.com/configurationmgr/2017/08/18/high-cpuhigh-memory-in-wsus-following-update-tuesdays/

NOTE2: It turns out there is a new issue from the August 2017 updates that “clears” the update history on a computer that will trigger a full client scan again. This will also cause high load on your WSUS server, although for slightly different reasons, however the suggestions here and the coming updates will help to resolve the load issue from that problem as well.

Microsoft have updated the August cumulative updates to resolve this issue, details here: https://support.microsoft.com/en-us/help/4039396/windows-10-update-kb4039396

 

NOTE3: Microsoft has now published some additional official guidance here: https://blogs.technet.microsoft.com/askcore/2017/08/18/high-cpuhigh-memory-in-wsus-following-update-tuesdays/

This issue is one I first encountered on only a couple of our WSUS servers (2 or 3 of 15 servers) last year in November 2016 after the new cumulative update process was introduced for patching. At first I assumed it to be a failure on my part to do more regular cleanup, or a result of the recent upgrade to ConfigMgr 1610, or an “end of year” rush of activity on the network. This isn’t unusual for the environment I currently manage (Education with approx. 370,000+ devices)

At first I looked at server bottlenecks (we run everything in VMWare) and even SQL DB corruptions. I tried doing WSUS resets, even recreating the database (this is a last resort in a large environment). I then thought maybe it was a Server 2012 WSUS issue as we had other Server 2012 related cases open with Microsoft. To test I rebuilt one server as 2012R2, but the problems persisted. Given it was only happening on a couple of server I assumed it was an issue with those servers in particular and didn’t suspect a larger issue.

Over the Christmas holidays things went quite, so there was nothing more I could do until school returned the following February.

Then everything basically exploded.

The first patch cycle we ran saw the WSUS server rocket to 100% CPU and stay there. Nothing I did could stop this reoccurring. I found ways to bring things under control for a few hours at a time. Endpoint definitions started falling behind because clients couldn’t scan for updates. Then it started happening on a couple more of the servers. At this point I conceded defeat and called in Microsoft. Unfortunately it was another 6 months before they finally identified it was a “function” of WSUS causing the grief and not the configuration or size of our environment.

The Problem

The most obvious symptoms will be clients failing to scan for updates and the WSUS server CPU (w3wp.exe) going very high. Some clients get through, many will fail. The main cause will be Windows 10 clients and the way WSUS has to process the Cumulative Updates.

Windows 10 Speech language missing

 

As part of the Insider program I get rather frequent upgrades to Windows 10. Each time the upgrade installs it resets my speech language to “English (United States)” which means Cortana stops working as my Region is set to “Australia”

I also use ConfigMgr to handle updates on my network (this would also apply for people using WSUS) so when I go into the Region & language settings I don’t get the Speech feature appearing under the “English (Australia)” options.

Luckily, it is relatively easy to sort out.

SOLUTION:

“Required” Automatic Deployment Rules

The Automatic Deployment Rule (ADR) feature in ConfigMgr2012 is quite handy, especially for people moving from WSUS that aren’t too worried about updates being automatically deployed.

Many larger organisations however tend to have a more controlling approach to which updates are approved for deployment and will approve/decline each update as required.

One thing I liked in WSUS was the ability to have updates automatically approved, but being able to set the client policy to say “Notify Only”. On my servers I could have them scan and determine applicable updates, but then I would manually approve them and reboot as required, or I could exclude some updates if they were causing problems on a per-server basis (e.g. .NET). Sure you could do all that through WSUS itself if you wanted to setup lots of different computer groups, but for small environments with half a dozen servers it’s easier this way.

In ConfigMgr2012, there is no way to “auto create” and update group unless you use an ADR. However the ADR configuration makes all deployments Mandatory with a deadline and does not give a “Required” notify only type option.

Migrate Approved update lists from WSUS to ConfigMgr 2012

If you work in a site that uses WSUS and just has auto-approve rules setup, then read no further. Just go and create Auto Deployment Rules in ConfigMgr that will continue to do exactly the same thing.

If you work in an organisation that reviews and approves each Microsoft update to be released, and over the years you have a fairly unfriendly looking list of approved and not-approved updates, then the thought of going through that list, manually selecting each update to add to an Update Group, and then repeating for each computer group, probably isn’t very appealing.

Finding myself in that situation I did what any rational lazy admin would do before scripting my own solution:

TO THE INTERNET

Luckily, I managed to stumble across this site which seems to do exactly that is required. I haven’t tested it out yet, but it’s published on a Technet Blog, so what could possibly go wrong?

UPDATE: Yes, it all works as required. One thing to watch for is ConfigMgr doesn’t let you approve/deploy superseded and expired updates, so you will probably notice your update groups have fewer approved updates than you have been deploying. The scripts also don’t do anything about approving the newer versions of those older updates so you’ll need to then check what updates are required. Not such a drama really as most of the heavy lifting has already been done by the scripts.

http://blogs.technet.com/b/manageabilityguys/archive/2012/08/25/migrating-from-wsus-to-configuration-manager.aspx