Microsoft Updates deployment with Configuration Manager – Braindump

Posted: May 13, 2014 in Configuration Manager, Information, System Center, Windows Update, Work in Progress
Tags: , , , ,

This post is going to be more of a “brain-dump” post of thoughts and ideas around the process of using ConfigMgr to deploy Microsoft updates in your environment. There are already numerous “guides” on the net already, so why?

This may or may not be a long read. It will just be my thoughts at this stage, with later posts (possibly) going into details of the pros and cons of different approaches and issues encountered. All feedback or experiences you would like to share are most welcome, and I’ll incorporate new ideas and points to consider into the post

In my work I tend to move from being an integrator/consultant for several years engaging with many different clients for relatively short periods of time (days, weeks, maybe months), then several years working directly for “clients” for extended periods (6-12+ months). This exposure  gives me perspective on the “design, deploy, document, leave” life of a consultant where things like “recommended” and “best practice” rule. I then spend time working with actual clients for those extended periods which allows me to experience the actual BAU operations of these environments and how those “consultant solutions” work in the real world. This is when you encounter the less-than-ideal way that processes are actually applied and where deviations sneak in that can turn the whole glorious solution into a “do whatever makes it work” thing.

“Best practise” is a term that has always annoyed me. It should really be called the “in a perfect world” or “serving suggestion” guidance. While it does provide a good starting point, the belief many people seem to hold that it should be followed to the letter make it virtually impossible to apply. I am of course applying very broad generalisations to world of “best practice” here, but I’m sure if you’ve been there you know what I mean. Best Practise works brilliantly in a Lab.

So I would argue that in many cases, “best practise” is a good place to start from. It is not the solution, it is not prescriptive, you won’t void your warranty if you do it differently.

I’m on a journey of discovery right now. Working with an exceptionally large installed client base with very significant challenges and requirements (you know, like every client has). I’m here on one of my extended direct to client engagements, and after working on designing and building the environment, I’m now involved in running it to “get out the kinks” in the processes before handing over to operations. I’ll probably be doing this for several months, so it gives me an excellent opportunity to really see what works and what doesn’t work.

I did of course start with “best practise”, and I’ve worked through a high level process that I thought would work well with their existing change control and deployment timeline and process. However, it’s not until you start doing the actual clicking on buttons thing and creating a real deployment that you start to spot the “gotchas”.

So the brain-dump (this will grow and change over time as I think of things to add)

Migration
  • Existing WSUS environment with computer groups and approvals
    • Creating ConfigMgr Update Groups with the same “approved” updates only (use cool scripts that make this easy)
    • Identifying updates that were approved in WSUS but are not visible or available in ConfigMgr (e.g. expired/superseded
  • Initial configuration of software update groups baseline
    • Build new “RTM” machine and identify all updates required?
    • Just choose currently required updates in environment?
    • Excluding updates that were not approved for whatever reason in the past, and ensuring previously “declined” updates don’t appear again as required
    • Keeping baseline group up to date as client start reporting in and machines that had “dropped off” the update process try to catch up
    • Identifying updates that may have been missed by previous update system and never reviewed
  • Co-existence of ConfigMgr and WSUS
    • Some computer will still need GPO for WSUS config while the ConfigMgr client deployment is under-way
    • There may be a migration period where WSUS is still used and CM12 is not enabled for updates until finished
    • There may be a need to “filter” the GPO to enable migrated clients to be managed by CM12 while leaving WSUS clients functioning
Client Configuration
  • Client Policies and settings
    • What settings, how do they relate to understanding of previous methods (e.g. WSUS)
  • Group Policy
    • While ConfigMgr will configure and control Microsoft Updates, “Windows Update” control panel appears as a warning saying automatic updates are not being applied
Target Collections
  • Specific collections used for Update deployment
    • Collection refresh times
  • Computers in multiple collections
    • Multiple deployments targeted at different times, unexpected install and reboot time
Software Update Groups
  • Different department reviews and approves updates to department that manages and deploys
    • Delays and lead time in assessment and approval
    • Companies that have long patch cycles (e.g. Servers every 6 months) and updates that expire without newer update being approved
    • Identifying superseding updates that are not cleared by the business, but where previous approved superseded update no longer available in CM
  • Creating a single “master” Update Group
    • Separate groups for Servers and Workstations, other?
  • Creating “monthly” update groups
    • Rolling-up into single “master”
  • Selecting Required updates is complex (see also reporting)
    • Console view does not allow filter by collections
    • SSRS reports to do not have mechanism to approve, or export/import to Update Group
  • Approving a single “security bulletin” may require selecting the multiple target versions of the update
  • Filtering and search can be very limited
    • Searches with many “Products” are painful to create (need to add them one at a time)
    • e.g. Filtering new “Internet Explorer” versions out, but keeping “Cumulative update for Internet Explorer”
    • No choice for Boolean “AND, OR” when searching/filtering
    • Concept of folders under “All Software Updates” is confusing
    • Lack of “search folders” rather than needing to select saved searches complicates operations
Deployment Packages
  • Creating a single “master” package source
    • New package when certain size
    • Issues with update “corruption” and process to remediate in ConfigMgr
    • Distribution of single package vs multiple
  • Creating “monthly” update package sources
  • Creating special packages e.g. service packs
  • Separate packages for servers, workstations or products?
  • Size of package store
    • Ensuring package source content still available if “old” package and source removed
  • No alerts for distribution status of specific important package like updates
Automatic Deployment Rules
  • Use for Endpoint Definitions
  • Use for computer update deployment
  • Use to “pre-download” required updates only (not auto-deploy or auto-distribute)
    • Create dummy Distribution Group with no members and use as target of ADR
    • Set to download updates at a given time (e.g. offpeak), but then only distribute the ones you want
  • Creating WSUS “like for like” hands off style approach
  • No alerts for specific package distribution
    • If Endpoint package fails to distribute, may not be noticed until “current” definitions are seen dropping
    • Current alert for “deployment success rate” needs extra “if for longer than xx hours/days” to prevent unwanted alerts from weekends when clients are offline
Maintenance Windows
  • Define on same “Update” collections or define specific maintenance window collections
  • Identifying when maintenance windows are in effect on clients
    • Troubleshooting deployment issues
  • Managing Server change windows, ensure set “outage” time only
    • No option to “expire” updates
    • Ensure servers that “miss” the required time don’t deploy the next night or next maintenance window unexpectedly
  • Complex maintenance and change schedules
    • e.g. Change window starting third Tuesday of each month for 3 days. Scenarios where the third Wednesday or Thursday do not actually follow the third Tuesda
Change Control
  • Change approval process requires specific updates only rather than simply “all required” updates
    • Manually picking out and selecting multiple updates can be tedious
  • Ensuring changes only occur during defined period
    • Avoid clients “catching up” the next morning, or same time next day
  • Reboot control
    • Servers, special sequence or order, linked services (one at a time)
  • Monitoring
    • SCOM or other monitoring raising “false” alarms from server patching and reboots
Test and Pilot
  • Collections for test and pilot phases
  • “Rolling up” approved tested updates for all machines
    • For “total” compliance, need Software Update group that has all updates that have ever been approved
  • “Rolling up” test and pilot computers as part of “all computers” (reporting)
    • When doing compliance reporting, need collections with all computers including test/pilot ones
  • Timing (as per Change Control section) with “2nd Tuesday” style deployments
Reporting and Compliance
  • Identifying required updates (SSRS, Consoles)
  • Native Console view can not filter by target collections or computers
    • Updates are either required or not, no way to say required by specific group of devices
  • SSRS reports do not provide any method to bulk approve or export/import a report list as an Update group
    • Compare WSUS where approvals could be done from a report
  • OOB reports are very basic
  • Reports for Endpoint deployments automatically filter on collections that have Endpoint deployments associated
    • This is both good and bad, mostly good
  • Reports for Microsoft Updates list all collections
    • This is both good and bad, mostly annoying (environment with LOTS of collections)
Remediation
  • Identifying computers failing to apply updates
  • How to redeploy once change window has passed
    • Create new collection of computers to remediate
    • Create new deployments (with reboot supressed)
    • Prevent computers (e.g. servers) retrying and applying updates and/or rebooting once change windows passed (no expire update option)
  • Numerous ConfigMgr Logs are hard to follow and often do not provide actual insight to if/what/when something is failing unless you already know what to look for
    • While vastly improved on previous versions, CM logging is still a mess
  • Computers (i.e. Servers) where Silverlight, and hence the Software Centre, are not installed
    • Visibility of progress, required or applied updates and failures
  • Matching the update “GUID” to an actual update entry
    • Internal GUID not visible through console, only by direct DB query
    • Client logs show GUID, but no other identification of which update it is
OS Deployment
  • Targeting updates for machine being used for reference image
    • No option in console to filter “required” updates by collection
  • Targeting updates to machines during Task Sequence build process
    • e.g. Single “All Workstations” Update group with deployment to OSD collection
    • Collection update speeds to target deployment when Unknown Computer
  • Issues in OSD where client update policy not applied before first update cycle starts
    • Clients attempt to connect to internet during OSD for update scan
  • “Schedule Updates” to inject updates into OS Images offers only very basic search filtering
    • No way to limit to a Software Group list of currently approved updates
    • No way to select only “required” updates
    • Wizard window cannot be resized and individual (un)selection of updates is tedious and unfriendly/error prone

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s