Potential USMT Errors & Resolution

, , , , , , , , ,
Potential USMT Errors

Failure when “trying to reboot into WinPE”
appears on the Task Sequence UI (forgot to take screenshot)

 Potential USMT Errors & Resolution


      Check the “SMSTS.LOG” “C:WindowsCCMLogs”


         In the screenshot below look for a line like “Unable to find a volume that is suitable for staging the boot image” in the log file 


Potential USMT Errors
       This is typically for 2 different reasons.
      1.  The drive is locked by bitlocker encryption and you need to first disable bitlocker in the task            sequence. once this is done then the task sequence engine can identify the drive to stage the        boot image locally
      2.   This can also be because the drive is still undergoing the encryption process. You must wait             until the drive is fully encrypted before you can execute this task sequence from windows.
       To verify the drive is finished encrypting launch powershell as an admin and type “Get-BitlockerVolume” and do not attempt to run the installation until the “Encryption KeyProtector Percentage” is are 100%  
Potential USMT Errors
NOTE: it is possible you receive this error when you are trying to stage the boot image onto the disk and the disk cannot be read b/c of a different encryption software locks the disk. For example if you are running Dell Credant you must be logged into the system then the TS Engine will be able to read the disk.
To identify Dell Credant systems via SQL…this is a hybrid of one my queries that I use in my production environment but you can modify it to abosrb only bitlocker information by commenting out not needed parts

SELECT Distinct

v_R_System.Name0 AS System,

Computer_System_DATA.Model00 AS [System Model],


V_R_System.AD_Site_Name0 AS [AD Site],

CASE V_R_System.Build01

When ‘6.1.7601’ THEN ‘Windows 7’

WHEN ‘10.0.14393’ THEN ‘Win 10 v1607’

WHEN ‘10.0.15063’ THEN ‘Win 10 v1703’

END AS [Operating System],

CASE V_R_System.Client0

When ‘0’ THEN ‘No Client’

WHEN ‘1’ THEN ‘Client Installed’

END AS [Client],

v_GS_ENCRYPTABLE_VOLUME.DriveLetter0 AS [Drive Letter],

–v_GS_ENCRYPTABLE_VOLUME.ProtectionStatus0 AS [Protection Status],


WHEN ‘0’ THEN ‘not encrypted’

WHEN ‘1’ THEN ‘encrypted’

WHEN ‘2’ THEN ‘Encrypted Requires Pin’

END AS [Bitlocker Status]



INNER JOIN v_R_System ON v_GS_ENCRYPTABLE_VOLUME.ResourceID = v_R_System.ResourceID

INNER JOIN Computer_System_Data ON V_R_System.Name0 = Computer_System_Data.Name00




AND Computer_System_DATA.Model00!= ‘VMware Virtual Platform’

AND Computer_System_DATA.Model00!= ‘Virtual Machine’

–AND v_GS_ENCRYPTABLE_VOLUME.ProtectionStatus0 = ‘0’ –for not bitlocker encrypted systems–

–AND V_R_System.Name0 = ‘P620268’

AND ___System_INSTALLED_SOFTWARE0.ARPDisplayName00 like N’Credant_WindowsShield%’


Potential USMT Errors


Failure when trying to connect to SMP Share
This failure is more often seen when trying to rerun on a failed system (can be seen during backup or restore part of the process) 
Potential USMT Errors
For this we corrected the issue is to open PowerShell and running the following.


Remove-Item -Path ‘HKLM:SOFTWAREMicrosoftSystemCertificatesSMSCertificates*’ -force; restart-service ccmexec

I would also recommend opening registry location to verify this has successfully been deleted.
Once this runs successfully you should then be able to re-run the task sequence successfully.
File Not Found: 
of course there is the standard make sure your commands are typed correctly. In the example below we see a file not found error. Make sure you type out your file names correctly etc.
Potential USMT Errors
Potential USMT Errors
NOTE: I have seen cases where a variable is set for Packages, but it does not always translate, so I tend to just hard code the package ID when I set restore/capture options into a variable see the example below
Potential USMT Errors
Connection to SMP Refused: 
Make sure when you being your deployment strategy that you plan for an adequate number of connections to the SMP. The default for this I believe is 100 connections, but that does not mean concurrent connections. The criteria that goes into the count is any established connection (completed or in progress) within the your deletion policy time period. If you have a problem where the connection is actively refused by the SMP you should increase the max allowed connections you have configured.
Potential USMT Errors
I tried to create as many Potential USMT Errors as I could think of to help out the community. If I  encounter anymore or can think of new ones I will add them to this blogpost.

USMT Estimation Report

, , , , , ,

USMT Estimation Report

USMT Estimation Report – One of the deliverable items at a customers site was to identify the amount of data each machine would would have to backup. This is important data to capture to help plan your estimated migration times as well as identify systems that will not be able to successfully perform a backup. This data is not something that SCCM will automatically absorb, but Jason Sandy already has a solution for that. We used his script here with a few mods for our environment we were able to capture this information.

let me point out that if you look inside of the MDT Toolkit for ZTIUserState.wsf you will see that it is estimated to need 1.1 times the size of the data you are trying to catpure. This is something that was pointed out from a co-worker.
USMT Estimation Report
OLD Report Visuals
USMT Estimation Report

New Report Visuals


USMT Estimation ReportUSMT Estimation Report


The new report allows you to look up specific computers, interactive sorting, better visuals, graphs, and more efficient SQL logic.

you will have to modify the CASE WHEN statements to fit your own environment. Please do some testing and then modify for the “USMT Only Time Estimate” column. Several factors go into consideration for determining this value. These are things such as server specs, amount of systems running the task concurrently, bandwidth, etc.

THE RDL: https://gallery.technet.microsoft.com/USMT-Space-Estimator-2c5d728b

Rebuild site servers without redistributing content over the WAN

, , , , , , , , , ,

Rebuild site servers without redistributing content over the WAN:

Outlining the Project


                In order to support the Windows 10 Migration project for this customer the hierarchy needed to be upgraded to a level to support deploying/managing of windows 10v1607. The current level of the environment is 2012 R2 SP1 and we will move to CB1606 and upgrade to CB1610. During this time I took the opportunity to simplify the architecture as there were dozens of unnecessary secondary sites. These locations would have the site replaced with a single distribution point. There was no getting around these secondary sites as the pre-req check would fail due to the unsupported version of SQL server express installed on the systems. SCCM does support in place upgrade of SQL on a site server but that is limited to full SQL and not the express version. The best practice would be to spin up the new servers, configure them as Pull Distribution Points, pull the content from the old servers, and then smoothly transition between the two. In this environment, however, we were not provided with new servers to use, and we were faced with the added difficulty of prohibitively slow WAN connections, requiring us to come up with an alternative solution.




Best Practice/Worst Practice


                What we came up with was a series of PowerShell scripts that eventually evolved into the Distribution Content Migration Tool-Kit module.  This module takes queries WMI to pull a list of all content assigned to a Distribution Point and runs on the Distribution Point to create prestage packages for all of that content.  Once the required roles have been removed, reconfigured, and added back to the server, the module then allows all of that content to be reassigned to the Distribution Point and subsequently extracted to complete the migration.



Prerequisite Components


                Because portions of this module had to run locally on our Secondary servers, we needed the Configuration Manager libraries loaded as well as the most recent version of the Windows Management Framework.  While it is possible to just copy over the required DLL files and import them into PowerShell, we did want to stick with something more reliable and consistent, so we installed .NET 4.5, WMF 5, which is required to install the Configuration Manager Console. These updates were all copied to the server and then installed to the clients with a quick PowerShell query to find all servers with the Secondary Site role installed piped into a Copy command.  A few reboots later, and the servers were primed to migrate.



Before Removing Roles


                Modify the site assignment and site server referenced on your boundary groups to talk to another site system server. This is set for your boundary by the boundary group applied to it. I changed the site assignment to my primary site server. I changed my site system servers to the MP on my primary and I left the DP blank as this was only an expected outage of less than 2 hours. Once the conversion is complete I will place my DP here for the boundary group. If you do not want to leave that blank you can use the closest DP that has open ports for communication. Remember do not remove any of the roles until we create the pre-staged content locally on the site server.



Prepare server for role removal


                The tool-kit is made up of four separate scripts written out as the functions Get-DPContent, Prestage-Content, Restage-Content, and Extract-Content.  



Script Step 1: Get-DPContent


                The Get-DPContent function requires you to specify a Distribution Point and will pull a list of every piece of content SCCM has assigned to it.  It returns an unformatted array of WMI SMS_PackageBaseClass objects which can look a little daunting, but can be easily formatted for reporting or further processing.


Title: A single DP Content Info object - Description: This is the raw dump of the information returned when you get the SMS DPContentInfo class


Figure 1 – Raw data produced by the SMS_DPContentInfo Class




Figure 2 – Table-formatted values for just PackageID and Name


                I’ve seen some guides online use Get-CMDeploymentPackage to get package info, but I’ve found that WMI works up to 3x faster when querying large data sets, and it runs without needing a connection to a CMSite drive, so it’s become my preferred method.



Script Step 2: Prestage-Content


                This function does the actual work of creating a prestage PKGX file based on the package ID you provide it, the Distribution Point that holds the content, and the location of the folder that will store the package for later use.  The ConfigurationManager PowerShell module actually comes with a cmdlet called Publish-CMPrestageContent, but because that cmdlet requires you to specify the type of item you’re prestaging, we wrote this function to make the WMI call, examine the package type, and issue the correct command for you.  For one off package prestaging, this is still far and away superior to going through right click menus, but where this function shines is when it’s used in conjunction with the Get-DPContent function. 



Figure 3 – Prestaging a single package





Figure 4 – Prestaging multiple packages via For loop


Remove roles from SCCM console.


After you successfully create the pre-staged content locally on the server we can move forward.   In our case, we needed to remove all roles assigned to the server and only add the DP role back. This action required us to remove the DP, MP, and SUP. After these roles are removed we can go ahead and remove the site server.


Note: when you are decommissioning the secondary site this will also uninstall the DP role naturally. Out of habit I recommend to remove all soles prior to uninstallation of the Site.


  • Validated via the distmgr.log on the primary site server
  • Validated by no longer being seen in distribution point configuration status in the console





Removing the site server


                From the console Administration > Expand Overview > Expand Site Configuratoin > Sites > Select the site and “Delete”. This will create a new dialog box and it is important that you read the differences btwn uninstall and remove. We will choose to uninstall.





From here you can see the state of the secondary site server has moved to “Deleting”



How to monitor and the site server uninstallation process


On your secondary site server you can monitor this from C:ConfigMgrSetup.log. The site server uninstallation process is roughly as follows.


1. ConfigMgr2012 Setup is started by system with command line options /deinstall / msg2parent /nouserinput

2. Information is checked, this will be things such as the following. FQDN, OS is verified, Checks for existing setup information, existing SQL information, existing configmgr installation and version number, etc.



3. removes SQL alias for sccm

4. Starts uninstallation of secondary site by first cleaning up SQl server replication data, start uninstallation of local dp (if applicable) Remove content SCCMContentLib, SMSPKG, SMSPKGF$, SMSSIG$ directories from the server. The process will also move through list of all SCCM Services and stop/uninstall them if present and then stop WMI



After services/connections are removed you will see a number of redlines in the log file. This is only b/c connection can not be established which is expected right after stopping WMI




5. Connect to database, drop schema SMS_SiteSystemToSQLConnection, drop database, and uninstall SQL (if applicable)



NOTE: ONLY If your admin installed SQL instead of letting SCCM perform the uninstall action during site install you will see this message



6. Attempt unregister list of Binaries



7. Attempt delete remaining folders/files from within the configmgr installation directory



8. remove registry keys, restart WMI, and other services then complete uninstallation of Configuration Manager Site.



NOTE: After site is uninstalled you might run into issues where the secondary server is still showing “Deleting” this can be resolved by my other blogpost HERE where I had to use the hierarchy maintenance tool.



Remove unnecessary items


Start by uninstalling SQL (if applicable) the only time you will have to uninstall this is if the admin installed / configured SQL on the secondary site instead of letting SCCM do this action. Remove any other roles/features that are no longer needed. For this environment I also removed WSUS as it is no longer needed nor will be able to patch win 10 when the server is on server 2008R2 w/ WSUS 3.0
since we uninstalled SQL this freed up two extra drives on the machine that stored the database and the log files. These were then reclaimed by the storage team. For the entire project this allowed 1200 GB to be reclaimed. Uninstall the sccm console as it is no longer needed.



Before reinstalling DP


I have performed a number of conversions in the past where there were problems reinstalling the DP role. Typically this process goes just fine but in rare instances I ran into issues and have to completely remove the client / delete from database / rediscover / reinstall client / reinstall role, so I recommend doing the following.  


  1. Completely uninstall SCCM Client
  2. Remove the following registry HIVE “HKEY_Local_MachineSoftwareMicrosoftSMS”
  3. Reinstall SCCM client



Reinstall DP Role


                There should not be any additional configuration needed as this server previously had the DP role. Make sure this is not configured for pull dp and you enable this for pre-staged content. You can track the installation process in 2 logs: DISTMGR.LOG on the primary site server installation path and and SMSDPPROV.LOG located SMS_DP$smslogs. You can also Track through monitoring on the console


Track though the distmgr.log on your primary server and smsdpprov.log on the DP.





Script Step 3: Restage-Content


                The Restage-Content function crawls through the list of packages we saved and tells SCCM to re-assign the content.  While the Prestage-Content and Extract-Content functions need to be run on the DP you’re migrating, this command, along with Get-DPContent, can be run from any computer as it is only interacting with meta-data on the SCCM server. All you need to specify here is the location of the prestage files and the name of the distribution point they’ll be assigned to.  



Figure 5 – Content restaging in progress



Figure 6 – Existing content will not waste time trying to reassign



Script Step 4: Extract-Content


This function takes input in the form of the prestaged content location and uses Microsoft’s ExtractContent.exe program to manually add them to the content library.  While we messed with the idea of having it prompt you for the location of the ExtractContent.exe utility, we eventually decided that it was simpler to just require the exe file to be in the same directory as the prestage packages.  This takes a while to run depending on the quantity and size of your PKGX files, and in the event that some do not sync properly when you check your Distribution Point Configuration Status messages, you can run this function again, and it will only try to extract content that isn’t flagged as State=0 (successful).



Figure 7 – Content extracting one package at a time




Final Product


In conclusion, while there are some tools and packages out there that are more “double click and go” automation, we’ve found that every environment is too different for one solution to work for everybody. With that in mind, we focused on developing a toolkit that could be adjusted and tweaked for any environment and then used that to simplify our infrastructure to make life easier for the local admins.  Our next step in this project is to begin the upgrade from SCCM 2012 R2 SP1 to Current Branch 1606, and eventually to Current Branch 1610.    When all is said and done, we’ll have converted nearly three dozen secondary sites, all with their own Distribution Point, Software Update Point, and Management Point roles over to just Distribution Points.  In addition to saving several hundred GB of content distribution traffic this conversion will have eliminated much of the unnecessary SQL and WSUS traffic we saw. The storage team was also thrilled to realize 1.2TB of storage can now be reclaimed.
ALSO SEE : Lockstate Object