Troubleshooting Delivery Optimization
In the past weeks I was heavily involved in some Delivery Optimization (DO) troubleshooting. Luckily we were able to solve the customer issues and I would like to share some of the lessons learned with you.
Things you should know about Delivery Optimization
Before we start with the general troubleshooting, you first need to know HOW DO works and there are some things that are not fully documented by Microsoft.
Update caching != update sharing
The most important information that you need to know is, that not all updates are shared all time. In fact, every device only shares four updates at the same time. Those updates are selected randomly by each client. You might have smaller offices with only a few devices, then it is might possible that all devices sharing the same updates while some updates are not shared at all. This shouldn’t be an issue in larger environments / network but is an important information.
Practically this means, even if an update is available in the DO cache, it doesn’t mean that other devices can download the update.
Device registration
Delivery Optimization is a Microsoft online service, that stores and provides the client information. This means, that if the device can’t reach out the DO service, the device will not be able to detect other peers in the network and the other way round.
Make sure that the devices can reach out to all the URL’s listed here:
https://docs.microsoft.com/en-us/windows/deployment/do/delivery-optimization-workflow
If you validated that the DO service is reachable, you now need to know that the clients will reach out to the DO service every 15 minutes to register the device. Clients that haven’t registered will be removed from the peer list and other clients will not be able to reach out to those devices.
In older Windows versions (1909 LTSC), I saw that some devices haven’t registered and that causes high bandwidth. There is no real way to trigger the registration but restarting the service seems to solve this issue.
Default settings are might to restrictive
There are some default settings in Windows that might blocking the caching of updates. Things like:
- Minimum RAM (default 4GB)
- Minimum file size (default 100MB)
- Monthly upload limit (default 20GB)
- Maximum cache age (default 3 days)
You might need to change those default settings to make sure that those settings fits to your client hardware.
Troubleshooting
Let’s get startet with troubleshooting.
Logging
After Windows 10 2004, Microsoft improved the PowerShell functionality and added the verbose logging function natively. You can now use
Enable-DeliveryOptimizationVerboseLogscommand to enable verbose logging. You will get an output that looks like this:
2022-03-25T15:09:54.2149992Z 1728 176C Info {CCore::_OssRecalcAll} All: 16, FG: 0, FG_sleeping: 0, BG: 0, BG_sleeping: 0, UP: 16
2022-03-25T15:09:54.2150091Z 1728 176C {CCore::_OssRecalcAll} Swarm: ec8a1f96ba65f834f24c5ba1b660b4c10da59e3b, upload state 2, priority: 688, sleeping? 0
2022-03-25T15:09:54.2150122Z 1728 176C {CCore::_OssRecalcAll} Swarm: 1c786def75f88dadcb3e552d0c694a51f07a1fab, upload state 2, priority: 114, sleeping? 0
2022-03-25T15:09:54.2150149Z 1728 176C {CCore::_OssRecalcAll} Swarm: 99283a80dbc065b6eb271df0f11f7a727296e56e, upload state 2, priority: 99.9, sleeping? 0
2022-03-25T15:09:54.2150172Z 1728 176C {CCore::_OssRecalcAll} Swarm: b56cfec6692828d2b240b0978e09fa22ddf23994, upload state 2, priority: 99.9, sleeping? 0
2022-03-25T15:09:54.2150267Z 1728 176C {CCore::_OssRecalcAll} Swarm: 66336e634eff1ead4e953f09165eda1da29298fb, upload state 5, priority: 99.9
2022-03-25T15:09:54.2150289Z 1728 176C {CCore::_OssRecalcAll} Swarm: 0ae328c2e2f04b7dc6c58ae3e733b8f1b6d06368, upload state 5, priority: 99.9
2022-03-25T15:09:54.2150311Z 1728 176C {CCore::_OssRecalcAll} Swarm: 2669b83a60c84e93f8f0514ffedea6f650df6199, upload state 5, priority: 99.9For a better understanding here is what the things mean:
| All | All available updates |
| FG | Foreground downloading (e.g. open the settings app and click on download update) |
| FG_sleeping | Paused downloads |
| BG | Background downloading |
| BG_sleeping | Paused downloads |
| UP | All updates that are currently available for sharing |
In the first line you’ll see the information about the current Delivery Optimization status.
In addition to the listed settings, there is something in the logs named “swarms” – which is basically the Update File ID which will you found under C:\Windows\SoftwareDistribution.
State 2 = “Caching” and state 5 = “Complete”.
Priority numbers higher than 100 indicate active peer connections – those swarms remain in state 2 as long as they have peers.
In my example you can see, that two updates are offered to peers while other updates were not requested and shared.
For a quick overview over the current status in the logs, you can use:
Get-DeliveryOptimizationLogAnalysisThis will provide an output like this:
Getting the current status
To get the current status of the updates, you can use
Get-DeliveryOptimizationStatusThis output will provide the current status of the updates that were used / shared by Delivery Optimization.
In the Get-DeliveryOptimizationStatus output, Status = Caching means “advertising for peers” (shareing this update to peers). Status = Complete means ready to share but not currently doing so.
A great feature that was added to Windows 10 2004 is the new -peerinfo parameter for the Get-DeliveryOptimazation command. You can run:
Get-DeliveryOptimizationStatus -PeerInfoAnd it will return the current active peer connections.
Differences between Foreground and Background Download
If you want to make sure that the update gets downloaded via P2P, you need to delay the update download via HTTP / internet. With the settings:
Delay background download from http (in secs) DODelayBackgroundDownloadFromHttp
Delay foreground download from http (in secs) DODelayForegroundDownloadFromHttp you can configure the delay in seconds. This will affect how the update installation will behave. Downloads from Windows Updates, Microsoft Store, and optional Windows features may appear stuck until the delay period has ended. They might show the well known “downloading – 0%” state. You can’t force the update to get downloaded immediately. The only option how you can “force” the download is, to have another device sharing the update.
This is where foreground and background download is making a difference. A shorter – or no – foreground delay means you’ll not have to wait for downloads when you have the Microsoft Store or Windows Update window open. As long as the window is open, you use the foreground setting. In other words, if you close the window, the system uses the background setting.
Empowering customers in client management since 2012.
Empowering customers in modern management since 2018.