Troubleshooting Delivery Optimization

• By -Grischa Ernst

Troubleshooting Delivery Optimization

• Posted on 2022-04-29
• Posted in Microsoft, Windows 11, Windows Desktop
• 1 Comment

In the past weeks I was heavily involved in some Delivery Optimization (DO) troubleshooting. Luckily we were able to solve the customer issues and I would like to share some of the lessons learned with you.

Things you should know about Delivery Optimization

Before we start with the general troubleshooting, you first need to know HOW DO works and there are some things that are not fully documented by Microsoft.

Update caching != update sharing

The most important information that you need to know is, that not all updates are shared all time. In fact, every device only shares four updates at the same time. Those updates are selected randomly by each client. You might have smaller offices with only a few devices, then it is might possible that all devices sharing the same updates while some updates are not shared at all. This shouldn’t be an issue in larger environments / network but is an important information.
Practically this means, even if an update is available in the DO cache, it doesn’t mean that other devices can download the update.

Device registration

Delivery Optimization is a Microsoft online service, that stores and provides the client information. This means, that if the device can’t reach out the DO service, the device will not be able to detect other peers in the network and the other way round.
Make sure that the devices can reach out to all the URL’s listed here:
https://docs.microsoft.com/en-us/windows/deployment/do/delivery-optimization-workflow

If you validated that the DO service is reachable, you now need to know that the clients will reach out to the DO service every 15 minutes to register the device. Clients that haven’t registered will be removed from the peer list and other clients will not be able to reach out to those devices.
In older Windows versions (1909 LTSC), I saw that some devices haven’t registered and that causes high bandwidth. There is no real way to trigger the registration but restarting the service seems to solve this issue.

Default settings are might to restrictive

There are some default settings in Windows that might blocking the caching of updates. Things like:

• Minimum RAM (default 4GB)
• Minimum file size (default 100MB)
• Monthly upload limit (default 20GB)
• Maximum cache age (default 3 days)

You might need to change those default settings to make sure that those settings fits to your client hardware.

Troubleshooting

Let’s get startet with troubleshooting.

Logging

After Windows 10 2004, Microsoft improved the PowerShell functionality and added the verbose logging function natively. You can now use

Enable-DeliveryOptimizationVerboseLogs

command to enable verbose logging. You will get an output that looks like this:

2022-03-25T15:09:54.2149992Z 1728 176C     Info {CCore::_OssRecalcAll} All: 16, FG: 0, FG_sleeping: 0, BG: 0, BG_sleeping: 0, UP: 16
2022-03-25T15:09:54.2150091Z 1728 176C          {CCore::_OssRecalcAll} Swarm: ec8a1f96ba65f834f24c5ba1b660b4c10da59e3b, upload state 2, priority: 688, sleeping? 0
2022-03-25T15:09:54.2150122Z 1728 176C          {CCore::_OssRecalcAll} Swarm: 1c786def75f88dadcb3e552d0c694a51f07a1fab, upload state 2, priority: 114, sleeping? 0
2022-03-25T15:09:54.2150149Z 1728 176C          {CCore::_OssRecalcAll} Swarm: 99283a80dbc065b6eb271df0f11f7a727296e56e, upload state 2, priority: 99.9, sleeping? 0
2022-03-25T15:09:54.2150172Z 1728 176C          {CCore::_OssRecalcAll} Swarm: b56cfec6692828d2b240b0978e09fa22ddf23994, upload state 2, priority: 99.9, sleeping? 0
2022-03-25T15:09:54.2150267Z 1728 176C          {CCore::_OssRecalcAll} Swarm: 66336e634eff1ead4e953f09165eda1da29298fb, upload state 5, priority: 99.9
2022-03-25T15:09:54.2150289Z 1728 176C          {CCore::_OssRecalcAll} Swarm: 0ae328c2e2f04b7dc6c58ae3e733b8f1b6d06368, upload state 5, priority: 99.9
2022-03-25T15:09:54.2150311Z 1728 176C          {CCore::_OssRecalcAll} Swarm: 2669b83a60c84e93f8f0514ffedea6f650df6199, upload state 5, priority: 99.9

For a better understanding here is what the things mean:

All	All available updates
FG	Foreground downloading (e.g. open the settings app and click on download update)
FG_sleeping	Paused downloads
BG	Background downloading
BG_sleeping	Paused downloads
UP	All updates that are currently available for sharing

In the first line you’ll see the information about the current Delivery Optimization status.
In addition to the listed settings, there is something in the logs named “swarms” – which is basically the Update File ID which will you found under C:\Windows\SoftwareDistribution.

State 2 = “Caching” and state 5 = “Complete”.
Priority numbers higher than 100 indicate active peer connections – those swarms remain in state 2 as long as they have peers.
In my example you can see, that two updates are offered to peers while other updates were not requested and shared.

For a quick overview over the current status in the logs, you can use:

Get-DeliveryOptimizationLogAnalysis

This will provide an output like this:

Getting the current status

To get the current status of the updates, you can use

Get-DeliveryOptimizationStatus

This output will provide the current status of the updates that were used / shared by Delivery Optimization.
In the Get-DeliveryOptimizationStatus output, Status = Caching means “advertising for peers” (shareing this update to peers). Status = Complete means ready to share but not currently doing so.

A great feature that was added to Windows 10 2004 is the new -peerinfo parameter for the Get-DeliveryOptimazation command. You can run:

Get-DeliveryOptimizationStatus -PeerInfo

And it will return the current active peer connections.

Differences between Foreground and Background Download

If you want to make sure that the update gets downloaded via P2P, you need to delay the update download via HTTP / internet. With the settings:

Delay background download from http (in secs)	DODelayBackgroundDownloadFromHttp	
Delay foreground download from http (in secs)	DODelayForegroundDownloadFromHttp	

you can configure the delay in seconds. This will affect how the update installation will behave. Downloads from Windows Updates, Microsoft Store, and optional Windows features may appear stuck until the delay period has ended. They might show the well known “downloading – 0%” state. You can’t force the update to get downloaded immediately. The only option how you can “force” the download is, to have another device sharing the update.
This is where foreground and background download is making a difference. A shorter – or no – foreground delay means you’ll not have to wait for downloads when you have the Microsoft Store or Windows Update window open. As long as the window is open, you use the foreground setting. In other words, if you close the window, the system uses the background setting.