Rethinking ChangeWindows

As time has moved on, the Windows Insider Program has changed, in much ways for the worst, but the way ChangeWindows reported on that has stayed the same. And now, it has become unsustainable.

Microsoft’s blog posts announcing new Windows Insider builds have always been unreliable. Back in the day, this was because little changes used to be left out and the focus in these announcements went to the major marketable changes. And all the marketing language in these blog posts didn’t help either. Regardless, put 2 similar devices next to each other, install the new build on one of them and then compare as much as you can and it was pretty easy to know what was new.

That changed.

As time went on, Microsoft began using features flagging/AB-testing or whatever you want to call it. As a result some users get new features while others don’t. So now we’re in a situation where a machine on an older build might have a new feature while the machine running the more recent build might not. Or a situation where a feature only pops up long after installing a new build.

As a result, blogs have become even more unreliable: if this is a feature worth mentioning, Microsoft often only mentions it multiple builds after people began to see it for the first time. Meanwhile, to give their blog posts some more content (I guess), Microsoft has begun mentioning changes in apps that aren’t part of these builds or have nothing to do with Windows whatsoever (PowerToys, for example), and often these changes might have rolled out weeks if not even months before, or have already gone public. This information, to us as a build-to-build comparison, is irrelevant.

As now both Microsoft’s blog posts and build-to-build comparisons have become extremely to unreliable, we have to realize that so have our changelogs. We depend on people mentioning that a feature has popped up in a build, and then we have to — somehow — figure out when that started. We can’t do that.

So it is time to rethink the way ChangeWindows reports on these things. A build-to-build changelog no longer makes sense.

Thus, we’re going to move to a release-to-release changelog. We’ll track all changes on 1 single page for each release. Not only will this make our final changelog more reliable than they are today, they make the progress as new releases go along more reliable too. This also means that we’ll stop tracking changes in cumulative updates to production builds (19H2 being the exception), we haven’t been doing that for a while now, but now we’re ripping it out entirely. Instead, under each release changelog, you’ll find the release history of that release.

This means that what is now the “Milestone”-page will basically take over the current “Build”-pages. For Known issues and Fixes issues, this means that only those of the last released build will be tracked up until the final release of a milestone. Beyond that, not much will change.

This is not a change that is happening right now. We’re working on updating the website to reflect these changes.