Public Folder Migration to Exchange Online (500k Folders) Part 1 — Preparing for Migration (1/2)
Quick Jump Guide
Part 1 — Preparing for Migration (1/2)
Part 2 — Preparing for Migration (2/2)
Part 3 — Starting the Migration Request
Part 4 — Monitoring and Fixing the Sync (1/3)
Part 5 — Monitoring and Fixing the Sync (2/3)
Part 6 — Monitoring and Fixing the Sync (3/3)
Part 7 — Completing the Batch
Part 8 — Testing & Unlocking Public Folders
Part 9 — Post-Migration Issues (Hierarchy Update)
Part 10 — Post-Migration Issues (Access Denied Deleting)
Part 11 — Post-Migration Issues (Add/Remove Permissions)
Part 12 — Decommissioning on-premises Public Folders
Part 13 — Troubleshooting Hierarchy Issues
So you are looking to migrate a large on-prem Public Folder estate to Exchange Online. The first question to you have to ask yourself is, are you insane?
Microsoft have recently made this process easier as they now officially support 500k folders, however the migration limit is still 250k (100k for Exchange 2013/2016). When we did our migration this was not the case.
The Background
We had very recently migrated Public Folders from Exchange 2007 to Exchange 2010 and a result we had replicas still on the on old Ex2007 servers. Now if you read the Microsoft migration TechNet article it gives you a few minor prerequisites to follow — there are more which I’ll go through!
If you are migrating this amount of folders, it’s very likely you will have Enterprise Vault (or some other archiving solution in place). For this example I’ll use Enterprise Vault. It is imperative that you size the migration correctly with the true folder sizes, otherwise you’ll have issues when you come to restore all archived emails.
By the time we migrated we had nearly 490k folders and over 26TB of data (including EV data).
On the weekend 5–7th May 2018 we successfully cutover 484k folders to Exchange Online Public Folders. Let the fun begin…
Prerequisites
Step 1 — Replicas
You can if you wish migrate the data with the replicas in-place, the choice is entirely yours. The question you need to ask yourself is, do you have up-to-date replicas?
Just say you have two replicas, Server A and Server B. When you configure the migration endpoint you have to specify a source server. All the data you send to ExO will be sent from that server. At the point the migration starts, the Microsoft Replication Service (MRS) pulls the data from your source server and if you have replicas it’s going to randomly choose one of them. If you don’t have up-to-date replicas then you are at serious risk of not migrating data…
If you are choosing to keep replicas then I strongly recommend you check the Item Count on all replicas to ensure they are the same.
Also under the Get-PublicFolder cmdlet, check the UseDatabaseReplicationSchedule and the ReplicationSchedule values. We had loads of folders configured never to replicate and as a result, data was missing on the replica.
We chose to remove all replicas and only migrate from Exchange 2010.
Another factor in this move was because we had Public Folder servers all over the world and as you can only ever had a single source server for the migration, if you leave it like that you will be pulling data over your network.
The below script should help you compare the data on Ex2007 and Ex2010. In the below example I have two servers on both versions.
Step 2 — Invalid Characters
This one is a huge pain and something that will need to be continuously monitored throughout your migration… users love to create to folders with \, / and all sorts of weird and wonderful formatting from Word — yes our users copied text from Word to paste into the folder names.
There is also another good reason to clean up your folder names. When you’ve created your migration mapping file, the catch-all mailbox is Mailbox1. So if you have any folders that cannot be read, they will all migrate into Mailbox1 and you are at risk of bloating that mailbox.
The below script is what I wrote to fix ours. I found \, /, <, >, _-_.
The elongated dash in Word (em-dash) looks identical to a dash, however it’s not and will cause PowerShell commands to fail.
I also found a space either side of a * causing issues. So I removed all these…
Step 3 — Permissions (Mass Changes)
This one is always forgotten but it’s so important. During our migrations (yes it took three attempts to migrate), this is the one that hurt us the most. As the syncing progressed and we passed 300 mailbox synced, we started to more and more “Transient Failures”, “Server Unhealthy”, “TargetServer_DiskLatency”, “TargetServer_CPU…” etc…
The issues (in this case) was down to the MRS just being unable to handle the sheer amount of changes we were making to your estate. Our provisioning teams were deleting leaver accounts who had access to vast numbers of Public Folders, all of these changes had to synced up to ExO — in one month alone we had over 70k folders that their ACLs changed.
We thought long and hard about what we could do to limit the number of changes, however in the end we decided to simply stop deleting leaver accounts until after the migration. This is harder then it sounds and it does depend on your organisation.
Step 4 — Permissions (Unresolved SIDs)
Following on from Step 3. Annoyingly we found this extremely late and as a result it broke one of our syncs, forcing us to delete everything and restart!
Microsoft made a change to the syncing of ACLs in 2016 which causes all permission errors in Public Folders to be flagged as Bad Items. While this is not a bad thing, during a Public Folder migration with nearly 490k folders be prepared for a lot of errors to be shown and unless you effectively set the Bad Item Count limit to “Unlimited” your migration will simply fail.
The issue is around unresolved SIDs that are left in the ACLs once an account has been deleted — how many of you actually go through and tidy these up?
IMPORTANT — This screenshot below shows some folders that have an unresolved SIDs in the ACLs. I strongly recommend you take action to fix these or at least reduce the number, as after you migrate you will have corrupted ACLs. Take a look at Part 10 of my series to see the issues for yourself.
As we didn’t want to run the risk of losing data, we fixed the majority of this issue. Bill Long has a good script here to remove all unresolved SIDs. The issue we found is that it won’t remove SIDs that are left with a permission settings of “None”. Luckily for us this was only about 20k out of a whopping 6.5 million entries!
You want to avoid mass changes after your sync has started. If you don’t clean-up the unresolved SIDs before you start the sync, you will most likely have to set the BadItemCount limit to unlimited — doing this you could be missing data once you cutover. The choice is yours…
If you run the clean-up after it’s started, then depending on how many items it deletes, you are at real risk of running into throttling limits and timeout settings with Microsoft. The end result will be you having to restart your sync from scratch.
Step 6 — Mail-Enabled Public Folders
If you read the migration TechNet article it simply says to download and run the Sync-MailPublicFolder script, job done… Not quite.
If you have a small environment or a fairly new Public Folder estate then yes it would be easier. If however you have had Public Folders for years, and have come from Exchange 2000/2003, you are in for a bit of pain here.
The Issues
You are likely (as we had) to have loads of mail-enabled Public Folders. This is because back in the old days, Microsoft thought it was a great idea to automatically mail-enable every folder. Luckily in Exchange 2007 this was stopped!
Out of our 490k folders, we have in excess of 50k folders that were mail-enabled. Of which only about 20 were actually used (verified by message tracking over a years worth of logs). We took this opportunity to clean-up the estate and remove all non-used mail-enabled Public Folders.
Now when you use Disable-MailPublicFolder you think it does what it says, if only life was that simple. Due to the folder becoming mail-enabled in Exchange 2000/2003, it’s a different attribute to what Disable-MailPublicFolder uses. The end result is you may have 20 folders that are enabled (checked in PowerShell), however the MESO OU (Micorsoft Exchange System Objects) will have 50k objects in it!
When you run the Sync-MailPublicFolders script, you will get thousands of errors as;
- The script will check against the objects in the MESO OU, not the folders that are enabled. So the script will attempt to create 50k objects in Azure because it believes you have that many.
- Also, another nugget of pain… is back in Exchange 2000/2003 the alias’ of the folders have spaces in them etc. These are no longer supported in Exchange 2007+. Therefore the script will fail to created the objects in Azure on thousands of folders because the alias contains a space and others.
- When you run the script it will generate a report, then you will know how good or bad your situation is. You will likely see loads of “Cannot bind parameter ‘Alias’ to the target. Exception setting expression “xxxxxxx” isn’t valid”
The Solution
1. Run an export in PowerShell to collect all mail-enabled Public Folders (you will need the alias). You can collect information by using the Get-PublicFolder cmdlet.
2. In AD Users & Computers open the MESO OU. You’ll need to add the column ‘Exchange Alias’ to the list.
3. Be careful in MESO OU as it does contain “msExchSystemMailbox” and “User” types as well as “Public Folder”. You can safety delete all unwanted “Public Folder” that you don’t see in the export you took under Step 1, as they are not mail-enabled in Exchange.
There are some scripts out there that can do this automatically for you (Bill Long has some published), however if you want control over it and ensure it’s done safely for your own peace of mind, do it manually.
Once you’ve cleaned it up, rerun the Sync-MailPublicFolders script and examine the export. You are likely to have to run this multiple times before you get this right.
WARNING — Without this being correct, Mailbox1 will fail it’s sync.
If it does fail during the sync you can recover from it and resume it again. I ended up having to manually create objects it was failing against. You will be given the “OnPremisesObjectId” during the sync failure so you use this to map them. Providing you get the Sync-MailPublicFolder script running with no errors, you should be OK.
