Geeks With Blogs
The Life and Times of a Dev Yes, we're really that weird

NOTE:  BY FOLLOWING THE STEPS HERE YOU WILL PERMENENTLY AND IRREVOCABLY DELETE USERS AND THEIR DATA!!!  MAKE SURE YOU HAVE BACKUPS (PST’S)!

Ugh.

First, I want everyone know know that this was a self-inflicted problem.  I thought I knew what I was doing, but really didn’t know.  I do wish that there were warnings letting you know you were about to ruin your life, but there isn’t, so this is an easy mistake to make.

Our company is moving to office 365 from Exchange 2010.  We want to keep a local on premise version for various reasons, so we’ve set up a hybrid exchange configuration.  We tested it by moving a couple of mailboxes over and everything appeared to be working great.

However, the person helping me quit his job at the company we were outsourcing our IT stuff to.  I thought I knew what he had done, but didn’t.

Here’s our configuration:

  1. Exchange 2010
  2. Office 365
  3. Federated Security for Single Sign On
  4. Active Directory Synchronization

The big mistake that I made was when we were ready to move mailboxes to the cloud, I used the Email Migration tool in the Office 365 Tenant Portal.  If you have a HYBRID exchange configuration with with active directory synchronization, DO NOT USE THE E-MAIL MIGRATION image BUTTON!!! You’ll be hosed if you do.  Instead, use the Remote Move Request feature of EMC or powershell.

Note that the pain will be somewhat mitigated if you leave your MX records pointing at your on premise server, since all mail will continue to be delivered to your on premise server.  However, if you moved it, like I did, you have more work.

Here’s some of the symptoms that you did the wrong thing:

  1. Archive folders (personal archives) don’t appear to have migrated.
  2. Users can log into both portal.microsoftonline.com and access outlook AND your on premises OWA site, and both still work.
  3. Autodiscover sends people, seemingly at random, to your on premise server and to the office 365 servers.
  4. You have mailboxes for users on both your on premise servers and in office 365.

As soon as you realize what you’ve done, you’ll want to crawl under a rock and die.  There is NO easy solution.  After many hours on the phone with Microsoft Tech Support, here are the steps you need to take to recover:

  1. Ensure that you have Rollup 3 for service pack 2 installed on your on premises internet facing hybrid server.
  2. If you’ve moved your MX record, MOVE IT BACK! Smile  Seriously, you’ll have to send it back to your on premise server.
  3. If you moved your MX record, you’ll need to export all new mail that is in office 365 into PST’s.  THIS IS REALLY IMPORTANT!  YOU’RE ABOUT TO PERMENENTLY DELETE DATA! 
    1. To do this, grant a NON-FEDERATED user (and admin user) full access to the mailboxes.  You can do this using EMC’s Manage Full Access permission, or by using a powershell script to do the same.  You’ll need to create the user in office 365.
    2. Set up an outlook profile (I set up an entire VM) and connect it to the user in step one.
    3. Wait forever for the mail to synchronize down to the outlook profile (you probably want to avoid Cached Exchange Mode).  If you’ve got lots of mail, you’re going to have to do this in groups.
    4. Using outlook, export the new mail to a pst (file, open, import, export to file, outlook pst, select the top level mailbox, put a filter in, change the filename to the name of the mailbox, no password, export . . . and yes, that was from memory . . .)
      1. I’d name the file after the user whose mailbox you’re exporting
    5. After PST creation, create a new profile (control panel, Mail, Show Profiles) for an on premise user THAT DOES NOT HAVE A 365 MAILBOX.  Again, you’ll probably need to create the user.
    6. Import the PST’s into the appropriate location in outlook (File, import, import from a file, outlook data file (.pst), select the pst file to import, select the appropriate mailbox from the dropdown list).
    7. Wait forever for mail to sync (you’ll probably want to avoid Cached Exchange Mode).
    8. You’re done.  Wasn’t that fun?
  4. Next, deactivate active directory synchronization in the Office 365 portal and then wait for up to 72 hours for it to complete.  Yup, 72 hours . . . sucks to be you right now.  (It actually took about 7 hours.  I’m not sure if MS made it happen faster because I was already in the tech support queue or not)
  5. Once deactivation is complete, delete the offending users and mailboxes.  You may need to use the Remove-Mailbox powershell command in remote powershell to get rid of the offending mailboxes. (See the additional details below)
  6. Once the users are gone, turn Active Directory synchronization back on and synchronize.
    1. This may take up to 24 hours.
  7. Move the mailboxes the RIGHT way using EMC and Remote Move requests.

I’m at Step 4 right now.  If anything else exciting comes up, I’ll update this post.  I couldn’t find many posts on this “fun” so I thought I’d create one.  I guess others are just more experienced than I am. Smile

 

UPDATE:

So step 5 needs way more detail.  After several more hours on the phone with Microsoft Support, we ended up doing the following to get rid of the bad mailboxes:

Did I mention that by following these steps you’re going to lose data if you don’t have it backed up?

  1. Remove the mailboxes using whatever means you desire (powershell, whatever, doesn’t really matter), which will also delete the users from the system
    1. At this point, you can still recover the users.  If you turn on ADSynch, the users will recover and be re-associated with the deleted mailboxes, which leaves you back where you started.
  2. Using remote powershell for microsoft online, get a list of all online users using the get-msolUser | fl DisplayName, ObjectId command
  3. One by one, delete the users from office 365 using Remove-MSOLUser –ObjectId {Guid from step 2}
    1. If you don’t want to permanently delete your users, you’ll need to get to the operations level at Microsoft.  Yeah, not much you can do about it.
  4. Once the users are deleted that have the duplicate mailboxes, flush the recycle bin using the following command:  Get-MsolUser –ReturnDeletedUsers | Remove-MsolUser –RemoveFromRecycleBin –force
  5. Once this is done, re-enable active directory synchronization, synchronize and your done.  Easy, no?

UPDATE:

This link might be helpful.  It talks about how to delete a user even with ADSynch turned on.

 

BIG Props to FP from Microsoft Support.  Without him I’d have been sunk!

Technorati Tags:
Posted on Thursday, August 16, 2012 8:54 AM | Back to top


Comments on this post: Well That Was Fun (The wrong way to migrate mailboxes in Office 365)

# re: Well That Was Fun (The wrong way to migrate mailboxes in Office 365)
Requesting Gravatar...
A friend of mine had a similar experience with the Migration - good to see that he is not alone :-)
Left by jana on Nov 07, 2012 8:34 AM

# re: Well That Was Fun (The wrong way to migrate mailboxes in Office 365)
Requesting Gravatar...
This happened to me, maybe my scenario was not the same but it started with sharepont online….
The hybrid was set, licensing set, but some users started to use sharepoint online. After this, the Exchange online mailbox was created. My solution was:
1 Set all these mailboxes on litigationHold, export to .pst
2 Remove the sharepoint and exchange plan for each user
3 Disable litigationHold
4 Wait for the mailbox to be removed (check the exchange console)
5 use the exported PSTs with the import-pst command
From here you must be in control now!
Left by Douglas Paz on Jun 24, 2016 12:56 PM

# re: Well That Was Fun (The wrong way to migrate mailboxes in Office 365)
Requesting Gravatar...
We expect that the new LG V30 will be a super success when it launches at the upcoming global event.
Left by Gary Lintz on Feb 25, 2017 3:17 AM

# re: Well That Was Fun (The wrong way to migrate mailboxes in Office 365)
Requesting Gravatar...
Nice post that have you shared way to migrate mailboxes in Office 365
Left by dav on Feb 22, 2018 11:26 PM

Your comment:
 (will show your gravatar)


Copyright © Robert May | Powered by: GeeksWithBlogs.net