http://marcdekeyser.com

Exchange 2013: The store process

Exchange 2013, the end of the store as we know it… In the years leading up to the development of Exchange 2013 we had a process called store.exe. Many an administrator cursed at this process for taking up so much memory but yet, things were good. But they could be better!

The store.exe prior to exchange 2013 was used to run all databases and keep reads and writes in the memory as much as possible. Exchange 2013 sees a continuation of keeping reads and writes in the memory (for obvious performance reasons) but some structural changes were made to address the lack of scale and to remove the single point of failure the "old" store.exe was.

The new store.exe

First of all, exchange 2013 introduces managed code in to the store process. This allows for more optimizations and easier debugging when needed. It also allows to scale out over more processors (in Exchange 2010 you would not see much performance increase when going over 8 cores and even a decrease when going over 12 cores. This was due to crosstalk…) and use process isolation, aka one store process per database.

The managed store actually has 2 or more processes running at all times! One global overseeing store.server.exe process and one store.worker.exe process for each database on the exchange server.
Now that global mast (store.server.exe) is a supervisor for each store.worker.exe process. It deals with lifetime management of those processes, deals with worker crashes and the worker processes absolutely depend on the store.server.exe process.

The store.server.exe process also manages the ESE cache memory allocation and the biggest change for exchange 2013 (on top of all other changes) in memory management is definitely that there is a maximum limit to the memory used by the store process. Only 25% of the total memory in a server can be allocated to the ESE cache based on the active and maximum active databases values. Do note that passive databases get only 20% of the memory allocated compared to an active database.

I already mentioned that there will be one store.worker.exe for each database on a server so let's skip repeating that… The worker processes (if you have more than one database on a server) manage client request sand are responsible for registering an RPC endpoint for its Database GUID. Other than that they don't have much responsibility or decision power, this is all overseen by the global store server process.

Performance improvements

Throughout the different versions of exchange (2003 up to 2013) the exchange product group have made significant efforts to reduce the IOPS footprint of databases. Now, in Exchange 2013, there is a 97% reduction in IOPS compared to Exchange 2003! Now there is a massive number to throw around!!

How would be the obvious question to ask here! In Exchange 2013 (and 2010 for that matter) the focus was to replace as many random IO with sequential IO to squeeze every bit of performance from the storage.

On the physical contiguity level there was a redesign of the leaf pages in the databases resulting in fewer and larger sized IO spanning multiple pages.

For the logical contiguity a folder, message and attachments table was introduced on a per mailbox level. Each of the message tables consists of physical columns and property blobs. Having a high level of messages per page resulted in fewer large IO to retrieve large number of messages for views.

Lastly the temporal contiguity was improved by updating the views and indexes only when they were being accessed by a user. This resulted in fewer and larger sized IO being performed together.

Overall the result of all these optimizations is a "sequentialish" IOPS pattern with massive improvements gains over older versions.

Scheduled maintenance

Mainly a leftover of the past, scheduled maintenance has been removed from exchange 2013. Maintenance is still recurring, but is now implemented within a time-based assistant infrastructure. Not only that, but background ESE database scanning has been throttled even further compared to exchange 2010. It is now targeted to complete only once every 4 weeks compared to Exchange 2010.

Support for large mailboxes

Yes, Exchange 2013 includes support for large mailboxes… Up to 100 Gb+ in size and over 1 million item counts. For the record, 1 million is the highest number which was tested with so it might be sustainable over that number.

Now large mailboxes introduce a possible problem for cached outlook users. Imagine having a 60 Gig + OST file… Hell, I'm certain that would kill your disk performance on your client! It is for that reason that a new feature was introduced in Outlook 2013 to control the amount of mail to keep offline, thus mitigating the effect of performance issues on your clients if large mailboxes are implemented.



Feedback

# re: Exchange 2013: The store process

Good and useful information. 3/19/2013 3:46 PM | Gulab

# re: Exchange 2013: The store process

Re: a 60 GB .OST file...Outlook 2010 and 2013 raised the hard limit from 20 GB to 50 GB (http://support.microsoft.com/kb/982577). It's been our experience that going over 40-45 GB seems to cause Outlook 2010, at least, to trigger validity checks on the .OST file, causing system slowness. 7/28/2014 12:40 PM | Steve

# re: Exchange 2013: The store process

Is it possible to raise the 25% for the ESE Cache to f.e. 40%? We've got 64 GB RAM and only 27 GB is used. I'd like to give the DBs a bit more cache (and no, we don't want to raise the 64 GB RAM :).

It looks like if this is not a concept where the programmers had an idea but didn't completed it. We've got more than 42% of unused RAM.... 7/29/2014 7:07 AM | Ben

# re: Exchange 2013: The store process

Unfortunately that limit is hardcoded so it cannot be adapted (yet)... The idea here was really to improve the memory usage on multirole boxes, assuring that the other services on the box still have enough memory available and the garbage collector doesn't go stark raving mad on the memory 7/30/2014 4:16 PM | Marc