Michael Van Cleave
Traveling the technical world, learning the language

Very Large SharePoint Database Files

Tuesday, June 03, 2008 3:06 PM

 

So you have installed SharePoint (WSS or MOSS) and you have been happily using it for a while now.  Everyone in the office has been taking part in creating some really cool content or loading their sites up with some very important information. 

Now fast forward a few months.  Your local administrator of the server gets in touch with you to let you know that your SQL Server is running low on memory.  What?  How could this be?  You start to look for the source of the issue and low and behold you run across a 120 GB database log file.  120 GB????  Yep.  120 GB.

You start to think to yourself, “Man, I didn’t know that the SharePoint databases were going to take up this amount of memory.  Maybe I need to purchase an extremely large drive to put the database files on”. 

Well you could do that.  Or you can modify a few settings and set up a few good maintenance plans and you can reduce that log file down to mere kilobytes.  This actually happened to a few clients of mine.  The main source of the problem is that when most folks install SharePoint they don’t realize that it creates the databases and sets them to Full Recovery Mode.  What this means is the database log (.ldf) file never gets shrunk.  Mainly this is so that if something happens to the database they you have a full history of all of the transactions that have taken place.  But, this is bad for many reasons as well.  Now don’t get me wrong many places have that grumpy DBA that will take care of all of this stuff for you, but I mainly see this in small to medium organizations that don’t have a full time DBA on staff and/or they roll a development farm to production without a lot of forethought on recovery if something goes bad.

Let’s see what we can do about this.

First open up your file explorer on your database server or whatever server your database files are located on and look at the size of the database .mdf and .ldf files.  If you have a default install of SQL Server you can find these files under C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\Data.  Usually if you find that the .ldf files are in the high MB or GB you definitely need to continue with this article.  If not at least check to verify that you have database backups and/or maintenance plans in place.

STEP 1: Backup all the SharePoint Databases
               If you are going to do anything with the databases it is always a great idea to do a Full backup on each database

STEP 2: Change the Recovery Mode of the database
              
This is a fairly easy thing to do.  You log in to SQL Server and navigate your way to each one of the databases you want to take care of.  The following screen shot is of the database properties dialog.  Just right click the database and select “Properties”.

DatabaseProperties

               The red outlined section is the recovery model setting.  Currently it is set to FULL.  The other selection “Simple” is the selection we need to use.  So select “Simple” and click OK.

STEP 3: Shrink that excessive log file!!!
              
So the database that I am currently using as an example the log file is nearly 2 GB (actually it is 1.795GB).  To shrink the database log file, right click the database and select “Tasks”, then select “Shrink”, then select “Files”.  You will get this dialog:

ShrinkFileDialog

               The database file type is what we are after.  We want to switch it to “Log”.

ShrinkLogFileLog

               Notice the red highlight.  Look at the log file size and the available free space.  You have the ability to recapture nearly 89% of the space that is currently allocated.  Make sure that the shrink action is set to release unused space and click OK.  Now your log file should shrink to only the needed space and it should end up around a few hundred KB.

               The important take away here is that when you set up SharePoint you need to be sure that you are actively maintaining your system or be sure that you have the proper processes in place for the database.  I have seen many of my clients that all this comes as a shock to them.  I don’t think it is really their fault, but more just a point that they never knew had to be considered.  Most clients that have DBA’s on staff will rarely run in to this problem since most DBA’s will be the first to be sure that these database are fully maintained and recoverable.

               On a side note, please be sure that you or someone that is qualified in your organization creates maintenance plans for the databases.  This will assist you in being sure that the database is backed up and maintained properly.  There are many articles out there on how to set these plans up on SQL Server 2005 so I won’t bore you with that.  J

 HTH.

Michael

 


Feedback

# re: Very Large SharePoint Database Files

Also, keep in mind if you are doing transaction log backups, log shipping or mirroring, you need to have the database in full recovery mode.

Microsoft has a list of tested operations done with maintenance plans that they have verified as safe for use with SharePoint
http://support.microsoft.com/kb/932744 and http://support.microsoft.com/kb/841057



6/4/2008 8:02 AM | Dave Wollerman

# re: Very Large SharePoint Database Files

Excellent point.

Thanks for the links.

Michael 6/4/2008 8:56 AM | Michael

# re: Very Large SharePoint Database Files

Hi Michael,

I done this process in my test server which is having 40 GB of log file. When I try to shrink the log file then it is reducing 2% only.
I tried in my production system it works good.
Is it depends on time factor ?
One request from my side(If possible) please post a blog on 'how to change our wss 3.0 from basic to form' (upgrading from WID to SQL Server 2005 Standard).

Thanks. 6/7/2008 5:32 AM | P.Venkateswara Rao

# re: Very Large SharePoint Database Files

Hi Michael,

Exelent post. There is another issue with this logfile. If you have your databases in full recovery mode (maybe other modes too, I didn't try) and set the database (+log) size to a size which is a fair amount under the total disk space available (so it will never eat up all that's left), you could run into the problem that deleting a site collection won't work.
Deleting a site collection seems to be transactional -> the logfile can't hold that information -> the transaction can't be completed, the site collection can't be deleted. The system freezes (seems to freeze) up on such an action, and you better pray it gets back.
If you're really without luck, it thinks it deleted the site collection, but it didn't, preventing you from accessing it and forcing you to delete the database (webapplication).. and start all over.
I have had the pleasure of dealing with both scenarios....
If you setup db and file sizes in such a way that the total disk space will never be taken completely, you will have a better chance that Sharepoint recovers itself and so only preventing you to do very large operations (like delete site collection).
If all disk size is eaten, you're in real trouble, f.i. preventing you to login on the server (especially through remote desktop) and the whole system state is kind of unpredictable. It can mean your site collection is gone but it still sits in your database 7/14/2008 8:03 AM | Mark Lammerse

# re: Very Large SharePoint Database Files

Hi

I have instaled sharepoint on my local machine and was working fine till I discovered that I have low disk space now I want to take all the stuff to centeral server how to carry out the steps

Thanks 10/7/2008 6:26 AM | Himanshu

# re: Very Large SharePoint Database Files

Himanshu,
Well, you have a couple of options that are available to you. You can backup the content that is in your local installaiton and then restore it to a fresh installation on another machine that has more space. Or you can just backup the databases and move them like a migration. Either one should work for you.

Michael 10/7/2008 8:44 AM | Michael

# re: Very Large SharePoint Database Files

Hi, if you cant connect to your sql express database, try this:

1) Start SQL 2005 Express Management Studio
ÙNOTE : SQL 2005 Express Management Studio must be installed locally on the
Sharepoint host.
2) In the Connect to Server window, enter the following in the Server Name
field
\\.\pipe\mssql$microsoft##ssee\sql\query
3) In the Autentication field, select Windows Autentication
4) Click Connect 2/4/2009 5:32 AM | Khabba

# re: Very Large SharePoint Database Files

or you could read the best practices docs for sharepoint and realize that flipping to simple recovery mode isn't the answer.

increasing the frequency of the log dumps.

4/22/2009 8:25 AM | Grumpy DBA

# re: Very Large SharePoint Database Files

We just followed the above steps and reduced a 60GB SQL DB down to <10GB. We could not believe that our content was breaking 60GB of data. We did a backup of the DB first, and then ran the steps above and then set up a maintenance plan.

Thanks for the info! 5/13/2009 9:34 AM | Bryan Luce

# re: Very Large SharePoint Database Files

Glad to hear that it helped.

Thanks for visiting my blog.

Michael 5/13/2009 10:21 AM | Michael

Post a comment





 




Archives

Post Categories

Great Links

Other Blogs

Pod Casts

Syndication: