ArchitectNow
Kevin Grossnicklaus' Blog

SQL Server 2005 Compact Edition performance

Friday, June 22, 2007 1:23 PM

The project I'm currently wrapping up is a large .NET Windows "Smart Client" application (which I'm sure you'll hear more about in upcoming posts).  It actually has the capability of working in "offline" mode and we have a pretty extensive infrastructure to cache data locally and store it for offline use in a SQL Server 2K5 CE database (which we auto-deploy with the rest of the app via ClickOnce).

Unfortunately, we've recently hit somewhat of a snag when our offline cache grows significantly.  We have a test case where (after loading a large amount of data into the application via a saved document) we end up with 90,000+ lines of data in a cache table that we want to write out to the SQL CE database when the application closes.  Basically, it requires 90K of simple insert statements in a loop.  This is where it's readily apparent that SQL CE is not a full SQL 2K5.  These 90,000 inserts take over 10 minutes to complete. 

We're currently evaluating any ways to increase the speed of this process (or provide the users the option to bypass).  We can show a progress bar but who likes to watch a progress bar for 10 minutes when all you want is to exit an application? :)

Other than performance in situations like this (and lacking the ability to use stored procs) we have been pleasantly suprised with the SQL CE option for offline data.  We've gotten pretty good at using it as well.

If you need this type of functionality (or want to know what else it is capable of) check it out here:

http://www.microsoft.com/sql/editions/compact/default.mspx

-Kev


Feedback

# re: SQL Server 2005 Compact Edition performance

Are you calling ExecuteNonQuery() 90,000 times? If so then try out concatenating 100 INSERT statements to a string, then issue 100 at a time. Something like:

int x=0;
while(notDone)
{ // Wicked loop doing 90,000 INSERTs

... Code that builds out insertStatement, perhaps adding uniquely named parameters based on x

lotsaInserts += " " + insertStatement;
if(++x % 100 == 0)
{
cmd.CommandText=lotsaInserts;
}
}

This should speed it up _significantly_.
6/22/2007 1:52 PM | Lorin Thwaits

# re: SQL Server 2005 Compact Edition performance

Oh, add these statements after "cmd.CommandText=lotsaInserts":

cmd.ExecuteNonQuery();
lotsaInserts=String.Empty;

Hope all that makes sense. 6/22/2007 1:55 PM | Lorin Thwaits

# re: SQL Server 2005 Compact Edition performance

It does actually :) We'd optimized the loop as best we could and were still looking at other possibilities but I hadn't thought of batching the calls. I'll give this a try and see if it helps.

Thanks a million for the suggestion.

-Kev 6/22/2007 4:33 PM | Kevin

# re: SQL Server 2005 Compact Edition performance

Unfortunately, it doesn't appear SQL CE supports concatenating the statements together. We've tried a number of delimeters and what works on a regular SQL 2K5 install is not supported by the CE version :(

We're still looking at it though...

-Kev 6/26/2007 12:08 PM | Kevin

# re: SQL Server 2005 Compact Edition performance

we got the same issues. ever gotten anywere with your research ?

regards 11/15/2007 1:41 AM | matthiasg

# re: SQL Server 2005 Compact Edition performance

Have you tried putting the INSERT's into a transaction? I'm not sure how the underlying mechanisms work but it might batch up the INSERT's and then apply the locks etc. in one go - rather than locking the table, inserting the row, freeing up the table 90k times it should only do it once.

I've just discovered CE so i'm doing some tests on it now... 4/23/2008 6:57 AM | Alan

# re: SQL Server 2005 Compact Edition performance

Use a result set object for inserts. 2/11/2009 10:50 PM | Ziad

# re: SQL Server 2005 Compact Edition performance

I found a solution on this article:
http://www.pocketpcdn.com/forum/viewtopic.php?t=11003

I was inserting 1188 rows in 1 minute and 23 seconds, and now, it took 18 miliseconds!

Really awsome!

Best regards,

Henrique 7/12/2009 5:02 PM | Henrique

Post a comment