<< XamlFest: WPF/Silverlight Tools | Home | Accidental Perplexity >>

Try/Catch Performance

I'm really not obsessed with performance -- honest!

However, when a co-worker asked me today exception handling was an acceptable way of coding defensively, my reaction was rather predictable. Exceptions are pure evil, and should be... well, exceptional.

Yes, you guessed it. The next question was "How bad is try/catch really?"

The short answer is that is involves minimal overhead... unless an exception is thrown. In that case, the .NET exception handling mechanism does a few nice things, like providing the stack trace as part of the exception. Great for debugging... lousy for performance. Of course, if exceptions are treated as being exceptional -- that is, they're only raised when something you haven't predicted occurs -- then you really shouldn't care about performance. Since your application is most likely left in an unknown or unstable state after an exception, the odds are you're going to reset a substantial part of your application, if not shut it down entirely.

As a rule, you can predict that users will provide data that is, to put it politely, suspect. This isn't a reflection on the users; if anything, it's more a comment on the tendency of developers to know how their own code works, and to continually feed it good data. But, both for this reason, and for security reasons, the ASP.NET mantra of "never trust any input" applies in pretty much any application. Code defensively against user input, so that you don't have to rely on try/catch blocks to deal with exceptions.

How significant is the overhead of catching an exception? Here's the code for a console application to illustrate:

  1: Module Module1

  2:     Sub Main()

  3:         TestNaive(1)

  4:         TestDefensive(1)

  5:         TestWithTryCatch(1)

  6:         TestWithException(1)

  7: 

  8:         Dim maxCount As Integer = 1000000

  9: 

 10:         Console.WriteLine("Naive: " & TestNaive(maxCount) & " ticks")

 11:         Console.WriteLine("Defensive coding, no try/catch: " & TestDefensive(maxCount) & " ticks")

 12:         Console.WriteLine("Naive with try/catch: " & TestWithTryCatch(maxCount) & " ticks")

 13:         Console.WriteLine("Exception: " & TestWithException(maxCount) & " ticks")

 14:         Console.ReadLine()

 15:     End Sub

 16: 

 17:     Private Function TestNaive(ByVal iterationCount As Integer) As Double

 18:         Dim sw As New Stopwatch

 19:         sw.Start()

 20:         For i As Integer = 1 To iterationCount

 21:             Dim x As Double = i / (iterationCount - i + 1)

 22:         Next

 23:         sw.Stop()

 24: 

 25:         Return sw.ElapsedTicks / iterationCount

 26:     End Function

 27: 

 28:     Private Function TestDefensive(ByVal iterationCount As Integer) As Double

 29:         Dim sw As New Stopwatch

 30:         sw.Start()

 31:         For i As Integer = 1 To iterationCount

 32:             Dim y As Integer = (iterationCount - i + 1)

 33:             If Not y = 0 Then

 34:                 Dim x As Double = i / y

 35:             End If

 36:         Next

 37:         sw.Stop()

 38: 

 39:         Return sw.ElapsedTicks / iterationCount

 40:     End Function

 41: 

 42:     Private Function TestWithTryCatch(ByVal iterationCount As Integer) As Double

 43:         Dim sw As New Stopwatch

 44:         sw.Start()

 45:         For i As Integer = 1 To iterationCount

 46:             Try

 47:                 Dim x As Double = i / (iterationCount - i + 1)

 48:             Catch ex As Exception

 49: 

 50:             End Try

 51:         Next

 52:         sw.Stop()

 53: 

 54:         Return sw.ElapsedTicks / iterationCount

 55:     End Function

 56: 

 57:     Private Function TestWithException(ByVal iterationCount As Integer) As Double

 58:         Dim sw As New Stopwatch

 59:         sw.Start()

 60:         For i As Integer = 1 To iterationCount

 61:             Try

 62:                 Dim x As Double = i / 0

 63:             Catch ex As Exception

 64: 

 65:             End Try

 66:         Next

 67:         sw.Stop()

 68: 

 69:         Return sw.ElapsedTicks / iterationCount

 70:     End Function

 71: End Module

My results tend to vary a little bit, since the work done in each loop so trivial that it is susceptible to noise. The first three tests are generally pretty close to each other. The fourth test -- testing with an exception raised -- is always much longer. Generally it's somewhat more than 5x longer than the other tests.

TryCatchPerfTest

TryCatchPerfTest2

What I find really interesting is that the inclusion of the try/catch block appears to make the naive implementation run faster. I don't have the .NET debugging symbols loaded on my machine, and dotTrace really didn't provide any insight. Anyone out there who has dug deeply into the CLR and can provide insight, it would be much appreciated.

However, the moral of the story is straight-forward. Treat exceptions as exceptional. Code defensively rather than catching exceptions wherever possible. No surprise there.

Print | posted @ Monday, December 01, 2008 5:06 PM

Comments on this entry:

Gravatar # re: Try/Catch Performance
by Alois Kraus at 12/2/2008 6:31 AM
The reason why your try/catch function is faster is easy:
The defensive version has an additional if clause which causes an conditional jump which ruins the day of the branch prediction unit of the processor ;-).


32: Dim y As Integer = (iterationCount - i + 1)
33: If Not y = 0 Then
34: Dim x As Double = i / y

46: Try
47: Dim x As Double = i / (iterationCount - i + 1)
48: Catch ex As Exception
49:
50: End Try

Yours,
Alois Kraus
Gravatar # re: Try/Catch Performance
by Jeff Certain at 12/2/2008 6:34 AM
Alois,

Perhaps I wasn't clear enough. I was referring to the difference between the naive test and the naive test with try/catch. There's no defensive coding there, but the version with try/catch appears faster.

Jeff
Gravatar # re: Try/Catch Performance
by Mark Brackett at 12/2/2008 9:25 AM
Not sure I can explain it off the top of my head, but compiled for AnyCPU or x64 on my x64 Windows results in the following timings:

Naive: 0.446562 ticks
Defensive coding, no try/catch: 0.344532 ticks
Naive with try/catch: 0.442742 ticks
Exception: 0.339604 ticks

An x86 build results in timings closer to yours:

Naive: 0.59108 ticks
Defensive coding, no try/catch: 0.205472 ticks
Naive with try/catch: 0.205776 ticks
Exception: 5.797074 ticks

Not that you should really care either way, as even at 6 ticks you're still doing around 1.5 million/second. Not really the bottleneck in any real world app, I'd think.
Gravatar # re: Try/Catch Performance
by Jeff Certain at 12/2/2008 9:46 AM
Mark,

Thanks for the feedback.

I am indeed building on an x86 box.

As far as this not being the bottleneck, I'd agree... at least on modern hardware, in most scenarios.

However, if you're coding in a constrained environment, it's sometimes necessary to eke out every bit of performance possible.

From a more philosophical point of view, I'd say that defensive coding is worthwhile for two reasons. The first is that I find it easier to understand what's going on, as well as the developer's intent. The second is more an issue of craftsmanship. While not as important as it used to be, writing code that performs well is worth doing in it's own right. And, in some cases, such as trying to optimize the throughput of a large number of concurrent processes, performance still matters.

Jeff
Gravatar # re: Try/Catch Performance
by Sean Stapleton at 12/3/2008 3:21 AM
I think you also need to be careful about what you are testing - you have established that exception handling code is really slow WHEN the exception is always raised, but that is hardly a surprising result..

I played around with your defensive coding functions and tested with the following:

1) removed the +1 from the denominator calculation, to ensure the branch/exception code is called once
2) added two "1inN" functions to test various percentages of exceptions.

E.g.,
For j = 1 to iters / N
for i = 1 to N
//exception or If Not code, no +1

Results for x86:
Defensive coding, NO try/catch: 0.187116 ticks
Defensive coding and try/catch: 0.185249 ticks

Defensive coding, NO try/catch, 0.1% div/0: 0.197068 ticks
Defensive coding, try/catch and 0.1% div/0: 0.184994 ticks

Defensive coding, NO try/catch, 1% div/0: 0.184682 ticks
Defensive coding, try/catch and 1% div/0: 0.189369 ticks

Defensive coding, NO try/catch, 10% div/0: 0.104836 ticks
Defensive coding, try/catch and 10% div/0: 0.270024 ticks

Defensive coding, NO try/catch, 25% div/0: 0.061308 ticks
Defensive coding, try/catch and 25% div/0: 0.436903 ticks

For AnyCPU / x64:
Defensive coding, NO try/catch: 0.211974 ticks
Defensive coding and try/catch: 0.209349 ticks

Defensive coding, NO try/catch, 0.1% div/0: 0.209431 ticks
Defensive coding, try/catch and 0.1% div/0: 0.20953 ticks

Defensive coding, NO try/catch, 1% div/0: 0.198279 ticks
Defensive coding, try/catch and 1% div/0: 0.197792 ticks

Defensive coding, NO try/catch, 10% div/0: 0.126249 ticks
Defensive coding, try/catch and 10% div/0: 0.139832 ticks

Defensive coding, NO try/catch, 25% div/0: 0.078965 ticks
Defensive coding, try/catch and 25% div/0: 0.092372 ticks

Conclusion: When testing for performance, measure the same things, and make sure you are focused on the appropriate use cases, and the appropriate code. For example, I assumed from previous experience in other languages that a major bottleneck in your code was the Dim inside the for loop. I tested it, and not so on my machine.

YMMV
Gravatar # re: Try/Catch Performance
by Jeff Certain at 12/8/2008 9:30 AM
Sean,

I deliberately hit the exception repeatedly to see what the impact of hitting it once was. (That is, the number I display is the average cost of raising the exception). Really, the point was to show that defensive coding performs better than catching exceptions -- and your tests seem to bear that out nicely.

Jeff



Gravatar # re: Try/Catch Performance
by Sean Stapleton at 12/11/2008 4:48 AM
What my results show is that *on my machine, x86* if you are doing *10M* divsion ops in a tight loop, *and* 10% of the operations are div/0, defensive coding will save you 0.17 seconds.

Conclusion: When testing for performance, make sure you are focused on the appropriate use cases, and the appropriate code
Gravatar # re: Try/Catch Performance
by Sky Sigal at 12/17/2008 8:08 PM
I did similar tests a while back that yielded that the cost of try/catch, although not insignificant, was -- for the benefit given -- acceptable.

The code used, and results, can be found at:

http://skysigal.xact-solutions.com/Blog/tabid/427/EntryId/318/C-The-cost-of-try-catchs.aspx

Your comment:

Title:
Name:
Email:
Website:
 
Italic Underline Blockquote Hyperlink