Concurrency

Parallel Computing with Visual Studio 2010 Beta 1

Jason Olson — Wed, 10 Jun 2009 06:44:34 GMT

One of the big areas of improvements coming to Visual Studio 2010 and .NET Framework 4 is in the area parallel computing. There are a LOT of new features and improvements to existing features here with this latest release of our developer tools.

Many people smarter and more capable than I have already written about most of this stuff, so there’s not much for me to add. What I did want to do though is to let you know all the places where you can find this wealth of information (and there is a lot of it). So, without further ado, here is a list of content for you if you are interested in Parallel Computing with Visual Studio 2010 and .NET Framework 4 Beta 1.

MSDN

What’s New…

Diving Deeper

Channel 9 Videos

Erika Parsons and Eric Eilebrecht : CLR 4 - Inside the Thread Pool

More Oldie-But-Goodie Channel 9 Videos

Misc…

Enjoy! There’s a lot here to digest. We definitely live in exciting times :).

Do you know of some links that you like that I missed here? Feel free to let me know by adding a comment here.

An Intro to Barrier, cont.d

Jason Olson — Tue, 24 Feb 2009 22:42:04 GMT

Well, I said I was going to be moving on to CountdownEvent, but I was wrong. there is one more aspect of using Barrier that I just had to share (thanks go to Stephen Toub on the Parallel Computing Platform team for bringing this up).

In my first Barrier post, I had mentioned:

Barrier is a great new synchronization primitive to use when there is a known amount of work to do that is being done by different workers that all have common synchronization points.

To make a long story short, this isn’t strictly true. Barrier can be used just fine when there is an unknown amount of work. The distinction between Barrier and other synchronization primitives isn’t about “known amount versus unknown amount of work”, it’s more about “usable once versus usable many times.” To explain this in more detail, let’s look back at the example from the last post (a Road Trip to Seattle).

Multiple Usages

First, it is perfectly allowed to use a Barrier more than once. Using the Road Trip metaphor, instead of going straight to Seattle after leaving the Gas Station, let’s have everybody stop for lunch on there way. So now there are two “sync points” that we have. First, we’ll wait for everybody to meet up at the gas station, then we’ll also wait for everybody to meet up for lunch after they’ve left the gas station.

Revising our DriveToSeattle method from the last post, we will simply call SignalAndWait() a second time to provide a second sync point where everybody meets for lunch:

static void DriveToSeattle(string name, TimeSpan timeDelay)
{
    // Perform some work
    Console.WriteLine("[{0}] Leaving House", name);
    Thread.Sleep(timeDelay);
    Console.WriteLine("[{0}] Arrived at Gas Station", name);

    // Need to wait for others
    sync.SignalAndWait();

    // Perform some work
    Console.WriteLine("[{0}] Leaving for Seattle", name);
    Thread.Sleep(timeDelay);
    Console.WriteLine("[{0}] Meet for lunch", name);

    // Need to wait for others
    sync.SignalAndWait();

    Console.WriteLine("[{0}] Leaving for Seattle", name);
}

In real-word usage of the Barrier primitive, this doesn’t have to be limited to two uses of the Barrier. We can use the Barrier any number of times we need to in order to provide these common synchronization points for our parallel code.

Unknown Number of Workers

I mentioned that it wasn’t strictly true that Barrier is great for a just a known amount of work. That’s because Barrier provides the functionality you need to even use it when the number of workers performing our work is unknown.

Back to our Road Trip. Dennis’s sister Dee is planning to come along on the Road Trip as well. However, she forgets to set her alarm and wakes up late. And unfortunately, Charlie, Mac, and Dennis have already left for Seattle. She still wants to go though so she plans to meet up with them for lunch.

What we need is for a way for Dee to flag “Hey guys! I’m coming!” And it just so happens that the Barrier class provides an AddParticipant method that we can use to make sure the extra “worker” (Dee in this case) is included the next time everybody waits for each other to “complete their work.”

In this case, Dee would simply “add herself” to the Barrier when her thread is spun up. To do this, let’s create a new DriveStraightToLunch method that will be used instead of the existing DriveToSeattle method we created before.

static void DriveStraightToLunch(string name, TimeSpan timeDelay)
{
    // "I'm coming along guys"
    sync.AddParticipant();

    // Meet for lunch
    Console.WriteLine("[{0}] Leaving for Seattle", name);
    Thread.Sleep(timeDelay);
    Console.WriteLine("[{0}] Meeting for lunch", name);

    // Need to wait for others
    sync.SignalAndWait();

    Console.WriteLine("[{0}] Leaving for Seattle", name);
}

And then we just spin up this new piece of work (added to our code in Main before all threads are joined):

... creation and start of Dennis thread

Thread.Sleep(TimeSpan.FromSeconds(6));
var dee = new Thread(() => DriveStraightToLunch("Dee", TimeSpan.FromSeconds(5)));
dee.Start();

dee.Join();
... join other threads

It’s that simple.

In Closing

I hope in the last two posts you’ve seen how useful the new Barrier synchronization primitive can be when writing Parallel Code. API-wise there really isn’t much to the new Barrier class. Of course, as is usual with writing parallel code, it’s learning to use it correctly or use it in the right situations that’s the tricky part :).

I would say "tune in next time when we chat about CountdownEvent” but considering this recent change of plans, I’m not going to tempt fate :). Later!

An Intro to Barrier

Jason Olson — Tue, 10 Feb 2009 11:17:39 GMT

In this first stage of our Tour de BCL, we will be passing through the new Barrier class.

So what is a Barrier? Let’s take a look at the ~~boring~~ technical description for a Barrier:

A Barrier is a synchronization primitive that enforces the stopping of execution between a number of threads or processes at a given point and prevents further execution until all threads or processors have reached the given point.

I don’t know about you, but sometimes technical descriptions like the above just sound like “blah, blah, blah” to me. What does a Barrier really mean to me, as a developer. Let’s break it down a different way by looking at a specific real-life scenario.

Think of a Road Trip

Instead of looking at the technical description above, think of the concept of a road trip.

There are three friends, Mac, Charlie, and Dennis. They live in a small town south of Seattle but want to take a road trip up to Seattle. So Mac calls up Charlie and Dennis and says “Hey guys, let’s take a road trip to Seattle. Meet up at the local gas station and then we’ll all leave from there.” So now they all now they need to go to the local gas station, wait for all of them to show up, and then they’ll leave for Seattle together and follow each other there.

Relating this scenario to some code, let’s say that Charlie, Mac, and Dennis are all individual threads that have one method of execution, the DriveToSeattle() method. Within that method, they are all going “to drive” to the same gas station and then continue to drive to Seattle.

If we don’t use the Barrier to synchronize the work (aka wait for each other to arrive at the gas station), each one of them will leave for Seattle the second they arrive at the gas station:

static void Main(string[] args)
{
    var charlie = new Thread(() => {
        DriveToSeattle("Charlie", TimeSpan.FromSeconds(1))
    }); 
    charlie.Start();

    var mac = new Thread(() => {
        DriveToSeattle("Mac", TimeSpan.FromSeconds(2))
    }); 
    mac.Start();

    var dennis = new Thread(() => {
        DriveToSeattle("Dennis", TimeSpan.FromSeconds(3))
    }); 
    dennis.Start();

    charlie.Join();
    mac.Join();
    dennis.Join();

    Console.ReadKey();
}

static void DriveToSeattle(string name, TimeSpan timeToGasStation)
{
    // Drive to gas station
    Console.WriteLine("[{0}] Leaving House", name);
    Thread.Sleep(timeToGasStation);
    Console.WriteLine("[{0}] Arrived at Gas Station", name);

    // Need to sync here

    // Perform some more work
    Console.WriteLine("[{0}] Leaving for Seattle", name);
}

Well, that’s hardly a road trip. If I were in Mac’s or Charlie’s shoes when Dennis left for Seattle without me, I know I would be a little upset. We obviously don’t want that to happen. So we can use the new Barrier to make sure they all “wait up” for each other at the gas station.

Using the new Barrier class

So how do you use the new Barrier class? It’s actually quite simple.The cool thing is that there are really only three things you need to learn to use the Barrier for this simple type of scenario: a single constructor and two method calls (both of which take zero parameters). Yes, it’s that simple to use.

Let’s look at what our new code using Barrier looks like (the three bold lines are the new ones):

static Barrier sync;

static void Main(string[] args)
{
    sync = new Barrier(participantCount:3);

    var charlie = new Thread(() => {
        DriveToSeattle("Charlie", TimeSpan.FromSeconds(1))
    }); 
    charlie.Start();

    var mac = new Thread(() => {
        DriveToSeattle("Mac", TimeSpan.FromSeconds(2))
    }); 
    mac.Start();

    var dennis = new Thread(() => {
        DriveToSeattle("Dennis", TimeSpan.FromSeconds(3))
    }); 
    dennis.Start();

    charlie.Join();
    mac.Join();
    dennis.Join();

    Console.ReadKey();
}

static void DriveToSeattle(string name, TimeSpan timeToGasStation)
{
    // Drive to gas station
    Console.WriteLine("[{0}] Leaving House", name);
    Thread.Sleep(timeToGasStation);
    Console.WriteLine("[{0}] Arrived at Gas Station", name);

    // Need to sync here
    sync.SignalAndWait();

    // Perform some more work
    Console.WriteLine("[{0}] Leaving for Seattle", name);
}

You can see now that nobody leaves for Seattle until everybody has arrived at the gas station. Essentially, with every call to SignalAndWait(), the number of signals received by the barrier is incremented. Once the number of signals received reaches the number of participants the Barrier was constructed with, all threads are then allowed to continue execution. And that’s all it takes to leverage the new Barrier class being introduced into the BCL with .NET Framework 4.0.

But Jason... What is up with that line of code: “sync = new Barrier(participantCount:3);”? I mean, what the heck is “participantCount:3”? How is that valid syntax?

Well, that’s easy. This is a new feature in C# 4.0 called Named Parameters where you can actually use a parameter’s name to specify what a value applies to. This becomes very handy in two places: 1) when combined with Optional Parameters, and 2) when used to clarify “magic numbers” (or “magic strings”, “magic booleans”, etc.) without having to introducing a temporary variable.

In this case, I could have left out participantCount and written just “sync = new Barrier(3);”. But I personally don’t like a magic number like “3” just floating around like this and Named Parameters give me an easy way to provide more context about the value “3”.

In Closing

So let’s look back at that ~~boring~~ technical description again:

A Barrier is a synchronization primitive that enforces the stopping of execution between a number of threads or processes at a given point and prevents further execution until all threads or processors have reached the given point.

Think about it this way:

“stopping of execution between a number of threads...” = waiting for each other to arrive
“… at a given point” = the gas station
“prevents further execution until all threads… have reached the given point” = can’t leave for Seattle until everybody shows up

See? Nice and simple!

Barrier is a great new synchronization primitive to use when there is a known amount of work to do that is being done by different workers that all have common synchronization points. However, if the amount of work needing to be done is not well known beforehand, Barrier is not the greatest primitive to use. Next we’ll look at another new synchronization primitive coming in .NET Framework 4.0 that is great to use when the amount of work is not known: the CountdownEvent.

I hope you all enjoyed this first stage in our Tour de BCL. Until the next stage, we’ll see you later.

Threading Basics: Race Conditions, Part 1

Jason Olson — Fri, 19 Sep 2008 08:13:42 GMT

I've talked in the past about the importance of parallel computing for all us developers. It's a trend in computer software and hardware architecture that is not a fad. Currently, in the US, it is nearly impossible to buy a new computer that has only a single core. We're even starting to see some of the first quad-core laptops hit the market.

It's becoming very important for developers to start dealing with parallel code. There's one problem: multithreaded development is hard. And in the class of Threading 101, Race Conditions is one of the first chapters you have to deal with.

So what is a Race Condition? According to Wikipedia, the source for all information on the Internet (emphasis mine):

"A race condition, or race hazard, is a flaw in a system or process whereby the output and/or result of the process is unexpectedly and critically dependent on the sequence or timing of other events. The term originates with the idea of two signals racing each other to influence the output first."

That's nice and all, but what does this mean to us developers? Let's look at a simple example. I have a Logger where I want to keep track of the number of messages that are logged.

   1: class Logger   2: {   3:     public static int TotalMessages = 0;   4:     5:     public void Log(string message)   6:     {   7:         // Do something with message   8:         TotalMessages++;   9:     }  10: }

Now let's call this method a bunch of times from four different threads and capture the total number of times Log message was called:

   1: int messagesPerTask = 200000;   2: int unitsOfWork = 4;   3:     4: var logger = new Logger();   5: Parallel.For(0, unitsOfWork, (taskNumber) =>   6: {   7:     for (int i = 0; i < messagesPerTask; i++)   8:     {   9:         logger.Log("Message " + i.ToString());  10:     }  11: });  12:    13: Console.WriteLine("{0} Expected", messagesPerTask * unitsOfWork);  14: Console.WriteLine("{0} Actual", Logger.TotalMessages);

What do you expect the output to be? The obvious answer is "well, we are adding 200000 messages on four different threads, so it's obviously 800000". Unfortunately, the obvious answer also happens to be the wrong answer. What's the right answer? It depends, and it can change every single time you run the application. For example, this were the results from the first time I ran the code:

800000 Expected
680354 Actual

Wait... how is this possible? Did we lose updates? Did I not call the method enough times? This surely must be a horrendous bug in the .NET Framework! Damn Bill and his blasted company. Trust me, it's not a bug. There's a good explanation of this behavior.

If you haven't written a lot of multithreaded code yet, the problem with the above code may not be obvious. Truth be told, there really isn't a problem with the code... IF you are only running the code on a single thread ever. But the likelihood of running on a single thread is becoming smaller and smaller. And as you can see, when we are running this in a multithreaded environment, there is most definitely a problem with the code.

A lot of .NET developers look at "TotalMessages++" and interpret it as one line of code, one execution, one instruction, etc. But there's the rub, it isn't. The code "TotalMessages++" is actually four instructions when compiled down to IL.

   1: L_0001: ldsfld int32 CSharpSandbox.Logger::TotalMessages   2: L_0006: ldc.i4.1    3: L_0007: add    4: L_0008: stsfld int32 CSharpSandbox.Logger::TotalMessages

These four lines essentially say the following:

Get the current value of TotalMessages
Get the value 1
Add the two numbers together
Save the result back into TotalMessages

Now let's imagine two threads executing the four steps above at the same time. Since threads won't be in perfect lock-step, they may be off by two steps (assuming the value of TotalMessages is currently 2):

[Thread 1] 1. Get the current value of TotalMessages (gets 2)
[Thread 1] 2. Get the value 1
[Thread 2] 1. Get the current value of TotalMessages (gets 2)
[Thread 2] 2. Get the value 1
[Thread 1] 3. Add the two numbers together (2 + 1)
[Thread 1] 4. Save the result back into TotalMessages (saves 3)
[Thread 2] 3. Add the two numbers together (2 + 1)
[Thread 2] 4. Save the result back into TotalMessages (saves 3)

So we've executed that code twice and at the end of the execution, TotalMessages has only increased by one. This is exactly what a race condition is. It's that simple. It's not as advanced as a concept as one might think.

This is why developers need to understand what race conditions are. A snippet of code that looks completely innocuous can cause some big problems when run in a multithreaded development. While future technologies will make writing parallel code easier, it won't prevent these types of problems from occurring. You still need to be aware of some of the common Threading 101 concerns/topics that exist today.

True CCR Concurrency - Using CCR without Iterators

Jason Olson — Sat, 21 Jun 2008 19:22:40 GMT

As developers get up to speed with the CCR, they quickly come across the (ab)use of iterators to achieve "concurrency." Okay, "(ab)use" may have been a bit of a low-blow. I call it an abuse as it is a totally unorthodox use of what iterators are usually used for, and I've came across my fair share of developers who find this unorthodox usage highly confusing. Not only is it confusing, but it's not what I would consider true concurrent code.

In a concurrent environment, if I create twenty different tasks that need to be executed, I expect several of those tasks to be executed at the exact same time (depending on the number of processors I have on my system). As a side effect of this concurrent execution of tasks, the order in which those tasks are executed is not guaranteed. If you don't have several tasks executing concurrently, and order of processing is indeed guaranteed, that's about as close you can get to being normal sequential code.

Look at this sample (example 18) from MSDN. If you execute and step through this code, you will find that the method actually executes sequentially. The code posts a number, processes the number, posts another number, processes that number, etc. This is highly sequential. One could argue this is simply not the best sample for CCR iterators. However, after using CCR when developing robots using Robotics Studio, you will find that this "sequential but not" pattern is a very frequent pattern that is used when using CCR iterators.

There's also the impact of having a method that essentially never exits due to the "while (true)" nature of the iterator-based method that is determining what tasks need to execute (leading to a highly coupled, monolithic application). This doesn't mean that the CCR itself isn't useful though. You just need to know how to leverage the framework while avoiding some of these "rough spots".

So let's take a look at an example of how to use CCR to achieve "true" concurrency with the CCR without the use of iterators (for you VB fans, the great part of this approach is that it is entirely possible with VB like I show here).

Dim myPort As New Port(Of Integer)

At the heart of the message-based infrastructure in the CCR is a Port. A Port is where messages are posted to, and pulled from when tasks are actually executed. As seen above, Ports are strongly typed through the use of generics, and so they support posting/pulling of largely any POCO (Plain Ole CLR Object) you wish. In the case of this very simple demo we just use integers, but in a real system this might be self-defined types like StockQuoteRequestMessage, FoodOrderMessage, etc.

    Sub Main()
        Dim taskDispatcher As New Dispatcher(4, "Sample Dispatcher")
        Dim taskQueue As New DispatcherQueue("Sample Tasks", taskDispatcher)

In the CCR, a DispatcherQueue contains a list of tasks to be executed, while a Dispatcher controls the execution of those tasks. Here we force our dispatcher to run four tasks concurrently even if we are on a single processor machine just so we can see the concurrency in action. Then we simply pass our dispatcher to the queue that contains the tasks that will be managed.

        ' Create the task that will handle our work
        Dim workReceiver = Arbiter.Receive(Of Integer)(True, _
                                myPort, _
                                Function(i) ProcessValue(i))

        ' Register the task with the CCR task queue
        Arbiter.Activate(taskQueue, workReceiver)

        ' Post some work to do
        For i = 0 To 9
            myPort.Post(i)
        Next

When we create our workReceiver, we are essentially telling the CCR that we want a persistent task (one that will last across the processing lifetime of many messages, possibly until the application exits), and that when a message comes in on our Port, we will process that message with our function ProcessValue (defined below). Once the receiver is created, we simply hook it up to the task queue, and we're off to the races.

Finally, all we need to do is wait until all values have been processed and exit the application.

        ' Wait until all tasks are finished
        While myPort.ItemCount > 0
            Thread.Sleep(TimeSpan.FromMilliseconds(200))
        End While

        taskQueue.Dispose()
        taskDispatcher.Dispose()

        Console.Write("Press any key to exit...")
        Console.ReadKey(True)
    End Sub

So when a message is posted to our port, CCR will call our function ProcessValue in order to process the message (as specified when we created the task to process messages). In our process function, we are simply going to write to the console at the beginning and at the end as well as sleeping our thread in the middle to simulate some work being done in the function.

    Function ProcessValue(ByVal value As Integer)

        Console.WriteLine("Begin " + value.ToString())
        Thread.Sleep(TimeSpan.FromMilliseconds(200))
        Console.WriteLine("End " + value.ToString())

        Return Nothing
    End Function

When I actually execute this sample, you can see the parallel nature of the execution by viewing the output (your output may differ as it can differ upon every execution thanks to the non-deterministic nature of concurrency):

Begin 0
Begin 1
Begin 3
Begin 2
End 2
Begin 4
End 1
Begin 5
End 0
Begin 6
End 3
Begin 7
End 4
Begin 8
End 5
End 6
Begin 9
End 7
End 8
End 9
Press any key to exit...

And that's all we need to do in order to use CCR to process incoming messages in parallel (and in a "true" concurrent fashion). Just as a recap, here's the entire sample in one place:

    Dim myPort As New Port(Of Integer)

    Sub Main()
        Dim taskDispatcher As New Dispatcher(4, "Sample Dispatcher")
        Dim taskQueue As New DispatcherQueue("Sample Tasks", taskDispatcher)

        ' Create the task that will handle our work
        Dim workReceiver = Arbiter.Receive(Of Integer)(True, _
                                myPort, _
                                Function(i) ProcessValue(i))

        ' Register the task with the CCR task queue
        Arbiter.Activate(taskQueue, workReceiver)

        ' Post some work to do
        For i = 0 To 9
            myPort.Post(i)
        Next

        ' Wait until all tasks are finished
        While myPort.ItemCount > 0
            Thread.Sleep(TimeSpan.FromMilliseconds(200))
        End While

        taskQueue.Dispose()
        taskDispatcher.Dispose()

        Console.Write("Press any key to exit...")
        Console.ReadKey(True)
    End Sub

    Function ProcessValue(ByVal value As Integer)

        Console.WriteLine("Begin " + value.ToString())
        Thread.Sleep(TimeSpan.FromMilliseconds(200))
        Console.WriteLine("End " + value.ToString())

        Return Nothing
    End Function

As a "hello world" sample for using the CCR, this is a very simple demo. But it shows that it is very easy to actually achieve concurrent applications with the CCR without having to use CCR iterators. At the same time, avoiding the use of CCR iterators enable us to write a message-oriented application that can be properly modularized by "separating concerns" into independent message processors through the use of Ports.

Of course, an added benefit of taking a "non-iterator" approach when using the CCR is that languages that don't have direct support for iterators (like VB.NET) can also develop CCR-based applications to achieve highly-parallel solutions.

Parallel Extensions to the .NET Framework, June 2008 CTP Released!

Jason Olson — Tue, 03 Jun 2008 08:01:41 GMT

The latest CTP for Parallel Extensions to the .NET Framework was released out to the intarw3bs today (Go Digg It on Digg and Kick It on DotNetKicks). This is the second major CTP for the Parallel Extensions.

If you're the kind of person who likes to just get the latest install and play around, you can find it here. If, however, you are a person who would like to read some resources about the new CTP first, there's a bunch of links for you as well:

Ed Essey announces the release on the Parallel Extensions Team Blog
Ed outlines the new features in this CTP
Ed talks about the known issues with this CTP
Joe discusses the new release
Ed on why feedback is important to the team

Here are some of the reactions (or re-announcements) from other bloggers:

Damon Payne on the release
Tamir Khason on the release
Brian Spann on the release

I think a very important quote from Joe is (emphasis mine):

"I'm really excited to see our entire stack finally shipping as one cohesive unit: the data structures we use throughout the implementation exposed publicly (what we now call CDS), a new scheduler built from the ground up, TPL and PLINQ better together, and lots more."

I find this release very exciting, especially the "cohesive unit" part. As Ed mentions in the new features post, PLINQ is now running directly on top of TPL, rather than using the ThreadPool like it did before. The data structures used internally are now exposed as CDS. TPL with many improvements to the scheduler. This is a very great CTP for the Parallel Extensions Team. So congratulations to the team!

If you want to play around with concurrent programming in .NET, Parallel Extensions to the .NET Framework is the place to be. The team really wants your feedback and, through this feedback, you can help shape the future of concurrent programming in .NET. How exciting is that?!? So go grab the bits, install them, and enjoy!

And if you're a Parallel Extensions fan, be sure to Digg It and Kick It!