Parallel Extension

.NET 4: “Slim”-style performance boost!

Vitus — Wed, 14 Apr 2010 20:28:24 GMT

RTM version of .NET 4 and Visual Studio 2010 is available, and now we can do some test with it.

Parallel Extensions is one of the most valuable part of .NET 4.0. It’s a set of good tools for easily consuming multicore hardware power. And it also contains some “upgraded” sync primitives – Slim-version.

For example, it include updated variant of widely known ManualResetEvent. For people, who don’t know about it: you can sync concurrency execution of some pieces of code with this sync primitive. Instance of ManualResetEvent can be in 2 states: signaled and non-signaled. Transition between it possible by Set() and Reset() methods call. Some shortly explanation:

Thread 1	Thread 2	Time
mre.Reset(); mre.WaitOne();	//code execution	0
//wating	//code execution	1
//wating	//code execution	2
//wating	//code execution	3
//wating	mre.Set();	4
//code execution	//…	5

Upgraded version of this primitive is ManualResetEventSlim. The idea in decreasing performance cost in case, when only 1 thread use it. Main concept in the “hybrid sync schema”, which can be done as following:

internal sealed class SimpleHybridLock : IDisposable

    private Int32 m_waiters = 0;

    private AutoResetEvent m_waiterLock = new AutoResetEvent(false);

    public void Enter()

        if (Interlocked.Increment(ref m_waiters) == 1)

            return;

        m_waiterLock.WaitOne();

    public void Leave()

        if (Interlocked.Decrement(ref m_waiters) == 0)

            return;

        m_waiterLock.Set();

    public void Dispose()

        m_waiterLock.Dispose();

It’s a sample from Jeffry Richter’s book “CLR via C#”, 3rd edition. Primitive SimpleHybridLock have two public methods: Enter() and Leave(). You can put your concurrency-critical code between calls of these methods, and it would executed in only one thread at the moment. Code is really simple: first thread, called Enter(), increase counter. Second thread also increase counter, and suspend while m_waiterLock is not signaled. So, if we don’t have concurrent access to our lock, “heavy” methods WaitOne() and Set() will not called. It’s can give some performance bonus.

ManualResetEvent use the similar idea. Of course, it have more “smart” technics inside, like a checking of recursive calls, and so on. I want to know a real difference between classic ManualResetEvent realization, and new –Slim. I wrote a simple “benchmark”:

class Program

    static void Main(string[] args)

        ManualResetEventSlim mres = new ManualResetEventSlim(false);

        ManualResetEventSlim mres2 = new ManualResetEventSlim(false);

        ManualResetEvent mre = new ManualResetEvent(false);

        long total = 0;

        int COUNT = 50;

        for (int i = 0; i < COUNT; i++)

            mres2.Reset();

            Stopwatch sw = Stopwatch.StartNew();

            ThreadPool.QueueUserWorkItem((obj) =>

                //Method(mres, true);

                Method2(mre, true);

                mres2.Set();

});

            //Method(mres, false);

            Method2(mre, false);

            mres2.Wait();

            sw.Stop();

            Console.WriteLine("Pass {0}: {1} ms", i, sw.ElapsedMilliseconds);

            total += sw.ElapsedMilliseconds;

        Console.WriteLine();

        Console.WriteLine("===============================");

        Console.WriteLine("Done in average=" + total / (double)COUNT);

        Console.ReadLine();

    private static void Method(ManualResetEventSlim mre, bool value)

        for (int i = 0; i < 9000000; i++)

            if (value)

                mre.Set();

            else

                mre.Reset();

    private static void Method2(ManualResetEvent mre, bool value)

        for (int i = 0; i < 9000000; i++)

            if (value)

                mre.Set();

            else

                mre.Reset();

I use 2 concurrent thread (the main thread and one from thread pool) for setting and resetting ManualResetEvents, and try to run test COUNT times, and calculate average execution time. Here is the results (I get it on my dual core notebook with T7250 CPU and Windows 7 x64):

ManualResetEvent	ManualResetEventSlim

Difference is obvious and serious – in 10 times!

So, I think preferable way is using ManualResetEventSlim, because not always on calling Set() and Reset() will be called “heavy” methods for working with Windows kernel-mode objects. It’s a small and nice improvement! ;)

Thread-safe data structures .NET 4.0 (part 2)

Vitus — Sat, 24 Oct 2009 23:01:12 GMT

Today I tell you about BlockingCollection<T>. It’s more complex and more interesting data storage.

BlockingCollection<T> is a thread-safe data structure, which is called “blocking”, because it based on following principles:

if the collection is empty, and some code try to take an element, the thread, that executes this code, is blocked while any at least one element is not added;
if the collection already contains maximum elements, the thread, attempting to add new element, will be blocked until free space for the element is not available.

Of course, it’s not a full description of the collection’s possibilities.

Let’s start to introduce with BlockingCollection<T>. It has few methods for manipulating data:

	Blocking methods	non-Blocking methods
Add	Add(T); Add(T, CancellationToken);	TryAdd(T) TryAdd(T, Int32) TryAdd(T, TimeSpan) TryAdd(T, Int32, CancellationToken)
Take	Take() Take(CancellationToken)	TryTake(out T) TryTake(out T, Int32) TryTake(out T, TimeSpan) TryTake(out T, Int32, CancellationToken)

How you can see, we have both types of methods – blocking and non-blocking (starts with “Try” prefix). Try*-method return true on success of adding or taking an element, else – false. Both types have overloads with cancellation token, which allow to cancel method async.

BlockingCollection<T> have mechanism of completion. Collection is completed, if we will not add any data in future. After completion all attempts to add some data will result in generation of InvalidOperationException. If we try to take an element from empty completed collection, we also have an exception. When we create a collection, it is not completed, and completion is doing by calling CompeteAdding() method. So, we must complete the collection explicitly. How you can use the completion mechanism? You can sync producer and consumer, i.e. parts of code, adding and taking data from the collection. Producer may notice, that new data will not be added by marking the collection as completed. So, consumer will not be wait for new elements.

Ok, here is some code, that demonstrate how to work with BlockingCollection<T>. First of all we instantiate the collection, and set max elements count. However, we can use constructor overload without parameters, that allow collection to grow “infinite”:

// create the collection

BlockingCollection<int> collection = new BlockingCollection<int>(10);

Next we will add some elements and print some properties of collection (I’ll talk about it later):

collection.Add(200);

collection.Add(300);

collection.Add(400);

collection.Add(500);

Console.WriteLine("Count: " + collection.Count);

Console.WriteLine("BoundedCapacity: " + collection.BoundedCapacity);

Console.WriteLine("IsCompleted: " + collection.IsCompleted);

Console.WriteLine("IsAddingCompleted: " + collection.IsAddingCompleted);

Console.WriteLine();

After that we start the timer. At the callback delegate more elements will be added, and collection is mark as completed:

Timer timer = new Timer(delegate

    Console.ForegroundColor = ConsoleColor.Red;

    Console.WriteLine(DateTime.Now.ToLongTimeString() + " Adding " + " 600");

    collection.Add(600);

    Thread.SpinWait(300000000);

    Console.ForegroundColor = ConsoleColor.Red;

    Console.WriteLine(DateTime.Now.ToLongTimeString() + " Adding " + " 700");

    collection.Add(700);

    collection.CompleteAdding();

},

  null, 3000, Timeout.Infinite);

Timer callback will be executed in other thread, than the Main() method. It allow us to model concurrent access to the collection. Then we start taking elements from collection (I use different font colors in different threads):

foreach (var item in collection.GetConsumingEnumerable())

    Console.ForegroundColor = ConsoleColor.Gray;

    Console.WriteLine(DateTime.Now.ToLongTimeString() + " Taking " + item);

Here is the result of work:

Some words on how this example is work. We create a collection, and add 4 element to it. Next we start the timer, so we have 2 threads:

the main thread, where method Main() is executing and taking elements from the collection;
the timer thread, where timer callback is executing and adding elements;

On the screenshot you can see that first 4 elements took in time less than 1 second. After that main thread is blocked and waiting for new elements. Timer starts with 3 seconds delay, you can see it: element 500 was took in 40 sec. and element 600 was added in 43 sec. In the same 43 sec. main thread is unblocked and took newly added element 600. Next, the timer thread is executing SpinWait(), after that it add element 700 and complete the collection by calling collection.CompleteAdding(). And the main thread is consuming this element and unblock, because the collection is completed and all elements are processed. If we comment CompleteAdding() call, the main thread will be blocked forever, because timer callback is returned and nobody mark collection as completed. And the main thread will wait for new elements infinite. So, the completion mechanism allows to avoid such situation and provide good way to solve the issue.

Some words on collection’s properties. I think, the most interest of them are following:

IsAddingCompleted – returns true, if collection is marked as completed, else false;
IsCompleted – return true, if collection is completed and empty, else false;
BoundedCapacity – return collection’s max capacity. Returns –1 in case of growing collection;

Take a look on the screenshot, and you can notice, when collection contains some elements, both of bool properties are false, because collection in not marked as completed. In the last moment it both are true, because all elements is taken and collection is completed.

Ok, and what about order of elements in the collection? You may notice, that it work as FIFO queue. Is this a predefined behavior? Or we can change it? Answer to these questions give us more careful look at the constructor. It have overload, which contains IProducerConsumerCollection<T> parameter. It implement a storage for the elements. You can use one of the the following collections (described in the part 1), included in the .NET 4 beta 2:

ConcurrentStack<T> – thread-safe stack;
ConcurrentQueue<T> – thread-safe queue;
ConcurrentBag<T> – thread-safe unordered collection, which allows duplicates;

So, BlockingCollection<T> is not a data storage. It implement a logic of access to the storage and mechanism of completion. By default it use ConcurrentQueue<T> as a storage, so you can see FIFO behavior at the samples. We can change it to stack in the constructor:

BlockingCollection<int> collection =

                new BlockingCollection<int>(new ConcurrentStack<int>(), 10);

and we saw following result:

How you can see, order of elements now conform to LIFO. You can realize custom data storage by implementing interface:

public interface IProducerConsumerCollection<T> : IEnumerable<T>, ICollection, IEnumerable

    void CopyTo(T[] array, int index);

    T[] ToArray();

    bool TryAdd(T item);

    bool TryTake(out T item);

In addition, I want to talk about some interesting static methods of the class BlockingCollection<T>. You can interact with a number of collections simultaneously with the following set of static methods:

Blocking
- int BlockingCollection<T>.AddToAny(BlockingCollection<T>[], T);
- int BlockingCollection<T>.TakeFromAny(BlockingCollection<T>[], out T);
Non-Blocking
- int BlockingCollection<T>.TryAddToAny(BlockingCollection<T>[], T);
- int BlockingCollection<T>.TryTakeFromAny(BlockingCollection<T>[], out T);

All of these methods take an array of BlockingCollection<T> elements – it’s an array of collections, with it set you’ll interact. They are work as described early Add() and Take() methods, i.e. with blocking logic. Methods return index of the collection, where data has been added or taken, or –1 when the action attempt is fail. More detail description you can read in the MSDN.

And one more interesting fact. In the MSDN you can read, that, for example, TakeFromAny() method return –1, if the taking action is not successful. It’s a blocking method, and when you try to take an element from set of empty non-completed collections, it will wait infinite while an element will be added to any of collection. Or if you complete any of collection, it will throw an ArgumentException. So, you wait, or take an element, or take an exception. And I can’t understood, when TakeFromAny() can return –1. I create a topic on MSDN Parallel Extension forums, and Reed Copsey, Jr accept, that it’s just a documentation issue. He create an appropriate item on the MS Connect. So, you need to be careful when read current MSDN docs about thread-safe collections, because it’s just a preview and may contains an errors.

In conclusion, other interesting samples of BlockingCollection<T> usage you can find at the MSDN code gallery. Now there are only samples for beta 1, but soon refreshed sample pack will be available. I advice you to download Visual Studio 2010 beta 2, that is published this week. You can try to work with new features. It contains the latest Parallel Extensions library, that allow you to use full power of modern multi-core hardware.

Thread-safe data structures .NET 4.0 (part 1)

Vitus — Sat, 24 Oct 2009 04:20:00 GMT

.NET 4 contains rich set of tools, that allow to create parallelized code more easy. But when we start processing chunk of data in parallel threads, we need to synchronize these threads, and we need some storage for the results of work. Now exist a big number of methods, solving sync issues, and we can realize in code any of them. But MS Parallel Extensions team already do it, and .NET 4 beta versions include set of thread-safe data structures. They implement some popular types of collections, and I want to do an overview of them.

1. Queue: ConcurrentQueue<T>

This is a classic FIFO queue, which can be safely accessed from few threads simultaneously. API of this collection is very simple:

ConcurrentQueue<int> queue = new ConcurrentQueue<int>();

queue.Enqueue(10);

int t;

Console.WriteLine(queue.TryPeek(out t));

Console.WriteLine(queue.TryDequeue(out t));

TryPeek() and TryDequeue() methods return an element from queue, but TryPeek() doesn’t remove it from queue. Both methods returns true, if they take an element, else they return false. You can add an element by calling Enqueue() method. Count of elements in the queue you can get by Count property, and if collection contains at least 1 element, property IsEmpty returns true.

2. Stack: ConcurrentStack<T>

This is a simple LIFO stack, that can be used in the concurrent environment. In contrast to queue, you can easily add or take the range of elements:

ConcurrentStack<int> stack = new ConcurrentStack<int>();

stack.Push(10);

stack.PushRange(new int[] { 1, 2, 3, 4, 5 });

int t;

if (stack.TryPop(out t))

    Console.WriteLine("Pop: " + t);

if (stack.TryPeek(out t))

    Console.WriteLine("Peek: " + t);

int[] ts = new int[5];

int count;

if ((count = stack.TryPopRange(ts, 0, 3)) > 0)

    Console.WriteLine("PopRange");

    for (int i = 0; i < count; i++)

        Console.WriteLine(ts[i]);

Here is the result of work of this code:

TryPeek() and TryPop() methods return bool values, and TryPopRange() – count of retrieved elements. You may add range of values by calling PushRange() method.

3. Collection: ConcurrentBag<T>

This is an unordered collection of data, which is looks like a set, but it can store duplicate items. It doesn’t guarantee any order of elements. I think, it’s a most simple data structure:

ConcurrentBag<int> bag = new ConcurrentBag<int>(new int[] { 1, 1, 2, 3 });

bag.Add(70);

int t;

bag.TryPeek(out t);

Console.WriteLine(t);

bag.Add(110);

Console.WriteLine();

for (int i = 0; i < 3; i++)

    bag.TryTake(out t);

    Console.WriteLine(t);

This piece of code produce following console output:

Looking at screenshot, you can notice, that elements retrieved in LIFO. I repeat: no any orders is guaranteed.

Take a look at the constructor of bag – you can set initial range of elements, and constructors of queue and stack have appropriate overloads too.

4. Dictionary: ConcurrentDictionary<TKey, TValue>

It’s just a variant of Dictionary<TKey, TValue>, which can be accessed concurrently. API for this collection has some thread-safe specific features:

ConcurrentDictionary<string, string> dict = new ConcurrentDictionary<string, string>();

dict.TryAdd("name", "OFC340");

dict.TryAdd("age", "25");

dict.TryAdd("age", "25");

Create/update/delete actions can be executed by calling one of Try* methods, which return true on success, else false. In the code sample, first attempt to add value “25” with key “age” is ok, and TryAdd return true. Second attempt is fail, and TryAdd() return false, and exception will not be generated. Other example:

string t = string.Empty;

Console.WriteLine(dict.TryGetValue("nokey", out t));

Key “nokey” is not present in the dictionary, but TryGetValue() method doesn’t generate any exception, and only return false. You can remove an element as shown here:

Console.WriteLine(dict.TryRemove("age", out t));

Using properties Values and Keys you can receive actual on the moment of call collections of values and keys of dictionary.

So, you can see, that these data structures is very basic, and don’t contains tons of functions – only necessary things. I like this concept – it’s a base structures and too many functions is redundant. If you need something else, you can easily add it at any time.

In addition to described data structures, .NET 4 beta 1 contains list ConcurrentLinkedList<T>. But I don’t cover it, because MSDN contains following warning: «ConcurrentLinkedList(of T) is planned to be removed prior to the final release of Visual Studio 2010. Please do not use this class». And in the .NET 4 beta 2 this collection is removed.

Ok, I show you 4 basic thread-safe collections of data. In the next part of the article I show you BlockingCollection<T> – more complex and interest thread-safe storage. Stay tuned! :)