Shaun Xu

The Sheep-Pen of the Shaun


News

logo

Shaun, the author of this blog is a semi-geek, clumsy developer, passionate speaker and incapable architect with about 10 years experience in .NET. He hopes to prove that software development is art rather than manufacturing. He's into cloud computing platform and technologies (Windows Azure, Aliyun) as well as WCF and ASP.NET MVC. Recently he's falling in love with JavaScript and Node.js.

Currently Shaun is working at IGT Technology Development (Beijing) Co., Ltd. as the architect responsible for product framework design and development.

MVP


This morning I received an email from the Microsoft said that, a meaningful company, Microsoft Open Technologies Inc., had just release their first pre-production, the Redis on Windows.

 

Redis

If you had been working in the Linux world or interested in NoSQL field you should have probably known or heard about the Redis. Redis is an open source, advanced key-value store. Someone categorized Redis to the distributed cache since it’s an in-memory key-value store. Someone categorized it to the NoSQL, since it provides the feature saving your data into disk. And someone categorized it to the distributed queue, since it supports storing your data into hash and list type and provides the enqueue, dequeue and pub/sub functionalities. You see, Redis is a very powerful tool that can be used widely in building especially a large scaling system, as a distributed cache, NoSQL and message queue.

The original Redis is written in ANSI C, which sponsored by VMWare, works in most POSIX systems like Linux, *BSD, OS X without external dependencies. Linux and OSX are the two operating systems where Redis is developed and more tested, and it’s recommended to use Linux for deploying. As you can see there’s no Windows in the list. Microsoft had submitted a patch into Redis repository to make it available on Windows, but unfortunately Redis doesn’t have any plan to merge it into the main branch. This means all Windows version of Redis are all unofficial, includes this one.

Different from the basic in-memory key-value store, such as the Memcached, the Redis not only provides storing objects by keys, it also can store your data in vary types such as strings, hashes, lists, sets and sorted sets. Redis also have the atomic operations on these types, like appending to a string; incrementing the value in a hash; pushing to a list; computing set intersection, union and difference; or getting the member with highest ranking in a sorted set.

In order to achieve its outstanding performance, Redis works with an in-memory data store. But depending on the use cases, we can persist it either by dumping the dataset to disk every once in a while, or by appending each command to a log. So it looks like a NoSQL database as well. Other features include a simple check-and-set mechanism, pub/sub and configuration settings to make Redis behave like a cache.

 

Download, Build and Install Redis on Windows

The Redis on Windows now is available on the GitHub of the Microsoft Open Technologies Inc. site. You can download the full source here, which including the Redis Service, Redis Client, Redis Benchmark, etc..

In Windows the simplest way to build the Redis is to use the Visual Studio. If you have installed the C++ component then you can open its solution file under the “msvs” folder. Then just build it.

image

Navigate to the “msvs\debug” folder there will be a lot of messing files appeared. I don’t have any experience on C and C++ but I can find something useful. The most imortant two files are redis-server.exe and redis-cli.exe.

redis-server.exe The execution file that start a Redis server process on your machine.
redis-cli.exe The command line client tool that can operate against a Redis server.
image

The simplest way to start a Redis server is just to open a command windows and go to this folder, execute the redis-server.exe then you can see the Redis is now running and on the screen it will show the status every 5 seconds by default.

image

Very easy, right? If you want to check whether the Redis is working or not, just execute the redis-cli.exe in another command window, and input the Redis command “ping”. If the server you are connecting is fine it will response a “PONG” back to you.

image

If you had been using the Linux version of Redis before you will find that the execution files, output log and the commands are exactly same. This means you can use any tools and clients you are familiar with which running on the original Redis to this new windows version. For example, let put a key-value pair into the Redis store through the client.

image

I added two key-value pairs into my Redis by using the SET command, and then searched all keys in the server. It gave me the result on the client screen. And then I got one of them by specifying the key. This is a very common and simple usage. You can find the full command list on this page.

 

Redis and C#

The Redis on Windows product just ship the windows version of Redis server, without the connection client for the .NET language. But this is not a problem. Since as I mentioned below, the Redis on Windows follows all behaviors and interfaces of the original one. This means we can use any connection client that works with the original Redis to this Windows version.

There are many clients available on this page, includes C, C++, Java, PHP, Ruby, Python, etc.. And of course, the C#. I  was using the ServiceStack.Redis as the connection client in one of my previous project. It worked well with my Redis running on Ubuntu, so it should be able to work on the Redis on Windows version as well.

You can download the ServiceStack.Redis here, build it can add the reference of ServiceStack.Interfaces and ServiceStack.Redis into the C# project.

When using the ServiceStack.Redis we need to instant its RedisClient class by specifying the Redis server name (or IP) and port. Then your can execute the Redis commands from vary methods in the RedisClient class. Let’s firstly have a try on how to use the list feature of Redis, which it works like a queue.

First of all, I will invoke the FlushAll method to clear all keys in my Redis. This will remove all items that stored in the server.

   1: using (var redis = new RedisClient("127.0.0.1"))
   2: {
   3:     redis.FlushAll();
   4: }

Then I will start a thread to let user input something and append it into one of the list (queue) of my Redis. The name of this list would be “default”.

   1: var senderThread = new Thread(() =>
   2: {
   3:     using (var redis = new RedisClient("127.0.0.1"))
   4:     {
   5:         Console.WriteLine("Input message: ");
   6:         var message = Console.ReadLine();
   7:         while (!string.IsNullOrWhiteSpace(message))
   8:         {
   9:             redis.EnqueueItemOnList("default", message);
  10:  
  11:             Console.WriteLine("Input message: ");
  12:             message = Console.ReadLine();
  13:         }
  14:     }
  15: });
  16: senderThread.Start();

Next, in the main thread the application will try to dequeue any items from the list, and print the value in the console.

   1: using (var redis = new RedisClient("127.0.0.1"))
   2: {
   3:     var result = string.Empty;
   4:     while (string.Compare("q", result, false) != 0)
   5:     {
   6:         result = redis.BlockingDequeueItemFromList("default", TimeSpan.FromSeconds(5));
   7:         Console.WriteLine("DEQUEUE RESULT = [{0}]", result);
   8:     }
   9: }

The full code would be like this.

   1: static void Main(string[] args)
   2: {
   3:     // clean up
   4:     using (var redis = new RedisClient("127.0.0.1"))
   5:     {
   6:         redis.FlushAll();
   7:     }
   8:  
   9:     // enqueue thread definition
  10:     var senderThread = new Thread(() =>
  11:     {
  12:         using (var redis = new RedisClient("127.0.0.1"))
  13:         {
  14:             Console.WriteLine("Input message: ");
  15:             var message = Console.ReadLine();
  16:             while (!string.IsNullOrWhiteSpace(message))
  17:             {
  18:                 redis.EnqueueItemOnList("default", message);
  19:  
  20:                 Console.WriteLine("Input message: ");
  21:                 message = Console.ReadLine();
  22:             }
  23:         }
  24:     });
  25:     senderThread.Start();
  26:  
  27:     // dequeue operation
  28:     using (var redis = new RedisClient("127.0.0.1"))
  29:     {
  30:         var result = string.Empty;
  31:         while (string.Compare("q", result, false) != 0)
  32:         {
  33:             result = redis.BlockingDequeueItemFromList("default", TimeSpan.FromSeconds(5));
  34:             Console.WriteLine("DEQUEUE RESULT = [{0}]", result);
  35:         }
  36:     }
  37:  
  38:     Console.WriteLine("Done!");
  39:     Console.ReadKey();
  40: }

As you can see after I input some strings it will be appended into the list, and then dequeued from the list at once.

image

In this example we only have one thread to dequeue the items. If there are more than one client which is dequeuing the same list, only one of them can retrieve the item. In message bus glossology this is named “queue”, which means only one consumer can retrieve an item. And if there’s no consumer available the items in the queue should be stay there until at least a consumer connected and dequeued.

image

If it’s a reliable queue, the items in the queue will be remained even though the message bus was terminated or the machine is crashed.

Next, let’s have a try on the pub/sub mode. Pub/Sub mode, also known as the “topic” mode in message bus field. Different from the “queue” mode I mentioned below, in “topic” mode a message will be received by all consumers that is subscribing on the topic. (In Redis it’s said “channel” rather than “topic”.) But if some consumers were not available at that moment the message will be lost and will never be delivered to them even though they got back later.

image

To use the Pub/Sub in Redis from C# as a consumer we also need to create the RedisClient instance, and use its CreateSubscrption method to bind it to a channel. And when a message comes it will raise the OnMessage event and we can handle it to do our own business logic.

   1: static void ConsumerAction(object name)
   2: {
   3:     using (var consumer = new RedisClient("127.0.0.1"))
   4:     {
   5:         using (var subscription = consumer.CreateSubscription())
   6:         {
   7:             subscription.OnSubscribe = (channel) =>
   8:             {
   9:                 Console.WriteLine("[{0}] Subscribe to channel '{1}'.", name, channel);
  10:             };
  11:             subscription.OnUnSubscribe = (channel) =>
  12:             {
  13:                 Console.WriteLine("[{0}] Unsubscribe to channel '{1}'.", name, channel);
  14:             };
  15:             subscription.OnMessage = (channel, message) =>
  16:             {
  17:                 Console.WriteLine("[{0}] Received message '{1}' from channel '{2}'.", name, message, channel);
  18:             };
  19:  
  20:             subscription.SubscribeToChannels("default");
  21:         }
  22:     }
  23: }

In order to demonstrate the behavior of Pub/Sub let’s create there threads as three consumers that are subscribing the same channel.

   1: var consumerThread1 = new Thread(new ParameterizedThreadStart(ConsumerAction));
   2: var consumerThread2 = new Thread(new ParameterizedThreadStart(ConsumerAction));
   3: var consumerThread3 = new Thread(new ParameterizedThreadStart(ConsumerAction));
   4: consumerThread1.Start("Consumer 1");
   5: consumerThread2.Start("Consumer 2");
   6: consumerThread3.Start("Consumer 3");

And use the PublishMessage method of the RedisClient to send some messages in the channel in the main thread. The full code would be like this.

   1: static void ConsumerAction(object name)
   2: {
   3:     using (var consumer = new RedisClient("127.0.0.1"))
   4:     {
   5:         using (var subscription = consumer.CreateSubscription())
   6:         {
   7:             subscription.OnSubscribe = (channel) =>
   8:             {
   9:                 Console.WriteLine("[{0}] Subscribe to channel '{1}'.", name, channel);
  10:             };
  11:             subscription.OnUnSubscribe = (channel) =>
  12:             {
  13:                 Console.WriteLine("[{0}] Unsubscribe to channel '{1}'.", name, channel);
  14:             };
  15:             subscription.OnMessage = (channel, message) =>
  16:             {
  17:                 Console.WriteLine("[{0}] Received message '{1}' from channel '{2}'.", name, message, channel);
  18:             };
  19:  
  20:             subscription.SubscribeToChannels("default");
  21:         }
  22:     }
  23: }
  24:  
  25: static void Main(string[] args)
  26: {
  27:     using (var redis = new RedisClient("127.0.0.1"))
  28:     {
  29:         redis.FlushAll();
  30:     }
  31:  
  32:     var consumerThread1 = new Thread(new ParameterizedThreadStart(ConsumerAction));
  33:     var consumerThread2 = new Thread(new ParameterizedThreadStart(ConsumerAction));
  34:     var consumerThread3 = new Thread(new ParameterizedThreadStart(ConsumerAction));
  35:     consumerThread1.Start("Consumer 1");
  36:     consumerThread2.Start("Consumer 2");
  37:     consumerThread3.Start("Consumer 3");
  38:  
  39:     using (var publisher = new RedisClient("127.0.0.1"))
  40:     {
  41:         Console.WriteLine("Input message: ");
  42:         var message = Console.ReadLine();
  43:         while (!string.IsNullOrWhiteSpace(message))
  44:         {
  45:             publisher.PublishMessage("default", message);
  46:             Console.WriteLine("Input message: ");
  47:             message = Console.ReadLine();
  48:         }
  49:     }
  50:  
  51:  
  52:     Console.WriteLine("Done!");
  53:     Console.ReadKey();
  54: }

Then let’s run our application and input some messages. You can see in this sample all of our three consumers received the messages and process their own business logic.

image

 

Summary

Distributed cache, distributed data store and distributed message queue are very important components when we build a large system with high scalability and high performance. Redis is one of the most powerful product. It can be used as a cache, a NoSQL database and message queue as well. And the performance of Redis is outstanding. But for the developers who is working on Windows platform it’s a little pity that Redis doesn’t support Windows by default.

In this post I forwarded the announcement that Microsoft Open Technologies had just published their first un-commercial Redis on Windows version. By using it we can deploy the Redis server on Windows, which means no need to learn on how to build and run it on Linux.

I also demonstrated how to use it from C# through the ServiceStack.Redis client library. Since the Windows version of Redis follows all APIs as the original one we can use anything we are familiar with.

Integrated with my previous post about using the WCF transport extension to build a high scalability system on top of the message bus, since the Redis supports the list data type which implements the message bus feature, we can use it as our underlying transportation as well.

 

Hope this helps,

Shaun

All documents and related graphics, codes are provided "AS IS" without warranty of any kind.
Copyright © Shaun Ziyan Xu. This work is licensed under the Creative Commons License.

Comments

Gravatar # re: Redis on Windows
Posted by Redisuser on 7/23/2012 7:15 PM
Hello is there similar effort for Redis queue or rq ?
Gravatar # re: Redis on Windows
Posted by Shaun Xu on 7/24/2012 9:41 AM
Redisuser,

Not sure if I understood your question but my first sample showed the queue capability of redis.
If the 'rq' means reliable queue, I don't have the answer I'm afraid. Sorry.

Shaun
Gravatar # re: Redis on Windows
Posted by Redisuser on 7/24/2012 2:08 PM
http://python-rq.org/
Redis queue I referred to is in link above
Gravatar # re: Redis on Windows
Posted by Shaun Xu on 7/24/2012 3:17 PM
Sorry I don't know about this one. I'm not familiar with Python.
Gravatar # re: Redis on Windows
Posted by Redisuser on 8/8/2012 2:55 PM
Is there a possible version for VS 2008 Pro ?
instead of VS'10 ?
Gravatar # re: Redis on Windows
Posted by Shaun Xu on 8/8/2012 3:35 PM
I think this is no problem. Just make sure you build the ServiceStack.Redis in .NET 3.5.
Gravatar # re: Redis on Windows
Posted by Shriram on 8/8/2012 3:34 PM
When I open the solution file to build it,
I get the error message below:
"The selected file is a solution file, but was created by a newer version of this application and cannot be opened."

Gravatar # re: Redis on Windows
Posted by Shaun Xu on 8/8/2012 4:11 PM
You might have to open the projects in VS2008 manually and then save the solution and build it.
Gravatar # re: Redis on Windows
Posted by Ramesh on 10/7/2012 4:41 AM
very nice!.. i just tried all your examples on windows..
great it worked!. Thank you. :)
Gravatar # re: Redis on Windows
Posted by Barut on 12/16/2012 12:43 AM
Hello Shaun, your Redis tutorial on page "http://geekswithblogs.net/shaunxu/archive/2012/04/27/redis-on-windows.aspx" is very great. I am a Informatics student and need your Help. I am using WindowsXp(32bit) and installed eclipse-indigo too. I have to benchmarking redis server on yahoo(ycsb) with my pc and report its performance . Nobody knows the answer. Can you declare me how I can run redis on YCSB? Thanks a lot.

Kind Regards

Barut
Gravatar # re: Redis on Windows
Posted by Shaun Xu on 12/16/2012 4:58 PM
Sorry Barut, this is out of my knowledge.
Post A Comment
Title:
Name:
Email:
Comment:
Verification: