Szymon Kobalczyk's Blog

A Developer's Notebook

  Home  |   Contact  |   Syndication    |   Login
  103 Posts | 6 Stories | 349 Comments | 365 Trackbacks

News

View Szymon Kobalczyk's profile on LinkedIn

Twitter












Article Categories

Archives

Post Categories

Blogs I Read

Tools I Use

Tuesday, August 31, 2010 #

For my next project I decided to try to upgrade my SERB robot that I showed here last year to use Netduino. This robot was designed by great inventors from oomlout and you can buy it as a nicely packaged kit from them. However because the design is open source you can also download the project files and if you have access to a laser cutter order all the acrylic pieces there. It is also very easy to build following the instructable (you can find few more photos from my build here).

Netduino Controlled Servo Robot

The original SERB robot uses Arduino as its brain, so it was quick upgrade to Netduino thanks to the same form factor and pin layout. On the above photo you can see the Netduino is mounted on the back below the prototyping shield. Oomlout’s instructable suggests to use separate 9V battery for the board, but I found that it works just fine using only 4xAA batteries to power whole setup (Netduino, Servos and Xbee).

Robot drive system

The robot is driven by pair of continuous rotation servos. You can buy such servos from various hobby electronics and robotics shops, but most ordinary servos can be modified this way. Some require to remove potentiometer and replace it with resistor divider, while with others you can leave potentiometer in place but release it from rotating and hold it in centered position. You can do it even with tiny servos. Either way you get a very cheap high torque reversible DC gearmotor. And the best part is that you can control it just as any other servo using PWM, with no need for additional motor driver circuits. Thus I could use the Netduino Servo class that was already written by Chris Seto (I only adjusted the pulse range so it matches default values from Arduino Servo library).

But with all good said above, the one drawback with using a servo is that we don’t have a much control over it’s speed. For most values it will jump right to the max speed. I made few test measuring how far my robot would drive at various power levels (by setting servo from 0 to 90 degrees). As you can see on the graph below for values above 20 it will always drive at maximum speed. However for values from 0 to 16 the speed increase is almost linear thus in my code I’m scaling the speed values to this range. I’m also using a timer to ramp up the speed in smaller increments so the robot doesn’t accelerate or stop to fast. Please keep in mind that this works well with the particular model of servos I’m using (Futaba S3001) but you might need to adjust it for your robot.

image

I encapsulated the code used to drive the robot in the Serb class located in the NetduinoSerbDemo project. It controls both motors and provides helper classes for various commands. The class RandomTest implements a simple random movement pattern using these commands.

Joystick

Now that we know that the basic drive operations work we can build some applications on top of it. Before we make the robot go around in full autonomous mode (which requires adding few more sensors so robot can navigate in its small world) lets drive it manually with remote control. It would be nice to have some kind of joystick for it, wouldn’t it? Sure you could use the Sparkfun’s joystick shield (which you need to modify a little bit). But since I don’t have one at hand for this project I’m going to use a Nintentdo Wii Nunchuck (besides oomlout uses the same in another instructable).

You see, smart people found out that Nintendo accessories are communicating with Wiimotes using the I2C bus, so it’s quite easy to interface with them from the microcontroller once you know the protocol. There is also available a handy adapter called WiiChuck that plugs directly to Arduino. In the end it’s a cheap way to get a good quality device with analog stick, two buttons and an accelerometer. I bought mine for under $10 – it’s not original but works just fine.

The I2C pins on Netduino are in the same place as on Arduino (SDA on analog pin4, and  SCL on analog 5), but Arduino can be also set to provide required power on neighboring pins (analog pin 2 for ground, and analog pin 3 for +5V). It seems that this can’t be done on Netduino so I had to plug the adapter to the breadboard and wire it properly instead.

_MG_1206

The second project called NetduinoSerbRemote contains the WiiChuck class with my implementation of .NET MF driver for Wii Nunchuck. It is based on the oryginal code by Tod E. Kurt, but I extended it with information found on WiiBrew, Arduino and other forums. For example it implements motion calculations to calibrate the accelerometer output (found in the oomlout version). It also has new I2C device initialization procedure that should disable encryption (this might improve compatibility with some wireless Nunchucks).

After creating the WiiChuck instance, you simply call the GetData method, and it will return true if it succeeded, or false otherwise.  Then you can get the extracted values from properties representing position of analog stick, buttons, and accelerometer. For the robot I’m simply mapping the joystick position to directly to speed and direction. Another mode is activated when you press and hold the Z button. Then it will use values from accelerometer instead of the joystick.

Remote control

Looking at the previous photos some of you might already guessed that I’m going to use XBee radios for communication from my remote to the robot. In fact this is the easiest way to add wireless communication between two microcontrollers. It uses serial ports, thus you only need to connect the corresponding UART pins to the radios to get it working.

Note that there are several models of XBee modules on the market. Some provide many advanced capabilities including mesh networking. But if you are going to use them only to transmit data from one device to another you should get an older version now called XBee 802.15.4 (series 1) that can be easily configures for point-to-point networking. I got mine from Adafruit with corresponding adapters that make it easy to plug the module on breadboard or directly to the USB FTDI-TTL-232 cable. You can learn much more from Lady Ada’s website and from Tom Igoe’s book Making Things Talk.

In this project I decided to use UART1 on both Netduinos, so I put a wire from Netduino digital pin 0 to TX pin on XBee, and from pin 1 to RX (and of course connected it to +5V and ground as well). For communication I adapted the simple protocol used in yet another instructable from oomlout. Each message begins with header ‘AAA’, followed by single letter command code, and optional parameter. Thus messages are always 5 bytes long. Here are currently implemented command codes:

private static class Command
{
   public const char Unknown = '?';
   public const char Forward = 'F';
   public const char Backward = 'B';
   public const char Left = 'L';
   public const char Right = 'R';
   public const char SetPower = 'P';
   public const char Stop = 'S';
   public const char SetSpeedLeft = 'X';
   public const char SetSpeedRight = 'Y';
}

On the Netduino acting as the remote the values are read from Wii Nunchuck, mapped to SetSpeedLeft and SetSpeedRight commands, and send to serial port. On the other side the SerbRemoteClient class parses incoming data, and interprets these commands accordingly. I hope the code will be easy to follow for everyone.

The solution also contains a very simple WPF application that I used for testing. To use this app I connected one of the XBee modules to PC via with FTDI USB cable. The name and baud rate of the serial port can be set in the app.config file.

Here is the source code for this project:


Saturday, August 21, 2010 #

Netduino Some time ago I wrote about FEZ Mini and FEZ Domino – first affordable development boards for .NET Micro Framework. Today I’m excited to tell you about another device called Netduino. Similar to FEZ Domino this board is pin compatible with Arduino, and therefore most of Arduino shields should work fine on Netduino. This makes transitioning your project quite easy. Only care should be taken to ensure that shield can run at 3.3V logic levels (because Arduino runs at 5V).

Of course Netduino is much more powerful than Arduino, thanks to Atmel 32-bit microcontroller running at 48Mhz, 128KB flash for code, and 60KB of RAM. What’s more, this is a first truly open source .NET MF device, meaning that you can modify and compile the source code of its firmware yourself. You will find more information and rapidly growing community of Netduino users at http://www.netduino.com/. I got mine yesterday and it looks great. But enough said about Netduino – lets put it to some good use.

You can find few introduction videos at http://www.netduino.com/projects/ to get you up and running with hardware and software configuration so I won’t repeat it here. The first tutorial shows you how to blink a single onboard LED, and it would be easy to do the same with more LEDs – you can simply wire them to all other digital I/O pins of the board. But instead of single LEDs you might want to use some other LED components, such as 10-segment LED bar graph or 7-segment display, and it will work the same way.

7-segement display and 10-segment bar graph componentsNetduino has 14 digital pins, and 6 analog pins that can be used as digital as well. But even with simple displays shown above we will quickly run out of available pins. In this tutorial I will demonstrate how you can use shift registers to extend the number of digital output pins available on the device.

Shift registers are very simple ICs, and one popular chip is 74HC595. It’s referred as 8-bit serial to parallel converter. It requires only three lines connected to the microcontroller (data, click, and latch) to get 8 outputs. But what’s cool is that you can connect more than one of such chips in series in case you need more outputs. You can learn all about shift registers from this Arduino tutorial so I won’t repeat it here, and I’ll only show how to use them with .NET Micro Framework devices.

Hardware setup

I’ve built a simple circuit with two 7-segment displays, so I’m also using two shift registers. You can build it on a breadboard as shown below, but I soldered it on a prototyping board to avoid messing with lots of wires. There is also an Arduino shield available called EZ-Expander Shield that you can use in your projects.

Here are the schematics of the circuit. I made it using Fritzing tool that I already presented here some time ago. The Fritzing file is included with the source code and I would like to thank Stanislav "CW2" Šimíček for creating a custom Netduino part for Fritzing:

diagram

A seven-segment display is made of 8 LEDs arranged in a matrix so that it can display decimal digits. It has 5 connectors on each side where central pin is used either as common ground or power (depending if you have a common cathode or anode version). Remaining pins are used to control individual segments. This diagram shows how the connections correspond to the LED segments:

You will notice that in my circuit I have common annode version. Also make sure to put 220Ohm resistor on other pins just like you do to protect regular LEDs. These pins are connected to the corresponding eight outputs of the shift register.

We need to connect only three pins to connect the shift register to netduino. In my circuit I connected these digital pins:

  • pin 11 to serial data input pin (DS)
  • pin 12 to shift register clock pin (SH_CP)
  • pin 8 to storage register latch pin (ST_CP)

Only the first shift register is connected directly to netduino, because we can wire more shift registers in series (cascade). Latch and clock pins are wired directly to same input, but the data pin for second register must be connected to the serial out pin (Q7") of first register. When we send the data signals to the first register after 8th bit it will overflow and forward these signals through the out pin to the 2nd register. You can find detailed explanation on how shift registers work in the article Controlling Your Festive Lights with the .NET Micro Framework at Coding4Fun.

You can download the source code for this project here. While it references Netduino assemblies it should work with little changes on other .NET Micro Framework devices like FEZ Domino, Meridian/P and others.

74HC595 and 7-segment display drivers

Now lets look at the code for controlling the shift register. I have separated this code to a class LED_74HC595_IOProvider. It implements simple interface ILED_IOProvider so it can be reused with different types of LED display components, and in the project you will find classes for 7-segment display and 10-segment bar graph.

Constructor of the LED_74HC595_IOProvider stores number of connected registers, and initializes output ports assigned to the corresponding shift register pins. Optionally you can set if data should be sent in MSB (most significant bit first) or LSB (least significant bit first) mode:

public LED_74HC595_IOProvider(short numRegisters, Cpu.Pin latchPin, 
    Cpu.Pin clockPin, Cpu.Pin dataPin, BitOrder bitOrder)
{
    _numRegisters = numRegisters;
    _bitOrder = bitOrder;

    _latchPort = new OutputPort(latchPin, false);
    _clockPort = new OutputPort(clockPin, false);
    _dataPort = new OutputPort(dataPin, false);
}

The Write method takes a buffer of bytes and sends them to the shift registers (each byte represents outputs of one shift register):

public void Write(params byte[] buffer)
{
    // Ground latchPin and hold low for as long as you are transmitting
    _latchPort.Write(false);

    for (int i = 0; i < buffer.Length; i++)
    {
        ShiftOut(buffer[i]);
    }
            
    // Pulse the latch pin high to signal chip that it 
    // no longer needs to listen for information
    _latchPort.Write(true);
    _latchPort.Write(false);
}

But all the heavy work is done inside the ShiftOut helper method that signals the bits in individual byte:

private void ShiftOut(byte value)
{
    // Lower Clock
    _clockPort.Write(false);

    byte mask;
    for (int i = 0; i < 8; i++)
    {
        if (_bitOrder == BitOrder.LSBFirst)
            mask = (byte) (1 << i);
        else
            mask = (byte)(1 << (7 - i));

        _dataPort.Write((value & mask) != 0);

        // Raise Clock
        _clockPort.Write(true);

        // Raise Data to prevent IO conflict 
        _dataPort.Write(true);

        // Lower Clock
        _clockPort.Write(false);
    }
}

Now let’s look at the SevenSegmentDisplay class. The ShowValue method will display the provided decimal number. It uses a static lookup table to map digits to corresponding pattern of LED segments. I simply mapped the patterns from wikipedia. Currently only decimal digits are supported, but one extension to this class might be to allow using hexadecimal output formatting as well.

public void ShowValue(int value)
{
    // Map digits to output patterns
    var buffer = new byte[_numDigits];
    for (int i = _numDigits - 1 ; i >= 0 ; i--)
    {
        byte digit = digits[value % 10];

        // Negate patten if its common anode display
        if (!_isCommonCathode)
            digit = (byte)(~digit);

        buffer[i] = digit;
        value = value / 10;
    }

    // Write output buffer
    _ledIoProvider.Write(buffer);
}

Example 1 – using potentiometer

For the simple test we will display a current reading of a potentiometer connected to one of analog inputs on netudiono. Don’t forget to connect Aref pin to 3.3V, otherwise the analog readings will be unknown. This is important fact to keep in mind because arudino has Aref internally connected by default.

This example is implemented in SevenSegmentLedWithPotentiometer class, and devices are setup in its constructor:

public SevenSegmentLedWithPotentiometer()
{
    // Create IO provider for shift registers
    display_IO = new LED_74HC595_IOProvider(2, LED_Latch_Pin, LED_Clock_Pin, LED_Data_Pin,
                        LED_74HC595_IOProvider.BitOrder.LSBFirst);

    // Create 7-segment display interface
    display = new SevenSegmentDisplay(2, display_IO, false);

    // Initialize analog input for potentiometer
    analogPort = new AnalogInput(Analog_Pin);

    // Set value range to display in 2 digits
    analogPort.SetRange(0, 99);
}

The main loop simple as well. Each iteration it reads the current value and sends it to the display:

public void Run()
{
    while (true)
    {
        // Read analog value 
        var value = analogPort.Read();

        // Show the value
        display.ShowValue(value);

        // Wait a little
        Thread.Sleep(100);
    }
}

Example 2 – using SHT15 temperature and humidity sensor

Now that we know that the display works fine lets use it to build a more interesting device, namely a digital temperature and humidity monitor. I’m going to use a Sensirion SHT15 sensor (SHT10, SHT11 and SHT15 works exactly the same and differ only by accuracy).  You can buy a simple to use SHT15 breakout board from Sparkfun.The communication protocol for this sensor is bit tricky, but fortunately Elze Kool has already written a .NET Micro Framework SHT1x driver so I’m going to use it in my example. 

This example is implemented in SevenSegmentLedWithTemperature class. Constructor adds initialization to the SHT1x sensor:

private void Setup()
{
    // Set initial button state
    ButtonPressed = buttonPort.Read();

    // Soft-Reset the SHT11
    if (SHT1x.SoftReset())
    {
        // Softreset returns True on error
        throw new Exception("Error while resetting SHT11");
    }

    // Set Temperature and Humidity to less acurate 12/8 bit
    if (SHT1x.WriteStatusRegister(SensirionSHT11.SHT11Settings.LessAcurate))
    {
        // WriteRegister returns True on error
        throw new Exception("Error while writing status register SHT11");
    }

    // Run timer for measurments
    sensorTimer = new Timer(MeasureTemperature, null, 200, 500);
}

In addition we want to setup the onboard button, so that when is pressed display will show humidity instead of temperature. I'm using InterruptPort that will trigger an event each time button is pressed on released. This way I don’t have to poll the button state each time in the main loop:

private void HandleButtonInterrupt(uint data1, uint data2, DateTime time)
{
        // Update button state
        ButtonPressed = (data2 != 0);
}

The temperature and humidity measurement takes a bit of time (maximum of 20/80/320 ms for 8/12/14bit resolution) and microcontroller has to wait for the measurement to complete. We don’t want to block the main thread waiting for this, so the obvious choice is to put the measurement in another thread. Here I’m using the Timer class to do measurements every 500 ms.

private void MeasureTemperature(object state)
{
    Stopwatch sw = Stopwatch.StartNew();

    // Read Temperature with SHT11 VDD = +/- 3.5V and in Celcius
    var temperature = SHT1x.ReadTemperature(SensirionSHT11.SHT11VDD_Voltages.VDD_3_5V, 
        SensirionSHT11.SHT11TemperatureUnits.Celcius);
    Debug.Print("Temperature Celcius: " + temperature);

    // Read Humidity with SHT11 VDD = +/- 3.5V 
    var humiditiy = SHT1x.ReadRelativeHumidity(SensirionSHT11.SHT11VDD_Voltages.VDD_3_5V);
    Debug.Print("Humiditiy in percent: " + Humidity);

    Debug.Print("Sensor measure time: " + sw.ElapsedMilliseconds);

    Temperature = temperature;
    Humidity = humiditiy;
}

After moving code for taking measurements and updating button state all that’s left in the main loop is simply updating the display when state changes (I added blinking LED to show how often the update is required):

private void Loop()
{
    if (_needRefreshDisplay)
    {
        _needRefreshDisplay = false;

        statusLEDPort.Write(true);

        // Display current value
        if (ButtonPressed)
        {
            led.ShowValue((int) Math.Round(Temperature));
        }
        else
        {

            led.ShowValue((int) Math.Round(Humidity));
        }

        statusLEDPort.Write(false);
    }
}

That’s all. The video below shows both completed exercises. Enjoy!


Friday, February 26, 2010 #

After seeing that some of my friends get fancy customized pictures for their Facebook and Twitter profiles I thought it would be cool to get one too. But not for me – for my wife. You see, she currently works as IT Security Officer (the best I know btw.) but earlier when she started as security auditor and she worked as part of the “tiger team”, she often said she wants her picture like the movie poster from "Men in Black”. So this is what I had in mind:

Menin_Black

Given the upcoming Valentine’s Day it occurred to me this could be the perfect gift (you see it gets hard over time in marriage to came up with something unique). But instead of going to websites that offer such service, I filled the job opening on oDesk.com (freelance job exchange – highly recommended). Couple days later I got handful of candidates to choose from. In the end I choose the graphics artist from Thailand, Mr. Wanchana Intrasombat (or Vic for short). After seeing Vic’s profile and gallery (and this picture in particular) I was sure this is the style I’m looking for and that he is the perfect guy for the job.

In the end I can say that you can’t go wrong with Vic. He made truly exceptional job, and working with him was fascinating experience on its own. He was very open to my ideas and quick to accommodate them (even when I changed my mind several times in the process).

So here is the final picture:

Avatar of Asia -final version

And here is another sketch he made just for fun of it:

Asia standing

Go check out Vic’s gallery – he has tons of good stuff there. If you need a great graphics artist or just want to get your own avatar feel free to contact him: vic__unforce (at) hotmail (dot) com

And tell us what you think about the pictures!


Thursday, February 25, 2010 #

Yesterday I picked up a nice package from TinyCLR.com. Yes, it is yet another robot (third in fact), but this time it runs .NET! How freaking cool is that!

FEZ Mini Robot Kit

This robot is controlled by FEZ Mini board that runs .NET Micro Framework. What’s interesting is that this board has pin-out compatible with Basic Stamp from Parallax. You can also uses it easily on a breadboard for prototyping (just like Boarduino). TinyCLR.com also offers a larger board called FEZ Domino that has pin-out compatible with Arduino so you can use your existing shields. It is more expensive but notice that this board already includes microSD card reader and USB host connector. Both boards are based on USBizi chipset from GHI Electronics that in turns runs on 72Mhz NXP ARM processor.

FEZ MiniFEZ Domino

But the huge thing for me is that I can program this robot in Visual Studio and benefit from all the debugging goodies (breakpoints, variable inspection, stepping, etc.).

The robot kit itself comes from INEX Robotics – company that makes many interesting educational kits for robotics and electronics. Over there the robot kit is called POP-BOT and only difference is that it is controlled by Arduino compatible board. But you can also download the POP-BOT Manual that has 130 pages of many cool project ideas (line tracking, object avoiding, controlling servo motor, and serial LCD). I’m sure it will be lots of fun converting these projects to .NET Micro Framework.

FEZ Mini Robot

Two reflective sensors are included in the kit (useful for line following and edge detection projects), and you can order additional components both from TinyCLR.com and other robotics sites. Many construction parts are included in the kit so it is very easy to attach additional sensors or other parts. As you can see on the picture above, I already added a Sharp IR distance sensor in front (so I can teach the robot to not bump on walls). I also added an Xbee expansion board on the back so one day I can control the robot remotely (and my Holy Grail is to connect the robot to Microsoft Robotics Developer Studio).

Here is a video of the robot running the default program:

And here is my little robot builder crew at work:

Antoś builds a robot

Of course both FEZ Mini and FEZ Domino boards can be used not only for robotics, but also in many different applications. Go to the TinyCLR.com website and check out the Projects tab. For quick introduction to .NET MF you should also download the Beginners Guide to C# and .NET Micro Framework, free ebook from Gus Issa.

In my previous post I complained how upset I’m that there is no cheap .NET Micro Framework hardware for hobbyists. Now I can take it back. IMHO we finally have a very powerful alternative to Arduino and similar platforms, with the price that won’t break the bank (especially with FEZ Mini).


Monday, January 04, 2010 #

Just after finishing my multicolor RGB controller shield for Arduino, I came across Fritzing, a program that lets you convert your breadboard prototypes into a physical PCB. Since I got one design working at hand, I decided to give it a try. The process is very straightforward.

First you simply put all the components in the breadboard view. The parts library contains most common parts, and you use wires to connect them – exactly the same as you would do on your physical prototype:

Fritzing - Breadboard viewNext you can switch to schematic view where you can check if all the connections are correct:

Fritzing - schematic viewFinally you go to the PCB view and layout your parts on the board. Then you need to route all traces, and while Fritzing does quite a good job with auto-routing, it is worth to spend some time and review the design. In my case I was able to eliminate all the jumper wires – needed otherwise because currently Fritzing supports only single-sided PCBs.

Fritzing - PCB view In my case I designed the board to fit as an Arduino shield. As you can see it consists of a 12V voltage regulator circuit for external power source and ULN2003 converter to drive two RGB outputs.

After you are happy with you can save it in multiple output formats. For me the the etchable PDF format was especially handy because I could print it as a paper prototype and check if all the physical parts will fit properly.

Fritzing also offers a website where you can upload your projects along with any additional documentation, images, and source code. You can find my project page here: http://fritzing.org/projects/multicolor-rgb-led-controller-shield-for-arduino/

On top of that, starting with January 2010 they start a Fritzing Fab Service, and produce the real, physical PCBs. In preparation for this, they selected 24 projects and made them manufactured. I was very happy that my design was selected as one of these projects.

I got my board short after Christmas, and it looks very professional (at least for me :-)):

And here is the final product:

P1150499

As a geek I always wanted to hack the hardware like this, but at the same time I was intimidated by my little knowledge of electronics. So its truly  amazing how much can be done with such simple tools, and how much I learned in past half a year since my good friend Marcin Książek introduced me to Arduino. For me it’s just the essence of the Arduino phenomen – perfect combination of open source hardware, software and the community.

On the side note, I’m bit upset that the same didn’t happened with the .NET Micro Framework, because I think on the software side it has even greater potential (like you can write and debug your code in Visual Studio you know and love). And although I think making the platform open source is a great move, the hardware costs are the biggest barrier right now. Especially if you consider using it for few hobby projects (I don’t know what’s the pricing looks like for mass production).

I will be exploring .NET Micro Framework shortly so stay tuned.


Saturday, January 02, 2010 #

For me one of the coolest new features added in Silverlight 4 is support for capturing video and audio using webcams and microphone. As we’ve seen during Scott Guthrie’s keynote demonstration it is now possible to capture image frames from video stream and apply some interesting effects to it. On top of that we can even process the video and audio streams directly on the client (i.e. inside the browser).

Because Silverlight 4 Beta was already available at PDC I could try the webcam support right away, and so I jumped to refresh my computer vision projects. Unfortunately I never did find time to finish this but I think it’s still worth publishing the code as it is.

One things I was playing with before was the Touchless SDK. It is an open source .NET library for visual object tracking created by Mike Wasserman. In short it identifies objects with specific color characteristics and sends you their position within the image scene enabling to use these objects as input devices. It means you can control your applications without touching the screen or keyboard (and thus it’s called touchless :-)

Original project page is here: http://touchless.codeplex.com/

Touchless SL

Over a year ago I successfully used this library in WPF so now it was quite easy to convert it again to Silverlight 4. You can check it out and download the source code on the demo page. Below I summarize some notes of things I learned about Silverlight 4 API in hopes that Silverlight team can address these issues before final release.

As a starting point I used this excellent summary by Mike Taulty. The main class that handles video processing is the CameraTracker. It extends from VideoSink in order to get access to incoming video samples. Each sample frame is represented by byte array which format is specified by VideoFormat structure. In order to get current VideoFormat you have to override the OnFormatChange method (call to videoSource.VideoCaptureDevice.DesiredFormat will throw exception).

I learned that although VideoFormat specifies the PixelFormatType, currently it is an enum with only two values: Format32bppArgb and Unspecified. It’s WPF counterpart – PixelFormat structure - offers much more including masks and bytes per pixel. And instead of enum we get a static PixelFormats class with number of common formats declared as properties. I hope this will be changed in Silverlight 4 RTM.

Another thing to be aware of is that the video frame buffer is flipped vertically – so you need to read the image lines from bottom to top. I handle this in the GetPixel and SetPixel methods of RawBitmapAdapter helper class:

public RgbColor GetPixel(int x, int y)
{
    int p = _scan0 + (y * _stride) + (x * _bytesPerPixel);
    byte b = _imageData[p];
    byte g = _imageData[p+1];
    byte r = _imageData[p+2];
    return new RgbColor(r, g, b);
}
public void SetPixel(int x, int y, RgbColor color)
{
    int p = _scan0 + (y * _stride) + (x * _bytesPerPixel);
    _imageData[p] = color.Blu;
    _imageData[p+1] = color.Grn;
    _imageData[p+2] = color.Red;
}

In both cases I first calc the offset of pixel in the byte buffer using the scan0 and stride values. Stride is specified in VideoFormat but we have to guess the scan0 value (it specifies the offset of the first line):

if (_stride < 0)
    _scan0 = -_stride * (_height - 1);

Would be helpful the get it in the VideoFormat too. One last comment regarding VideoFormat is I think it’s constructor should be made public.

Before initializing the camera app displays a list of available devices. I tried to bind the collections from my ViewModel however it turns out that calls to GetAvailableVideoCaptureDevices and GetDefaultVideoCaptureDevice return different instances of CaptureDevice so you can’t do something like this:

CaptureDevices = CaptureDeviceConfiguration.GetAvailableVideoCaptureDevices(); 
SelectedCaptureDevice = CaptureDeviceConfiguration.GetDefaultVideoCaptureDevice();

Also I noticed that none of the devices returned by GetAvailableVideoCaptureDevices has IsDefaultDevice flag set, so I had to use FriendlyName to match it.

Because CaptureSource is used in similar way to MediaElement I think it should expose similar API. In particular it would be helpful to get events when it state changes. We should also get some notification when camera is connected or disconnected.

In order to show the markers recognized by Touchless I paint these areas on an overlay. In current version I tried to implement this using a custom MediaStreamSource with double buffering as demonstrated by Pete Brown. It works but you can see the overlay lags significantly in respect to camera video. So I’m going to switch back to WriteableBitmap instead, but I’m not sure how it will perform either. In particular I’m concerned that WriteableBitmap doesn’t have a way to indicate that we are going to begin updating the buffer. In the WPF version you first need to issue Lock to get access to the BackBuffer. I was bit surprised that it’s not required in Silverlight version and I’m concerned how efficient it is without it. 

Altogether the camera API looks very promising and definitively opens many possibilities. However I think the most common scenario people will be trying to implement will be some sort of online IM client, and for this we would also need some generic video codecs for streaming. Right now Silverlight doesn't even have codecs for Jpeg/PNG images so I think it should be added in the first place. In the meantime you can try to use these as an alternative: http://imagetools.codeplex.com/ Updated: Please see this great post by Rene Shulte on saving webcam snapshots to JPEG using FJCore library.

Last thing, unrelated to camera API, is regarding support for commands that was also added in Silverlight 4. I found that the command CanExecute method won’t be required automatically on user input as it happens in WPF so you need to manually send CanExecuteChanged event for each command.


Friday, November 27, 2009 #

I’ve came across this interesting thread on Arduino forum about using IKEA DIODER and other RGB LED strips to build mood/ambient light. This allows you to create the same effect that you can now find on some TVs. I thought it would be a fun little project to do, and here is a short clip to show you the result:

In case you would like to build it yourself here is how I did it.

The Hardware

To get multicolor LEDs some people are hacking IKEA DIODERs but it’s quite expensive, so I followed the advice found here and ordered two RGB Multicolored 1-Meter LED strips from DealExtreme. Those are flexible plastics strips with 30 multicolor LEDs on each (you can also get it as a bar). It has 4 wires with +12V common anode and separate cathodes for RGB channels.

To control the lights I adopted the circuit shown in Markus post. I put it together on Adafruit’s Proto Shield for Arduino (which is nice because it had the IC pattern for the chip). 

P1140742

Instead of multiple transistors this circuit uses the ULN2003 chip, so that each PWM pin from Arduino is connected to one side of the chip and the RGB channels from LEDs are connected to the other side. Because LEDs require 12V voltage there is a separate power connector (Arduino is still powered from USB). Since I had only a 16V power supply I’m also using a 12V voltage regulator. Next to it I have a push switch to turn the LEDs on/off quickly. The 10K potentiometer on the other side is used to select color hue for one of the manual modes. One push button is wired to Arduino reset pin, and the other is used to switch modes.

I know it could be made prettier but I yet have to learn how to design and produce a custom PCB (please contact me if you’d like to help). If you don’t want to solder this yourself here are some other alternatives that you might consider:

For initial testing I used the Arduino sketch written by Markus. It has three modes: specific color selected by a potentiometer, pulsate the specified color, and smooth transition between random colors. You can find more details in his post.

The Software

Of course it would be hard to get ambient light working for every video source you feed to your TV, but its lot easier to do it when signal is going through a PC. This summer I build a new HTPC based on a Zotac ION-ITX board (cause its super small and quiet). Recently I ditched Windows 7 Media Center in favor of open source XBMC, mostly to get hardware accelerated VDAUP playback. Here are the steps to run it on a minimal Ubuntu install (but I admit I took me three days to figure out how to do it properly).

Apart from the video player we also need additional piece of software that will constantly sample the image visible on screen and produce average color for each LED strip. Thankfully there are existing programs that can do this for us. One of them is called Boblight and turns out to work quite well both in Windows and Linux (but again you get a little speed boost on the later).

In essence Boblight runs in backgrounds, accepts color input from clients and converts them to commands that it sends to the devices. I have to say I was pleasantly surprised to see how flexible it is, and how easy it was to configure it for my setup. You can find more details on the configuration on this page, and my config file is included in the download below.

The final piece is the code running on the Arduino that will translate these commands to corresponding light levels for each channel. Boblight uses a simple serial protocol called LTBL (Let There Be Light) that is described here. In the download you will find my sketch that handles this communication.

Currently I have Boblight setup to automatically start before XBMC loads so it will turn on the lights on startup (some advice can be found here). This setup serves me well, but as I said Boblight runs Windows too. And you can also try other software like Momolight, AtmoLightVLC Media Player, and other projects found at Ambilight4PC.


Wednesday, October 28, 2009 #

Now that .NET Framework 4.0 Beta 2 is out let’s look again what is available for building multi-touch application in WPF. In Beta 1 we got only a preview of manipulation and inertia components. With Beta 2 we finally get access to whole touch input system, and it looks very close to what was shown on PDC last year. Here is an overview from MSDN:

Elements in WPF now accept touch input. The UIElement, and UIElement3D, and ContentElement classes expose events that occur when a user touches an element on a touch-enabled screen. The following events are defined on UIElement and respond to touch input. Note that these events are also defined on UIElement3D and ContentElement.

In addition to the touch events, the UIElement supports manipulation. A manipulation is interpreted to scale, rotate, or translate the UIElement. For example, a photo viewing application might allow users to move, zoom, resize, and rotate a photo by touching the computer screen over the photo. The following events have been added to UIElement:

To enable manipulation, set the IsManipulationEnabled to true.

This is exactly what we were waiting for. I started playing with this new API and I will try to share some samples soon. But here is one problem I run into already. When I tried to use touch in ordinary WPF application, I noticed that all Buttons have a weird behavior, so that when I touched the Button it didn’t go into a pressed state immediately (I’m testing this on HP TouchSmart). Instead it only fires the Click event when I lift the finger. Testing it more I noticed that on any UIElement it won’t fire the MouseDown event until I lift the finger or slide it quite a bit.

To help me diagnose this issue I created a simple test application shown here:

multitouchtest

From left to right I have gray rectangles that react to input events from mouse, stylus and touch. Each rectangle will change color to orange when mouse or stylus is over, and to green when is pressed. In addition, on press I capture the appropriate device and during capture the border changes color to red.

This test confirms that when I use touch panel then the xxxDown events don’t arrive immediately regardless of the device used. I spent quite a while trying to figure it out and trying various settings in both the Touch and Pen Control Panel and the NextWindow’s USB Config utility.

Finally I noticed that somehow this works perfectly fine when I use manipulations. From this I quickly found that this problem goes away when you set IsManipulationEnabled property on the element. Turns out that the side effect of setting this property to true is also changing the stylus properties to disable PressAndHold and Flicks gestures on the element. This explains this problem because stylus engine has to postpone these events trying to interpret the gestures. You can see the difference by selecting the appropriate checkbox on the window.

However although now we get the xxxDown events immediately (which will be very useful for manipulations) this doesn’t fix the problem with Button pressed state. You will notice that MouseDown event is still deferred regardless of the IsManipulationEnabled settings. I believe that in this case this might be caused by input logic in Windows itself. In fact the only way I could affect this was to force the touch panel to report input as mouse events (using NextWindow’s USB Config tool).

In the end I believe the proper fix for this would require extending all WPF built-in controls so that they understand the touch events and react accordingly. At the same time some controls might get some multi-touch specific behaviors as well. For example it was already announced that ScrollViewer will be enhanced to support multi-touch panning. In case of Button it was mentioned several times, that when simultaneous touches occur the correct behavior should be to fire the Click event only after the last touch is lifted. I hope that we get some of these enhancements in the RTM timeframe.

You can download the sample code here.


Monday, July 06, 2009 #

Last Friday I’ve received new Futaba S3001 servos. Turns out they are more easier to modify to Hitec’s, and at the same seem more reliable. I’ve simply followed this Instructable, and you can also find some additional photos in my gallery. What most important the servos are also easier to calibrate, so now my SERB will actually stop in place when I want it.

Below is a short video of the completed SERB robot where it just runs in random directions:

This weekend I’ve also started looking at Microsoft Robotics Development Studio 2008. This environment promises easy entry level to robotics. However this is true only when you have a compatible robotics platform (like Lego Mindstorms NXT, fishertechnik, iRobot and few others). There are pretty good tutorials on how to control these robots both from C# and from VPL. However before I can apply them to my SERB robot, first I need to implement the hardware interface and generic services to control the robot. Not much information on this and closest what I found so for is the Robotics Tutorial 6 – Remotely Connected Robots. It would be cool to get more detailed example on writing such services, but fortunately source code of services for other robots is included. In particular I’m trying to understand the code for iRobot and this additional service for VEX Robotics. If you know of any other good examples or tutorials please let me know.


Thursday, July 02, 2009 #

Few weeks ego we went to an exhibition in one of Kraków’s malls to celebrate the 100 year anniversary of robotics. Besides some nice exhibits there was also a place where kids could try to build a robot from Lego Mindstorms NXT and Fishertechnik sets. We spent few hours there with my older son Jaś and even managed to build a walking robot.

It was fun for both of us so I started looking if we could continue playing with robots at home. Unfortunately most of the sets I found are pretty expensive (starting at $200). This is where my good pal Marcin Książek turned me to the Arduino electronic prototyping platform.

Arduino is really a simple and cheap microcontroller kit that can be easily connected to a PC and programmed. What makes it fun is that there are also many extensions that come as “shields” and adapters for the board and add capabilities such as XBee and Ethernet communication, accelerometer, DC motor control, GPS or even a touch LCD. The platform is open source so there are tons of information and fun projects on the web. Good place to start would be the Arduino Experimentation Kit (ARDX) – you can buy the kit from oomlout, but it’s also open source so you can download all guides and buy the parts in your local electronics store yourself. I also enjoy reading the book “Getting started with Arduino” by Massimo Banzi. And make sure to send the Arduino gift guide to your relatives. There are even Arduino user groups such as Howduino in Liverpool.

[In Poland you can buy Arduino from Nettigo]

All of this makes Arduino a great basis for building a simple robot. And in fact there is already a project on Instructables that shows exactly How to build a Arduino Controlled Servo Robot. Again this project is open source so you can buy a kit from oomlout or download the blueprints and find all the parts yourself. The body of the robot is done from acrylic sheets and I found a local shop with a laser cutter that could cut the parts for me. Most of the other parts can be easily found in your local DIY store.

One exception are the servo motors. Normal servos for RC models are constrained to movement in 180 degrees range, but what we need here is a “Continuous Rotation” servo. You can get one that was pre-modified such as Parallax, GWS S35 or Hitec HSR-1425CR. Another option is to modify the servo yourself, and there are also many instructions on how to do it on the web. I found that it’s really not hard to do, but you have to work a bit on the calibration.

So having all the parts I could finally begin the assembly. Below you can see is a photo gallery where we build our first SERB robot. As you can see I’m one happy geek dad, because both kids were pretty interested with this.

Right now the robot can’t do much besides running around in random directions, but I have some plans to make it smarter.

  1. Add remote control via Xbox pad or WiiMote. There is already a project showing how to use a netbook to control the robot via Skype. But being a .NET geek I would like to use Microsoft Robotics Studio for this.
  2. Add wireless connection with XBee so that robot can roam freely without cables the but still being controlled from the PC.
  3. Add some sensors so the robot gets more information from the world. I think first I would add bumpers to detect obstacles as shown in this project. Later I might also add wheel encoders (such as Nubotics WheelWatcher) for odometry.
  4. Play with some AI algorithms.

So it looks like I already have found myself a summer project :-)