The recently ratified C++11 language specification provides a range of cool new features. Many of which have been part of other programming languages for some time now. One such new feature that I value a lot is the concept of lambdas.
Lambdas are great in many ways. They enable you to create callbacks that are called for specific items, for instance, or you can implement events with them, as is the case in this blog post. The following is an example of an event implemented in C++11 using lambdas.
RREvent1Arg<int> e;
e += [](int i) { printf( "%d", i ); };
e( 100 );
This looks exactly like C# style events and makes for a very neat way of creating notifications that the user can request to be triggered at pre-specified points. The method introduced here was not possible in previous versions of C++ but with the introduction of lambdas we now have the power to define these kinds of constructs in a developer friendly manner.
The lambda is actually a nameless function body wrapped with a nameless type. This type can also store captured stack variable values if you so choose. This is possible by specifying the captured fields between the [ ] brackets. You also have the option to specify default capture behavior. If you wish to capture values you use by reference you should just write the letter ‘&’ between the brackets and if you wish to capture by value, you should write the letter ‘=’. You can also prefix the name of a single field with the letter ‘&’ to capture it by reference. You can use all the fields you have captured, either explicitly or implicitly, within the function body. The following shows a few examples.
int stackValue = 200;
RREvent1Arg<int> e;
e += [=](int i) { printf( "%d\n", i + stackValue ); };
e += [&](int i) { printf( "%d\n", i + stackValue ); stackValue += 100; };
e += [&stackValue](int i) { printf( "%d\n", i + stackValue ); };
e += [stackValue](int i) { printf( "%d\n", i + stackValue ); };
e( 100 );
// Prints:
// 300
// 300
// 400
// 300
Because lambdas are defined as nameless types, you can’t directly define the type of a variable that will contain a lambda. This is one major reason why C++11 introduces the “auto” keyword. When using the auto keyword, as is done in the following example, the compiler actually deduces the type of the variable without you having to explicitly specify it.
auto l = [](int i) { printf( "%d\n", i ); };
What about templates then? We’d like to define a template called RREvent1Arg that does just what we’ve seen at the beginning of this post. We’d like to be able to create instances of references to the lambdas that we’re adding to the event. This is where std::function comes in. This template class allows for defining a function prototype that the lambda function must comply with and then pointing to the given function with the given prototype. We’re also storing an array of these constructs thus allowing the user to register multiple event handlers.
Cutting a long story short, the magic behind the event implementation above goes as follows.
template<typename T1>
class RREvent1Arg
{
public:
typedef std::function<void (T1)> Func;
public:
void Call( T1 arg )
{
for( auto i = m_handlers.begin(); i != m_handlers.end(); i++ )
{
(*i)( arg );
}
}
void operator ()(T1 arg)
{
Call( arg );
}
RREvent1Arg& operator += ( Func f )
{
m_handlers.push_back( f );
return *this;
}
RREvent1Arg& operator -= ( Func f )
{
for( auto i = m_handlers.begin(); i != m_handlers.end(); i++ )
{
if ( (*i).target<void (T1)>() == f.target<void (T1)>() )
{
m_handlers.erase( i );
break;
}
}
return *this;
}
private:
vector<Func> m_handlers;
};
Note that you have to define a new class for each parameter count. This is not pretty, but if you’re working with Visual Studio 2010 it’s a must because VS2010 does not support variadic templates. So you’ll have to wait for VS11 to be able to make the code prettier.
Of course lambdas are not the only way of implementing these kinds of events. We’ve actually been using this kind of event structure since the dawn of time in our game engine at Raccoon Interactive. We’re actually not able to implement events with lambdas at this point in time due to limitations in compilers on some of the major platforms we’re targeting (Android being one example) but instead we simply store pointers to this call methods.
To conclude all this, lambdas open up a range of new ways in which we can be more productive bringing C++ closer to languages like C# in those terms. The updated STL actually uses lambdas a lot allowing you to write much simpler iteration code, for instance. Events are just one example of how C++11 is bringing C++ back to the “mainstream” in not just the embedded world but for desktop applications as well.
C++ is a very powerful language. Well written native C++ code can perform much better than managed languages like C# and Java due to optimizations that the managed systems are not able to perform during run-time compilation (if this is done at all, that is). This is great for developers who work on gaming technology for instance. For people concerned with game logic, performance isn’t necessarily priority number one, but productivity and the ability to express oneself without too much head banging.
When implementing game logic most of the code expresses certain operations that need to be performed. Usually this means initiating an operation and waiting. The actual execution of the operation usually involves starting and updating animations, allocating and loading resources as well as doing rendering. These operations should be implemented in optimized code to get the most out of the platform but the control logic can be implemented in a language that facilitates higher productivity.
Because of this, game engine core functionality is usually implemented in C/C++ and a scripting layer is built to utilize those platforms facilities. Some major game engines implement their own scripting languages with Unreal Engine being the most notable example. These days there are a few options worth considering before starting down that path, however. A few examples include LUA, AngelScript and C# on Mono.
Mono is the system that I’m going to discuss in this post. There are many benefits to using Mono. First of all Mono provides basically all of the benefits of .NET/Java code. The most notable of these being productivity. Hardly anyone can argue that writing C# code with a good IDE like Visual Studio 2010 is not productive and developers can benefit from the same tools that business software developers get to use.
The second major benefit over many other scripting languages is performance. Yes, I did talk about game logic not having to be as optimized as platform operations but it doesn’t hurt if it is. Mono actually implements a JIT compilation process similar to the Microsoft .NET framework. At run-time it compiles the IL code generated by the Visual Studio compiler and emits machine code executable by the processor. This makes it very fast but not necessarily quite as fast as C/C++, mind you.
As a side note, for the doubtful ones, the performance of C++ stems from a couple of major aspects unique to native programming. First of all, you have full control over the memory you’re accessing. Because modern processors rely heavily on preloading blocks of memory into multiple on-chip caches, knowing what memory you’re accessing becomes very important. Also, modern processor architectures exhibit NUMA (non-uniform memory architecture), which means having your memory close to where it’s being used (processor/core wise) helps you get an additional boost. There are other factors as well but this article is not about C++ performance.
Going back to Mono, multi-platform support is also one of the major benefits of using Mono. Mono is written in such a way that it’s possible to compile it for most popular platforms and since it’s open source and if some platform is not supported, you can always switch into do-it-yourself gear. Perhaps the platforms of most interest to my readers are Windows, Linux, Android, XBox 360, PlayStation 3, Nintendo Wii and iPhone. If your favorite platform is not listed it doesn’t mean it’s not supported 
I’d like to explain to you, the reader, how to get started with embedding Mono to gain productivity for logic and still retain the effectiveness of native programming.
Compiling and initializing
Personally the most interesting scenario for using Mono is embedding it in a C/C++. This allows for building an interface on top of a C/C++ application to enable higher productivity while moving optimized code to the native host. In this blog post I will be using Mono on Windows.
To embed Mono in your C/C++ application you must first generate an import library using the module definition file found here. By the way, I see they have finally added mono_domain_create_appdomain, that we’ll be using later on, to the definition 
The exact command for generating the import library is as follows:
lib /nologo /def:mono.def /out:mono.lib /machine:x86
The command generates a mono.lib file that your application can link against. Once the import library is generated you will have to call mono_jit_init to initialize the Mono run-time and mono_jit_cleanup to cleanup. The function takes as its only parameter an assembly that will be loaded into the created application domain. It’s also possible to call mono_jit_init_version, which allows for initializing a specific version of the Mono run-time.
My personal recommendation is not to use this application domain for basically anything. This is because dumping the main application domain, i.e. ripping down the run-time, and then re-initializing it crashed the application. I’m not sure if this is fixed yet, however. A better option is to create a dummy assembly that will be loaded into this dummy application domain. This domain will only function as a default application domain that enables Mono to function correctly.
You can create this new application domain by calling mono_domain_create_appdomain (before calling this function ensure that the main dummy application domain is activated by calling mono_domain_set). This will create a new isolated virtual process execution environment with it’s own heap and stack. This means that application domains can be dropped at any time thus releasing assemblies loaded into the application domain for modifications.
As I already mentioned an application domain defines an isolated virtual process execution environment that has it’s own memory space. This means that you can load assemblies (managed DLL files) into an application domain and run code in it, letting it allocate objects as it wishes, and then drop the application domain, thus freeing all the allocated memory and the references to the loaded assemblies it owns.
Creating a new application domain for running your script code, for instance, is important, because of the fact that you can unload scripting components without having to restart the application. You might ask why not just unload the assemblies that are not required anymore? The answer is short: because you can’t. The .NET framework, and thus Mono also, forbids unloading assemblies. This is due to the complicated interdependency of objects and their types. This might result in missing dependencies in the middle of application execution, which is hardly a good thing.
This brings us to what we do at Raccoon Interactive. We use an application domain to host the game script environment that is created to wrap an existing game environment inside our editor. If we make any changes to the built script assemblies we simply stop the game execution within the editor at which point the editor restores the state of the game to what it was before jumping into the game and dumps the application domain. After this we can simply jump into the game again and see the new modified scripts in action.
Loading and running code
To actually run your code in the application domain you have just defined you must first load the assembly into the domain. To load the assembly into the domain do the following.
MonoAssembly *pAssembly = mono_domain_assembly_open( pDomain,
“MyAssembly.dll” );
if ( pAssembly == NULL )
return NULL;
MonoImage *pImage = mono_assembly_get_image( pAssembly );
The mono_assembly_get_image function gets the assembly image required by many Mono operation, like reflection (getting type information).
Assemblies can be reflected for the type information they contain. You can also create objects once you know their types and call arbitrary methods to get your game code running. To get this type information you can call a couple of functions, my favorite being mono_class_from_name. This function takes the image of the assembly, the namespace of the class and the name of the actual class as parameters. The following is an example of getting a type.
MonoClass *pClass = mono_class_from_name( pImage,
"MyNamespace", "MyClass" );
Great! Now we have loaded an assembly and we have a pointer to the class that we’re interested in. Mono also allows creating objects using the type information we have just requested and invoking methods on that object (or invoking static methods, for that matter).
To create an object and invoke a method on it, do the following.
// Get the constructor of the class.
MonoMethod *pConstructorMethod = mono_class_get_method_from_name_flags( pClass,
".ctor", -1, METHOD_ATTRIBUTE_SPECIAL_NAME );
// Create a new instance of the class.
MonoObject *pObject = mono_object_new( pDomain, pClass );
// Acquire a GC handle if the object will not be rooted in the CLR universe.
guint32 GCHandle = mono_gchandle_new( pObject, false );
// Invoke the constructor.
MonoObject *pException = NULL;
mono_runtime_invoke( pConstructorMethod, pObject, NULL, &pException );
The most important thing to note here is that the object is actually not initialized straight away as we create a new instance of the class. We first have to get the constructor with the quite peculiar name and manually invoke it. Here are are using a default constructor that does not take any arguments.
Also, note that we are acquiring a GC handle to the object. If you wish to use the object from native code and not have it rooted anywhere in the CLR universe (the Common Language Run-time universe) you must acquire a GC handle so that the garbage collector will not steal the object from you.
Once you are finished running your code you should call mono_domain_unload to unload the application domain that houses your code. You can also call this function to release a domain without actually quitting the application.
Next, I’ll cover manipulating fields of objects and going deeper into invoking methods and parameter passing.
Manipulating fields and invoking methods
Often, when working with script objects, you are interested in directly modifying an object in the script universe. You might, for instance have a native pointer in the object that defines a native resource to which the object is bound and you wish to initialize that directly. Mono provides a set of functions for doing just this.
When using Mono reflection, you must always get information about the certain type of primitive. For instance, to invoke methods, you must first get method information, as I showed in the previous section. To manipulate fields you must do the following.
MonoClassField *pField = mono_class_get_field_from_name( pClass,
"_myField" );
You pass the method the class information and the name of the field. Simple, eh? Setting field values is a bit more complicated and requires some information about argument passing in the Mono run-time. Documentation on conventions is pretty much non-existent (as is all but simple initialization).
To se the value of a field of the type float you must do the following.
gfloat value = 1.0f;
gfloat *pValue[1];
pValue[0] = &value;
MonoException *pEx = NULL;
mono_field_set_value( pField, pObject, (void **)pValue, &pEx );
Not that pretty, is it? mono_field_set_value takes as parameters the field that you are setting, the object that contains the field, a pointer to a pointer to data and a pointer to a pointer to an exception. The idea behind the third parameter is that it collects the argument data from a pointer array that points to the data. You should always use type names prefixed with the character g (their part of GLIB). This will assure that the internal representation correctly maps to your data.
To set a field to reference a Mono object, do the following.
MonoObject *pObject = ...
gpointer pValue[1];
pValue[0] = pObject;
mono_field_set_value( pField, pObject, (void **)pValue, &pEx );
As before, we are passing an array of pointers where we now point to a single Mono object. To set properties, just use the mono_property_set_value function on a MonoProperty instance returned by mono_class_get_property_from_name, for instance.
Conclusions
In this article we covered first of all why using a managed layer on top of your native code is sometimes quite useful. On the technical side we covered initializing Mono, creating objects, manipulating them and calling methods on them. There are a lot of specifics that I have not covered and will get into them in future posts. These include boxing and registering internal call methods that enable managed code to call back into native code.
As always, if there are errors in this article or something does not work, please don’t hesitate to drop a comment and I’ll try to answer your questions to the best of my abilities!
Until next time!
Our engine is written in almost pure C++ with some part in C# that run on Mono. The code base is huge which mans that we’re primarily looking for platforms that support C++ development. Looking at the current set of platforms on the market this is really not a big issue with Windows Phone being the only platform that does currently not support native development. Sadly this means that we’re currently not planning a Windows Phone release in the near future.
Looking at other platforms, about a year ago Android was not really an option for us at Raccoon Interactive. It seemed like a ragtag platform with only Java support. Not only was there almost no good support for C++ development but the Java run-time was/is very slow to say the least. The Android NDK now provides support for native development and builds on the SDK which contains all the basic tools like the emulator and the Android Debug Bridge (ADB). That and the fact the Android is quickly gaining momentum meant that we’d just found our first platform.
We’re not quite there yet with the Android version of our engine but I thought I’d give you some insight on the intricacies of actually developing for that platform while maintaining our productivity (read: continue using Visual Studio). I’ll most likely be posting more about Android development as the port progresses, but today I’m going to focus on debugging.
To start with, we use vs-android instead of the NDK build pipeline, which means that there’s no need for Cygwin and we’re able to build our engine from within Visual Studio in the same way we build our Windows version. vs-android consists of a set of MSBuild targets that execute the compiler and linker commands provided by the NDK. The targets also include steps for packaging and deploying the native application.
Another method for building Android native applications from within Visual Studio is WinGDB that has one huge advantage over vs-android: you can actually debug the application remotely within Visual Studio! At this point you might be asking “Why use vs-android then?” There’s actually a simple answer to this. WinGDB requires you to create a new project instead of just adding a build setup for the already existing project. I was also not able to link my library, that is loaded by the native activity, with another library. This was crucial for us because we have multiple libraries that form the whole engine and it’s out of the question to pack everything into a single library.
All this means that when we wish to build and test our engine on Android, this is pretty much what we do:
- Develop normally within Visual Studio.
- Build the project for the Android build target.
- Deploy the APK onto the phone (this can be done automatically by setting the vs-android Ant build to handle the deployment).
- Launch the application on the phone/emulator.
- Manually set up the GDB server on the phone/emulator and the GDB client on the Windows host to communicate with each other.
- Debug the application.
There are some pit falls when setting up a debugging session, however. First of all, you can’t use the ndk-gdb script that comes with the Android NDK because we don’t have a makefile project to start with, which the script uses to set up the debug session. Also because we don’t use the typical ndk-build script for building the makefile project (which we don’t have) we don’t get the gdbserver application deployed onto with the APK by default.
All this means hat setting up the debugging session becomes somewhat tricky. First of all you have to deploy the gdbserver application with the APK. The way we solved this was copying it to the libs folder that also contains the libraries built by vs-android. The libs folder is not cleared during building so the application stays there when we put it there. This way gdbserver will be found under the folder /data/data/com.ri.MyAndroidApplication/lib with the correct user access rights associated with it. The correct gdbserver executable can be found under NDK_PATH/toolchains/arm-linux-androideabi-4.4.3/prebuilt.
The second part is setting up TCP forwarding from the phone to your PC. This happens by running the following command (note that I expect <ANDROID_SDK>\platform-tools to be in your PATH environment variable).
adb forward tcp:1234 localfilesystem:/data/data/com.ri.MyAndroidApplication/debug-pipe
|
The command redirects TCP connections to localhost:1234 on the PC to the named pipe on the Android system. We then configure the GDB server to listen to debug-pipe on the emulator/phone. This happens by executing the following command on the emulator/phone. First you have to determine the application process ID (pid). This can be done by executing “adb shell ps” and looking up the correct process (com.ri.MyAndroidApplication in our case).
adb shell run-as com.ri.MyAndroidApplication
/data/data/com.ri.MyAndroidApplication/lib/gdbserver +debug-pipe
--attach <process-pid>
|
The run-as command executes the application as the user that also runs the application com.ri.MyAndroidApplication. This is kinda neat since Android runs all applications under different user accounts to protect your phone from malicious users. The run-as command will look up the user that would run the application and then run the application with he correct user rights.
So, now we have the server running. But like with all servers in the world, you also need a client for any of this to be meaningful. The Android NDK comes with a GDB client that you’re supposed to use. This client can be found under “<NDK_PATH>/toolchains/arm-linux-androideabi-4.4.3/prebuilt/windows/bin” and is called arm-linux-androideabi-gdb.exe.
We will actually be debugging a process named app_process that resides on the emulator/phone. This executable has to be pulled from the emulator/phone so we can inspect it for symbols. We also need to pull the correct libc.so version so the debugger can be correctly setup. To perform these steps execute the following commands in a new command prompt.
adb pull /system/bin/app_process app_process
adb pull /system/lib/libc.so libc.so
|
Great, let’s get to the actual beef, connecting to the GDB server. Run the GDB client executable inside the command prompt and execute the following commands.
(gdb) set solib-search-path <NDK_PATH>/platforms/android-9/arch-arm/usr/lib
(gdb) directory <NDK_PATH>/platforms/android-9/arch-arm/usr/include;
<NDK_PATH>/sources/cxx-stl/system
(gdb) file app_process
(gdb) target remote :1234
|
At this point you should have successfully connected to the the GDB server and should be able to work with the debugger. I’m not going to go into detail about GDB commands but a few handy ones are b (breakpoing), c (continue), step, next and info threads (listing threads when developing a multithreaded application).
GDB is actually quite handy once you get over the shock of not having a GUI. But hey, real programmers only need their keyboard 
Just recently there was a great talk held by Herb Sutter at the C++ and Beyond conference. You can see the whole presentation at Channel 9.
So why does C++ really matter anymore? All this time we’ve been talking about managed languages (Java, C#, Visual Basic.NET, the list goes on) increasing the productivity of development and being really the holy grail of computing where we don’t have to focus on memory allocation or really knowing anything about the underlying platform, for that matter. All this has recently come to an end when we think about cutting edge applications that will form the basis of the future of computing.
We’re moving more and more into the cloud and building huge data centers, all the while moving our mindset towards mobility. Datacenters have huge compute capacity while mobile devices have to do with very little. Both topics are concerned with power usage, also. Having datacenters use less energy means less money spent on cooling and electricity bills. On the phone this means longer battery life (making the battery of your Android device last over the weekend).
What we have to learn here is that there’s not just a lot of time and money to be saved here, but also focusing on performance means we’re able to build more complicated applications that do even more exciting things (and put them into your pocket too).
Modern processors are built on NUMA architectures, use multiple layers of fast memory caches and run on multiple cores. Not just this, but recent systems are also heavily moving towards heterogeneous computing, which means using thousands of simple cores to perform massively parallelized computations at blazingly high speeds. NUMA and caching, especially, are beasts that will get us into trouble if we’re not careful with where we do things on the chip as well as when we do it.
Managed languages abstract control over memory access as well as the generated code. It’s a step forward that they do JIT compilation but going that extra mile means having the ability to go deeper. Native development is all about that. Allocating memory near to a specific core is easy. Having memory that is used rarely separated from memory that is very hot is also easy. You know where your memory is and you know what the processor is doing and at what point.
Believe it or not, managed languages are not the holy grail. You just can’t optimize resource access when you’re not supposed to care about it. Neither is C++ and native programming a holy grail. It’s often not as productive (this has partially been because of language syntax and libraries, but anyway).
The point: Keep the balance. Do things where you wish to be productive in .NET/Java but when you need performance and/or low power consumption, C++ with it’s wide adoption and object oriented programming style is your choice. Also keep an eye on what’s coming in the new C++ standard, because productivity is just going to get a huge boost.