One of the advantages of my favorite IRC client, Klient, is the flexible programmability: it provides object oriented scripting in 6 different languages and sports an API for DLL plugins.
To act as a plugin, a DLL need only export one function:
int __stdcall KlientScriptStartup(HWND AppHandle, int Id, char *ScriptName)
The Id belongs to this plugin, and is used when registering callbacks for various events. ScriptName must be filled in before returning, and is the name shown in Klient's plugin dialog.
Two other functions can be optionally exported for load and unload notifications:
void __stdcall KlientInitialize(void)
void __stdcall KlientFinalize(void)
A plugin talks to Klient by calling a function in KlientEntry.dll:
int __stdcall KlientEntryPoint(int Identifier, void *Data)
In this case, Identifier is a command constant, and Data is a pointer to a command-specific struct containing additional information. Identifiers are provided to register callbacks, send data, run user commands, and query information.
Callbacks are registered for two basic things: aliases, which are custom commands for users; and events, which are raised by Klient for things ranging from new IRC channel messages to the manipulation of Klient's various windows. Each callback has one argument, a pointer to a struct containing alias or event-specific data:
int __stdcall callback(void *data)
Fairly straightforward stuff.
The Microsoft Script Control is used to manage scripts, and Klient provides 72 objects containing 1379 properties/methods. Most of them are either wrappers around the same API exposed to DLL plugins, or around basic Win32 functionality. An integrated script editor makes creating scripts easy:
There's also interest in the community for .NET plugins. Besides bringing along several more languages, the Framework library contains a lot of really useful tools. A common complaint is Klient's lack of support for script-based GUIs, and WinForms is one way to partially address that. Many people are more comfortable with WinForms than they are with the Win32 API ;)
Since Klient is an unmanaged application and doesn't use COM for plugins, there's no direct way to interoperate with .NET components. However, the CLR has support for custom hosts, so it should be possible to create a Klient plugin that loads and "multiplexes" .NET plugins.
I created a v1.1 proof-of-concept to explore this option. The design was fairly simple. An unmanaged DLL is loaded by Klient as a plugin, and uses the CLR hosting APIs to create a new appdomain and load a managed assembly into it. The assembly contains a PluginManager class with an exposed COM interface, which the unmanaged DLL uses to talk to it (startup, shutdown, and display dialog). A dialog allows the user to browse for managed DLLs to load as plugins. When one is chosen, the PluginManager creates a new appdomain, loads its own assembly into it, and instanciates a Plugin class, giving it the plugin assembly's filename. This Plugin class loads the assembly, reflects over it to find a class implementing a specific IPlugin interface, and instanciates it. Finally, a .NET plugin is loaded.
Also in the managed assembly is a set of public classes for .NET plugins to use. These classes simply P/Invoke to KlientEntryPoint, marshalling data as needed. Callbacks for events and aliases map to delegates on the managed side, exposed to .NET plugins as events. A managed wrapper on the delegates is responsible for catching and displaying exceptions not handled by the plugin itself.
Two major issues became apparent with this approach. First, it's imperative that alias and event callbacks be unregistered with Klient before the managed delegates are destroyed; otherwise the unmanaged thunk goes away and Klient is apt to crash when attempting to raise an event. The AppDomain object provides an Unload event, with one caveat: it's raised on a worker thread! Klient's DLL plugin interface is not threadsafe, and events cannot be unregistered on threads other than the main one.
The second issue is exception handling. The CLR propagates exceptions back to the caller: in the case of delegates, this is using the OS's natural SEH mechanism. A delegate wrapper on the managed side is sufficient for normal exceptions thrown inside the .NET plugin, but not for asynchronous exceptions thrown by the CLR itself. A prominent one is ThreadAbortException, which is thrown on all unmanaged threads that attempt to enter an appdomain in the process of being unloaded, or when another thread calls Thread.Abort(). This particular exception has the property of being "undeniable": it can be caught, but the CLR will immediately rethrow it afterward. Klient's event loop is not prepared to deal with such exceptions.
Now v2.0 of the CLR is out, and it brings some changes, including a much expanded hosting API. SQL Server 2005 is also a customer, hosting the CLR in its own process for stored procedures. One of the important things about SQL Server is that it must not crash. This means all the tools needed for reliably running managed plugins in Klient are present, and it's just a matter of figuring out how to use them.
So what exactly needs to be covered?
Reliability. The CLR is a sandbox, so managed plugins should be completely unable to crash Klient. Exceptions unhandled by a plugin need to be caught, logged, displayed, and the plugin unloaded -- without propagating the exception back to Klient. This includes the dreaded asynchronous exceptions: OutOfMemoryException, ThreadAbortException, and StackOverflowException. Klient is a single-threaded application, so if additional managed threads are created, they must not be allowed to call into Klient.
Security. As a sandbox, there's a unique opportunity here for integration of the CLR and Klient. In all existing IRC clients I'm aware of, scripts are able to do anything they wish to; they run in the same security context as the client itself. For .NET, Code Access Security can be used to keep plugins from doing unwanted things on the computer. But I can do better than that: create custom permissions for various Klient operations. Not only can you (dis)trust a plugin to not delete arbitrary files, you can also be sure it can't create an alias to intercept outgoing messages, or send silent ones without your knowledge. Usability pipe dream? Perhaps, but one worth exploring.
Performance. Klient's plugin API is pretty chatty: it takes individual function calls to retrieve a channel name, current modes, a nickname, its associated address, etc. Worse, Klient's not a Unicode application. Not only is there the basic overhead of unmanaged<->managed transitions, but also ANSI<->Unicode translations on every call involving strings. I'll need to minimize overhead, reeduce the number of transitions, and implement caching on the managed side to avoid repetitive callouts.
Usability. There's a wide range of things to do here. Klient's plugin API is procedural, so one immediate need is a managed OO wrapper for it. It should be patterned after the existing script model to maintain consistency and familiarity. Many of Klient's events have special filters, and rather than create methods just to add delegates with those filters, custom attributes can decorate the methods used to handle them. Reflection could also be used to find such methods and hook them up to the appropriate events at load time. Documentation generation and enhanced support for tools (debuggers, IDEs) also needs to be investigated.
And possibly some other things I'm forgetting at the moment. This project looks more daunting written down than it did in my head!
In future blog entries, I'll discuss options and lay out solutions as I go, with whatever discoveries I make along the way.