When it comes to displaying the progress of a long-running operation to the user, there are many options available to a GUI designer:

  • Change the cursor to an hourglass
  • Change the caption on the status bar
  • Show an animated glyph on the window
  • Show a dialog box

A number of factors could affect which method you choose:

  • How long does the operation typically run for? A few seconds? Minutes?
  • Is the operation synchronous or asynchronous? If the operation is asynchronous, are there any restrictions on what the user can do while the operation is running?
  • Can the user cancel the operation?
  • Is the length of the operation known, or can it be easily estimated? Is it unpredictable?
  • Does the operation provide meaningful feedback to the user? (e.g. status messages)

ProgressDialog example

Shell Progress Dialogs provide a mechanism for displaying the progress of a long-running operation. They are suited to non-trivial operations that exceed 10 seconds, showing status messages and a progress bar. They facilitate both synchronous and asynchronous operations (though the latter are always preferable). They provide a mechanism for aborting the operation and for estimating the time remaining to complete the operation (computed using the progress percentage and time measurements taken as the operation progresses). If progress is not measurable (and the user is running Windows Vista or above), a marquee can be displayed instead of a normal progress bar.

This type of dialog box is part of the Windows shell, and is used by the operating system for a variety of purposes, from copying files to running the Windows Experience Index tests. Microsoft makes the dialog available to third party code via COM; exposing the IProgressDialog interface and a concrete implementation of the type (CLSID_ProgressDialog). This makes it easy to use progress dialogs in C/C++ applications, but what about managed code and .NET?

Instantiating COM Types in .NET

As with COM interfaces in general, the .NET Framework has a number of features to enable interoperability. In general, to instantiate a COM type in .NET, you need to:

  • Define the interface implemented by the type (in this case, IProgressDialog), based on its specification. Add any required metadata for marshalling method parameters and return types, as well as any enumerations required. Above all, remember to add the [ComImport] and [Guid] attributes to the interface definition.
  • Obtain a managed Type object for the COM type (not the interface), using the Type.GetTypeFromCLSID() method.
  • Use Activator.CreateInstance() on the Type to instantiate it. Cast the resulting object to the interface type, and you’re ready to start calling methods.
  • Don’t forget to call Marshal.FinalReleaseComObject() on the instance when you’re finished with it. A convenient way to ensure this is to place the cleanup code in a finally block, or implement IDisposable if the object is stored at instance scope in a class.

For IProgressDialog, the CLSID of the interface is {EBBC7C04-315E-11d2-B62F-006097DF5BD4} and the CLSID of the concrete class is {F8383852-FCD3-11d1-A6B9-006097DF5BD4}.

Using IProgressDialog

The MSDN Documentation for the Windows Shell API details the lifecycle of the IProgressDialog object and how to use it. In simple terms:

  • Set up the progress dialog before it is displayed to the user by calling SetTitle() and SetCancelMsg().
  • Show the dialog using StartProgressDialog(). A series of PROGDLG flags control the behaviour and appearance of the dialog box.
  • Show status messages using the SetLine() method. You can use up to 3 lines of text, 2 if you allow the estimated time remaining to be automatically calculated.
  • As you perform work, call SetProgress() to update the progress bar. Each call lets you specify the current value as well as the maximum value for the bar. The user can click the cancel button at any time, therefore you should also check its status using the HasUserCancelled() method.
  • When the operation has finished, call StopProgressDialog(). Note that, once you close the dialog box, you cannot show it again; you must create a new instance for each operation.
Type progressDialogType = Type.GetTypeFromCLSID(new Guid("{F8383852-FCD3-11d1-A6B9-006097DF5BD4}"));
IProgressDialog progressDialog = (IProgressDialog)Activator.CreateInstance(progressDialogType);

try {
    // set up dialog and display to the user
    progressDialog.SetTitle("Progress dialog");
    progressDialog.SetCancelMsg("Aborting...", null);
    progressDialog.StartProgressDialog(Form.ActiveForm.Handle, null, PROGDLG.AutoTime, IntPtr.Zero);

    // do work
    progressDialog.SetLine(1, "Working...", false, IntPtr.Zero);
    progressDialog.SetLine(2, "Please wait while the operation completes", false, IntPtr.Zero);
    for (uint i = 0; i < 100; i++) {
        if (progressDialog.HasUserCancelled()) {
            break;
        }
        else {
            progressDialog.SetProgress(i, 100);
            Application.DoEvents();
            Thread.Sleep(150);
        }
    }

    // close dialog
    progressDialog.StopProgressDialog();
}
finally {
    Marshal.FinalReleaseComObject(progressDialog);
    progressDialog = null;
}

A Managed Wrapper for IProgressDialog

Working directly with COM types is not recommended; due in part to the need to manually release resources, as well as having to work with the archaic programming model, marshalled types, structures, etc. COM types are also limited in comparison to .NET types, in that they do not (natively) support events or properties. For these reasons, I decided to create a managed wrapper for IProgressDialog.

As with other common dialog types already available in .NET (e.g. OpenFileDialog), my implementation inherits from the Component class, allowing it to be dropped onto the design surface of a Form or other component in Visual Studio. Component also has the advantage of implementing the IDisposable pattern, removing the need to write some boilerplate code.

My goals for the wrapper were to:

  • Replace method calls with properties (where possible).
  • Provide get accessors to complete the functionality of each property.
  • Replace the flag options with separate properties (to improve ease of use and intuitiveness).
  • Provide sensible default values for properties.
  • Simplify the operations by removing parameters and relying more on the object state.
  • Provide cleanup code in a Dispose() method.
  • Remove the need to re-instantiate the component for each operation.
  • Further simplify the use of the component by automatically closing the dialog when the progress reaches 100%.

With my wrapper class, ProgressDialog, you can configure the dialog at design time and thus write less code:

// set up dialog and display to the user
wrapper.Show();

// do work
wrapper.Line1 = "Working...";
wrapper.Line2 = "Please wait while the operation completes";
for (uint i = 0; i < 100; i++) {
    if (wrapper.HasUserCancelled) {
        break;
    }
    else {
        wrapper.Value = i;
        Application.DoEvents();
        Thread.Sleep(150);
    }
}

// close dialog
wrapper.Close();

Final Words

As I said in my introduction, there are many different ways to show progress in a GUI application. The real advantage of Shell Progress Dialogs is that they are rendered by the operating system (thus their appearance is ‘upgraded’ automatically on new versions of Windows, and they always fit the OS theme), and present users with a familiar and consistent interface. They’re not suitable for all applications, but I hope you find my implementation useful if you think they suit the needs of your app.

Download

ProgressDialogs.zip (9KB, includes example)

In many applications, the ability to dynamically load and execute third-party code (in the form of plug-ins) is highly desirable. Plug-ins can be used to provide:

  • Alternative implementations of built-in features
  • Completely new features (given some kind of framework on which they can be built)
  • All functionality in the application (e.g. in a test harness, compiler or other modular system)

There are a number of mechanisms within the .NET Framework to facilitate plug-in code:

Reflection enables assemblies to be dynamically loaded (removing compile-time references), and to iterate through the types in an assembly. To build a plug-in system based entirely on reflection, however, would be limiting, unreliable and very slow. The overheads involved in calling methods via reflection are high. Also, in the absence of a compile-time reference to an object, you lack the ability to verify whether the object contains the method/property you’re trying to access.

Interfaces are one of the key primitives in object orientated programming. They allow you to define the public methods, events and properties of a type without specifying how it should be implemented; or indeed, how it should behave. You can toss around a reference to an object using only its interface type, and still be able to do everything with the object that you could do with a concrete class (save for instantiating it, of course).

An obvious way to implement a plug-in system, therefore, would be to define a set of interfaces that were common to both the application and its plug-ins, implement them in the plug-ins and load them into the application using reflection. The advantage of this approach is that you have a contract at compile-time against which you can guarantee that the methods/properties you’re accessing exist on the object you’ve loaded.

There is quite a significant downside to this approach, however: You are loading third-party, potentially malicious code directly into your application’s memory space. The plug-in code could use reflection to access and manipulate everything in your application, not to mention crash it. It’s not hard to see why this would be a bad idea.

Application domains are an important (though perhaps not widely-understood) part of the .NET Framework. The vast majority of applications will only ever use a single AppDomain but, when utilised, they can be very powerful. Application domains sit at a high level in the runtime, providing a memory space into which the code you reference and execute is loaded. The only way to access managed objects from outside their AppDomain is to serialise them (a process over which you can exercise a lot of control) or to marshal them via remoting. An AppDomain can be secured to prevent another AppDomain from seeing inside it, loading assemblies and reflecting types. It can also raise and handle its own exceptions, keeping it isolated from the main process. This is definitely a solid foundation for a plug-in system.

It makes a lot of sense to load any plug-in code into a separate AppDomain, then ferry objects between the two domains in a controlled, sandboxed fashion. A general rule to consider when writing plug-in code is:

  • Use binary serialisation (the Serializable attribute or the ISerializable interface) when passing data between two application domains. Only ever pass an instance of a type that is common to both domains; e.g. a standard .NET type or a type defined in an assembly that is referenced by both domains. Any operations performed on serialisable types will run in the AppDomain that calls them (i.e. the main application).
  • Use remoting (handled transparently by MarshalByRefObject) when calling methods or raising events. Operations performed on remotable types will run in the AppDomain in which they are instantiated (i.e. the plug-in domain).

Why Not WCF?

At this point, you might well ask, “Why not use WCF as the basis for a plug-in system?”. It’s true, some developers do advocate this practice, but I personally do not. WCF imposes a number of show-stopping limitations on plug-in code:

  • WCF handles events poorly, requiring callback interfaces to be manually defined and wired up. Remoting handles events transparently.
  • Operations on WCF services can only exchange serialisable data. You can’t, for example, pass a remotable object to a WCF service to enable two-way communication.
  • WCF is primarily designed to be stateless. Plug-in code is almost always stateful. Although WCF handles sessions and concurrency, these can be difficult to use.
  • WCF uses XML serialisation based on public members of a type. Remoting uses binary serialisation and has the necessary permissions to access private members of a type.
  • WCF is optimised for interprocess and network-based communication, not communication between two application domains within the same process.

And, of course, it’s worthwhile to note that WCF itself is built on top of .NET Remoting; it is under no threat of deprecation, as it is a fundamental part of the framework.

More About MarshalByRefObject and Remoting

MarshalByRefObject is essential to any non-trivial cross-AppDomain functionality. It is handled specially by the .NET Framework; all you have to do is inherit from MarshalByRefObject and the framework will generate a transparent proxy for your class, automatically marshalling calls between the application domains for you.

Some important things to remember about writing classes that extend MarshalByRefObject:

  • Any objects you pass-to or return-from a MarshalByRefObject must be serialisable, or themselves a MarshalByRefObject.
  • You must remember to mark any types you derive from Exception or EventArgs with the [Serializable] attribute.
  • IEnumerable sequences created using the yield statement or LINQ cannot cross an application domain (because the compiler does not mark them as serialisable). Don’t return sequences from a MarshalByRefObject; instead, copy the elements into a collection (or use the ToList() method) and return the collection.
  • If you pass a delegate to a MarshalByRefObject – and the method it points to is in a different AppDomain – you must ensure that the method belongs to a MarshalByRefObject as well. Do not pass delegates to static methods, because they will be called on the wrong AppDomain (since there is no object instance to marshal the call to).

There are some other common types that can’t cross application domain boundaries:

  • DataObject (used for drag-and-drop, clipboard and other OLE functionality) is neither MarshalByRefObject or serialisable. You can either extract the data and pass it directly to the other AppDomain, or create a wrapper that implements IDataObject and inherits from MarshalByRefObject.
  • Image/Bitmap, although marked as serialisable, may not cross AppDomain boundaries. You should pass a Stream or byte[] containing the image data instead.

Lifetime Services and ISponsor

To further complicate matters, remoting uses lifetime services (rather than ordinary generational garbage collection) to determine when instances of a MarshalByRefObject should be cleaned up. By default, you have a window of 5 minutes in which to use an object obtained from a foreign AppDomain before the proxy becomes disconnected and an exception is thrown upon access. This is a necessary evil, because the garbage collector can’t count references to a remotable object inside a different application domain; that would break the isolation provided by application domains in the first place.

There are two methods to get around this, however:

  • Override the InitializeLifetimeService() method and return a null reference. This instructs remoting not to clean up instances of your object in another AppDomain. This has the potential to create memory leaks, so you can really only use this technique for singleton classes.
  • Obtain the lifetime service object (ILease) from the MarshalByRefObject using the RemotingServices class and register an ISponsor object to keep the instance alive.

Sponsorship works by renewing the lease on a MarshalByRefObject; it does this by returning a TimeSpan indicating how much longer the object is needed. Remoting will periodically call the Renewal() method on an ISponsor object until it returns a timespan of zero, or the sponsor is unregistered.

// register a sponsor
object lifetimeService = RemotingServices.GetLifetimeService(myMarshalByRefObject);
if (lifetimeService is ILease) {
    ILease lease = (ILease)lifetimeService;
    lease.Register(mySponsor);
}

// unregister a sponsor
object lifetimeService = RemotingServices.GetLifetimeService(myMarshalByRefObject);
if (lifetimeService is ILease) {
    ILease lease = (ILease)lifetimeService;
    lease.Unregister(mySponsor);
}

In practice, what this means is that you should hold a reference to a sponsor for any MarshalByRefObject you obtain from another AppDomain for as long as you need to access the object. When the sponsor object becomes eligible for garbage collection, it will also take out the remotable object which it sponsors. Ideally, implementations of ISponsor should be serialisable.

In my implementation of a plug-in system, I created a convenient generic class, Sponsor<TInterface>, which is simultaneously responsible for registering/unregistering a sponsor, accessing the remotable object itself and providing the renewal logic. You hold a reference to the sponsor object in your class, then call its Dispose() method when the remotable object is no longer needed. My plug-in system centers around the Sponsor class; ensuring that objects from the plug-in AppDomain are always wrapped in a Sponsor instance and never returned directly to user code without one.

Design for a Plug-In System

As I have alluded to, a plug-in system based on reflection, interfaces, remoting and sponsors is built around two application domains. The main AppDomain uses the PluginHost class to create the plug-in AppDomain and remotely instantiate PluginLoader, the class that loads plug-ins and instantiates remotable objects:

// create another AppDomain for loading the plug-ins
AppDomainSetup setup = new AppDomainSetup();
setup.ApplicationBase = Path.GetDirectoryName(typeof(PluginHost).Assembly.Location);

// plug-ins are isolated on the file system as well as the AppDomain
setup.PrivateBinPath = @"%PATH_TO_BINARIES%\Plugins";

setup.DisallowApplicationBaseProbing = false;
setup.DisallowBindingRedirects = false;

AppDomain domain = AppDomain.CreateDomain("Plugin AppDomain", null, setup);

// instantiate PluginLoader in the other AppDomain
PluginLoader loader = (PluginLoader)domain.CreateInstanceAndUnwrap(
    typeof(PluginLoader).Assembly.FullName,
    typeof(PluginLoader).FullName
);

// since Sandbox was loaded from another AppDomain, we must sponsor
// it for as long as we need it
Sponsor<PluginLoader> sponsor = new Sponsor<PluginLoader>(loader);

PluginLoader dynamically loads the plug-in assemblies (located in a subdirectory) into the plug-in AppDomain:

foreach (string dllFile in Directory.GetFiles(pluginPath, "*.dll")) {
    Assembly asm = Assembly.LoadFile(dllFile);
    Assemblies.Add(asm);
}

PluginLoader keeps a cache of ConstructorInfo objects for each interface implementation it discovers, so it can quickly instantiate objects. It exposes GetImplementations (returns IEnumerable<TInterface>) and GetImplementation (returns the first implementation of TInterface).

private IEnumerable<ConstructorInfo> GetConstructors<TInterface>() {
    if (ConstructorCache.ContainsKey(typeof(TInterface))) {
        return ConstructorCache[typeof(TInterface)];
    }
    else {
        LinkedList<ConstructorInfo> constructors = new LinkedList<ConstructorInfo>();

        foreach (Assembly asm in Assemblies) {
            foreach (Type type in asm.GetTypes()) {
                if (type.IsClass && !type.IsAbstract) {
                    if (type.GetInterfaces().Contains(typeof(TInterface))) {
                        ConstructorInfo constructor = type.GetConstructor(Type.EmptyTypes);
                        constructors.AddLast(constructor);
                    }
                }
            }
        }

        ConstructorCache[typeof(TInterface)] = constructors;
        return constructors;
    }
}

private TInterface CreateInstance<TInterface>(ConstructorInfo constructor) {
    return (TInterface)constructor.Invoke(null);
}

public IEnumerable<TInterface> GetImplementations<TInterface>() {
    LinkedList<TInterface> instances = new LinkedList<TInterface>();

    foreach (ConstructorInfo constructor in GetConstructors<TInterface>()) {
        instances.AddLast(CreateInstance<TInterface>(constructor));
    }

    return instances;
}

PluginHost calls the GetImplementation/GetImplementations methods on PluginLoader to return transparent proxies to the remotable objects instantiated from the plug-ins. It wraps them in a Sponsor instance and returns them to the user. PluginHost also handles reloading/unloading of the AppDomain.

Putting It All Together

The general usage pattern for my plug-in system would be:

  1. Create a series of interfaces and place them in a common assembly.
  2. Create one or more plug-in assemblies containing types that implement the interfaces.
  3. Create an application which references only the common assembly.
  4. Instantiate PluginHost, passing the path to load plug-ins from.
  5. Call the LoadPlugins() method and check for success.
  6. Instantiate implementations of the plug-in interfaces using the GetImplementations() or GetImplementation() methods.
  7. Keep a reference to the Sponsor<TInterface> object returned from the above methods until the object is no longer required.
  8. Unload the plug-in AppDomain by calling Dispose() on the PluginHost object.

You can see an example of this in the included example project.

Final Words

Nobody can deny that loading third-party code in a separate application domain is regarded as best practice. Hopefully, this task is greatly simplified through the use of the plug-in system i’ve provided. It is, of course, simply a proof of concept implementation. Other things you might want to consider would be:

  • Applying security to the plug-in AppDomain to further sandbox the environment.
  • Filtering the plug-in assemblies loaded; either according to digital signatures, implementation of marker interfaces or particular metadata.
  • Making metadata about the plug-ins available to calling code.
  • Handling exceptions more gracefully.

In any event, I hope it demonstrates the basic idea behind cross-AppDomain programming in .NET.

Download

PluginSystem.zip (Visual Studio 2010 solution, zipped)

David Anderson recently responded to a question I asked on stackoverflow regarding how to access binary resources in a .resx file as a Stream rather than a byte[] (which ResourceManager exposes by default). It goes into detail about how ResourceManager always copies the data into memory, and how to use manifest resource streams to overcome this problem:

Getting a stream to a resource without copying the contents into memory on David Anderson’s blog

Background

I’m currently doing some work in the area of cross-AppDomain development, where all objects are either marshalled (transparently) using .NET Remoting or serialised. It turns out that, when DataSet objects are serialised, their extended properties are serialised as strings. This means that, when operating on a DataSet which has come from another AppDomain, the complex data type you added as an extended property has unexpectedly become a string representation of it.

There’s a quick and easy solution to this problem; manually serialising extended properties using the built-in .NET XML serialisation mechanism. However, not all .NET types are fully supported by the XML serialiser, as I discovered while trying to serialise an object with a property of type System.Drawing.Color. Here’s a quick and easy way to work around this issue:

Solution

The solution is two-fold; firstly, we should exclude the Color property from XML serialisation (using the XmlIgnore attribute), because we know it’s not supported:

[XmlIgnore]
public Color TextColor {
    get;
    set;
}

Next, we need to provide an additional public read/write property that is XML-serialisable:

public string TextColorHtml {
    get { return ColorTranslator.ToHtml(TextColor); }
    set { TextColor = ColorTranslator.FromHtml(value); }
}

The property needs to be public and provide a getter and setter, because the XML serialiser will invoke it, and does not execute with high enough privileges to be able to read into private/protected/internal properties or set the values of backing variables. I’ve chosen to serialise the HTML representation of the colour, because it is already XML-friendly and translates back and forth easily.

The advantage to using this approach is that you do not have to change the representation used by your class; the Color structure is very useful and very pervasive in .NET. Also, since the additional property has no backing variable (it is entirely procedural), no additional memory is needed to store each instance of the type.

From time to time, it’s a good idea to go back and re-examine old code. With time comes greater knowledge and new tricks; and in the case of two of my earlier custom Windows Forms controls, this was certainly true. You may remember my ComboBox control with grouping support, as well as my ComboBox with a TreeView drop-down – I have revisited these controls, with particular mind to implementing what I learnt from my experience with the Buffered Paint API.

New versions of GroupedComboBox and ComboTreeBoxGroupedComboBox

To refresh your memory, this control extends ComboBox and adds a new property; GroupMember. Using PropertyDescriptor and owner-drawing, it groups together items whose grouping values are equal.

What’s new:

  • On Windows Vista and 7, the control did not appear in the ‘button style’ consistent with normal ComboBox controls in DropDownList mode; this is a limitation of owner-drawing, but one that can be overcome through the use of VisualStyleRenderer. This is a slightly dodgy hack in that it pre-supposes the appearance of the ComboBox in this mode, but the functionality is only applied when the OS is identified as Vista or 7.
  • The control is inconsistent with other ComboBox controls because it is not animated. With the BufferedPainter<T> class I developed, implementing buffered animation was simple.

ComboTreeBox

The ComboTreeBox is a control developed entirely from scratch (i.e. its base class is Control) that uses a hierarchical/tree structure as its data representation (instead of a flat list). As such, it has behaviours in common with both ComboBox and TreeView. The drop-down/pop-up portion of the control is implemented using ToolStripDropDown, bitmap caching and custom rendering.

What’s new:

  • As with GroupedComboBox, the control did not appear in the Windows Vista/7 style. As the control has no editable portion and is entirely owner-drawn, it was easy to simulate the appearance under the appropriate OS. Under XP, it falls back to the default visual style. When themes are disabled, it draws in the classic style.
  • The control was not animated. Once again, BufferedPainter<T> was used to implement animation.
  • Due to a strongly-typed reference to its designer class, the control previously did not work with the “Client Profile” version of the .NET Framework. Rather cheekily, I have substituted my custom designer for the built-in designer that the DomainUpDown control uses – this works well for constraining the height of the control and avoids the need to reference the System.Design assembly (included only with the full version of the framework).
  • Minor tweaks were made to the way focused items are drawn; this is only apparent if focus cues are enabled in Windows.

Download

Please visit the project page here: Drop-Down Controls