The Gallio Book

The Gallio Documentation Team

The Gallio Book
by The Gallio Documentation Team
3.0.6.0 beta
Copyright © 2009 Gallio Project
Table of Contents
Preface ........................................................................................................................... viii
Why This Book? ..................................................................................................... viii
Who is This Book For? ............................................................................................ viii
I. Introduction .................................................................................................................... 1
1. About Gallio .......................................................................................................... 3
What is Gallio? .................................................................................................. 3
History ............................................................................................................. 3
2. Installation ............................................................................................................. 4
The Gallio Zip Distribution .................................................................................. 4
The Gallio Installer ............................................................................................. 4
Plugins ................................................................................................... 4
Runners .................................................................................................. 5
Tools integration ...................................................................................... 5
Documentation ......................................................................................... 5
Extras .................................................................................................... 5
3. Getting Started ....................................................................................................... 6
II. MbUnit v3 .................................................................................................................. 15
4. About MbUnit ...................................................................................................... 17
What is MbUnit? .............................................................................................. 17
MbUnit v2 vs. MbUnit v3 .................................................................................. 17
Feature Matrix .................................................................................................. 17
Requirements ................................................................................................... 17
5. Unit Testing ......................................................................................................... 18
6. Data Driven Testing .............................................................................................. 19
7. Web Testing ......................................................................................................... 20
8. Database Testing ................................................................................................... 21
9. Recipes ............................................................................................................... 22
10. Contract Verifiers ................................................................................................ 23
What are contract verifiers? ................................................................................ 23
The Exception Contract Verifier .......................................................................... 23
The Equality Contract Verifier ............................................................................ 25
The Comparison Contract Verifier ....................................................................... 27
The Immutability Contract Verifier ...................................................................... 29
The Collection Contract Verifier .......................................................................... 29
The List Contract Verifier .................................................................................. 29
11. Extending MbUnit ............................................................................................... 30
How MbUnit Works .......................................................................................... 30
How to create an MbUnit attribute ....................................................................... 30
How to create an MbUnit assertion ...................................................................... 30
How to create an MbUnit data source ................................................................... 30
How to create a custom type converter and formatter ............................................... 30
12. Troubleshooting .................................................................................................. 31
13. Migration Guide .................................................................................................. 32
III. Test Framework Adapters ............................................................................................. 33
14. csUnit ................................................................................................................ 34
15. MbUnit v2 ......................................................................................................... 35
16. MSTest .............................................................................................................. 36
17. NUnit ................................................................................................................ 37
18. xUnit.Net ........................................................................................................... 38
IV. Test Runners .............................................................................................................. 39
19. Icarus GUI ......................................................................................................... 40

iv
 The Gallio Book

20. Sail, a Lightweight Visual Studio Test Runner .......................................................... 41

21. TestDriven.Net .................................................................................................... 42
22. Visual Studio Team System ................................................................................... 43
23. ReSharper .......................................................................................................... 44
24. Echo Command Line ........................................................................................... 45
25. Powershell .......................................................................................................... 46
26. MSBuild ............................................................................................................ 47
27. NAnt ................................................................................................................. 48
V. Test Tools ................................................................................................................... 49
28. Ambience ........................................................................................................... 51
29. AutoCAD Integration ........................................................................................... 52
Introduction ..................................................................................................... 52
Using the AutoCAD Test Runner ........................................................................ 52
How Gallio Finds AutoCAD ....................................................................... 52
30. NCover .............................................................................................................. 53
31. Pex ................................................................................................................... 54
32. TypeMock .......................................................................................................... 55
33. WatiN ............................................................................................................... 56
34. CruiseControl.Net ................................................................................................ 57
35. TeamCity ........................................................................................................... 58
VI. Developer Guide ......................................................................................................... 59
36. Getting the Code ................................................................................................. 60
Prerequisites ..................................................................................................... 60
Required Software .................................................................................... 60
Optional Software ..................................................................................... 60
Checking out the Source Code ............................................................................ 62
Working with the Gallio and MbUnit v3 Source Code ............................................. 62
How to Run a Build .................................................................................. 62
Understanding the Source Tree .................................................................... 63
Working with the Solution ......................................................................... 63
Running Tests in Visual Studio ................................................................... 65
Working with the Gallio Book ............................................................................ 65
How to Run a Build .................................................................................. 65
Working with DocBook ............................................................................. 65
Working with the Gallio Web Site Projects ........................................................... 66
How to Run a Build .................................................................................. 66
37. Understanding the Gallio Architecture ..................................................................... 67
Gallio Runtime ................................................................................................. 67
Services and Components ........................................................................... 67
Plugins .................................................................................................... 67
The Runtime Lifecycle .............................................................................. 67
38. Creating Plugins .................................................................................................. 69
39. Creating Test Frameworks ..................................................................................... 70
Concepts ......................................................................................................... 70
The EasyTest SampleFramework ......................................................................... 70
EasyTest syntax ................................................................................................ 70
Low-Level Implementation ................................................................................. 70
High-Level Implementation ................................................................................ 70
Summary ......................................................................................................... 71

v
List of Tables
36.1. Source Tree Layout ................................................................................................... 63

vi
List of Examples
37.1. Using the Runtime .................................................................................................... 68

vii
Preface
Why This Book?
One of the biggest problems in many open source projects is the lack of proper documentation. This is a
shame because there is not point in implementing a thousand features if no one knows about them. The
Gallio development team was very aware of this issue, so documentation was given a high priority from
the beginning of the project. The API reference documentation, which is automatically generated from
the source code, was the first sign of this concern, but the team knew users need more: they need a guide
to show them how to use the multiple features, the runners and so on under different scenarios. So the
idea of writing a book was there from the beginning, in our wishlist, but we knew it was a big task so
it was posponed.

Around February in 2008, the topic arised again in a conversation between Jeff Brown, Gallio's lead and
main developer, and I. The project was approaching a new alpha release and a beta version was not too
far away either, so there was a lot to write about. I decided I'd go ahead and write the book myself, even
though I felt (and still feel) I was not the best person to do it, hoping that the rest of the team and the
community would help me to fill the gaps in my English and my time.

I'm glad to see we are publishing the first draft of the book. I sincerely hope it'll help you to leverage the
full power of this exciting platform and all the tools that are been built around it. Happy testing!

Who is This Book For?

• Developers.

• Testers.

• People wanting to extend Gallio.

viii
Part I. Introduction
Table of Contents
1. About Gallio .................................................................................................................. 3
What is Gallio? .......................................................................................................... 3
History ..................................................................................................................... 3
2. Installation ..................................................................................................................... 4
The Gallio Zip Distribution .......................................................................................... 4
The Gallio Installer ..................................................................................................... 4
Plugins ........................................................................................................... 4
Runners .......................................................................................................... 5
Tools integration .............................................................................................. 5
Documentation ................................................................................................. 5
Extras ............................................................................................................ 5
3. Getting Started ............................................................................................................... 6

2
Chapter 1. About Gallio
What is Gallio?
The Gallio Automation Platform is an open, extensible, and neutral system for .NET that provides a
common object model, runtime services and tools (such as test runners) that may be leveraged by any
number of test frameworks.

History
In January 2004 Marc Clifton, a frequent contributor at Codeproject, wrote a series of articles that sought
to expand the unit testing discussion. Among other things, Marc proposed a formalization of various test
patterns beyond basic TDD. Marc then took his ideas into code as AUT (Advanced Unit Testing), an
independed project that you can find at Codeproject.

Two months later, Jonathan "Peli" de Halleux took a look at Marc's proposals and created gUnit (which
was later renamed to MbUnit) while recovering from surgery in a hospital. In fact, Peli wrote most of
MbUnit while still in the hospital.

MbUnit had some new ideas and concepts and it caught the attention of Jamie Cansdale who while on a
trip to Brussels hooked up with Peli to work on an add-on for TD.net. TD.net started life as a NUnit project
and so this made MbUnit the next framework after NUnit to be supported by TD.net, as such since the
very early days of this great tool there has been MbUnit support.

In 2005 Peli made MbUnit opensource and continued working on the framework while finishing his PhD.
Shortly after completing his PhD he accepted a job with Microsoft as a SDE\T on the CLR team. Unable to
carry on MbUnit, he handed it over to Jamie Cansdale as short time caretaker. Peli blogged about needing
someone to take on MbUnit and shortly after Andy Stopford as a long time MbUnit user read this and
stepped up.

Since then MbUnit has grown as a framework and project, with two major releases and triple the downloads
per release it has firmly rooted itself in main stream Microsoft .net culture as a viable unit test framework
next to NUnit.

In the autumn of 2007, MbUnit v3 - a ground up rewrite of MbUnit, started. In one of those funny turn
of events, v3 was to be code named "Gallileo" but due to a typo became "Gallio". The name stuck and
development continued on MbUnit v3, code name: Gallio.

With MbUnit V3 developmement well under way, long time MbUnit core member Jeff Brown attended
the Alt.Net conference in Austin, Texas. Following discussions with other programmers at the conference,
Jeff made the case to the MbUnit team that there was value to the community at large in isolating the test
runner capabilities of the system to create a neutral platform upon which MbUnit could then be hosted as
one of many supported frameworks. Other open-source and commercial projects would be able to leverage
the platform's services to create rich, interoperable and extensible testing solutions, thereby adding great
value to the community.

After much discussion, the decision was made to separate the test runner from MbUnit and Gallio the
Automation Platform was born.

Going forward the Gallio Project seeks to become visible to other open source projects so that the
capabilities of the platform can bring unity and value to the many projects in the testing space.

3
Chapter 2. Installation
Gallio comes in two flavors: an installer and a zip file. The installer is probably better for individual
development. If you work on a team you might prefer the zip distribution so you can put the files in a
convenient, source-controlled location.

In any case you can download the latest release from http://www.gallio.org/.

The Gallio Zip Distribution

The Gallio Installer

Plugins
Plugins add support for running tests created with frameworks other than MbUnit v3. The currently
available ones are:

• MbUnit v2: Allows you to run MbUnit v2 tests with Gallio

• MSTest: Allows you to run MSTest tests with Gallio (only if you have Visual Studio 2008 / Visual
Studio Team System installed)

• NUnit: Allows you to run NUnit tests with Gallio

• xUnit.NET: Allows you to run xUnit.NET tests with Gallio

Unless you need to run tests created with these frameworks, you don't need to install these plugins.

4
 Installation

Runners
Runners are programs or plugins for other programs that provide different ways to run Gallio. The chosen
runner has no effect on the availability of plugins, that is, you can for example run NUnit tests with any
of the runners provided you have installed the right plugin.

• Echo: The console test runner

• Icarus: The graphical test runner

• MSBuild task: Allows you to run Gallio from an MSBuild build script

• NAnt task: Allows you to run Gallio from a NAnt build script

• PowerShell commands: Allows you to run Gallio from a PowerShell script

• ReSharper v3 plug-in: Allows you to run Gallio from ReSharper

• TestDriven.NET runner for MbUnit3: Allows you to run MbUnit v3 tests with Gallio from the
TestDriven.NET Visual Studio add-in

• TestDriven.NET runner for Other Supported: Allows you to run tests from the supported frameworks
(MbUnit v2, NUnit, xUnit.NET and MSTest) with Gallio from the TestDriven.NET Visual Studio add-
in

You probably don't need the MSBuild and NAnt tasks unless you are implementing a build server.

Tools integration
• AutoCAD plug-in: Allows tests that depend on the AutoCAD API (ObjectARX) to run inside the
AutoCAD process.

• NCover integration: Allows you to enable code coverage with NCover by simply setting a property

• TypeMock integration: Allows you to enable TypeMock.NET by simply setting a property

Documentation
• Standalone Help Docs: Install the CHM documentation

• Visual Studio 2005 Help Docs: Install the integrated documentation for Visual Studio 2005

Extras
• CruiseControl .NET extensions: Provides an extension to allow downloading attachment from the
CCNet build report. It's only useful if you are implementing a build server.

The installer will create a folder in the Start Menu, where you will see shortcuts for the Icarus GUI runner,
the MbUnit website, the offline documentation, the online documentation and the unistaller.

5
Chapter 3. Getting Started
In this chapter you will see step by step how to create and run a simple test project using Gallio and its
default test framework, MbUnit v3. It's assumed that you already know how to create projects and add
references in Visual Studio, but no prior knowledge of unit testing or test frameworks is assumed. The
screenshots were taken in Visual Studio 2008, but the steps are the same for Visual Studio 2005 as well.

Let's start by creating a new class library project in Visual Studio called SimpleLibrary. Delete the default
class (normally Class1) that's added by Visual Studio and add a new one called Fibonacci. It will be a
pretty simple public class with only one method called Calculate, as shown here:

namespace SimpleLibrary
{
public class Fibonacci
{
public static int Calculate(int x)
{
if (x <= 0)
return 0;
return Calculate(x - 1) + Calculate(x - 2);
}
}
}

Public Class Fibonacci

Public Shared Function Calculate(ByVal x As Integer) As Integer

If x <= 0 Then
Return 0
Else
Return Calculate(x - 1) + Calculate(x - 2)
End If
End Function

End Class

We will write some tests to verify that this method is working properly. Many people like to write the
tests first, in what is called "Test driven development", usually abreviated "TDD". If we were working in
a TDD way we would have created a test, made it fail and only then we would have implemented the code
required to make it pass (in this case the Calculate method's body), just to start over in what is known as
the "red, green, refactor" cycle. Since the chosen development methodology doesn't affect the way we use
the framework, we won't follow anyone in particular.

Tests should never be put in your production code, but in separate projects/assemblies. Add a new class
library project to the SimpleLibrary solution called SimpleLibrary.Test, delete the default class and add
a new one called FibonacciTest. In the Gallio source code the convention is to name a test project after
the project it's testing plus the suffix '.Tests', and to name a unit test class after the class it's testing plus
the suffix '.Test'. This is a popular convention, but as you may guess everyone has its own preferences.

Now, still in the test project, you need to add a reference to the SimpleLibrary project, and also to the
Gallio.dll and MbUnit.dll assemblies that you can find in the bin folder of your Gallio install folder. In

6
 Getting Started

case you don't know the path, look for a shortcut to it in the Gallio program group of the Start menu.
Note that the common practice is to have this assembly as well as other referenced assemblies in a custom
folder in your solution, mainly for location independency and versioning issues, but we will cover this
scenario in other chapters.

At this point, your solution should look like this:

With the project structure and references in place we can now write the first unit test. The code is the
following:

using System;
using SimpleLibrary;
using MbUnit.Framework;

namespace SimpleLibrary.Test
{
public class FibonacciTest
{
[Test]
public void FibonacciOfNumberGreaterThanOne()
{
Assert.AreEqual(Fibonacci.Calculate(6), 8);
}
}
}

Imports SimpleLibrary
Imports MbUnit.Framework

Public Class FibonacciTest

<Test()> _
Public Sub FibonacciOfNumberGreaterThanOne()
Assert.AreEqual(Fibonacci.Calculate(6), 8)
End Sub

End Class

Notice the things we did in the previous code:

• We imported the SimpleLibrary namespace

7
 Getting Started

• We imported the MbUnit.Framework namespace

• We made the FibonacciTest class public (you may need to explicitly do so in your language of choice)

• We added a new public method called FibonacciOfNumberGreaterThanOne, which doesn't return a

value

• We decorated the FibonacciOfNumberGreaterThanOne method with the [Test] attribute

• We added a call to the Assert.Equal method

You may be wondering what's the call to the Assert.AreEqual method supposed to do. Not too much in
fact: it only checks that the return value of the call to Fibonacci.Calculate(6) is equal to 8. The Assert class
is very important because it contains a lot of helpful methods that make writing tests easier.

Now we have our first unit test, but how do we execute it? Gallio comes bundled with many different
runners, that is, programs or plugins for other programs that allows you to execute tests. This time we
will use Icarus, the graphical runner, because it's a standalone application (meaning that you don't need
to install anything else to run it). The Gallio installer creates a shortcut for it in the programs folder of the
Start menu, so it's pretty easy to launch it:

Once Icarus is running, go to Assemblies->Add Assemblies... in the menu:

Browse to the folder where you put the solution and look for the SimpleLibrary.Test.dll assembly (probably
located in bin\Debug under the test project's folder). The test tree is populated as shown in this screenshot:

8
 Getting Started

You can see the MbUnit version under the Root node, and under it the names of the assembly, the fixture
and the test method. The next step is to execute the test, for which you only need to press the Start button.
After doing so we see that it passes:

But what does it mean for a test to "pass"? It means that it was executed, all its assertions were true and no
exception was thrown. This is the basic structure of a test in the so called state-based testing: you create
one on more objects, call a method and assert over the state of it after doing it.

So far so good, but we need to test more scenarios to make sure that the test didn't pass because we were
lucky, but because the tested method is actually working. In other words, we need to test for different
values of x. Since we cannot test every possible value, the common practice is to pick a few representative
cases. In our case, it's clear that we also need to test at least the following three scenarios:

• x less than 0

• x equal to 0

• x equal to 1

We face an interesting case: what should we do if x is a negative number? The Fibonacci function is only
defined for numbers greater or equal than zero, so it makes sense to throw an exception in that case. But
how do we test that behavior? We said a test will only pass if no exception is thrown during its execution.
One option would be to put the call to the Fibonacci.Calculate in a try-catch block, calling the Assert.Fail
or throwing a new exception method in case the one we are waiting for is not thrown, but a better option
is to use the handy ExpectedException attribute that will do just that for us:

[Test]
[ExpectedException(typeof(ArgumentException))]
public void FibonacciOfNegativeNumberDoesNotExist()
{
Fibonacci.Calculate(-1);
}

<Test()> _
<ExpectedException(GetType(ArgumentException))> _
Public Sub FibonacciOfNegativeNumberDoesNotExist()

Fibonacci.Calculate(-1)

9
 Getting Started

End Sub

We run the test and we see it fails (but note that FibonacciOfNumberGreaterThanOne keeps passing):

This is what we would expect, since we are not trowing any exception in the Calculate method. Let's fix
that:

public static int Calculate(int x)

{
if (x < 0)
throw new ArgumentException("x must be greater than or equal to zero");

return (x > 1) ? Calculate(x - 1) + Calculate(x - 2) : x;

}

Public Shared Function Calculate(ByVal x As Integer) As Integer

If x < 0 Then
Throw New ArgumentException("x must be greater than or equal to zero")
End If

If x < 2 Then
Return x
End If

Return Calculate(x - 1) + Calculate(x - 2)

End Function

We execute the tests again and all of them pass this time:

10
 Getting Started

All that's left is to add tests for x equal to 0 and 1. But instead of writing a test for each of them, we will
use a powerful feature called row testing to reduce the amount of coding we need to do. Here's the code:

[Test]
[Row(0, 0)]
[Row(1, 1)]
public void LowerBoundsTest(int x, int Fibonacci)
{
Assert.AreEqual(Fibonacci.Calculate(x), Fibonacci);
}

<Test()> _
<Row(0, 0)> _
<Row(1, 1)> _
Public Sub LowerBoundsTest(ByVal x As Integer, _
ByVal expectedFibonacciNumber As Integer)

Assert.AreEqual(Fibonacci.Calculate(x), expectedFibonacciNumber)

End Sub

There are a few differences between this test and the previous we wrote:

• We applied a Row attribute for each of the scenarios we want to test (two in this case)

• This test method receives 2 parameters, just as the number of items in each Row attribute

So what does this all mean? Gallio will create test instances for each Row attribute, and will pass each
value of it as a parameter to the test method (in the same order by default), converting to the right type if
necessary. We run the tests in Icarus again and voilá!, they pass:

11
 Getting Started

Row testing is one of the simplest cases of data-driven testing. As we will see later, Gallio has powerful
data-binding capabilities, supporting a heterogeneous set of datasources and many ways to manipulate
and scope them.

As the last thing in this chapter, we will see how to run the tests we created with the TestDriven.NET
add-in, which is one of the most popular ways for developers to run tests in Visual Studio.

You can download TestDriven.NET from its website, http://www.testdriven.net/. It's a commercial
product, but at the time of this writing there's a personal version of it.

After you have downloaded and installed TestDrive.NET, open the sample solution again and right-click
on the SimpleLibrary.Test project, as shown in the screenshot:

In a few seconds you see this output:

12
 Getting Started

The real benefit, however, of using the TestDriven.NET add-in comes when you want to run individual
tests:

In this case, the output window shows pretty useful information, like which test instances were executed,
the parameters they were passed and the outcome of each one:

Also there is a link to the generated HTML report. If you Control-click it it will be opened inside Visual
Studio:

13
 Getting Started

This report is always generated in a folder called Gallio.TDNetRunner within your temporary files folder.
Unfortunately the link is not displayed when executing tests at the assembly level, but the report is
generated anyway (this is the way TestDriven.NET works, so there's nothing we can do about it other than
asking its author, Jamie Cansdale, to change it).

You know now pretty much everything you need to get started. But keep reading! There is still much more
to discover in the next chapters.

14
Part II. MbUnit v3
Table of Contents
4. About MbUnit .............................................................................................................. 17
What is MbUnit? ...................................................................................................... 17
MbUnit v2 vs. MbUnit v3 .......................................................................................... 17
Feature Matrix .......................................................................................................... 17
Requirements ........................................................................................................... 17
5. Unit Testing ................................................................................................................. 18
6. Data Driven Testing ...................................................................................................... 19
7. Web Testing ................................................................................................................. 20
8. Database Testing ........................................................................................................... 21
9. Recipes ....................................................................................................................... 22
10. Contract Verifiers ........................................................................................................ 23
What are contract verifiers? ........................................................................................ 23
The Exception Contract Verifier .................................................................................. 23
The Equality Contract Verifier .................................................................................... 25
The Comparison Contract Verifier ............................................................................... 27
The Immutability Contract Verifier .............................................................................. 29
The Collection Contract Verifier .................................................................................. 29
The List Contract Verifier .......................................................................................... 29
11. Extending MbUnit ....................................................................................................... 30
How MbUnit Works .................................................................................................. 30
How to create an MbUnit attribute ............................................................................... 30
How to create an MbUnit assertion .............................................................................. 30
How to create an MbUnit data source ........................................................................... 30
How to create a custom type converter and formatter ...................................................... 30
12. Troubleshooting .......................................................................................................... 31
13. Migration Guide .......................................................................................................... 32

16
Chapter 4. About MbUnit
What is MbUnit?
MbUnit is a unit testing framework in the tradition of xUnit frameworks such as JUnit. In addition, MbUnit
includes a rich suite of features designed to simplify other automation tasks that arise during integration
testing.

MbUnit was originally created in 2005 by Jonathan "Peli" de Halleux. It introduced and popularized novel
ideas such as "RowTests" and "CombinatorialTests".

The present incarnation of MbUnit, MbUnit v3, represents a complete rewrite and redesign of Peli's
original work to improve the end-user experience, consolidate features, enhance extensibility, and enable
advanced integration testing and reporting. From this synthesis, Gallio was born.

For more information about the early history of MbUnit, refer to the introductory chapter about Gallio.

MbUnit v2 vs. MbUnit v3

The subject of this section is MbUnit v3. However, we will occasionally refer to the older MbUnit v2.
This section will explain a few of the differences between these versions.

MbUnit v3 is a .Net 2.0 based framework. It uses generic types and methods where possible to encourage
code reuse. It also provides additional features for .Net 3.5 clients in a separate assembly. MbUnit v3
leverages the Gallio test automation platform heavily to provide integration with numerous other tools and
to enable functionality such as rich reporting.

MbUnit v2 is a .Net 1.1 based framework with a few .Net 2.0 add-ons. It is stand-alone framework that
includes its own suite of test runners. Gallio includes an adapter plugin so that Gallio-based tools may also
be used with MbUnit v2 (when running tests in a .Net 2.0 environment). However, since MbUnit v2 was
not originally designed for Gallio, it does not provide as many advanced features as MbUnit v3.

MbUnit v2 is being maintained concurrently with MbUnit v3 for the benefit of existing projects based on
MbUnit v2 that have not yet migrated to MbUnit v3. For new projects, we recommend adopting MbUnit v3.

MbUnit v3 is mostly backwards compatible with MbUnit v2 except for some APIs that have been renamed
or redesigned. Transitioning to MbUnit v2 is relatively straightforward.

For more information about the differences between MbUnit v2 and v3, please refer to the Migration
Guide chapter.

Feature Matrix
TODO

Requirements
.Net 2.0 runtime (CLR or Mono)

TODO

17
Chapter 5. Unit Testing
TODO

18
Chapter 6. Data Driven Testing
TODO

19
Chapter 7. Web Testing
TODO

20
Chapter 8. Database Testing
TODO

21
Chapter 9. Recipes
TODO

22
Chapter 10. Contract Verifiers
What are contract verifiers?
The contract verifiers are built-in test fixtures for very common contractual objects such as custom
exceptions or types implementing the generic IEquatable interface. A contract verifier is a complete
and configurable test suite. It automatically enables several test methods which evaluate exhaustively the
functionalities and the behaviors of your custom contractual type.

The contract verifiers are declared as public read-only fields of your test fixture. They are marked by the
special attribute VerifyContract as well. An hypothetical declaration of a contract verifier for the Foo
contract would be declared like this:

using System;
using MyLibrary;
using MbUnit.Framework;
using MbUnit.Framework.ContractVerifiers;

namespace MyLibrary.Test
{
[TestFixture]
public class MyFooTest
{
[VerifyContract]
public readonly IContract FooTests = new FooContract
{
Option1 = value1,
Option2 = value2
};
}
}

Remark the different named options set at the declaration of the contract verifier. Those options are used
to configure the contract verifier. They also make it more flexible by letting you enable or disable certain
features at your convenience.

The Exception Contract Verifier

The Exception Contract Verifier adds to a fixture, several tests which evaluate a custom user exception. The
contract verifier is declared as a public read-only field of the test fixture. It has the following specifications:

public class ExceptionContract<TException> : IContract

: where TException : Exception
{
public bool ImplementsOperatorOverloads
{
get;
set;
}

23
 Contract Verifiers

public bool ImplementsStandardConstructors

{
get;
set;
}
}

A simple and typical usage is:

using System;
using MyLibrary;
using MbUnit.Framework;
using MbUnit.Framework.ContractVerifiers;

namespace MyLibrary.Test
{
[TestFixture]
public class MyExceptionTest
{
[VerifyContract]
public readonly IContract ExceptionTests = new ExceptionContract<MyExcept
{
ImplementsSerialization = true; // Optional (default is true)
ImplementsStandardConstructors = true; // Optional (default is true)
};
}
}

The contract verifier takes only one single required type parameter defining the type of the tested
exception; MyException, in the example above. The optional properties ImplementsSerialization and
ImplementsStandardConstructors disable some of the test methods described below.

The verifier adds at runtime, five test methods to the test fixture:

• HasSerializableAttribute verifies that the tested exception type has the attribute System.Serializable.
Setting the property ImplementsSerialization to false in the definition of the contract verifier,
deactivates that test.

• HasSerializationConstructor verifies that the tested exception type has a non-public

constructor similar to System.Exception(SerializationInfo, StreamingContext). Setting the property
ImplementsSerialization to false in the definition of the contract verifier, deactivates that test.

• DefaultConstructor verifies that a public default constructor is present. If the property

ImplementsSerialization was not set to false, it verifies that the message and the inner
exception properties are preserved by a roundtrip serialization as well. Setting the property
ImplementsStandardConstructors to false in the definition of the contract verifier, deactivates that
test.

• MessageConstructor verifies that a public constructor with one single parameter for the message is
present. If the property ImplementsSerialization was not set to false, it verifies that the message and
the inner exception properties are preserved by a roundtrip serialization as well. Setting the property
ImplementsStandardConstructors to false in the definition of the contract verifier, deactivates that
test.

24
 Contract Verifiers

• MessageAndInnerExceptionConstructor verifies that a public constructor with two parameters for

the message and the inner exception is present. If the property ImplementsSerialization was not set
to false, it verifies that the message and the inner exception properties are preserved by a roundtrip
serialization as well. Setting the property ImplementsStandardConstructors to false in the definition
of the contract verifier, deactivates that test.

The Equality Contract Verifier

The Equality Contract Verifier adds to a fixture, several tests which evaluate the correct implementation
of the genertic interface System.IEquatable. The contract verifier is declared as a public read-only field
of the test fixture. It has the following specifications:

public class EqualityContract<TTarget> : IContract

where TTarget : IEquatable<TTarget>
{
public bool ImplementsOperatorOverloads
{
get;
set;
}

public EquivalenceClassCollection<TTarget> EquivalenceClasses

{
get;
set;
}
}

A simple and typical usage is:

using System;
using MyLibrary;
using MbUnit.Framework;
using MbUnit.Framework.ContractVerifiers;

namespace MyLibrary.Test
{
[TestFixture]
public class FooTest
{
[VerifyContract]
public readonly IContract EqualityTests = new EqualityContract<Foo>
{
ImplementsOperatorOverloads = true; // Optional (default is true)
EquivalenceClasses =
{
{ new Foo(1) },
{ new Foo(2) },

25
 Contract Verifiers

{ new Foo(3) }
}
};
}
}

The contract verifier takes one single required type parameter defining the type of the tested equatable
type; Foo, in the example above. The optional property ImplementsOperatorOverloads disables some
of the test methods described below.

The property EquivalenceClasses is mandatory. It feeds the contract verifier with various valid object
instances grouped by equivalence classes. An equivalence class contains a collection of distinct objects
which are expected to be equal together (in the sense of your implementation of the equality for the tested
type.) Instances contained in different equivalence classes are expected to be not equal. The equivalence
classes make possible the contract verifier to test both for equality and inequality between objects. Thus,
it is important to feed appropriately the contract verifier with a set of relevant and common objects based
on real use cases. Null references are not considered as a valid input for an equivalence class. Indeed,
the test engine determines automatically whether the tested type is nullable, and modifies the necessary
tests accordingly.

There are two possible ways to construct and initialize a collection of equivalence classes. You can either
use the list initializer syntax or directly the constructor. The two examples below create a collection which
contains three equivalence classes.

EquivalenceClasses =
{
{ new Foo(1),
new Foo("One"),
new Foo(Digit.One) },
{ new Foo(2),
new Foo("Two"),
new Foo(Digit.Two) },
{ new Foo(3),
new Foo("Three"),
new Foo(Digit.Three) }
}

The constructor does not allow the equivalence classes to contain more than one instance each.

EquivalenceClasses = new EquivalenceClassCollection<Foo>(

new Foo(1),
new Foo(2),
new Foo(3))

The constructor might still be useful in the case you want to feed the collection with an existing enumeration
of distinct objects.

EquivalenceClasses = new EquivalenceClassCollection<Foo>(MyDistinctInstanceProv

The verifier adds at runtime, five test methods to the test fixture:

• ObjectEquals verifies that the method bool Object.Equals(object obj) behaves as expected. Your type
should indeed override that method to provide appropriate equality results, even when it is boxed as
a System.Object.

26
 Contract Verifiers

• HashCode verifies that the method int Object.GetHashCode() behaves as expected. Your type should
override that method and return the same hash value for objects equal together.

• EquatableEquals verifies that the method bool Equals(TTarget other) works correctly. The
implementation of that method is required by the generic equatable interface.

• OperatorEquals verifies the equality operator (==) behaves as expected for the tested type. You must
overload the static equality operator to make it pass. The static equality operator has the following
syntax:

public static bool operator ==(TTarget left, TTarget right)

Setting the optional property ImplementsOperatorOverloads to false in the definition of the contract
verifier, deactivates that test.

• OperatorNotEquals verifies the inequality operator (!=) behaves as expected for the tested type. You
must overload the static inequality operator to make it pass. The static equality operator has the following
syntax:

public static bool operator !=(TTarget left, TTarget right)

Setting the optional property ImplementsOperatorOverloads to false in the definition of the contract
verifier, deactivates that test.

The Comparison Contract Verifier

The Comparison Contract Verifier adds to a fixture, several tests which evaluate the correct
implementation of the genertic interface System.IComparable. The contract verifier is declared as a
public read-only field of the test fixture. It works very similarily to the equality contract verifier. It has
the following specifications:

public class ComparisonContract<TTarget> : IContract

where TTarget : IComparable<TTarget>
{
public bool ImplementsOperatorOverloads
{
get;
set;
}

public EquivalenceClassCollection<TTarget> EquivalenceClasses

{
get;
set;
}
}

27
 Contract Verifiers

A simple and typical usage is:

using System;
using MyLibrary;
using MbUnit.Framework;
using MbUnit.Framework.ContractVerifiers;

namespace MyLibrary.Test
{
[TestFixture]
public class FooTest
{
[VerifyContract]
public readonly IContract ComparisonTests = new ComparisonContract<Foo>
{
ImplementsOperatorOverloads = true; // Optional (default is true)
EquivalenceClasses =
{
{ new Foo(1) },
{ new Foo(2) },
{ new Foo(3) }
}
};
}
}

The contract verifier takes one single required type parameter defining the type of the tested comparable
type; Foo, in the example above. The optional property ImplementsOperatorOverloads disables some
of the test methods described below.

The property EquivalenceClasses is mandatory. It feeds the contract verifier with various valid object
instances grouped by equivalence classes. Unlike for the equality contract verifier, the order in which the
equivalence classes are specified is important for the comparison contract verifier. Equivalence classes are
expected to be declared in the ascendent order. Every object in a given equivalence class is expected to be
greater than the objects contained in a previous class, and less than the objects in the next classes.

The verifier adds at runtime, five test methods to the test fixture:

• ComparableCompareTo verifies that the method int CompareTo(TTarget obj) behaves as expected.
Your type should indeed implement that method to provide appropriate comparison results.

• OperatorGreaterThan verifies that the "Greater Than" operator (>) behaves as expected for the tested
type. You must overload the static operator to make it pass. The operator has the following syntax:

public static int operator >(TTarget left, TTarget right)

Setting the optional property ImplementsOperatorOverloads to false in the definition of the contract
verifier, deactivates that test.

• OperatorGreaterThanOrEqual verifies that the "Greater Than Or Equal" operator (>=) behaves as
expected for the tested type. You must overload the static operator to make it pass. The operator has
the following syntax:

28
 Contract Verifiers

public static int operator >=(TTarget left, TTarget right)

Setting the optional property ImplementsOperatorOverloads to false in the definition of the contract
verifier, deactivates that test.

• OperatorLessThan verifies that the "Less Than" operator (<) behaves as expected for the tested type.
You must overload the static operator to make it pass. The operator has the following syntax:

public static int operator <(TTarget left, TTarget right)

Setting the optional property ImplementsOperatorOverloads to false in the definition of the contract
verifier, deactivates that test.

• OperatorLessThanOrEqual verifies that the "Less Than Or Equal" operator (<=) behaves as expected
for the tested type. You must overload the static operator to make it pass. The operator has the following
syntax:

public static int operator <=(TTarget left, TTarget right)

Setting the optional property ImplementsOperatorOverloads to false in the definition of the contract
verifier, deactivates that test.

The Immutability Contract Verifier

TODO

The Collection Contract Verifier

TODO

The List Contract Verifier

TODO

29
Chapter 11. Extending MbUnit
This chapter explains how to extend MbUnit with new features such as custom metadata, test decorators,
and new data sources,.

How MbUnit Works

Ever wonder how MbUnit attributes work? Well, it's nothing too special. Each attribute is just an
ordinary .Net class with some code that defines how it contributes to the semantics of the test cases. So
we can define new attributes by subclassing the appropriate built-in attributes and providing different
semantics.

Likewise, MbUnit assertions are quite literally just a collection of static methods defined on the Assert
class. However, the assertion methods follow an implementation pattern that is designed to improve the
consistency and readability of assertion failure messages when things go wrong. While it's not possible
to add new methods to the Assert class itself, we can certainly create new assertions methods within
our own custom assertion class.

Some features, like the automatic conversion of test argument values from one type to another, work by
resolving services registered with the Gallio Runtime. New services, such as data type converts, can be
defined by creating a simple Gallio Plugin.

Pretty much everything in MbUnit can be extended in some way. The following sections explain some of
the mechanisms involved. You may also find it useful to download the MbUnit source code to see how
it works.

How to create an MbUnit attribute

TODO

How to create an MbUnit assertion

TODO

How to create an MbUnit data source

TODO

How to create a custom type converter and

formatter
TODO

30
Chapter 12. Troubleshooting
TODO

31
Chapter 13. Migration Guide
TODO

32
 Part III. Test Framework Adapters
Gallio provides test framework adapters for numerous test frameworks. This part describes how to use each of them.
Chapter 14. csUnit
TODO

34
Chapter 15. MbUnit v2
TODO

35
Chapter 16. MSTest
TODO

36
Chapter 17. NUnit
TODO

37
Chapter 18. xUnit.Net
TODO

38
 Part IV. Test Runners
Gallio's test runners integrate with a variety of tools. Some test runners are standalone applications, others integrate
with the IDE, and a few provide specialized tasks for release engineering purposes. This part describes how to use
each of them.

Choose your favorite!

Chapter 19. Icarus GUI
TODO

40
Chapter 20. Sail, a Lightweight Visual
Studio Test Runner

41
Chapter 21. TestDriven.Net

42
Chapter 22. Visual Studio Team System

43
Chapter 23. ReSharper

44
Chapter 24. Echo Command Line
TODO

45
Chapter 25. Powershell

46
Chapter 26. MSBuild

47
Chapter 27. NAnt

48
Part V. Test Tools
Table of Contents
28. Ambience ................................................................................................................... 51
29. AutoCAD Integration ................................................................................................... 52
Introduction ............................................................................................................. 52
Using the AutoCAD Test Runner ................................................................................ 52
How Gallio Finds AutoCAD ............................................................................... 52
30. NCover ...................................................................................................................... 53
31. Pex ........................................................................................................................... 54
32. TypeMock .................................................................................................................. 55
33. WatiN ....................................................................................................................... 56
34. CruiseControl.Net ........................................................................................................ 57
35. TeamCity ................................................................................................................... 58

50
Chapter 28. Ambience
TODO

51
Chapter 29. AutoCAD Integration
Introduction
Programs that depend on ObjectARX, the API that Autodesk provides for extending AutoCAD, can be
difficult to test because ObjectARX imposes several restrictions on applications that depend on it:

• The ObjectARX API is only usuable within an "ObjectARX Host Application" such as AutoCAD or
one of its product verticals.

• The ObjectARX API can only be accessed by a single thread at a time. Additionally, many operations
such as calling the destructor on an AcDbObject must be performed on a specific thread provided by
the host application.

• The ObjectARX API makes use of Microsoft's OpenMP implementation. This implementation does not
allow for non-default AppDomains to be created within the same process that OpenMP (vcomp.dll) is
loaded in.

Gallio's AutoCAD support allows for tests to be run within this set of restrictions.

Using the AutoCAD Test Runner

The AutoCAD test runner is not a full-fledged test runner such as Icarus or Echo. Instead it enables the
actual test runner to perform the run within AutoCAD.

To use the AutoCAD test runner specify "AutoCAD" as the runner type. This option is only be available
on systems that have Gallio's AutoCAD integration components installed.

The runner type is specified in different ways depending on the test runner used. In Gallio Icarus, it appears
as a preference item in the GUI. With Gallio Echo, it appears as a command-line argument. Likewise the
MSBuild, NAnt and PowerShell tasks accept a runner type as an argument. Refer to the documentation
of your test runner for more information.

The following example shows how to run tests from the command-line using AutoCAD.

Gallio.Echo {/runnerType:AutoCAD} paths-to-test-assemblies...

How Gallio Finds AutoCAD

Gallio finds AutoCAD by looking in the Windows registry to see what AutoCAD application is registered
to handle the .DWG file extension. Typically this is whatever instance of AutoCAD ran most recently. If
you have multiple versions of AutoCAD installed and want to test with a specific one, run the one you
want before starting your tests.

52
Chapter 30. NCover
TODO

53
Chapter 31. Pex
TODO

54
Chapter 32. TypeMock
TODO

55
Chapter 33. WatiN
TODO

56
Chapter 34. CruiseControl.Net
TODO

57
Chapter 35. TeamCity
TODO

58
 Part VI. Developer Guide
This part explains how Gallio works, how to extend it, and how to use it to build new tools of your own.

Let's peer into the guts of the machine...

Chapter 36. Getting the Code
This chapter explains how to get the Gallio source code, build a local copy, and test it. If you intend to
participate in the development of the Gallio Project, start here!

Gallio is intended to support many extensions without requiring direct modification of its code. (We
attempt to follow the Open-Closed Principle of software development.) Consequently, if your purpose is
to develop a new extension, you probably do not need to get the source code; unless you want to...

Prerequisites
To build Gallio locally and work with the source code effectively, you will need to install some tools.
Some tools are optional: if they are missing, you can still perform a local build but it will be missing certain
extensions. Several tools are included in the source tree itself for convenience: they are not listed here.

Required Software
• A Subversion client

Note
We recommend TortoiseSVN for Windows systems but any client will do.

• .Net Framework 3.5

Note
If you have everything else installed then you probably already have this...

• Visual Studio 2008 SP1

Note
Strongly recommended for working with Gallio source code effectively. Older versions of
Visual Studio will not work.

• PowerShell

Note
Required to run a few of the helper scripts as well as for building and testing the PowerShell
extensions.

Optional Software
• Visual Studio 2008 SDK

Note
Required to build and test the Visual Studio 2008 extensions as well as the integrated help
documentation.

• Debugging Tools for Windows

60
 Getting the Code

Note
Required to generate Source Server annotations in the PDBs. This feature enables users to
automatically download Gallio source code from within the Visual Studio debugger when
Source Server support is enabled.

• Perl

Note
Requires by the Source Server scripts in the Debugging Tools for Windows package. The
ActivePerl package for Windows works fine. You can strip out all of the documentation,
examples and optional services.

• Visual Studio 2010

Note
Required for building and testing the experimental Visual Studio 2010 extensions.

• ReSharper v3.1, v4.0 and/or v4.1

Note
Required for building and testing the ReSharper extensions.

• Pex

Note
Required for building and testing the Pex extensions.

• NCover v2

Note
Required for building and testing the NCover v2 extensions.

• TypeMock v5

Note
Required for building and testing the TypeMock v5 extensions.

• Microsoft FXCop 1.36

Note
Required for code analysis in Release builds.

• TestDriven.Net v2.18 (or newer)

Note
Useful for running tests in Visual Studio but not required for builds.

61
 Getting the Code

• AutoCAD

Note
Required for testing the AutoCAD extensions.

• Mono v1.9.1 (or newer)

Note
Required for testing the Mono runtime support.

Checking out the Source Code

The source code is hosted on Google Code. Using your Subversion client, check out one of the following
repositories:

• Gallio and MbUnit v3 Source: http://mb-unit.googlecode.com/svn/trunk/v3

• Gallio Book: http://mb-unit.googlecode.com/svn/trunk/Book

• Gallio and MbUnit WebSite: http://mb-unit.googlecode.com/svn/trunk/WebSites

If you are a committer, you must use https and specify your Google user id and generated Google Code
password. This is not the same as your orginary Google password.

For more information visit: http://code.google.com/p/mb-unit/source/checkout.

Working with the Gallio and MbUnit v3 Source

Code
This section explains how to work with Gallio and MbUnit v3 source code. We will assume you have
checked out the source code from Subversion into the C:\Source\Gallio folder but you can put it
anywhere you like.

How to Run a Build

Probably the first thing you will want to do is to run a build to be sure everything has been checked out.
Fortunately this is quite easy:

> cd C:\Source\Gallio
> Build

After a little while, you should get some binaries created in the C:\Source\Gallio\build\image
folder. You can run Gallio directly from that location if you like, or create an installer.

> Build /dist

To clean up the contents of the build folder, run the following command:

62
 Getting the Code

> Build /clean

The Build.bat command has many other options. Here are a few more interesting combinations.

> Build /?
> Build /test
> Build /clean /build /image /test /dist
> Build /configuration Release
> Build /version 3.1.0.0
> Build /verbosity detailed

Understanding the Source Tree

The source tree layout reflects the modules that make up Gallio.

Table 36.1. Source Tree Layout

Path Description
/bin Scripts used to build Gallio.
/build An automatically created folder that contains the
output from the build. This folder is deliberately not
checked into Subversion.
/compat Modules used to perform code analysis with FxCop
and to compare new and old versions of Gallio
with LibCheck to help identify binary compatibility
issues.
/docs A little documentation about how to work with the
source tree.
/libs Shared libraries used by multiple Gallio
components. Libraries used by only one component
are generally stored in private "libs" folders within
that component's folder.
/src Root of the source tree.
/src/Gallio Gallio core runtime, base libraries and tests.
/src/MbUnit MbUnit v3 framework and tests.
/src/Installer.Wix The Gallio installer packages built using the WiX
compiler.
/src/Extensions Contains all optional extensions to Gallio including
all of the standard tools. Each one is held is a
separate folder.
/tools Third party tools and dependencies used to build
Gallio.

Working with the Solution

First, open up the All.sln with Visual Studio 2008. If you are working with Visual Studio 2010, use
All.vs2010.sln.

63
 Getting the Code

The first thing to notice is that there are many projects due to the large number of extensions. The projects
have been grouped into Solution Folders by type and purpose. These folders mirror the structure of the
code in the source tree.

Some projects may not load correctly because they depend on third-party tools that are not installed. This
is normal. If you choose to install one or more of these tools later, be sure to reopen the solution.

Important
The projects have been customized by hand editing of the *.csproj files.

When creating a new project, be sure to perform the following steps:

1. Create the project within the appropriate folder of the build tree. When creating a new extension, be
sure to give it its own folder.

2. Change the output path for the project in both the Debug and Release configuration to bin.

3. Enable the Warnings as Errors option in both the Debug and Release configuration.

4. Delete the [AssemblyVersion] and [AssemblyFileVersion] attributes from AssemblyInfo.cs.

Customize the other attributes as appropriate. Be sure to look at how the attributes in other Gallio
projects have been set.

5. Save the project.

6. Edit the *.csproj Visual Studio project file.

If the assembly must be signed then include the following elements in the first PropertyGroup element:

<SignAssembly>true</SignAssembly>
<AssemblyOriginatorKeyFile>..\..\Key.snk</AssemblyOriginatorKeyFile>

Tip
Compare against other projects for examples.

7. Include the project in a *.module MetaBuild module file. If the project is part of a new component,
then you might need to create a new module file to describe it. Be sure to specify items for files to be
copied, XML documentation to be preprocessed, and tests to be executed.

Tip
Compare against other modules for examples or read the MetaBuild build system
documentation.

8. Include the project in a *.wxs WiX package fragment file. If the project is part of a new component,
then you might need to create a new fragment to describe it.

Tip
Compare against other package fragments for examples or read the WiX installer compiler
documentation. Also be sure to give each WiX compnent a unique GUID.

64
 Getting the Code

Running Tests in Visual Studio

The easiest way to run tests for Gallio within the IDE is to install TestDriven.Net. Then run the following
script from the command-line to register the Gallio TestDriven.Net extensions from within the source tree.

> src\Extensions\TDNet\Install-TDNetRunner.bat

Once the extensions have been registered, you should be able to run tests directly from Visual Studio using
the TestDriven.Net Run Tests command. To run tests with the debugger, use Test With... Debugger.

For efficiency, TestDriven.Net will keep Gallio in memory across multiple test runs. Occasionally this
will cause problems when performing a build from Visual Studio because one or more of the assemblies is
still in use. To ensure TestDriven.Net releases its hold on these files, be sure to close the TestDriven.Net
agent (right-click on the rocket ship icon in the system tray) beforehand.

Tip
When developing Gallio, we recommend uninstalling any copy you may previously have
downloaded and installed. Installed Gallio features, such as the ReSharper extensions, may
conflict with the development copy of Gallio in the source tree and result in crashes, locked files,
or other unusual behavior!

Working with the Gallio Book

This section explains how to work with Gallio Book content. We will assume you have checked out the
source code from Subversion into the C:\Source\GallioBook folder but you can put it anywhere
you like.

Tip
You will need a good XML editor. We recommend oXygen because it comes bundled with
DocBook support and is relatively cheap but any editor will do...

How to Run a Build

Building the book is much like building the Gallio and MbUnit v3 source code since it uses the same
underlying collection of scripts. Most of the time, this is all you will need to do.

> cd C:\Source\GallioBook
> Build

The resulting compiled documentation (in several different formats) will be placed in the /build/image
folder within the source tree.

Working with DocBook

The book is arranged in a tree of folders corresponding to its parts and chapters. Files and other dependent
resources are generally stored in the same folder as their associated fragments. These fragments are
composed together hierarchically using XInclude tags.

65
 Getting the Code

For more information about DocBook, visit the DocBook website: http://www.docbook.org/.

Working with the Gallio Web Site Projects

This section explains how to work with Gallio Web Site content. We will assume you have checked out
the source code from Subversion into the C:\Source\GallioWebSites folder but you can put it
anywhere you like.

The web sites are just ordinary ASP.Net projects implemented in C#.

How to Run a Build

Building the web sites is much like building the Gallio and MbUnit v3 source code since it uses the same
underlying collection of scripts. Most of the time, this is all you will need to do.

> cd C:\Source\GallioWebSites
> Build

The resulting compiled web sites will be placed in the /build/image folder within the source tree.

66
Chapter 37. Understanding the Gallio
Architecture
Gallio, as a platform, is built up in layers.

TODO, include diagram here...

Gallio Runtime
The Gallio Runtime is the subsystem responsible for initializing Gallio-based applications such as test
runners, managing global configuration parameters, loading plugins, and enumerating services. It binds
the whole platform together.

At its core, the runtime is essentially a service locator or inversion of control container. It reads declarations
of components from plugin metadata files and provides a factory for instantiating them on demand. It
also tracks the lifecycle of components such that they can be decommissioned (disposed) gracefully upon
shutdown.

Services and Components

[Definition: A service is a description of an interface (contract) that may be implemented by one or more
components. A service may specify that its components are required to provide declarative metadata in
addition to executable code. Metadata is used to describe properties of a component without having to load
its code. It may also be used to specify activation criteria for the component.] [Definition: A component
is a class that implements the operations described by a given service. A component may also include
metadata.]

For example, Gallio's core library declares a service that describes an interface for a test framework and
its metadata. Each test framework plugin then defines a component to provide a reference to its executable
code along with its name, description, version and an icon.

The runtime is optimized to defer the instantiation of components until they are needed. This helps Gallio
to minimize its startup time and footprint even in situations where many plugins have been installed but
are not yet required for execution.

Plugins
[Definition: A plugin is a unit of extension supported by the runtime.] [Definition: A plugin manifest is a
simple XML file that defines the services and components that constitute a plugin. XML is used specifically
to avoid having to load plugin assemblies until they are required for execution.] [Definition: A plugin
assembly is a .Net assembly that extends the Gallio Platform in some way by providing executable code
and resources associated with services or components.]

A plugin is generally represented as a collection of files including a plugin manifest, one or more plugin
assemblies and additional resources as required. This layout is described in a later section.

The Runtime Lifecycle

Applications based on the Gallio Platform must initialize the Runtime before calling into most Gallio
services. When the runtime is initialized, Gallio populates its list of services but it does not instantiate
them until they are needed.

67
 Understanding the Gallio Architecture

While an application runs, it has free access to all runtime services. The following basic operations are
provided: getting descriptions of plugins, services and components; activating and deactivating plugins;
resolving components; obtaining plugin resources.

When an application is about to exit, it should dispose the runtime. This ensures that Gallio services are
shut down appropriately.

Example 37.1. Using the Runtime

Runtime.Initialize(NullLogger.Instance);

IMyService component = Runtime.Components

.Where(componentDescription => componentDescription.Metadata["Name"] ==
.Resolve<IMyService>();
component.DoSomething();

Runtime.Shutdown();

68
Chapter 38. Creating Plugins
TODO

69
Chapter 39. Creating Test Frameworks
Concepts
TODO

The EasyTest SampleFramework

The EasyTest sample illustrates the process of creating a new test framework using Gallio. Gallio provides
the platform and test runners so the framework is only concerned with the syntax.

Our sample test framework will be called EasyTest. We present two different implementations of EasyTest
dubbed Low-Level and High-Level.

EasyTest syntax
To keep the sample simple but interesting, EasyTest defines only a few different syntax elements.

TODO

Low-Level Implementation
The low-level implementation is a bare-metal implementation of the framework on Gallio. It directly
implements the ITestFramework interface and performs its own reflection.

Working in this way, a framework has full control over its syntax. It can generate test structure using
reflection and other means at will.

The low-level interfaces shown here are also applicable to the implementation of adapters for existing test
frameworks. A test framework adapter only differs in how it generates the structure of its tests. Instead
of performing reflection, it may ask the adapted test framework to generate a tree of tests which is then
transformed into a Gallio test tree.

Likewise, the low-level interfaces may be used to construct tests from metadata stored in other forms
such as XML-based test specifications. Or adapters may be created for test frameworks not implemented
with .Net (by means of pipes, for example).

The low-level interfaces expose all of the power of the Gallio test framework API. However, if you are
developing a new test framework from scratch you may leverage more Gallio infrastructure using the high-
level interfaces described next.

TODO concepts

TODO implementation walkthrough

High-Level Implementation
The high-level implementation leverages an abstract test framework provided by Gallio, called the Pattern
Test Framework.

70
 Creating Test Frameworks

The Pattern Test Framework is itself a low-level implementation of a test framework based on reflection
over attributes. Each attribute implements a distinct pattern: a rule for interpreting the semantics of the
attribute and collaboratively populating the test tree.

Defining new rules is as simple as subclassing existing attributes and overriding parts of their
implementation to define new semantics. We will discuss this process in greater detail below.

For branding purposes, a test framework built on the Pattern Test Framework should implement
IPatternTestFrameworkExtension. This ensures that the name of the derived framework can be presented
to the user as appropriate.

The Pattern Test Framework is somewhat abstract but very powerful. It is designed to simplify the task of
constructing test frameworks with a highly composable and general feature set. MbUnit v3 itself is based
on it. However, if you require more control over the process of test exploration and test execution you may
prefer using the low-level interfaces directly instead.

TODO concepts

TODO implementation walkthrough

Summary
In this chapter we have discussed two different implementation strategies for custom test frameworks on
Gallio.

The low-level interfaces expose all of the power of Gallio and enable infinite customization of the
test framework syntax. The high-level interfaces, in the form of the Pattern Test Framework, provide a
convenient foundation for a powerful and composable test framework.

Build your own.

71