Universal NAnt Script for MbUnit

If you run automated builds using NAnt, you may or may not know about the MbUnit Task you can add to NAnt.  MbUnit has pretty good directions for how to do get their task setup. I've been out to their site a couple of times trying to remember the exact details of the mbunit NAnt task.  Dru put the thought into my head, why not make the concept of running the tests a more generic and universal item so that you can just drop a couple of files into a project and be done?

I thought about this for awhile and realized it can be done pretty easily because the mbunit task can use pattern matching for finding assemblies to include in running tests.

If you follow a simple rule for testing, you can use the following ideas to run in an automated build setting and on your build server (Continuous Integration anyone?). When you are creating tests, always separate your tests into their own projects with the word "Test" (I prefer the plural "Tests") in the resulting DLL. A good example of this is Company.Project.Area.Tests.

Also, always separate your integration tests into different projects with the word ".Integration" for the integration tests). A good example of this is Company.Project.Area.Tests.Integration. I even go as far as to put database type tests into another test project that has the word Database and the word Tests (or Test) somewhere in it.

This allows you to separate your fast running tests (your logical and unit type tests) from your tests that are dependent on external factors and subject to possible failure for a plethora of possible reasons. You still want to run the integration tests to verify your code, but not when you are quickly verifying it after small changes and check-ins (commits).

Note: This doesn't always mean putting your tests in separate projects as long as they are separated when compiled into DLLs.

Why Do We Need a NAnt Task for Testing? OR Why Should I Read the Rest of this Post? I Already Run Tests in My Automated Build

Now before we get started, let's talk about why you would even want to go to this work when you have a perfectly good way to test your application in Visual Studio.

Assuming you do automated builds and don't yet also do tests on the build server, you add the ability for your build server to run automated tests.

Second, it allows you to go to a command line and type test and hit {Enter}.  And outside of visual studio your code is compiled, tested, and the results are displayed for you in Internet Explorer (or another means if you want). Five keystrokes and BAM! you've just verified your code!  You also have a nice display of a mbunit report in your browser showing you any issues that may exist (instead of just telling you that there were failing tests).  You have just become much more productive than you were using just Visual Studio.

image

NOTE: Another file to have around is the build batch file which does the default build, compile, testing, and packaging of your project.  That's very nice to have for both you and the build server.  In my mind the test batch file we will be creating serves a slightly different purpose, and that is one of getting test results to the developer quickly.

NAnt Script for MbUnit

The first thing we need to do is create our NAnt Script.  We will create a file called mbunit.build and then open it with notepad.  Let's put the following xml into the file.

<?xml version="1.0" encoding="utf-8" ?>
<project name="TestRunner" default="run">
  <property name="build.directory" value=".\bin" overwrite="false" />
  <property name="build_artifacts.directory" value="${build.directory}\build_artifacts" overwrite="false" />
  <property name="test_results.directory" value="${build_artifacts.directory}\mbunit" overwrite="false" />
  <property name="test_results.file" value="mbunit-results" overwrite="false" />

  <target name="run" depends="cleanup, run_tests" description="Tests" />

  <target name="cleanup">
    <echo message="Removing and adding ${test_results.directory}."/>
    <delete dir="${test_results.directory}" />
    <mkdir dir="${test_results.directory}" />
  </target>

  <target name="run_tests" depends="cleanup" description="Running Unit Tests">
    <echo message="Running tests using MbUnit and putting results in ${test_results.directory}."/>
    <mbunit
        report-types="Html;Xml;Text"
        report-filename-format="${test_results.file}"
        report-output-directory="${test_results.directory}"
        halt-on-failure="true"
        failonerror="true"
        >
      <assemblies>
        <exclude name="${build.directory}\*Database*dll" />
        <exclude name="${build.directory}\*.Integration*dll" />
        <exclude name="${build.directory}\TestFu.dll" />
        <include name="${build.directory}\*Test*dll" />
      </assemblies>
    </mbunit>
  </target>
  
  <target name="run_tests_all" depends="cleanup" description="Running All Unit Tests">
    <echo message="Running all tests (including integration tests) using MbUnit and putting results in ${test_results.directory}."/>
    <mbunit
        report-types="Html;Xml;Text"
        report-filename-format="${test_results.file}"
        report-output-directory="${test_results.directory}"
        halt-on-failure="true"
        failonerror="true"
        >
      <assemblies>
        <exclude name="${build.directory}\TestFu.dll" />
        <include name="${build.directory}\*Test*dll" />
      </assemblies>
    </mbunit>
  </target>
  
  <target name="open_test_results">
    <echo message="Opening test results at ${path::get-full-path(test_results.directory)}\${test_results.file}.html."/>
    <exec 
      program="${environment::get-folder-path('ProgramFiles')}\Internet Explorer\iexplore.exe"
      commandline="${path::get-full-path(test_results.directory)}\${test_results.file}.html"
      />
  </target>

</project>

What is this file doing? By default it calls the run target. That depends on the cleanup and run_tests, so it runs them in order of the dependencies from left to right.  Cleanup does basically what it says. It cleans out the mbunit folder and prepares it for the next set of results from testing.

The run_tests target runs, yep, it runs the tests using MbUnit. The mbunit task actually puts the results files in the folder after it runs the tests. And you may have noticed that we get three formats of our result set:HTML, XML, and TEXT. We have uses for the HTML and XML, but we include TEXT to be more generic. The HTML is for humans. It's readable and printable. The XML is for other applications, such as CruiseControl.NET (our CI server) to merge in the results so you get more information on the build server for each build.

Most of the time when we come to this build file, the run target is the behavior we expect out of it.  Yet we have two more targets to talk about.

We also have a run_tests_all target that we can call separately to run all tests. This can be used both locally and for running a nightly build. There is a little bit of repetition here in the actual target, but it is for clarity. It's nearly repeating run_tests (the only difference is that the included assemblies is broader) because I haven't come up with a better way yet.

Our last target, open_test_results, is also not referenced by the rest of the file, and for good reason: we don't want our build server EVER calling this target (whether by accident or on purpose). We want to call this task explicitly so that us developers can get to the details of the test results even faster, which makes us more productive with less work! I am opening the results in Internet Explorer because it is pretty universal on most developer machines (I haven't tested this on Vista yet).  Of course you are not bound to using Internet Explorer if you don't want to.  In that case just change out the program you call in this target.  But please make sure everyone else on your team can also use whatever program you decide to use without much work on their part.

Reference the Build File from the Main Build File

Second, we want to reference this build file from our main build file (if we don't include it in our main build file).

<target name="run_tests">
    <nant buildfile=".\mbunit.build" inheritall="true" />
</target>

Here we are referencing our sub build file using NAnt's nant task.

Add MbUnit Files to NAnt's Directory

We also want to make sure we have the correct files sitting in the NAnt directory.  From the MbUnit site:

  1. Copy the following DLLs from the MbUnit installation directory to the NAnt bin directory.
    • MbUnit.Tasks.dll
    • MbUnit.Framework.dll
    • QuickGraph.dll
    • QuickGraph.Algorithms.dll

Create a Test Batch File

Once we have everything else completed, we can create a batch file specifically for running tests and displaying the results. So we create a file called test.bat (in the same directory as our build files) and insert the below into it.

.\libs\Nant\nant.exe /f:default.build compile
.\libs\Nant\nant.exe /f:mbunit.build %1
.\libs\Nant\nant.exe /f:mbunit.build open_test_results

Here we are calling our default build file to compile the code for us. Then we are calling the mbunit build to run tests.  And here is the key part, once we run and save the test results, we want to see them.  So we call back into mbunit.build asking for the open_test_results target.  What's nice about this is that when it opens IE it automatically shifts focus to it. When we close IE ({Alt}+{F4}), our focus is shifted back to the command line.

Conclusion

All this goodness gives us five keystrokes of work to run tests from now on. And you can drop this stuff into any project (or retrofit an existing project) in a relatively short amount of time.  I literally dropped this into an existing project while writing this post and only had to change where I referenced NAnt and the target to call on the default build for it to compile the code. And you can see those changes only had to be made in the test.bat file.

Happy Building!

Print | posted @ Tuesday, January 27, 2009 10:40 PM

Comments on this entry:

Gravatar # re: Universal NAnt Script for MbUnit
by John Miller at 2/20/2009 1:40 PM

Hey Rob, Not sure how I didn't see this post before but I like the idea of running a *pretty* report from the command line. And a quick google search showed that there's an nunut2report task in nantcontrib for us nunit guys.

Awesome post man!!
Gravatar # re: Universal NAnt Script for MbUnit
by Robz at 2/22/2009 6:19 PM

@John: Thanks man!!
Gravatar # re: Universal NAnt Script for MbUnit
by Robz at 3/24/2009 11:50 AM

@TriLK: Try downloading from here: http://www.mbunit.com/Downloads.aspx
Gravatar # re: Universal NAnt Script for MbUnit
by CA Targets at 1/21/2010 3:59 AM

And a quick google search showed - there's an nunut2report task in nantcontrib for us nunit guys.Awesome post man!!
Comments have been closed on this topic.