Thursday, July 17, 2008

Integrating your HUnit (or other) tests into your cabal package

As part of one my side projects I'm working on learning haskell, and creating a simple DNSBL lookup tool. An important part of any project, especially one which you plan to build on top off is its testing. There are quite a few options for Haskell testing such as quickcheck,etc., and the one that I ended up using HUnit. Most developers that I know have been burned at one time or another by making some "trivial" change building it and checking it in to only have it fail when running there tests would have caught it.

The solution of course is to integrate your automated tests into the build process. Cabal is a wonderful package build system for haskell. Unfortunately, very few (if any) people seem to use the provided mechanism for hooking in tests. The first set is opening up the Setup.hs (or lhs) file and overriding the default test hooks.
Start off by importing the necessary libraries

import Distribution.Simple
import Distribution.PackageDescription(PackageDescription)
import Distribution.Simple.LocalBuildInfo(LocalBuildInfo)
import System.Cmd(system)
import Distribution.Simple.LocalBuildInfo

Then we need to replace the default main (main = defaultMain) with a main with our command hooks:

main = defaultMainWithHooks (simpleUserHooks {runTests = runzeTests})

Now we need to define runzeTests so that our tests get run. You might be tempted to just through your tests inside runzeTest, but thats a bad idea since your module isn't visible. So we just make the test hook call out to our program:

runzeTests:: Args -> Bool -> PackageDescription -> LocalBuildInfo -> IO ()
runzeTests a b pd lb = system ( "runhaskell ./tests/mytests.hs") >> return()

Now if you've put your sources in a subdir (lets say src) you need to tell runhaskell where to find it, so instead do this

runzeTests a b pd lb = system ( "runhaskell -i./src ./tests/mytests.hs") >> return()

And now its done! You can run your tests with cabal by running

runhaskell Setup.hs test

The alternative approach, if you are using darcs, is to set darcs to run your tests:

darcs setpref test "runhaskell test/test.hs"

Or for the best of both worlds, configure cabal as above and set darcs to run your cabal tests:

darcs setpref test "runhaskell Setup.hs test"

For those of us who are running in git we can add it to our git pre-commit hooks to keep us from accidentally checking in broken code, just add:

runhaskell Setup.hs test

to the bottom of your pre-commit hook file (don't forget to make it executable :)).
We can even make the commit fail if our tests fail by returning the status from the test. Just store the return code where you run the test:

runhaskell Setup.hs test
code = $?
...[someother precommit hooks]...
exit code


Jeremy Shaw said...

Cool, I've been meaning to learn how to do this. Thanks for the information!

As as side note, I have implemented a little wrapper that allows you to use QuickCheck Properties from in HUnit.

The source is available via darcs at

Holden Karau said...

Thanks :)

Thats really cool :) More integration between the testing frameworks is winsauce :) said...

Thanks for the step-by-step. I had to change the darcs command to include calling 'configure' first before this would work:

darcs setpref test "runhaskell Setup.hs configure; runhaskell Setup.hs test"

jetxee said...

Thanks! That was useful. I just had to remember to change my HUnit script so that it actually exitFailure-s if there are errors.

Free Blog Counter