Thursday, December 08, 2011

31 Days of Testing—Day 8: Pay Attention to Your Tests’ Setup!

Updated: Index to all posts in this series is here!

In yesterday’s post I laid out some fundamentals of what a test looks like in C# with NUnit, plus how you’d go about running it and getting results. Keep in mind that while I’m using NUnit and C# to illustrate fundamentals for these few columns, the basic tenets apply regardless of your platform or testing framework/toolset.

Today I want to focus on handling setup conditions for your tests. In yesterday’s examples, every test created a new WageComputer instance. This sort of duplication gets tedious and can lead to more maintenance hassles since it violates the DRY principle (Don’t Repeat Yourself). If you scatter creation of prerequisite classes, services, data, etc. across multiple tests, you’ll have to go to those same multiple tests to fix busted instantiations when those prerequisites change.

Instead, look to push this sort of work off to a centralized location. In many cases, you can use your test framework’s features to do this. NUnit and many other frameworks support setup methods at the fixture and namespace scope. Moreover, you can have these setup methods run once per class/fixture, or once per test. This lets you have great control over where you create prerequisite objects, load/create data, etc.

Here’s how this looks in NUnit using my examples from the previous post:

public class When_working_with_an_hourly_worker
    private WageComputer computer;
    bool isHourlyWorker;
    public void Run_once_before_any_tests_run()
        isHourlyWorker = true;
        computer = new WageComputer();
    public void Computing_with_40_hours_at_rate_5_returns_200()
        var computedWages = computer.ComputeWages(40, 5, isHourlyWorker);
        Assert.AreEqual(200, computedWages);
    public void Computing_with_41_hours_at_rate_5_returns_207_5()
    var wages = computer.ComputeWages(41, 5, true);
    Assert.AreEqual(207.50, wages);

The TestFixtureSetup attribute will cause the NUnit runner to execute the method Run_once_before_any_tests_run when this class/fixture is first loaded. We’ll stand up a new instance of the WageComputer and set isHourlyWorker true. This is very similar to a class constructor, but the lifecycle is managed by the test framework, not the .NET internals. (Which I couldn’t explain to you. Go ask Jon Skeet or Bill Wagner.)

There’s a similar attribute called SetUp which executes before each test. This is handy if you need to freshly initialize something before each test. (Remember, tests shouldn’t rely on any state set in another test.)

What gets set up may need to get torn down. Frameworks and tools generally support some form of TestFixtureTeardown or Teardown approach to clean up after each fixture or each test, respectively. These are great places to stuff transaction rollbacks to clean up after database interactions, for example.

This is a very simplistic, trivial example, but I’m sure you grok the general concept here. You can use fixture setup and teardown methods to deal with fixture-related prerequisites. Very handy.

Now for some important caveats.

While you want to avoid too much duplication, you can easily get carried away and overly clever with inheritance and abstraction of your setup actions—to the point where it gets extraordinarily difficult to understand what’s being set up where.

In a previous life I worked with an internally created test framework based off a Behavior Driven Development framework. We let ourselves get perhaps a little overly clever with our inheritance and setup chain, and it became very difficult to learn and understand. (OK, there’s no “perhaps” about it. We did, and I was a major part of letting that happen. Bad Jim. Bad Jim!)

Using some psuedo code, the inheritance and setup chain looked something roughly like this:

Integration_test  (Base)
  //sets up database
Functional_test (Inherits from Integration_test)
  //sets up browser, configures UI
Feature_test  (Inherits from Functional_test)
  //configures system
Module_test  (Inherits from Feature_test)
  //creates a module
//Now we get to the actual tests!
When_creating_something_as_regular_user  (Inherits from Module_test)
  //creates new user
  //FINALLY does some testing!

Imagine you’ve got a failure in a test. You could potentially have to walk back up four layers of inheritance to understand exactly what’s going on where. This is an overly clever approach to setup which makes understanding of your tests extremely difficult. Remember that code is read and re-read many more times than it’s written.

In these sorts of cases it’s OK to relax a bit on the Don’t Repeat Yourself principle. There’s an extremely applicable saying for this situation: “Keep your system DRY and your tests slightly moist.” What that means is, it’s OK to duplicate some setup steps in your tests if it makes the fixture/class/spec more understandable.

The Bottom Line

Leverage your testing platform/framework’s features for helping you get prerequisites for your tests set up, just don’t get so convoluted that you can’t easily understand what’s going on in the test when you open it up a couple weeks or months after you’ve written it.


MK23 said...


IMHO: You will miss bugs that way.

Types are critical.

Jim Holmes said...

@mk23 I, and a huge group of folks using dynamic languages, vehemently disagree WRT "Types are critical" but that's a much longer discussion better had over adult beverages. :)

Jasmine said...

Except that the use of var in this case doesn't get you anything other than hiding the type. The function you're calling is strongly-typed, so the "var" will always be "double" (I think) and writing "double" instead of "var" would make the code more understandable.

But I was reluctant to say that because I'm loving your series :)

Nick said...

I like var, if you want to test for type either write another test or add an assert that checks for type.

I don't like using setup and teardown for cases like this because it hides what is going on. ie, there is nothing under the //Arrange comment. I like to make a descriptive method name that does the same thing and call that instead. Problem is when you need to return more than 1 thing from setup.

Subscribe (RSS)

The Leadership Journey