31 Days of Testing—Day 14: Tests as Specifications

Updated: Index to all posts in this series is here!

Your automated tests, be they unit, integration, or functional, are amazing things. They validate parts of your system are working as expected; they free you from tedious, repetitive manual testing so you can focus on high-value exploratory testing; etc., etc. etc. Automated tests are great.

What they aren’t always good at, in their basic form, is providing a good communication channel between the technical folks, testers, and business owners/stakeholders/folks who are writing the checks for your work. Take a look at any of the Selenium examples I’ve used in previous posts. Can you really expect your stakeholders to understand what’s going on? Should you? Likely not, but isn’t this a missed opportunity to get some great communication around what you, your devs, and your stakeholder actually want the system to do?

To steal a phrase from the mobile world, there’s an app for that. Well, not an app, specifically, but a broad set of tools, frameworks, and nifty software that can help you greatly facilitate those conversations.

Your non-dev folks on your team likely can’t understand something like the cascading drop down menu test I wrote up the other day:

[Test]

public void Working_with_no_content()

{

    string make = "Acura";

    string model = "Integra";

    string color = "Sea Green";

    browser.Navigate().GoToUrl(

    "http://www.asp.net/ajaxLibrary/AjaxControlToolkitSampleSite/CascadingDropDown/CascadingDropDown.aspx");

    var listOfMakes = browser.FindElement(By.Id("ctl00_SampleContent_DropDownList1"));

    WebDriverWait wait = new WebDriverWait(browser, TimeSpan.FromSeconds(10));

    wait.Until<IWebElement>((d) =>

    {

    return d.FindElement(By.XPath(

        "id('ctl00_SampleContent_DropDownList1')/option[text()='"+make+"']"));

    });

    var makeOptions = new SelectElement(listOfMakes);

    makeOptions.SelectByText(make);

    var listOfModels = browser.FindElement(By.Id("ctl00_SampleContent_DropDownList2"));

    wait.Until<IWebElement>((d) =>

    {

    return d.FindElement(By.XPath(

        "id('ctl00_SampleContent_DropDownList2')/option[text()='"+model+"']"));

    });

    var modelOptions = new SelectElement(listOfModels);

    modelOptions.SelectByText(model);

    var listOfColors = browser.FindElement(By.Id("ctl00_SampleContent_DropDownList3"));

    wait.Until<IWebElement>((d) =>

    {

    return d.FindElement(By.XPath(

        "id('ctl00_SampleContent_DropDownList3')/option[text()='"+color+"']"));

    });

    var colorOptions = new SelectElement(listOfColors);

    colorOptions.SelectByText(color);

    browser.Manage().Timeouts().ImplicitlyWait(TimeSpan.FromSeconds(10));

    Assert.IsTrue(browser.FindElement(By.Id("ctl00_SampleContent_Label1")).Displayed);

}

but what they can understand is something like this:

Given I am at the main page

When I select Acura for Model

And I select Integra for Make

And I select Sea Green for Color

Then the result message shows ‘You selected a Sea Green Acura Integra!’

This idea of being able to specify your system’s behavior in plain English, or something close to it, is the fundamental concept of tools like Cucumber, JBehave, MSpec, and a host of other tools. Rather than losing behavior in the depths of C#, .NET, Ruby, or some other language we can bring it up to something in plain English like Cucumber, or a set of clearly understood tables in the case of Fitnesse.

Instead of me spending a lot of bandwidth trying to rehash all this, I am going to give you some reading assignments.

First, go grab a copy of Gojko Adzic’s Specifications By Example. He does an amazing job of taking the technology out of the picture and laying out a wonderful case for why you should look to specifications. He’s backed up his position with a number of great case studies. Perhaps one of the best books I’ve ever read on testing.

Secondly, go read Elisabeth Hendrickson’s awesome post on selecting test automation tools. She does a wonderful job of laying out how you might approach investigating and selecting some tools to help you bridge these communication gaps. The only additions I’d make to her great article is to say

1. I don’t think you necessarily need to write tests in the same language as your system. Figure out what tool works best for your team.
2. Think about your customer’s skills and teams if you’re handing off your system and tests. Will your customer be able to deal with all those Fitnesse or Cucumber fixtures? Just because you as a consultant think it’s cool and useful doesn’t mean it’s going to be great for your customer.

Finally, do your own trolling around for topics like Behavior Driven Development, or Acceptance Test Driven Development. There’s some wonderful stuff being written by wicked smart folks.

Tomorrow I have a post on using Cucumber in the real world written by my good pal Tim Wingfield. I think that will be a great practical follow on to this post!