31 Days of Testing—Day 13: Functional Test 201 (Common Problems)

Updated: Index to all posts in this series is here!

Updated: (Updated Case 2. Adam Goucher pointed out I referenced the Wait class which is for explicit waits, yet this case is about implicit waits. Yeesh. Major writing failure. Thankfully I have good pals who keep me straight.)

NOTE: I missed posting this yesterday due to a long trip back and forth to visit the amazing new expansion at the Kalahari in preparation for CodeMash. That coupled with a sick kid left me behind the power curve. Sorry!

Functional testing at the UI, particularly for web tests, can be a pile of convoluted, confusing poo. Headaches from AJAX dynamic calls, oddball JavaScript events interfering with fields you’re trying to work with, horrific DOM models, the list goes on and on.

There’s a nice opener for you, but never fear, I’m going to help you through a few of those—because I’ve suffered through them and would prefer you didn’t have to.

Testing a web application means you need to get down and jinky with exactly how your page is loading. You’ll need to work closely with your developers to understand what sorts of JavaScript you have on your page. You’ll need to look closely at your testing framework to understand how it interacts with the DOM and how it handles dynamic content situations.

Let’s look at three common situations you’ll likely run in to on web applications:

1. JavaScript is bound to an input field and is disrupting your test’s input to that field.
2. You’re working with an AJAX-ified page where the element you need isn’t yet loaded.
3. You’re working with an AJAX-ified page where the content you need isn’t yet loaded.

Case 1: JavaScript is Messing With My Fields!

Problem: You’ve got a JavaScript validation attached to one or more fields you’re trying to interact with. For example, on the Test Studio demo app we use JavaScript to check that the password and username fields both have content before we enable the login button. This JavaScript is causing your test framework some grief and the validation isn’t working to enable the button.

Answer: Varies by framework. When using Selenium, the field’s onchange event only fires when focus leaves that field. You’ll need to click to another field to get the validation to work properly. That might look something like this:

browser.FindElement(By.Id("username")).SendKeys("testuser");

browser.FindElement(By.Id("password")).SendKeys("abc123");

browser.FindElement(By.Id("username")).Click();

In Test Studio, we inject text directly to the DOM, so no events at all get fired on fields. You’ll need to use the “SimulateRealTyping” property to push the input text through the browser itself. This is a property on that specific test step:

Other testing tools/frameworks will have their own approaches to solving this problem. Point being, spend time understanding your app and your tools.

Case 2: AJAX (or some dynamic system) hasn’t yet loaded the element I need!

Problem: The elements you need to work with haven’t yet been loaded on the page.

Have a look at the ASP.NET AJAX demo page for a drop down menu. Use some tool like Firebug to inspect the current DOM and you’ll see the menu elements aren’t actually loaded on the page yet. The menu elements (the ones you want to interact with!) don’t get loaded until you click on the pull-down list. You can’t validate or interact with elements that aren’t yet there!

Answer: Use Selenium’s implicit wait feature  to help out with this. You set a default timeout for the browser session you’re currently using, after which Selenium will wait for the configured amount of time for the elements to appear. If the elements don’t show up in the DOM, you’ll see an Exception, which is what you want – the elements aren’t appearing, so your test is failing. In Selenium this looks like:

[Test]

public void Validate_correct_choices_generate_correct_answer()

{

    browser.Navigate().GoToUrl(

        "http://www.asp.net/ajaxLibrary/AjaxControlToolkitSampleSite/DropDown/DropDown.aspx");

    browser.FindElement(By.Id("ctl00_SampleContent_TextLabel")).Click();

    browser.Manage().Timeouts().ImplicitlyWait(TimeSpan.FromSeconds(10));

    browser.FindElement(By.Id("ctl00_SampleContent_Option1")).Click();

    Assert.IsTrue(browser.FindElement(By.Id("ctl00_SampleContent_lblSelection")).Displayed);

}

Note we’re setting the ImplicitWait property to ten seconds. The test will hang around that long waiting for the service call to finish and the DOM to update with the element we need to work with.

In Test Studio we already handle this sort of implicit wait without any extra steps.

Not sure how Watir handles these sorts of situations, but I would assume (yes, bad word!) that it’s very similar.

Case 3: AJAX (or some dynamic system) hasn’t yet loaded the ContentI need!

Problem: The content you need to work with hasn’t yet been loaded on the page.

Answer: Use explicit waits to delay your test’s execution until the content is loaded.

Have a look at the ASP.NET AJAX demo page for cascading menus.

Use some tool like Firebug to inspect the current DOM and you’ll see the menu elements are there, but not their content. Make some choices for the menus, look at the DOM. Now you’ll see the content exists after having been loaded by an AJAX callback. Because the elements exist, but not their content, we need to handle this with an explicit wait. (See the awesome Selenium article on explicit and implicit waits. You need to read this!)

In the example above we need to navigate to the page, then make a series of selctions. In each dropdown selection list, we’ll need to get that selection element, then wait for the specific item we want to appear. That content is loaded by the previous selection, or in the case of the Make selection, by the actual page loading.

Here’s how we write the first section of code to wait and select our option in the Make pulldown. After that, we use WebDriverWait to explicitly wait until our target element (the make of “Acura” in this case) appears. Once the content is loaded in to the options we can then select the specific item from the option list.

[Test]

public void Working_with_no_content()

{

    string make = "Acura";

    string model = "Integra";

    string color = "Sea Green";

    browser.Navigate().GoToUrl(

        "http://www.asp.net/ajaxLibrary/AjaxControlToolkitSampleSite/CascadingDropDown/CascadingDropDown.aspx");

    var listOfMakes = browser.FindElement(By.Id("ctl00_SampleContent_DropDownList1"));

    WebDriverWait wait = new WebDriverWait(browser, TimeSpan.FromSeconds(10));

    wait.Until<IWebElement>((d) =>

    {

        return d.FindElement(By.XPath(

            "id('ctl00_SampleContent_DropDownList1')/option[text()='"+make+"']"));

    });

    var makeOptions = new SelectElement(listOfMakes);

    makeOptions.SelectByText(make);

Actions on the remaining two lists are the same. For the final confirmation message, also AJAX-ified, we can go back to our implicit wait pattern shown earlier—that message is in an element which isn’t loaded on the page. (NOTE: The test for the confirmation message isn’t a great one. A better one would be to ensure the proper combination of make+model+color is rendered as well as the general message – I took this shortcut for brevity’s sake because this example’s already somewhat lengthy.)

browser.Manage().Timeouts().ImplicitlyWait(TimeSpan.FromSeconds(10));

Assert.IsTrue(browser.FindElement(By.Id("ctl00_SampleContent_Label1")).Displayed);

The entire test looks like this:

[Test]

public void Working_with_no_content()

{

    string make = "Acura";

    string model = "Integra";

    string color = "Sea Green";

    browser.Navigate().GoToUrl(

        "http://www.asp.net/ajaxLibrary/AjaxControlToolkitSampleSite/CascadingDropDown/CascadingDropDown.aspx");

    var listOfMakes = browser.FindElement(By.Id("ctl00_SampleContent_DropDownList1"));

    WebDriverWait wait = new WebDriverWait(browser, TimeSpan.FromSeconds(10));

    wait.Until<IWebElement>((d) =>

    {

        return d.FindElement(By.XPath(

            "id('ctl00_SampleContent_DropDownList1')/option[text()='"+make+"']"));

    });

    var makeOptions = new SelectElement(listOfMakes);

    makeOptions.SelectByText(make);

    var listOfModels = browser.FindElement(By.Id("ctl00_SampleContent_DropDownList2"));

    wait.Until<IWebElement>((d) =>

    {

        return d.FindElement(By.XPath(

            "id('ctl00_SampleContent_DropDownList2')/option[text()='"+model+"']"));

    });

    var modelOptions = new SelectElement(listOfModels);

    modelOptions.SelectByText(model);

    var listOfColors = browser.FindElement(By.Id("ctl00_SampleContent_DropDownList3"));

    wait.Until<IWebElement>((d) =>

    {

        return d.FindElement(By.XPath(

            "id('ctl00_SampleContent_DropDownList3')/option[text()='"+color+"']"));

    });

    var colorOptions = new SelectElement(listOfColors);

    colorOptions.SelectByText(color);

    browser.Manage().Timeouts().ImplicitlyWait(TimeSpan.FromSeconds(10));

    Assert.IsTrue(browser.FindElement(By.Id("ctl00_SampleContent_Label1")).Displayed);

}

Test Studio handles this in a similar fashion: wait for the item you need to work with to appear, then interact with it. If you’re working straight in code with the Telerik Testing framework, then it’s not overly different from above in concept. If you’re working with the Test Studio standalone edition the test would look like this:

Wrapping Up

There you have it. A quick walk-through of three common situations on web pages that can give automation folks grief.

31 Days of Testing—Day 12: Functional Test 101

Updated: Index to all posts in this series is here!

In my last post I gave you a bit of a drive-by spewing of some of the things I think are critical to keep your automation maintainable, particularly in the functional test world.

Now let’s take a look at what a functional UI test looks like. Again, there’s a huge difference in the specifics between platforms and frameworks/tools. I’m going to focus on a very simple web UI test to kick off here. I’m using Selenium 2.15. Selenium’s an open source web testing framework available on nearly every platform you could care to write code for. Watir for Ruby is another example of an open source web testing framework. Test Studio, the commercial product I’m involved with, has a free testing framework as its underpinnings.

These tools are used in stand alone tests, but they’re also used extensively as backing for other specification-style testing tools such as Cucumber, Fitness, Capybara, and others.

All these frameworks use roughly the same approach to testing your web applications:

• Start up a browser
• Navigate somewhere
• Find things of interest on the page to interact with
  • Inspect text
  • Input text
  • Click
  • Scroll
  • Blow up

(I may be exaggerating with the Blow Up part.)

Without further ado, let’s take a look at how a functional test with Selenium is built. For simplicity’s sake and readability I’m leaving this test a bit more linear than I normally would. I’ll show you some potential refactorings/optimizations in later posts—including some discussion of the Page Object Pattern I’ve mentioned previously.

You may need a separate test framework to actually execute your functional framework inside of. Selenium doesn’t come with any form of test runner, reporting engine, or even Assert support. The examples below use NUnit to drive the tests around

The app we’ll be testing is the Test Studio demo app hosted at Heroku. We’ll use two simple tests: check for the login link on the home page, and validate that we can log on to the system.

First test first. Skipping over setting up the project and references, the first thing you’ll need to do is declare a test fixture, plus a variable for the browser we’ll be working with. Selenium supports many different browser drivers, all of which implement the IWebDriver interface.

[TestFixture]

public class Home_page_displays_correctly

{

    IWebDriver browser;

Now we can use a fixture setup method to instantiate our browser. I’m using the FireFox driver, so there are a few profile and binary executable file things I need to deal with. NOTE: In a true implementation, I’d have this instantiation handled by some sort of provider so I could flexibly stand up Firefox, Chrome, IE, or whatever browser I wanted this run to be with.

[TestFixtureSetUp]

public void Run_once_before_anything()

{

    var profile = new FirefoxProfile();

    var exe = new FirefoxBinary();

    browser = new FirefoxDriver(exe, profile);

    browser.Navigate().GoToUrl("http://growing-planet-634.herokuapp.com/welcome")

}

The last step is to actually navigate to the URL/page we’re going to work with. You need to understand that the navigation will cause WebDriver to wait for the page’s “onload” event to fire. This is a handy blocking step, meaning you don’t need to worry about adding in code to pause and wait for the page to load. You also need to understand this does not have anything to do with dynamic content. You’ll need to handle that yourself. More on that later.

We’ve navigated to the home page, and now it’s time to find the elements we want to test for. Each framework is subtly different, but the general idea is the same: specify an HTML ID, CSS selector, XPath, or some other form of find logic so the framework can locate the element you want. In our WebDriver test we do the following:

[Test]

public void Login_link_is_correct()

{

    IWebElement loginLink = browser.FindElement(By.Id("login_link"));

    Assert.IsTrue(loginLink.Text.Equals("Login"));

    Assert.IsTrue(loginLink.GetAttribute("href").Contains("/login"));

}

We’re grabbing the element with the ID of “login_link”, then we’re checking text and attribute values on that element.  We’re using NUnit Asserts to make the actual test.

How do we know which IDs to work with? First, best answer: talk with the developers who created that UI. Second, nearly as good an answer: use a tool like Firebug, IE Developer Toolbar, or the DOM Explorer in Test Studio to help you walk through the page’s DOM.

We can also interact with elements on the page in the same fashion. If I want to log in, I’ll use these steps right after the initial navigation step:

browser.FindElement(By.Id("login_link")).Click();

browser.FindElement(By.Id("username")).SendKeys("testuser");

browser.FindElement(By.Id("password")).SendKeys("abc123");

browser.Manage().Timeouts().ImplicitlyWait(TimeSpan.FromSeconds(10));

browser.FindElement(By.Id("login_button")).Click();

I can interact with elements by first finding them, then using sensible commands like Click() for buttons/links, or SendKeys for input fields.

The fourth statement in this group sets an implicit wait for the navigation after the final Click() event  on the login button. While WebDriver nicely waited for the page to completely load after the Navigate call above, it doesn’t know to wait for a page load after clicking a link. We’re putting in an implicit wait to have Selenium pause before failing subsequent actions after that navigation.

In this case, that subsequent action is another test validating the presence of the logoff  link.

[Test]

public void Logout_link_displays()

{

    Assert.IsTrue(browser.FindElement(By.LinkText("Logout")).Displayed);

}

Without the implicit wait, WebDriver would blow past this assert, failing it because the page hadn’t properly loaded.

Implicit and explicit waits are one of the most critical things an automation developer needs to understand. The Selenium documentation has a great article on it which you need to read and understand—you’ll save yourself a lot of pain!

This brief overview showed some common themes regardless of the automation framework you’re using. You need to create a browser, have it navigate somewhere. That navigation may or may not handle delays necessary for your testing. Once you’re at a page you can find and interact with elements on that page.

I’ll follow this with a post on dealing with some common troublesome issues like AJAX or other dynamic content, pop up windows, file system dialogs, and others.

31 Days of Testing—Day 11: Maintainable Functional Automation

Updated: Index to all posts in this series is here!

I wanted to write a blog post on “Introduction to Functional Automation Testing” but I realized I firmly believe there are a number of things to know before you ever start down the road of functional automation so that you don’t get bushwhacked a few weeks or months in to your effort.

In my view, functional tests via a browser or desktop automation tool, are by far the slowest, most brittle, most curmudgeonly type of automated test. Maybe that’s why I like them so much…

I’ve been fortunate to have spent a few trips over the last month working with customers to train them up on Telerik’s Test Studio, the great automation tool I’ve joined Telerik to help promote. I think the customers are a little taken aback when I get very frank and animated about the difficulties around functional test automation. I also make it very clear that everything you do around functional test automation is closely tied to how well your tests will be running in four weeks or four months.

I’ve also been spending a LOT of time wandering around to various conferences and user groups giving my talk on Automation Isn’t Shiny Toys. That talk really drives home the point that you’ve got to carefully focus on your automation strategy if you want to succeed and stay sane.

In this post I want to highlight a few of the things I discuss in that talk. I’ll dive in to greater detail on a few of those points in later posts, but the post you’re reading now will give you the broad view.

I’ve seen automation fail, or cause tremendous stress, in a number of situations: Projects I’ve been on, projects I’ve seen colleagues on, projects I’ve talked about with friends and other conference attendees, and projects I’ve seen at customer sites. I’ve come to see three common threads among these conversations: long-running tests, brittle tests which break too frequently, and tests which take too much time to maintain over the life of the project. (I also admit that items two and three are likely the same root cause.)

Long Running Tests

Long-running test suites can suck the trust out of your project. Functional tests at the UI level will always be much slower than other sorts of tests, but you can’t have a productive, trustworthy automation environment if your tests can only be run once a month over a weekend. You need a moderately short execution time so you can have your UI tests running several times a day.

Keep in mind the scale of time in this context. 800 tests taking 30 seconds per test will take you nearly seven hours to execute. Small changes across many tests will net you huge gains.

Here are a few causes I’ve seen for overly long execution times:

• Poor infrastructure. You shouldn’t be running your UI tests in series. Scale out to some form of a grid and parallelize your execution. Get reasonable hardware to host your app and execution agents on. Get this all in place early in your project so you don’t have to sweat it when the pressure’s on.
• Using the browser for setup. Browser actions are SLLLLLOOOOOOOOW. They’re brittle, too. Never use the browser for setting up prerequisite data or configuration, always look to build a backing API to handle this sort of stuff. You can cut huge amounts of time out of your tests by pushing setup out of the browser and to an API.
• Testing too much. Just because you can automate something doesn’t mean you should. Focus on high-value tests around your most critical business needs, or the highest-risk areas. Keep other tests for your eyeballs and manual exploratory testing.
• Navigating where you don’t need to. Navigation is expensive. Wherever possible, bypass logons, navigation, and other unnecessary steps. You can even look to modify your system to be able to bypass these actions. Test these steps elsewhere, of course, but not every time!

Brittle Tests

Brittle tests break for odd reasons. Brittle tests work in the developers’ environments, but fail in the QA or staging environments. Brittle tests fail when run in your suite, but pass when run individually.

Here are a few causes of brittle tests I’ve run in to (or shot myself in the foot with!):

• Side effects. Every test absolutely must be self-sufficient and granular. A test needs to set up its own prerequisites and, where possible, clean up after itself. (Database transaction rollback, FTW!) Tests may not ever rely on state set by other tests! This doesn’t mean you can’t use a baseline dataset to configure broad sets of prerequisites, but you darn well better make sure to clean up after yourself.
• Bad environments. You can’t expect horrible environments to be stable. I once worked on a project where our functional tests were expected to run on our CI server. The server was continually slammed by builds from 20 devs. Moreover, the CI server had a database on it used by other servers. And it was a virtual machine on a poor host. Priceless. Get realistic environments. You’ll be happy you did.
• Complex, badly designed tests. This dovetails in with the side effects mentioned above. It also ties in to maintainability which I’ll address shortly. Mixed concerns, too many responsibilities, tight coupling to the UI or other components—this sounds just like problems we see when building systems. You know what? That impacts our tests, too! Pay attention to how you’re building your tests.

High-Maintenance Tests

If you’re not careful, you can easily end up spending more time maintaining your existing automated tests than you spend creating new ones or doing your exploratory testing. Roy Osherove, in his amazing book The Art of Unit Testing talks about a project that failed because his team spent too much time maintaining tests. Note they didn’t abandon testing on their project—the entire project failed! Ouch.

In my experience, for functional/UI tests, the number one cause of high-maintenance tests is poor element locator management. Side effects and poor test design contribute, but locators is the number one PITA around. Locators are how your testing framework/tool finds elements on the page or in the application. (I’ll have much more on locators in a following post.) If you aren’t careful, you’ll end up spending huge amounts of time fixing up your tests’ locators every time the UI changes even a small amount.

Two things can greatly help you cut down pain around your locator management.

1. Centralize your locator definition. This is one place to learn, live, and love the DRY principle. Having locators defined in more than once place in your codebase is a recipe for misery. If you’re writing purely code-based tests then look to something like the Page Object Pattern, or perhaps even a simple dictionary or set of read-only fields. Tools like QTP or Test Studio help you by creating either a database (QTP) or element repository (Test Studio) which centralizes locator storage and maintenance. With a centralized locator storage you only have to go to one place to update your locators. Every test refers back to that repository to actually read the locator’s definition. It’s a thing of beauty.
2. Prefer good locator types where possible. Locators are generally based on HTML attributes, CSS, or the Document Object Model (DOM) structure. Not every tool or framework will support every locator type. More on a few of the most common locator types:
   • HTML ID values are the number one, mostest bestest way to specify your locators. IDs are unique on the page, so you’re never ambiguous. IDs don’t change based on their position on the page, so if a field is moved elsewhere on the page your tests should work fine.  IDs are also the fastest to resolve.
   • Text or link. Some frameworks will let you locate an element/target by its link text or text in the field. This is handy; however, may not be unique on the page.
   • CSS selectors are things like CSS class names. These are fairly fast to resolve, but you’re not guaranteed uniqueness on the page, so they may not work for you.
   • XPath. XPath works with the DOM structure based on paths and functions. I have a love/hate relationship with XPath. The geeky part of me that loves regular expressions (I’m taking counseling for it) also loves XPath. I can solve some very tricky problems with XPath, which is neat; however, XPath is hard to grok, and it’s extraordinarily brittle. If anything moves the slightest bit in the page’s DOM then your XPath is totally borked. Moreover, XPath is extremely slow in Internet Explorer. Finally, XPath under the .NET framework support isn’t fully implemented, so XPaths which work fine in tools like the XPather for Firefox may not work in testing tools built on the .NET Framework.

Wrapping Up

I’ll be doing some follow on posts introducing you to functional test automation; however, past experience has taught me that you’ll see the most success when you keep your automation suite’s maintainability in mind from the start. Gee, isn’t that sort of the same approach great software developers use? Maybe you should treat your testing codebase like production code—because it is production code!

I hope you’re enjoying this series so far and are finding it useful. I’m enjoying writing it!

31 Days of Testing—Day 10: Mocking Out Dependencies

Updated: Index to all posts in this series is here!

Frankly, I’ve dreaded writing this post. Mocking is something that’s difficult to clearly get across, and the approaches and tooling for mocking vary perhaps even more widely than those for testing frameworks. There’s also an extraordinary range of terms people use to describe subtly different aspects of mocking—because we software folks don’t have enough confusing terminology in our lives already.

While I’m being specific with a mocking implementation in this post, it’s important that you keep focused at the higher level of the concepts I’m trying to get across. Look to the broad concepts and go do more exploration in the mocking tools and approaches specific to your platform.

Enough disclaimers. Let’s roll.

Why Mock?

In testing, mocking is a technique to let you cheat your system to isolate away dependencies external to the area of code you're trying to test. This is particularly important in unit testing where you want to focus on one small portion of your system—and you can’t do that if you’re having to deal with calls to external services, databases, security providers, etc.

Let’s have a look at one example. Below is some code for updating an employee’s information. This sort of action can be quite sensitive, especially since there is privacy and salary information involved. Ergo, you want to ensure there’s very solid security around this use case. My example below uses a security provider service which checks the current user and sees whether that user is permitted to do the specific actions on the target employee.

public class EmployeeUpdater

{

    private readonly IUpdateEmployeePermissible _updateEmployeeSecurityProvider;

    public EmployeeUpdater(

        IUpdateEmployeePermissible updateEmployeeSecurityProvider)

    {

        _updateEmployeeSecurityProvider = updateEmployeeSecurityProvider;

    }

    public Employee UpdateEmployeeRate(Employee targetEmployee, float newRate)

    {

        Employee updatedEmployee;

        if (_updateEmployeeSecurityProvider.CanUpdateEmployeeRate(targetEmployee.EmployeeId))

        {

            updatedEmployee = new Employee(targetEmployee.EmployeeId, targetEmployee.FirstName

                                           targetEmployee.LastName,

                                           targetEmployee.Address, newRate);

        }

        else

        {

            throw new SecurityException(

                "Invoking user doesn't have permissions to update target target employee. "

                + targetEmployee);

        }

        return updatedEmployee;

    }

}

I’m leaving out the implementation details for the security provider in an attempt to keep this as readable as possible. What’s important is an external (to this class) service figures out whether or not the current user can modify the target employee.

Design Impacts

Depending on your tool and platform, mocking may impact how you design your system. Note the security provider is passed in to the EmployeeUpdater’s constructor. Injecting dependencies this way, or through some form of setter, is an extremely preferred design approach for many reasons.

First, the EmployeeUpdater shouldn’t have to know how to create anything. Single Responsibility Principle says this class should know how to do its job really well, and not worry about other things. If the manner of creating a new security provider changed, then we'd have to modify the EmployeeUpdater, too. That’s nuts. Let things get built elsewhere and simply passed in to objects that need them.

Secondly, injecting the dependency to the class gives us a seam where we can substitute the real item with a fake one. We can use mocking tools, or just flat code, to create that faked out item and pass it to the portion of the system we’re trying to test.

One more consideration for mocking which can impact your design: different mocking toolsets on different platforms have limitations around what sorts of objects/classes they’re able to work with. In the .NET world, for example, many mocking tools are unable to deal with static or concrete classes. You’ll need to provide interfaces or abstract implementations for the classes you’ll want to mock out. Some tools will deal with static classes, some will deal with concretes, some won’t at all, and then some platforms lend themselves to completely different approaches.

Simple Stub Mocks

There are a number of different types of mocked objects. Mocked objects may have some behavior defined in them, or they may just be empty shells which do nothing more than satisfy your test’s need to have something in the “shape” of the real object.

In my examples below I’m using RhinoMocks for .NET. RhinoMocks uses the term “stub” to refer to a mock with no behavior, and “mock” for a faked object which has some behavior defined for it. The first test I’ll show you simply checks that an Employee object is returned with the correct updated pay rate. I don’t want to deal with the external security provider on this; the test is solely focused on whether or not the update happens properly. Ergo, I want to decouple the system and ignore the real security provider—I just want that to tell the system, “Yes, the current user is authorized.”

[Test]

public void Updated_employee_has_correct_rate()

{

    int userId = 1;

    int oldRate = 5;

    int newRate = 10;

    var stubProvider = MockRepository.GenerateStub<IUpdateEmployeePermissible>();

    stubProvider.Stub(x => x.CanUpdateEmployeeRate(userId)).Return(true);

    Employee current = new Employee(userId, "Jim", "Holmes", "Doghouse", oldRate);

    EmployeeUpdater updater = new EmployeeUpdater(stubProvider);

    Employee updated = updater.UpdateEmployeeRate(current, newRate);

    Assert.AreEqual(updated.HourlyRate, newRate);

}

The first statement in this test creates a stub of the security provider as defined by the interface IUpdateEMployeePermissible. (Dear .NET folks: can we please lose the last vestige of the silly Hungarian notation and just drop the “I” from interface names?).  The second statement defines an implementation that says “Whenever your method CanUpdateEmployeeRate is called, return true.”

Next we create an Employee object as test data and an EmployeeUpdater to actually test. Note that we pass in our stubbed out provider to the constructor. The EmployeeUpdater will now use the fake provider we created earlier.

The rest of the test is straight forward: invoke the method, check the results.

Because we’re using a stub, the fake provider won’t even care if we invoke CanUpdateEmployeeRate with a bogus ID. It will simply return true regardless of when or how it’s invoked.

Mocks With Behaviors

Now for the next test. We want to ensure that our security provider’s CanUpdateEmployeeRate is always invoked—this is a critical part of our overall business rules to ensure proper security for updating employees, after all.

What would happen if someone was working on the CanUpdateEmployeeRate method and commented out the security check for some troubleshooting, then mistakenly checked that in to source? You’d completely bypass the security check in production. Ick. Like that sort of thing never happens…

Mocking frameworks generally provide you ways to deal with this sort of interaction. In RhinoMocks you can use a “mock” to handle this.

[Test]

public void Verify_security_provider_is_called()

{

    int userId = 1;

    int oldRate = 5;

    int newRate = 10;

    var mockProvider = MockRepository.GenerateMock<IUpdateEmployeePermissible>();

    mockProvider.Expect(x => x.CanUpdateEmployeeRate(userId)).Return(true);

    Employee current = new Employee(userId, "Jim", "Holmes", "Doghouse", oldRate);

    EmployeeUpdater updater = new EmployeeUpdater(mockProvider);

    Employee updated = updater.UpdateEmployeeRate(current, newRate);

    mockProvider.VerifyAllExpectations();

}

Several major differences here: First, we’re generating a Mock, not a Stub in the first real statement. Secondly, we’re calling mockProvider.Expect instead of .Stub. The Expect call sets an expectation that CanUpdateEmployeeRate will be called once with the specific value set in userId. The next several lines are all the same; however, the final statement in this method isn’t the usual Assert we associate with tests – it’s a call to the mocked out security provider’s VerifyAllExpectations method.

This final statement asks the mocked object to check whether or not the system under test did the things we expected it to. In this case, we expected the system under test to invoke the CanUpdateEmployeeRate method with a value of 1.

The verification will fail if the CanUpdateEmployeeRate wasn’t invoked at all, or if it was invoked with a value other than 1.

Mocking frameworks can provide a wide range of expectation checks. You can look to ordered expectations (method A called before method D), constraints (input value between x and z), and many other validations.

Tools for Creating Mocks

You might start to see that creating a series of mocks could be quite complex. For example, if my security provider depended on something else which in turn depended on something else, then I’d have a nasty tree of objects I needed to deal with. Tools exist in every platform to help ease this. Factory Girl in Ruby is one, StructureMap and Ninject in the .NET world are others.

Look up these approaches to help you ease the burden of dealing with complex object graphs both in your testing and production worlds.

Wrapping Up

This is an extremely simplistic, high-level view of mocking. My goal was to simply show where mocks might be used, and expose you to a few of the fundamental concepts around them. Mocks are an extremely powerful tool, but it’s extremely easy to use them too much, or in too confusing a manner. It’s also easy to end up testing the mock you just created instead of the system itself!

Don’t stop with this post. Go read more around mocks and figure out how they can help you better test your systems!

31 Days of Testing—Day 9: Readable Tests

Updated: Index to all posts in this series is here!

This post is an off-the-cuff tangential post based on some interesting comments to my previous posts on test fundamentals.

A couple folks commented on my use of C#’s “var” keyword, pointing out that it made the tests less readable in their view. I think it raises an interesting point of discussion that is an extremely large debate on the value of strict typing. I’m going to forgo that conversation, and instead try to focus down on readability of tests.

In the following test, what’s most important?

[Test]

public void Computing_with_40_hours_at_rate_5_returns_200()

{

    WageComputer computer;

    bool isHourlyWorker = true;

    var computedWages = computer.ComputeWages(40, 5, isHourlyWorker);

    Assert.AreEqual(200, computedWages);

}

This test is validating that the ComputeWages method is correctly working and returning the correct values for my inputs. This test isn’t looking to validate the return type. Frankly, I don’t care what the return type is. I’m more concerned that the algorithm is functioning properly.

If I’m only concerned about the value coming out of the method, then, quite frankly, anything extra at all on the left-hand side of that statement is pure noise to me. Look at these two different examples:

double computedWages = computer.ComputeWages(40, 5, isHourlyWorker);

var computedWages = computer.ComputeWages(40, 5, isHourlyWorker);

That’s how my mind looks at these sorts of things. Frankly, I’d be happier if the compiler just got rid of the entire type declaration. The compiler knows what the right-hand side is returning. Even “var” is just noise to me. Something like this would be a thing of beauty:

computedWages = computer.ComputeWages(40, 5, isHourlyWorker);

(By the way, that’s not my idea. My pal Leon Gersing put this idea in front of me during an interesting discussion several years back at a .NET training day we put on.)

Moreover, explicitly using strong typing makes your tests more brittle. If you ever change the return type from ComputeWages you’ll need to go back and update every test using that class. (In this case that’s a somewhat contrived example. Please look at the larger point here.)

If you feel the need to have a test specifically checking the return type, then write a separate test for that.

Carry this over to everything else about your tests. What’s noise? What’s really important? Can you easily grok a test you wrote three weeks ago? Three months ago?

No? Why not?

Great software engineers, or the latest trend of labeling folks who care as “craftsmen,” understand the criticality of writing code that’s easily readable.  Tests need to have the same care, because tests are production code.

Take a close look at what you really for your test and cut out everything else. It’s just noise.

Updated: By the way, in case you get put off my my frequently curmudgeonly sounding tone, I really do love this sort of feedback and questioning on my posts. I do not have all the answers. Your questions and points make me constantly re-evaluate how I’m doing things, and that’s a good thing! Please, keep up comments and questions. I love ‘em!