Wednesday, April 29, 2009

Challenge to MS Evangelists: Real Examples. Tests, Even

For some time I’ve been giving Jeff Blankenburg, our region’s Microsoft Developer Evangelist, a good-natured ribbing over the lack of tests or real-world architectures or design in his demos. This isn’t just a problem I have with Jeff, it’s a problem I have with our community of speakers in general, both inside and outside Microsoft.

Here’s the gist of my gripe: Our industry’s in an awful place right now regarding how we use the tools/platforms/whatever that Microsoft delivers. Don’t believe me? Brush up on software design patterns and practices (read Uncle Bob’s Clean Code or Bill Wagner’s Effective C# or McConnell’s Code Complete), then take another look at how you write your own code. Go look at various open source projects. Look at the examples and guidance that come out of Microsoft. Rethink how the last 100-level “Intro to <X>” session you saw was presented.

Take a look at a number of those and maybe you’ll see my point: We’ve got way too many examples and demos from way too many smart folks and groups which show zero tests or awful implementation practices. The 100-level introductory presentations we see at way too many user groups and code camps do nothing to help us software engineers decide HOW we should implement something.

Changing this situation requires a concerted, long-term effort on many parts:

  • User group leaders and conference organizers should start demanding this of folks presenting at their events
  • Speakers need to step up to the plate and speak to the real world sharp edges of implementing the latest, greatest tool or technology
  • Attendees at these events need to lose their apathy and get engaged. Demand better from speakers: “How would you test that? How does that fit in a real implementation? What lessons have you learned from solving problems with this?”

I understand the perceived difficulty of trying to get a topic introduced in a clear, understandable fashion while trying to show that topic’s real world implementation. Too bad. Step up to the plate and re-think how you’re doing your presentations. Scale back scope. Rethink how you demo things. This issue can and must be solved.

As part of this, I want to offer up a challenge to Microsoft evangelists anywhere, not just Heartland, who are presenting to companies, user groups, or events: step up to the plate. Speak to how the tools, frameworks, or whatever need to fit into the real world. Show tests. Show the hard edges. Encourage your audiences to use critical thought in fitting these technologies into their environments.

Why am I singling out MS Evangelists? Simple. Scalability. MS Evangelists speak to a huge number of folks in any given month. That’s their job.  I do four, maybe six presentations in a year (mostly because I suck, but that’s beside the point). Jeff Blankenburg  did seven gigs last month and 20 over the previous three.

Like it or not, what comes out of Redmond is too often taken as gospel. Bad examples or demos which show zero tests or have everything including data access wrapped up in a code behind are far too prevalent.  This isn’t anywhere near Microsoft’s fault – folks who blindly take those examples as gospel are failing to use any critical thought.  I’ve had long discussions with my pal Leon about this, and we even drew poor Mike Wood into the discussion when he was trapped in a car with me and Leon for nine or ten hours on the way to and from the Kalamazoo X conference last weekend.

Here’s the meat and potatoes of my challenge: I’m willing to step up to the plate with something on my side: Cold, hard cash. During the months of May, June, and July, I’m offering to donate my entire monthly discretionary spending budget of $200 to charities of your choice if you’ll meet my challenge.  Here are the rules:

  • Present at an event, company, or user group
  • Show a demo which includes unit, integration, or feature tests
  • Show a demo where you lay out real-world implementations with a solid architecture
  • Discuss pros and cons of the technology you’re presenting on

Do that, then mail me (contact info on right sidebar) a summary of your talk.  I’ll write out a check for $20 to your favorite charity.  I’ll do that each time until I’ve spent all my discretionary allowance for that month. That’s ALL my blow money for the next three months I’m putting on the line. No saving for the new home server I need. No beers after my .NET user group meetings. No purchases of MP3s from No purchasing coffee beans to roast once I run through my remaining stash of unroasted beans.

Terms and conditions: Don’t be a nitpicker. Assert.IsTrue(true); isn’t the kind of test I’m talking about!  Make a solid effort to at least show some good tests. I don’t expect perfection, but I do expect a good shot.  No more than three contributions per evangelist per month.

Don’t get started on how this is only one tiny piece of the puzzle, or how we shouldn’t rely on Microsoft to do all our thinking for us, or … <insert_fave_argument_here>. I’m all over all those issues.  However, changing our .NET industry will take time. This is just one small kick in the rear to try and get that change happening a bit faster.

Wednesday, April 22, 2009

Nice Acceptance Test Framework: Cucumber

I’ve spent some time over the last month fooling around with using Cucumber to write some acceptance tests for bits and pieces of the next version of Community Server. Cucumber’s great advantage is its lovely expressive grammar – a clear grammar for testing is critical to my view because it helps ensure developers, analysts, and the customer are all on the same page with how the system should behave.

For instance, we need to ensure a role has the appropriate permissions set for it. In this case, the Site Members role, by default, has 25 separate permissions assigned to it:

I could write up a Selenium test to navigate through all this and do individual assertElementPresent or some variant on each one of those, but that would be time consuming and frankly, the test report wouldn’t be all that clear, even if I used good names for the Selenium test suite and individual cases like I do for other tests:

Flip this around. Imagine that the person owning the feature can write a simple text file with contents like this:

Feature: Default Member Permissions in a Public Open Group
  In order to ensure the system is properly secured
  As a site administrator
  I want to check that roles are assigned 
    the correct default permissions for Public Open Groups
  Scenario: Check default permissions for the Members role
    Given I created the PermTest group as a Public Open group
    When I examine the Members role for the PermTest group
    Then the total number of Granted Permissions should be 25
    Then Granted Permissions should include Blog - Create Comments
    And Granted Permissions should include Blog - Read Posts
    And Granted Permissions should include Forum - Attach Files Locally
    And Granted Permissions should include Forum - Attach Files Remotely
    And Granted Permissions should include Forum - Create New Threads
    And Granted Permissions should include Forum - Post Replies

That snippet clearly states the goal of the Feature, plus it gives an exact scenario to test with a specific Given/When/Then set of conditions.

I can use Cucumber to run that clear language snippet through a framework and run tests against each of the “Then” conditions laid out above. (Code below) I’ll get a nice output with familiar colors (red, green) for the state of my conditions:

To me, this is just huge. I’ve got a specification written by someone who understands WHAT the system is supposed to do and HOW it’s supposed to behave. I can do a number of very cool things with that spec. First off, it can and should live in the source control repository where updates can be versioned. Secondly, I might be able to link to that item in source control from whatever project management tool I’m using – why duplicate that text in a project system when I can pull it straight in from Subversion, Git, etc. ? Finally, that expressive language is a thing of beauty for crossing the communication/understanding barriers between devs and feature owners/stakeholders.

My level of excitement around this is very, very high, in case you haven’t figured that out already.

Here’s the details of the implementation behind that spec above.  First off, you obviously need the Cucumber framework. Secondly, you’ll need Ruby.  There’s some C# examples in Cucumber, but I’ve not fooled with C# yet.

The spec above is written in plain text and saved as “MemberInAPublicOpenGroup.feature” in a unique folder.  We’re going to use Watir to handle this test, so in a folder called “support” we’ll need to have an “env.rb” file which is responsible for firing up our Watir IE instance and doing a bit of work:

require 'spec/expectations'
#require 'firewatir'
#Browser = FireWatir::Firefox
require 'watir'
Browser = Watir::IE
# "before all"
browser =
Before do
  @browser = browser
  @browser.speed=:zippy #for Watir with IE only. Also try :fast
# "after all"
After do, "Log Out").click

Note that I’ve got two lines referencing Firewatir commented out. I’m able to easily switch back and forth between browsers without altering my test code!

Cucumber expects the actual implementation code to follow that convention: step_definitions\<FeatureName>_steps.rb. We’ll write the actual test code in a file called “MemberInAPublicGroup_steps.rb.”

require 'spec/expectations'
Given /I created the PermTest group as a Public Open group/ do
    @browser.goto "http://localhost/csfunctionaltests", "Sign in").click
    @browser.text_field(:id, "ctl00_bcr_ctl03_ctl07_username").set("admin")
    @browser.text_field(:id, "ctl00_bcr_ctl03_ctl07_password").set("pa$$word")
    @browser.button(:id, "ctl00_bcr_ctl03_ctl07_loginButton").click, "Control Panel").click, "Group Administration").click    
When 'I examine the (.*) role for the PermTest group' do |roleName|, "Dashboard").click, "Group Administration").click, "//div[@id='Common']/div[3]/div/table/tbody/tr/td[1]/div/table/tbody/tr[2]/td/strong/a").click
    @browser.span(:xpath, "//span[text()='PermTest']").click
    @browser.button(:xpath, "//input[@value='Edit']").click, "Permissions").click
    @browser.select_list(:id, "ctl00_ctl00_ctl00_OuterTaskRegion_TaskRegion_TaskRegion_TaskRegion_ctl02_ctl02_GroupRoles_RoleList").select(roleName)
Then /^the total number of Granted Permissions should be (\d+)$/ do |numGrantedPermissions|
    selectListItems = @browser.select_list(:id, "ctl00_ctl00_ctl00_OuterTaskRegion_TaskRegion_TaskRegion_TaskRegion_ctl02_ctl02_GroupRoles_listGrantedPermissions").getAllContents.length
    selectListItems.should == numGrantedPermissions.to_i
Then /^Granted Permissions should include (.*)$/ do |permissionName|
    @browser.element_by_xpath("//select[@id='ctl00_ctl00_ctl00_OuterTaskRegion_TaskRegion_TaskRegion_TaskRegion_ctl02_ctl02_GroupRoles_listGrantedPermissions']/option[text()='" +permissionName+ "']").should_not be_nil

The main parts here are the blocks for Given, When, and Then. Cucumber looks for those blocks, then grabs the regular expressions after those keywords. Matching blocks are executed and asserts/expectations are checked. Passing expectations means the item is written back to the screen in green; failures in red.

Keep something in mind here: This particular _step and feature is using Watir to test at the UI level. There’s no reason whatsoever that you couldn’t use this tool for testing at the integration or even unit level. Joe O’Brien, wicked smart guy driving EdgeCase around, said via Twitter that they’re using Cucumber at all sorts of different levels in their testing.

There are some cons to consider: you can still have an implementation disconnect if the dev writing the _steps file does something wrong. If you’re at the UI level then you’re also hostage to the pains of Selenium, Watir, Firewatir, etc. I continue to run into troubles with timing, but I’m working around those.

One tip: When you’re first starting out, add a line in your scenario which should fail. When you run the feature through Cucumber you ought to see a red line. If not, you’ve goofed up something in your step implementation. I learned this the hard way…

Use something like this

Scenario: Check default permissions for the Members role
  Given I created the PermTest group as a Public Open group
  When I examine the Members role for the PermTest group
  Then the total number of Granted Permissions should be 25
  Then Granted Permissions should include Blog - Create Comments
  And Granted Permissions should include Blog - Read Posts
  And Granted Permissions should include THIS SHOULD FAIL

to generate this:

This has saved me some time after some initial self-inflicted wounds.

I’m still evaluating this, and I’ve not wrapped it into any automated builds or CI reporting process. I’ll pass on updates as we continue to explore this.

Monday, April 06, 2009

VS Missing TestDriven.NET’s “Test With” Menu

Problem: The “Test With” menu from TestDriven.NET isn’t showing up in Visual Studio’s context menu when you’re right-clicking in a test file. No debugging, no coverage, no other goodies. Bah Humbug!


  1. Rename "%USERPROFILE%\AppData\Roaming\Microsoft\VisualStudio\9.0\1033\CmdUI.PRF" *.bak
  2. Start Visual Studio 2008
  3. Check the box the the left of TestDriven.Net on the 'Add-in Manager' dialog

Use “VisualStudio\8.0” if you’re using Visual Studio 2005.

Thanks to Jamie Cansdale, TDD.NET’s author, for reaching out to me directly in e-mail with this solution!

Thursday, April 02, 2009

Twitter Poll Results: Favorite Visual Studio Add Ons

Last night I posed the question on Twitter: Aside from ReSharper, what are your favorite add ons for Visual Studio?  Below are the responses – it’s a great list of tools which can really boost your productivity. Note that I asked to exclude ReSharper because I would have gotten a huge number of replies endorsing it!

Thanks to everyone who responded!

  1. Ivan Zhakovchemodax@JimHolmes Ctrl+Enter should help.about 5 hours ago from web in reply to JimHolmes

  2. Richie Hindlerichiehindle@JimHolmes Favourite Visual Studio addin: (partly 'cos it's great, and partly 'cos I wrote it. 8-)about 6 hours ago from web
  3. mgrovesmgroves@JimHolmes GhostDocabout 10 hours ago from PockeTwit in reply to JimHolmes
  4. Dan Hounshelldanhounshell@JimHolmes MRU Cleaner, Start Page Killer, TestDriven.NET, VisualSVNabout 10 hours ago from twhirl in reply to JimHolmes

  5. Rachel AppelRachelAppel@schambers @jimholmes +1 Visual SVN rocks!about 10 hours ago from web in reply to schambers

  6. Steve Hornstevehorn@JimHolmes Copy as HTML for blogging code snippets.about 10 hours ago from TweetDeck in reply to JimHolmes

  7. Sean Chambersschambers@JimHolmes visualsvnabout 10 hours ago from TweetDeck in reply to JimHolmes

  8. George Mauertogakangaroo@JimHolmes CodeRush Refactor Pro TestDriven.NET AnkhSVN but that's boring. Fun one: CR_ClassCleanerabout 10 hours ago from web

  9. Brian GenisioBrianGenisio@JimHolmes TestDriven.NETabout 10 hours ago from Tweetie in reply to JimHolmes

  10. Steve Schoonsteveschoon@JimHolmes CodeRush/Refactorabout 10 hours ago from web in reply to JimHolmes

  11. Steve Wrightunderwhelmed@JimHolmes CodeRush, Resharper Pro, VisualSVN, Copy to HTMLabout 10 hours ago from twhirl in reply to JimHolmes

  12. Steve J WallaceSteveJWallace@JimHolmes Gallio, AnkhSvn, CodeRush are the ones that pop into mind quickest.about 11 hours ago from twhirl

  13. Charlie SearsCharlieSears@JimHolmes VisualSVNabout 11 hours ago from twhirl in reply to JimHolmes
  14. Michael Eatonmjeaton@JimHolmes ViEmuabout 11 hours ago from twhirl

Here’s the list of add ons I’m currently a big fan of:

  • Cool Commands 4.0: It’s undocumented and you have to use a hack to get it in to VS 2008, but it’s a great set of commands to pimp up VS
  • GhostDoc: I work hard to make methods, classes, and parameters extremely clear when I name them. GhostDoc helps me get XML comments for APIs quickly built from those good naming practices.
  • VisualSVN: Just started using it, but I love it.
  • CopyAsHtml: It’s undocumented, and the bits for VS 2008 are hidden away or hacky, but it’s da bomb for pasting formatted code snippets.  Currently I’m using the Code Formatter for WLW for blog snippets. (See this post for great CSS additions.)
  • ReSharper: Can’t emphasize the goodness of this.
  • TestDriven.NET: Ultimate in low-friction test running.
  • CodeRush: I don’t use its templating. What I get from it are terrific visualizations of code, plus IN YOUR FACE! metrics for cyclomatic complexity on every method in the file in front of you. Absolutely something you should have in place if you’re trying to keep your codebase in order.
  • Visual NDepend add-in: Again, a great tool for quickly figuring out where your codebase needs attention.

There’s empirical data that I’m a tool whore, so you can imagine my list of widgets goes on quite a bit longer than this; however, these (at the moment!) are a few of my favorites.

Subscribe (RSS)

The Leadership Journey