Sunday, February 07, 2016

Demonstrating Risk, Stability, and Testing

Learning Through Games

Many coaches use the Jenga game to help illustrate the importance of testing early and often. It’s a fun activity that demonstrates the concepts in a light-hearted but eminently clear way.

I’ve wanted a similar approach to help drive home the idea of stability, risk, and how testing helps mitigate those issues. I also wanted a clear expression of the always-present constraint of time. Moreover, I want to explicitly demonstrate that stable components don’t need anywhere near the amount of testing as less stable components.

I came up with the following approach several months ago, and it’s been very effective the three or four times I’ve run it so far.


The general idea is to build a structure of a specific minimum height in a constrained amount of time. Attendees are given different materials each iteration with the same height and time constraints. The different materials inject stability issues, making it harder to meet the minimum height requirement within the time constraint. This tends to be a lot of fun, because as with the Jenga game there can be a lot of crashing as structures fall apart.

I’ll start the exercise talking about value and risk—two things I continually hammer home during my coaching sessions—and talk about how this exercise is a tradeoff of value and risk. I don’t specifically call out stability during my explanations. I like that come to the forefront as the game unfolds. Your mileage may vary.

Rock, Dice, Marbles Game

(Apologies to Rock, Scissors, Paper.)

The Goal

Illustrate the interaction of risk, stability, value, and how testing comes in to play helping stabilize things and mitigate risk.

Materials required:

  • One set of dice per team – nine to 12 dice
  • One small rock per team. Rock should be somewhat die-sized and roughly square. You don’t want them too flat—part of the point is the rocks being unstable…
  • Two marbles per team
  • One small blob of putty, PlayDo, or otherwise squishy stuff. This blob should be fairly small. It’s meant to illustrate testing, a constrained resource.
    Dice, clay, rocks, and marbles.

  • Break up your group in to teams. Keep teams under six if possible. Three to five per team seems to work best.

  • Give each team one set of dice. Keep the other materials hidden.

Iteration One

Rules for I1:

  • Must build a tower at least eight dice high. I’ll also often use the sides of my Moleskein as a measure instead of just saying “Eight dice” or whatever.
  • Time limit of 90 seconds

Running I1
Explain the rules, then start the timer and have teams build their towers.
When the timer finishes you’ll have some successful teams and some who had trouble with falling towers.

This is the first place to start discussing the main point: stability. The structure’s a bit wobbly, even when using dice. Sometimes teams will use interesting approaches to building. Use opportunities as they arise to discuss how different approaches help mitigate instability, risk, etc.

Iteration 2

Distribute a rock to each team. This is generally met with some groans and pained laughter. Play that up! The game’s meant to be fun in addition to educational!

Rules for I2
* Must start tower from scratch. No building on top of existing ones!
* Must build a tower the same height as I1
* The rock must be used as part of the tower. It must contribute to the tower’s height, meaning you can’t just set it off to the side of the tower. It has to have something on it, or be on top of something else.

(I’ve had some creative attempts to get around the intent of this. Testers. Go figure. Again, use this as an opportunity to inject some humor as you describe the rule and ask the teams to honor the intent of it!)

Running I2
Same as before: Start from scratch, 90 seconds, fire the teams off and see what happens!

More instability, this time in a big way. I’ve had some teams build smaller towers that included the rock. This is actually a great discussion point: those teams shipped some value! That’s not a bad thing—it offers a great opportunity to discuss value versus risk trade offs!

Look at people who succeeded in any fashion and talk about how they managed their risk/stability.

Now carry the discussions over to software delivery. Make a connection that the stability in this game is a metaphor for different components/systems we work with.

Iteration 3

Pass out two marbles to each team. Yet another opportunity for injecting some serious humor. “Oh no! You’re giving us another rock???” “No! I’d never do anything so mean! Here, have marbles.”

Rules for I3
Same height and time constraints. Start from scratch. Rock and one marble must add to the height of the tower.

One critical modification here: The stakeholder must have a tower at the minimum height. A shorter tower will not deliver him the business value he needs.

Running I3
Same as before. There should be a nice amount of crashes, laughter, and muttering.

Huge stability issues. Connect back to software: stable components and systems. Less stable or newer components which haven’t been used for as long. Brand new systems, or ones written very, very badly.

Iteration 4

Now distribute out the small ball of clay/putty. Talk about stabilizing as you’re doing it. Don’t tell teams how to use the clay.

Rules for I4

Start from scratch, same height, 90 seconds. Rocks and marbles must contribute to height.

Running I4
Same as before. Start it off, let it roll.

Talk over how teams used clay for stability. I’ve seen a wide range of uses: dots between dice, nothing on dice, big blobs on the marbles, etc. One team made a large pancake of clay and wrapped their rock and die in it—what a great metaphor to tie directly to software wrappers around unstable components!

If the teams haven’t made the connection of clay == testing, help guide them along that. Now’s the time to really head off down the road of “You have 35 - 40 hours a week to apply all your testing expertise. Where are you and your team going to focus? What’s the scary unstable stuff? Where’s the high-risk stuff?”

This is also the spot in this exercise where I really hit home on careful thinking around test coverage, regression, and dear God PLEASE STOP TESTING ALL THE THIRD PARTY TOOLS YOU HAVEN’T CUSTOMIZED.

Finishing Up

You should definitely poll your attendees to see what they got out of this. I always get some neat feedback, and it’s important to validate the attendees got some glimpse of the goal of the exercise.

Note on Licensing of this Exercise

I invented it, I’m licensing it.

Here’s my terms: Use this as much as you want, whenever you want, wherever you want. Pass the concept on to others as often as you want. The sole term in this license is if we are ever in the same place at the same time you must buy me a [drink] and share your experience with the game I mean exercise. Let me know if it worked, and how you’ve changed it.

Awful terms. I get it.

[drink]: Coffee, soda, or beer. Scotch is best; however, I’m happy with anything as long as you share your experience!

Wednesday, December 16, 2015

Give Yourself a Great Christmas Present: A Job With Pillar

Want to start off 2016 with a job that’s challenging, interesting, and fun? Come join me at Pillar Technology!

We’ve got a significant number of job openings across a lot of disciplines. We’re looking for people to fill roles as Delivery Leads, DevOps, Experience Architects (and here at Pillar that is a real role, not some vaporware handwavy marketing thing!), and of course Software Craftsmen!

I joined Pillar last March and it’s been a tremendous time since then. I’ve know the folks at Pillar for years, and I love how they’ve grown in to a focused, passionate, awesome organization. If you looked at Pillar some time ago, I’d encourage you to give us another consideration. This is the best environment I’ve had in my career, and I love the clients I’m working with.

Interested? Drop me a note either at my work address or personal and I’d be happy to chat with you via mail, Skype, phone, or face-to-face. Sorry, no Morse code. I forgot everything after having earned the Signaling merit badge 40 years ago…

Tuesday, December 01, 2015

ISTA Conference Keynote

For those interested, my keynote from ISTA Conference 2015 in Sofia, Bulgaria, is live now on YouTube!

This keynote meant a lot to me. It was my first one, which was nerve-wracking, plus the organizers were kind enough to give me free rein to do a somewhat quirky format. I wanted badly to do a shorter keynote, and specifically stay away from the hour-long breakout-session-disguised-as-a-keynote format.

My topic for the talk was "Why?" which is a question we don't ask anywhere near enough. The talk's just under 30 minutes, so it's an easy to watch/listen to chunk.

I hope you'll find it interesting.

Thursday, October 01, 2015

Keynoting at ISTACon in November

I’m honored to announce I’m going to be keynoting at the ISTA Conference in Sofia, Bulgaria 18-19 November! This is exciting for me on a number of levels.

enter image description here

First, it’s my first-ever keynote, which is pretty exciting. I’ve never spoken to more than ~150 attendees, so being up front before 700-800 will be a new experience!

Second, I get to return to Sofia, Bulgaria, which is one of my favorite cities in the world. I had extraordinary experiences there when I worked for Telerik, and I’m looking forward to being back there.

ISTA’s organizers were kind enough to let me go a bit astray from “normal” keynotes. Instead of an hour talk I’ll be doing a much shorter session on the lines of a KalamazooX talk. My topic is “Why?” and I’m hoping to fire up attendees to ask “Why?” more often as we work to deliver value to our customers.

I’ll also be presenting my Leadership 101 workshop. This workshop’s from the heart, so it’s always a rewarding but exhausting experience.

ISTA’s registration is already open. I hope folks in the region will be interested enough to attend—it’s a great conference with a lot to offer! If you can’t attend please at least share the info to your friends, colleagues, and community members.

Please look me up onsite if you’re going to attend!

Thursday, September 03, 2015

Don't Ignore Big Deltas in Your Teams' Estimates

Estimates suck. They’re always wrong, and of course estimates somehow end up getting treated as exactimates, or worse yet, commitments.

enter image description here

I’m not here to tell you how to do estimates, or how to follow the #NoEstimates movement and not do estimates at all.

What I am here to do (well, in this article at least), is tell you to pay attention when you have wide estimate ranges within your team. It’s a situation that isn’t just an indicator of a skills gap. It’s a warning flag that maybe you’re missing something. (It’s also a great learning opportunity!) Regardless whether the range in estimates comes from junior to senior, or between similarly experienced folks, take some time to dive down into a conversation about the delta.

What’s causing the big disconnect? Is it indeed a skills/experience gap? Do the junior team members misunderstand the complexities involved? Take a few minutes and have someone more senior walk them through things—and make sure they get an understanding of what’s involved in this particular work item.

This quick learning moment doesn’t have to extend out to a huge training/knowledge sharing session. Keep it tight and focused; after all, the primary purpose is to get the estimating done. If needed, take the conversation offline so it doesn’t derail whatever meeting you’re in at the moment.

An even more interesting situation is when you have people with similar experience differing widely on work. Don’t simply take averages of the estimates and move on. Take time to explore what’s the rationale behind the extreme delta. In this case, you do want to spend as much time’s needed to clarify the delta.

Is the delta due to a lack of clarity around the work item? Spend more time getting the details around acceptance criteria, documentation, whatever’s necessary to ensure everyone’s on the same page.

Is the delta due to a disconnect in complexity of the work? If needed, whiteboard out workflows, data structures, dependencies, etc. until everyone’s got roughly the same idea of complexity involved.

Is the delta due to misunderstanding of the amount of testing involved? If so, move along, BECAUSE THAT NEVER HAPPENS. Hah. Of course it happens! This situation needs time to rectify, because the team has to make sure testing aligns with the information the business/product owner/customer needs to make effective risk/benefit decisions. Are the testers over-focusing on edge cases that don’t directly apply to highest priority risk/values? This can be a regular issue with less mature testers. Is the team missing critical integrations or dependencies that might impact how much testing is really needed? What about coverage matrices for things like browsers, devices, operating systems, etc.?

Work through the testing requirements, but ensure you’re focusing on what the business needs from you. Remember: The testers don’t assure quality, nor does the delivery team, frankly. The business/product owner/customer is the only one who can do that. The team is responsible to pass along the right information to help those people make informed decisions.

Got big estimate deltas? Make sure you’re using that as a trigger for further discussion!

How have you approached handling big deltas in your teams’ estimates? Let me know in the comments!

Wednesday, August 26, 2015

Sorry, I Can't Talk to You This Iteration

No, seriously, I actually heard that on a project once upon a time.

The client’s team had gotten far too focused on capacity and lost sight of delivery. Management wanted more stuff faster, so they focused on full utilization and commitment. As a result they were in a place where time for collaboration, knowledge transfer, and you know, teamwork had to be scheduled. Management had clearly lost sight that 100% utilization means no slack for conversations. They’d missed the problem that 100% utilization turns freeways—and software teams—into parking lots.

Intended or not, teams and individuals on the project ran with that mindset to an extreme. Collaboration and cross-team assistance fell off drastically, as did the sense of a shared goal: working software delivered to the customers.

Such an emphasis on capacity and commitment isn’t healthy. I’ll go so far to say it’s a cancer. Regardless whether you’re “agile” (whatever that means), waterfall, or something else, you must leave slack time in your schedule. Without slack you’ll never get your teams working on the most important aspect of any software methodology: continual improvement and adaptation.

Healing up this mindset and culture isn’t easy. It’s an issue with management’s pressure and unreasonable expectations. That can be a result of a lack of skills with the delivery teams’ ability to ship; however, honest change needs time for that change.

Generally management is open to discussions around impacts to delivery or business value. Remember, misguided as the decisions might be, they’re an attempt to meet deliverables and ship software. Frame your case in that context and you might have some luck.

Start gathering hard data around situations like:

  1. Work items blocked by inability to get time for discussions. Can’t get time with Team <X> to discuss the API work item you’re on right now? Note that and consolidate with other data.
  2. Work items slipping past the iteration. Are those blockages from #0 bad enough that you can’t finish your work? Make sure the reason behind it is clear to people up the chain.
  3. Lack of knowledge transfer. Shared codebases are critical, even on large projects. Can’t get time to understand critical aspects of what you’re working on? Document, escalate.

Make sure you’re approaching this from a positive, not negative, direction. Don’t even get out of bed if you’re planning on a “This is stupid and everyone sucks!” pitch. Instead, frame it as “Look, this is costing us <X> hours, <Y> blocked user stories, and <Z> <fill_in_blank/>.”

Additionally, come to the table with a suggested approach for fixing it. “Look, can we reserve some slack time each iteration for more ad-hoc discussions? Having some honest time for collaboration will help us prevent the <X> hours lost, <Y> blocked stories, and <Y><fill_in_blank/> we’ve seen the last couple iterations.”

Every team’s primary focus should be working software in production. Blocks to effective communication and collaboration are a direct impediment to that. Work to cure that cancer!

Tuesday, May 05, 2015

So Long Heartland

Back in the summer of 2001 my family (just three, back then) moved to Beavercreek, Ohio. For those of you who are, like me, math-challenged that’s 14 years. We’d moved here for my wife’s final assignment during her 20 years of service in the USAF.

At the time I was working as a customer relations manager for a small Department of Defense contactor engaged on a long-running program run out of Wright Patterson AFB. I was handling customer conversations and some project management, plus I was doing “testing” for the system we were developing.

enter image description here

Over those 14 years my family grew to four, we put 45 cubic yards of dirt in for flower beds, well over 100 cubic yards of mulch, we added on an 800 ft2 addition for my Mother-in-Law (hold the jokes. She’s awesome.), 1500 ft2 of laminate flooring, and 80 gallons of paint. Unlike other situations, those are accurate figures, not me exaggerating!

Also along the way I managed to hook up with the amazing Heartland’s amazing developer community. Yes, I know, I wrote amazing twice. It’s that amazing.

I was lucky enough to get engaged with the community here at a really interesting timeframe. In 2005 there weren’t any conferences being in the Heartland other than the occasional commercial thing like No Fluff Just Stuff. What was here was a nascent, primordial bubbling goo formed by some really smart, motivated folks who wanted to reach out to other like-minded folks.

I was fortunate to hook up with people like Drew Robbins, James Avery, Brian Prince, Josh Holmes, Mike Wood, and others who were starting to get various conferences off the ground. (I’m missing lots of people off that list. Sorry.) That group exploded out and created an incredible number of community events.

By 2008 you could, literally, hit a one-day conference at least once a month if you wanted to. Days of .NET, Code Camps, Give Camps, Days of Agile, and others had sprung up from Grand Rapids down through Nashville. Larger conferences like DevLink, StirTrek, KalamazooX, and of course CodeMash took off over the following years. User groups were springing up all over the place.

Fast forward to 2015: the pace of Code Camps, Days of .NET, etc. has died down a bit, but larger regional conferences like StirTrek, CodeMash, etc. have exploded. User groups have gone the way of tribbles and have spun off into an insane amount of local meetups, user groups, hangouts, whatever. I stopped trying to keep track years ago.

Heartland’s community is full of smart, passionate, welcoming folks who opened their arms, brains, and hearts to those interested in improving their various crafts, skillsets, and general outlook. These folks helped get me motivated to get out of the soul-crushing work I was involved with at the time. I trace a line from the awesome job I’m currently doing directly back to influences, motivations, and friendships I developed amongst Heartlanders over the years.

I can also point to at least three jobs I got directly because of my involvement with the Heartland community. I once got a job offer from a Heartland homie three or four days after I reached out. When I hung my shingle out earlier this year I had seven leads on independent contracts, five writing leads, and four job interviews—within four days.

Folks in this region are simply amazing. I’ve been blessed to have spent so many years here.

With all that said, it’s time for me and my family to bid adieu to the Heartland. My wife and I’ve always known Dayton wasn’t even close to our “forever home.” No mountains and too far from my family.

On 9 May my son, brother-in-law, and I load up a 27’ UHaul truck and minivan and head west for Ashland, Oregon. With two cats and two dogs in the minivan. Please send your prayers for what little is left of my sanity. The rest of my family will follow in stages, but we’ll be settled in our new home in early June.

I can’t list out all the people I really need to thank. There are hundreds of you, maybe even thousands. Literally. Those of you who’ve suffered through assorted tech crashes during my talks, those of you who’ve patiently listened to various rants, those of you who’ve made CodeMash so awesome, those of you who’ve lent me energy and strength at various dark phases of my life.

If you’re ever out in southern Oregon, please drop me a note either via mail ( or Twitter (@aJimHolmes). I’d love to meet up for a coffee, craft beer, or good glass of PNW vino.

To all I’ve met over the years: Thank you all. You’ve given me far more than I gave.

Subscribe (RSS)

The Leadership Journey