Wednesday, April 29, 2009

Challenge to MS Evangelists: Real Examples. Tests, Even

For some time I’ve been giving Jeff Blankenburg, our region’s Microsoft Developer Evangelist, a good-natured ribbing over the lack of tests or real-world architectures or design in his demos. This isn’t just a problem I have with Jeff, it’s a problem I have with our community of speakers in general, both inside and outside Microsoft.

Here’s the gist of my gripe: Our industry’s in an awful place right now regarding how we use the tools/platforms/whatever that Microsoft delivers. Don’t believe me? Brush up on software design patterns and practices (read Uncle Bob’s Clean Code or Bill Wagner’s Effective C# or McConnell’s Code Complete), then take another look at how you write your own code. Go look at various open source projects. Look at the examples and guidance that come out of Microsoft. Rethink how the last 100-level “Intro to <X>” session you saw was presented.

Take a look at a number of those and maybe you’ll see my point: We’ve got way too many examples and demos from way too many smart folks and groups which show zero tests or awful implementation practices. The 100-level introductory presentations we see at way too many user groups and code camps do nothing to help us software engineers decide HOW we should implement something.

Changing this situation requires a concerted, long-term effort on many parts:

  • User group leaders and conference organizers should start demanding this of folks presenting at their events
  • Speakers need to step up to the plate and speak to the real world sharp edges of implementing the latest, greatest tool or technology
  • Attendees at these events need to lose their apathy and get engaged. Demand better from speakers: “How would you test that? How does that fit in a real implementation? What lessons have you learned from solving problems with this?”

I understand the perceived difficulty of trying to get a topic introduced in a clear, understandable fashion while trying to show that topic’s real world implementation. Too bad. Step up to the plate and re-think how you’re doing your presentations. Scale back scope. Rethink how you demo things. This issue can and must be solved.

As part of this, I want to offer up a challenge to Microsoft evangelists anywhere, not just Heartland, who are presenting to companies, user groups, or events: step up to the plate. Speak to how the tools, frameworks, or whatever need to fit into the real world. Show tests. Show the hard edges. Encourage your audiences to use critical thought in fitting these technologies into their environments.

Why am I singling out MS Evangelists? Simple. Scalability. MS Evangelists speak to a huge number of folks in any given month. That’s their job.  I do four, maybe six presentations in a year (mostly because I suck, but that’s beside the point). Jeff Blankenburg  did seven gigs last month and 20 over the previous three.

Like it or not, what comes out of Redmond is too often taken as gospel. Bad examples or demos which show zero tests or have everything including data access wrapped up in a code behind are far too prevalent.  This isn’t anywhere near Microsoft’s fault – folks who blindly take those examples as gospel are failing to use any critical thought.  I’ve had long discussions with my pal Leon about this, and we even drew poor Mike Wood into the discussion when he was trapped in a car with me and Leon for nine or ten hours on the way to and from the Kalamazoo X conference last weekend.

Here’s the meat and potatoes of my challenge: I’m willing to step up to the plate with something on my side: Cold, hard cash. During the months of May, June, and July, I’m offering to donate my entire monthly discretionary spending budget of $200 to charities of your choice if you’ll meet my challenge.  Here are the rules:

  • Present at an event, company, or user group
  • Show a demo which includes unit, integration, or feature tests
  • Show a demo where you lay out real-world implementations with a solid architecture
  • Discuss pros and cons of the technology you’re presenting on

Do that, then mail me (contact info on right sidebar) a summary of your talk.  I’ll write out a check for $20 to your favorite charity.  I’ll do that each time until I’ve spent all my discretionary allowance for that month. That’s ALL my blow money for the next three months I’m putting on the line. No saving for the new home server I need. No beers after my .NET user group meetings. No purchases of MP3s from Amazon.com. No purchasing coffee beans to roast once I run through my remaining stash of unroasted beans.

Terms and conditions: Don’t be a nitpicker. Assert.IsTrue(true); isn’t the kind of test I’m talking about!  Make a solid effort to at least show some good tests. I don’t expect perfection, but I do expect a good shot.  No more than three contributions per evangelist per month.

Don’t get started on how this is only one tiny piece of the puzzle, or how we shouldn’t rely on Microsoft to do all our thinking for us, or … <insert_fave_argument_here>. I’m all over all those issues.  However, changing our .NET industry will take time. This is just one small kick in the rear to try and get that change happening a bit faster.

13 comments:

Jesse Liberty said...

I couldn't disagree more. Sorry.

First, I believe that a good demo teaches one thing at a time, and makes it blindingly clear.

Second, teaching best practices, testing, etc. is teaching a skill, and as such needs, in my experience and in my opinion, to be taught as such.

Thus, teaching a new skill in (e.g., Silverlight) and teaching (or even just showing) full testing, exception handling, bullet proofing, design and so forth at the same time is a recipe for teaching nothing at all.

Do we need reference applications that bring it all together? Sure, absolutely (though the implication that we'll have agreement on what is the best or even an acceptable way is arguable).

But, for example, at Tech Ed I'm demonstrating a number of new features in Silverlight 3, such as bitmaps, perspective 3d, Easing, Local Messaging, etc. Each will be demonstrated individually, in as simple and stripped down an example as I can, with zero exception handling or unit testing. I believe strongly that given 75 minutes that is the right thing, and so I respectfully decline the challenge.

-jesse

Jim Holmes said...

@Jesse

We'll have to agree to disagree. Strongly.

This is a massive failure of our community. Look to other segments of the industry where this mindset of ignoring testing or practical implementation details is flat out unacceptable. Unacceptable.

You're welcome to decline the challenge. I'm welcome to decline to support your mindset and approach which will do nothing to improve the quality of our community.

Jim Holmes said...

@Jesse

My last para in my first response to you is over the top. I don't believe in pulling comments from my blog -- even my own -- so I wanted to follow up and let you know I should have waited a bit longer to respond.

Obviously you hit a sore spot. :)

Anonymous said...

@Jesse

I think you're missing the point here. I don't think Jim is stating that you have to TEACH testing AND your topic at the same time. What he is trying to get across is we need to provide tests WITH our samples/presentations in order to get people in the habit and to show them how to do it correctly.

What I have been doing lately is driving presentations from unit test code. Do I show them how to unit test? Hell no, this would be an hour presentation in itself. However, what it does do is allows them to download a sample that they can look through that SHOWS them HOW to test, shows them what they need and how to wire it all up and as a side effect, gives a real nice flow when doing the presentation.

Here is a link to a recent presentation I did on SOLID principles with unit testing code. I stepped through the unit test to describe what was happening with the SUT and actually helped them learn what was going on and the context of each sample: http://github.com/schambers/solid-principles/tree/master

I actually had a few people thank me for including tests as it shows them how proper unit tests should look to properly test/decouple code in the SUT

I think just throwing up our arms and saying "I'm not going to show testing because they just won't get it" is doing a disservice to the community, especially with someone that is looked to as an expert in a specific field. For instance, I have NO idea how to test a silverlight app, no clue at all as I don't do silverlight dev. If I saw a presentation that actually had sample code and unit tests with it that would be excellent. Not only do I get sample code, but I can see HOW to test the code at the same time.

Reference applications are great, but they take time to pick through and I personally would rather see a presentation and get a small slice of unit testing code at the same time.

When using unit testing in applications they also have the benefit of being documentation in the form of code. I can't remember the last time I read through an api's docs before looking at the unit tests that it has. It's a great way to get an idea of whats happening very quickly. I think you under estimate the power of small focused tests.

Hopefully this comment didn't come out negative, I was just trying to further explain Jim's point, if that is what Jim was actually trying to get across :D

Travis Illig said...

I would think showing more "real examples" sorts of demos would be a selling point on the technology being demonstrated, but Jim's right - we don't see these sorts of demos.

For example, I can't tell you how many "intro to ASP.NET MVC" demos I've seen that show the exact same thing, but where are the in-depth localization and validation demos - the stuff I'll actually want to be using when the rubber meets the road? How about integrating Windows Workflow Foundation into MVC? Where's that demo? (And when you show me that demo, also show me how to test it.)

I agree to some level that showing everything all at once can be confusing to the newbie. I also think there's a happy medium that can be drawn, and Jim mentions this - almost everything we see is a 100 level presentation. I may slightly disagree with Jim in that I'm happy to let some best practices slide in a 100 level presentation. I feel like 100 level presentations are sort of like "just prove this new feature can be exercised." Spike code. See if it works, show it working.

If you're showing a 100 level presentation, sure, omit the extra stuff to get a beginner point across... but quit making every presentation 100 level, and when you get to the 400 level presentations, make sure you're putting some real content in there.

Gregory A. Beamer said...

Jim:

I am between you and Jesse on this one.

On a TechEd demo, I can see a "I am going to teach you A" with the understanding, preferably explicit, that the code is just demo code.

On the other hand, as I wrote a few months ago (http://snurl.com/h3h3), the prepoderance of really crappy demos is leading to worst practices in newbie developers (and many junior and senior devs, as well).

I agree with making demos better, but it has to be an "across the board" effort. As long as the books still suck, you have not conquered the problem.

Peace and Grace,
Greg

Michael Kimsal said...

This is pretty prevalent in the PHP projects/communities I've been a part of as well, and doesn't seem to be *too much* better in many other camps. Was there a particular area of software that you think does it better? Just wondering how much of this may be a case of the grass being greener elsewhere.

James Bender said...

I think Jim and Jesse both have good points:

I can understand the idea of keeping a demo simple and focused on the particular task at hand. To that end, where I may have used console apps to demonstrate things in the past, I've started to use tests. Their both just ways to run code and verify a result. But I understand that doesn't work in all cases. In that case I think it's a judgment call. BUT, even if I don't show TDD in my demo, I talk it up. A lot.

I will say though that I have worked with clients who I've tried to get on the TDD bandwagon. When their response is "Well, we have all these Microsoft books, and they don't use tests in any of their examples." I know it's gonna be a tough sell.

Just my $0.02

Jim Holmes said...

To be clear: I am NOT proposing we take massive amounts of time teaching TDD in demos. What I am pushing is that we at least show how one should test things. Even better, take a step in Sean's direction and use tests to show how something works. Moreover, skipping discussion of the pros and cons of these tools/technologies is an epic fail, IMO.

Let me be clear on something else: I skip TechEd and larger conferences like DevConnections because of issues like this. I get very little value from 90% of the sessions at those conferences. Why? Because they're lightweight demos which offer me nothing I can't find online.

Very few of the sessions at those conferences have given me any sense of how things work in the real world. A few exceptions would be the SharePoint sessions I saw from Neil Iverson, Todd Baginski, and Ben Curry at DevConnections in '07.

We're software engineers. Or craftsmen, if you prefer. Offering up sessions which do NOTHING to promote critical thinking about implementation is a disservice to our community.

Anonymous said...

I have to agree... there is no reason we can't have a properly implemented project, and just highlight the pieces that we are demoing.

Our industry needs better direction on proper implementations, instead of everyone rolling their own.

Or at the very least, Microsoft should have a corporate standard and hold everything, including posted demo code, to this standard.

Lead by example. Otherwise posted shortcut code will (like it or not) make it into someone's production code.

At the heart of the issue is this: What about a properly structured program makes it difficult to highlight concepts? As a structural engineer, I might walk into a building and be shown a feature... just because the rest of the building is built well doesn't mean I can't understand the feature I'm looking at.

Too many demos I've seen waste time building a project from scratch. Show me a working project, and explain to me the how and why of what is being demoed... and then give me that project so I can fully understand the application of those ideas!

Dave

Kirk Evans said...

@Travis - there are a number of demos that show how to host Windows Workflow Foundation in an ASP.NET application, which are applicable to both ASP.NET MVC as well as web forms. And ASP.NET MVC has rich support for unit testing. Since there is ample material on these, the area where there's not an abundance of information on is how to unit test WF.

There are 2 approaches. First, unit test the outcome of your workflow by checking the WorkflowCompletedEventArgs.OutputParameters property to inspect the outcome of your workflow. The second approach is to unit test your custom activities. If you search for "unit test Windows Workflow Foundation", it will yield several resources such as Ron Jacobs' blog entries, samples, and the MSDN Virtual Lab on unit testing WF.

WorkflowCompletedEventArgs.OutputParameters:
http://msdn.microsoft.com/en-us/library/system.workflow.runtime.workflowcompletedeventargs.outputparameters(VS.85).aspx

MSDN Virtual Lab on WF and Unit Testing
https://www.microsoft.com/resources/virtuallabs/step3-msdn.aspx?LabId=c4a993a5-d498-4d5c-9f98-476c1f496d15&BToken=reg

So, if I were giving a presentation on Windows Workflow Foundation and tried to code this demo from scratch, I could absolutely build custom activities, wire them up into a workflow, and even show how to use the OutputParameters property. If I were to create the *unit tests* for this, it would add a significant amount of time to the demo, greatly reducing the amount of information I could share on WF, ASP.NET MVC, the integration between them, the pitfalls to watch out for when hosting WF in ASP.NET, patterns for building custom activities, etc.

I take pride in not giving 100 level pre-canned demos. I take pride in being able to code demos from scratch and not following a simpleton script because I know the technology deeply enough to explain it.

I agree that there is some culpability on the evangelists here, but there is far greater culpability on the community. While an evangelist might give 20 talks in 2 months focused on introducing a technology or concept, that number is DWARFED by the number of user groups, code camps, conferences, study groups, lunch and learns, etc that the community hosts without an evangelist present. You don't need and shouldn't wait for evangelists to do this, it is much more appropriate for people who code day-in-day-out using unit tests to increase that chatter.

If you want true scale for this type of effort, start with INETA. Coordinate a national effort across all of INETA to have material focused on code quality and unit testing. You don't need (or even really want) me waxing philosophic about unit testing or even burning precious demo minutes trying to subclass a WF Activity just so I can get to the ActivityExectionStatus.

When I read blog posts from the community (I read a lot), I don't see the focus on unit testing or code quality, either. Sure, evangelists need to participate, but I can't accept the notion that you are listening to an evangelist because you want to understand best practices or code quality... you listen to evangelists to hear about very recently introduced technology, stuff you probably haven't touched yet but want to see how it works.

I am not rejecting this challenge, not in the least. In fact I have tried to find ways to accept it. For the technologies that I typically demonstrate (WF, SharePoint), I find it extremely difficult to show unit testing in a manner that actually adds value to the demo.

The frustration here is not that evangelists don't use unit tests, but rather the frustration is with the perceived diminished skill level of the developer pool. I can tell you first hand that is not universally the case, but rather a function of how many corporations recruit and view developers and IT as an expense they wish to minimize rather than an investment they wish to capitalize upon. There are a bunch of really great developers out there, the challenge continues to be how to find them.

If you can't find them, grow them. The place to start is locally, within your own user groups.

Jim Holmes said...

@Kirk,

Thanks for jumping in here.

I'm fine with you doing your presentations from scratch. That's cool -- but my question to you is, HOW are you doing that? Does all your behavior end up in code behind, or the equivalent of whatever you're presenting on?

If so, that's a massive fail, IMO, because all you're doing is continuing to foster the impression that THIS IS HOW IT SHOULD BE DONE. Fail.

If, however, you're doing things like discussing "Hey, don't put your data access here. Instead farm that out to the equivalent of a model class. Here's how that looks. Moreover, note that I'm not doing a lot of exception checking in these areas, but you'll definitely need to do that in your production code." then I'm utterly hip with that. If you're doing that amount of sensible discussion, then frankly I'm more than willing to forgive the lack of tests.

The frustration is not around developer skills. Folks come to events to RAISE their skills. The frustration is around how material is presented to those looking to improve themselves.

I'm not bothering to reach out to INETA on this. I've had zero success over four or more years with trying to get them out of their very narrow box. Good folks, all, but the organization doesn't meet my needs in many ways.

We both agree that this is a community issue, but I'm unwilling to let Evangelists just throw up their hands and say "Get that kind of info elsewhere, it's not our job."

Hooey. It's part of everyone's job who speaks about a technology, as far as I'm concerned. Even if you don't have tremendous real-world experience with something you can still talk to implementation design. Again, not simply tests, but discussion around HOW something's supposed to be used.

Kirk Evans said...

Sweet! Then we are completely on the same page.

Subscribe (RSS)

The Leadership Journey