Wednesday, December 29, 2010 == “Sorry, that page doesn’t exist!”

In case you’re looking for me on Twitter, I’m not.

Twitter turned in to a massive time sink for me (and I didn’t have time to sink in the first place), and I was letting myself vent way too many frustrations there. It was time to take my own advice: “Enough griping, enough bitching already. Roll up your sleeves already and get to work.”

‘Nuff said.

WinDirStat–Nice Disk Space Usage Visualizer

Hard disks are cheap and greatly ease the past pain of having to manage free space, but that’s not the case for me: I have a small-ish Crucial C300 128GB Solid State Drive as my C: volume in my work laptop. It’s been absolutely awesome for working with the various VMWare virtual servers I deal with on a daily basis. The speed on the SSD goes to 11.

The only downside is I have to be very careful about free space on the SSD. SSDs need breathing room to deal with various low-level things (go read Wikipedia’s article on them for a great overview), so I try to keep things fairly tidy and clean up unneeded files regularly.

One of the best tools I’ve found to help me with this is WinDirStat, a free GPL tool. No installation, just download and throw in a util folder somewhere. Launch it via SlickRun to be really cool.

You get an easy to understand UI with clear indicators where your space pain points are.

Besides, I’m a tool whore and this is yet another great thing for me to play with.

Wednesday, December 22, 2010

HTTP 500 Errors With Invalid Request Format After Upgrading Site to .NET 4.0

Problem: After upgrading an ASP.NET site to 4.0 you get HTTP 500 errors with exceptions when Javascript is making web service calls: “System.InvalidOperationException: Request format is invalid: application/json; charset=utf-8.” You’ve used IIS Manager to change the application pool to specify .NET 4.0 but the problem persists.

Solution: Change to the 4.0 framework folder (C:\Windows\Microsoft.NET\Framework64\v4.0.30319 in my case) and run aspnet_regiis.exe. You’ll need to determine which option you want – check the help before running and determine what you need.

For whatever reason, simply changing the application pool’s version value in IIS Manager didn’t work. Using the big hammer of aspnet_regiis.exe fixed things up.

Duplicated Setup Code in Your Tests? Refactor to a Factory!

Your test code is production code. Treat it the same way. Don’t let cruft build up, and follow good software engineering principles – like Don’t Repeat Yourself (DRY).

I was looking through some test code the other day and found a great opportunity to refactor out some common behavior to a factory. I do this quite often in the test frameworks I build – shove off responsibility for data and object creation to a factory so I can keep things much clearer in my tests.

In this case, we’re testing a parser class that creates reports based on an XML file. A number of tests check the parser’s handling of malformed XML or missing elements. The basic pattern below is repeated across eight or nine tests: hardwire an XML string in the test, feed that to the parser, then check if appropriate errors and messages are generated by the parser.

   1: var reportXml =
   2:     @"<report version='4.0'>
   3:             <key>FiscalYearByBrowser</key>
   4:             <description>Shows cool stuff.</description>
   5:             <connectionString>TelligentAnalyticsAnalysisServer</connectionString>
   6:             <query>
   7:                select some SQL-fu from a magic place and return unicorns
   8:             </query>
   9:       </report>";
  11: _report = ReportParser.Parse(reportXml);

This is repetitious, duplicated, copied cruft. Small joke there. Moreover, it’s not readily apparent what the XML snippet’s intent is. I have to read the entire XML, then mentally unwind what I know of the structure to figure out that, oh, this snippet is actually missing the title element.

Compare the above with this section of refactored code:

   1: _report = ReportParser.Parse(TestReportFactory.GetWithoutTitle());

The implementation of the TestReportFactory isn’t overly important [1]; what’s important here is that duplicated behavior should be pushed out to a common area, regardless of whether it’s in the system under test or your test code itself.

Your tests are production code. Treat them as such!

(NOTE: Some folks, particularly the Really Smart Guy Jeremy D. Miller, make the absolutely solid case that you should never sacrifice readability in your tests for the sake of cutting duplication. It’s good to push duplicated behavior out if you can keep your tests completely clear. It’s not ok to push behavior up to base classes or elsewhere if your test turns into a vague skeleton. Readability trumps everything else.)

[1] The TestReportFactory simply builds up a string of XML, creates an XElement from that, then uses XElement.remove() or XAttribute.remove() to drop elements or attributes as needed. The entire class is pasted below. Yes, there are a number of other ways to skin this cat. I could make things a bit more dynamic, but this worked for what I needed right now. And yes, I drove design and implementation of this factory with a separate set of tests I wrote first…

   1: internal class TestReportFactory
   2: {
   3:     private static string _reportOpen = @"<report version='4.0'>";
   4:     private static string _key = @"<key>FiscalYearByBrowser</key>";
   5:     private static string _title = @"<title>Fiscal Year by Browser</title>";
   7:     private static string _description =
   8:         @"<description>Shows page views per browser (IE, FireFox, etc..)
   9:          for each fiscal year.</description>";
  11:     private static string _connectionString =
  12:         @"<connectionString>TelligentAnalyticsAnalysisServer</connectionString>";
  14:     private static string _query =
  15:         @"<query>
  16:             select non empty [Time].[Fiscal Year].Members on columns,
  17:             non empty [Dim Browser].[Browser Name].Members on rows
  18:             from [Evolution Reporting]
  19:          </query>";
  21:     private static string _chartTypes =
  22:         @"<chartTypes>
  23:                                 <add type=""fancy 3d chart"" />
  24:                                 <add type=""plain 2d chart"" />
  25:                               </chartTypes>";
  27:     private static string _reportClose = @"</report>";
  30:     public static string GetValidXml()
  31:     {
  32:         XElement doc = CreateValidReportXElement();
  33:         return doc.ToString();
  34:     }
  36:     private static XElement CreateValidReportXElement()
  37:     {
  38:         var _outgoing = new StringBuilder();
  39:         _outgoing.Append(_reportOpen);
  40:         _outgoing.Append(_key);
  41:         _outgoing.Append(_title);
  42:         _outgoing.Append(_description);
  43:         _outgoing.Append(_connectionString);
  44:         _outgoing.Append(_query);
  45:         _outgoing.Append(_chartTypes);
  46:         _outgoing.Append(_reportClose);
  47:         XElement doc = XElement.Parse(_outgoing.ToString());
  48:         return doc;
  49:     }
  52:     internal static string GetWithoutQuery()
  53:     {
  54:         return CreateXmlStringWithoutElement("query");
  55:     }
  57:     private static string CreateXmlStringWithoutElement(string elementName)
  58:     {
  59:         XElement doc = CreateValidReportXElement();
  60:         XElement element = doc.Element(elementName);
  61:         element.Remove();
  62:         return doc.ToString();
  63:     }
  66:     internal static string GetWithoutKey()
  67:     {
  68:         return CreateXmlStringWithoutElement("key");
  69:     }
  71:     internal static string GetWithoutDescription()
  72:     {
  73:         return CreateXmlStringWithoutElement("description");
  74:     }
  76:     internal static string GetWithoutConnectionString()
  77:     {
  78:         return CreateXmlStringWithoutElement("connectionString");
  79:     }
  81:     internal static string GetWithoutChartTypes()
  82:     {
  83:         return CreateXmlStringWithoutElement("chartTypes");
  84:     }
  86:     internal static string GetWithoutTitle()
  87:     {
  88:         return CreateXmlStringWithoutElement("title");
  89:     }
  91:     internal static string GetWithoutVersion()
  92:     {
  93:         XElement doc = CreateValidReportXElement();
  94:         XAttribute ver = doc.Attribute("version");
  95:         ver.Remove();
  96:         return doc.ToString();
  97:     }
  99:     internal static string GetInvalidXml()
 100:     {
 101:         return "<report>";
 102:     }
 103: }

Wednesday, December 15, 2010

I’m Looking for A Great QA Team Member!

Want to be part of a team that’s deeply involved in fundamental changes to how a company builds its software? Don’t mind hollering “Sancho! My armor!” because with some regularity you defeat those windmills?[1]  Passionate about learning and continually improving how you do work? Care more about enabling the team’s success than ticking boxes on your own resume? Think that lots of collaboration and communication with developers and stakeholders is more important than pages of requirements documents and specifications?

Got your interest yet? If so, open up a conversation with me about joining my QA team at Telligent. We’re going to be opening up a new position in short order, and I’m looking for someone who can help us continue driving some amazing, fundamental transformations[2] at the company.

Check out my post from last year describing the job and a bit about the environment. Most everything there still applies.

While the general skillset and duties in that other post are still applicable, there are a few tweaks to responsibilities and skills for this new position. Our new team member will be responsible for

  • Writing automated acceptance and integration tests
  • Helping drive our Section 508 and WCAG 2 accessibility efforts
  • Assisting ongoing efforts to improve our overall development processes

If you read my blog or follow me on Twitter you know I’m much more interested in what you’ve got in your head and heart than what you’ve got on your resume. That means I’m not overly concerned if you’re not an expert in C#, Selenium, or even .NET. I am concerned that you’re driven to learn new things. I am concerned that you’re passionate about driving cultural change. I am concerned that you think it’s better to talk frequently with folks at the start of the development lifecycle rather than wait until a dev-complete feature lands in your lap near the end of the lifecycle. (Dear fans of Six Sigma, Rational ClearQuest, and Waterfall: you likely will not thrive working with me.)

Interested? Drop me a line! Use the contact link on this blog’s sidebar, or mail me directly: JHolmes AT Telligent DOT com.

[1] Don’t mind working for a boss who (likely far too often) throws out weird references to books, movies, and oddball stories?

[2]“Transformations” is lousy, over-used blabberspeak marketing crap. But you know what? The entire group my team fits in is making amazing changes to how we create our software. In this case it fits.

Monday, November 29, 2010

Geekspeak: Ten Tips for Making an Effective Pitch

You’re fired up about something, but you need approval to make it happen. Maybe it’s asking for approval to go to a great developers’ conference. Maybe it’s asking for a new hire to expand your team. Maybe it’s asking for some software to help you write better code. Maybe it’s pushing for a raise for one of your subordinates.

If you care enough about something to make a pitch for it to your management, care enough to make a GOOD pitch. Writing up a passionate, long-winded ranting e-mail or white paper might make you feel better, but you’re not going to have much of a chance of success. Spend some time preparing your pitch and you’ll have a much greater chance of winning folks over to your side.

Here are a number of things which I’ve learned over the years through some great mentoring and a lot of hard knocks. I don’t claim to have a magical approach, and I’ve failed at a number of pitches, but overall I’ve had some pretty good success following these general concepts.

Focus your idea

Make sure you know what it is you want. Make sure you’re pitching for something concrete and obtainable. “We need to train our dev staff!” isn’t a focused idea, “We need to train our dev staff and can do it by sending them to CodeMash! It’s CHEAP!” is.

Broad, unfocused ideas don’t make good pitches. They make for awesome places for you to start focusing down to a specific plan.

Know your audience

Understand who you’re pitching to, and what sort of information they’ll need. The bigger your pitch (read “the more it will cost”), the higher up the food chain you’ll need to go. Ensure you’re pitching to the right folks – drafting something for marketing won’t help you if you’re really trying to get new headcount approved…

You need to do your homework and learn as much as possible about the folks who have approval authority. You’ll need to understand their motivations for the organization so that you can tailor your pitch to what they’re interested in.

Just as importantly, you need to understand whether your goals match up with theirs. Earlier in my career I spent months, literally months, polishing up a pitch for something I felt extremely passionate about. I had to work closely with mid-level management to create a significant package and presentation for upper management, and it wore me out. The pitch ultimately was rejected because upper level leaders had completely different views and core values than I did – and I had no idea that was the case. I wasted huge amounts of my time and spent a lot of political capital at lower levels only to find out I never had a chance.

Losing that particular pitch was one of the most demoralizing things I’ve ever had happen in my career. Not because the pitch didn’t get accepted, but because I’d so completely misunderstood the culture at the top levels of that particular company.

Save yourself that heartache: do your homework up front by testing the waters, floating trial balloons, or whatever metaphor you prefer. See if you’ve got a reasonable chance at succeeding before you charge that hill.

Speak the language

More likely than not you’ll be making your pitch outside your circle of geeky colleagues to a manager somewhere in your organization. Phrase your pitch in terms your audience feels important – and that’s nearly always some form of specific, quantifiable value delivered back to the organization or your customers.

This bears repeating because it’s critical: Tie back to specific business value. Your pitch will fail if you don’t, regardless of how big an item you’re asking for.

Speaking about improving quality or increasing productivity is nice, but get more specifics: speak to cutting support costs. Speak to delivering features more quickly. Talk to how your idea helps the bottom line or improves relationships with your customers.

Believe it or not, but business people actually do care about many of the same things we geeks do, they just speak a different dialect. Bridging that language gap is critical for an effective pitch.

Understand the budget process and cycle

Money matters. Sorry, it does. You need to know how your request will fit in your organization’s budget process and cycle. The larger your request the more you need to know.

The budget process generally involves how requests are filed and who has approval authority for what amounts. Understanding how far up the chain your request will need to go helps you get a feel for how much prep work you’ll need to do. A $250 expense for a new SSD drive may only need your direct manager’s approval based on an e-mail. A $30K training workshop may need to go all the way to your CFO with a significant amount of backup material.

Budget cycles matter because you’ve got to understand when you’ve got a window for a successful pitch. Larger purchases such as new hardware generally have to be planned out well in advance. You may need to be making your pitch for that $30K performance testing lab a year in advance in order to get it on next year’s budget.

Knowing the annual budget cycle may not be enough; you may also need to understand the organization’s spend plan, too. Financial folks in companies love stability of income and expenses. They don’t like huge spikes and generally try to schedule out large expenses through the year rather than drop huge amounts of cash in one month or quarter. Looking for a really significant expense? Ensure you’re not stacking up your request when other big expenses are scheduled. Make your pitch include a target timeframe which won’t rock the spending plan. (Too much…)

The executive summary is critical

The higher up your pitch will go, the more important a tight, well-written opening is. You need to have one paragraph, perhaps two, which gets right to the point of what, why, and how. This opener might end up being targeted to a different audience than the rest of your pitch.

Write your opener after everything else in your pitch is done, not before. Get all the details in place, then work REALLY HARD to distill everything down to the bare essence.

Keep in mind this summary may be the only thing read by the person who will give your pitch the thumbs up or down. Spend the time to get this part right.

Back it up with data

The summary is critical, but you need meat and potatoes behind it. (Or tofu and potatoes if you’re vegetarian.) Dig around and root up some metrics that back up your pitch.

I once had to write up a salary increase proposal for one of my subordinates. He was a very valued member of our team and was ridiculously underpaid. I compared his salary to standard wages in the region (“He can get $10K more tomorrow by quitting and going to our competitor.”). I was also able to show how his great development skills boosted the team’s velocity by 30% by tying feature estimates to actuals when colleagues worked on infrastructure he’d implemented.

If your pitch deals with the hard engineering side of things (training to help improve how you write code, hardware for performance testing, new dev machines, etc.) then gather up metrics from tools like NDepend,  NCover, simple perf test tools, etc. Tie this to industry standards, your own Service Level Agreements, or other similar items which your audience understands – as with everything about your pitch, tie it back to specific, understandable business value.

Don’t get carried away with flooding your audience with data, but by all means do get enough detail to make your case.

Show options and pros/cons

Your idea may not be the only way to skin the cat. Your idea may have drawbacks. Lay those out with the same care you spend on your idea. Doing so shows your audience that you’ve thought long and hard about your pitch and are being honest about alternatives and potential drawbacks.

Now, that said, you don’t need to go overboard propping up the alternatives and cons. After all, you want your idea implemented!

Be prepared to fail

Harsh reality: your idea may not be the greatest thing since sliced bread. Others in the organization may have different opinions. The organization may not have budget. There may be a hiring freeze. There may be things going on in the organization that you don’t have knowledge of.

Do be optimistic when making your pitch. Do work hard at getting your pitch right. Do be passionate about it.

Just don’t let a rejected idea completely sour you on life.

That said, if the pitch was something that’s central to your core values remember this: you can change where you work, or you can change where you work.

Less is more

Perhaps the overall most important point is this: Be concise. Spend the time to write a lean, concise pitch. Cut out all unnecessary cruft. Re-read, edit, re-edit, then have someone else read the pitch.

Be sure everything in your pitch helps make your case, and ensure you’re not adding in extra fluff which doesn’t help your case.

Don’t write ten points when nine will do.

Tuesday, October 26, 2010

Examples from Unit Testing 101 Talk

I’ve given my Unit Testing 101 talk several times in the past month or two. I really love this talk because it’s fun to give and it’s obviously something I’m more than a little passionate about. It’s also a lot of fun because, unlike the majority of my talks, I’m spending the majority of the talk writing code. I don’t get to do that anywhere near enough in my day job, so it’s a kick to go do it while gabbing with community hommies!

You can find the before and after VS 2010 solutions here on my site. The “slide deck” is really a Prezi which you can find online here.

Tools used

  • NUnit – the test framework I wrote all the tests in
  • Rhino.Mocks – the framework I demonstrated for mocking out dependencies
  • ReSharper – Gives me a test runner, quick templates, and scads of other goodies
  • CodeRush – Another productivity tool for a runner and tons of other joy
  • AutoHotkey – Enables the next tool
  • JP Boodhoo’s BDD AutoHotKey script – makes it easy to write test_cases_with_underscores

References I mentioned at the end of my talk:


Thanks to everyone who attended my talks, particularly the great folks at MIGANG – that’s one of the best interactive audiences I’ve had in years!

Sunday, October 24, 2010

Book Review: Managing Humans

Managing Humans by Michael Lopp. Published by Apress. ISBN 159059844X.

I’m torn by this book. There’s a lot of really useful, interesting articles in the book, but it feels like much of the content is barely reworked from the author’s blog, Rands In Repose, and the amount of over-the-top gratuitous profanity really make the book feel unpolished. (And those who know me well are howling that I’m complaining about profanity.)

Putting aside those two objections, I found a number of good pieces in the book. There’s a whole lot of great things about dealing with people in tense situations, and many of the pieces end up pointing back to how you should have been more aware of what was going on. For example, one article discusses how to deal with an employee who explodes about something he feels is important. The article walks through getting the person calmed down and reaching a good resolution with them – but the last paragraph throws it right back in your face with something along the lines of “But you need to remember that the person felt the only way to get your attention was to explode.”

There’s plenty more like this, with the book covering dealing with your people, the people who manage you, and keeping a close eye on empowering your own career. Lopp uses lots and lots and lots of examples cut right out of his own career, which is really nice. The topics he cover are obviously things which are real world issues, not contrived, theoretical dealings.

The rest of the book keeps up this same style: some good reading, some uneven content, some throw-away F-bombs and excrement quips, and occasional amazing flashes of “Wow, I really need to take a closer look at myself in the mirror” moments.

Overall it’s a fine read. I’m not sure I got my money’s worth out of it, but it’s a good read all the same.

Wednesday, October 06, 2010

Upcoming Webcast: Three Tips to Improve Your Dev Process

I’m giving my “Three Tips to Improve Your Development Process” talk tomorrow (Thursday, 7 Oct) at the Cincinnati Programmer’s Guild.

This talk covers how you can improve your development process by implementing daily standups, regular retrospectives, and improving your estimation. These tips work regardless of whether you’re in an Agile, CMMI, or complete chaos process environment.

I’ve just found out the Guild’s leader, Troy Davis, will be streaming the presentation on UStream. You can find the stream here. The show should start about 6:30 if everything goes well and the AV gods behave themselves.

Monday, October 04, 2010

Upcoming Speaking Engagements

I’ve got a somewhat busy October for speaking!

  • This Thursday (7 Oct) I’ll be talking at the Cincinnati Programmer’s Guild hosted at Max Training. I’m giving my Three Tips to Improve Your Development Process talk. This is one of my favorite talks because it hits a few specific things you can do to improve your development environment regardless of where you work: Start daily standups, use retrospectives, and get a handle on your estimation.
  • Saturday I’ve been invited to kick off the Software Quality Firestarter in Mason (Cincinnati) with my talk on Testing Web Applications with Selenium. This is an intro-level talk, plus I pass on a LOT of hard knocks lessons we’re learning while using Selenium at my work. It’s a 60 – 75 minute talk with an incredible amount of goo crushed in to it. (There are still seats open for this event, so go register!)
  • Finally, Justin Kohnen and I are teaming up at the Dayton .NET Developers Group on 27 Oct to cover a few odds and ends in Visual Studio 2010 and .NET 4.0. I’ll be showing off some of the neat goodies helping you get rolling with Test Driven Development. I will also talk a bit about the web test framework and show some uses of it for creating simple performance tests. Justin will be covering some of the goodness around ASP.NET, configuration deployment support, and some other things. I always love co-presenting with Justin because he’s wicked smart and geeks out on all these cool toys.

Let me know if you’re going to be at any of these events. It’s always nice to put a face to blog readers or Tweeps!

Tuesday, September 28, 2010

Intro to Unit Testing Presentation Video & Materials

A couple weeks ago I drove up to the Great Lakes Area .NET Users Group to give my Intro to Unit Testing talk. The MIGANG crowd is always great to speak to – they’re interactive (always a win for a speaker), they’re interested, and there are always a bunch of pals in the audience.

Dave Giard recorded my presentation and posted it up at the MIGANG site. You can find it here. It’s a two hour session – not just because I was long-winded (which I am), but because there was lots of great discussion about various testing topics.

You can find my presentation materials here. I’ve got my “slide” deck (in Prezi form) and the starting/ending versions of my demo code.

Speaking at MIGANG is always a great time, and totally worth an eight hour round trip for me from Dayton. Thanks to the gang for having me up to talk!

Wednesday, June 23, 2010

Managing Your Hiring Effort

Congratulations! Your company is doing well enough that you need to find new folks for your growing team. I’ve been on the hiring side of the equation several times and thought I’d pass on some things I’ve learned about managing your hiring effort.

(I’m purposely skipping a number of topics in this post, such as getting approval for your headcount increase, dealing with your internal hiring processes, etc. There’s only so much I can write about in one post… I’m also assuming you’re the person responsible for managing the flow of candidates through the system.)

As you move through the process of sifting through piles of resumes, you need to keep a solid grip on your one primary goal: finding a candidate that will help you deliver great solutions. Hiring new team members is a time consuming[1], frustrating, expensive process, and it can potentially impact a large number of folks in your organization. Every time you look at a cover letter, resume, or speak to a candidate you must be asking yourself “Will this person improve my team?” and “Am I making the best use of the time of other interviewers on my team by sending this candidate to them?” If the answer isn’t a resounding “Yes” then you need to thank that candidate for their submittal and move on to the next candidate on your list. Focus on improving your team. Focus on ensuring you’re not wasting colleagues’ time by passing on mediocre candidates who likely won’t improve your team.

Every hiring effort is different, and each new open position will have different requirements, but in general, here are the things I always look for at each stage when considering a candidate.

  • Passion. Does the candidate care about what they’re doing?
  • Culture/mindset. Will the candidate fit in with the team?
  • Drive for learning. What’s the candidate doing for self-improvement?
  • Reliability. Can the candidate be counted on to meet all their commitments (get to work, attend meetings, be responsible for their tasks, etc.)

Once the job is advertised you’ll have to deal with the flow of incoming candidates, tracking which ones you’re advancing and which ones you’re rejecting. I use a combination of Excel and Outlook Inbox folders to mange my flow. (Maybe you’re in a huge corp with some specialized system to manage all this. In which case feel free to ignore all this…)

Outlook with numbered folders to mimic my flow:

Folders for hiring workflow

Excel sheet for tracking my contacts with each candidate and notes about them. I can sort on the various Y/N columns to keep progressing candidates at the top of the view.

Excel sheet for hiring workflow

Here’s how my workflow generally rolls:

  1. Receive notice of a new candidate from HR or some other source. Log the initial contact in my Excel sheet.
  2. Immediately respond directly to the candidate and thank them for their interest. You absolutely must respond to each and every candidate who contacts you. I can’t emphasize this enough. Far too many companies simply take the black hole approach to candidate submissions – you mail in a nice cover letter and a resume, and you have no clue if the company got it, much less whether or not they’re interested in you. No response, no thanks, nothing. Moreover, too often you’re given the run around or rude responses if you call the company to check whether they’ve received your submission. This is utterly unacceptable. Treat each and every candidate with respect. Let them know you’ve received their submission and that you’re evaluating it. That’s a human on the other end of the line. Treat them as such. (I sent out over 250 resumes when living in Germany while my wife was stationed there. I got acknowledgements from the vast majority of those companies. Awesome.)
  3. I’ll wait to read the resume and cover letter until after I’ve sent the initial response. Now it’s time for my first-level screen. Is there a cover letter or introduction of some sort? No? Two strikes against the candidate. Cover letters or introductions matter. Is the resume neat? Is it coherent? Grammar and punctuation correct? Do the broad domain skills match up? Does the resume show any passion or drive in the candidate for self-improvement and craftsmanship? I weed out roughly 50% - 80% of candidate submissions at this point.
  4. If the candidate’s cover letter and resume appear worthwhile, I’ll take the next step in my screening process: having the candidate fill out a questionnaire. This is a great way to get an indication of what the candidate’s like, and how they approach things. I wrote about my questionnaire in an earlier blog post. This approach helps me continue to trim the initial pool of candidates. I’m down to about 10% of the initial candidate count by the time I’m done evaluating questionnaire responses.
  5. If the candidate’s responses to my questionnaire are good, then I’ll schedule an initial phone screen with them. This is a 30 minute phone or video call where I spend some time laying out what the job’s like, how big a pain in the butt I am to work with, and what the company’s about. The rest of the call is me getting a feel for how well the candidate matches up with the broad topics I laid out in bullet points near the top of this post. I’ll also clearly lay out what the salary range is, and what date I’m looking for folks to start on. You can’t afford to waste anyone’s time if the candidate isn’t willing to accept an offer in your price range. Clear this up right at the start.
  6. If the phone screen goes well, then I’ll schedule the candidate for a loop of interviews. If you’ve done your screening right, only a handful of candidates will make it to the more formal interview process. I get three or four colleagues at various levels and roles to talk with the candidate for up to an hour. My colleagues’ focus should be the deeper technical interviews, plus getting a feel for whether or not the candidate will be a good fit culturally. Again, each interviewer should be asking “Improve team, yes or no?” You also need to ensure the candidate is able to speak with your HR folks. First, HR has a completely different view into candidates and you absolutely need their take on the candidates. Secondly, you need to make sure your candidates are getting all the information possible on benefits, policies, etc.
  7. Now you’re able to evaluate your colleagues’ feedback on candidates and figure out who you want to have a final screen with. You should already have a clear picture of who your top two or three choices are. I’ll do a final phone screen with my top choices to make sure they’re still interested in the position, all their questions are answered, and they’re willing to accept a specific salary and start date. At this point there’s likely a bit of juggling going on. You have a top candidate in mind to extend an offer to, but you also have to keep your communications open with your second and third choices – your top choice may back out at the last minute for any number of reasons, and you need to have options until your primary choice has signed an offer letter.
  8. Get an offer letter out to your top choice. Give them five working days, max, to accept the offer. Nothing in the offer letter should be a surprise at this point, so it really should be a matter of them confirming the details and signing.
  9. Once your top choice has accepted, then close the loop with your other top choices. Call them up personally, don’t mail.  They’ve gone through lots of interviews and they’ve had their hopes raised. Treat them with respect. Moreover, these folks are your top choices for any future open positions you may get, so you want to ensure you keep a good relationship with them.

Take care managing your hiring process. Make sure you’re respecting the time of everyone involved, and never, ever lose sight of your goal: advancing only candidates who will improve your team and help you create great software.

[1] Some staffing and consulting companies push off hiring personnel for opportunities until they’ve signed contracts. This is a tough juggling act and injects some serious time pressure, but I’ll push back on folks in those hiring situations: re-think your approach to hiring. Good personnel aren’t cogs you get off a shelf somewhere on a moment’s notice. If you think they are then you deserve what you get.

Thursday, June 17, 2010

Initial Screening Questionnaire

When I started hiring to build up my QA team I took a great idea from a friend who was hiring for her team: give potential candidates a questionnaire to fill out as one of the first steps of the interviewing process.

This turned out to be a huge timesaver – as someone managing a hiring effort, you have to absolutely focus on finding the small number of candidates who have the potential to improve your team. You don’t have enough time to talk with every potential candidate, and just reading through resumes doesn’t give you enough information to decide whether or not a 30 minute call is worth your while.

My screening process runs like this: My first screen is the cover letter/e-mail – if it’s incoherent or badly done then I don’t move that candidate forward. Next comes the resume, where I continue with folks who’ve mentioned community involvement, testing, and a number of other things I care about. At this point I’ve likely cut my potential candidate pool in half. I’ll send them the questionnaire and ask them to return it to me in a reasonable amount of time.

The last questionnaire I used included these questions:

  1. Describe in detail your approach to testing a software feature with which you are not familiar.
  2. Next I give them a use case where I point them to a public feature on our site and ask them how they’d test it.
  3. Describe when in the software development process you as a QA engineer would like to get involved in fleshing out the testing requirements for a feature?
  4. Describe how you see your interactions with the following team member roles:
    • Software Developer
    • Program Manager
    • Feature stakeholder
    • Technical writers
    • Support
  5. Do you have a blog? If so, what’s its URL?
  6. What have you done in the last week to improve your skills?
  7. Why do you want to work for Telligent?

This questionnaire focuses on a few key points for the type of folks I look for: are you detail oriented? What’s your approach to testing? How do you view the members of the team? Do you care about improving your skills.

The responses to these questionnaires are always very interesting and are a huge help in deciding who to continue the interview process with – at this point I’m now better able to decide who I’ll have an initial phone screen with. If that call goes well then we’ll move forward with the regular interview process.

For what it’s worth, only 10% of the candidates who submitted for positions on my team made it to the initial phone screen stage.

Feel free to steal, adapt, or ignore this concept. It’s worked very well for me!

Subscribe (RSS)

The Leadership Journey