Managing Your Hiring Effort

Congratulations! Your company is doing well enough that you need to find new folks for your growing team. I’ve been on the hiring side of the equation several times and thought I’d pass on some things I’ve learned about managing your hiring effort.

(I’m purposely skipping a number of topics in this post, such as getting approval for your headcount increase, dealing with your internal hiring processes, etc. There’s only so much I can write about in one post… I’m also assuming you’re the person responsible for managing the flow of candidates through the system.)

As you move through the process of sifting through piles of resumes, you need to keep a solid grip on your one primary goal: finding a candidate that will help you deliver great solutions. Hiring new team members is a time consuming[1], frustrating, expensive process, and it can potentially impact a large number of folks in your organization. Every time you look at a cover letter, resume, or speak to a candidate you must be asking yourself “Will this person improve my team?” and “Am I making the best use of the time of other interviewers on my team by sending this candidate to them?” If the answer isn’t a resounding “Yes” then you need to thank that candidate for their submittal and move on to the next candidate on your list. Focus on improving your team. Focus on ensuring you’re not wasting colleagues’ time by passing on mediocre candidates who likely won’t improve your team.

Every hiring effort is different, and each new open position will have different requirements, but in general, here are the things I always look for at each stage when considering a candidate.

• Passion. Does the candidate care about what they’re doing?
• Culture/mindset. Will the candidate fit in with the team?
• Drive for learning. What’s the candidate doing for self-improvement?
• Reliability. Can the candidate be counted on to meet all their commitments (get to work, attend meetings, be responsible for their tasks, etc.)

Once the job is advertised you’ll have to deal with the flow of incoming candidates, tracking which ones you’re advancing and which ones you’re rejecting. I use a combination of Excel and Outlook Inbox folders to mange my flow. (Maybe you’re in a huge corp with some specialized system to manage all this. In which case feel free to ignore all this…)

Outlook with numbered folders to mimic my flow:

Excel sheet for tracking my contacts with each candidate and notes about them. I can sort on the various Y/N columns to keep progressing candidates at the top of the view.

Here’s how my workflow generally rolls:

1. Receive notice of a new candidate from HR or some other source. Log the initial contact in my Excel sheet.
2. Immediately respond directly to the candidate and thank them for their interest. You absolutely must respond to each and every candidate who contacts you. I can’t emphasize this enough. Far too many companies simply take the black hole approach to candidate submissions – you mail in a nice cover letter and a resume, and you have no clue if the company got it, much less whether or not they’re interested in you. No response, no thanks, nothing. Moreover, too often you’re given the run around or rude responses if you call the company to check whether they’ve received your submission. This is utterly unacceptable. Treat each and every candidate with respect. Let them know you’ve received their submission and that you’re evaluating it. That’s a human on the other end of the line. Treat them as such. (I sent out over 250 resumes when living in Germany while my wife was stationed there. I got acknowledgements from the vast majority of those companies. Awesome.)
3. I’ll wait to read the resume and cover letter until after I’ve sent the initial response. Now it’s time for my first-level screen. Is there a cover letter or introduction of some sort? No? Two strikes against the candidate. Cover letters or introductions matter. Is the resume neat? Is it coherent? Grammar and punctuation correct? Do the broad domain skills match up? Does the resume show any passion or drive in the candidate for self-improvement and craftsmanship? I weed out roughly 50% - 80% of candidate submissions at this point.
4. If the candidate’s cover letter and resume appear worthwhile, I’ll take the next step in my screening process: having the candidate fill out a questionnaire. This is a great way to get an indication of what the candidate’s like, and how they approach things. I wrote about my questionnaire in an earlier blog post. This approach helps me continue to trim the initial pool of candidates. I’m down to about 10% of the initial candidate count by the time I’m done evaluating questionnaire responses.
5. If the candidate’s responses to my questionnaire are good, then I’ll schedule an initial phone screen with them. This is a 30 minute phone or video call where I spend some time laying out what the job’s like, how big a pain in the butt I am to work with, and what the company’s about. The rest of the call is me getting a feel for how well the candidate matches up with the broad topics I laid out in bullet points near the top of this post. I’ll also clearly lay out what the salary range is, and what date I’m looking for folks to start on. You can’t afford to waste anyone’s time if the candidate isn’t willing to accept an offer in your price range. Clear this up right at the start.
6. If the phone screen goes well, then I’ll schedule the candidate for a loop of interviews. If you’ve done your screening right, only a handful of candidates will make it to the more formal interview process. I get three or four colleagues at various levels and roles to talk with the candidate for up to an hour. My colleagues’ focus should be the deeper technical interviews, plus getting a feel for whether or not the candidate will be a good fit culturally. Again, each interviewer should be asking “Improve team, yes or no?” You also need to ensure the candidate is able to speak with your HR folks. First, HR has a completely different view into candidates and you absolutely need their take on the candidates. Secondly, you need to make sure your candidates are getting all the information possible on benefits, policies, etc.
7. Now you’re able to evaluate your colleagues’ feedback on candidates and figure out who you want to have a final screen with. You should already have a clear picture of who your top two or three choices are. I’ll do a final phone screen with my top choices to make sure they’re still interested in the position, all their questions are answered, and they’re willing to accept a specific salary and start date. At this point there’s likely a bit of juggling going on. You have a top candidate in mind to extend an offer to, but you also have to keep your communications open with your second and third choices – your top choice may back out at the last minute for any number of reasons, and you need to have options until your primary choice has signed an offer letter.
8. Get an offer letter out to your top choice. Give them five working days, max, to accept the offer. Nothing in the offer letter should be a surprise at this point, so it really should be a matter of them confirming the details and signing.
9. Once your top choice has accepted, then close the loop with your other top choices. Call them up personally, don’t mail.  They’ve gone through lots of interviews and they’ve had their hopes raised. Treat them with respect. Moreover, these folks are your top choices for any future open positions you may get, so you want to ensure you keep a good relationship with them.

Take care managing your hiring process. Make sure you’re respecting the time of everyone involved, and never, ever lose sight of your goal: advancing only candidates who will improve your team and help you create great software.

[1] Some staffing and consulting companies push off hiring personnel for opportunities until they’ve signed contracts. This is a tough juggling act and injects some serious time pressure, but I’ll push back on folks in those hiring situations: re-think your approach to hiring. Good personnel aren’t cogs you get off a shelf somewhere on a moment’s notice. If you think they are then you deserve what you get.

Initial Screening Questionnaire

When I started hiring to build up my QA team I took a great idea from a friend who was hiring for her team: give potential candidates a questionnaire to fill out as one of the first steps of the interviewing process.

This turned out to be a huge timesaver – as someone managing a hiring effort, you have to absolutely focus on finding the small number of candidates who have the potential to improve your team. You don’t have enough time to talk with every potential candidate, and just reading through resumes doesn’t give you enough information to decide whether or not a 30 minute call is worth your while.

My screening process runs like this: My first screen is the cover letter/e-mail – if it’s incoherent or badly done then I don’t move that candidate forward. Next comes the resume, where I continue with folks who’ve mentioned community involvement, testing, and a number of other things I care about. At this point I’ve likely cut my potential candidate pool in half. I’ll send them the questionnaire and ask them to return it to me in a reasonable amount of time.

The last questionnaire I used included these questions:

1. Describe in detail your approach to testing a software feature with which you are not familiar.
2. Next I give them a use case where I point them to a public feature on our site Telligent.com and ask them how they’d test it.
3. Describe when in the software development process you as a QA engineer would like to get involved in fleshing out the testing requirements for a feature?
4. Describe how you see your interactions with the following team member roles:
• Software Developer
• Program Manager
• Feature stakeholder
• Technical writers
• Support
5. Do you have a blog? If so, what’s its URL?
6. What have you done in the last week to improve your skills?
7. Why do you want to work for Telligent?

This questionnaire focuses on a few key points for the type of folks I look for: are you detail oriented? What’s your approach to testing? How do you view the members of the team? Do you care about improving your skills.

The responses to these questionnaires are always very interesting and are a huge help in deciding who to continue the interview process with – at this point I’m now better able to decide who I’ll have an initial phone screen with. If that call goes well then we’ll move forward with the regular interview process.

For what it’s worth, only 10% of the candidates who submitted for positions on my team made it to the initial phone screen stage.

Feel free to steal, adapt, or ignore this concept. It’s worked very well for me!

Article on Selling Developer-Level Testing

I wrote an article on successfully selling developer level testing in your organization for nPlus1. The article hits a lot of things I think are really important when talking to your management and team about implementing some form of developer testing.

You really do need to make sure you present an honest, well-thought case for testing before rolling in to it. Hopefully this article will help you if you’re searching for ammunition to help win that debate!

Go read it. Then go write some tests.

Video: Panel Discussion on “Drinking From The Firehose”

I was flattered to be asked to sit on a lunchtime discussion panel at last month’s Ann Arbor Day of .NET. David Giard moderated the discussion and Aydin Akcasu captured it all on video.

Dave’s posted the video on his terrific Technology and Friends site, and it’s well worth you spending 37 minutes to watch it. Not because I’m on the panel, but because the discussion covered a lot of things I’m especially passionate about.

Some of those things would include, in no specific order:

• Technology’s moving fast, and coming in waves – but we’re empowered to do so many more things than we were even five years ago. I can focus on delivering value instead of writing infrastructure or tooling.
• Companies need to invest in their workers’ training.
• Workers need to get emphatic about owning their own careers. Companies need to support them, but you, not your company, own your destiny.
• Managers need to support workers with time sliced out of the schedule for continuous, ongoing training. (I didn’t even ask permission with my management. I just directed my team to take two hours a week and go explore. That’s my job as a boss: get forgiveness instead of permission…)
• Not learning every new shiny thing is OK. Maybe learn something about what biz value new technologies offer, but carefully pick and choose what you’re willing to dive in to.
• Do spend some time exploring new things, though, because that learning will inform and impact how you work with your current technology.

Props to Dave for a great show, and thanks to fellow panelists Mike Eaton, Jay Harris, Patrick Steele, and Jason Follas. It was a pleasure being up on stage with y’all.

Slides & Demo Solution for Selenium Test Presentation

I’ve uploaded a zip with the slides and demo code from my “Web Testing with Selenium” at Saturday’s Central Ohio Day of .NET. You can find them here.

The zip is 16MB in size – sorry, but that’s including the 15MB Selenium v1 jar file. There’s a README file in the root explaining the solution and some of the various support files.

If you attended the session, thanks! I enjoyed talking about Selenium and hope you got some good info.

Exploring Selenium 2: Introduction & Lessons Learned in Selenium 1

We use Selenium (version 1.x) at work as a regression / functional test tool. We’ve gone through a number of iterations over the last year and a half since I first started pimping Selenium and learned some valuable lessons. Now we’re in the process of migrating to Selenium 2/WebDriver and I’ve been really pleased with the results so far.

I thought I’d write a short series of short articles showing some of the things we’re learning about Selenium 2 / WebDriver.  To kick off the series I thought I’d lay out an overview of the journey we’ve taken so far, and some rationale why I knowingly chose a less-than-optimal solution at the start.

First, a quick overview of the bits and pieces of Selenium v1, the toolset we started out with:

• Selenium IDE (SIDE): A Firefox plug in that lets you record browser actions (navigation, clicks, input) and play back those. You can write assertions to check for things like text, elements, etc. Scripts can be saved out as HTML or exported to Java, Ruby, C#, and a few others. You can write your own templates for outputting scripts.
• Selenium Remote Control (Selenium RC): A Java-based server app that launches, manages, and kills browser sessions. You start up RC, then run tests written in one of the various supported languages (lots of client libraries around), and RC does the rest. RC lets you run your tests against different browser platforms. If your scripts are simple, you can run one test suite against Firefox, IE, and Chrome without re-writing. Theoretically…
• Selenium Core: This is the basic framework for both SIDE and RC. When you spin up RC you’re getting core as part of the environment.
• Selenium Grid: Manages splitting off test suites to run your tests in parallel, thereby massively speeding up your tests. (I haven’t used Grid.)

Initially we started out using Selenium IDE, the record and playback tool, for writing our tests. Some caveats should be noted about our test goals: we were building out automated test suite from ground zero and were focusing on very fundamental, simple functionality tests. Think CRUD operations, navigation confirmation, etc. No fancy UI manipulation, no dealing with Ajax or Javascript. With this in mind, we could use SIDE and save off the scripts as HTML files and feed those to RC. This approach had some pros and cons.

Pros:

• At the time, our Program Managers were our primary testers. SIDE’s record/playback made it easy to get some tests written. ( Some tests > no tests)
• HTML based scripts made it easy for folks writing tests to alter the scripts as needed because SIDE’s recording actions didn’t deal well with dynamic content or a number of other things. We’d have to clean all that up manually.
• RC takes HTML scripts as an input, so we could automate what tests we were able to come up with.
• HTML files are easily wrapped in to our source control system. Treat your tests like production code, because they are production code. Get your test files, regardless of format, in to your source control. Go on, I’ll wait here while you do it.

Cons:

• Record and playback tools are generally failures for anything above a trivial one-off script. The tools create fragile, wonky scripts which require a lot of tweaking to get right and more tweaking to maintain. (Yes, yes, yes, you may have had glorious success with tools from Rational or Mercury. 99% of the rest of the testing world can’t afford the hundreds of thousands of dollars for those tools.)
• SIDE and Selenium don’t do well with relative links in their HTML files. This made it impossible to centralize common functions like logging in to and out of the system, creating users, etc. This is perhaps the biggest strike – if ANY link changes anywhere, you’ve got to fix up every occurrence of it. Massive violation of DRY, massive fail.
• Managing Selenium RC’s server app during a build cycle can get tedious.
• RC turned out to be even more finicky with timing issues around scripts. Any web UI script is already very sensitive to timing issues around your page loading and rendering. HTML scripts were even more so.

I’d known about some of the SIDE limitations going in to the effort; however, I needed something that would be a quick win to show how functional tests could help the entire team. We had some great success with those initial tests, but I had always wanted to push to “real” Selenium tests via Ruby or .NET. I knew we’d have much better results, and it was also a way to get me back in to writing code on a regular basis. (Cue evil mastermind’s laugh.)

Making the case for moving to code for our Selenium was greatly simplified because I’d show the value prop of test automation already. We’d caught a number of regression bugs simply through the automation tests chugging away and finding things like “Whoops. The logon link isn’t on the main page anymore…” Sometimes you need to start really small and get a few victories on the board before pressing on to the better solution.

With all this in mind, I started writing Selenium tests using the .NET libraries and running those via RC. I started a migration path converting the old HTML tests to C#-based tests and passing those off to RC for execution. More pros and cons:

Pros:

• I get to write real code. I’ve been in a Practice Lead, Program Manager, or Quality Lead for several years now and haven’t been able to crack open Visual Studio but once every few months. Writing code makes me very happy, so quite frankly I’m not ashamed to list this as a pro.
• Using a “real” language to write tests instead of HTML scripts lets us start to build up a common library for frequently used calls. I can centralize things like log in / log out, content creation, etc. (I’m not centralizing tests for those steps, mind you, just the actions which might be prerequisites for other tests.) Building up infrastructure/an API for your test environment is critical to your success – speed, maintainability, readability, the benefits just go on and on.
• I can now use a languages tools like iterators, branches, etc. to start writing much more powerful tests and support functions.
• Did I mention I get to write real code again?
• Tests in native code run through RC are much, much more stable and performant. We saw a significant reduction in false failures and tests ran faster.

Cons:

• Your tests are now in real code. If you were hoping to leverage non-coders to help write your tests then you’ve cut off that resource. Now you’re looking at finding time from your dev team to write your tests, or you’re looking at finding QA resources who can write code well enough to write your tests. (I found a great one of the latter. No, you can’t have her.)
• Selenium RC is still in the picture. You have to spin it up, and you have to deal with false failures injected into your tests due to RC being the middle man broker.

At this point I’d been fortunate enough to get another couple full-time, dedicated QA resources on board. This gave me the flexibility to start looking at other tools. Based on a couple comments from Jeremy Miller and Adam Goucher I decided to look in to Selenium 2.

Selenium 2 is combining with WebDriver, a nifty pairing that lets you get rid of having to deal with the separate Selenium RC server and directly manipulate the browser. Less moving parts is a huge win in my book. The API is a lot cleaner, much more readable, and solves a number of technical issues around timing. The tooling is officially in “Alpha” status; however, it’s quite full-featured already and quite stable. We’ve been having great success with it so far.

I’ll follow this introductory post with several others walking through our move from Selenium 1 to Selenium2/WebDriver.

MSBuild: Targeting a Configuration from the Command Line

Update: Read the comments for some good tips from other readers.

Firing up Visual Studio to simply build a solution is nuts. Do it from the Visual Studio Command Prompt instead and save yourself time.

What I always forget, though, is how to specify a release build, ergo I’m writing this blog post so I can quickly find it the next time I need it.

From the command line, use the “/p:” switch and pass the value “Configuration=Release” like so:

msbuild CommunityServer.Sync.sln /p:Configuration=Release

I’m also in the habit of first doing a clean build before the release build to ensure I’ve got no leftover cruft:

msbuild CommunityServer.Sync.sln /target:clean

(Actually, I tie all this stuff together with SlickRun shortcuts and batch files, which is even better!)

Central Ohio Day of .NET is Soon!

This year’s installment of the Central Ohio Day of .NET is coming up soon! This year’s event is just a couple weeks away on 5 June. We’re returning to the handy Roberts Centre in Wilmington, nicely located equidistantly between Dayton, Columbus, and Cincinnati.

If you haven’t already, go have a look at the terrific session list, then go register.

We’ve had great turnout for each event – last year’s was over 200. If you don’t go you’ll be missing a terrific opportunity to improve your skills and network with your peers!