Handling State for Browser Testing

Marc Escher asked a great question on Twitter about browser testing and state. He actually wrote a Gist on it, but I thought the question merited a response here.

State in Tests

Handling "state" of a system for any test automation is an involved process that should evolve as your test suite does. All the same good design/engineering practices that we use for building good software (abstraction, DRY, readability, etc.) should be applied to our test automation suites. Because, you know, test code is production code.
Here are some thoughts on Marc's great set of questions:

1. What are patterns for creating expected state prior to running browser tests?

"State" is a broad term and many folks may think of different definitions. I think of it as data, environment, configuration. There's a difference between setting up background "state" for a test suite, and having each test set up its prerequisites.
Setting up an environment for a test suite run is a conscious decision based on what's needed for that particular suite to succeed. I regularly use a combination of loading baseline datasets, altering system configuration (turn off CAPTCHA, select a simple text editor, swap mail providers, etc.), and leveraging test-specific libraries, infrastructure, APIs to get the starting environment set up.
Also, it's very, very critical to understand there's a difference between system-wide state (baseline datasets, eg) and state needed for individual tests. No test should ever, ever rely on sharing state with another test. The risk of side effects, especially in distributed/parallel testing scenarios is just too high. You'll regret it if you rely on this. Ask me and Jeremy Miller how we know this.... (ie, school of painfully learned hard knocks.)

2. Is it appropriate for the browser tests to communicate directly with the database of the system under test, creating state in the same manner that you would in an integration test?

I generally prefer to use the system under test's own APIs to create state for a test. These APIs ensure CRUD operations are obeying the system's own rules for its data. Using APIs means you don't have duplicated logic around CRUD ops for your tests, which is a good thing. This way you're not having to maintain separate helper methods to handle changes when the app's underlying DB structure changes, for example.

3. How important is it that browser tests be decoupled from the application under test, such that the only thing required to run tests are the tests themselves, which can be pointed at any applicable URL (dev, test, staging)?

There are a couple different aspects to this, IMO. You want your test suite coupled to the application in the sense you need to leverage the system's own APIs for setup, teardown, configuration, and parts of your test oracles[1]. That said, these calls to the system should be abstracted into a set of helper libraries. The tests themselves should never, ever know how to communicate with the system itself.
For example, you don't want individual tests that know how to invoke a web service endpoint to create a new contact in your Customer Relations Management system. If that endpoint changed, you'd have to touch every test that used that endpoint. That's a recipe for a lot of extra scotch consumption.
Instead of telling the system how to do something, tests should tell a helper function/library what they want done. That library in turn knows how to call a stored procedure to create a user. With this approach, no test ever has to be updated if, for example, the system changes from a stored procedure to a web service for creating new contacts.

Write Tests Like You Write Code

I've generalized a number of things in the above thoughts, but the bottom line is this: use the same ideas approaches for your test suites as your production code. Because tests are production code.
[1] For automated tests, an "oracle" or "heuristic" is the final step in your test after you validate the UI is behaving as expected. It's not enough to leave the test off at that point. You need to check the database to ensure items were properly created, updated, or deleted, for example. You might have an oracle that checks the filesystem to ensure a datafile was properly downloaded.

TIL: Arrowhead Anti-Pattern

Today I Learned: The term “arrowhead anti-pattern” from Jeremy Miller’s tweet. (If you’re interested in software craftsmanship and you’re not following him, change that.)

The “arrowhead” describes the visual pattern made by a nasty set of nested conditionals:

if
   if
     if
       if
         do something
       endif
     endif
   endif
 endif

The above snippet was lifted straight from the great article on the C2 wiki. If you want to take the visual aspect a step further, go see the Daily WTF’s article Coding Like the Tour de France.

Nested conditionals are awful. Avoid them. Read up on cyclomatic complexity and learn to handle things differently in your code. You’ll be happy you did. (See Chris Missal’s article on Los Techies for some other good discussion.)

I’ve long known the troubles this sort of code causes. I just didn’t know the cool name for it.

Learn something every day…

Dear Jeff: Good. For. You.

Jeff: You don't know me, we've never met, but you've run across a nasty spate for your post on how men can help address some of the awful things that happen in our industry. I wanted to add my voice of support to you.

The diversity movement, or whatever that movement names itself, is trying to bring some much needed change to our world. Good for them. Some of their leaders have suffered some horrible things in life, and they've got some powerful stuff they're trying to spread awareness of. Good for them too.

That said, I'm incredibly saddened at the venom and bile you've had thrown at you. Too many in the diversity movement have no room for any voices or any views other than exactly what's put forth by those leaders. Ironically, there's no tolerance for diversity of views or voices.

Worse yet, there are some in that movement who explicity want white males to have no voice whatsoever. They'll couch it in fluffy phrases like "people with power should restrict their speaking to amplifying the words of those without" but at the core they're demanding to stereotype and marginalize a group of people while trying to uplift others who've been stereotyped and marginalized. In what twisted universe is that even logical, respectful, or even close to right?

I'm especially saddened with the reaction you received because you made some terrific points which were shouted down or ignored. Looking to Martin Luther King as a model instead of (literally) screaming profanities at people? How's that wrong?

Emphasizing the need to start early in life and focus on kids? We need more people with your reach talking about this. Asking people to line up with Hacker School Rules and calling out guys for bad behavior? YES YES YES!

Yes, you should have done a few things better. I do wish you'd referenced Shanley Kane's post from the get-go. It's obvious she gave you some bit of inspiration or motiviation. I really dislike her tone and approach, but I think it's important folks know more about her--even if for no other reason to say "I can't agree with her approach, but there's a few things to be learned."

At the end of the day you were adding your voice and thoughts to raise awareness in a horribly sensitive, messy, human problem area. My good pal Leon Gersing has a number of amazing presentations he's done in front of thousands of people. One of his best thoughts is something in the lines of (paraphrased) "You don't need anyone's permission to do what you feel is right."

I'm glad you wrote your post.

Good.
For.
You.

Webinar Recap: EuroStar 'Four Tips for Web UI Automated'

Today the great folks at EuroStar conferences graciously hosted me for a webinar. I walked through my conference session "Four Tips for Web UI Automation.”

EuroStar hosts a post-webex discussion at TestHuddle. You can find discussion around my talk in the appropriate forum there. I'm told a recording of my session will show up at that link shortly as well.

As always, my slides are hosted on SpeakerDeck, but here's this deck embedded.

My New Job: ALM & Testing at Falafel Software!

Today is my first day at Falafel Software. I’m extraordinarily happy to be on board with Lino and his great crew of people!

I’m Falafel’s new Vice President for ALM and Testing. This means I get to focus on things in my wheelhouse: helping organizations and teams improve their processes, not just around testing, but across the entire delivery chain: envisioning, prioritization, building/testing (THEY’RE THE SAME THING), and final delivery.

A major part of my work will continue to be around test automation, but I’ll be broadening out to help teams figure out what tools best meet their needs: WebDriver, TFS, SmartBear, Telerik, etc. And of course, in line with my general philosophy, testing is NOT just about tools.  Ergo, I’ll be helping teams understand where automation fits in and doesn’t. I'll be helping teams build up their testing skills, not just automation skills.

This is a great opportunity for me, and I’m thankful to the Falafel team for bringing me on. It’s going to be an exciting role!

(Are you or your organization looking for something alongthe lines I mentioned above? Ping me and let’s see if I can help! Jim AT Falafel.com)

Audio Gear for Geeks

I spend a lot of time in my day job building up various recordings for videos. Having high-quality sound for videos makes a huge difference with your audience. As a very experienced podcasting pal once told me, “Don’t give them an excuse to turn your stuff off.” He used a word other than “stuff.”

In The Beginning

When I first started out in my evangelism (now developer advocate) role, I used this Logitech H390 headset for my recordings.  It’s a great headset which does a fine job with audio. The quality was completely acceptable. I still use this headset for all my webinars, webexes, online meetings, Skype calls, etc. It’s comfortable and the inline mute/volume switch is awesome. Plus at $28-ish it’s an insanely great value.

 

 

Upgrading

Nowadays I’m using the setup below, and it’s getting me awesome results:

Microphone: The Audio Technica AT2020 USB condenser mic. It’s just plain awesome. Carl Franklin gave this a big thumbs up as he used this extensively in his professional sound studio for years. This mic is roughly $100, but it’s an incredible value. I know folks who are using $300 mics for podcasting/video recording and their audio doesn’t sound any better than what I get out of the AT2020.

The best thing about this mic is its utter elimination of echoes in my recording room. My home office is a small room with extremely nasty echoes from the hard plaster walls. With earlier recording devices I’d tried all kinds of workarounds, including (honestly!) recording while hiding under a blanket draped over my workstation. The AT2020 scoffs at echoes in my office and gives me great, clean sound.

Shock Mount: A “spider” mount isolates the mic from the boom. This Samson SP01 mounts on the boom listed below and holds the mic in a cool web. Vibrations can’t pass the web mount. Neat. (Insert trite Gandalf “THOU SHALL NOT PASS!” joke here.)

 

 

Boom: You know you want more boom in your life. Get some. A mic boom helps isolate the mic from noises and vibration, and it’s great for swiveling around so I can record while futzing around between different systems and sitting positions. It’s surprising how big a difference this “simple” gadget made. This one’s a Rhode PSA1, and it comes with a couple different mounting options: a clamp for your desk and also a more permanent threaded nut/bolt arrangement if you have a hole available in your table. (Or pull out a drill…)

An additional benefit from the boom: I can swivel the mic down enough to get nice recordings from my acoustic guitar. (No, I am not sharing those! )

Pop Filter: I got one because I see all the cool-looking videos of recording artists and actors in ADR. OK, actually I got this Nady 6”  pop filter because they really do help out with eliminating harsh sounds while you’re talking. I’m not sure how it works; I just know it does.

This clamps on the boom and has a nice flexible arm to position it around. I found the filter arm tends to droop a bit, so I use a small velcro strap to help hold it in place.

Putting It All Together

I tried getting a shot of my setup in the home office, but I couldn’t get anything I was satisfied with. (And I actually even cleaned my desk.) All this stuff goes together quite easily, and the swivel lets me keep the gear out of my way when I’m not recording.

You’ll spend a couple hundred dollars on all the gear above, but it’s really worth it if you want to step up the quality of audio you’re creating.

So Long, CodeMash!

After eight amazing years it’s time for me to step aside from CodeMash and let others bring new vision and energy to CodeMash. As of last Friday I’m no longer the President of the Board. I’ve stepped down, and Brian Prince is taking over.

It’s been eight wonderful years of working with amazing, tiny crew. We started off year zero of CodeMash with something around 220 attendees, speakers, and staff. I used “Almost 300 attendees!” as marketing schtick when trying to pimp the next year.

Oh how we’ve grown!

This year we had 220 family members signed up—the same amount as the first year’s total attendance! KidzMash (something I dreamed up with Jason Gilmore while on a short phone call during one of my long commutes) has grown to the point where they need two rooms and were nearly overflowing during the raffle.

This year there were roughly 2,000 attendees, speakers, and staff in attendance, and Jon Skeet’s morning talk in the main ballroom had more folks than either of our first two conferences!

CodeMash has been an incredible success for many, many reasons:

• An wonderful venue whose staff are truly partners, and have become part of the CodeMash family
• A tiny core of five to eight core organizers who are close friends. We succeed by absolute delegation and total responsibility for owing every detail of execution on our ideas.
• A ruthless focus on keeping to our main ideals: kickass content on a broad range of topics. We constantly deflect neat ideas that don’t align with our mission.
• Content selection committees that have sifted through hundreds of submissions each year to distill out amazing material. We’ve had between 600 – 800 submissions the last five years and that’s an incredible amount of work and stress to deal with!
• World-class speakers who donate their time. (CodeMash’s financial model doesn’t enable us to cover travel expenses or stipends. Speakers are losing money while at CodeMash, although I’m proud that unlike many larger conferences we do cover speakers’ hotel nights.)

CodeMash isn’t unique in those things; however. There are larger conferences that have professional full-time staff. They plan and execute their conferences flawlessly with a great lineup of content presented at cool venues. What makes CodeMash truly different?

THE ATTENDEES.

We’ve been blessed with a core of amazing attendees who bring great passion, true passion unlike the asinine “passion” marketing blabberspeak facile crap, to the conference. Attendees who spend as much time on the couches in the hallways talking in tiny groups as they do in breakout sessions with 120 folks. Attendees who honestly care about productive, not divisive, discussions with geeks who live and breathe in different platforms. Attendees who take what they’ve learned at CodeMash and return to work and life re-energized and willing to try a few new things.

It’s YOU who make CodeMash so great.

Thank you for all you’ve given me. It’s dwarfed what I’ve given.

So long, and thanks for all the fish. Or bacon.