The Value of Specializing Generalists

I've never claimed to know all the answers.  Actually, I'd be happy if I could answer more questions than not...  That said, if I can't answer a question, I generally know right who to reach out to for answers.

This comes in to play time and time again in our line of work, particularly since we're expected to be able to solve so many different problems in so many different technologies/platforms/whatever.  I ran in to this recently with a performance issue and was able to solve part of the issue by reaching out to Bruce Lindman, the senior DBA we have on staff.

I'd isolated some connection timeout issues to a query that was taking almost four minutes to return.  The query was built to handle sorting out some permissions across a large set of data and was working with a lot of records from a master table and association table.  The developer who had originally written the SQL had broken out the sproc into several different functions in order to modularize it for testing and for clarity.  Everything looked pretty good, and all of our moderately complex use cases passed just fine in the test environment.  Unfortunately when we rolled in to the test environment at the client site we found the really bad timing issue.  Ick.

Bruce immediately identified the problem: the user defined functions were returning tables of result sets -- and those tables aren't indexed.  Large amounts of data for intermediate results. No indexes.  Small problem there.    Bruce spent 30 minutes or so rewriting the query to bring the functions back in to the main sproc, tweaked a few things, and ended up cutting the query from nearly four minutes down to 22 seconds.

That's a great improvement in a technology area that I'm, well, frankly, sucky at best.  Being able to call in a specialist let me focus on a couple other issues which were also contributing to the performance problem.  Those other areas (overly heavy biz objects, pooling issues) were in my area.  SQL complexities aren't.

Our industry's grown too broad for us to be experts at everything, even if clients expect us to be.  I think all we can ask for is that we form groups or teams of specializing generalists: folks who are very solid at a wide range of skills, but experts in one area.  That lets us have a cadre of resources to reach out to when we're in over our heads, which happens more often than we'd like to admit...

UPDATE:  One blog reader, blocked from commenting by an evil corporate proxy, pointed out a great article by Scott Ambler on this same topic.