In a job market rich with highly qualified applicants, CIOs use on-the-job testings, unusual questions and users to hire more than technical talent.

Robert Norris, CIO, Pinnacol Assurance

Test application ability

The traditional assessment of skills and experience often gets you a ton of folks in the door who know what they're doing, but can't always actually apply it. Once we've done that initial traditional filter, we move on to an in-depth problem solving and analytical assessment. We spend the better part of a day on this in order to do more than just check a box for "can apply skills."

We put candidates directly into the work environment with our teams and give them hands on challenges, such as solving problems on the help desk or building part of a new application. Cultural fit is also important. A final assessment is conducted in which a candidate is interviewed by a panel of all members of IT management. We make sure that the candidate will be a good fit within the team and will interact well with IT's customers.

So far, this has had a terrific payoff. We have been taking this formal approach for more than a year now, and the new employees we've hired have been outstanding.

Mary Finlay, Deputy CIO, Partners HealthCare System

Ask unexpected questions

I like to go beyond the standard "challenge" questions during the interview. One I always return to is about integrity. I ask people to provide an example of when they have been personally or professionally challenged, and to describe how they responded. I don't frame the question as whether they've ever been confronted about a particular issue. What caused the challenge of integrity is really less of a concern than the candidate's response to the situation, to the person doing the challenging and the effect on the relationship.

It's illuminating how many are thrown off by that question. I take that seriously, even if they seem like they would fit in here in other ways. The answer, or lack of it, tells me a lot about the person and how they handle adversity and relationships. When you get past all of the technical requirements, having an idea of how someone believes they will react in a charged situation like that can be the differentiator in finding the right people for your organisation.

Brian Lurie, VP of IT, Stryker Orthopaedics

Use customers as screeners

In traditional IT recruiting, you often look for two types of professionals: technical specialists to complete project work and liaisons who interface with clients. At Stryker, we want that specialist to be the liaison. We want everyone, even our database administrators, to be able to answer questions if they run into the client in the hallway. Because of this high standard, we probably see more candidates than other organisations.

The final step of our evaluation process is for candidates to talk not only with peers and IT management, but also with the clients with whom they would be working. We include clients in the hiring process in order to start a partnership before the candidate is hired. It's the strength of those relationships that helps make our IT professionals successful. It's no longer "us and them", we're business partners.