Adding up the last two years of Forrester research into the product management/product marketing role in the tech industry, it's easy to see why it can be hard to hire a good PM. Many of the skills that define successful PMs aren't easy to detect in a few hours of interviews. Since these traits generally don't reveal themselves until the PM is on the job, perhaps you should simulate the job in the interview.

While many technology vendors have a muddled idea of what PM success looks like, they usually have some notion of the outcomes they don't want to see. Can't build good working relationships with Development? Black mark. Can't say no to a sales enablement request? Black mark. Can't manage competing inbound and outbound priorities? Black mark. Can't give a detailed description of the specific roles to which we're marketing and selling? Black mark. 

And the list goes on. The candidate sitting on the other side of the desk might have done an outstanding job, as it says on the resume, launching products at various companies. But what does "launching products" mean, exactly? Did the candidate keep the project on track, or was he just someone in the congratulatory photo op at the end? When you're interviewing a developer or QA person, you have a pretty good idea what their contribution was. The exact contribution of PM can be a bit more obscure, though certainly no less important.

To make matters worse, what worked in one team might not work in another. In a previous job, a product marketer might have had a jolly time working with the corporate marketing people. In this job, where corporate marketing has a different corporate personality, and a different set of challenges, the product marketer in question might not have a clue.

Stock interview questions can go only so far. At a certain point in the interview process, it's time for a little role-playing.

I'm a big advocate of game-like activities in many business settings, and role-playing has direct relevance to this hiring conundrum. If you're concerned that the PM won't be able to manage a huge stream of sales enablement requests, create a 15-minute scenario in which the candidate has to tell a real salesperson (recruited for the interview) that it may not be a good investment of time to fly across the country for one customer meeting, no matter how strategic that customer is. Ditto for dealing with the development team: Give the candidate the task of dealing with an ornery engineer's complaint that the requirements documents are useless.

While the official point of this exercise is to test the candidate's mettle, there's also an unofficial objective: getting the people with whom the candidate will work to invest more in the success of anyone who takes that job. Role-playing exercises reveal a lot about everyone who participates, not just the test subject. For example, after one round of interviews, an ornery engineer with whom I worked admitted that he wasn't terribly clear what, for him, constituted "useful" requirements.

Role-playing takes more preparation than the typical interview session. Therefore, I would limit its use to the "short list" candidates whom you bring back for a second round of interviews. It's your choice whether to warn the candidate in advance. (Heh.)