[Last in this series about the Tech Product Management/Marketing Job And Department Profiler. Previous posts cover its origins, the design decisions that went into it, and the job specializations it profiles.]

From my grad student years, I know that a lot of mathematical models fall flat on their simulated faces because they don’t get the testing they should. In the case of the PM job calculator/department profiler, I had to apply three different tests.

The tasks are many and varied
Last year, when I did the survey research on the state of the product management/product marketing job in technology companies, I vetted the list of tasks with a lot of PMs. Using their feedback, I added tasks, rethought others, and even merged a couple. The resulting list represented everything that PMs might be doing, depending on where they work.

That was the easy part.

The tasks define the jobs
A major goal of the Profiler is, as I explained in an earlier post, classifying PMs according to their specialization. A requirements-focused PM is not a product manager, who is not a release manager, who really isn’t a product owner, who can’t find the spare time to also be a community manager…You get the idea.

Consequently, the next test involved showing these job categories—defined by the tasks these roles perform, and the priority each task receives—to people who run PM departments. That group included the heads of product management teams, product marketing groups, and combination product management/product marketing departments.

Since no one’s job description for any of these roles exactly resembles anyone else’s, the goal wasn’t to find perfect agreeement among job categories as they’re defined in particular organizations. Instead, I did a version of “ideal type” analysis: does this definition offend your sense of what’s essential about this role? Over the course of several conversations, I managed to develop a set of ideal types that everyone found more or less acceptable.

The jobs define the department
When I started this project, the stretch goal was finding a way of using the job-level results to help define what mix of people you might need at a departmental level. Surprisingly, that stage of the project was easier to complete than the job descriptions.

Perhaps that’s because every senior director or VP of a PM organization focuses first on the type of person needed to perform a job, steps back a bit, and then figures out how many people, at what level of seniority, need to appear in the group portrait.

The most important choice was, which variables mattered? There are dozens you might consider, from the type of product to the size of the company. While all of these variables have some influence, they don’t all have the same weight in determining what kind of team you need to build.

Once again, tasks proved to be the right starting point for this project, for a variety of reasons. For example, the degree to which someone can focus on their core responsibilities has a definite impact on how many of those people you need.

This last stage of validation revolved around the question, “You put this information into the black box, and out comes this picture of the kind of department you need. Does this picture make sense?” The results had to be good enough, since a perfect calculation is probably not possible. (Again, many variables have relevance, but it’s necessary to focus on the ones that matter the most.) Perhaps not all of these people in this hypothetical department, in practice, report into the same group, but they have to exist somewhere in the company to provide adequate support and leadership for the product or service under development.

Don’t let the perfect be the enemy of the good
Even though perfect accuracy in the prediction of staffing needs—how many, in which roles, at what level of seniority—may be impossible, the “good enough” picture has definite value. For example, during the validation discussions, some PM department heads were interested in showing the results to their management to (surprise!) justify their staffing needs. Others spent time discussing the things they hadn’t considered, such as tuning their organization to better support the sales team. Someone else spent a long time talking with me about where to shift particular tasks, if you take them off the shoulders of PMs.

In other words, the Profiler provides a starting point for a discussion about organizational changes. Without it, you might not be able to have that discussion, since the proverbial devil is truly in the details. Making a particular task, such as maintaining the product roadmap, a higher or lower priority affects other PM responsibilities. It also begs a lot of questions about skills, time management, deliverables, and compensation.

However, if you didn’t have some kind of detailed portrait of what PM does and doesn’t do, it’s difficult to get to these second-order questions, such as professional development. If you don’t force yourself to be specific about PM priorities, then there’s always a case for something else to get added to the to-do list. Unfortunately, that often leads to the situation in which, if everything is a priority, then nothing is.

[Cross-posted at The Heretech.]