Meaningless Metrics – Combined Experience

From time to time, I run across a vendor who says:

We have over 40 years of combined experience…

What does that really mean?

The intended meaning is that, if there are 4 people in the room, they each have about 10 years of experience.

But, it could also mean that one of them has 39 years of experience, and the other three have been in the business for only 4 months!

Further, let’s say that this particular vendor is an AI (Artificial Intelligence) consultant – why would I want 40-year-old advice from the mid 1970’s, when the largest computers of that era didn’t have as much computing power as my smart phone?

Senseless vs. Meaningful Calculations

Essentially, we have a set of scalars that are supposed to represent the respective sizes of each individual’s experience base within the group.

Let’s say we have 4 people:

• Timmy:  12 years of AI experience
• Stacy:  15 years of AI experience
• Johnny:  8 years of AI experience
• Alice:  10 years of AI experience

This gives us a set of 4 scalars:  {12, 15, 8, 10}

Although there are many valid ways to compare and combine these numbers, there are also very many ways to combine them, that don’t make sense.

At the end of the meeting, Stacy proudly proclaims, “we have 45 years of combined experience“, because she added all of these scalars, but what does that really mean?

It’s not like the team are simply ONE person who becomes eminently more qualified with combined magnitude.  Take the case of our most junior member – in theory, if we give Johnny another year of experience, he still sits within the footprints of all three of his other team members.  The team’s range of experience is really based on it’s most senior member (Stacy).

It’s not like there is some kind of historical significance, as if getting to some magic number of combined experience qualifies the team for an historical marker.  If they make it to 100 years of combined experience, they can’t proclaim “experience since 1917!”.

Talking about a set of numbers that each represents a constant value is just like combining height:  Unless you plan to have them stand on each others’ shoulders, or make them lay on the ground end-to-end, the “combined height” would be completely pointless.  The statement, “we have 22 feet of consultants visiting us today…” just doesn’t make any sense.  Nor would it make sense to say, “it took 600 pounds of consultants to fix this problem”.

Instead, if we understand that we have a set of 4 people, and each scalar in the set represents ONE of the four people, we can come up with some meaningful metrics by comparing rather than combining:

• The team has an average of 11 years of experience (indicating a consistent qualitative aspect to the work product)
• The team has an experience base of 15 years (perhaps the senior team member has hands-on experience with older, but still practical technologies that are only taught in textbooks today)
• Everyone here has between 8 and 15 years of experience  (Even the most junior member has a solid experience base, and practical knowledge)

Likewise, if we’re talking about numbers that reflect a rate, such as salary or billable rate, it might make sense to say, we had four consultants come in for a meeting.  Their combined billable rate was $800/hr, and the meeting lasted two hours, so the cost of the meeting was $1,600.

Conclusion

When you use math to compare or combine a set of numbers, ultimately, you have to maintain perspective about what those numbers really mean.

If you combine numbers in ways that don’t make sense, you might create a meaningless metric.