September 1, 2021

Lesson 7: Quality Assurance

Lesson 7: Quality Assurance

If you’ve ever attended a rodeo, you know the highlight is the bull riding competition. The crowd greatly anticipates the moment when bull riders mount up on the back of an animal filled with murderous rage. When the signal is given, a gate opens and the bull leaps into the arena where it spins around and bucks violently while exclaiming through angry snorts, “Get of my back! Get off my back! Get off my back!” The bull rider tries to stay on the bulls back for 8 seconds.

As a PM, you are the bull rider.

The bull is your product.

Opening the gate is your release process where you desperately hang on and hope that nothing catastrophic happens.

Everyone holds their breath.

As you build your PM career, quality will be your greatest conundrum. If you embrace quality, everyone will hate you for not getting anything done. If you ignore quality, everyone will hate you for not having anything the market can use. Regardless of what you do, everyone is going to hate you. You will then embrace the fact that you’ve made a terrible career choice…

…it ain’t easy.

I remember a specific job interview where my potential new boss was an actual user of the product at my current job.

“Oh, I use that product,” the potential new boss said as I tried to hide my embarrassment and shame, thinking of how I was going to weasel my way out of this one. “I’m having a really hard time logging in.”

“I, uh, only work on the part after the login,” I stammered, realizing it was the dumbest thing I had ever said.

I didn’t get the job.

What was I supposed to say? The truth?

  • My boss is shortsighted and demands everything be done within an unreasonably time period.
  • Our development environments are broken and nothing can be deployed reliably.
  • Our QA manager is a pot-smoking hippie whose only concern is the quality of his weed.

None of these matter. As long as you have a product in the market, a product manager is judged by quality and nothing else.

Yet ironically, to get a product to market, you gotsta get paid. And usually getting paid is the result of a mandate like this from an executive:

“We finally have a paying customer but they think our product is real and I told them it would be ready next week.”

Uh oh. Now your skills as a Product Manager are under pressure. Do you have enough clout to push back against the executive? Do you have enough goodwill with the dev teams to request a ridiculous favor? Do you have enough PTO days to go on vacation and get away from this problem?

It always ends the same way.

By executive mandate to make this happen, you now must bribe some engineers to hack up something over the weekend. Yes, it makes you sick to compromise and take shortcuts. Yes, It makes you sick to think of the unhealthy amount of Monster Engergy Drinks you are about to consume. But what makes you most sick is that you will be dealing with this decision for the rest of your now pathetic days at this job. Sure, you try to find consolation by telling yourself this is only temporary and you will “productize” it later…

…you never get the chance.


Because more client promises are made and more features are built on this compromised solution. The next thing you know you have an entire product and business built around what was essentially a weekend hackathon fueled by caffeine and self-loathing. And that’s your foundation of quality for the next 10 years.

Every profitable software company in existence shares this exact same story.

The most ironic thing in all of this is that there will be someone in your company with the job title of QA. These poor people. They are essentially the equivalent of pedestrian signals in New York City. Sure, they exist as warnings, but it doesn’t keep you from crossing whenever the hell you feel like it. Cars honk, taxi drivers swear, but you march forward in oblivion.

“We strongly advise against this product release due to this list of dozens of coding and user experience issues we found,” the QA team tells you the night before a release.

“Are any of them showstoppers?” you ask.

“On Android, a hellish demon is released that eats the soul of our user's first born child?” QA says.

“I’m sure it’s a low percentage that will be impacted. Seems like an edge case,” you say as if you are the judge of your customers’ eternal destiny. “Besides, most of our customers don’t believe in demons anyway,” you justify.

Whether it is a soul-sucking demon or a minor CSS issue, you are the one that makes the decision to release an imperfect product…

…after all, all products are imperfect, so it’s your call, the industry-appointed CEO of the product, to determine the threshold of imperfection.

This means you must put on your chaps, mount your product, and ride it into the open market, hoping to stay on longer than 8 seconds…

…Here we go…

…Ouch. The bull knocks you off quicker than expected.

“Who gave the green light to release this broken feature?” The services executive barks. Every eyeball in the room turns your way and pierces you with accusation.

It’s true. You gave the green light after smoke testing the most obvious happy path on Chrome. What you forgot is that none of your customers use the software as intended and your most critical clients inexplicably use a version of Internet Explorer that not even Microsoft supports anymore.

“Let’s take this offline,” you say to the angry exec.

“That’s fitting, because due to your incompetence, that’s exactly what our application is: offline,” the exec replies.

You then schedule the obligatory post-mortem assessment where you try to convince everyone how valuable this experience was as a learning opportunity…

…It doesn’t work…

…your job is once again on the line…

…the single wringable neck.

…it ain’t easy.

When code inevitably breaks and fails in production, you learn a lot about your organization based on who takes the blame. Of course, in good product organizations, the team works as a cohesive unit, accepting blame together and fixing the solution in unison. Good teams, spearheaded by a strong Product Manager, know it is not one person’s fault. It takes the entire high performing team to produce quality products.

In bad product organizations, the fingers start pointing. Designers didn’t get the designs done on time, quality assurance is lazy, product managers didn’t write enough documentation, engineers play too much ping pong, and finally everyone rallies around a common message of leadership being incompetent.

Unfortunately, in the blame game, PMs become the easiest scapegoat. It’s very easy to notice your lack of details in story descriptions.

“The PM didn’t tell us we needed to account for demonic Android soul-sucking,” the engineers say.

You then try to defend yourself by saying that it should be common sense to not release devilish fiends into the world. But at this point, common sense no longer applies. You all have a heated debate about whose responsibility it is to define and write quality code and inexplicably everyone looks at you again, the least qualified person in the room to have any jurisdiction over code and the quality thereof…

…You then kick off a series of meetings to define the definition of done…

…which usually means you must document every business case, use case, test case, and marketing case in exquisite and excruciating detail…

…You’ll then wonder how you got this deep into the bowels of hell…

…it ain’t easy.

The truth is that we all know the answer to quality. It only requires 2 things: (1) Quality must be at the top of company values and then (2) you must build teams around people who give a crap.

On the other hand, if you don’t have good motivated people then by all means, come up with a rigorous process that treats people like factory cogs and completely removes any essence of personal motivation, pride in work, or crap givingness.

So what’s the lesson:

The lesson is that only great people can create great products and high performing teams will only release quality products. As the Product Manger, it is not your job to define done. It is your responsibility to lead product teams so that they own it together, collaborate together, and produce valuable quality products together.

In these conditions, even though every product release will continue to feel like opening the chute and riding a chaotic bull, at least you’ll have a high chance of putting on a good show for the crowd and winning the competition.