I’ve developed several products that were initially sold to directly to individual users that I then transitioned up-market.

We went from offering a straightforward product wedge directly to users to working directly with larger entities that controlled the activities of thousands and tens of thousands of users.

It required a lot of work. Our sales team made promises - metaphorically writing checks we didn’t have money in the bank for. We lost several customers after they came on board and were disappointed with the state of the offering.

What went wrong? Turns out, we under-estimated the amount of effort it would take.

The enterprise value proposition

To actually move up-market, your product needs to have some value proposition that applies to the enterprise - the larger organization that sits above your actual individual users.

What are you giving the organization purchasing your product?

• Greater control? Do they get more administrative tooling, policy enforcement levers, or access controls?

• Greater visibility? Do they get reports across multiple users, actionable insights, or usage?

• Greater interoperability? Do they get the capability to integrate with other systems they use, such as login providers, SIEMs, CRMs, or ERPs?

• Greater scale? Do they get access to capabilities useful at scale, such as automations, faster resources, or SLAs?

Is it surprising these don’t sound like anything related to your core product? Remember that this value proposition can be quite distinct from your actual product offering, or may not even be related at all.

A lot of product managers get this wrong and imagine moving upmarket as giving more advanced, broader, or deeper features of the wedge, when in reality the purchasing decision may be made because of the color of the doorknob.

I once worked with a potential customer who loved every single capability of our product, except for the fact we used Amazon Web Services to host our offering. They were a retail organization and as a result had internally banned usage of Amazon, whom they viewed as their largest competitor. To sell them on our offering, we had to explore transitioning their instance to Microsoft Azure - a completely irrelevant and useless detail in the grand scheme of what we offered, and a lot of work to boot.

Your product will change

When you “go enterprise”, you’ll likely end up incorporating some elements of the below into your product. Enterprise product functionality that companies eventually expect include:

• Account management

• Account provisioning

• Analytics

• Auditing

• Authorization

• Billing

• Branding and white-labeling

• Configurations

• Cost projection

• Custom domain names

• Data interchange options / integrations

• Data residency, lineage, and ownership rules

• Email control

• Exports

• Infrastructure

• Ingestion

• Invoicing

• License management

• Reporting and insights

• Security controls

• SOC2 or other compliance (eg. WCAG, PCI)

• SSO

• Team member management

• Webhooks

These represent the iceberg that is enterprise products. It’s not just offering a tool or wedge to “process a payment” or “streamline a process”. It’s offering that on top of a cohesive platform. In effect, your product offering becomes your meta-product.

Your org will change, too

There’s also enterprise capabilities that your organization will need to be able to support and operate via various programs:

• Completing RFCs, RFPs, and RFIs

• Supporting unique billing such as invoicing

• Multi-tenancy and data segmentation

• Monitoring and reporting, service SLAs

• Disaster recovery, including failovers, backups, RPOs and RTOs

• White-glove service, account management, escalation paths, response SLAs

• Support plans and roadmap influence

• Legal ownership of intellectual property ad data ownership

• Compliance with laws and regulations, both real and perceived

• Change management, including notification

These need to be defined, operationalized, documented, and facilitated for customers and prospects. It also increases internal coordination needs as well - for example, you wouldn’t want a Salesperson committing to an Uptime SLA that the Engineering organization couldn’t meet.

The secret - you may not need all of these

The dirty secret is that some of these may be absolute requirements to sign the contracts, but ultimately doesn’t matter to the customer after contract signing. They just needed to check a box or meet some internal political need.

What you need to commit to and what you actually need to implement can sometimes be determined by people who weren’t part of the contracting and purchasing process at all - the operations and implementation team on the customer’s side.

While the purchasing team may happily be operating under the assumption they received everything, sometimes only a few capabilities end up getting used operationally.

It’s important to identify and prioritize the capabilities that are actually used, and find wiggle room in the ones that aren’t actually important. Be up-front and really dig in to why something is being asked for.

Sometimes you have negotiating leverage to strike away an item. Other times you can get a feel during negotiations that an item isn’t actually all that important post-signing.

Don’t lie to the customers - that’s a fast way to lose all reputation and irreparably harm your organization plus open yourself up to lawsuits. However, you may be able to find an alternate, simpler solution that mutually fulfills the customer’s internal requirements even if it doesn’t meet the letter of the contract while allowing you to spend your investment allocation chips elsewhere. Working directly with your customers on finding these alternate approaches can be in everyone’s best interest.

I once spent several months developing an enterprise-customizable role-base access management system to enable an enterprise prospect. When they finally started using the product, the overwhelmed operations admin couldn’t make the time to learn how to configure it, and asked our internal account management team for help. We set it up internally, and they were quite happy, and never changed the tool again. What was initially a critical contract piece ended up never being used by the customer once they started using the product.

Operationalization

Of course, not everything can be deprioritized. At some point, you’re going to have to actually deliver on the capabilities you agreed to. That’s where you have to recognize that it’s not just about the Product.

Enterprise capabilities are just that - enterprise capabilities. They cross functional boundaries and impact almost every part of the organization.

You have to work with many, many different functions to deliver successfully on the contract. This means playbooks and runbooks for various processes that multiple parts of the organization participate in. Documentation is

key here, as is training and pro-active facilitation of any new process: you can’t just release a new process and expect it to be followed.

Even something taken for granted such as “onboard a new customer” may have a surprising amount of nuance and complexity that involves:

• Reviewing the contract terms

• Provisioning the account according to those terms

• Configuring the account according to the customer’s specific needs

• Training the customer on how to use the product

• Answering questions from the users

• Coordinating with marketing for announcements, if any

• Monitoring usage to encourage adoption

• Scheduling and developing any committed customizations

• Starting billing and invoicing

• Preparing tier-1 support teams for new questions

Things can start to fall through the cracks as communication and steps occur between roles like account management, sales, solutions, legal, marketing, finance, support, security, and product. Without effective tracking, checklists, and process, you’d end up with an inconsistent, non-repeatable customer experiences with a lot of key steps being missed.

Multiply that by the number of other things you have to do to support enterprises, and you start seeing why the product itself is just the tip of the iceberg.

Larger companies have dedicated people for each of these steps, but in smaller companies it may be a single person performing all of these steps on top of their day job. At prior small companies, I’ve had to do all of the above on top of actually developing the product!

To move to supporting enterprises successfully, you need strong relationships with all of your peer functions and disciplined facilitation and execution. No customer will remain with you if you string them along or break promises. Think deeply about not just the product, but how the very act of selling to a new kind of customer changes how you operate at the company.

Product Managers are responsible for proposing informed changes to products. They propose these changes to drive towards an outcome that is hopefully beneficial to the company.

I have worked with many Product Managers who made decisions, but lacked intentionality. Their proposals were arbitrary and seemingly random. They often didn’t have the desired effect, or caused significant issues.

A lot of this came down from the fact that they did not know what they were doing - quite literally. They didn’t take the time to truly think about what they were attempting to do, and therefore wasted significant time and effort of teams of developers.

When they did have wins, it often amounted to sheer luck due to some outcome or interaction they didn’t even think about. Sometimes progress was made in spite of their contribution, thanks to the hard work by the team to fill gaps.

What they were doing wasn’t product management, it was product gambling.

The best product managers I’ve worked with had intent behind every decision they made. Right or wrong, they could articulate and trace their thinking through some logical path, effectively identify their assumptions, perform validations to de-risk their plans, and rapidly incorporate feedback to self-correct.

It doesn’t mean they were always right - product management is a job of probabilities, not certainty. However, they were right more and more often as time went on, and even when they were wrong - they could course-correct to be eventually right. As a result, their wins compounded and their losses had minimal negative impact.

The Intent behind Product Management

To be intentional, you need to at least know your train of thought that led to the conclusion you had. When you develop a proposal to change the product, you should be able to articulate the train of thought, and take people along that journey.

• Discover and define your problem space

• Define and propose an approach

• Form a hypothesis

• Articulate your assumptions

• Validate your assumptions

• Decide and act

• Monitor results and outcomes

Note: if your process is “make proposal → see results”, then you’re just throwing spaghetti at the wall. Any random monkey can do that - that doesn’t add value to the team.

Discover and define the problem space

Discovery can have a lot of different sources - from feedback to user interviews to hunches - you become aware of the existence of a problem.

Once you have a general awareness - you want to define it.

You can do that by first articulating the theoretical framework for how your problem space ladders into your company-level goals. This problem space might ladder up into a variety of other spaces that ultimately feed into your bottom line - typically revenue or some other business consideration.

An example

Suppose you have a hunch that users aren’t using the search bar.

You might’ve heard an off-hand comment, or maybe you saw a bit of a strange delay from watching a new user navigate the system. Perhaps you overheard a support agent talk about some convoluted filtering a user is doing that is more complicated than searching, and that set off your product instincts. Maybe you saw unexpectedly low usage in analytics.

Whatever the cause of the hunch, you now have a problem space - users not searching.

The next step - you can ladder up this problem space into a higher-level goal:

• users not leveraging searching → opportunity to improve user efficiency → improve user time share spent on impactful events vs. non-impactful events → increase rate of impactful events which drive business outcomes → if impactful event is monetized, revenue

You don’t need any numbers right now - you just want to develop a theoretical understanding of how this area might contribute to your company’s goals and objectives.

Define and propose an approach

Once you understand the ladder, you can define and propose an approach.

Your approach proposal isn’t the definition of the solution itself, it’s a proposed way to solve, in part or whole, the problem space you are exploring.

Your proposal might be “I want to improve the visibility of the search bar to address the problem of users lacking search awareness” or it might be “I want to do a training video on making sure users know how to use the search bar”.

Both can address the problem of users not leveraging search.

Note: a key callout here - there may be many different ways to solve the problem - from product changes to marketing pushes to communication adjustments - don’t get stuck in a “Product is the solution” box.

Form a hypothesis

With the theoretical chain of how the problem space or solution creates an impact, you can now form a hypothesis.

Your hypothesis should be simple - describable in three simple sentences.

A hypothesis is an assertion of what you believe will happen given an action you take.

Your hypothesis could be:

By improving visibility of the search bar, we will be able to increase awareness, and thus increase usage of search functionality, replacing time-consuming behavior of manual filtering and paging.

We project this will improve the number of impactful events by 3 / day / user by freeing up time otherwise spent on non-impactful events.

At a monetization of $25 / impactful event, this is estimated to have an impact of $25 / day / user, or $25,000 / day across our 1,000 users.

If you can’t describe your hypothesis in 3 sentences, you likely haven’t defined it well enough.

Your hypothesis should have:

• What you intend to do or solve

• How it relates to larger goals

• What the expected impact of it is

Don’t have a solution, or work in an organization where solution ideation happens downstream? That’s fine too - you can start at the value of the problem.

Articulate your assumptions

Once you have your hypothesis, you can start to point out any expectations and assumptions you have made in creating that hypothesis.

An assumption is a belief that you have that isn’t proven by any data, that you are basing other decisions on or that has an interaction with your problem space that changes the feasibility or ROI proposition (return on investment).

For the above example, you might list your assumptions and expectations as:

• Users aren’t using the search bar.

  • Are they actually not using it? How do you know?

• They don’t know it exists.

  • Why do you think they don’t know it exists?

• Making the search bar more visible will increase usage.

  • Do you actually know that’s why the search bar isn’t being used today? Could it be another reason?

• Users will spend any time not spent manually filtering with doing impactful events.

  • Why do you believe the time will immediately go to impactful events? Would it go to getting a cup of coffee instead?

• This time spent will translate into 3 impactful events / day / user.

  • Why do you think time is the limiting factor of adding 3 impactful events? Could there be other factors?

• The monetization rate of an impactful event is $25 / day / user.

  • Why do you think the monetization rate of an impactful event is the same as others?

• You can sustain the monetization rate of an impactful event.

  • Do you think there might be a chance of diminishing returns?

• 100% of the user base is suffering from this problem.

  • Why do you think 100% of the user base is suffering? Could it be 50%? Is it still worth it at 10%?

Listing these assumptions and asking yourself why you believe these to be true is extremely valuable to pressure-test your theories before investing further time into them. By listing the dots, you can validate that you are connecting them logically.

Assumptions are expectations that you haven’t validated. Technically, every expectation is an assumption before anything is given to users. Some of the above examples, like $25 / day / user, may have already been validated by others, or may have obvious answers (eg. “we have a contract”) but the explicit callout is useful.

Anything you haven’t quantified can also be an assumption. If you are’t quite sure the number of impact events per day per user it might be worth validating just to shore up your ROI calculation.

Ask yourself questions

Having trouble figuring out what your assumptions are? Fear not - it breaks down into asking yourself two simple questions:

What are you expecting, and why do you expect that?

• Are you expecting your users to do anything? Why do you believe they well?

• Are you expecting your users to change their behavior? Why do you believe they will?

• Are you expecting a specific amount of impact or change? Why do you believe it will?

• Are you expecting a particular size of users or customers to be impacted? Why do you believe it will?

• Are you expecting to address a cause to a particular problem? Why do you believe that is the cause?

You can also categorize assumptions, and verify that you’ve covered each area with at least one factor:

• Assumptions about user behavior

• Assumptions about sustaining or improving trends

• Assumptions about the actual cause of the problem

• Assumptions about the scope or reach of the problem

• Assumptions about the customer

• Assumptions about the value

Validate your assumptions

Once you have your assumptions, you should validate them. Validation means proving or disproving the premise of your theories.

The point of validation is to reduce time spent working on the wrong thing. If you’re able to invalidate an assumption early, you don’t have to incur the cost of developing a solution because you’ve already determined it won’t work. If you validate all of your assumptions and they hold, you likely have a solid path forward to value and impact.

Start with your core assumptions. Some assumptions are not core to your problem. That is, if they don’t hold, your general premise might take a hit, but still holds.

For the above example, if you had assumed 100% of the user base was suffering from this problem, and it turns out only 75% was, your solution may still have an impact - it just won’t be the same magnitude as initially expected. That’s not a core assumption - it just impacts the expected ROI.

However, let’s say you had assumed users weren’t using the search bar because it wasn’t visible, but it turns out it was because they want to search using something that your search doesn’t search by. In this case, your fundamental assumption is flawed - your efforts to improve awareness of search will have no impact.

If you don’t have the data to test an assumption, get the data. Whether that’s analyzing existing data, implementing additional instrumentation, making a small change to test a specific assumption, performing additional customer or user interviews, or creating a paper prototype an sharing it with customers - there’s dozens of tools available to you. Use this data to inform your assumption.

At that point, you’ve disproven your theory and hypothesis and unless there’s another compelling reason, your solution likely won’t solve their problem.

Decide and act

Go through this process a few times and you’ll have set of problems, possible approaches, and opportunities.

At this point - you can stack rank, prioritize, and apply various frameworks or factor tradeoffs to make a decision and act.

Prioritization is a nuanced skill, highly dependent on your context. Whether you use something as simple as Eisenhower framework (urgent/important) or more complex like RICE - you should be able to take these opportunities and stack rank them to identify high value ones to act on.

Once you decide - make sure your team understands all this thinking that occurred, and ensure you effectively define the problem and the solution space, and manage the stakeholders, shepherd the go-to-market approach, and any other activities that are necessary to successfully release your proposal to users.

Monitor results and outcomes

This is a key part of post-action. One you and the team actually implement your approach, be sure you watch the results. Look at analytics, do additional post-release interviews, check in with your users and customers.

If you assumptions held and it had the impact. you expected - great! If not, use those. to learn and adjust. Did you encounter an unexpected result? Did it have more or less success than you envisioned? Was your central premise valid?

All of these things should be factored into the next decision you make and. the next action you take.

The key, once again, is intentionality. Think through things and have the follow through after release to incorporate the learnings.

If the spaghetti stuck to the wall - why? If the spaghetti fell off the wall - why? If someone came by and ate the spaghetti - why?

Avoid the analysis paralysis pitfall

One pitfall that Product Managers fall into is fear and demand for certainty. Very little in life has guarantees.

Confidence must be balanced with the risk of the assumption - does it invalidate your entire premise, or does it make your solution less effective? Does it change the ROI outcome, or does it cause possible negative considerations?

Sometimes you have to take a leap of faith. If the cost is especially low - eg. a few hours of development time - it might just be worthwhile to try it and see what happens instead of over-thinking it and potentially letting a larger problem fester.

Just remember - your users and customers can say anything. You can think and believe anything. The true proof is in the behavior and actions that are demonstrated.

note: one side benefit of acting with intention - it becomes significantly easier to communicate the “why?” to team members, since you’ve already done the thinking!

As a product manager, you have to be able to articulate, separate, and define your risks - the potential negative effects of whatever you are trying to do. Articulating them helps you frame and compare them appropriately - ignoring low risks and mitigating higher ones.

These are the levers you use and the plates you balance as a product manager to create and deliver value.

Impact risk

Impact risk is the risk that the work you are doing is not worth doing for your company. That is, it is the risk that what you and your team are working on has little to no value to your company, relative to other opportunities, or even worse - may have negative impact.

Impact is a particular thing to every company and different teams within the same company will gauge their success by different outcomes. Alignment with your company and your leadership is critical to ensuring you are mitigating impact risk by working on impactful items.

This question of whether your work is impactful to the company or not is, unfortunately, extremely context-specific. Every company is different. Every team is different.

Instead of telling you the answer, I’ll tell you how you can find out the answer for your specific context, by answering:

• How does your company make money?

• What is important to your company?

• Where do you and your team’s efforts fit in?

How does your company make money?

Fun fact - companies mostly exist to make money. Even social good companies need money to run and operate, so it’s never not a factor.

As a Product Manager, the more you can tie your work into the company’s bottom line, the more impact you’ll have and the less likely it is you’ll have no or negative impact.

To do this, you need to understand how your company actually makes money:

• Ad-supported (eg. social media, search)

• Selling products (eg. storefronts, eCommerce)

• Take rate (eg. marketplaces, service providers, transaction facilitators)

• Subscription fees (eg. SaaS)

• Negotiated annual contracts (eg. B2B, enterprise)

• Usage-based (eg. PaaS, AI)

• Kickbacks (eg. referrals, affiliate marketing)

• Other

Once you know how your company makes money, it’s easier to tie your work into it.

What is important to your company?

Different companies prioritize different things at different stages of their lifecycle.

Early-stage startups might optimize getting a product built and finding product market fit. Mid-stage startups might prioritize exponential customer growth or revenue. Later-stage companies might prioritize profitability and cost reductions.

If you don’t know what the company currently views as important, then stop and find out. Talk to your leadership, and then listen to what they say.

I knew a Product Manager who spent an entire quarter working on accessibility improvements - a key pain point from users that the Product Manager took it upon himself to fix. He made a tremendous amount of progress with his team, labeling images, improving screen reader compatibility, text contrast, and keyboard navigation. The efforts genuinely created a lot of user value - the application had a somewhat high number of users that benefited from these improvements.

When the Product Manager asked for a raise that year, he got denied. While his efforts improving accessibility created user value, they didn’t have any meaningful impact to the company, which was at the time prioritizing customer monetization efforts. Because the efforts he and his team worked on had no impact, he couldn’t justify his request for a raise.

Where do you and your team’s efforts fit in?

Product Managers should have a sense of the teams in their company and what areas they are working on, and how their efforts tie into larger company goals. They should important, know what their own team has been tasked to achieve.

Know what you are expected to be working on. Some broad categories might be:

• Growth. You’re being given a specific metric outcome to improve, such as increasing transaction take rate by 3%, improve user activation by 15%, or increasing the usage of debit cards for payments by 15%.

• Differentiators. You’re being tasked with identifying and developing things that separate you from competitors in your space. Differentiators can be used as talking points in sales organizations or even investors.

• Moat. You’re being tasked with keeping competitors out of your space by strengthening your company’s position. Whether that looks like securing exclusivity deals, or becoming an industry-level source of truth - the company wants to not have competitors.

• Special interests. Sometimes we do things because someone wants to. Maybe the CEO wants to make a key investor happy, or the CPO has an unvalidated hunch.

• Operations. You’re being tasked with keeping the company running, and possible to improve operations efficiency.

• Threats. You’re being tasked with addressing a particular threat or risk area the company is facing. Perhaps its keeping an eye on customer sentiment, or maintaining parity with a key competitor.

• Activity. You’re tasked with increasing activity in the product. User engagement, transaction volume, etc. are all areas where increased activity can positively impact bottom lines.

• Monetization. You’re tasked with improving revenue generated from business. Whether that’s increasing margins or improving take rate.

• Innovations. You’re tasked with identifying new areas for the business - whether that’s a new business line, a new product, or moving up/down market.

Know that not everyone is working on the same thing. Companies have areas that they have to or want to pursue. You, as the Product Manager, likely aren’t choosing the area to work in at most companies. If you ignore your assigned space because you want to pursue some other area, then you’re leaving an area your company wants to cover without appropriate coverage, which obviously causes issues to your own longevity at the company. Never pursue new opportunities at the expense of your assigned area without appropriately communicating and securing buy-in from your leadership.

This means if you’re asked to sustain operations, don’t let the day-to-day of the company fall over because you’re busy chasing down revenue opportunities. If you’re tasked with building a moat, don’t get jammed up trying to figure out how to monetize it. If you’re working on a special interest project, don’t change approaches significantly just because you’ve find a better way to grow a related outcome. Do the thing you’re asked to first exceptionally well, and then branch out with the increased credibility and trust you have built.

Articulating the impact of your work

Once you understand your area, you need to understand your problem and solution space, and how that impacts the primary metrics of the area you are responsible for.

As an example, suppose you are working on a project to improve search awareness. How do you answer what the impact is to the company to improve the users’ awareness of search?

Don’t worry about quantification just yet. First - start at the theory. You can construct a sequence of metrics, drivers, and levers back into a top-line revenue goal:

Revenue

• Monetization rate of impactful events that drive business outcomes

  • Impactful events that drive business outcomes

    • User time spent on impactful events vs. non-impactful events

      • User efficiency to create more time for impactful events

        • User time spent on searching for records

          • Area: Search

            • Search awareness - User awareness of search feature

              • Problem: Users use sub-optimal means of locating their records, including paginating through dozens of pages, exporting data, and even calling support

              • Hypothesis: users are unaware of search capabilities, what they can search for, and whether they can search at

            • Search capability - Users searching in unsupported ways

              • Problem: Users are searching with terms we do not currently support, increasing time spent locating records

              • Hypothesis: by improving search capabilities, we can reduce wasted time

Once you have the theory on how the levers cascade into the company goal, you can then start quantifying each specific area and its relative impact.

List the facts you know, and then start connecting the dots together:

• You know each impactful event is worth $25 to your company.

• You know you have 10,000 users.

• You know that users spend 7 hours of their day doing “impactful events”

• You know an average user does 10 impactful events in an hour.

• You know that users spend 1 hour of their day searching for records

• You believe you have an idea that can reduce time spent searching by 30m.

This means if you successfully deliver on the outcomes of that idea, then you can increase the number of impactful events by a theoretical 5 events / user / day, or an impact of $250,000 / day across all the users.

Comparing impactful things

Once you have a set of problems and levers you can ladder up into a top-level impact, you can start comparing opportunities with each other to identify what the highest impact items are, not just in your ladder but across the entire set of opportunities and problem spaces. This allows you to effective manage your investment portfolio - the solutions and areas you are making bets in to hopefully drive the impact you want to achieve.

Look broadly across your opportunity space. For example, if you spend all your time optimizing the efficiency of users, you may be working on something significantly less impactful than working in another area like acquiring new users. You incur an opportunity cost when you work on something that’s less impactful than another thing you could have been working on instead.

Avoid this by prioritizing the set of items that have likelihood of achieving high impact. Use the framework and context of the company outcome you are trying to influence, and select the ones that have higher impact in a shorter amount of time, balancing cost, confidence, impact, and risk.

Just remember - you can’t always quantify it.

One sticking point I see is a lot of product manager attempting to quantify everything. They get stuck in analysis paralysis, taking longer to gauge whether it’s worth doing than it would have taken to get the work done in the first place.

Make sure your quantification attempts are proportionate to the development cost risk. That is, don’t let anxiety about quantifying the impact of things cause unnecessary delays in delivering on things that are fast and cheap and likely have value. Balance risk and cost appropriately with impact.

Product Manager is not a job of precision, it’s a job of probabilities.

Keep an eye on your metrics and KPIs

You may have theories on what your changes will do. You may even be 100% confident in the impact it will have.

Make sure you monitor it anyways. Implement instrumentation, use analytics, and keep an eye on not just the KPI you’re trying to influence, but all of the outcomes and KPIs that might be impacted. Theories fall prey to reality all the time, and data will show you the truth and facts.

Have a wide lens, and act quickly if you start to see dips in trends that are out of the ordinary or possibly caused by changes you’ve made to the product.

Impact is a context dependent definition, and what’s impactful for one company or one stage isn’t for another. Be aligned with your company and leadership, understand how your proposals can theoretically influence outcomes before you embark on work, prioritize the valuable items alongside other risks and factors, and make sure you monitor your results afterwards.

Product analytics. It’s one of the most valuable quantitative tools for making effective decisions because it tells you exactly what your user-base is doing with your product, if properly instrumented.

Yet, I’ve worked with many product managers who have underutilized this value, unable or unwilling to perform this analysis or even consider it a factor in their requirements. Some have legitimate reasons - perhaps their company doesn’t allow it. Others, less so - some view the work as beneath them, or others are simply intimidated.

Yes - intimidated. Because it seems technical and it seems heavy on data, product managers can potentially shy away from digging deeply into learning how to wield analytics. Instead, they toss any analysis needs over to a data science organization, greatly extending their feedback-decision-action loop.

This behavior prevents them from being able to deeply understand their users’ behavior. If all you’re getting is a summary of an analysis, then how do you quickly get answers to ad-hoc questions? How do you truly understand what your users are doing at scale?

A solid understanding of product analytics, instrumentation, and analysis is a must-have for Product Managers to be maximally effective.

Product Managers must know:

• The mental models of product analytics

• How to identify what to instrument

• How to perform basic analysis themselves (funnels, trends, tracing)

• How to effectively incorporate instrumentation needs into their requirements

Regardless of the vendor (eg. Mixpanel, Amplitude, Fullstory, home-rolled, etc.), Product Managers must know the fundamentals.

What should you instrument?

It depends, but below is what I consider table-stakes questions for any product manager to be able to answer.

• Who is the user performing the action?

• What action is the user taking?

• What context is the user acting in?

These three questions can help you trace user behavior, and construct trends and funnels for analysis.

Who is the user performing the action?

To identify this, we can break this down into three different categories:

• User identity

• User demographics

• User segmentation and categorization

User identity

User identity are the unique properties that allow you to uniquely identify the user so you can trace a particular user’s behavior as they use the product, and cross-analyze across other datasets if you so desire.

The primary one you should, at minimum, have is:

• A record identifier tied to the specific user record in your system (eg. a user_id or account_id)

• An internal identifier to identify a specific user across multiple accounts, if needed, such as a randomly generated UUID associated with multiple accounts representing a single person.

Other identifiers can be used as well - just note there may be pitfalls:

• Email address is useful to easily find an account, but can change or not be present, especially if you have phone number login or other capabilities.

• Phone numbers are useful, but remember that phone numbers can change hands over time.

• IP address is not uniquely identifying enough - IP addresses can be shared, can change frequently, and can even represent thousands of other people (eg. a public hotspot).

Regardless of what identifier you use, associate the User Record Identifier and the Internal Identity identifier to the User’s profile for tracking. It’s sometimes helpful to also associate these details with the event as well.

Why a record identifier AND another internal identifier?

Some products require the creation of multiple accounts, which means a single person may have 2, 3, or even 4 accounts. Think multi-tenant systems, or corporate payroll systems segmented by company. Some security policies also mandate the usage of more than one account (eg. separation of admin accounts and user accounts).

If you only have a record identifier, you may get confusing or skewed analysis results.

To correct for this, you’ll need some sort of universal user identity your product generates that is associated with other accounts but separate from the account identity.

Even if you don’t think that’s the case, just remember: users do strange things. Product analytics often lose track of users due to cache, cookie settings, and multi-device behavior.

User demographics

Demographics might include age, income, country of origin, etc. and can often be natural segmentation points where user behavior starts to differ or key information useful for market expansion. Teens are likely to engage completely differently with your product than the elderly. Perhaps you find yourself getting visitors from another country where English isn’t the primary language, and you want to inform whether it’s useful to embark on an internationalization effort.

It’s harder to do that without understanding the demographics of your users.

Just be careful not to over-collect information. If you’re running a financial applicatio, then asking for income might be contextually appropriate. However, users might find it weird if you have a photo-sharing product and ask that same question.

Be judicious and make sure it’s actually relevant and useful information to collect before collecting it.

note: some analytics products auto-collect and interpret information like IP address an turn it into country. It’s imprecise, but accurate enough.

User segmentation and categorization

You’ll probably have some unique characteristics particular to your industry, offerings, and market that you may want to segment on.

Maybe you have some vetted personas that you want to tie all of the users to. Perhaps your company has separated its customer base by a metal level such as bronze, silver, and gold. Maybe you want to track how big of an organization the user is in.

Whatever you want to use to categorize and segment users by, you can track so that you can look at behaviors across you various categories.

Note: Categorization can be anything that’s relevant to your company, product, and industry. In the past, I’ve categorized users by the kind of sport they played, which was strongly correlated with success outcomes in the product I was working on.

What action is the user taking?

Once you have identity, you should also be tracking what the user is actually doing.

There are two kinds of “doing” events:

• Interaction events

• Business events

Interaction events

Interaction events are things that the user has done to interact with your product or system.

They are typically abstract, generic, and mechanical in nature, such as:

• Viewing a page or element

• Clicking/tapping/holding a button, link, heading

• Hovering on / off an element

You’ll often want to answer questions like “How many times did the user click the thumbs up button?”. Tracking interaction events at a generic, automated level lets you answer those quickly.

These are also useful for constructing interaction funnels - you can quickly chain together a set of views and clicks to see where users are dropping off a funnel.

However, because users can interact with thousands, tens of thousands, or even hundreds of thousands of elements in a single product, you want to track a second, more specific set of events to make you analysis even more useful / faster: business events.

Business events

The second set of events to collect are business events. These are events that are relevant to your business.

They aren’t specifically measuring a user interaction (such as a click or a view) but rather some sort of business-relevant outcome or action which may or may not be triggered by the user via an interaction.

Generic product events might include things like:

• Signed up

• Logged in

• Deleted account

• Changed password

• Requested password reset

• Reset password

• Changed email

• Subscribed to newsletter

• Unsubscribed from newsletter

If you had an eCommerce site, some business events might be:

• Item added to cart

• Item removed from cart

• Item purchased

• Item page viewed

• Cart viewed

• Checkout started

• Checkout completed

• Checkout failed

• Upsell displayed

• Return requested

• Return denied

Examples for a business dashboard might be:

• Chart viewed

• Export downloaded

• Export requested

• Search conducted

• Record approved

• Record denied

• Team member added

• Billing info updated

• Billing type changed

You get the picture. The important part is to identify what events might be important to you or the business to track - which events lead to outcomes?

What context is the user taking action in?

Finally, you’ll likely want to be able to segment and analyze behavior depending on the context of the user’s actions.

Some products offer a multi-tenant whitelabel capability to customers. Events might occur particular to a specific customer’s environment or whitelabel, and you may want to track these events separately from the global perspective.

Perhaps your application has multiple entry points, and you’ve identified that users behave different depending on which upstream they came from. You can track this as well to establish the context of the user’s interaction with your product.

Useful contextual items include:

• The environment or account being acted on (eg. a specific customer environment)

• Where the user came from (eg. what page) - also known as a referrer

• The device of the user (eg. mobile, tablet, desktop, laptop)

• The browser of the user (eg. Chrome, Safari)

Context is, well, contextual - so be sure to think through how you might want to segment and analyze your data.

Add properties to your events so you can analyze them

You’ll always want to make sure that each of your events has a set of properties that describes that you can pivot on to answer your questions.

For example, the following events might have as properties:

• page viewed event

  • page name

• checkout completed event

  • items in cart

  • total price

  • payment type

  • prices in cart

  • coupon code used

• search conducted event

  • search term used

  • filters applied

  • # results returned

  • search time taken

If we used the above as an example, we could answer questions like:

• Which coupon codes resulted in an increase in total price for a checkout vs. a decrease in total price of a checkout?

  • Checkout completed, summing up total price, broken out by coupon code compared to
    Checkout completed, summing up total price filtered by those without a coupon code as a baseline

• What are users attempting to purchase or search for that we might want to find sellers to stock?

  • Search conducted, filtered by those returning no results, broken out by search term used

• Is there a correlation between payment method and the total amount of a checkout?

  • Checkout completed, summing total price, broken out by payment type

• Did users who viewed the new “Halloween special” marketing page buy more items than users that didn’t?

  • Checkout completed, broken out whether there was a Page view for “Halloween special” or not, summing up total price

The questions are obviously limitless. Having the right properties and events will help you answer them. Whatever you might need to be able to answer a question you want to ask is fair game.

Nothing is worse than releasing a product and then wanting to answer a question, only to realize you can’t answer it because you didn’t define the instrumentation for it so you aren’t capturing the events or properties you need.

Note: data never tells you WHY your users did something, only WHAT they did. To get the “why?”, you need to talk with your users. For example - a user may have bought more items on the halloween special because it was a great upsell, or it might be because the halloween special had a defect that added warranty card products to the cart, or maybe it was because the halloween page simply showed more of a catalog than your other landing page and had easier add-to-cart UX. You’ll never know the actual reason through data alone.

Putting events together and analyzing them

With interaction events and business events, you can now start performing analysis of the events - building visualizations of funnels, trends, and behavior across the range of products.

Analyzing a funnel

Funnels are used to identify points in a flow where the conversion rate can be improved.

Funnels are constructed with a sequence of events you expect the user to go through in order, and then you can use the number of people who successfully reached the next step within a set amount of time as a “conversion”.

Analytics products typically have funnel feature built in, but if not, you can construct your own by counting the number of events or users that went through that flow fully, step by step.

For example, if you want to construct an end-to-end funnel for a checkout flow, you could look at these events in sequence:

Item viewed → Item added to cart → Checkout started → Checkout completed

Each column above shows the number of sessions that reached that step. This means you can identify areas with higher than expected drop-off to optimize.

The above example shows a total end-to-end funnel conversion rate of only 23% for events that occurred in a month. That is, 77% of people who view an item do not make it to completing a checkout (“Item viewed” → “Checkout completed”).

Looking more closely, a large portion of drop-off is in the step from “Checkout started” → “Checkout completed”. This area is ripe for improvement: starting a checkout is a high-intent signal, so optimizing this process is likely to significantly improve conversion rates.

The drop-off in sessions moving successfully from “Checkout started” → “Checkout completed” is 6.7k sessions - a staggering 70% drop-off. If your average Checkout size is $50 per checkout, that’s a potential missed opportunity of $335,000 per month.

Improving the flow might increase that to only a 35% drop-off, which effectively doubles your business outcomes from checkouts.

note: some sequences of funnels naturally have high drop-offs. Look up industry standards as a benchmark.

Analyzing a trend

Funnels provide a snapshot in a specific moment in time. It’s useful for optimization, but if you want to see results over a long period if time, you’ll want to look at a trend.

Let’s suppose you map the trend from the above example for the step “Checkout started → Checkout completed” after pivoting your team towards making improvements into that funnel.

You might see something like below:

The above shows the effects of your team’s improvements on optimizing the checkout funnel, having a larger and larger impact month over month until around October, when something occurred that dropped the conversion rate. After that, it stabilizes in December. Perhaps in October, you deployed something that had a negative impact. Unfortunate, but in any case - you can see a massive long-term impact with your other improvements throughout the year.

Within a 12-month period, you were able to improve the rate from 30% to 70%, more than doubling the conversion rate. If you applied that to the same number of funnel sessions from above, this translates into an additional 3,834 successful checkouts, or capturing an additional $191,700 per month, assuming a $50 average checkout.

note: rates are not the only thing that matters. If it’s cheaper to increase entries into the funnel (that is, more people starting a checkout), then you can gain even more benefit from assuming the conversion rate holds. Always check the raw numbers in addition to the percentages. A 10% boost in top-of-funnel for 100,000 events (+10,000) is going to be significantly more impactful than a 100% boost in conversion rate for a step that only receives 1,000 events (+1,000).

Analyzing the behavior of a specific user

You may be interested in how a particular user interacted with your product. Sometimes it’s for auditing or legal reasons. Other times you may be trying to trace down a reported defect, or something that user is doing is making them particularly effective.

Because you associated your events with a particular user record identifier, you can find all events associated with that particular user, and view it as a sequenced list of interactions.

This is useful for tracing behavior, diagnosing the chain of events leading up to an error, and other items:

• User ID 12345 - item viewed - 1685

• User ID 12345 - item viewed - 8573

• User ID 12345 - item added to cart - 8573

• User ID 12345 - item added to cart - 1685

• User ID 12345 - checkout started

• User ID 12345 - checkout failed - unknown error

• User ID 12345 - checkout failed - unknown error

• User ID 12345 - checkout failed - unknown error

• User ID 12345 - cart viewed

• User ID 12345 - item removed from cart - 1685

• User ID 12345 - checkout started

• User ID 12345 - checkout completed

The above example shows there’s likely some issue with item 1685 that cause some unknown error occurred.

This is also how many “session replay” tools like FullStory work. They capture all of the user’s events and play it back for you, in sequence. It’s easier than interpreting a list of text-based events, although with current pricing significantly more expensive.

Your company very likely already has a lexicon or other events and properties already in use. Find or create a data dictionary to start becoming familiar with existing events.

Effectively defining instrumentation in requirements

In order to get any of the value from above, you need to be able to communicate to your engineers what you want to instrument. You should know what events and properties you want to capture as users interact with your product.

It’s not hard - instrumentation is just about implementing the measurements you need. However, engineers aren’t going to read your mind - you need to tell them what you’re looking for. While great product engineers might fill in the gaps, you shouldn’t cede your responsibility to them - make sure you are in lockstep.

Some ways you can provide that clarity:

• Define the specific events that are important to you and the business

• Explicitly write the questions you want to be able to answer

• Write down factors you may want to perform an analysis on

• Establish an expectation of instrumenting interactions by default

Define the specific events that are important to you and the business. You should know what you business events are important to you, and what milestones and checkpoints you want to capture as users navigate through your product. Explicitly write them down for your engineers in your requirements. For example, if you’re working on a job board product:

• Major events to instrument

  • “Job listing created”

  • “Job listing archived”

  • “Job listing edited”

  • “Job listing published”

  • “Job listing removed”

  • “Job listing filled”

  • “Applicant applied to job listing”

  • “Applicant viewed job listing”

  • “Applicant shares job listing”

  • “Company shares job listing”

Explicitly write the questions you want to be able to answer. This helps engineers think about additional events you might want to be able to capture, and helps crowdsource to fill in gaps in your own knowledge. Be clear and detailed, and form and write down questions. State why you want the question answered. Set an expectation that the way to answer these questions is provided. For example:

• Questions I want to be able to answer:

  • Does the length of the job listing influence the number of applicants it receives?

    • We want to inform best practices in the future.

  • Does sharing a job listing lead to more views?

    • We want to inform whether investing in better sharing tools will improve effectiveness of the listing.

  • Why was a job listing archived?

    • We want to understand whether a job listing is removed because it was filled elsewhere, or if it was not effective, or the position is no longer available.

  • How many applicants, on average, view a job listing before it is filled?

    • We want to be able to communicate to customers and in marketing the efficiency of our job board.

  • Did a specific applicant get their job after viewing the job listing?

    • We want to be able to have direct, specific conversion information for renewal negotiations.

Write down factors you may want to perform an analysis on. These can be lists of properties you want to be able to pivot on, so that engineers can ensure these are instrumented. For example:

• Number of applicants

• Type of company

• Type of job

• Salary stated in job listing

• Remote-friendliness

• Company size

• Company location

• Industry

Establish an expectation of instrumenting interactions by default. Some engineers try to instrument things on a case-by-case basis. They have to always remember to add and instrument basic user interactions like clicks. Instead, try to make room or time for engineering to work on a way for instrumentation of user interactions to be always enabled by default in a standardized way. This might require the creation of a small but powerful framework or library that automatically collects events for clicks, views, and other items. This will greatly help as you can, at minimum, reconstruct a user interaction i you forget to add a business event or find some surprising outcome.

Note: remember - not all data is valuable, and not all data is easy to get. It should be a conversation with your team to figure out which data is available and easy to get, and which might be difficult or lead to scope increases. There might even be other ways to answer the questions you have. Make sure it’s a dialogue.

Deeper analysis

That’s it on the basics, which will take you quite far. Remember - product management relies on quantitative data to inform decisions. The faster and better you get at using it, the more informed you can be.

To be effective:

• Be intentional - understand what you want to answer, know, and why

• Be clear - don’t hand-waive instrumentation concerns away when talking with your engineers - they need to understand what you need

• Be curious - dive deep into the tools and questions to get answers, and leave no stone unturned when asking and answering questions

This is just the start. There’s significantly deeper analysis you can do with the proper tool and instrumentation. Analyzing user cohorts, interpreting A/B test results, cross-referencing with internal data - the list goes on.

That will be for another post, though.

I once worked with a Product Manager that was assigned to do a single task: get donors to give our company more money. A typical growth area within transactions and monetization.

To do that, he had several levers he could pull:

• Increase donation size

• Increase donation volume

• Increase donation take rate

He was a smart guy and he hit the ground running. He explored and delivered on several different areas in his first few months:

• He created leaderboards, attempting to increase donor engagement to instill competitive spirit in donors, hoping to increase donation size.

• He encouraged campaign leaders to run multiple campaigns, hoping to increase donation volume in addition to their annual giving days

• He added a tipping feature, an automatically applied amount on top of the donation that went straight to the company, hoping to increase take rate.

• He reduced the number of steps to make a donation, hoping to optimize conversion rates.

Many of these were extremely successful. Donation size increased by 3%. Campaign leaders ran more 300% more campaigns. The tipping feature led to a 2% increase in donation take rate. The donation conversion funnel improved by 10%.

The Product Manager was extremely pleased. He proudly showcased these results in his quarterly review and presentations.

At the review cycle, he was fired. Leadership had lost confidence in him.

What? Why?

The importance of combatting myopia

As a Product Manager, he focused exclusively on his goals with a laser-like focus. He monitored the outcomes he was trying to impact carefully - but nothing else. This laser focus would be his downfall.

He hadn’t considered the other effects beyond his immediate impacts:

• He created leaderboards, which revealed donor names when they wanted to be private. This led to reduced trust in the platform, leading to a decrease of 10% in the number of donations, including larger donors who preferred to remain anonymous.

• He encouraged campaign leaders to run multiple campaigns, which caused campaign leaders to not run their annual campaign and move towards an ongoing gift model. This started to reduce a large source of company revenue, which revolved around providing white-glove support for these special annual days.

• He added a tipping feature, which caused complaints as users thought that the donations were going to the organization when in reality it lined the company’s pockets - campaign leaders were deeply unnerved by the loss of trust, and vocalized their complaints to the executive team.

• He reduced the number of steps to make a donation, removing key anti-fraud checks like CVV and Zip Code, leading to expensive automated credit card scanning by fraudsters and accompanying chargebacks.

Individually, he thought he had met each of his goals and desired outcomes. Taken together, he caused significant harm to the company’s bottom line and positioning.

After he left, I took responsibility of fixing the issues with his execution and ideas:

• Creating the capability to post anonymous donations, carefully preserving privacy.

• Updating go-to-market practices to emphasize the annual giving as the special focus campaigns, and the ongoing campaigns as general funds, re-positioning our company as thought leaders for campaign management.

• Providing users the ability to opt-out of tipping, and being extremely clear the donation was going to support the platform, and not the organization.

• Re-adding key security fields, developing anti-fraud and rate limiting controls, nd adding the capability for campaign leaders to issue refunds for erroneous donors.

With the appropriate balancing factors considered and their risks mitigated, overall outcomes turned net positive and the ideas that originally had harmed in their initial execution brought it roaring back to life.

Having a wide lens

This is the importance of having a wide lens and not being myopic.

No change takes place in isolation. Every change has first, second, and third-order effects that have to be considered.

Product Managers must consider them, and understand the constraints and anti-goals they operate under. Otherwise, they can over-focus on the positives.

As a product manager, you have to be able to articulate, separate, and define your risks - the potential negative effects of whatever you are trying to do. Articulating them helps you frame and compare them appropriately - ignoring low risks and mitigating higher ones.

These are the levers you use and the plates you balance as a product manager to create and deliver value.

$1,125,000 in development costs. Nine months of development. This centralized, curated, tagged database of every single possible customer in the country was going to drive forward expansion like never before. With precision information, the sales force could go directly to the largest, most valuable customers and capture large swaths of the market - customers we didn’t even know existed before. We could compare our user performance with the centralized database to identify what markets were under-served, over-served, and adjust the sales strategy accordingly, and bake these insights into the product.

The project, dubbed TAMMY (Total Addressable Market Maximizing Yield), was the brainchild of Jen, the product manager working closely with the executive team and the sales oganization. It was the largest technical integration the company ever undertook.

A year after release, the project had achieved none of its desired results. Nobody was using it except a couple of data analysts who already had access to the underlying data even before the project started. It hadn’t driven strategy, and the intended users weren’t using it at all.

In short, it wasn’t valuable - a huge waste of money, time, and effort because it solved a problem nobody actually had.

Value risk

Value risk is the risk that your users and customers don’t get value from your solution.

It’s often conflated with adoption risk because what people don’t get value out of, they likely won’t use or buy. Product Managers instead spend all their efforts focusing on increasing adoption.

Value risk is a distinct risk area with different levers to mitigate and address.

Mitigating value risks

Mitigation of value risk requires significant skill by the product manager:

• Deeply understand your product

• Deeply understand your users and customers

• Identify and understand the problem spaces and levers

• Be able to quantify the value

• Move in small iterative, increments

• Avoid over-indexing

Deeply understand your product

Your product has a set of features that provide capabilities to solve problems that the users and customers have.

Can you articulate those problems? Can you name and understand the capabilities and how they solve those problems? Do you know how to configure, use and set up those features to leverage those capabilities? Do you know how it is being discussed and sold to potential customers? Do you know how users use the product? Do you know where it fits into your company’s narrative?

If not, you need to educate yourself on the product.

• Use the product yourself. There’s no replacement for using the product yourself. Eat your own dog-food. Familiarize yourself with its capabilities and limitations. Feel the pain your users feel.

• Read documentation. Take advantage of any articulated knowledge that has been written in the past. Requirements documentation, meeting notes, decision logs, marketing collateral, historical Slack/Teams messages, specifications - these are all valuable context to build an understanding of why things were done, and how things behave under the hood. Just keep in mind they might not be accurate.

• Watch users use it. If you have access to user session playback tools like Fullstory, or recorded user testing sessions, watch them! If you have access to a user, ask if you can shadow them as they use the product. Keep in mind that users can change behavior when they know they are being watched.

• Look at product analytics. Product analytics tools like Amplitude, Mixpanel, etc. are great. If the tool has proper instrumentation, you can piece together funnels of user behavior. Identify whether there’s an up-to-date lexicon so you can see which event goes to which page, or if one doesn’t exist, create one. If not properly instrumented, you can still look at the URL properties of events, which most analytics tools automatically track to construct rudimentary funnels.

• Read the code. If you’re somewhat technical and have access to the codebase, you can read the code to identify what’s happening. The code is the truth of how things behave in the product. If you can’t read code, you can at least read change logs, commit messages, or Pull Request discussions to gain context.

• Talk to engineering. Engineering built the product, so they likely can tell you all the messy details of how it behaves. Build relationships with engineering partners so you can ask questions and gain deeper understanding. Just be sure to do your leg work first, and take notes!

• Talk to support. Support is an excellent source of nuances. Just keep in mind that they often operate without technical support, so workarounds and explanations they have for why things are the way they are may not be reflect of the reality, but are more cargo-cult in nature*.

• Talk to anyone else. Don’t disregard knowledge others might have. From Account Management to Sales to Marketing - identify the keepers of knowledge and invite them to chat.

You have many sources to understand your product - there’s no excuses: get at it!

*Cargo cults

The term “cargo cult” come from World War II. Indigenous tribes saw American troops build and operate air fields and receive supply drops. After they left, the indigenous people built their own air towers out of wood, copied the motions of the soldiers, and went through rituals of marching up and down, expecting that this summoned the planes to bring down equipment.

Obviously, the planes never arrived. The term refers to ritualized behavior that people do because they mis-interpret the chain of cause and effect - continually taking actions that don’t actually lead to the effect they desire.

Deeply understand your users and customers

It’s not enough to understand your product. You have to understand and get into the minds of the people buying and using your product if you want to create value.

Your product is just a small sliver of your users’ and customers’ daily, weekly, and monthly experience. If you over-emphasize your current product, you might think everything is fine because they are generally happy with your product and there’s no complaints, only to see mass migration once a competitor solves an intense pain point you weren’t even aware existed.

There’s no replacement for deeply understanding the user and cusomer.

Talk to them, a lot

Deep understanding requires you to talk to them frequently.

A lot of product managers fall down here, opting to look purely at quantitative data, or do external-focused industry and market research. Don’t get me wrong - these are valuable, but data will never tell you why something occurred, only what has occurred.

To truly understand the why, you need to build and cultivate consistent feedback loops with your users. customers, and intended audience.

Remember - it’s not just about their experience with your product. It’s about understanding their total experience, period.

• What does the user’s day look like?

• What are their pain points in their day-to-day?

• What part of your product is used during what part of their day?

• What are users doing in other products, or outside your product?

• What do they actually want to accomplish and do every day?

• What is the long-term desires and goals of the user or customer?

• What constraints is the user or customer operating under?

There’s a lot of frameworks and practices that help you articulate and map this knowledge out into something insightful and actionable.

• Jobs To Be Done

• Process Mapping (User and Business)

• Value stream mapping

If these frameworks are too complex or too daunting to start using, then start simple. You don’t have to get fancy.

A quick-start guide to gaining understanding

If you find frameworks complex, then do a simple process:

1. Create a table, with columns for every day of the week, or month Sunday through Saturday.

2. Create four rows:

   • Morning

   • Noon

   • Afternoon

   • Evening

3. Talk to your users an customers. Ask them what their day and week looks like. Be specific - ask open-ended questions like“what do you do in the morning?” or “do you do that every day, or only on Mondays”. Ask them what the most painful and time consuming parts of their days are. Ask them what tools they use during what parts of the day. Ask what business processes are done, when, and why.

4. Find commonalities and put them in the appropriate cell - one line for each thing they mention.

Soon, you’ll have filled the table with a general picture of the week or month consisting of activities, pain they’ve highlighted, and even possible mentions of other tools they use.

Identify and understand the problem spaces and levers

At this point - you should have an understanding of your existing product, and you should at least have a general understanding of your users and customers.

Use that understanding as your starting point. Now that you know what your users’ and customers days looks like at a high level, you can drill down into specific points of interest, and start mapping where your product fits in or doesn’t fit in. If you see that an item in a day has a particular applicability for something your product can address - either a pain point or an opportunity - congratulations! You’ve just identified a potential problem space.

Is there anything in your product today that is causing pain? Create a list of these pain points of existing items. Identify the simplest, lowest-cost solutions to remediate as many of those pain points as possible, and then deliver on them. These are low-hangers, and are often table-stakes - the 1% incremental units of friction that aren’t valuable independently, but collectively causes decreased user satisfaction. Solving them in one go helps you create momentum and avoid their future distractions as you focus on larger problems.

Identify the biggest problem spaces. What are the parts of the journey (both within and outside of your product) have been brought up? Pay attention to items that have been:

• Mentioned the most frequently

• Take up the most time

• Take up the most attention

• Are the most important to get right

• Have the lowest success rate

• Are the most complicated to do

• Require high skill to do

• Require many steps to do

These are opportunities to streamline processes, remove pain, decrease needs, etc. The more users and customers in common that feel it, the more you reduce value risk. The more intense the pain, the more likely it is a solution that solves that pain will be valuable.

Identify what you’re trying to do for the customer or user - these are your levers to pull to solve the problems they are encountering:

• Save them time? Find the items that take up the most time in their day to do. Reduce steps, streamline them, or make it so they don’t have to do them at all. Sometimes users do a large time-consuming process just to calculate a value or find a piece of information you could have displayed in the product easily.

• Save them focus? Continued focus on something can be mentally draining. What are they focused on? Can you create some sort of alerting mechanism so they “fire and forget” and switch attention to something else?

• Save them repetition? Identify things you can automate or reduce the steps on. Tasks that are mechanical and wrote, such as updating records with the same set of steps or same value, or checking for discrepancies for review, are great candidates for bulk modification enhancements or automated difference highlighting.

• Prevent them from making mistakes? Errors can cause rework or have downstream consequences. Find out what the most common mistakes made are, and create in-product guardrails to help automatically detect and prevent them.

• Allow them to delegate? Sometimes a user does a combination of things with your product. Perhaps they select a set of records, download the data, slice and dice to a set of columns, and then upload that back into the system. Doing this correctly requires skill and knowledge they may not have been able to reliably impart on others. Automating that process or creating a guided mechanism can allow them to delegate that task to other people.

• Give them a new capability? If you allow your users or customers to do something they’ve never been able to do before, that opens up a new area your product can fit into their day-to-day. Sometimes you don’t even have to give them the power to do it, but the information to do it.

From there, you can quantify the user value in terms of time saved, errors reduced, capabilities created, steps reduced, etc. This can also translate into customer value from a revenue and cost perspective depending on the kind of organization you’re in.

Be able to quantify the value

You could, if desired, back this into a currency value for your users and customers.

Let’s suppose there’s a 12-step process a customer has their users to use in your product:

Process details:
12 minutes per process execution (12 steps, 1 minute / step)
25 process executions / day / user

Calculating total minutes spent per day per user:
25 executions x 12 minutes / execution = 300 minutes / day / user

Assuming a customer has 100 users:
100 users x 300 minutes = 30,000 user minutes / day for customer

Assuming an hourly pay of $20 / hour (33 cents / minute) for each user:
33 cents x 30,000 user minutes / day = $9,900 / day customer cost

Assuming a customer base of 30 customers:
$9,900 / customer x 30 customers = $297,000 / day cost across all customers

In this scenario, your users are spending 300 minutes / day on a process, which translates to a customer cost of $9,900 / day. If you managed to reduce the number of steps by 3 (25%), you can quantify the expected customer and user value:

Process details:
12 minutes per process execution (6 steps, 1 minute / step)
25 process executions / day / user

Calculating total minutes spent per day per user:
25 executions x 9 minutes / execution = 225 minutes / day / user

Total value per user:
300 minutes / day - 225 minutes / day = 75 minutes / day

Assuming a customer has 100 users:
100 users x 225 minutes = 22,500 user minutes / day for customer

Assuming an hourly pay of $20 / hour (33 cents / minute) for each user:
33 cents x 22,500 user minutes / day = $7,425 / day customer cost

Assuming across a customer base of 30:
$7,425 / customer x 30 customers = $222,750 / day cost across all customers

Total cost savings for a customer:
$9,900 old cost - $7,425 new cost = $2,475 / day value for customer

Total cost savings across all customers:
$2,475 / customer x 30 customers = $74,250 / day value across all customers

In the scenario above, by shaving 3 steps off of the existing process, you managed to create a user value of 75 minutes saved per day, which translates to a customer value of $2,475 / day in cost reductions.

Amplified over an entire year (250 business days), that’s a total user value of 18,750 minutes (or roughly 13 days) saved per user, translating to a savings per customer of $618,750 / year, or $18,562,500 per year in total customer value across 30 customers.

Move in small, iterative increments

Once you identify a problem and have a belief in a solution, the remaining way to mitigate value risk is to consider another risk: cost risk.

Building the wrong thing isn’t actually bad in and of itself. It’s the fact that companies don’t have unlimited resources that acts as a constraint. Spending a lot of time building the wrong thing with no value means that you’ve incurred an opportunity cost when you could have been building the right thing that did have value.

Cost then becomes the counter-factor when thinking about value. The return of investment calculation is simple - you want your costs to be lower than the value you get times some multiple of return you expect.

To balance the ROI and make sure you don’t invest six months of development timeon something with no value, you move in small expenditures of cost. That is, step-by-step.

Do a small, scoped release and see if it creates value. If it doesn’t, then you can stop working on it before you spend any real money. If it does create value, or you get feedback that there’s a lot of potential there, you can invest in the next increment and continue development, with the added benefit of getting feedback to inform the direction early and often.

Value, then, is delivered step-by-step, controlling for cost to ensure the return on investment is positive, with the ability to pivot before wasting a lot of time, effort, and money.

That’s the essence of agile - iterative, incremental value development to ensure you’re building the right thing over time.

Note: opportunity cost risk is a somewhat related, but different risk. That risk is about not working on a higher ROI item, even if the items independently are valuable.

Avoid over-indexing

One key piece of advice: make sure you’re not over-indexing on a single customer or single user’s feedback.

Some Product Managers go down the rabbit hole of listening to a particular large client to inform changes to the product, at the expense of everyone else who doesn’t quite experience the same challenges.

This results in a customized, client-specific experience that doesn’t resonate when applied to the larger market of users and customers.

It’s very easy to lose product market fit going down the rabbit hole of entertaining customization requests or doing work based on insights of a small group of users that aren’t representative of the broader user base.

Always think about your user and customer makeup, segments that are being addressed, and avoid making the experience worse for everyone except a specific client. You may have to take on customizations here and there to keep a client happy, but don’t let the tail wag the dog.

Addressing and mitigating value risk is the reason product managers exist. Their responsibility is to deliver and create value, and to do that requires a deep understanding of many facets of the market, industry, user, company, customers, and product. They orient their teams around working on things that create value and translate their understanding into actionable next-steps to solve the problems an deliver the value.

As a product manager, you have to be able to articulate, separate, and define your risks - the potential negative effects of whatever you are trying to do. Articulating them helps you frame and compare them appropriately - ignoring low risks and mitigating higher ones.

These are the levers you use and the plates you balance as a product manager to create and deliver value.

Adoption risk

Adoption risk is the risk that your users don’t use your solution.

If you go through the trouble and expense of doing discovery, finding a meaningful problem to solve, and then releasing something with great fit that still doesn’t get traction, you’ve suffered from the negative impact of adoption risk.

What are some of the contributors?

• Your users aren’t aware of your solution

• It’s difficult to start using or use

• You don’t actually want adoption

Your users aren’t aware of your solution

While you may think of the product you work on 24/7, your users likely don’t. For most products, only the most dedicated power users are keeping up with release notes and change logs.

To fix awareness issues, you have to have an effective go-to-market motion. If you’re lucky enough to have product marketing peers, great! Support them with collateral, explanations, ahead-of-time information, and keep them coordinated and informed of changes and releases.

If you don’t, you should roll up your sleeves and work on getting some of the table stakes of awareness in place:

• Actually release it. Hilariously, I’ve seen some product managers mistakenly believe they released their feature when they hadn’t. They would fret over the lack of adoption and usage and wonder what they did wrong with extreme anxiety. A quick check and I had to point out that they never actually took the steps to enable the feature for the users - oops! Don’t assume it is released to users - verify it.

• Have some way in the product itself to communicate a new release. This could be a “What’s new?” section, a notification or indicator when the user logs in, a highlight that makes a new button or navigation item obvious, or some other element that tells the user “Hey, I’m new - look at me.”

• Write a “quick start” guide and share it. Talk up your new feature, describe step-by-step how to quickly get started, and some frequently asked questions you can think about. If your product has a support knowledge base, you can make a section for new releases and point users to it.

• Ensure your internal teams are aware. Write up an internal explanation, with screenshot or video, of what the feature is, how to use it, how it works, and what the value proposition is for your internal teams. You want key teams in your company to be aware it exists - Marketing, Support, Sales, Account Management, Solutions, etc. can all drive adoption, but only if they know about it and can confidently answer questions.

• Tell your users. In user interviews, tell them about the release. Personally send users you’ve built relationships with a quick email. Do a new release email blast at whatever tempo is appropriate for your situation. Post on social media. If you know specific users have sent feedback in the past about the issue, tell them directly that you’ve solved their pain!

• Gauge interest before even starting. The best way to mitigate adoption risk is to gauge user interest before you even embark. You can do surveys, customer interviews, feedback analysis, etc. I like to do fake-door tests, where I add a button as if the functionality existed, and then track how many users opt-in to being notified of when it releases.

Make sure your awareness strategy fits the cohort

If you know your users are aware, but they still aren’t using it, audit how difficult it is to get started. Actually use your product and click through the end-to-end experience both from the perspective of a new user as well as a returning user.

Audit the UX from a new user perspective

New users should have a different experience than users who have been around for a while. Their goals are different, the foundation of knowledge they have are different, and what you want them to do is different.

Make sure your feature is appropriately positioned for new users. You may even want to not show them a new feature just to avoid distracting them from doing the things that make them more effective and stickier.

Ask the question - is your new feature appropriate for a new user, or does it distract them from onboarding?

An example

I one worked on a product that leveraged a lot of onboarding modals. Every time a new feature was released, they would create a modal announcement for that feature that would show to all users. Great for awareness, plus you can track the user’s interaction with the modal.

The problem was they didn’t maintain their modals over time, and weren’t careful to remove older modals that were no longer relevant. As a result, a brand new user would start seeing a new modal every time they changed the page.

The announcement stack may have been 30 modals deep. Any new feature release announcement would get drowned out in a sea of modals announcing new features from yester-years.

Relevancy was also questionable. All of our features were new features from the perspective of a new user - did we really have to disrupt the new user with an announcement of a feature they have no context on in the moment they should be focusing on their account setup?

I had the modals removed and banned their usage.

Audit the UX from a returning user perspective

A returning user has a larger foundation of knowledge you can build on. They’ve already completed their account setup, they may be using the tool regularly or relatively frequently enough to be able to introduce new concept and features to them to make them more efficient or stickier.

They are likely used to using the tool the way they are using it. This means that there’s an inertia acting against change - they may not want to see changes, or they may not go to the place where changes are being made.

Getting things in front of these users, possibly with engrained habits, is really about adjusting their habits.

Put the entry point close to the problem it is going to solve.

If your new feature is about automatic merging of duplicate records, don’t put them in some settings page - returning users would almost never see it since they’ve already set up their account, an very few users regularly check a settings page. Instead, put it on the page with the list of records with duplicates and highlight it.

It doesn’t have to be where it lives all the time, you just want to have users start using it, and the best way is to put it where they are already habitually looking.

A banner that says “Duplicates found, do you want to automatically merge them? Yes / No” that’s directly on the place where duplicates exist is going to be way more effective than a “Remove duplicates” button hidden in a dusty settings page.

Make a drastic change to force intentional thought

Frequent users are used to how your product works. On some tools, they may be on auto-pilot mentally, knowing exactly where to click, where to type. If you’ve ever looked at your users navigating your application, you can see them moving their cursor to a place where they know a button will appear before it even appears.

This means even if you add a new feature, they may not see it at all if they are on auto-pilot. It’s the fastest way to complete the task of whatever it is they are doing.

In order to raise awareness to these users, you need to apply some friction to break their habits. Move a button from its typical location. Change their initial landing page. Offset a navigation item. Whatever gets them to pause and direct their focus on the application to identify what’s changed.

Yes, it will likely irritate them. However, if the benefit is higher than the cost, it can be extremely valuable to create a little disruption for a future long-term gain.

Note: make sure you’re actually delivering user value. Changing an application just to change it is annoying, and too many changes can harm user efficiency and retention. Like all things, it’s depends on what you are balancing and trying to achieve.

It’s difficult to start using or use.

If your feature requires complex setup to be maximally effective, it a friction that reduces the likelihood of it being used. If you are trying to address adoption risk, simplifying setup and configuration removes one extra drop-off point in user adoption. Even raising awareness that setup has to be done, and providing simple guiding documentation can help drive adoption.

Improve the UX

The simplest thing to do is to make sure the user experience is streamlined. Work closely with your designer and prioritize the user experience during development. Ease of use isn’t always about optimization or increasing user efficiency - some things require a minimum bar of usability to even get started. At least meet the bar.

Add sensible defaults

Identify a set of configurations that works for most people, and automatically set that up for accounts. If a large portion of your users don’t need to set anything up, then that’s better than all of your users having to set something up to use your feature.

If you don’t have sensible defaults, you can still add an initial example of what the tool could look like to demonstrate its value, either as empty-state or an example the user can later delete or modify. If users see what the end-state looks like, it might encourage them to go through the trouble of setting it up.

Effective defaults can drive adoption. I once created a customizable bulk email segmentation feature to allow admins to send email blasts to targeted subsets of their mailing list - donors across the country. Our customers were widely varied - from religious organizations to educational institutions to special interest non-profits across hundreds of industries. There was no one-size fits all default. We released, and we just saw crickets. Nobody really bothered to set it up or use it, sticking instead to their standard of using spreadsheets.

I knew the feature was valuable, so I decided to create a set of five lists for all customer accounts to get them started:

• All donors

• Top 10% of donors

• Bottom 20% of donors

• Donors who hadn’t donated in the past 3 months

• Donors who had donated within 3 months

Having functional examples was like a spark that lit the fuse. The simplicity and power was immediately evident to our users. Once they used the existing lists to try out the segmentation feature, they started configuring it to suit their needs quite rapidly - first by editing the existing lists, and then making their own.

You could see in the instrumentation that the first thing most people did was to use an existing list, which led to configuration, which led to comfort setting up their own lists, deepening user adoption.

Set it up yourself

Something I see a lot of product managers shy away from is doing the work of setting up an account themselves.

Yes, it can be difficult and time consuming. But providing white-glove service can help drive adoption of your feature. At the end of the day - you generally want your feature to be effective, and you should what needs to be done to support that.

Working with your engineering partners is extremely valuable here - they may be able to write a script to automate the initial setup for many people, reducing a friction point in adoption. If you incorporate that into your release strategy, you can significantly reduce adoption risk.

Note: Be willing to do things that don’t scale. It might mean sitting down with the user yourself, or working with support on a process for them to set it up for the users. Painstaking, time-consuming, but it can be very effective. Do what makes the product successful.

You don’t actually want adoption

Product Managers should never consider immediate and sustained adoption of their feature as the primary goal by default. In some situations, it’s actually quite a big mistake to do.

They should always consider what appropriate user adoption looks like. In many situations, user adoption behaves counter-intuitively, and measures like Daily Active Users (DAU) aren’t appropriate.

The feature is naturally only used sporadically

Some product managers see an initial spike in user adoption and usage that rapidly fades. They get despondent as the DAU or WAU trends downward, disappointed at the fact it didn’t seem to have sticking power.

Maybe it didn’t, or maybe it is a tool that naturally has periodic or seasonal usage.

I once worked on a document management tool for large expenditures. Think a review and approval process for permission to move tens and hundreds of of millions of dollars at a time. Document approvals happened in the last two weeks of every quarter.

Before and after that, there was zero usage of the tool. Zero. A Product Manager who didn’t think about the periodicity of usage would think that it had zero sticking power if they were only looking at the initial usage.

You need to zoom out and consider the seasonality.

The feature is not meant to be used very often

Some features and functionality can’t be measured effectively by usage rates or be gauge on its success by adoption.

For example, think of an audit log feature intended to track activity across an account and display it to the user. In this case, the fact that the capability exists is what matters - adoption is a secondary concern, if at all.

In the few times it is used (such as when a company has to perform an investigation of usage or traffic), having the capability available when it is needed is what matters, not that it is used frequently. You may even wish for a world where it doesn’t have to be used at all.

The feature shouldn’t be used at all

Some features and functionality can’t be measured effectively by usage rates. It may even be an anti-goal for it to be used.

A clear example is a “cancel subscription” feature. You definitely don’t want to increase usage of it. Measuring success on an increase on the number of times an account is cancelled is obviously counter-productive. Adoption in this case is an anti-goal.

Note: obviously, you should still instrument such a feature and use it as a lagging indicator of customer satisfaction, and there’s plenty of things you actually can measure, such as effectiveness of subscription save attempts that prevents cancellation.

It’s not always clear-cut

In certain situations, adoption as a risk is less obvious. Think of a feature that lets users export and download their data as a CSV. You may need to balance competing interests, such as:

• Your desire to keep data in-house to aid retention
  vs.
  The user’s desire in slicing and dicing data as they need

• Your desire to keep users inside of the product
  vs.
  The desire of users to connect with other platforms to increase their effectiveness when using your product

In these cases - is adoption desired? Is adoption not desired? What does that balance look like?

Only your context will dictate the answer. An internet blog post can’t tell you that.

Tidbit: high, regular usage of exports that aren’t from departing users might indicate a missing capability within your product that users are making up for in another system, or the desire to have your data in some other system that is a core part of their value stream (eg. a CRM). It’s a signal - use it to learn more.

Adoption risk can be addressed in many ways, an Product Managers should ensure they keep in mind that it’s not the end-all be-all. Premature attempts to address adoption risk can also waste time - contextually-appropriate balance is required. Sometimes it’s more valuable to wait and see if you actually experience adoption issues before attempting to address the risk - risks are, after all, merely probabilities.

As a product manager, you have to be able to articulate, separate, and define your risks - the potential negative effects of whatever you are trying to do. Articulating them helps you frame and compare them appropriately - ignoring low risks and mitigating higher ones.

These are the levers you use and the plates you balance as a product manager to create and deliver value.

Fitness risk

Fitness risk is the risk that your solution will not actually solve the problem it was built to solve. That is, the solution is not fit for the purpose it was developed.

Suppose you had a problem: users are not referring their friends to the platform.

There’s a huge world of solutions that you could explore and implement. Many sound exciting and seem like they can solve the problem. You spend your time developing one of them, only to see zero effect - users still aren’t referring their friends. After addressing all of the other possible risks and causes, you realize you’ve built a dud that doesn’t actually solve the problem

That’s fitness risk.

An illustrative example

A while ago, a Product Manager, Emily, was working on a B2B enterprise insights dashboard with a bunch of cool reports. The primary users were executives that used the tool to gather rapid insights from a variety of data sources. The users loved the tool - it let them answer questions extremely quickly and present data to stakeholders that helped sell their ideas, secure funding, and reinforce narratives.

Pain heard and listened to

During Emily’s quarterly business reviews with her customers, there was a pain point that was heard over and over: the users wanted to be able to share access to the reports with others, but wanted to limit the visibility and lock down permissions to edit the reports.

Each of the users had a different set of permissions they want to give to govern who could do what on the reports. Some wanted the ability to only view, others wanted edit, but not the ability to share with others, and others wanted absolute full control to do anything they wanted. Nobody would be happy without their special rules.

Permissions, roles, security, data visibility, access control. Emily connected all the dots - this called for an customizable authorization product.

The product was simple on the surface but internally complex. It would give users the full ability and control by letting them create their own roles and define what permissions those roles have. The users could then have the full authority to grant those roles to any user account that was a member of their team. They would fully own the access configuration! It was genius - solving all of the competing desires in one fell swoop.

All that hard work and…

Emily and her team of 6 spent that summer building a dynamic role-permission management system into the reporting product. It was brilliant. It allowed for the creation of roles, the assignment of dozens of granular permissions, and even had an impersonation mode so the user could see what people in that role could see with their own eyes. On top of that, the UX was spotless.

At the launch party, she bought the entire team pizza.

The next week, after the marketing blast, Emily looked at the adoption rate, excited. She was disappointed by what she saw. Nobody was using it.

Strange.

She checked back a week later - still, nobody. A week after that - finally! Two customers had set up a new role and permission. She looked at who and got disappointed - one was just a demo account, and the other was an account manager who had set it up for his client.

A month later, with almost nearly no usage, Emily sunset the feature and called it a loss.

The pain was still there. The solution had allowed that need and pain to be solved. The UX was optimized. The awareness was high. The feature was free of defects. The documentation was clear. What went wrong?

What didn’t fit?

There were quite a few problems, actually.

The solution didn’t fit the mental model. Emily created a feature that was about configuring permissions, roles, and authorizations - concepts that were unrelated to the initial problem and pain of sharing a report. While they can be used to solve the problem, it certainly wasn’t the thing her executive-level users envisioned as they were describing their experience, and it certainly wasn’t in their wheelhouse of familiarity.

The solution didn’t fit the audience. Emily created a tool requiring intimate understanding of roles and permissions to achieve the result for an audience made up of executives. Most executives aren’t IT administrators who are familiar with clicking around in a SaaS configuration page to tailor suit the permission mechanism.

The solution didn’t fit the job being done. Emily’s product was for executives who were using it to rapidly obtain reports. When do executives need to rapidly obtain reports? When they’re in a rush. When are executives in a rush to get data? Stressful times when they’re being put on the spot, like during a board meeting, a presentation to a key client, or an investor call. A busy, stressed, rushed executive was not going to take the time to set up administrative permissions, even if they had full knowledge of how.

The solution didn’t fit the user adoption lifecycle. Emily accidentally created a chicken-and-egg problem. The users, executives, needed to give access to one of their IT members in order to set up access and reports. However, they couldn’t give anyone access unless they set up the permission and role system properly, which is what they wanted their IT person to do. It was a catch-22 - they would never set up those permissions. In fact, the only way that chicken and egg problem was fixed was when the account manager did the work.

While each of these was fixable and addressable over time, these challenges spoke to a fundamental issue with the solution: that it wasn’t fit for the purpose. This fitness risk could have ben addressed before development effort was wasted.

Addressing fitness risk

Emily could have addressed the fitness risk far before the build. She could have taken any number of approaches, all of which involved validating fitness with the smallest possible solution:

A research-based approach. Emily could have drawn some mocks, wireframes, or prototypes and shared it with her users for feedback through user interviews, or user testing tools. Their feedback could have given early signs of being on the wrong track.

A competitor-based approach. Emily could have taken a step back to better understand the problem space of sharing, seeing how competitors and other products solved that problem. She would have likely seen different patterns used and been able to take more into consideration.

An increment-based approach. Emily could have built and released a smaller scoped version of the product she built, such as something that only gave view access without the ability to configure it. Feedback and analytics from that increment could have quickly revealed fitness risk with limited development cost.

A sense-based approach. Emily could have thought through the problem more deeply, considering the needs of her audience, matching them with a deep understanding of her users and their jobs and context, thinking through how it would be used and set up from the ground up. This thinking could have afforded opportunities to realize the pitfalls of her solution before she had her team develop it.

The key to these is that they are minimal-to-low investments to get something in front of the user that helps get feedback early, before you invest significant effort from your development team in going down the wrong path. Whether that’s a paper prototype, a working increment, or a detailed thought experiment backed by deep thinking and understanding - all of these are cheaper than building the wrong thing.

To use the example above, an alternative solution with better fit might have been a report-specific link share option that allowed whoever had the link to view or edit the report for a specific amount of time. It likely would have required significantly less development, and fit the need of a rapid way to share reports on the fly.

Validation comes in many different forms and different approaches can give you different levels of confidence you’re headed in the right direction. What level of confidence is appropriate or not depends on the context. Early validation helps mitigate fitness risk by giving you confidence you’re building the right thing.

As a product manager, you have to be able to articulate, separate, and define your risks - the potential negative effects of whatever you are trying to do. Articulating them helps you frame and compare them appropriately - ignoring low risks and mitigating higher ones.

These are the levers you use and the plates you balance as a product manager to create and deliver value.

Development cost risk

Development cost risk is the easiest risk to think about for Product Managers, but it’s also the one they misuse the most.

Development cost is the cost to develop the solution. Every single solution being built has a quantifiable amount of time it will be expected to take. That is the development cost. To identify this amount, Product Managers work with their engineering partners to obtain estimates, or look at historical trends, or if they bad, just make something up.

Once that time is obtained, you can multiple by the average developer salary to quantify the development cost in terms of dollars:

For example, supposing there are 2 developers and they are working on something that’ll take them 2 weeks to build, at $4,000 / week per developer cost:

2 developers
x 2 weeks
—————
4 developer-weeks
x $4,000 / week average salary
—————
$16,000 development cost

You now know that it’ll cost $16,000 to develop the solution,

The development cost risk here is that the cost of developing the solution will not be worth the impact it has. For example, if the impact ends up being estimated to be $100, it probably wasn’t a good tradeoff.

The second development cost risk is that a cost over-run occurs, and the solution is more expensive to develop than previously believed. This still results in the same efect as the solution not being worth the impact, and is more of an engineering concern to monitor and avoid - outside the scope of this particular post.

Addressing the risk

As a Product Manager, you can address the risk a variety of different ways.

The first option is to reduce development costs by cutting scope. Can you solve the problem with a simpler solution? If so, you can change the ROI calculation for the solution.

Many Product Managers don’t actually cut scope, though. They haven’t thought about the solution space enough, or maybe they can’t envision what the smallest possible increment could be.

Instead, the first tool they often reach for is to say “no” and to remove that solution from consideration. They don’t do it at all, instead looking for more valuable items to work on.

Ka-ching! You just saved the company $16,000…maybe.

A key pitfall in saying “no”

There is a subtle nuance in addressing development cost risk that most Product Managers forget when working in a product-based organization: you are always paying it regardless of what you are working on or not working on.

Developers in long-lived teams are typically salaried - they get paid every two weeks regardless of whether they do 1 thing or 100 things. This cost is a recurring amount. It doesn’t stop if there’s no backlog of items to work on.

Product Managers attempting to prioritize solely from the lens of “saving developer bandwidth” or”avoiding something that is too costly to build” can potentially act counter-productively, incurring an even higher cost than time they actually saved from not doing a piece of work.

This is because many well-intentioned Product Managers turn down decent, valuable items to prepare or work on something even more valuable, without having something else ready for development. With no backlog and no ready work, the developers just sit around with nothing to do or tinker on technical concerns of intellectual value but no meaningful impact, or work on lower value items than the one that was discarded.

Remember: in an organization with salaried engineers, you are always incurring development cost if you don’t have something ready for the developers to do. Efforts to address development cost must factor in that this is the floor of cost.

Counter-intuitively, this means that declining to do something because of concerns of development cost can actually lead to lower overall value delivery in some cases.

An alternative approach

An alternative model I prefer to use when thinking about development cost is percentage of developer bandwidth. It’s a common one I use as an Engineering Manager when I think about team capacity.

It assumes that you get a fixed amount of bandwidth per time period (eg. a month). The items you work on, as estimated in your preferred dev-unit (eg. dev-weeks), take up a percentage of that total time available.

You can get the total time available by multiplying the number of developers you have with the number of units in your time period, with a fudge factor for holidays, weekends, etc. This would give you your maximum theoretical development capacity.

As time marches on through your time period, your theoretical development capacity decreases to zero.

For example, to look at a month:

2 developers
x 3.2 theoretical dev-weeks in a month (to factor in meetings)
—————
6.4 theoretical developer-week capacity per month

Then, you can look at you estimate in terms of percentage of development capacity:

2 developers
x 2 developer weeks (estimate)
——-
4 developer weeks (estimate)
÷ 6.4 theoretical developer-weeks capacity per month
—————
62.5% of available developer-week capacity per month

You then know that you’ll be using up 62.% of your theoretical development capacity for that month.

It’s clearly visible that when you remove that item that was going to be worked on and you don’t replace it, you’ll end up using 0% of your theoretical development capacity, which isn’t actually a savings: it’s waste.

This makes it much clearer than if you had merely used dollar values to represent the cost - it feels different because it illustrates not just what you save, but what you waste.

Note: if you pay by the hour, then this development cost is as straightforward as it sounds and utilization can be less needed.

In practice

I’ve found this model useful when product managers get stuck in analysis paralysis. Some product managers become fixated on identifying the highest value item, that they do so at the expense of their teams having anything to do, sometimes for months on end. Telling product managers that they should at least achieve a certain minimum amount of utilization can help spur them towards becoming comfortable making decisions and acting, even if that action isn’t the highest value item in the moment.

By forcing them to realize the cost isn’t saved, but wasted, it makes the picture clearer and helps them make sub-optimal, but still valuable decisions. Worrying about min-maxing value makes sense at higher levels of utilization, but not necessary at lower levels.

It then becomes an exercise of achieving a balance in developer utilization, cost, and value. Applying a constraint creates a forcing function that helps narrow down an infinite set of decisions into a path forward.

A skillset every product manager should have is the ability to synthesize.

Product Managers get exposed to a wide variety of information. They take in feedback through dozens of different avenues: conversations, side comments, research, analysis, metrics, interviews, intuition.

While Product Managers should share their original sources, they must have the ability to construct a cohesive narrative out of all of those different sources.

A Product Manager who just info-dumps without filtering and combining the results together isn’t doing their job. Anyone can take notes. Anyone can present a stream of consciousness and have others read it.

If a Product Manager stops there, they are abdicating a key responsibility: storytelling.

How to synthesize

If you’re crafting a narrative, add a structure.

• Problem, Solution, Value

• Situation, Action, Result

• Intent, Rationale, Detail

• etc.

Pick the appropriate structure. For example, for Intent, Rationale, Detail, you can do these steps below:

First - identify what it is you want your communication to achieve.

• Are you making a recommendation?

• Are you making a decision?

• Are you soliciting information?

• Are you sharing information?

Second - be explicit and state it, in no uncertain terms. Put that at the top of whatever it is you are communicating.

• “I am making a recommendation to do <X>.”

• “I have decided to do <Y>”

• “I am asking for information or answers to <X> topic or questions”

• “I am raising awareness of this information for consideration into <Y> discussion.”

Third - give the rationale. This is where you summarize your findings.

The reasons are below:

• Dozens of user interviews (link to repository) have indicated <X> sentiment.

• Our product analytics have shown a <Y>% increase since <Z> occurred.

• Specific competitors like <A>, <B>, and <C> have all explored this space.

Fourth - show the details. This is where you can show your work.

User interviews

• We conducted <X> user interviews.

• Interview with top 5% of customers showed <X> positive assessment.

  • “We absolutely love <Y>.“

Analytics

• This Amplitude dashboard (link) shows usage at <X>% since launch, highlighting increasing adoption and demand.

Competitor behavior

• <R> competitor recently hired <A>, who’s background is in this space.

• <Z> competitor recently announced joining a business group focused on bringing <B> to market

What should you synthesize?

Pretty much everything.

• User interviews

• Meeting notes

• Requirements

• Specifications

• Problem statements

• Decision logs

• Release notes

• etc.

Remember - synthesizing something doesn’t mean you don’t link to further details. It just means you paint a clear picture with broad strokes before starting to focus on the specific details. You want people to see the forest before focusing too much on the trees. Link back to the specific details, especially if they are important.

A contrived example of an unsynthesized user interview

Suppose you received this set of notes from your Product Manager:

support interview

Attendees - joseph, melanie, brent, brent, carl

started meeting

talk about new problem

lots of problems, finding ticket bad, can’t find ticket, lots issues

search for ticket

daily report

filter list

build it

melanie doesn’t like it

The problems

The above is useless because it is a ton of detail with more questions than answers. It lacks organization, which leaves questions unanswered and makes it harder to follow:

• What was the meeting about?

• What problems were discussed?

• Why this problem?

• What didn’t Melanie like?

• What should we build, and why?

• What are the other options?

You could still parse it out to figure it out, but you shouldn’t have to fill in those blanks. Even if it was verbatim and had full details, Product Managers are not transcribers. We need to provide more value than that to be useful.

An alternative

Imagine the same meeting, but with the below notes instead:

Interview with Support - 2/11/2019
We decided to create a new Filter on the Support Ticket tool that will filter the list by the ticket status. This will help ease support time burden in locating a ticket. We considered other alternatives like searching or a recurring report, but these had drawbacks.

Meeting notes
Intent
discuss challenge with support ticket navigation

Attendees
Product and Engineering: Joseph, Melanie
Support: Brent B., Brent C., Carl

Discussion
Support ticket navigation

• Problem - unable to find ticket

  • Wastes significant time going through pages of tickets

  • Reported to happen 10x per day per agent

• Solutions proposed

  • Search for ticket status

    • Requires cross-team interaction

  • Daily report of tickets by status

    • Melanie doesn’t like it - technical feasibility concerns

  • Filter list by status

    • Easier to do, can be done with single team

    • DECISION - Decided to do this

Decisions:

• Create Filter list by status

Even if not perfect, this is a much clearer set of notes. The synthesis at the top also helps frame the intent of the message to act on.

Tips

Don’t start with details. People need to know what is expected of them. Otherwise, it can derail a conversation or lead people down a line of thinking that wasn’t the intent.

Understand the content. Some Product Managers don’t actually understand what they are working on, but attempt to communicate it anyways. This usually results in fantastic blowups as the situation devolves. Don’t be afraid to ask questions if you don’t understand. Make sure you do understand what’s being said and what’s happening - people will quickly find out if you’re just making up stuff.

Be organized and consistent. Know how to write well. Use headers, lists, typography changes (bold, italics, etc.), section breaks, tables, etc. to communicate.

Know your audience. Is your audience a group that likes numbers and metrics? Add them. Is your audience more about user sentiment and impact to reputation? Highlight it. Is your audience more about comprehensive considerations? Highlight the risk areas you’ve explored and addressed. Know what your audience cares about, and ensure your synthesis considers it. Know what they don’t care about, and save the time and energy.

Don’t be too long. Nobody wants to read 50 pages to get to the point.

Be as long as you need to be. Don’t be too short, either. If there’s key callouts you want to make, or particular pieces of evidence you want to highlight, you should do so, provided the information is not duplicative of points already sufficiently made.

Link to the details. Some people are keenly interested in the details. You don’t want to mix up or muddle your message with over-detail. Instead, link to supplemental materials or add footnotes where details might matter.

Be medium-appropriate. Don’t paste a 1,000 line message in a chat. Don’t write a short-hand message with typos on Confluence. Understand the characteristics and expectations of the communication medium you are using, and stick to it.

• Documentation stores should have formal, clear, concise written text with long-lived content.

• Instant messages should be 1-3 sentence summaries with links to documentation.

Sync content across mediums. If you’re making a decision and you’re communicating it in one medium, make sure it reflects in another medium. For example, if you make a key decision on a Slack thread, be sure to update your decision log on Confluence immediately.

Be correct. When you synthesize, you lose a bit of message and communication fidelity. Make sure that your message is still accurate, or else you may be unreliable delivering incorrect information that would change a decision.

Get feedback. Getting someone to look over your synthesis and give you feedback can be nerve-wracking or even uncomfortable, but extremely valuable learning opportunity. I’ve seen many Product Managers get stuck at a low quality because they dismiss the feedback they get or don’t ask for it, assuming everyone likes what they are doing.

A failure to synthesize

I worked with a Product Manager a long time ago that had extreme difficulty synthesizing information.

They did great at talking to users and getting tons of information from them. But, when it came time to communicate that to the broader team, they gave word vomit, stream-of-consciousness notes instead of clear, concise statements.

When they did a user interview, they shared unfiltered, disorganized notes. When they wrote a problem statement, they blurted out every single thing they knew without a cohesive narrative.

Nobody could make heads or tails of what was being asked, what the intent was, why it was happening, or what the details were. The 10+ page documents that were shared had great information here and there, but could have easily been reduced to 1-3 key sentences. The team, being busy, rarely read the poor quality, unclear documents.

Eventually, the team had enough and just started ignoring the Product Manager and did their own thing. When the Product Manager complained to upper leadership, they couldn’t help because they didn’t understand what the Product Manager was saying, either.

As the relationship between the Product Manager and the team soured, so too did the quality of information back to the Product Manager regarding feasibility, progress, or other insights. The Product Manager started to lose accurate ground-truth visibility into the product they were responsible for, which in turn negative affected their communications to others even further as the information was now starting to become inaccurate on top of being disorganized and unclear.

Eventually, that Product Manager was side-lined. Nothing they said was ever trusted or had any influence simply because they couldn’t effectively communicate it. They eventually were shifted out of that role and left the company.