-
How and When to Control for Confounders During Product Usage Analyses
2021-06-12
Note: This post was originally published on heap’s blog We all know that correlation isn’t causation, but when we’re assessing the impact of a feature we’ve just shipped or searching for an “aha moment” that leads to better retention, it’s easy to forget this. It’s tempting to look at the increased conversion rates of users who did X versus users who didn’t, and conclude that our feature is working or that we’ve found the “aha moment.…
-
Why PMs Should Study Statistics: An Interactive Essay
2019-06-02
Marty Cagan – seasoned product manager and author of a book and blog that makes practically every recommended reading list for new product managers – says that there are two academic courses that “every product manager should take”: finance and computer science. In this interactive essay, I suggest we add another course to this list: statistics. A strong understanding of statistics facilitates three key responsibilities of product managment: understanding analytics, implementing cooprorate change, and making accurate forecasts.…
-
How Startups Can Do Better Cohort Analyses
2018-12-29
If you’ve ever looked at analytics for software products, you’ve probably run across a graph that looks like this: Graphs like this one depict cohort analyses.1 This particular graph is from Google Analytics. Apple also has one for app analytics. So does Fabric.2 Cohort analyses can be very useful. For example, Eric Reis, in The Lean Startup, recounts how cohort analysis helped his startup realize that their efforts at improving their product weren’t working:…