Product Analytics for Engineering Leaders
Product Analytics for Engineering Leaders
To Own Your Product You Must Understand How It's Being Used
Is product analytics an engineering thing?
You can be a great developer, focusing intently on algorithms, design patterns, code structure, etc. However, the true meaning of your work is revealed when you observe how your product is utilized by customers and other stakeholders.
Moreover, according to DORA research, a user-centric approach to building applications and services is one of the strongest predictors of organizational performance. To improve this performance, you must develop a deep understanding of your users.
You can read more, here: Key Insights for Engineering Leaders from DORA research.
The approach to user centrism varies depending on the team you lead:
Product development teams primarily focus on understanding user needs and behavior to tailor their work accordingly.
Platform teams should understand how developers work since they are their "users.” The focus goes mostly on identifying areas of friction through Developer Experience practices (feedback surveys, internal monitoring systems, SLI/SLOs, etc.).
To narrow the subject, this article will concentrate on the former - customer-facing teams and how to enhance our understanding of end-user behavior.
Understanding the Product You Build
The classic, though oversimplified, approach to product analytics involves:
Engineers developing a new feature,
Analysts examining behavioral data and drawing conclusions,
Product managers sharing these insights with all stakeholders, including engineers.
This process is already recognized as a good practice because it enables developers to grasp the impact and significance of their work.
However, truly empowered engineering teams take several additional steps to minimize the feedback loop. Delving into data, defining personas and cohorts, or calculating sophisticated business metrics can be a full-time job; But it should not prevent engineers from acquiring a basic understanding of the product they are building.
As an engineering leader, you should always be aware of certain core metrics about the feature or service you manage:
The volume of users and events,
Seasonality (like peak hours or days),
User flow through your feature,
Critical events (often referred to as conversions),
Errors (not just technical, but also those from a UX perspective),
A/B test success metrics.
If you rely on a Product Manager or Analyst to inform you about these, your feedback loop is already too extensive. As an engineering leader, you should be the first to know these metrics.
Here's why it matters: Three Levels of Engineering Teams Engagement.
This high-level understanding of your product is also critical for framing the problems you aim to solve. Instead of waiting for a Product Manager to define WHAT and HOW a certain challenge should be addressed, you can form your own perspective on the issue ahead of time.
More about framing problems can be read at The Problem Solving Framework.
Let's Build Your First Product Analytics Dashboard
To translate theory into practice, let's experiment with code and create our first product analytics dashboards. Treat this implementation as a proof-of-concept. The goal here is to demonstrate the high-level concept of product analytics, not to showcase high-quality coding practices.
A bonus point - if, like me, you are not a fluent JavaScript/React developer, you will see here a project that was created almost entirely by ChatGPT and Github Copilot. If you're interested in a separate article on how I did that step-by-step, let me know by clicking here.
Here's the agenda:
Project overview
Build a simple React application
Add product analytics
Deploy it to Vercel
Build product analytics dashboards with Amplitude and Mixpanel
Project Overview
We will build a 3-page web application (just a mock, with no backend logic) to simulate basic functionalities:
A landing page with basic navigation,
A call-to-action page resulting in success or error (e.g., a payment process),
A contact page form with simple validation.
This web application will include a few analytics events upon which we will base our example product analytics dashboards in Amplitude and Mixpanel.
You can be a great developer, focusing intently on algorithms, design patterns, code structure, etc. However, the true meaning of your work is revealed when you observe how your product is utilized by customers and other stakeholders.
Moreover, according to DORA research, a user-centric approach to building applications and services is one of the strongest predictors of organizational performance. To improve this performance, you must develop a deep understanding of your users.
You can read more, here: Key Insights for Engineering Leaders from DORA research.
The approach to user centrism varies depending on the team you lead:
Product development teams primarily focus on understanding user needs and behavior to tailor their work accordingly.
Platform teams should understand how developers work since they are their "users.” The focus goes mostly on identifying areas of friction through Developer Experience practices (feedback surveys, internal monitoring systems, SLI/SLOs, etc.).
To narrow the subject, this article will concentrate on the former - customer-facing teams and how to enhance our understanding of end-user behavior.
Understanding the Product You Build
The classic, though oversimplified, approach to product analytics involves:
Engineers developing a new feature,
Analysts examining behavioral data and drawing conclusions,
Product managers sharing these insights with all stakeholders, including engineers.
This process is already recognized as a good practice because it enables developers to grasp the impact and significance of their work.
However, truly empowered engineering teams take several additional steps to minimize the feedback loop. Delving into data, defining personas and cohorts, or calculating sophisticated business metrics can be a full-time job; But it should not prevent engineers from acquiring a basic understanding of the product they are building.
As an engineering leader, you should always be aware of certain core metrics about the feature or service you manage:
The volume of users and events,
Seasonality (like peak hours or days),
User flow through your feature,
Critical events (often referred to as conversions),
Errors (not just technical, but also those from a UX perspective),
A/B test success metrics.
If you rely on a Product Manager or Analyst to inform you about these, your feedback loop is already too extensive. As an engineering leader, you should be the first to know these metrics.
Here's why it matters: Three Levels of Engineering Teams Engagement.
This high-level understanding of your product is also critical for framing the problems you aim to solve. Instead of waiting for a Product Manager to define WHAT and HOW a certain challenge should be addressed, you can form your own perspective on the issue ahead of time.
More about framing problems can be read at The Problem Solving Framework.
Let's Build Your First Product Analytics Dashboard
To translate theory into practice, let's experiment with code and create our first product analytics dashboards. Treat this implementation as a proof-of-concept. The goal here is to demonstrate the high-level concept of product analytics, not to showcase high-quality coding practices.
A bonus point - if, like me, you are not a fluent JavaScript/React developer, you will see here a project that was created almost entirely by ChatGPT and Github Copilot. If you're interested in a separate article on how I did that step-by-step, let me know by clicking here.
Here's the agenda:
Project overview
Build a simple React application
Add product analytics
Deploy it to Vercel
Build product analytics dashboards with Amplitude and Mixpanel
Project Overview
We will build a 3-page web application (just a mock, with no backend logic) to simulate basic functionalities:
A landing page with basic navigation,
A call-to-action page resulting in success or error (e.g., a payment process),
A contact page form with simple validation.
This web application will include a few analytics events upon which we will base our example product analytics dashboards in Amplitude and Mixpanel.
Disclaimer: This article does not promote Amplitude or Mixpanel. I use them as example tools based on my experience. Feel free to use any other Product Analytics tool of your choice.
The final implementation of the app can be found on my GitHub: Analytics-example-app.
Building a Simple React Application
To create the example app, I utilized the following setup & tools:
Nvm to manage Node.js on my machine,
Visual Studio Code for app development,
(paid) Github Copilot for code completion and enhancements to the existing codebase,
(paid) ChatGPT for generating the project from scratch. Github Copilot Chat could have sufficed, but I found ChatGPT's responses more tailored to my needs, so I opted for this route,
(free/paid) Vercel as the hosting platform for the web app. The app is simple and does not require any premium features. It is currently hosted on Vercel completely for free.
As an alternative to setting up my machine, I also recommend exploring Github Codespaces, which provides Visual Studio via browser and prepares a machine for work within a few seconds.
In fact, parts of this project were edited in Codespaces on my iPad:
