ClassPass Consumer Diary Study

Understanding how ClassPass users make booking decisions and explore new offerings.

ROLE

User Researcher

COMPANY

ClassPass (Mindbody)

YEAR

Aug-Sept ‘23

METHOD

Diary Study

Context

In H2 2023, I worked with ClassPass Product leadership to identify “Consumer Activation” (converting ClassPass trialers and new subscribers to long-term subscribers) as a big area in need of research. The org lacked the foundational insights to strategically inform relevant roadmap commitments.

I set out to investigate: How and when do newer ClassPass users establish wellness routines through the app?

My Involvement

  • I was the lead researcher, closely working with 1 product manager and 1 product designer to identify gaps in foundational understanding about our users

  • I pioneered the adoption of a diary study platform (dscout) and the methodology itself for the company

  • I crafted the research approach, execution, and analysis

  • I built trust with stakeholders as one of the first UX researchers to work with the ClassPass team to influence decision-making for roadmap commitments

Research Questions

My desire to maximize the amount of evergreen research insights through this project led me to focus on the following questions:

  1. What do newer ClassPass users do in their first few months after signing up for ClassPass?

  2. What are critical experiences in the product that impact users’ attitudes toward ClassPass?

  3. What encourages people to try new services/classes on ClassPass?

  4. How do ClassPass users build wellness routines and make booking decisions?

Most importantly, tied to a 2023 OKR: How can ClassPass aid the discovery process to encourage a higher conversion (booking) rate?

The Approach

To answer the above, I decided to conduct a 2-week diary study with 20 participants for the following reasons:

Why a diary study?

  • The questions we wanted to answer were behavioral. A diary study would allow me to observe that behavior in a self-documented environment, rather than having the time or peer pressure that comes from a survey or moderated session.

  • A diary study would allow me to collect various forms of data – qualitative, quantitative, and media (screen recordings and screenshots).

  • I wanted to document and get close to the natural rhythms of people’s wellness routines. By allowing self-documentation through diary study, participants would be able to recount their experiences at their own pace, and closer to the time of booking / completing the activity. This gets us closer to real-time interaction, rather than forcing participants to recount an incident in an artificially prompted time.

Why 2 weeks?

  • I wanted to allow for ample time and opportunity for behaviors to be repeated and routines to reveal themselves. If something happened in the span of 1 week, I wanted to see if it would happen again during the 2nd week.

  • A 2-week study would allow me more time to see how far in advance participants typically sign-up for wellness activities.

TL;DR – a diary study was the best way to capture user behavior and frequency.

Methodology

Recruitment

We aimed to secure 20 participants through a mix of dscout panelists (to test dscout’s recruiting offerings) and manually selected recruits.

  • Parameters:

    • Early-lifecycle ClassPass users who as been subscribed to ClassPass between 3-6 months

    • Displayed consistent usage of ClassPass credits

    • Based in regions with large market supply

Why 20 participants?

  • The Nielsen Norman Group’s recommendation for large discovery projects with a fairly heterogeneous user group is 12-30 people.

  • Using NNG’s recommendation as a starting point, I accounted for potential fall-out over the course of the diary study. (I’ve found that longer studies over 2-weeks struggle to retain active engagement from participants.)

Structure

We wanted to capture wellness routine-related behaviors that naturally occurred for participants, while gathering contextual data. This was done by utilizing a mix of required prompts and voluntary prompts that were answered when a natural behavior occurred. The structure was as follows:

  • Activity 1: Introduction

  • Activity 2: Using the ClassPass app

  • Activity 3: Completing a non-CP wellness activity

  • Activity 4: Completing a CP activity

  • Activity 5: Week 1 - end of week reflection

  • Activity 6: Week 2 - end of week reflection

Sample Questions

Activities included a range of prompts and response styles. For example, Activity 2 included the following:

–––––

Any time you are about to use the ClassPass app to explore options or book a service, please submit an entry to this Activity. You will submit entries to Activity 2 over the entire 2 weeks of this mission.

  • You have 14 days to complete this activity. Please submit an entry each time you use the ClassPass app.

  • In an up to 5 minute screen recording, please record as you use the ClassPass app. As you use the app, talk aloud and let us know what you are doing and why.

  • What were you doing/where were you when you opened the app today? For example: in between classes at school, lunch break, etc

  • What made you interested in booking that particular service at that business?

For additional details about the approach, please reach out to me at diane.hahn9@gmail.com. 📬✨

Insights

Though my NDA limits disclosing the detailed takeaways from this research, we uncovered clear themes through this study that revolve around:

  • What factors into ClassPass booking decisions

  • Wellness is a very personal journey

  • When + why users decide to explore “new” offerings

  • Why people are drawn to ClassPass

  • … and more!

For additional details about insights + results please reach out to me at diane.hahn9@gmail.com. 📬✨

Impact

With the successful launch and execution of the diary study, we were able to answer all research questions. This resulted in the following:

  • Team defined 2 roadmap items for 2023 H2 that ladder into ClassPass subscriber growth OKR

  • Significant contribution to evergreen data for the ClassPass side of the business to continually come back to for strategic product decisions

  • Solidified diary study as a methodology for the team’s UXR toolkit and guided 1 teammate through using the method for the Mindbody side of the business

Reflection

Challenges

  • A challenge I encountered was underestimating the sheer volume of feedback I was collecting. As participants uploaded screen recordings of their booking process (on avg. 5-min each) multiple times a week, I struggled to keep up with watching the videos in addition to going through the text data from open-ended reflection questions.

  • This hindered my ability to quickly churn out research tidbits during the diary study, as I had wanted to send a steady stream of insights over the 2-weeks before the official report.

  • However, I adapted quickly to give stakeholders overarching themes while prefacing that they were preliminary observations from the amount of data I could ingest. This still allowed me to keep my product partners engaged and invested in the process as we waited for the granular, finished report.

Reflections

  • If I could re-do this study, I would recruit all 20 participants through a manual recruit rather than using the dscout panelists. Though it is great to test out the platform’s capabilities, the dscout panelists surprisingly had a higher drop-off rate than the manual recruits.

  • I developed an appreciation for the diary study’s longer format and timeline because it captures nuances that a 30-minute moderated session cannot, when it comes to understanding day-to-day behaviors. It made me excited to continue expanding my research toolkit!

Spoiler alert!

I found that users heavily rely on the map view + Home tab to explore, and seek personalized recommendations to pre-filter workout options based on their mood or availability.

The team committed to 2 roadmap items in 2023 based on these findings to increase conversion rates.