top of page
Evaluative and Generative Research for
UsabilityHub's Customer Dashboard
2021.9 - 2021.12
Context
I worked as a contract UX researcher at Curiosity Tank, a design research firm, between September 2021 and December 2021. Curiosity Tank was collaborating with UsabilityHub to understand people's experience using the platform's customer dashboard.
UsabilityHub is an all-in-remote user testing platform with access to over 340k panelist. As it gets more users, the customer service team receives feedback requesting different features and actions in their dashboard. UH wants to conduct both evaluative and exploratory research to know what improvements should be made to the dashboard.
I was responsible for the study focusing on power users and there's another researcher working in the project focusing on novice users. We worked together, identified several issues of the dashboard and uncovered opportunities for improvement.
Project Restrictions
-
Working across time zones with stakeholders (based in Australia) and teammates (based in Dubai and San Francisco) leads to asynchronous communication
-
Limited time for actionable research insights
* The project is under Non-disclosure agreements, so I can only share the research process without confidential information in the following case study.
Research Objectives
After several rounds of stakeholder meetings, we were in alignment with UH about the objectives of the study: the primary objective is to understand how users use UH’s customer dashboard and identify areas of improvement. I listed out assumptions based on feedback from the customer service team as well as communications with UH's designer and PM. Here are the final objectives for the dashboard study:
-
Evaluative how users are using the customer dashboard
-
Identify current frustrations and barriers about the customer dashboard
-
Uncover opportunities to improve the dashboard for "power users"
Methodologies
Screener survey
User interviews
Thematic analysis
Process
Plan:
Stakeholder interviews - Assumptions - Research Questions
Conduct research:
Screener survey (68 valid responses) - 10 interviews (each interview 45 minutes)
Analyze & Synthesize:
Debrief Grid - Dovetail Tagging - Thematic analysis in Miro
Deliverables:
Highlight Reels, Topline Report, Presentation Deck
Find the Right People to Talk
I talked to my stakeholders as well as the customer service team from UsabilityHub to clarify the recruiting criteria for power users. These criteria include 1. The number of tests they created on UsabilityHub, 2. The number of people who shared the same account, 3. Frequency, and length of their usage.
I conducted the screener with the other researcher who studied novice users. We created the screener on Typeform with 17 questions, and the UsabilityHub team shared the screener through its customer pool.
Screenshots of recruiting messages through UsabilityHub customer pool
We received 68 complete responses from both novice and power users. I managed to conduct 10 1:1 45 minutes interviews with power users. These users came from 4 different countries and 60% of them are researchers while 40% of them are designers.
Analyze & Synthesize
After 10 interviews, I analyzed and synthesized all the data through thematic analysis using both Dovetail and Miro. 3 major themes and opportunities were
Screenshots of Affinity diagram
Deliverables
The final deliverables of this study include raw data from in-depth remote interviews, highlight reels, a topline report with key themes and descriptions, and a presentation deck highlighting findings, actionable insights, and recommendations.
Screenshots final deliverables
Reflection
-
Don't decide by the committee of end users. We spent time talking to users as we wanted to ensure that the product fulfilled the users' wishes and needs. However, we need to keep user feedback in its proper context, as what we really want is a user-centered rather than a user-centered approach.
-
Find an appropriate way to collaborate remotely. One of the biggest challenges for this 8-week project is that the teammates and stakeholders were based in different time zones. To make everyone on the same page, we spent too much time finding the best way to do asynchronous work with the help of slack, Google doc, and Miro. However, I find it still necessary to do synchronous work. Combining these two working styles would benefit the team most in a remote project.
-
Working as a team while thinking as an individual. UX research is teamwork, but the actual doing part should be independent, as every researcher comes with different opinions and angles.
bottom of page