Design Usability Test Plans for Actionable Insights

Designs comprehensive usability test plans that generate actionable product insights - covering research objectives, participant recruitment, task design, data collection, remote vs. in-person logistics, and issue severity analysis, following HCI research practices.

Use this skill when planning a new usability study for a product or prototype, designing realistic task scenarios that avoid leading participants, calculating an appropriate participant sample size, choosing between moderated and unmoderated or remote and in-person testing, or structuring findings into a prioritized, actionable report. Not a fit for quantitative-only A/B test design in isolation, or for general market research unrelated to usability/UX.

Core principles include a user-centered approach (observing authentic behavior over confirmation bias), ecological validity (mirroring real-world usage context), triangulation (combining behavioral observation, think-aloud, interviews, and SUS scores), and statistical guidance that 5-8 participants per user segment typically achieve 80% problem discovery.

Provides a test plan template with research questions (primary/secondary/tertiary) and success metrics (task completion rate >85%, time-on-task benchmark, error recovery threshold, SUS score >68). Includes a participant recruitment framework specifying primary user criteria, edge-case segments (accessibility, novice users), recruitment methods (user panel, social recruiting, on-site intercept), and exclusion criteria.

Task design methodology contrasts a poor leading task prompt against a good realistic scenario-based prompt, defines primary/secondary/tertiary task metrics, and lays out a four-stage task complexity progression (warm-up, core tasks, edge cases, open-ended exploration). Data collection covers quantitative task metrics (completion rate, time on task, clicks, error rate, help-seeking) and an SUS scoring formula, plus a qualitative think-aloud protocol with example prompts and a behavioral coding schema (navigation patterns, hesitation points, error types, emotional indicators).

Remote testing guidance lists specific tool categories (screen recording, prototype sharing, real-time note-taking) and environment-control practices, alongside a moderated-vs-unmoderated tradeoff comparison. Analysis guidance defines a three-tier issue severity classification (critical, major, minor) and an impact-vs-frequency prioritization matrix, plus a structured finding format (evidence, user quote, recommendation, design implication, success metric). Advanced techniques cover combining usability findings with A/B test validation and running longitudinal studies with consistent methodology. Ethical guidance covers informed consent elements and participant wellbeing practices (avoiding genuine frustration, breaks for long sessions).

References common usability research tooling categories - screen recording (Lookback, UserTesting, Zoom), prototype sharing (Figma, InVision), and collaborative note-taking (Dovetail, Miro) - without being tied to a specific platform.

UX researchers and product teams planning a usability study who need a complete methodology - from recruitment criteria through task design to severity-classified, actionable findings - rather than an introductory overview of usability testing concepts.

Finding: Users struggle to locate the search function
Evidence: 7/8 participants took >30s to find search
Recommendation: Move search to primary navigation