Description
Observe people unboxing and setting up a physical product unaided, revise guides and packaging, and retest weekly until a simple time or complaint threshold is met.
Connected and consumer hardware products often fail at first contact: people skim quick-start guides, misread status lights and are unsure of cable orientation. Out-of-box testing hands participants the product exactly as shipped and watches unboxing, guide use, physical connection and registration without help. Its distinctive rigour is a simple stopping rule. BRCK set a 10 to 20 minute set-up target; Sifteo, unable to change a battery mechanism, tested ten arrow-sticker versions and each week had 10 to 15 people try set-up until fewer than 2 in 10 complained. Because problems are often found after hardware is frozen, fixes target the cheapest layers first: instructions, labels, packaging, videos and online help. Support contacts after launch continue the evaluation.
Attributes
- Discipline
- Usability & Evaluation
- Loss type
- Siloed
- Stage
- Evaluation, Post-launch
- Data
- Mixed
- Effort
- Weeks
Steps
- Give participants the packaged product exactly as shipped, with no briefing beyond the scenario.
- Observe unboxing, use of printed guides, physical connection, power-up and registration without help.
- Record confusions and time to a working state.
- Set a simple threshold, such as set-up within 10 to 20 minutes or fewer than 2 in 10 participants complaining.
- Revise the cheapest layers first when hardware is frozen: instructions, labels, stickers, packaging, videos and online help, keeping print and on-screen diagrams identical to the shipped part.
- Retest weekly with 10 to 15 fresh participants until the threshold is met.
- After launch, keep monitoring set-up problems through support contacts.
Sources
- Designing Connected Products (Rowland et al., 2015)
Also known as
- Out-of-Box Setup Testing with a Complaint Threshold
- Unboxing usability test
- First-run setup testing
- Out-of-box experience testing
Categories
- User Research
- Usability & Evaluation