Description
Load pages repeatedly from controlled locations, browsers and connections, and benchmark perceived speed with Speed Index, start render and filmstrips before and after each change.
Synthetic tests load a page from a third-party location and device so a team sees performance outside its own fast office set-up. Hogan recommends running five tests with first and repeat views and taking the median, inspecting the request waterfall for slow or blocking requests, and recording time to first byte, start render, visually complete, fully loaded and Speed Index, the average time at which visible parts of the page appear. Filmstrips at 0.1 second intervals show what the user sees and when. Tests are repeated for each responsive breakpoint, after every large change or experiment, against a performance budget and against competitors. Side-by-side filmstrip videos from several locations make the wait tangible for stakeholders. It measures perceived load, which ties performance to experience rather than to server metrics alone.
Attributes
- Discipline
- Usability & Evaluation
- Loss type
- Siloed
- Stage
- Design, Evaluation, Post-launch
- Data
- Quantitative
- Effort
- Hours
Steps
- Pick the major pages and test settings: location, browser, device and connection speed representative of users.
- Run five tests with first and repeat views and take the median run to avoid flukes.
- Inspect the waterfall for unusually long or render-blocking requests.
- Record time to first byte, start render, visually complete, fully loaded and Speed Index.
- Review the filmstrip at 0.1 s intervals to see when meaningful content appears.
- Repeat for each responsive breakpoint and compare results with the performance budget.
- Re-run before and after every large change or experiment, and optionally compile side-by-side filmstrips against competitors for stakeholders.
Sources
- Designing for Performance (Hogan, 2014)
Also known as
- Synthetic perceived-performance testing
- Speed Index benchmarking
- WebPagetest filmstrip comparison
- Synthetic performance testing
Categories
- User Research
- Usability & Evaluation