Description
Rates a study's measurement situation on four ordered dimensions of similarity to the real system, so its realism and generalisability can be planned and judged.
Simulation is not all-or-nothing: every study decides how much of the operational system and environment to reproduce, a matter of hypothesis and judgement. Meister's profile rates four dimensions, each ordered from most to least realistic. Measurement environment: actual operational setting, operational setting reproduced on film or audio, test site, high-fidelity simulator, low-fidelity or part-task simulator or mock-up, laboratory. Measurement unit size: full system, subsystem, task or workstation, part task, single control or display, synthetic system. Performance measured: mission, function, task, subtask, non-equipment task. Task characteristics: real-world activity, simulation of it, abstraction of it, non-equipment task, non-task activity. Physical fidelity is assumed to yield behavioural fidelity if participants resemble real users, but this is untested. Coding two years of conference papers showed 46% were in the laboratory and 50% used non-operational equipment, a measure of how far research sits from real use.
Attributes
- Discipline
- Research Design & Rigor
- Loss type
- Latent
- Stage
- Evaluation, Design
- Data
- Analytical / expert
- Effort
- Hours
Steps
- Decide which operational characteristics plausibly affect performance for the question at hand, and ignore those that do not.
- Locate the planned or published study on each of the four dimensions: environment, unit size, performance measured and task characteristics.
- Where realism matters, move up the relevant dimension (field site, high-fidelity prototype, live product) within budget; otherwise accept the lab.
- Record the profile in the study plan and report so that readers can judge generalisability.
- Use the profile to audit a body of research (for example a research repository) and spot where evidence relies only on low-fidelity situations.
Sources
- Conceptual Foundations of Human Factors Measurement (Meister, 2004)
Also known as
- Simulation dimensions (Meister 1998)
- Measurement realism profile
Categories
- Research Operations
- Research Design & Rigor