Design Considerations: Which method should I use?

Which method should I use?

One way to think about which method I should use can be made by answering following questions (refer to as ‘PRET A Reporter’ framework)

• Purpose of the evaluation: what are the goals of testing, or the detailed questions to be answered in the study? (this will find need and requirement that method was developed to address)

• Resources and constraints: what resources are available for conducting the study and conversely what constraints must the study work within? this will address the challenges that might come during testing

• Ethics: what ethical considerations need to be addresses? In method testing, the main ethical concerns are likely to relate to privacy of participants information.

• Techniques for gathering data must be identified, such as controlled experimentation, questionnaire(design, length, structure, wording, types and scales), interviews (structure, style, settings, and recording) and focus groups (break into groups and facilitate discussion) and eye tracking

• Analysis techniques must be selected such as GOMS [2], KLM [3],

• Reporting of findings.

Many of the tests of methods that have been conducted are comparative

--

[2] John, B. E. and Kieras, D.E 1996a, ’The GOMS family of user interface analysis technique: comparison and contrast.’ ACM Transaction on Computer-Human Interaction 3(4): 321-55

[3] Kieras, D.E 2001, ‘Using the keystroke-level model of estimate execution times.’ Unpublished manuscript, University of Michigan