Evaluating Children's Interactive Products - Principles and Practices for Interaction Designers

FRONT COVER

EVALUATING CHILDREN'S INTERACTIVE PRODUCTS

COPYRIGHT PAGE

CONTENTS

PREFACE

ABOUT THE AUTHORS

PART 1 CHILDREN AND TECHNOLOGY

CHAPTER 1 WHAT IS A CHILD?

Age and Children

Learning about Children

Theories of Child Development

Perspectives on Child Development

Biological

Psychodynamic

Learning

Cognitive-Developmental

Sociocultural

Summary

Typical Stages of Development

Physical

Socioemotional

Cognitive

Child Development and the Evaluation of Interactive Products

The Temperament of Children

Temperamental Dimensions

Reducing the Effects of Temperament

Summary

Further Reading

CHAPTER 2 CHILDREN AND INTERACTIVE TECHNOLOGY

Interactive Products

What Is Interactivity?

Measuring Interaction

Interactive Products for Children

Entertainment Products

Education Products

Enabling Products

How Children Use Interactive Products

The PLU Model

Interactive Technology and Evaluation Studies

How Good Is Interactive Technology for Children?

Gender and Technology

Summary

Further Reading

CHAPTER 3 THE INTERACTIVE PRODUCT LIFECYCLE

Interaction Design and Evaluation

The Product Lifecycle

The Waterfall Model

The Usability Engineering Lifecycle Model

Using Lifecycle Models

The Place of Evaluation in the Lifecycle

Using Prototypes in Evaluation

Medium Used

Scope

Fit in the Lifecycle

Purpose

Involving Children in Design and Evaluation

Summary

Further Reading

PART 2 EVALUATING WITH AND FOR CHILDREN

CHAPTER 4 ETHICAL PRACTICE IN EVALUATIONS

Ethical Principles, Approaches, and Codes

Cultural Issues

Ethical Codes

Ethics and the Law

Safety and Risk Assessment

Consent

Consenting to Take Part in the Study

Deception

Covert Observation

Consent for Photographs and Audio and Video Recordings

Inducements for Taking Part

Ethical Selection of Participants

Privacy

Getting Advice and Permission

Further Reading

CHAPTER 5 PLANNING THE EVALUATION STUDY

Defining the Purpose of the Evaluation

Diagnostic Evaluations

Formative and Summative Evaluations

Exploratory, Measurement, and Experimental Studies

Evaluation Criteria

Choosing Evaluation Methods

Reliability of Evaluation Results

Field versus Lab: More Than Just a Location!

Control and Realism

Evaluating Products with Children in the Field

Goals and Testing Context

The Evaluation Plan—Why Make One?

Context Definition

Evaluation Goal

Evaluation Method

Location and Profile

Summary

Further Reading

CHAPTER 6 BEFORE THE EVALUATION

Deciding on People and Places

Confirming a Location and a Time

Selecting Child Participants

Selecting the Evaluation Team

Preparing Technology and Evaluation Instruments

Preparing the Technology

Designing Test Tasks

Evaluation Instruments and Instructions

Planning the Day

Space

Managing Children, Tasks, and Technology

Training Evaluators and Pilot Testing

Communicating the Details

Next Stages

CHAPTER 7 DURING THE EVALUATION

Arrival and Setup

Preparing the Evaluation Team

Setting up the Space

Introductions and Instructions

Children in Adult Places

Adults in Children's Places

Giving Instructions

Managing Time and Space

Working with Schoolchildren

Monitoring the Tasks

Dealing with Problems on the Day

Closing the Evaluation

Leaving the Area Tidy

Saying "Thank You!"

CHAPTER 8 AFTER THE EVALUATION

Getting the Data into Shape

Coding and Analyzing the Data

Coding and Analyzing Quantitative Data

Presenting Summaries of Quantitative Data

Analyzing Qualitative Data

Reporting Results

Know Your Audience

Structure and Style

Oral Presentation of Results

Care of Data

How Long Should Data Be Kept?

Reflecting on the Process

Summary

PART 3 METHODS OF EVALUATION

CHAPTER 9 RECORDING AND LOGGING

Automated Logging

Recording the Data

Software for Logging

Practicalities

Video and Audio

The Technology

Setup and Use in Usability Labs and Elsewhere

Gaze Tracking

Gaze-Tracking Measurements

Gaze-Tracking Technology

Hardware for Participants

Setup and Calibration with Children

Using Gaze Tracking with Children

Summary

Further Reading

CHAPTER 10 OBSERVATION METHODS

Types of Observation

Realism in Observation Studies

Participant Observation

Passive and Naturalistic Observation

Structured Observation

Determine the Focus

Develop Guides and Forms

Recruit and Train Observers

Carry out the Observation

Analyze and Interpret Findings

Using Standard Coding Schemes

Summary

CHAPTER 11 VERBALIZATION METHODS

Types of Verbalization Methods

Think-Aloud

Variants of Think-Aloud

The Picture Cards Method

Other Variants of Think-Aloud

Dialogue between the Administrator and the Testers

Active Intervention

Post-Task Interview

Robotic Intervention

Interaction between Testers

Constructive Interaction

Peer Tutoring

Methodological Issues of Verbalization Methods

Advantages and Disadvantages

Performance

Which Children Should You Test?

Analyzing Verbal Protocols

Summary

Further Reading

CHAPTER 12 THE WIZARD OF OZ METHOD

Wizard of Oz Studies

Variability

Studies with Children

When to Use a WOz Study

A Walkthrough of a WOz Evaluation

The Wizard Interface

Overcoming Problems

Managing the Evaluation Study

Stages in Planning a Study

Problems

Children as Users of Technology

The Wizard Is a Human Being

Summary

Further Reading

CHAPTER 13 SURVEY METHODS

What Is a Survey?

The Purpose of Surveys in Evaluation

Why Use a Survey Method?

Designing a Survey

Designing the Questions

What Can Go Wrong?

Satisficing and Optimizing

Specific Question Formats

Language Effects

Carrying out the Survey

Minimizing Suggestibility

Reliability of Findings

The Fun Toolkit

Guidelines for Using the Fun Toolkit

Summary

Further Reading

CHAPTER 14 DIARIES

Field Evaluation

Using Diaries to Evaluate Interactive Products with Children

Design the Study

Prepare the Diary Questions and Materials

Monitor and Encourage Diary Keeping

Perform the Debriefing Interview

The Parent Evaluator Method

Procedure

Structuring Diaries

How Good Is the Method?

More Guidelines for Diary Studies

Summary

Further Reading

CHAPTER 15 INSPECTION METHODS

Heuristic Evaluation

The Process

Nielsen's Usability Heuristics

Problem Reporting

Problems with Heuristic Evaluation of Children's Products

Using the Heuristic Evaluation Method with Other Heuristics

Heuristics for Evaluating Fun

Heuristics for Evaluating Games

Heuristics for Evaluating Web Sites

Using Children as Evaluators

The SEEM Method

Persona-Based Evaluation

How Good Are Inspection Methods?

Summary

Further Reading

PART 4 CASE STUDIES

CHAPTER 16 CASE STUDY 1: GAME-CONTROLLING GESTURES IN INTERACTIVE GAMES

Finding a Suitable Evaluation Method

The Study

Preparations

Interactive Prototypes

Test Setup

Participants

Test Space Design

Procedure

Children's Movement Analysis

Preliminary Video Analysis

Experimenting with Movement Description Methods

Results from the Study

Intuitive Movements for the Swimming Game

Commentary on the Study

Ethics

Test Arrangements

Pilot Analysis of the Test Data

Summary

CHAPTER 17 CASE STUDY 2: EMBEDDING EVALUATION IN THE DESIGN OF A PERVASIVE GAME CONCEPT

The Design Project Context

Children and the Design of Camelot

The Mission from Mars Method

The Interview

Paper Prototypes with Observations and Picture Card Interviews

Paper Prototype Evaluations

Reflections on Concept Testing

Evaluating Interaction Styles with Peer Tutoring

The Camelot Game

Peer Tutoring Evaluations

Surveying Form Preferences

The Paired Comparison Test

Evaluation Summary of the Camelot Design Concept

Play Testing of the Camelot Prototype

Summary

CHAPTER 18 CASE STUDY 3: USING SURVEY METHODS AND EFFICIENCY METRICS

The Study

Training

Test Task Design

Constructs Being Evaluated

Results

Commentary

Efficiency

Satisfaction

Summary

REFERENCES

INDEX

A

B

C

D

E

F

G

H

I

J

K

L

M

N

O

P

Q

R

S

T

U

V

W

Y