Multi-Modal User Interactions in Controlled Environments

Foreword

Preface

Acknowledgements

Contents

Chapter 1 ntroduction

1.1 Introduction

1.2 Objective

1.3 Practical Applications

1.4 Research Challenges

1.4.1 Event Detection

1.4.2 Flow Estimation

1.4.3 Gaze Estimation

1.4.4 Role of the Context

1.4.5 Societal Issues

1.5 Technical Contribution

1.6 How the Book is Organized

Chapter 2 Abnormal Event Detection

2.1 Introduction

2.2 Related Work

2.3 Proposed Approach

2.3.1 Low-Level Features

2.3.1.1 Motion Heat Map

2.3.1.2 Points of Interest

2.3.1.3 Tracking Points of Interest

2.3.1.4

2.3.2 Intermediate-Level Features

2.3.2.1 Motion Area Ratio

2.3.2.2 Direction Variance

2.3.2.3 Motion Magnitude Variance

2.3.2.4 Direction Histogram

2.3.2.5 Direction Map

2.3.2.6 Difference of Direction Map

2.3.3 Other Intermediate-Level Features

2.3.3.1 Motion Continuity Factor

2.3.3.2 Motion Description Factor

2.3.3.3 Motion Trajectory of the Blob

2.3.4 High-Level Features

2.3.4.1 Detecting Collapsing Events

2.3.4.2 Detecting Opposing Flow Event

2.4 Group Detection and Tracking

2.4.1 Detection and Tracking of PoIs

2.4.2 Direction and Magnitude Models

2.4.3 Block Clustering

2.4.4 Group Tracking

2.5 Detecting Multiple Flows and Events in a Crowd Scene

2.5.1 Multiple Flow Detection

2.5.2 Event Recognition

2.5.3 Running and Walking Events

2.5.4 Crowd Convergence and Divergence Events

2.5.5 Results

2.6 Method Adaptation to Context

2.6.1 Overview

2.6.2 Context Factors

2.6.3 Method Extensions

2.6.4 Experiments

2.6.4.1 Some examples

2.6.4.2 Data Set

2.6.4.3 Methodology

2.6.5 Results

2.7 Conclusion

Chapter 3 Flow Estimation

3.1 Introduction

3.2 Related Works

3.2.1 Methods based on Motion Detection and Analysis

3.2.1.1 Approach Proposed by Xu and al. [149]

3.2.1.2 Approach Proposed by Zhang and Chen [153]

3.2.2 Methods Based on Contour Analysis

3.2.2.1 Approach Proposed by Bozzoli and al. [19]

3.2.3 Template-Based Methods

3.2.3.1 Approach Proposed by Sidla and al. [120]

3.2.4 Stereovision-Based methods

3.2.4.1 Approach Proposed by Terada and al. [130]

3.2.5 Spatio-Temporal Methods

3.2.5.1 Approach Proposed by Albiol and al. [1]

3.2.6 Commercial Applications

3.2.6.1 Cognimatics

3.2.6.2 Infodev

3.2.6.3 Eurecam Sarl

3.2.7 Contribution

3.3 Approach Steps

3.3.1 Blob Detection

3.3.2 Count Estimation

3.4 Experiments and Results

3.5 Conclusion

Chapter 4 Estimation of Visual Gaze

4.1 Human Vision System

4.2 History of Gaze Tracking

4.3 Gaze Tracking Techniques

4.3.1 Intrusive Systems

4.3.1.1 Electro-Oculography

4.3.1.2 Contact Lenses with Magnetic Coils

4.3.1.3 Localization of the Limb

4.3.1.4 Analysis of Eye Images

4.3.2 Non-Intrusive Systems

4.4 Applications

4.4.1 Interaction During Meetings

4.4.2 Driver Monitoring

4.4.3 Virtual Reality

4.4.4 Human Computer Interaction

4.4.5 Extraction of Saliency Maps in Images

4.4.6 Store Marketing

4.5 Contribution of Head Pose in Visual Gaze

4.5.1 Database

4.5.2 Calculating the Contribution of Head Pose

4.5.3 Prediction of the Target

4.6 Estimating Gaze Direction Based on Eye Localization Only

4.7 Head Pose Estimation

4.7.1 State of the Art

4.7.1.1 Definition

4.7.1.2 Human Capacity for Estimating Head Orientation

4.7.1.3 Problems Encountered when Estimating Head Poses

4.7.2 Image Datasets

4.7.2.1 Building Image Datasets

4.7.2.2 Utilized Image Database

4.7.3 Estimation of Head Pose Based on Global Appearance

4.7.3.1 Utilized Image Dataset

4.7.3.2 Feature Selection

4.7.3.3 Experimental Results

4.7.4 Cylindrical Model for Head Tracking

4.8 Conclusion

Chapter 5 Visual Field Projection and Region of Interest Analysis

5.1 Visual Field Estimation

5.1.1 Physiological Data

5.1.2 Visual Field Estimation and Fixation Point for Frontal Pose

5.1.3 Visual Field Adaptation to Head Orientation

5.1.3.1 Matrix Approach

5.2 Visual Field Projection

5.2.1 Point Projection

5.2.2 Perception Volume Projection

5.3 Visual Field Display and Projection on an Image

5.4 Region-of-Interest Extraction

5.4.1 Representation of Gaze Information

5.4.2 Gaze Point Correction

5.4.3 Calculation of Tilt and Pan Angles Corresponding to a Gaze Point

5.5 Metrics for Gaze Analysis

5.5.1 Construction of a System Measuring Media Relevance

5.5.1.1 Raw Data Collection

5.5.1.2 Identification of Fixations

5.5.2 Metrics Related to Fixation Distribution

5.5.3 Experiment

5.5.3.1 Images

5.5.3.2 Videos

5.5.4 Discussions

5.6 Conclusion

Chapter 6 Conclusion

6.1 Challenge

6.2 Perspectives

References

Appendix A Societal Recommendations

A.1 Societal Recommendations

A.1.1 Public Awareness

A.1.1.1 Observation and Surveillance Technology Awareness

A.1.1.2 Legal Awareness

A.1.1.3 Recommendation 1

A.1.2 Public Policy of Research and Development

A.1.2.1 EU R&D Awareness

A.1.2.2 Recommendation 2

A.1.2.3 Democratic Discussion

A.1.2.4 Recommendation 3

A.1.2.5 R&D Program Evaluation

A.1.2.6 Recommendation 4

A.1.2.7 Management and Human Science Researchers’ Role

A.1.2.8 Recommendation 5

A.1.3 Democratic Requirement for OST Regulation

A.1.3.1 Reinforcement of Public Authorities’ Assets with Regard to the Protection of Privacy and Data Protection

A.1.3.2 Recommendation 6

A.1.3.3 Intelligibility of the OST Systems

A.1.3.4 Accessibility to the OST Systems

A.1.3.5 Recommendation 7

A.1.3.6 Legitimacy of the OST’s Finalities

A.1.3.7 Recommendation 8

A.1.3.8 Recommendation 9

A.1.3.9 Privatization of Public Issues

A.1.3.10 Recommendation 10

A.2 Legal Recommendations

A.2.1 Data Protection and Privacy Issues

A.2.1.1 Recommendation 11

A.2.1.2 Recommendation 12

A.2.1.3 Recommendation 13

A.2.1.4 Recommendation 14

A.2.1.5 Recommendation 15

A.2.1.6 Recommendation 16

A.2.1.7 Recommendation 17

A.2.1.8 Recommendation 18

A.2.1.9 Recommendation 19

A.2.1.10 Recommendation 20

A.2.1.11 Recommendation 21

A.2.2 Beyond Data Protection

A.2.2.1 Data Protection and Fundamental Liberties

A.2.2.2 Recommendation 22

A.2.2.3 Recommendation 23

A.2.2.4 Recommendation 24

A.2.2.5 Specific Provisions about Terminals and Infrastructures

A.2.2.6 Recommendation 25

A.2.2.7 Recommendation 26

A.2.2.8 Recommendation 27

A.2.2.9 Recommendation 28

A.2.2.10 Recommendation 29

A.2.2.11 Recommendation 30

A.2.2.12 Towards a Regulation of Profiling Activities

A.2.2.13 Recommendation 31

A.2.2.14 Recommendation 32

A.2.2.15 Recommendation 33

A.2.2.16 Recommendation 34

A.2.2.17 Recommendation 35

A.2.3 Consumer Protection

A.2.3.1 Recommendation 36

A.2.3.2 Recommendation 37

A.2.3.3 Recommendation 38

A.2.3.4 Recommendation 39

Glossary

Index