AR Magic Lenses - Computer Science and Software Engineering

261 Pages·2008·5.55 MB·English

AR Magic Lenses: Addressing the Challenge of Focus
and Context in Augmented Reality
A thesis
submitted in partial fulfilment
of the requirements for the Degree
of
Doctor of Philosophy
in the
University of Canterbury
by
Julian Looser
Examining Committee
Associate Professor Andy Cockburn Supervisor
Professor Mark Billinghurst Co-Supervisor
Professor Bruce Thomas Examiner
Professor Hans Gellersen Examiner
University of Canterbury
2007
Declaration
All contributions described in this thesis are entirely my own, with the
exception of cases where I collaborated with other researchers. The follow-
ing list identifies such parts of my research, and clarifies the roles of the
contributors.
Chapter 5 - Stencil Lenses: Theworkonstencil-basedARMagicLenses
was primarily my own. I developed the software, explored the application
space, and wrote the majority of the paper describing the work. The publi-
cation, presented at GRAPHITE 04, was written with the assistance of my
supervisors, Mark Billinghurst and Andy Cockburn.
Chapter 5 - TankWar: The Augmented Reality game, TankWar, de-
scribed in this chapter was the idea of Trond Nilsen, and became the topic
of his Masters research (Nilsen 2006). My contribution to TankWar was
software development of the first OpenGL version, which included the AR
Magic Lens interaction techniques and a transitional (AR-VR) interface. In
addition, I provided technical advice on the development of Trond’s sec-
ond iteration of TankWar, built with Open Scene Graph. I co-authored a
paper about TankWar with Trond and Steven Linton (Nilsen, Linton and
Looser 2004), and helped demonstrate TankWar at the New Zealand Game
Developers Conference in 2004.
Chapter 6 - osgART: The software library, osgART, was originally my
invention, as I was individually responsible for developing the first version.
osgART has since become a collaborative project, being actively maintained
and improved by myself, Hartmut Seichter, Rapha¨el Grasset and Phil Lamb.
osgART has been released under GPL and commercial licenses, and was
presentedattheIndustrialWorkshopatISMAR06(Looser,Grasset,Seichter
and Billinghurst 2006).
Chapter 7 - AR Selection Experiment: The experiment evaluating
AR Magic Lenses as a selection technique was designed and run by myself.
The resulting publication, presented at GRAPHITE 07, was written with the
assistance of Rapha¨el Grasset and Mark Billinghurst.
Chapter 8 - Flexible Sheet Lenses: Theinventionofflexiblesheetlenses
is my own, and I wrote the software implementation. This work was ex-
ploratory and I had many discussions about potential applications and inter-
action techniques with Rapha¨el Grasset. Our publication about the flexible
lens, presented by Rapha¨el at ISMAR 07, was written with the additional
assistance of Mark Billinghurst.

Dedicated to
Alan and Frieda

Abstract
In recent years, technical advances in the field of Augmented Reality
(AR), coupled with the acceleration in computer and graphics processing
power, have brought robust and affordable AR within the reach of the wider
research community. While the technical issues of AR remain heavily re-
searched, there is also a growing amount of work on user interface develop-
ment and evaluation, heralding the convergence of traditional Human Com-
puter Interaction (HCI) and AR.
Magic Lenses are 2D interface components that provide alternative rep-
resentations of objects seen through them. In this way, they can be used
to provide Focus and Context in the interface, especially when visualising
layered information. There are very few, if any, formal evaluations to guide
the development of lens-based interfaces.
This thesis describes the development and evaluation of Magic Lenses as
a tool for AR interfaces. The work starts with a comprehensive survey of
many Focus and Context techniques, which are classified based on the way
they present views to the users { for example, a Magic Lens is a spatially sep-
arated multiple view technique. A formal evaluation of 2D Magic Lenses in
a GIS scenario found that users strongly preferred the lens-based interaction
technique to others, largely because it reduced the effort of interaction. Ac-
curacy was high with the lenses, but a simple “global view” interface allowed
significantly faster performance.
This positive result motivated further work on Magic Lenses within AR,
where the lens metaphor can reinforce the tangible interaction methods that
link virtual and real content. To support rapid exploration of interaction
alternatives with AR Magic Lenses, I describe the design and architecture of
osgART, an AR development toolkit that is available to the research com-
munity as open-source software.
Object selection and manipulation is a fundamental interaction require-
ment for all AR interfaces, and I establish an empirical foundation of per-
formance in this task with a variety of AR interaction techniques, including
Magic Lenses. Results show that performance with all techniques is suc-
cessfully modelled by Fitts’ Law, and that Magic Lenses outperformed other
techniques.
Finally, I examine new interaction techniques based on Magic Lenses,
particularly a Flexible Sheet Lens, which allows concurrent bimanual speci-
fication of multiple parameters within the visualisation.
Table of Contents
List of Figures v
List of Tables x
Chapter 1: Introduction 1
1.1 Overview of Thesis and Research Approach . . . . . . . . . . . 1
1.2 Chapter Summary . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Chapter 2: Related Work 6
2.1 Focus and Context . . . . . . . . . . . . . . . . . . . . . . . . 6
2.1.1 An Overview of Graphical User Interfaces . . . . . . . 6
2.1.2 MVC: The Separation of Views from Models . . . . . . 10
2.1.3 Multiple View Systems . . . . . . . . . . . . . . . . . . 11
2.2 A Taxonomy of Views for Supporting Focus and Context . . . 13
2.2.1 Single View Focus and Context . . . . . . . . . . . . . 14
2.2.2 Multiple View Focus and Context . . . . . . . . . . . . 33
2.2.3 Summary of Focus and Context . . . . . . . . . . . . . 45
2.3 Magic Lenses . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
2.3.1 2D Magic Lens Systems . . . . . . . . . . . . . . . . . 49
2.3.2 3D Magic Lens Systems . . . . . . . . . . . . . . . . . 55
2.3.3 Summary of Magic Lens Systems . . . . . . . . . . . . 62
2.4 Augmented Reality . . . . . . . . . . . . . . . . . . . . . . . . 62
2.4.1 The History of Augmented Reality . . . . . . . . . . . 65
2.4.2 Providing an AR Experience . . . . . . . . . . . . . . . 66
2.4.3 Usability Testing in Augmented Reality . . . . . . . . . 72
2.4.4 Applications for AR . . . . . . . . . . . . . . . . . . . 74
2.4.5 Summary of Augmented Reality . . . . . . . . . . . . . 76
2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
Chapter 3: Fundamentals of Magic Lenses 77
3.1 Properties of Magic Lenses . . . . . . . . . . . . . . . . . . . . 77
3.2 Analysis of Magic Lens Applications . . . . . . . . . . . . . . 84
3.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
Chapter 4: 2D Magic Lens Evaluation 90
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
4.2 Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
4.3 Experimental Design . . . . . . . . . . . . . . . . . . . . . . . 93
4.3.1 Apparatus . . . . . . . . . . . . . . . . . . . . . . . . . 93
4.3.2 Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . 94
4.3.3 Participants . . . . . . . . . . . . . . . . . . . . . . . . 96
4.3.4 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . 99
4.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
4.4.1 Subjective Results . . . . . . . . . . . . . . . . . . . . 107
4.5 Additional Analysis of Manipulation Cost . . . . . . . . . . . 110
4.5.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
4.5.2 Visual Analysis of Mouse Movement . . . . . . . . . . 115
4.5.3 Summary of Manipulation Cost . . . . . . . . . . . . . 115
4.5.4 Reducing the cost of Magic Lens Manipulation . . . . . 117
4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
Chapter 5: Augmented Reality Magic Lenses 124
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
5.2 Motivation for AR Magic Lenses . . . . . . . . . . . . . . . . . 124
5.3 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . 127
5.4 AR Magic Lens Applications . . . . . . . . . . . . . . . . . . . 132
5.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
Chapter 6: osgART 144
6.1 Scene Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
6.2 Open Scene Graph and ARToolKit Integration . . . . . . . . . 145
6.2.1 Implementation of osgART . . . . . . . . . . . . . . . . 147
6.2.2 osgART Example Code . . . . . . . . . . . . . . . . . . 149
6.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
ii