Automatically Generating High-Quality User Interfaces for Appliances

358 Pages·2006·6 MB·English

by  Jeffrey Nichols

Automatically Generating High-Quality
User Interfaces for Appliances

Jeffrey Nichols

December 2006
CMU-HCII-06-109

Human Computer Interaction Institute
School of Computer Science
Carnegie Mellon University
Pittsburgh, Pennsylvania 15213

Thesis Committee:
Brad A. Myers, Chair
Scott E. Hudson
John Zimmerman
Dan R. Olsen, Jr., Brigham Young University

Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy

Copyright © 2006 Jeffrey Nichols. All rights reserved.

This research was supported in part by the National Science Foundation (through the author’s Graduate Re-
search Fellowship and grants IIS-0117658 and IIS-0534349), Microsoft, General Motors, the Pittsburgh
Digital Greenhouse, and Intel. Equipment supporting this research was generously donated by Mitsubishi Elec-
tric Research Laboratories, VividLogic, Lantronix, Lutron, IBM Canada, Symbol Technologies, Hewlett-
Packard, and Lucent. The views and conclusions contained herein are those of the author and should not be
interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of any
sponsoring party or the U.S. Government.
Keywords: Automatic interface generation, aggregate user interfaces, handheld computers,
personal digital assistants, mobile phones, home theater, appliances, personal universal con-
troller (PUC), user interface description languages (UIDLs), remote controls, multi-modal
interfaces, speech recognition, Smart Templates, user interface consistency, personal consis-
tency, familiarity
ii
Automatically Generating High-Quality
User Interfaces for Appliances

Jeffrey Nichols

iii
iv
Abstract
In this dissertation, I show that many appliance usability problems can be addressed by mov-
ing the user interface from the appliance to a handheld device that the user is already
carrying, such as a Personal Digital Assistant (PDA) or mobile phone. This approach, called
the Personal Universal Controller (PUC), takes advantage of the increasing pervasiveness of
wireless communication technologies that will allow handheld devices to communicate di-
rectly with appliances. Automatic generation of the appliance user interface allows the PUC
to create interfaces that are customized to the platform of the controller device, the user’s
previous experience, and all the appliances that are present in the user’s current environment.
This dissertation makes several contributions to the state of the art in automatic interface
generation:
• Automatic generation that makes use of dependency information to determine a better
structure for the generated user interfaces
• The general Smart Templates technique for incorporating domain-specific design con-
ventions into an appliance specification and automatically rendering the conventions
appropriately on different platforms and in different interface modalities
• Algorithms that apply knowledge of similarity between specifications and interfaces the
user has previously seen to generate new interfaces that are personally consistent
• Algorithms that use a model of the content flow between appliances to generate task-
based interfaces that combine functionality from multiple appliances
An evaluation of the PUC system compared the automatically generated interfaces for two
all-in-one printers with the manufacturer’s interfaces for the same two appliances and found
that users of the automatically generated interfaces were twice as fast and four times as suc-
cessful for both common and complex tasks. The evaluation also shows that the PUC’s
consistency features allow users to be twice as fast when using a new appliance that is similar
to an appliance they have previously encountered. This evaluation is the first known user
study of automatically generated interfaces compared to human designs, and it shows that
automatic generation of user interfaces for end users is now viable for interactive systems.

v
vi
For my parents

vii
viii
Table of Contents
Abstract.................................................................................................................................v
List of Figures...................................................................................................................xvii
List of Tables.....................................................................................................................xxv
Acknowledgements.........................................................................................................xxvii
1 Introduction 1
1.1 The Personal Universal Controller.....................................................................7
1.2 Outside the Scope............................................................................................13
1.3 Contributions..................................................................................................14
1.4 Dissertation Overview......................................................................................15
2 Related Work 17
2.1 Control of Appliances......................................................................................18
2.1.1 Commercial Products.............................................................................18
2.1.2 Commercial Standards...........................................................................20
2.1.2.1 INCITS/V2 Standard.............................................................20
2.1.2.2 Universal Plug and Play..........................................................21
2.1.2.3 Digital Living Network Alliance..............................................22
2.1.2.4 Home Audio-Video Interoperability.........................................22
2.1.2.5 JINI.......................................................................................23
2.1.2.6 OSGi.....................................................................................23
2.1.3 Research Systems....................................................................................23
2.1.3.1 Universal Interactor................................................................23
2.1.3.2 IBM Universal Information Appliance.....................................24
2.1.3.3 ICrafter..................................................................................24
2.1.3.4 Xweb......................................................................................25
2.1.3.5 Ubiquitous Interactor..............................................................25
ix
2.1.3.6 Analyses of Actual Remote Control Usage..................................26
2.1.3.7 DiamondHelp.........................................................................26
2.1.3.8 Roadie....................................................................................26
2.2 Automatic & Guided User Interface Design.....................................................27
2.2.1 Early Model-Based Systems.....................................................................28
2.2.1.1 Mickey....................................................................................28
2.2.1.2 Jade........................................................................................29
2.2.1.3 UIDE.....................................................................................29
2.2.1.4 Humanoid..............................................................................30
2.2.1.5 Mastermind............................................................................30
2.2.1.6 ITS........................................................................................30
2.2.1.7 TRIDENT.............................................................................31
2.2.2 Model-Based Systems for Very Large Interfaces and Platform Independence31
2.2.2.1 Mobi-D..................................................................................32
2.2.2.2 ConcurTaskTrees....................................................................32
2.2.2.3 XIML.....................................................................................33
2.2.2.4 IBM PIMA and MDAT.........................................................33
2.2.2.5 UIML and TIDE...................................................................33
2.2.2.6 TERESA................................................................................34
2.2.2.7 USIXML................................................................................34
2.2.2.8 XAML and XUL.....................................................................34
2.2.2.9 SUPPLE................................................................................35
2.3 Aggregate User Interfaces.................................................................................36
3 Preliminary User Studies 39
3.1 Hand-Designed User Interfaces........................................................................39
3.2 User Studies.....................................................................................................42
3.2.1 Procedure..............................................................................................42
3.2.2 Evaluation............................................................................................42
3.3 Study #1..........................................................................................................43
x