Dynamic physical affordances for shape-changing and deformable user interfaces

222 Pages·2015·31.52 MB·English

Dynamic Physical Affordances for Shape-Changing
and Deformable User Interfaces
AF
by
Sean Weston Follmer "^*OSoFFC TECEHTNSO NLTYITUTE
B.S., Stanford University (2009)
JUN 12 2015
S.M., Massachusetts Institute of Technology (2011)
LIBRARIES
Submitted to the Program in Media Arts and Sciences,
School of Architecture and Planning
in partial fulfillment of the requirements for the degree of
Doctor of Philosophy in Media Arts and Sciences
at the
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
February 2015
Massachusetts Institute of Technology 2015. All rights reserved.
Signature redacted
Author.........................................
Program in Media Arts and Science
January 9, 2015
Signature redacted
Certified by ..............
(4 Prof. Hiroshi Ishii
Jerome B. Wiesner Professor of Media Arts and Sciences
Program in Media Arts and Sciences
s Supervisor
Signature redacted
Ac cepted by ...........................
Prof P tie Maes
Alexander W. Dreyfoos (1954) Prorssor of Media Arts and Sciences
Associate Academic Head
Program in Media Arts and Sciences
2
Dynamic Physical Affordances for Shape-Changing and
Deformable User Interfaces
by
Sean Weston Follmer
Submitted to the Program in Media Arts and Sciences,
School of Architecture and Planning
on January 9, 2015, in partial fulfillment of the
requirements for the degree of
Doctor of Philosophy in Media Arts and Sciences
Abstract
The world is filled with tools and devices designed to fit specific needs and goals, and their
physical form plays an important role in helping users understand their use. These physical
affordances provide products and interfaces with many advantages: they contribute to good
ergonomics, allow users to attend to other tasks visually, and take advantage of embodied
and distributed cognition by allowing users to offload mental computation spatially. How-
ever, devices today include more and more functionality, with increasingly fewer physical
affordances, losing many of the advantages in expressivity and dexterity that our hands can
provide.
My research examines how we can apply shape-changing and deformable interfaces to
address the lack of physical affordances in today's interactive products and enable richer
physical interaction with general purpose computing interfaces. In this thesis, I introduce
tangible interfaces that use their form to adapt to the functions and ways users want to
interact with them. I explore two solutions: 1) creating Dynamic Physical Affordances
through shape change and 2) user Improvised Physical Affordances through direct defor-
mation and through appropriation of existing objects. Dynamic Physical Affordances can
provide buttons and sliders on demand as an application changes, or even allow users to
directly manipulate 3D models or data sets through physical handles which appear out of
the data. Improvised Physical Affordances can allow users to squeeze, stretch, and deform
input devices to fit their needs, creating the perfect game controller, or shaping a mobile
phone around their wrist to form a bracelet.
Novel technical solutions are needed to enable these new interaction techniques; this
thesis describes techniques both for actuation and robust sensing for shape-changing and
deformable interfaces. Finally, systems that utilize Dynamic Physical Affordances and Im-
provised Physical Affordances are evaluated to understand patterns of use and performance.
My belief is that shape-changing UI will become increasingly available in the future, and this
work begins to create a vocabulary and design space for more general-purpose interaction
for shape-changing UI.
Thesis Supervisor: Prof. Hiroshi Ishii
Title: Jerome B. Wiesner Professor of Media Arts and Sciences
Program in Media Arts and Sciences
3
4
Dynamic Physical Affordances for Shape-Changing and Deformable User
Interfaces
by
Sean Follmer
n
0I
Signature redacted
/
Thesis Reader ...............
Bj6rn Hartmann
Assistant Professor of Electrical Engineering and Computer Science
University of California, Berkeley
5
6
Dynamic Physical Affordances for Shape-Changing and Deformable User
Interfaces
by
Sean Follmer
redacted
Signature
T h esis R ead er ................ ...................................................
Joseph Paradiso
Associate Professor of Media Arts and Sciences
Massachusetts Institute of Technology
7
8
Acknowledgments
I would like to thank Prof. Hiroshi Ishii, my advisor and mentor for my time at the
MIT Media Lab. His support and inspiration has made all of this work possible. Hi-
roshi would always challenge me to think bigger, often asking, "What is your vision?"
But most of all, his excitement was contagious and his energy and passion undying,
which will always give me something for which to strive. In addition, I would like
to thank my thesis committee, Prof. Joseph Paradiso and Prof. Bj6rn Hartmann.
Their advice and guidance through my research process has been invaluable. Bj6rn
has long been guiding and inspiring me, ever since I was an undergrad at Stanford,
and has served as a model and a benchmark for what good HCI research should look
like. I would also like to thank Hayes Raffle and Scott Klemmer, two people who
helped me find my way to the Media Lab and both showed me how much fun HCI
could be.
I have had the pleasure of collaborating with some of the most interesting and in-
spiring researchers at the Media Lab. Specifically, Daniel Leithinger, whose friendship
and collaboration I am most thankful for. The rest of the members of the Tangible
Media Group have been a pleasure to bounce ideas off of and work shoulder to shoul-
der with in the late nights leading up to sponsor weeks or CHI deadlines. Alex Olwal,
though not always in TMG, has been a close collaborator and a source for helping to
organize and order our crazy ideas. Beyond TMG, many others at the lab have made
me consider new ideas and crazy schemes over drinks at the Muddy Charles, tea at
Voltage, or lunch at Sloan or Momogoose (formerly Gooseberry), like Dave, David,
and David. I especially enjoyed borrowing things from High Low tech as an excuse to
hang out in Sam, Jenifer, and Dave's office. Thanks to Seth, Marcelo, and Jamie for
meeting with me even before I started and for helping me find my way when I did.
J.B. for guiding me when I first got to the lab. James Patten and Jen Lusker for their
awesome help with the TRANSFORM and Cooper Hewitt version of inFORM. Brian
Mayton helped me understand electric field sensing. Mark Feldmeier always helped
to solve any electrical engineering problem that ever came his way. Even though he
9
might not believe it, I think that Matt Hirsch was the best student at the Lab, and
I thank him for helping me and for being around.
This work could not have been possible without the tremendous help and collab-
oration of MIT undergraduate researchers. Akimitsu Hogge was always there when
we needed him. A large cast of amazing mechanical engineers helped with the de-
sign and implementation of inFORM: Guangtao Zhang, Alyx Daly, Cheetiri Smith,
Keenan Sunderwirth, Pat Capulong and David Doan.
So many wonderful people at MIT have greased wheels and done too many favors
for me. Specifically, Mary Tran Niskala, who has long been my office mate and a all
around great person to be around. Natalia, Sarah, and Lisa for helping out before
that. It has been a pleasure over the years to get to know Kevin Davis one of the nicest
and most helpful people I have ever met. I thank Tom Lutz and John DiFrancesco for
their help in the shop. Also, the amazing press and communications staff at the Media
Lab: Alexandra and Stacy. MAS staff Aaron, Bill, and Keira always brightened my
day. And, last but not least, the great Linda Peterson.
Finally, I would like to thank my family for their support and love. My dad for
taking me to SIGGRAPH '98 in Orlando, not only because of Disney World, but also
because of the amazing inventions and technologies on display. My mom for always
encouraging me to study, and for instilling a life long love of education and reading
in me. And Kendall, for putting up with me even when I was stressed about finishing
this document, and for fixing all of my spelling mistakes!
This research was supported in part by the National Science Foundation Graduate
Research Fellowship under Grant No. 1122374. In addition I was supported as a
Research Assistant at the MIT Media Lab through the Media Lab Consortium.
10