Egocentric interaction for ambient intelligence

260 Pages·2012·8.08 MB·English

Department of Computing Science
Recognizable
Umeå 2012 Set
Egocentric Interaction
for Ambient Intelligence

Dipak Surie

PhD Thesis, January 2012

Department of Computing Science
Umeå University
SE-901 87 Umeå
Sweden
Department of Computing Science
Umeå University
SE-901 87 Umeå, Sweden

[email protected]

This work is protected by the Swedish Copyright Legislation (Act 1960:729)
Copyright © by the authors
Copyright © for figures and tables by the original publishers
Figures and tables are used for academic purpose

Excerpts of this thesis have been published in journals, book chapters,
conference and workshop articles as well as research deliverables, most
notably (Backman, 2006, Lindgren et al., 2011, Pederson et al., 2010,
Pederson et al., 2011, Pederson and Surie, 2007, Surie et al., 2010a, Surie et
al., 2007a, Surie et al., 2008, Surie and Pederson, 2007, Surie et al., 2010b,
Surie et al., 2007b, Surie et al., 2012)

ISBN: 978-91-7459-352-5
ISSN: 0348-0542
UMINF: 12.01

Electronic version available at http://umu.diva-portal.org/
Printed by: Print & Media, Umeå University
Umeå, Sweden 2012
3

Abstract
Ambient intelligence refers to the vision of computationally augmented
everyday environments that are sensitive, adaptive and responsive to humans
and intelligently support their daily lives. Ambient ecologies are the
infrastructures of ambient intelligence. To enable system developers to frame
and manage the dynamic and complex interaction of humans with ambient
ecologies consisting of a mixture of physical (real) and virtual (digital) objects,
novel interaction paradigms are needed.
Traditional interaction paradigms like the WIMP (windows, icon, menus,
and pointing devices) paradigm for desktop computing operate in a closed
world, unaware of the physical, social and cultural context. They restrict
human perception and action to screen, mouse and keyboard with the
assumption that human attention will be fully devoted to interaction with the
computer. Emerging interaction paradigms for ambient intelligence are
typically centered on specific devices, specific computing environments or
specific human capabilities. Also, many of them are driven by technological
advancements rather than viewing the human agent as their starting point. A
principled, theoretical approach centered in the individual human agent, their
situation and activities that are comprehensive and integrated while at the
same time instrumental in the design of ambient ecologies has been lacking.
This thesis introduces egocentric interaction as an approach towards the
modeling of ambient ecologies with the distinguishing feature of taking the
human agent‟s body, situation and activities as center of reference, as opposed
to the more common device-centric approaches in facilitating human-
environment interaction. Egocentric interaction is encapsulated in a number
of assumptions and principles such as situatedness, the proximity principle,
the physical-virtual equity principle, perception and action instead of “input”
and “output,” and activity-centeredness. A situative space model is proposed
based on some of these principles. It is intended to capture what a specific
human agent can perceive and not perceive, reach and not reach at any given
moment in time. The situative space model is for the egocentric interaction
paradigm what the virtual desktop is for the WIMP interaction paradigm:
more or less everything of interest to a specific human agent is assumed and
supposed to happen here.
In addition, the conception and implementation of the easy ADL ecology
based on egocentric interaction, comprising of smart objects, a personal
activity-centric middleware, ambient intelligence applications aimed at
everyday activity support, and a human agent literally in the middle of it all
is described. The middleware was developed to address important challenges
in ambient intelligence: (1) tracking and managing smart objects; (2) tracking
a human agent‟s situative spaces; (3) recognizing human activities and
actions; (4) managing and facilitating human-environment interaction; and
(5) to ease up the development of ambient intelligence applications.
The easy ADL ecology was first simulated in immersive virtual reality,
and then set up physically as a living laboratory to evaluate: (1) the
technological and technical performance of individual middleware
components, (2) to perform a user experience evaluation assessing various
aspects of user satisfaction in relation to the support offered by the easy ADL
ecology, and (3) to use it as a research test bed for addressing challenges in
ambient intelligence. While it is problematic to directly compare the “proof-of-
concept” easy ADL ecology with related research efforts, it is clear from the
user experience evaluation that the subjects were positive with the services it
offered.
5

Dedication

―To Thomas and Lars-Erik for painting novel ideas into a beautiful picture‖
Scenario
Before reading this thesis, the readers are
requested to go through the scenario(s) about
Gunnar Sjögren from a first person‟s perspective.
The scenario “Getting into Gunnar‟s shoes” is
intended for introducing one potential application
area where the work to be presented in this thesis
fits in. The scenario “Gunnar steps into the easy
ADL (Activities of Daily Living) ecology” is used as
a science fiction scenario that motivate and guide
further research in the field of Ambient Intelligence
(Aarts and Grotenhuis, 2009, Aarts and Ruyter,
2009, Aarts and Wichert, 2009, Aarts and
Encarnação, 2008). The scenarios are also useful for
introducing this thesis to a wide range of audience
with varying knowledge in the field
of ambient intelligence.

Getting into Gunnar’s
shoes
I like to browse through my photo
album with a cup of coffee when I
feel lonely. This photo was taken
during my working days as a
carpenter. I had a head injury
during a construction work in
Lycksele in 1992. The injury has
affected me psychologically leading
to small memory problems throughout my life. I am Gunnar Sjögren, a 64
year old pensioner living in Umeå, Sweden.

This is not my favorite picture, my stuga (cottage) in Ammarnäs half
burnt. I forgot to turn off the stove and went to take a sauna. When I came
back, all that I saw was my home burning like
the wooden logs in a sauna. This picture
reminds me that I need to turn off the stove
after cooking. Ting ting, the door bell is
ringing… the postman! I have got a new post
from Norrlands University hospital. I have got
an appointment on the 26th for my fortnightly
checkup. I will be out of town on this date, and
should probably call them tomorrow, if I
remember! I am currently under medication,
but I often forget to take my medicines on time.
That‟s when the whole world spins on my head.
7

The situation is never
better when I forget if I
had taken my medicines
or not. I panic and call
my care-giver. Linda is
nice, but I feel
embarrassed to call her
for such simple reasons
often.

Look here, this
photo was taken during
the millennium New Year celebrations. Yes, I look pretty cold. It was a
freezing evening and I forgot my gloves, cap and scarf at home. After the
celebrations, I rushed back to my home and began to fiddle around with my
apartment key using my frozen hands to unlock the main door, and you know
what, I ended up disturbing Linda‟s New Year party to open my apartment‟s
main door. I am not in full control inside my home either. I often forget where
my toilet is during night times and land up in the kitchen. Then, I usually try
calling Linda and since my
phone buttons are too
small, I end up calling
wrong numbers. I often
wonder why can‟t there be
a system that can guide
me to the toilet at nights. I
need more coffee before I
can continue. “Hej coffee
machine, can you prepare
more coffee?” Nej! I am
searching for kanelbullar (Swedish cinnamon roll) that I prepared with
Ingrid, a friend of mine last time around. Here it is, but looks old and so are
many other food items here. I should have thrown them away long time ago!

My phone is ringing, it
is Erik my son, wait a
minute. He is busy as
usual, wants me to prepare
the shopping list so that we
could go to the supermarket
this evening. Preparing a
shopping list is not the
easiest thing in the world. I
don‟t want to buy the whole
supermarket, but don‟t
want to miss something
important either. Damn, I
have spilled some coffee on
my shirt. Now, I remember,
there are many other clothes that are to be washed. But why have I not
washed them? Yeah, I was out of washing powder last time around, should
probably include it in the shopping list. I am getting confused, what else to
include in the shopping list? Milk, shower cream, tooth paste, veggies … Oj
då, the shower handle has fallen on the floor, and there is so much water in
the bathroom. That‟s why; the tap seems to be open. I am usually cautious in
these circumstances since I have a history of falling down in the bathroom.
Injuries, frustration, phone calls, hospital… I want to avoid it!

That‟s a gift from Ingrid. I have not opened it for a week now, just want
to keep it secretly for when I open it, my imagination might fade. We take
dinner together at times, and usually I want to be the chef with some surprise
menu. But, I find it hard to decide on the menu based on the ingredients that
I have at home, and these old recipe books. Mind you, I want to prepare a
good dinner, but then who can guide me through with some interesting
recipes. Ingrid checks my refrigerator and tells me what I could potentially
prepare, but that‟s not
interesting from my
perspective. Also, it
would be nice to have
perfect lighting and
music to go with some
good food.

I like to be social,
but then it gets really
difficult to move out of
my home to meet
friends. I used to
discuss politics for
hours with my friends
at the biblioteket
Walls that leave me isolated!
(library) cafeteria
when I was young and healthy. I can check if any of my friends are available
online. I like what
Pay my apartment bill properly! this computer does,
but then it is tricky
to use it at times.
Paying my monthly
bills using internet
is challenging.
Checking horse
racing updates,
playing online
memory games, and
reading books are
no picnic either. I
prefer to read coffee
table books or play
9

physical memory games instead of using a computer for entertainment, but
then computers bring into play unlimited games, books, movies… That‟s my
granddaughter Jenny in the desktop background. I tell her night time stories
using this computer. I have a webcam, but it more often doesn‟t work. Erik
has tried hard to teach me how to configure my webcam. I am not an expert in
computers though I can manage other household equipments like vacuum
cleaners, washing machines, coffee makers...

That‟s a chart that contains information about the activities that I should
perform daily. Linda keeps track of them when she visits me daily. But when
she is not there, I usually have a hard time completing some of those
activities. Doctors say I have mild-dementia, but I do not believe it, or at least
I don‟t want to believe it. I have been an independent man throughout my life
and would like to be so. I need to get going with other stuffs for the moment.

Gunnar steps into the easy ADL ecology
I like the coffee table, and of course this new coffee flavor from Gevalia. I
purchased it in…hmm…just forgot! My coffee cup says ICA Maxi
(supermarket), yeah it‟s true. I moved into the easy ADL ecology about a year
ago. I was part of Bostaden‟s (housing company) smart home campaign and
became lucky to get this apartment when the other applicants were reluctant
to live in an ambient ecology. The initial thought sounded geeky and even
scary, but after a year in the easy ADL ecology, I do not have many regrets.
This coffee table gives me all the news updates that I crave for, allows me to
play memory games online by placing and moving around stuff on it, acts as a
photo album that I can share with my friends online, and most of all, it allows
me to enjoy a nice fika (Swedish coffee break).

The informative art painting in front of me is a gift that I got from Erik
for my Birthday last month. You see Erik climbing the Sognefjord along with
Jenny in Norway last
summer. They are about to
reach the summit, which is
actually an indication that
my kanelbullar in the oven
is about to be ready. The
same painting facilitates
other services like bus
timings, weather
information, discuss politics
with friends, and activity
reminder information. It
feels like a well thought out
gift for me. Let me ask my
coffee machine to prepare
some coffee, love it with a
kanelbullar. Ursäkta (excuse
me), need to visit the toilet