simulating gaze behavior of virtual crowds by predicting interest points

66 Pages·2016·27.1 MB·Turkish

SIMULATING GAZE BEHAVIOR OF
VIRTUAL CROWDS BY PREDICTING
INTEREST POINTS
a thesis submitted to
the graduate school of engineering and science
of bilkent university
in partial fulfillment of the requirements for
the degree of
master of science
in
computer engineering
By
Umut A˘gıl
July 2016
SIMULATING GAZE BEHAVIOR OF VIRTUAL CROWDS BY
PREDICTING INTEREST POINTS
By Umut A˘gıl
July 2016
Wecertifythatwehavereadthisthesisandthatinouropinionitisfullyadequate,
in scope and in quality, as a thesis for the degree of Master of Science.
Ug˘ur Gu¨du¨kbay (Advisor)
¨
Ozgu¨r Ulusoy
˙
Veysi I¸sler
Approved for the Graduate School of Engineering and Science:
Levent Onural
Director of the Graduate School
ii
ABSTRACT
SIMULATING GAZE BEHAVIOR OF VIRTUAL
CROWDS BY PREDICTING INTEREST POINTS
Umut A˘gıl
M.S. in Computer Engineering
Advisor: U˘gur Gu¨du¨kbay
July 2016
Creating realistic crowd behavior is one of the major goals in crowd simula-
tions. Simulating gaze behavior and predicting interest points of virtual charac-
ters play a significant role in creating believable scenes, however this aspect has
not received much attention in the field. This study proposes a saliency model,
which enables virtual agents to produce gaze behavior. The model measures the
effects of distinct pre-defined saliency features that are implemented by examin-
ing the state-of-the-art perception studies. When predicting an agent’s interest
point, we compute the saliency scores by using a weighted sum function for other
agents and environment objects in the field of view of the agent for each frame.
Then we determine the most salient entity in the virtual scene according to the
viewer agent by comparing the scores. We execute this process for each agent in
the scene, thus agents gain a visual understanding about their environment. Be-
sides, our model introduces new aspects to crowd perception, such as perceiving
characters as groups of people, gaze copy phenomena and effects of agent velocity
on attention. For evaluation, we compare the resulting saliency gaze model with
real world crowd behavior in captured videos. In the experiments, we simulate
the gaze behavior in real crowds. The results show that the proposed approach
generates plausible gaze behaviors and is easily adaptable to varying scenarios
for virtual crowds.
Keywords: crowd simulation, saliency, gaze behavior, perception, interest point
detection, gaze copy.
iii
¨
OZET
˙ ˙ ˙
ILGI NOKTALARINI TAHMIN EDEREK SANAL
˙ ˙
KALABALIKLAR IC¸IN BAKIS¸ DAVRANIS¸I
˙ ¨
SIMULASYONU
Umut A˘gıl
Bilgisayar Mu¨hendisli˘gi, Yu¨ksek Lisans
Tez Danı¸smanı: Ug˘ur Gu¨du¨kbay
Temmuz 2016
Ger¸cekc¸i kalabalık insan grubu davranı¸sı yaratmak kalabalık simu¨lasyonlarının
en temel amac¸larından birisi olarak yer alır. Sanal karakterlerin bakı¸s
davranı¸sını simu¨le etmek ve ilgi ¸cekici noktaları tahmin etmek inandırıcı sahneler
olu¸sturmada ¨onemli rol oynamaktadır. Fakat bu yakla¸sım bu zamana kadar bil-
gisayar grafig˘i alanında yeteri kadar ilgi ¸cekmemi¸stir. Bu ¸calı¸sma sanal karakter-
lerin bakı¸s davranı¸sı u¨retmesini sag˘layan bir dikkat ¸cekicilik modeli sunmaktadır.
Model algı u¨zerine yapılmı¸s gu¨ncel ¸calı¸smaları ele alarak dikkat ¸cekici unsurlar
belirler ve bu unsuların etkilerini o¨l¸cer. Bir karakterin ilgisini c¸eken noktayı tah-
min ederken sahnede karakterin go¨ru¨¸s alanınında bulunan di˘ger karakterler ve
nesnelerin dikkat ¸cekme skorları bir ag˘ırlıklı toplam fonksiyonu ile hesaplanır.
Ardından bu skorlar kar¸sıla¸stırılarak izleyici karaktere g¨ore sahnedeki en dikkat
¸cekici varlık bulunur. Bu i¸slem sahnedeki her bir karakter i¸cin hesaplanır ve
bo¨ylece tu¨m karakterler ¸cevreleri hakkında g¨orsel bir algıya sahip olurlar. Bunun
¨
yanında, modelimiz kalabalık algısına yeni bakı¸s ac¸ıları da getirmektedir. Ornek
olarak, kalabalıkların gruplar halinde ele alınması, bakı¸s kopyalama, hızın dikkate
¨
etkisi verilebilir. Onerilen modelin deg˘erlendirmesi i¸cin elde etti˘gimiz dikkat
¸cekme modelini videolardan elde ettig˘imiz ger¸cek du¨nya kalabalık davranı¸s ve-
rileri ile kar¸sıla¸stırdık. Hazırlanan senaryolarda, ger¸cek kalabalık davranı¸slarının
simu¨lasyonunu yaptık. Elde edilen sonu¸clar modelin ger¸cek¸ci bakı¸s davranı¸sı
sunan ve farklı senaryolara kolaylıkla uygulanabilir oldug˘unu go¨stermektedir.
Anahtar s¨ozcu¨kler: kalabalık simu¨lasyonu, dikkat c¸ekme, bakı¸s davranı¸sı, algı,
ilgi ¸cekici noktaların tespit edilmesi, bakı¸s kopyalama.
iv
Acknowledgement
First and foremost, I would like to express my gratitude to my supervisor
Prof. Dr. U˘gur Gu¨du¨kbay. Without his guidance and assistance, it would not be
possible to complete this thesis.
¨
I also would like to thank to rest of my thesis committee, Prof. Dr. Ozgu¨r
˙
Ulusoy and Prof. Dr. Veysi I¸sler for evaluating this thesis.
Besides, I am grateful to Dr. Erdal Yılmaz for his advice with the development
of the framework and the evaluation of the model.
Thanks to Tarık Yal¸cın and Sec¸kin Yal¸cın for their assistance on creating
virtual scenes and preparing videos.
I would like to thank to Onur Polat for his companionship during this process
and for his help in capturing videos.
˙
I am grateful to my friends Volkan, Faruk, Emre, Kubilay, Gu¨lfem, Istemi,
Olcay, Serkan, Yusuf and all the people who have shared their valuable time.
Special thanks to the members of ”Motor Oil Co.”, who have always kept my
spirit high, for all the goy they have given.
Finally, I must express my sincere gratitude to my family, who have supported
me throughout my years of study and through the entire thesis process. This
accomplishment would not have been possible without their unconditional love.
v
Contents
1 Introduction 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Research Questions . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.3 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2 Background and Related Works 6
2.1 Saliency in Computer Graphics . . . . . . . . . . . . . . . . . . . 6
2.2 Crowd Simulations and Crowd Models . . . . . . . . . . . . . . . 7
2.3 Attention Models and Applications on Crowds . . . . . . . . . . . 9
2.4 Human Visual System and Gaze Behavior . . . . . . . . . . . . . 11
2.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3 Simulating Interest Point Detection and Gaze Behavior 13
3.1 Interest Point Detection . . . . . . . . . . . . . . . . . . . . . . . 14
3.1.1 Attributes of Characters and Objects . . . . . . . . . . . . 15
vi
CONTENTS vii
3.1.2 Saliency Score Computation Parameters . . . . . . . . . . 16
3.1.3 Saliency Scoring . . . . . . . . . . . . . . . . . . . . . . . . 17
3.2 Adjusting Gaze . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.2.1 Gaze Copy . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.2.2 Gaze Shifting Animation . . . . . . . . . . . . . . . . . . . 28
3.2.3 Gaze Duration . . . . . . . . . . . . . . . . . . . . . . . . 28
4 Implementation of the Framework 30
4.1 Graphics Models . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
4.2 Animation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.3 Architecture of the Simulation . . . . . . . . . . . . . . . . . . . . 32
5 Evaluation and Results 35
5.1 Scenario Simulations . . . . . . . . . . . . . . . . . . . . . . . . . 35
5.1.1 Scenario 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
5.1.2 Scenario 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
5.1.3 Scenario 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
5.1.4 Scenario 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
5.1.5 Scenario 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
5.1.6 Scenario 6 . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
CONTENTS viii
5.1.7 Scenario 7 . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
5.1.8 Scenarios 8 . . . . . . . . . . . . . . . . . . . . . . . . . . 42
5.1.9 Scenarios 9 . . . . . . . . . . . . . . . . . . . . . . . . . . 43
5.2 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
6 Conclusion 48
Bibliography 50
List of Figures
1.1 Gaze behavior in popular computer games . . . . . . . . . . . . . 3
4.1 The relationships between agent components. . . . . . . . . . . . 32
4.2 Gaze decision logic. . . . . . . . . . . . . . . . . . . . . . . . . . . 33
5.1 Effects of proximity and periphery: (a) video, (b) simulation, and
(c) simulation (weight=0). . . . . . . . . . . . . . . . . . . . . . . 36
5.2 Effects of agent velocity: (a) video, (b) simulation, and (c) simu-
lation (weight=0). . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
5.3 Effects of agent attractiveness: (a) video, (b) simulation, and
(c) simulation (agent’s attractiveness=0.4). . . . . . . . . . . . . . 38
5.4 Effects of distinctiveness, curiosity and height: (a) video, (b) simu-
lation, (c) distinctiveness = 0, (d) curiosity=0, and (e) child height
= 1.65. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
5.5 Still frames showing the effect of agent shyness: (a) video frames,
(b) shyness = 0.25, (c) shyness = 0.7. . . . . . . . . . . . . . . . . 40
5.6 Effects of object distinctiveness: (a) video, (b) simulation, and
(c) simulation (weight=0). . . . . . . . . . . . . . . . . . . . . . . 41
ix
LIST OF FIGURES x
5.7 Still frames showing the effects of gaze copy: (a) video frames and
(b) simulation frames. . . . . . . . . . . . . . . . . . . . . . . . . 42
5.8 Effects of periphery: first row: the real video, second row: the
simulation result. . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
5.9 Still frames from a 40 second video: left column: the real video,
right column: the simulation result. . . . . . . . . . . . . . . . . . 44
5.10 Graphs depicting (a) maximum frame computation times (msecs)
and (b) frame rates (frames per second) for different number of
agents. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
5.11 A crowd simulation with 100 agents. Gaze behavior scripts are
(a) disabled and (b) enabled. . . . . . . . . . . . . . . . . . . . . . 47