Master thesis

159 Pages·2010·15.85 MB·English

Master thesis
"Analysis of the influence of
facial recognition applications
on privacy in Web 2.0 services"
Technische Universität Darmstadt
Fraunhofer-Institut SIT Darmstadt
Arne Siebo Oncken
First examiner: Prof. Dr. Claudia Eckert
Second examiner: Andreas Poller
30 July 2010
Contents
1 Abstract 1
2 Introduction 4
2.1 Description of the problem . . . . . . . . . . . . . . . . . . . . . . . 7
2.2 Objectives and motivations . . . . . . . . . . . . . . . . . . . . . . . 9
3 Fundamentals 11
3.1 Web 2.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.1.1 Social network sites . . . . . . . . . . . . . . . . . . . . . . . 17
3.1.2 Flickr . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.2 Fundamentals of facial recognition . . . . . . . . . . . . . . . . . . . 27
3.2.1 Face detection . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.2.2 Face recognition approaches . . . . . . . . . . . . . . . . . . 29
3.2.3 Facial recognition in everyday life . . . . . . . . . . . . . . . 30
4 Facial recognition applications 32
4.1 Model of a Web 2.0 application . . . . . . . . . . . . . . . . . . . . 32
4.2 Model of a Facebook account . . . . . . . . . . . . . . . . . . . . . 34
4.3 General model of a Facebook application . . . . . . . . . . . . . . . 36
4.4 General model of a facial recognition application . . . . . . . . . . . 38
4.5 Requirements for the use of a facial recognition application . . . . . 41
4.5.1 Facebook . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4.5.2 Flickr . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.6 Polar Rose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.6.1 The first use of Polar Rose - from a user’s point of view . . . 59
i
Contents ii
4.6.2 The first use of Polar Rose - from a technical point of view . 66
4.6.3 Polar Rose and Flickr . . . . . . . . . . . . . . . . . . . . . . 72
4.7 Face.com Phototagger . . . . . . . . . . . . . . . . . . . . . . . . . 82
4.7.1 The first use of the Face.com Phototagger - from a user’s point
of view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
4.7.2 The first use of the Face.com Phototagger - from a technical
point of view . . . . . . . . . . . . . . . . . . . . . . . . . . 86
4.8 Other facial recognition software . . . . . . . . . . . . . . . . . . . . 93
5 The Facebook API 97
5.1 General information about the Facebook API . . . . . . . . . . . . 98
5.2 The Revealer application . . . . . . . . . . . . . . . . . . . . . . . . 106
5.2.1 An exemplary use of the Revealer application . . . . . . . . 108
6 Conclusions 114
6.1 Data privacy analysis summary . . . . . . . . . . . . . . . . . . . . 114
6.1.1 Effectiveness of privacy settings . . . . . . . . . . . . . . . . 115
6.1.2 Effectiveness of changes in privacy settings . . . . . . . . . . 120
6.1.3 Vulnerabilities of the access control of the Facebook API . . 124
6.1.4 Attack scenarios . . . . . . . . . . . . . . . . . . . . . . . . . 125
6.1.5 Summarized evaluation of Polar Rose . . . . . . . . . . . . . 126
6.1.6 Summarized evaluation of Face.com Phototagger . . . . . . . 127
6.2 Ideas for improvements . . . . . . . . . . . . . . . . . . . . . . . . . 127
6.2.1 Demands on the Facebook privacy settings . . . . . . . . . . 130
6.2.2 The Flickr API . . . . . . . . . . . . . . . . . . . . . . . . . 133
6.2.3 Demands on the examined facial recognition applications . . 134
6.2.4 Advice to users . . . . . . . . . . . . . . . . . . . . . . . . . 134
6.2.5 Judicial comment . . . . . . . . . . . . . . . . . . . . . . . . 134
6.2.6 Future prospects . . . . . . . . . . . . . . . . . . . . . . . . 135
7 Addendum: Content of the enclosed CD 138
8 Statement of authorship 139
Contents iii
Bibliography 140
List of Figures 146
1 Abstract
The present work analyzes the handling of photos, which were entrusted to social
network sites, by the social network sites itself and by third party applications
that access such photos through for these purposes established APIs. The goal
is to evaluate how the privacy of users and non-users appearing on photographs
is respected and to develop suggestions for a better privacy protection in social
networks and in facial recognition applications.
Facial recognition applications offer their users the possibility to manage their
photos, view their photos and to add information to their photos. Users can tag
their photos with the names of the photographed persons. After the user manually
added the name of a person, the facial recognition application tries to recognize
the person’s face on other available pictures. With each new name tag, the facial
recognition performance of the applications is enhanced. Name tags do not have to
be bound to a user account, also people who are not member of a social network
can be tagged.
Web 2.0 services like Facebook provide APIs for third party developers like the
developers of facial recognition applications that allow them to access user data if
corresponding authorizations are granted. These APIs can be used for Facebook
intern applications, which can be added by Facebook users and they can be used for
Facebook extern applications, for example as a single-sign on access control.
The market dominating social network site Facebook offers an API that can be
accessed by any developer. Before users grant a permission to applications of third
1
1 Abstract 2
party developers, Facebook neither informs about the full extent of the granted
authorization, nor does Facebook control if a developer exploits the obtained user
data.
After the reader is introduced into the eponymous subject of the present work,
some fundamentals are explained, including the term Web 2.0 and basic facial
recognition techniques. Starting with the abstract model of a Web 2.0 application
and of a Facebook account, first a general model of a Facebook application and
later of a facial recognition application - of the observed type - are developed.
The usage of a facial recognition application is outlined and in detail inspected
for the two closer examined applications Polar Rose and Face.com Phototagger.
The Facebook API is explained and a sample application, which was developed
exclusively for this work, is presented. It accesses the Facebook API and visual-
izestheopportunitiesofFacebookapplicationstoobtainphotosfromFacebookusers.
The most important results of the work are as follows:
• Facebook empowers third party developers to build databases with biometrical
reference data of their users and their users’ contacts - amongst others via the
default privacy settings and the offline access authorization.
• Polar Rose attempts to display photos to its users with the publisher’s consent,
but stores the user photos permanently. In consideration of other projects
that Polar Rose is and was involved in, data privacy conscious users cannot
be advised to entrust Polar Rose with their photos.
• The Face.com Phototagger displays all accessible photos to its users, without
an observable concern about the consent of the photographed persons. Since
the Phototagger application imports images for the recognizing process and
sinceitisabletostorethephotos, thisapplicationalsocannotberecommended
to data privacy conscious users.
• The analysis of the facial recognition applications had the result that the ap-
plications retrieve photos and make them available for themselves independent
1 Abstract 3
of the Facebook API. Their proceedings bypass the Facebook access control.
• Facebookusersshouldutilizethehardtofindprivacysettingsoftheinformation
that is accessible through friends and protect their data from the developers
of applications, games and Facebook extern websites.
• People that are not member in any social networks are not protected from the
affiliation in a biometrical database.
• If Facebook aims to justify the trust of their users, changes to the privacy
concept should be made. The most important claim is that most user data
should be private by default, until the user chooses to share it with unknown
Facebook members or applications (opt-in).
2 Introduction
A growing number of people is spending their freetime with Web 2.0 applications.
Many people are disclosing a lot of details from their private life without reasoning
about any possible consequences for themselves and for others. Careless behavior
makes it easy for recruiters, advertisers or others to develop personality profiles.
On their website, Facebook.com claims to have 400 million active users world-
wide, whereof 200 million users log in to their Facebook account every day. Shortly
before the closing date of this work, Facebook announced in its blog that the 500
millionth user has signed up to Facebook on 21 July 2010 [Faca]. The total time
that users spend on Facebook.com is, according to Facebook, 500 billion minutes
every month. The average user has classified 130 other users as friends [Fact].
Facebook.com is visited by people from all parts of the world, 70% of the users is liv-
ingoutsidetheUnitedStates. 70differentlanguagesareavailableforFacebook[Fact].
Facebook claims that their users share 25 billion pieces of content, like photo
albums or web links. They can interact with 160 million objects, as pages, groups
and events. The average user is connected to 60 pages, groups and events and creates
70 pieces of content every month [Fact].
This content can be evaluated, shared or merely displayed by applications of third
party developers that access the Facebook API. Over a million developers from
more than 180 countries worldwide created 550,000 active applications on Facebook
Platform and one million websites integrated Facebook into their external websites
4
2 Introduction 5
with Facebook Connect. Half of the websites ranked in comScore’s Global Top 1001
has integrated Facebook.
Trading with user and customer data is a growing market. It is interesting for
advertisers to know more about possible customers and to address their advertise-
ments merely to people from the product’s target group - so-called direct marketing.
Loyalty programs in stores or frequent flyer programs from airlines promise their
members privileges and are interested in consumer retention, but also in the pri-
vate data of their members, which is helping companies to create user profiles. In
Germany, the volume of sales that is captured with cards of the loyalty program
Payback is 14.5 billion e, causing a demand of 180 employees [PAY]. The holding
company Loyalty Partner had revenues of 209 million e in 2009 [Loy].
Google AdWords claims that their service helps advertisers to spent their bud-
get more target-oriented, because their advertisements will be only taken out to
users that enter specific search words in Google [Gooa]. Google AdSense is going
a step further and analyzes the content of a website to place matching ads [Good].
Google’s email service Gmail evaluates their user’s email for advertisement purposes.
The Privacy Policy states [Goob]
"Google’s computers process the information in your messages for various
purposes, including [...] delivering advertisements and related links [...]
and other purposes relating to offering you Gmail."
The ranking "Top 100 most valuable global brands 2010", which is published by
Millward Brown Optimor, ranks Google as the most valuable brand worldwide
[Mil]. This reputation that is also substantiated by Google’s financial results [Gooc],
constitutes a lot on the value that Google’s users provide with their usage data.
This excursion is supposed to demonstrate the high value of user data. User photos
1 ComScore is a marketing research company and provides marketing data and services. More
information on comScore can be found on their website http://www.comscore.com/About_
comScore.
2 Introduction 6
that compose a biometrical database, are also of great value.
Web 2.0 services give advertisers great opportunities for direct marketing. Since the
content in Web 2.0 services (which are defined in the subchapter 3.1) is generated
mainly by the services’ users, advertisers and other interest groups can discover a
lot of information about the users, their interests, personal information and photos.
This information is of great value and therefore a popular trading good.
The content of social network sites like Facebook essentially consists of information
about their users (social network sites are defined in subchapter 3.1.1). Third party
applications, that access Facebook’s API for Developers, can, if a user granted the
specific permission, obtain the information and photos of this users - and also of the
users in his contact list.
The Web 2.0 application Flickr is a service that hosts images. Its users entrust
Flickr to 4 billion photos [Yaha], which are not only of great financial value, but
give Flickr also the great responsibility to provide the privacy to the photos that
the users wish. Also Flickr offers an API that empowers to access Flickr profiles if
the users agree to.
Both mentioned APIs are accessed by applications that use facial recognition tech-
niques to identify persons in the inspected photos. After obtaining a user’s photos,
these applications locate the faces on the photos and ask their users to name the
photographed persons. When the application is trained with some photos, it recog-
nizes persons in newly added photos itself (more on facial recognition can be found
in the last subchapters of chapter 3, facial recognition applications are described in
chapter 4).