ERIC EJ1064273: Exploring the Usage of a Video Application Tool: Experiences in Film Studies

2011·1.6 MB·English

by  ERIC

InformaticsinEducation,2011,Vol.10,No.2,163–181 163
©2011VilniusUniversity
Exploring the Usage of a Video Application Tool:
Experiences in Film Studies
NazlenaMohamad ALI1,AlanF.SMEATON2
1InstituteofVisualInformatics,UniversityKebangsaanMalaysia
43600Bangi,Selangor,Malaysia
2CLARITY:CentreforSensorWebTechnologies,DublinCityUniversity
Glasnevin,Dublin9,Ireland
e-mail:[email protected],[email protected]
Received:May2011
Abstract.Thispaperexploresourexperiencesindeployingavideoapplicationtoolinfilmstudies,
anditsevaluationintermsofrealisticcontextualend-userswhohaverealtaskstoperforminareal
environment.Wedemonstrateourexperiencesandcorelessonlearntindeployingournovelmovie
browserapplicationwithundergraduateandgraduatestudentscompletingaFilmStudiescoursein
DublinCityUniversityoverasemester.WedevelopedasystemcalledMOVIEBROWSER2that
hastwotypesofbrowsingmodes:AdvancedandBasic.Ingeneral,studentsfoundthatthefeatures
we provided were beneficial for their studies. Some issues or mismatches arose during the trial.
A‘wish-list’wasdrawnupthatmightbeusefulforthefuturesystemdeveloper.Thecontribution
andachievementsreportedinthisarticleareonthedemonstrationandexplorationofhowadvances
intechnologycanbedeployed,andmediacanbeaccessedinthecontextofarealusercommunity.
Exploringtheusageindicatesapositiveacceptanceamongstudents,besideslessonslearnedthat
areimportantforfurtherinvestigation.
Keywords:usagedataanalysis,digitalvideoapplication,filmstudies.
1. Introduction
Asaresultofdevelopmentsinmultimediatechnologies,theprospectofwideandubiq-
uitous use of new media is promising in various domains; in particular, in education
including the use of software, hardware, Internet applications and other related activi-
tieswhichcanfacilitateandimprovelearning.TheuseofvideoinFilmStudiesteaching
and learning, for example, can bring overall usefulness and benefit, as video can be an
importantresourceforstudents(MohamadAliandSmeaton,2009).Filmandmediaare
more accessible and useful for a variety of purposes with the advances of information
technology,andhaveplayedapartintheevolutionofscholarlyresearchandeducational
goals. Innovative educational technologies are possible to develop as a showcase with
numerouspotentialbenefits.Researchtowardsintegrationofbothtechnicalmultimedia
researchanduserorientationplaysasignificantroleintoday’slearning.Anumberofac-
tivitiessuchasforumsandworkshopsonhowtodelivermultimediaresearchtoenduser
164 N.M.Ali,A.F.Smeaton
experiencearestartingtohappen(i.e.,QoMEX,2009),showingagrowingawarenessof
theimportanceofrelatingMultimediaresearchtothe‘realworld’.
Much work has been performed regarding the technical possibilities for multimedia
content analysis, particularly video. These works usually process a video element with
various automatic outcomes and analyses, such as shot boundary detection, image seg-
mentation, and automatic detection of semantic concepts. (Mas and Fernandez, 2003;
Smeaton,2004;Smeatonetal.,1999).Asitsgoal,muchoftheworkhasafocusonthe
evaluation and measurement of precision and recall for video search tasks, of the effi-
ciencyandaccuracyofthetechniquesdeveloped.Userinvolvement,ifany,willnormally
beinvolvedonlyatthetestingstage.Suchevaluationsareperformedinpre-definedlab
settings or sessions. Various applications of the new technologies can be developed as
ashowcaseinordertocarryouttheevaluation.However,notmuchworkhasbeencar-
ried out beyond this boundary where new technology in video analysis techniques are
deployed and used by real end-users in real settings and with real tasks involved. User
interactionswithnewsoftwareapplicationsthatincorporatethesekindsoftechnologies
are among the contributions demonstrated in this paper. Such true user interactions are
almostnevercapturedorconsidered.Theobjectiveofthispaperistodiscussouranalysis
whenexploringtheusageofavideoapplicationtoolasanadvancedtechnologyforreal
usersinaFilmStudiescontext.
Thepaperisorganizedasfollows.Inthefollowingsection,somebackgroundworkin
videocontentanalysisandexamplesofworkingapplicationisintroduced.Thisincludes
some works in deployment that combine technology into practical usage. In Section 3
and 4, we elaborate on our developed system MOVIEBROWSER2 and explain the de-
ploymenttrialtoFilmStudiesstudent.Itisfollowedbyourfindingsanddiscussionsthat
alsocoversomelessonslearned,basedonourdeploymentexperiencesandanalysis.We
endwithconclusionsandlimitationofthework.
2. BackgroundWork
Anumberofoperationalvideoapplicationshowcaseshavebeendevelopedapplyingthe
underlying techniques of video content analysis; particularly related to movies such as
MovieBrowser (Lehane et al., 2007), Videana (Ewerth et al., 2005), and a project at
theINRIA(Ronfard,2004).Theseworksareamongthemanyworksthatarenoveland
consider the technical perspective of the analysis (i.e., measuring Precision/Recall and
accuracy).Fromanotherperspective,theylackassessmentfromtheaspectofrealusage
monitoringandevaluation.Notmuchworkhasbeenperformedwhichgoesbeyondthe
boundary of end-user orientation,where the usage of these kinds of tools or showcases
areactuallymonitoredandassessed.
In addition to pure technology-driven evaluations, there is also work reported that
combinestechnologicaleffortsintoadeploymentstagewhereitissubsequentlyusedby
users like Físchlár (Lee et al., 2000), Newsblaster (McKeown et al., 2002), iTV (Bern-
hauptetal.,2007)andSportsAnno(LanaganandSmeaton,2007).TheFíschlárDigital
ExploringtheUsageofaVideoApplicationTool:ExperiencesinFilmStudies 165
VideoLibrarySystemwasdevelopedatDublinCityUniversitytosupportcapture,index-
ing,browsing,searching,andsummarizingofdigitalvideo,andhasbeendeployedinto
four separate video content collections for a variety of users and application scenarios
(Smeatonetal.,2004a).ThefourversionsoftheFíschlársystemincludeTVprograms
(Smeaton et al., 2004b), TV news (Smeaton et al., 2004a), TRECVid video track par-
ticipation (Browne et al., 2003), and nursing educational videos (Gurrin et al., 2004).
Informationprovidedtousersinthesysteminterfacearebasedonfindingandselectinga
videoprogrameitherusingtextormetadata.Supportedinterfaceelementsincludedakey
frameslideshow,ahierarchicalkeyframebrowser,andatimelinebrowser.Físchlár-News
wasoneofthecollectionsdesignedtosupportanarchivetothemaineveningTVnews
broadcast.Itincorporatesanumberofmultimediaandrecommendationtechniquesand
wasdeployedwithinaUniversitycampusforseveralyears,inwhichlargescaletesting
andevaluation(performanceandusability)hasbeencarriedout(Smeatonetal.,2004a).
Methods used from video content analysis include shot boundary detection, key frame
extraction,captureofclosedcaptions,andthesystemallowsfortextsearching,browsing
andplayback.AnextendedliveusagestudyhasbeenperformedonFíschlár-Newswith
16usersusingahighlyqualitativeandethnographicdiary(Leeetal.,2006).Thisstudy
mainlyemphasizedtheunderstandingofrealuse,thedevelopmentofuseovertime,and
theuseofnewtechnologies.
NewsblasteratColumbiaUniversity(McKeownetal.,2002)isanexperimentalsys-
tem incorporating natural language processing techniques to automatically crawl news
websitesonadailybasis,andsummarizeandpresentthemtowebusers.Thesystemhas
been deployed since 2001 and was developed mainly to demonstrate its summarization
robustness and the use of TDT (topic detection and tracking) technology. Researchers
on the Newsblaster system had started conducting a large online evaluation to measure
the usage and preferences. The system has also undergone a number of experiments,
mainly on summarization efficiency (Precision and Recall), as reported in (McKeown
etal.,2001).
AnAustrianinteractiveTV(iTV)trial(Bernhauptetal.,2007)deployedanovelTV
application to a local cable TV provider in Salzburg, Austria, and ran for 4 months in
2004–5.Inthispaper,theauthorstriedtoovercomethelackofresearchintheareathat
canprovideactualusagedata;theresearcherscarriedoutevaluationandgeneratedresults
fromausabilitytestandthefindingsontheuseoftheiTVservices.Inordertoassessac-
tualusagedata,thesystemwasdeployedintomorethan300households.Apartfromthat,
thedesignofthesystemalsofollowedauser-centereddesignapproachinordertocollect
moreusabledata.Theresearchersgotlowfeedbackfromthetrialduetosometechnical
constraints. The system received some suggestions for improvement from the usability
test findings; these are tackled in subsequent projects. Data collections from the server
logs,questionnaires,andinterviewswereusedtoillustratethefindings.Useracceptance
ofthenewtechnologyisconsideredlowasthisareawasstillimmature.Thedevelopers
followedauser-centereddesigninordertoprovideausablesystemfortheuser,butthen,
due to time constrains, the improvements could not be carried out for the deployment
test. The research proceeded with the improved version and was further re-deployed in
166 N.M.Ali,A.F.Smeaton
theirfuturework.Whentryingtoimprovethesystemtheystucktoauser-centereddesign
approach,wherebytheytookintoaccountusercharacteristics,experiences,expectations,
andpreferences.Theusertestshowedthatuserspreferredasimplenavigationaldesign.
The research outputs also noted several issues from the trial, such as technical, usabil-
ity, and user acceptance issues. For instance, during the field trial, users reported some
technicalproblemswhichprovedtobeoneofthereasonsforlowusage.Theyidentified
from the deployment conclusion that usability should be taken seriously to ensure user
acceptance.
SportsAnno is a video browsing system designed and developed to allow users to
make comments and share opinions and ideas on soccer events with other registered
users(LanaganandSmeaton,2007).SportsAnnowasdeployedduringthesoccerWorld
Cup 2006. Users can browse soccer video, and at the same time, make comments and
annotate while reading texts of newspaper reports related to the matches. Usage data
on sports event segmentation was gathered during the deployment stage. A number of
usage monitoring issues were pointed out from the deployment, such as low responses
forsomespecificgamescomments.Amongthesolutionsproposedaresuggestionsthat
thesystemshouldhaveanalertnotification,whichwouldinformorsignalthatthereare
newcomments.Someotherlessonslearnedwere:toincludedirectlinkingandannotation
that allow easier navigation; and locating video events within the report. By viewing
commentsofothers,userswillcatchadditionalinformation,notavailableintheoriginal
newspaperreport.
Examplesgivenabovearesomeoftheworkindeploymenteffortsthatcombinetech-
nology into practical system development. These trial efforts also indicate that there is
growing awareness of the importance of user evaluation in realistic environments. This
researchdirectionwillbringanewperspectiveintovarietyofresearchagendainvisual
informaticsthatextendsinteractionandcommunicationbasedonvisualinterfaces;thus,
couldsupportusersinunderstandingmediacontent.
3. TheMOVIEBROWSER2Application
We developed the MOVIEBROWSER2 to incorporate the use of a number of content
analysistechniques,particularlythosethatidentifymoviesceneboundariesandcatego-
rizethemintoexciting,montage,anddialoguescenes.MOVIEBROWSER2usesseveral
recentmultimediatechnologiestoautomaticallyprocessdigitalvideocontent,butatthe
sametime,weusedausabilityengineeringprocesstorelatethesetechniquestothereal
tasksofrealusersintheirrealenvironments.OurMOVIEBROWSER2interfaceisshown
inFig.1(a)and(b).Thedesignedinterfaceincorporatessomeadvancedfeaturessuchas
atimelinebarnearthetopofthescreen(Fig.1(b)).Thetimelinebarwillappearinthree
differentcolors,eachrepresentingdifferenttypesofscenes.Forexample,thegreenband
representsthosesceneswithDialogues;theyellowbandrepresentsExcitingscenes;the
pinkbandrepresentsthosesceneswithMontage.Theseinteractivetimelinefeatureswere
developedtoprovideassistanceinbreakingdownconventionalpatternsofviewingand
ExploringtheUsageofaVideoApplicationTool:ExperiencesinFilmStudies 167
Fig.1.MOVIEBROWSER2screenshot(a)mainpage,(b)mainbrowsingmovieclips.
168 N.M.Ali,A.F.Smeaton
helpingtheusertoappreciatethegrammarandstructuresoffilmmaking,andmighthelp
to break down the narrative and style of the film into its constituentparts. Enablingthe
film to be seen and navigated with a timeline bar outlining exactly where the sequence
is,inrelationtothewholefilm,mightbeusefulwhenstudentsaretryingtounderstand
narrativeconstructionandthedifferentfunctionsofanygivensceneinafilm.Suchinitial
impressions are necessary to later help develop a more reflective as well as an intuitive
engagement with the movie content (Brereton, P., personal communication, August 5,
2009).
MOVIEBROWSER2 provides the main features that, we believe, could support the
process of reading a movie for Film Studies students. The function can be divided into
fourmajorcategories:
(1) Selectingamovie–listofmovieposterswithsomeotherinformation(i.e.,titles,
genre,year)ontheleftsideofthescreenasshowninFig.1(a).
(2) Browsingwithin-movie–theusercanbrowseusingthetimelinebarontheupper
sideofthescreenandusingakeyframeviewasshowninFig.1(b).Eachtimeline
representsasegmentedsceneofdialogue,exciting,ormontagethatisidentifiedin
eachmovie.
(3) Playing a movie – standard buttons as can be seen in the normal media player
wereprovidedinMOVIEBROWSER2;namely,play,pause,stop,fullscreen,vol-
umeadjustmentsandmutefortheusertoplaymovieclips.Theplaybackareain
MOVIEBROWSER2canbeseeninFig.1(b)attherightsideoftheinterface.
(4) Socialinteractions–weprovideafeaturesothat,atanypointduringtheplayback,
theusercanmakenotesbyclickingonthe‘MakeSomeNotes/Comments’button
underneaththeplaybackarea,andthentypeanoteonaninterestingsceneorthe
wholemovie;which,perhaps,canbesavedandviewedlaterandevensharedwith
otherstudents(Fig.1(b)).
4. DeploymentTrialMethodology
We deployed our MOVIEBROWSER2 system in groups of Film Studies students in
SchoolofCommunications,DublinCityUniversity.Studentsusedtheconventionalap-
proach,wherebytheygotDVDsfromtheUniversitylibraryorrentalshopandusedthese
asthebasisforreadingsofmoviesaspartofcourseassignments.Themainrestrictions
contributing to the problems in their studies were, getting short-term loans, and a lack
of DVD resources (i.e., old production year, obsolete format). In preparing to write a
textualanalysisfortheircourse,studentsofFilmStudiesaregivenalistoftopicstobe
chosen from prior to that. Once decided on the particular topic, students start looking
for related DVD movies. The length of the textual analysis varies across modules from
1,000to3,000words,whichdependsonthelevelofthemodule.Thereisnouniquefor-
mulatodecodeafilm.Atextualanalysismustengagewiththegrammarofthecinema
and the most important thing is to concentrate on how the meaning is created (Brere-
ton,2008a;Brereton,2008b).ThetypicaltasksofFilmStudiesstudentsaretoreadand
ExploringtheUsageofaVideoApplicationTool:ExperiencesinFilmStudies 169
analyze movie sequences. Reading a movie in their context refers to the process of un-
derstandingandanalyzingmoviecontentclosely,lookingfordifferentlevelsofmeaning
andcritiqueforexamplefromelementslikeframing,depthoffield,plot,shots,camera
angle, lighting and so on. On a broader level it also involves an understanding of the
genericconventionsandnarrativestructureofindividualmovies(Brereton,2008a).Our
currentdesignofbrowsingmoviecontentcouldalsoprovideapositiveimpactandwill
aid in film analysis and could provide an indirect process of reading a film. In relation
totheMOVIEBROWSER2systemwedeveloped,webelievethatthisnewstrategyand
‘tool’ is also useful to help students to acquire the skills of reading film and there are
some elements or interface features that could maximize the potential of their learning.
For example, enabling the film to be seen and navigated with a timeline bar, outlining
exactlywherethesequenceis,inrelationtothewholefilm,forexample,mightbevery
useful when students are trying to grapple with narrative construction and the different
functionsofanygivensceneinafilm.
4.1. Participants
OurgroupofrealusersatthedeploymentstagewerestudentsfromthemodulesCM272
NationalandIrelandCinema,whichisasecondyearundergraduatelevelcourseofone
semester (12 weeks classes), and CM135 Analyzing Media Content, a first year under-
graduatelevelmodule,alsoofonesemesterduration.Thecombinedclassgroupstotaled
268students(CM272=76students,CM135=192students).Fortheformermodule,the
lecturerusedmostlyanIrishcinemamakerastheirmainfocus,whilethelattermodule
used contemporary Hollywood movies. Students taking the modules were from differ-
entdegreeprogrammesincludingtheBAincommunications,theBScinmultimediaand
theBAinjournalism.Inbothmodules,thestudents’maintaskaspartoftheirpractical
assessment is to “read” movies and analyze the film in detail based on what they have
learnedinclass(i.e.,widerangetheories,historicalcontextofcinemaandetc)andfrom
this“reading”theyarerequiredtoproduceanessayonacertaintopic.Thereisnowritten
examinationattheendofeithermodule.Studentassessmentsarebasedontheessaysthey
producefromreadingthemovies.Forinstance,intheCM135module,studentsneedto
produceanessayonanychosenmovie,ofapproximately1000wordsinlength.Thistask
involves an individual reading of a sequence of any chosen movie. Each student needs
toanalyzeamoviefromvariousaspectssuchasgrammarofthecinema,mise-en-scene
(whatisintheframe),capturingtheessenceinthemovie,focusingonaspectslikemusic,
camera,lighting,andothers.
4.2. Tasks
Aspartofourend-userinvolvementeffort,wedeployedourapplicationformanagingand
browsingmoviecontentstofilmstudystudents(takingmodulesCM272andCM135)for
thedurationofthewholespringsemester.Weprovidetwotypesofbrowsingfeaturesin
ourtool;BasicandAdvanced interface.Theadvancedtypeofbrowsing(Fig.2(b))has
170 N.M.Ali,A.F.Smeaton
Fig.2.Exampleofscreenshotpage(a)basicinterface(b)advancedinterface.
features that could enhance user browsing, like the inclusion of a visual timeline, sup-
port for key frame browsing, note taking features. On the contrary, some of the movies
werelistedunderthebasiccategoryinterface-typeofbrowsingthatprovidesonlyaba-
sic playbackfunctionwithnormalstandardDVD-likeplayerinterfacefacilitiessuchas
play,pauseorsliderbar(Fig.2(a)).Thecollectionof30moviescamefromvariousgen-
res (i.e., comedy, drama, romance, and action), ranging from contemporary Hollywood
moviestooldIrishmovies,withproductionyearsfrom1952to2004aslistedinTable1.
Table1
ListofMoviesinMOVIEBROWSER2.
Withadvancedfeatures Withbasicfeatures
Shrek(2001) AmericanBeauty(1999)
AboutAdam(2000)∗∗∗ TheMostFertileManInIreland(1999)∗
CircleofFriends(1995)∗ HighFidelity(2000)
Intermission(2003)∗ Oceans11(2001)
IntoTheWest(1992)∗ TheMagdeleneSisters(2002)∗
ManAboutDog(2004)∗ LiloandStitch(2002)
MichaelCollins(1996)∗ RoadToPerdition(2002)
OnTheEdge(2001)∗ ThisIsMyFather(1998)∗
SpinTheBottle(2003)∗ TheRoyalTenenbaums(2001)
TheQuietMan(1952)∗ EatThePeach(1986)∗
TheButcherBoy(1991)∗ Poitin(1979)∗∗∗
Korea(1995)∗∗∗ After68(1993)∗∗∗
Nora(2000)∗∗∗ GoldfishMemory(2003)∗∗∗
– TheShipper(1993)∗∗∗
– TheVisit(1992)∗∗∗
– TheBallroomofRomance(1982)∗∗∗
– BentOutofShape(1995)∗∗∗
Note:∗Irish-directedmovies;∗∗movieusedinCM272module.
ExploringtheUsageofaVideoApplicationTool:ExperiencesinFilmStudies 171
There were a few movies that were short in duration (less than 1-hour) that were used
in class such as After 68 [25 minutes], Bent Out Of Shape [27 minutes], The Visit [19
minutes] and The Ballroom of Romance [50 minutes]. These movies were categorized
under the standard browsing features due to their short lengths and difficulties in gen-
erating event detections and classifications as a result of that short length. We tried to
balancethenumberofmoviesthatwerecategorizedinto‘Advanced’or‘Basic’interface.
Movieswith(**)areexamplesofIrish-directedmoviesthatarealsousedaspartofthe
discussionsintheCM272module.
4.3. Procedures
Weprovidedasystemdemointheearlierpartofthesemesterforeachclass.Thelectur-
ersforeachmodulewereacknowledgedandinformedofthewholeprocedurebeforeany
deploymentwascarriedout;theygavegreatsupportandencouragementfortheprocess
toproceed.Abriefexplanationwasgiventoeachclassregardingtheusageofthesystem,
duringademopresentationofabout15minutes,afterwhichanemailwassenttoallthe
students,informingthemoftheirindividualsystemusernameandpassword,alongwith
information on the system (i.e., web links, player installation and other requirements).
Follow-upemailremindersweresent a few timesduringtheperiodof deployment.We
administeredanonlinequestionnaireinweek-13andweek-14ofthisinitialdeployment.
Students’usagedatalogswerecapturedautomatically.Attheendofthedeployment,an
emailwassenttostudentstothankthemandacknowledgetheirresponsesandcoopera-
tion.
4.4. DataCapture
Weseparatedthecaptureddataasfollows:
• Usage Logs: Our objective in performing the deployment on the students in the
University was to monitor and assess their actual-usage data of the developed
moviebrowsingandplaybacktool.Amongthelogscapturedweretheinteractions
or user actions with the features that we provided on the screen interfaces, either
BasicorAdvanced.
• Questionnaires: The objectives of the administered online questionnaires was to
measuresubjectivesatisfactionandoverallacceptancewiththenewdeployedsys-
tem, and to collect quantitative and qualitative data on the system we deployed.
Thisincludedquestionsonthedemographicsonstudents’gender,age,coursein-
formation,andotherpreliminaryinquiries;aswellastheiroverallreactiontothe
system in terms of perceived satisfaction, features provided, and opinions on the
valueoftechnologyincorporated.
5. Findings
Our findings from our various types of logging and qualitative assessments are divided
intousers,anduserinterfacefeatures,whichwenowpresent.
172 N.M.Ali,A.F.Smeaton
5.1. Users
Out of a total of 268 students in both classes, 107 students (40%) accessed the
MOVIEBROWSER2atleast2timesduringthedeploymentperiod,whichtellsusthat
they were not all wholly engaged with the system. 17 users (16%) each accessed the
system for up to 8 hours in total, most over several sessions. As shown in Table 2, the
total access duration time was approximately86 hours (CM272 = 57 hours, CM135 =
29hours).All7Hollywoodmoviesthatwerestoredinthesystemlibrarywereaccessed
a total of 73 of times (39%), with Shrek (2001) being mostly accessed and viewed by
students(Fig.3).Atfirstglancethismaynotseemsubstantial,butdeeperanalysisshows
thattheaccesswasveryfocusedonthestudents’tasks,thuswasatime-savingservice.
Table2
AdvancedandBasicpagehoursspent
Type Hoursspent
Advancedscreen 45
Basicscreen 33
Totalplaybackactivities(AdvancedandBasic) 78
Totalaccessedhoursinthesemester 86
Otheractivities* 8
*Otherthanplaybackactivities.
Fig.3.Advancedvs.Basicviewedmovies.