Robotic Haptic Proxies for Collaborative Virtual Reality

4.5 MB·

by  Zhenyi He

Robotic Haptic Proxies for Collaborative Virtual Reality
ZhenyiHe FengyuanZhu AaronGaudette
FutureRealityLab,NewYork FutureRealityLab,NewYork FutureRealityLab,NewYork
University University University
[email protected] [email protected] [email protected]
KenPerlin
FutureRealityLab,NewYork
University
[email protected]
7
1 ABSTRACT realobjectsintheenvironment, suchasauser’swristsora
0 WeproposeanewapproachforinteractioninVirtualReality workstation/table. With this knowledge of global state, we
2 (VR)usingmobilerobotsasproxiesforhapticfeedback. This utilize an outside-in approach to control lightweight robots
n approachallowsVRuserstohavetheexperienceofsharing (proxies) in realtime with a high degree of accuracy. The
a andmanipulatingtangiblephysicalobjectswithremotecol- systemisphysicallyconciseandsimpletosetup.
J laborators. Becauseparticipantsdonotdirectlyobservethe
Weimplementedmultipleprototypeapplicationsthataddress
1 robotic proxies, the mapping between them and the virtual
someofthemanypossibilitiesthistechnologyenables. Be-
3 objectsisnotrequiredtobedirect. Inthispaper,wedescribe
cause the Holojam SDK primarily supports multi-user VR
ourimplementation,variousscenariosforinteraction,anda
experiences,wechosetofocusoncollaborative,sharedexperi-
] preliminaryuserstudy.
C ences.Weimplementedprototypesinvolvingusersinthesame
H room(shared-space)aswellasprototypesinvolvingusersin
ACMClassificationKeywords
differentphysicallocations(remote-space),withsupportfor
. H.5.1. Multimedia Information Systems: Artificial, aug-
s varyingspatialconfigurations.
c mented,andvirtualrealities;H.5.2UserInterfaces:Interaction
[ styles Wedefinedthreepossiblemappingsbetweentherepresenta-

tionofphysicalproxies,whichare"invisible"whenwearing
1
AuthorKeywords aVRheadset,andvirtualobjects: one-to-one,many-to-one,
v
9 TangibleUserInterface;Physicalpresence;VirtualReality; andone-to-many. Multiplevirtualobjectsmayberepresented
7 Remotecollaboration;Haptics; physicallybyasinglerobot,orviceversa,dependingonthe
8 numberanddistributionoftherobotsinthescene,aswellas
8 INTRODUCTION userproximity(shared-spaceversusremote-space).
0 Inthepastdecade,virtualreality(VR)hasreachedagreater
1. level of popularity and familiarity than ever before. VR is Inremote-space,oursystemallowsmultipleusersto"share"
touchonthesamevirtualobject,enablingnewformsofcol-
0 gainingtractionintheconsumermarket,andintegratingphys-
laborationforremoteteamwork. Inshared-space,oneproxy
7 icalandvirtualenvironmentsseamlesslyisawell-recognized
orseveraldistributedproxiescanbesynthesizedviagesture
1 challenge. Hapticfeedbackcanbeapowerfulcomponentof
recognitionandpredictiontofacilitateswifthapticresponse
: immersionandcanbeusedtoimproveuserexperienceinVR
v foralargenumberofvirtualobjects,aswellas"command-
togreateffect.
Xi ing"ofphysicalobjectstomovewithouttouchingthem. We
In this paper, we present a generic, extensible system for executedalloftheseideasasprototypeapplications.
r
a providinghapticfeedbacktomultipleusersinvirtualrealityby
Ourmaincontributions:
intelligentlydirectingoneormoremobilerobots.Theserobots
actasproxiesforreal,physicalobjectsorforces,assistingthe • RemotesynchronizableroboticproxiesforVR.Basedon
environment by providing a deeper level of immersion at a roboticassistants,peoplecoulddophysicalcollaboration
lower cost. Using the Holojam Software Development Kit remotely.
(SDK)alongwithOptiTracktrackingtechnology,weareable
to determine the absolute position and rotation of various
• Mappings between virtual objects and invisible physical
robots.Weextendthepossibilityviaimplementingdifferent
mapping styles. Therefore, we could control one virtual
objectbymultiplerobots,orcontrolmultiplevirtualobjects
byonlyonerobot.
• Augmenttheexperiencebycombinationofphysicalfeed-
backandvirtualscene.
RELATEDWORK control distant objects without feedback. Also with shared
Severalmethodshavebeenadvancedthatsimulatethesense workspacesthatcancaptureandremotelyrendertheshapes
oftouch. However,manyareeithernotasmobileornotas ofpeopleandobjects,userscanexperiencehapticfeedback
lightweightasoursystem. basedonshapeddisplays[13].
HapticInterfaces MOBILEPHYSICALPROXIES
Recentlymoreandmoreresearchhasarisenfocusedonthe Tosupporttheillusionofphysicalsharingbetweencollabo-
intersectionbetweenhapticfeedbackandcollaboration. In- rators,oursystemrequiresoneormoremobilerobots,which
Form[6]proposedutilizingshapedisplaysinmultipleways mustbeintelligentlycontrolledinamannerthatrespectsthe
tomanipulatebyactuatingphysicalobjects. TangibleBitswas constraintsoftherobotsandtheenvironment. Weaccomplish
proposedtoempowercollaborationbymanipulatingphysical thiswithanoutside-inapproach,managingglobalstateand
objectsat1997[9],andtheideawasextendedin2008[8]. PSy- sendingrobotcontrolmessagesfromasingleservermachine.
Bench,aphysicalsharedworkspace,presentsanewapproach
Usingthead-hocservermodelintheHolojamSDK,wefirst
toenhanceremotecollaborationandcommunication,based
ascertainglobalstatebyreadingrealtimetrackingdatafrom
on the idea of Tangible Interfaces at 1998[3]. The concept
theOptiTracksystem,thencalculatethetargetpositionsfor
ofsynchronizeddistributedphysicalobjectswasmentioned
anyrobotsthatarecurrentlyactiveintheenvironment,based
in PSyBench[3], which demonstrated the potential of phys-
ontheapplicationrequirements. Ourad-hocservermaintains
ical remotecollaboration. One contribution in this paper is
asmanyroomsasisnecessaryforremote-spaceapplications
toshowhowpeoplecanexperienceconsistentphysicalfeed-
(alloftheexamplesinthispaperuseeitheroneortworooms)
backoverdistance,regardlessofthephysicalconfiguration
andcommandstheconnectedrobotstomovewhenneeded.
ofthecorrespondingremotespace. PSyBench[15]onlyhad
Becauseboththerobotsandoptitrackcamerasaresmall,and
1-to-1mappingwhileweextendedthemappingstyleandkind
thead-hocservercanrunonamidrangecomputer,theentire
ofscenarios. Alsoobjectscouldnotbeliftedanddisplayed
setupisfairlylightweightandportable.
thesamemovementwithoutVRsupport. InForm[6]didnot
supportcollaborationandthematerialsarefixedonthetable,
whileweofferamorelightweightapproach.SnakeCharmer[1]
hadsimilarideasaboutone-to-manymapping. However,we
supportcollaborationandwireless.
HapticFeedback
HapticfeedbackhasbeenmentionedfrequentlyinVRtraining,
especially in the medical field. The sense of touch is the
earliest developed in human embryology and is believed to
Figure1.Robotswithtrackedconfiguration
be essential for practice [5, 4]. Robot-assisted minimally
invasivesurgery(RMIS)holdsgreatpromiseforimprovingthe
accuracyanddexterityofasurgeon[14]. Hapticfeedbackhas
RobotDesign
potentialbenefitsnotonlyintraining,butinotherinteractions
Beginningwithseveralm3pirobots,weattachedaconfigu-
as well. Reality based interaction was proposed for post-
rationoftrackedmarkerstoeachoneinordertodetermine
WIMPinterfaces[11]. Tangibleinteractionsareobservable
its absolute position and orientation via the OptiTrack sys-
withbothvisualandhapticmodalitiesthatcouldhelppeople
tem. Wecreatedacylinder-shapedacryliccoverfortherobots
utilize basic knowledge about the behavior of our physical
(Figure1)inordertofacilitateeasygripping. Thisgripwas
world[18].
standardized across all the robots, with a minimalist, scale-
matchedvirtualrepresentationavailableforwheninteraction
HapticRe-Targeting
wasnecessary.
Manipulatingmultiplevirtualobjectsisalwaysachallenge,in
thatprecisely-locatedhapticproxyobjectsarerequired. [2]
MappingsBetweenPhysicalandVirtualSpace
proposed multiple approaches to align physical and virtual
Onechallengeofphysicallyrepresentingavirtualenvironment
objects. Redirectedtouching[12]consideredtheinflexibility
isthemethodusedtomapvirtualobjectstotheirphysicalcoun-
ofpassivehapticdisplaysandintroducedadeliberateincon-
terparts(orviceversa).Conventionally,virtualobjectshaveno
sistencybetweenrealhandsandvirtualhands. Inredirected
physicalrepresentationandcanonlybecontrolledindirectly
touching,asinglerealobjectcouldprovidehapticfeedback
orthroughtheuseofastandardinputdevice(suchasagame
for virtual objects of various shapes to enrich the mapping
controller). Also,thesetofphysicallycontrolledobjectsthat
betweenvirtualobjectsandphysicalproxies.
are visually represented in VR have been traditionally lim-
itedtotheinputdevicesthemselves. Incontrast,oursystem
Telepresence
providesthreedifferentmappingmechanismstoaugmentthe
C-Slatepresentedanewvision-basedsystem,whichcombined
relationshipbetweenphysicalandvirtualrepresentation.
bimanualandtangibleinteractionandthesharingofremote
gestures and physical objects as a new approach to remote • One-to-OneMapping
collaboration [10]. [7] tried an augmented reality way to Thisisthestandardandmoststraightforwardoption. When
users interact with a virtual object in the scene, they si-
multaneously interact with a physical proxy at the same
location.
• Many-to-OneMapping
When multiple proxies represent one virtual object, we
define the mapping as "many-to-one." This is useful for
remote-spaceapplications: avirtualobjectcouldexistin
thesharedenvironment,whichcouldthenbemanipulated
bymultipleusersviatheirlocalproxies.
• One-to-ManyMapping
Variousconstraintsincludingcostandspacelimitthecapa-
bilityofmaintainingaphysicalcounterpartforeveryone
of a large number of virtual objects. When less proxies
areavailablethanvirtualobjects,oneofthetotalavailable
proxiescouldrepresentagivenvirtualobjectwhenrequired.
Forexample,auserwithadisordereddeskinVRmaywant Figure3."ClinkingDrinks"Scenario
toreorganizealloftheirvirtualitems. Ineachinstanceof
theuserreachingtointeractwithavirtualitem,thesingle
(ornearest,iftherearemultiple)proxywouldrelocateto latencytestconcludedrangesfrom1-1.5seconds. Therefore
the position of the target item, standing by for pickup or we "faked" the scene that user supposed to see in order to
additionalinteractions. Thevirtualexperienceisseamless, eliminatethefeelingoftruelatency.
whilephysicallyasmallnumberofproxiesisconstantlyin
motion. Intheexperimentsectionwetestusersontheirperceptionof
thislatency,withpositiveresults. Forshared-spaceapplica-
LocalizationofUserEnvironments tions,weforegolatencyandinsteadrendera"loading"sprite
Oursystemenablesteamstocollaboratewitheachotherin tosignifytotheuserthattheproxyhasnotyetarrivedatits
remote-space. Hereweassumetheenvironmentisaworksta- finaldestination.
tionforthesakeofexample. Oneissueisthatnotallusers
maysharethesamespatialconfiguration. Toaddressthis,we PROTOTYPEAPPLICATIONS
builtagenericmodelforsynchronizingproxiesonatable.
PasstheMug
In this scenario, we demonstrate a novel method to extend
thecapabilityofhumaninteractionwithphysicalandvirtual
objects. Weemployacustomgesturerecognitiontechnique
alongsideourphysicalproxies, enablinguserstocommand
thepositionofamugusinghandmotions. Utilizingasimple
configurationoftrackedmarkersthatreportseachuser’swrist
positionandorientationviatheOptiTracksystem,userscan
pushaproximatemugacrossthetableviaapushinggesture,
pull a distant mug towards them with a pulling gesture, or
Figure2.Simulatedremote-spacesetupwithbothVRviews
motioninanotherdirectiontoslidethemugtowardsanother
partofthetable.Usersmustfirstselectatargetbyaimingtheir
Inademonstration,weassignedtwoparticipantstoseatsar- palmatanobject–theobjectwillshaketoindicateitisbeing
ranged perpendicular to each other, with tables unequal in targeted–afterwardstheobjectwillglowtoindicateitisunder
size (Figure 2). In VR, each participant observed the other gesturemode. Userscanstillpickupobjectsandinteractwith
asdirectlyoppositethemselves, withaconsistenttablerep- themnormallywhentheyarecloseenoughtoreach.
resentation. Thisdesignallowsuserstoexperienceanatural
viewpointregardlessofphysicalsetup. Theroboticproxies, ClinkingDrinks
meanwhile,adjustthemselvestooperatewithintheminimum Thisscenarioutilizesaone-to-onemapping,simulatingtheef-
boundary,relativetoeachuser’sindividualview. fectoftwousersstrikingtheirmugstogetherwhileinseparate
physicallocations(seeinFigure3).Ineachlocalenvironment,
Wealsochosetointroducelatencyforsynchronizinguserac-
theuser’smug’spositionisgovernedbyanon-robotictracked
tions in remote-space. Movements involving the proxy are
physicalobject,andtheforceofthestrikeissimulatedviaa
delayedbyaspecifiedtimeinterval(weuseadelayofonesec-
synchronizedmovingproxy.
ondforthedemonstrationabove),togivetherobotsufficient
timetocatchupwitheachuser’sactions. Thissmoothsthe
Tic-Tac-Toe
hapticoperationsconsiderably.
Herewerecreatedtheclassicboardgame,Tic-Tac-Toe,inVR,
Weadded"artificial"latencytoimprovetheuserexperience. but for users in remote locations. We utilize a "controller"
Basedontherobotwechoseandthesizeofworkstation,our object,allowingplayerstoselectatileforeachgamestep(see
the transparent cylinder holding by user in Figure 4). This themtoseeeveryoneelseasanavatar,walkaroundthephysi-
applicationimplementsmany-to-onemapping: twoproxies calworld,andinteractwithrealphysicalobjects. Youcansee
aresynchronizedallthetime,withasinglevirtualcontroller theheadsetandpairedgloveinFigure2.
synchronized across both spaces so that a player can reach
Wecancalculatethehead’spositionbasedonmarkerinfor-
for it on their turn and experience haptic feedback. When
mation. Theusersaretrackedallthetime. Intheframework
oneplayerisholdingthecontrollerandconsideringtheirnext
of Holojam, we have a server to receive data from outside
move,theotherplayercanobservethismovementintheirview
like OptiTrack and send customized message to all clients
aswell. Eachgamestep,aplayer’sselectedtiledisplayseither
(phones). Thatiswhyuserswearingheadsetcouldseereal-
an"O"oran"X"shape. Turnsareprocessedsequentially. The
timescene. Themarkersweusedareonlyfixedonheadsets
techniquesusedinthisprototypecouldeasilybeexpandedto
andhand-madeglove. Userscouldonlyfeelthegloveitself
othertabletopgames.
insteadofthemarkers,seeFigure2. Actuallyitisjustaband
aroundthepalm. Fromthefeedback,nouserscomplainedon
thesimplifiedglove.
Inourdesign,weneedalightweightphysicalobject,whichis
bothtrackedandmovable. Them3pirobot[17]isacomplete,
high-performancemobileplatformwithanm3piexpansion
boardasitssecondlevel. Youcanseethecombinationofthe
m3pirobotsandmarkersinFigure1.
For both experiments, the participants’ wrists were tracked
Figure4."Tic-Tac-Toe"Scenario byhandstrapscontainingtrackedmarkers,andtwotablesof
differentsizeswerealsotracked.
CityBuilder Participants
TheCityBuilderdemovisualizesavirtual3Dminiatureofa Sixteen participants invited to join our study, three female.
cityonaflatsurface,whereusersareabletopickupanybuild- Agesrangedfromtwenty-twotofifty-two,andthemedianage
ingandmoveittosomewhereelseonthemap. Eachuser’s wastwenty-sixyearsold. Allparticipantswereright-handed.
wrististrackedinthesamemannerasinthe"PasstheMug" Seventy-fivepercenthadexperiencedVRpriortothestudy.
demo, and aproxy is fittedbehind-the-scenes tothe virtual
buildingwhichisvisuallynearesttoagivenuser’shand–an Tic-Tac-Toe
exampleofone-to-manymapping. Inthisimplementation,the Inthisexperiment,theparticipantexperiencedamultiplayer
movementandpositionoftherobotsiscompletelyinvisible (two-person)playthroughofbothourprototypeapplication
totheuser. version of Tic-Tac-Toe and a purely virtual version, where
theirtrackedhandwasusedfortileselectionsinsteadofthe
EXPERIMENTANDRESULTS controller.
Wedesignedtwotestsforouruserstudy,modifiedfromour
Thetimeelapsedforeachgamewasrecorded,aswellasthe
prototypeapplications. Thefirst,aduplicateofour"Tic-Tac-
participant’sresponsestothefollowingquestions:
Toe" application, demonstrates many-to-one mapping. We
includedaversionofTic-Tac-Toethatwaspurelyvirtual(no • Did you feel like you and your testmate were sitting at
haptics)forcomparison. Thesecondexperiment,"Telekine- differenttables?
sis",waslooselybasedonour"PasstheDrinkingMug"ap-
plication,demonstratingone-to-onemapping,andmeasuring • Didyoufeellikeyouropponentwasmovingnaturally?
users’abilitytolearnthegesturesystemandtocommanda
• Wereyoumorecomfortablewiththephysicalorthevirtual
targetatvariouslocations.
versionofthegame?
Following each test, users filled out a survey of their expe-
Onascalefromonetofive:
rience. Of the twelve questions on the survey, three were
general,sixregardedtheTic-Tac-Toetest,andthreeregarded • Howmuchdelaydidyouexperiencebetweenyouractions
theTelekinesistest. andtheirexpectedoutcomes? (Notmuch–unendurable)
HardwareandSoftware • Howwelldidyoufeelyoucouldmanipulateobjectsinthe
ParticipantsworeSamsungGearVRheadsetswithOptiTrack virtualenvironment? (Notwell–verywell)
trackedmarkers,asperthecurrentspecificationsoftheHolo-
jamSDK.Holojam[16]isanuntetheredvirtualrealityheadset • Didyouunderstandthegame? (No–enjoyable)
systemthatenablespeoplehaveshared-spaceVRexperience.
HolojamhasbeendemoedinSIGGRAPH2015performance Telekinesis
session. Briefly:eachparticipantneedstoputonalightweight Inthisexperiment,wesplitthevirtualtableintonineparts,se-
wirelessmotion-trackedGearVRheadsetaswellasstrap-on quentiallyplacingatargetineachoneofthesubsections. Par-
glove markers, tracked by OptiTrack. These devices allow ticipantswerefirstgivenuptotwominutestolearnthegesture
Table1.Amountforthedecisionondirectlygrabbing Table2.Tic-Tac-ToeSurvey
positiononthetable 1 2 3 4 5 6 7 8 9 Questions Results
...differenttables? 2/16
amountofgrabbing 3 1 1 4 5 0 6 5 4
...opponentmovingnaturally? 16/16
...comfortablewiththephysicalversion? 11/16
...experienceddelay? 2.31/5
system,thenweobservedtheircommandchoice(including
...feelyoucouldmanipulateobjects? 4.02/5
non-gesturalphysicalinteraction)foreachtargetposition.
...understandthegame? 4.38/5
Thetimeelapsedforeachchoicewasrecorded,aswellasthe
participant’sresponsestothefollowingquestionsonascale
Table3.TelekinesisSurvey
from1to5:
Questions Results
• Hownaturaldidthemechanismforcontrollingmovement ...feltnaturalforcontrollingmovement? 3.5/5
throughtheenvironmentfeel? (Notnatural–verynatural) ...compellingsenseofobjectsmoving? 4.0/5
• Howcompellingwasyoursenseofobjectsmovingthrough
space? (Notcompelling–verycompelling)
wayfortilesfour,five,andsix(atamediumdistancefromthe
Followingthis,theparticipant’saccuracywastestedonone player),39outof48(81.3%)weremovedusinggestures. And
final command, given a target direction. We recorded the fortheclosetiles–seven,eight,andnine–33outof48(68.8%)
participant’sresponsetothefollowingquestion: weremovedusinggestures.
• Which direction matched your expectation of the com- Theaveragetimeelapsedwhenphysicallymovingtheobjects
mand’sresultthebest? (Forward/Backward/Left/Right/ was 1.5 seconds (ranging from 0.2 seconds to 5 seconds),
None) whereas the average time elapsed when using gestures was
1.97seconds(rangingfrom0.5secondsto3.2seconds).
Analysis
From this data, we determined that directly picking up an
The average length of a Tic-Tac-Toe game was one minute
objectiseasierthangesturing,especiallyatclosedistances,
andfortysecondsfortheversionusingourphysicalcontroller,
but gesturing is faster ifoptimized, and ismore stable over
and one minute and five seconds for the purely virtual ver-
distance.
sion. Althoughthephysicalversiontooklongeronaverage,
twenty-fivepercentofalltheparticipantsplayedfasterwith Whilemostparticipantspreferredtousegesturesfordistant
the physical controller than without it. From this data we targetsandphysicalinteractionforproximatetargets,onepar-
concludedthattheversionusingourphysicalcontrollerwas ticipantmentionedthattheypreferredgesturesoverphysical
comparableinaccessibilitytothepurelyvirtualversion–that interaction because the gestures were more fun. Two other
istosay,wedidnotfindasignificantdifferencebetweenthe users both chose to use gestures to push proximate targets
two play-styles. We would postulate that participants were awaybeforepullingthembackagain. Weconcludedfromthis
morefocusedonhowtowinratherthanthemethodofinput. thattheinteractionwedesignedisnotonlymeaningfuland
Ideally,ourphysicalcontrollersaresoseamlesslyintegrated useful,butenjoyableaswell.
thattheyareunnoticeable.
Survey
Figure5.Overviewoftabletileswithplayerposition
Table1neededtobeexplainedwithFigure5,whichshowed Figure6.Preferenceofdirectionfortelekinesis
thedistributionoftiles. Eachtilerepresentstheareawhere
thetargetwouldbedirected. Inthefirstrowoftable1, the
Figure 6 shows the preference of direction for telekinesis.
number representing positions on the table meant the same
After the tests, participants were asked to chose the easiest
thinginFigure5. Thesecondrowshowedhowmanytimes
direction for them to move. 37.5 percents chose forward
theychosetograbinsteadofusegesturestocontrolthetarget.
becausethegestureiseasyandtheeffectisimpressive. 31.3
Fortilesone,two,andthree(farfromtheplayer)inFigure5, percents chose right rather than left probably because they
theamountofgrabbingissumof3,1and1,whichif5. We are all right-handed. We concluded "picking is easier than
calculatedthat5outof48(10.4%)weremovedbygrabbing gesturingforcloser"basedonthetimeconsumedandtheir
and43outof48(89.6%)weremovedusinggestures. Same choices.
SenseofReality 3. ScottBrave,HiroshiIshii,andAndrewDahley.1998.
ViaTable2,onlytwoparticipants(12.5%)feltisolatedduring Tangibleinterfacesforremotecollaborationand
theexperiment. Allusersagreedthattheiropponentmoved communication.InProceedingsofthe1998ACM
naturally. Fromthisweconcludethatourremote-spacerepre- conferenceonComputersupportedcooperativework.
sentationwasacceptable. ACM,169–178.
Preference 4. BrianDunkin,GLAdrales,KApelgren,andJD
ViaTable2,elevenofthesixteenparticipants(68.8%)chose Mellinger.2007.Surgicalsimulation: acurrentreview.
thephysicalcontrollerovernocontrolleratallforthe"Tic- Surgicalendoscopy21,3(2007),357–366.
Tac-Toe"test. Althoughthevaluewasnotashighaswewould
5. P-JFagerandPervonWowern.2004.Theuseofhaptics
have liked, we concluded that our physical controller was
inmedicalapplications.TheInternationalJournalof
acceptable.
MedicalRoboticsandComputerAssistedSurgery1,1
(2004),36–42.
Feasibility
Forthequestion"Howwelldidyoufeelyoucouldmanipulate
6. SeanFollmer,DanielLeithinger,AlexOlwal,Akimitsu
objectsinthevirtualenvironment"inTable2,elevengradeda
Hogge,andHiroshiIshii.2013.inFORM:dynamic
4,fourgradeda5,andonegradeda3(average4.02). From
physicalaffordancesandconstraintsthroughshapeand
thisdata,weconcludedthatarobotproxyisacceptablefor objectactuation..InUIST,Vol.13.417–426.
remoteinteraction.
7. ZhenyiHeandXuboYang.2014.Hand-basedinteraction
InTable3,forthequestion,"Hownaturaldidthemechanism
forobjectmanipulationwithaugmentedrealityglasses.
forcontrollingmovementthroughtheenvironmentfeel?",81%
InProceedingsofthe13thACMSIGGRAPH
(average3.5)ofparticipantsfeltitwasnaturalenough(≥3)
InternationalConferenceonVirtual-RealityContinuum
tocontroltheobjectsusinggestureswithoutactuallytouching
anditsApplicationsinIndustry.ACM,227–230.
the objects. In addition, the average score for the question,
"Howcompellingwasyoursenseofobjectsmovingthrough 8. HiroshiIshii.2008.Tangiblebits: beyondpixels.In
thespace?"was4,with87.5%gradingcompelling(≥4). Proceedingsofthe2ndinternationalconferenceon
Tangibleandembeddedinteraction.ACM,xv–xxv.
Feeling
Forthequestion"Howmuchdelaydidyouexperiencebetween 9. HiroshiIshiiandBryggUllmer.1997.Tangiblebits:
youractionsandtheirexpectedoutcomes?"inTable2,more towardsseamlessinterfacesbetweenpeople,bitsand
than75%feltitwasendurable(≤2). Theaveragevaluewas atoms.InProceedingsoftheACMSIGCHIConference
2.31,lowervaluesbeingbetter(lessperceivedlatency). onHumanfactorsincomputingsystems.ACM,234–241.
10. ShahramIzadi,AnkurAgarwal,AntonioCriminisi,John
CONCLUSIONANDFUTUREWORK Winn,AndrewBlake,andAndrewFitzgibbon.2007.
Inthispaper,weproposedanewapproachforinteractionin C-Slate: amulti-touchandobjectrecognitionsystemfor
virtualrealityviarobotichapticproxies,specificallytargeted remotecollaborationusinghorizontalsurfaces.In
towardscollaborativeexperiences,bothremoteandlocal. We HorizontalInteractiveHuman-ComputerSystems,2007.
presentedseveralprototypesutilizingourthreemappingdefi- TABLETOP’07.SecondAnnualIEEEInternational
nitions,demonstratingthat–usingoursystem–severalvirtual Workshopon.IEEE,3–10.
objectscanberepresentedphysicallybyoneormorerobotic
proxies,andmultipleuserscansharetouchonthesamevirtual 11. RobertJKJacob,AudreyGirouard,LeanneMHirshfield,
object,orusegesturestocommandobjectswithouttouching MichaelSHorn,OritShaer,ErinTreacySolovey,and
them. Ourpreliminaryexperimentsreturnedpositiveresults. JamieZigelbaum.2008.Reality-basedinteraction: a
Inthefutureweplantoundertakeamorecomprehensivestudy frameworkforpost-WIMPinterfaces.InProceedingsof
focusingondeeperapplicationinteractionsandexpandedhard- theSIGCHIconferenceonHumanfactorsincomputing
warecapability. systems.ACM,201–210.
12. LuvKohli.2010.Redirectedtouching: Warpingspaceto
REFERENCES
remappassivehaptics.In3DUserInterfaces(3DUI),
1. BrunoAraujo,RicardoJota,VarunPerumal,JiaXianYao,
2010IEEESymposiumon.IEEE,129–130.
KaranSingh,andDanielWigdor.2016.SnakeCharmer:
PhysicallyEnablingVirtualObjects.InProceedingsof 13. DanielLeithinger,SeanFollmer,AlexOlwal,and
theTEI’16: TenthInternationalConferenceonTangible, HiroshiIshii.2014.Physicaltelepresence: shapecapture
Embedded,andEmbodiedInteraction.ACM,218–226. anddisplayforembodied,computer-mediatedremote
collaboration.InProceedingsofthe27thannualACM
2. MahdiAzmandian,MarkHancock,HrvojeBenko,Eyal
symposiumonUserinterfacesoftwareandtechnology.
Ofek,andAndrewDWilson.2016.HapticRetargeting:
ACM,461–470.
DynamicRepurposingofPassiveHapticsforEnhanced
VirtualRealityExperiences.InProceedingsofthe2016 14. AllisonMOkamura.2009.Hapticfeedbackin
CHIConferenceonHumanFactorsinComputing robot-assistedminimallyinvasivesurgery.Current
Systems.ACM,1968–1979. opinioninurology19,1(2009),102.
15. EsbenWarmingPedersenandKasperHornbæk.2011.
Tangiblebots: interactionwithactivetangiblesin
tabletopinterfaces.InProceedingsoftheSIGCHI
ConferenceonHumanFactorsinComputingSystems.
ACM,2975–2984.
16. KenPerlin.2016.FutureReality: Howemerging
technologieswillchangelanguageitself.IEEEComputer
GraphicsandApplications36,3(2016),84–89.
17. Pololu.2016.Pololum3piRobotwithmbedSocket.
https://www.pololu.com/product/2151/resources.(2016).
Accessed: 2017-01-30.
18. BryggUllmer,HiroshiIshii,andRobertJKJacob.2005.
Token+constraintsystemsfortangibleinteractionwith
digitalinformation.ACMTransactionson
Computer-HumanInteraction(TOCHI)12,1(2005),
81–118.