Table Of Content

An Improvement to the Domain Adaptation Bound in a PAC-Bayesian context 5 PascalGermain AmauryHabrard 1 IFT-GLO,Universite´Laval LaHC,UMRCNRS5516 0 Que´bec(QC),Canada Univ.ofSt-Etienne,France 2 [email protected] [email protected] n a J FrançoisLaviolette EmilieMorvant IFT-GLO,Universite´Laval LaHC,UMRCNRS5516 3 1 Que´bec(QC),Canada Univ.ofSt-Etienne,France [email protected] [email protected] ] L M Abstract . t a ThispaperprovidesatheoreticalanalysisofdomainadaptationbasedonthePAC- t s Bayesiantheory. Weproposeanimprovementofthepreviousdomainadaptation [ boundobtainedbyGermainetal. [1]intwoways. Wefirstgiveanothergeneral- 1 izationboundtighterandeasiertointerpret.Moreover,weprovideanewanalysis v oftheconstanttermappearingintheboundthatcanbeofhighinterestfordevel- 2 opingnewalgorithmicsolutions. 0 0 3 1 Introduction 0 . 1 Domainadaptation(DA)ariseswhenthedistributiongeneratingthetargetdatadiffersfromtheone 0 fromwhichthesourcelearninghasbeengeneratedfrom. Classicaltheoreticalanalysesofdomain 5 adaptationproposesomegeneralizationboundsovertheexpectedriskofaclassifierbelongingtoa 1 hypothesisclassHoverthetargetdomain[2,3,4]. Recently,Germainetal. havegivenageneral- v: izationboundexpressedasanaveragingovertheclassifiersinHusingthePAC-Bayesiantheory[1]. i Inthispaper,wederiveanewPAC-Bayesiandomainadaptationboundthatimprovestheprevious X resultof[1].Moreover,weprovideananalysisoftheconstanttermappearingintheboundopening r thedoortodesignnewalgorithmsabletocontrolthisterm. Thepaperisorganizedasfollows. We a introducetheclassicalPAC-BayesiantheoryinSection2. Wepresentthedomainadaptationbound obtainedin[1]inSection3. Section4presentsournewresults. 2 PAC-BayesianSetting inSupervised Learning In the non adaptive setting, the PAC-Bayesian theory [5] offers generalization bounds (and algo- rithms)forweightedmajorityvotesoverasetoffunctions,calledvoters. LetX ⊆ Rd betheinput space of dimension d and Y = {−1,+1}be the outputspace. A domain P is an unknowndis- s tributionover X ×Y. The marginaldistribution of P overX is denotedby D . LetH be a set s s of n voterssuch that: ∀h ∈ H, h : X → Y, and let π be a prioron H. A prior is a probability distributiononHthat“models”someaprioriknowledgeonqualityofthevotersofH. Then,givenalearningsampleS = {(x ,y )}ms ,drawnindependentlyandidenticallydistributed i i i=1 (i.i.d.) accordingto the distributionP , the aim ofthe PAC-Bayesian learneristo find a posterior s distributionρleadingtoaρ-weightedmajorityvoteB overHthathasthelowestpossibleexpected ρ risk, i.e., the lowest probability of making an error on future examples drawn from D . More s precisely,thevoteB anditstrueandempiricalrisksaredefinedasfollows. ρ 1 Definition 1. Let H be a set of voters. Let ρ be a distribution over H. The ρ-weighted majority voteB (sometimescalledtheBayesclassifier)is: ρ ∀x∈X, B (x) d=ef sign E h(x) . ρ h∼ρ (cid:20) (cid:21) ThetrueriskofB onadomainP anditsempiricalriskonam -sampleS arerespectively: ρ s s 1 ms R (B ) d=ef E I[B (x )6=y ], and R (B ) =def I[B (x )6=y ]. Ps ρ (xi,yi)∼Ps ρ i i S ρ ms ρ i i i=1 X whereI[a 6= b]isthe0-1lossfunctionreturning1ifa = band0otherwise. UsualPAC-Bayesian analyses[5,6,7,8,9]donotdirectlyfocusontheriskofB ,butboundtheriskofthecloselyrelated ρ stochasticGibbsclassifier G . Itpredictsthelabelofanexamplexbyfirst drawinga classifier h ρ fromHaccordingtoρ,andthenitreturnsh(x). Thus,thetrueriskandtheempiricalonam -sample s S ofG correspondtotheexpectationoftherisksoverHaccordingtoρ: ρ R (G ) =def E R (h) = E E I[h(x )6=y ], Ps ρ h∼ρ Ps (xi,yi)∼Psh∼ρ i i 1 ms and R (G ) =def E R (h) = E I[h(x )6=y ]. S ρ S i i h∼ρ ms h∼ρ i=1 X Note that it is well-knownin the PAC-Bayesian literature that the risk of the deterministic classi- fierB andtheriskofthestochasticclassifierG arerelatedbyR (B )≤2R (G ). ρ ρ Ps ρ Ps ρ 3 PAC-BayesianDomainAdaptationofthe Gibbs classifier Throughouttherestofthispaper,weconsiderthePAC-BayesianDAsettingintroducedbyGermain etal. [1]. ThemaindifferencebetweensupervisedlearningandDA is thatwe havetwo different domainsoverX×Y:thesourcedomainP andthetargetdomainP (D andD aretherespective s t s t marginalsoverX). TheaimisthentolearnagoodmodelonthetargetdomainP knowingthatwe t onlyhavelabelinformationfromthesourcedomainP . Concretely,inthesettingdescribedin[1], s we have a labeled source sample S ={(x ,y )}ms , drawn i.i.d. from P and a targetunlabeled i i j=1 s sample T = {x }mt , drawni.i.d. fromD . One thusdesires to learnfromS andT a weighted j j=1 t majorityvotewiththelowestpossibleexpectedriskonthetargetdomainR (B ),i.e.,withgood Pt ρ generalizationguaranteesonP . RecallingthatusualPAC-Bayesiangeneralizationboundstudythe t riskoftheGibbsclassifier,Germainetal.[1]havedoneananalysisofitstargetriskR (G ),which Pt ρ alsoreliesonthenotionofdisagreementbetweenthevoters: R (h,h′) d=ef E I[h(x)6=h′(x)]. (1) D x∼D Theirmainresultisthefollowingtheorem. Theorem1(Theorem4of[1]). LetHbeasetofvoters. ForeverydistributionρoverH,wehave: RPt(Gρ) ≤ RPs(Gρ)+disρ(Ds,Dt)+λρ,ρ∗T , (2) wheredis (D ,D )isthedomaindisagreementbetweenthemarginalsD andD , ρ s t s t dis (D ,D ) =def E (R (h,h′)−R (h,h′)) , (3) ρ s t (h,h′)∼ρ2 Ds Dt (cid:12) (cid:12) wwihtehreρ2ρ(∗h,=h′a)rg=minρ(hR)×(ρG(h)′)i,stahnedbλ(cid:12)(cid:12)(cid:12)esρt,ρd∗Tistr=ibuRtiPotn(Gonρ∗Tth)e+tarRgeDttd(oGmρa,Ginρ.(cid:12)(cid:12)(cid:12)∗T) + RDs(Gρ,Gρ∗T), T ρ Pt ρ NotethatthisboundreflectstheusualphilosophyinDA:Itiswellknownthatafavorablesituation forDAarriveswhenthedivergencebetweenthedomainsissmallwhileachievinggoodsourceper- formance[2,3,4]. Germainetal.[1]havethenderivedafirstpromisingalgorithmcalledPBDAfor minimizingthistrade-offbetweensourceriskanddomaindisagreement. NotethatGermainetal.[1]alsoshowedthat,foragivenhypothesisclassH,thedomaindisagree- mentofEquation(3)isalwayssmallerthantheH∆H-distanceofBen-Davidetal.[2,3]definedby 12sup(h,h′)∈H2|RDt(h,h′)−RDs(h,h′)|. 2 4 New Results 4.1 ImprovementofTheorem1 First,weintroducethenotionofexpectedjointerrorofapairofclassifiers(h,h′)drawnaccording tothedistributionρ,definedas e (G ,G ) =def E E I[h(x)6=y]×I[h′(x)6=y]. (4) Ps ρ ρ (h,h′)∼ρ2(x,y)∼Ps Thm2belowreliesonthedomaindisagreementofEq.(1),andonexpectedjointerrorofEq.(4). Theorem2. LetHbeahypothesisclass. Wehave 1 ∀ρonH, R (G ) ≤ R (G )+ dis (D ,D )+λ , (5) Pt ρ Ps ρ 2 ρ s t ρ whereλ isthedeviationbetweentheexpectedjointerrorsofG onthetargetandsourcedomains: ρ ρ λ d=ef e (G ,G )−e (G ,G ) . ρ Pt ρ ρ Ps ρ ρ (cid:12) (cid:12) (cid:12) (cid:12) Proof. First,notethatforanydistribut(cid:12)ionP onX×Y,withmargina(cid:12)ldistributionDonX,wehave 1 R (G ) = R (G ,G )+e (G ,G ), P ρ D ρ ρ P ρ ρ 2 as 2R (G ) = E E I[h(x)6=y]+I[h′(x)6=y] P ρ (h,h′)∼ρ2 (x,y)∼P (cid:16) (cid:17) = E E 1×I[h(x)6=h′(x)]+2×I[h(x)6=y]I[h′(x)6=y] (h,h′)∼ρ2 (x,y)∼P (cid:16) (cid:17) = R (G ,G )+2×e (G ,G ). D ρ ρ P ρ ρ Therefore, 1 R (G )−R (G )= R (G ,G )−R (G ,G ) + e (G ,G )−e (G ,G ) Pt ρ Ps ρ 2 Dt ρ ρ Ds ρ ρ Pt ρ ρ Ps ρ ρ 1(cid:16) (cid:17) (cid:16) (cid:17) ≤ R (G ,G )−R (G ,G ) + e (G ,G )−e (G ,G ) 2 Dt ρ ρ Ds ρ ρ Pt ρ ρ Ps ρ ρ 1(cid:12) (cid:12) (cid:12) (cid:12) = (cid:12)dis (D ,D )+λ . (cid:12) (cid:12) (cid:12) (cid:12) ρ s t ρ (cid:12) (cid:12) (cid:12) 2 TheimprovementofTheorem2overTheorem1 reliesontwo mainpoints. Onthe onehand,our newresultcontainsonlythehalfofdisρ(Ds,Dt). Ontheotherhand,contrarytoλρ,ρ∗T ofEq.(2), thetermλ ofEq.(5)doesnotdependanymoreonthebestρ∗ onthetargetdomain. Thisimplies ρ T thatournewboundisnotdegeneratedwhenthetwodistributionsP andP areequal(orveryclose). s t Conversely,whenP =P ,theboundofTheorem1gives s t RPt(Gρ)≤RPt(Gρ)+RPt(Gρ∗T)+2RDt(Gρ,Gρ∗T), wwhheicnhthiseastuplepaosrtt2oRfρPta(nGdρρ∗T∗).inMHoriesocvoenr,sttihtuetetedrmof2atRlDeats(tGtwρ,oGdρi∗Tff)eriesngtrcelaatsesrifitehrasn. zero for any ρ T 4.2 ANewPAC-BayesianBound NotethattheimprovementsintroducedbyTheorem2donotchangetheformandthephilosophyof thePAC-BayesiantheoremspreviouslypresentedbyGermainetal.[1]. Indeed,followingthesame prooftechnique,weobtainthefollowingPAC-Bayesiandomainadaptionbound. Theorem3. ForanydomainsP andP (resp. withmarginalsD andD )overX×Y,anysetof s t s t hypothesisH,anypriordistributionπ overH,anyδ ∈ (0,1],anyrealnumbersα > 0andc > 0, with a probability at least 1− δ over the choice of S ×T ∼ (P ×D )m, for every posterior s T distributionρonH,wehave c′ α′ KL(ρkπ)+ln3 R (G ) ≤ c′R (G )+α′ 1dis (S,T)+ + δ +λ + 1(α′−1), Pt ρ S ρ 2 ρ c α m ρ 2 (cid:18) (cid:19) c 2α whereλ isdefinedbyEq.(6),andwhere c′ d=ef , and α′ d=ef . ρ 1−e−c 1−e−2α 3 4.3 OntheEstimationoftheUnknownTermλ ρ Thenextpropositiongivesanupperboundonthetermλ ofTheorems2and3. ρ Proposition4. LetHbethehypothesisspace.IfwesupposethatP andP sharethesamesupport, s t then ∀ρonH, λ ≤ χ2 P kP e (G ,G ), ρ t s Ps ρ ρ whereePs(Gρ,Gρ)istheexpectedjointerroqron(cid:0)thesour(cid:1)cedistribution,asdefinedbyEq.(4),and χ2 P kP isthechi-squareddivergencebetweenthetargetandthesourcedistributions. t s (cid:0) (cid:1) Proof. SupposingthatP andP havethesamesupport,thenwecanupperboundλ usingCauchy- t s ρ Schwarzinequalitytoobtainline4fromline3. λ = E E I[h(x)6=y]I[h′(x)6=y]− E I[h(x)6=y]I[h′(x)6=y] ρ (cid:12)(h,h′)∼ρ2(cid:20)(x,y)∼Pt (x,y)∼Ps (cid:21)(cid:12) (cid:12) P (x,y) (cid:12) =(cid:12)(cid:12)(cid:12)(h,hE′)∼ρ2(cid:20)(x,yE)∼PsPst(x,y)I[h(x)6=y]I[h′(x)6=y]−(x,yE)∼PIs[h(x)6=y]I[h′(cid:12)(cid:12)(x)6=y](cid:21)(cid:12) (cid:12) P (x,y) (cid:12) =(cid:12)(cid:12)(cid:12)(h,hE′)∼ρ2(x,yE)∼Ps(cid:18)Pst(x,y) −1(cid:19)I[h(x)6=y]I[h′(x)6=y](cid:12) (cid:12)(cid:12) (cid:12) (cid:12) (cid:12) P (x,y) 2 (cid:12) ≤(cid:12) E t −1 × E E (I[h(x)(cid:12)6=y]I[h′(x)6=y])2 s(x,y)∼Ps(cid:18)Ps(x,y) (cid:19) r(h,h′)∼ρ2 (x,y)∼Ps P (x,y) 2 ≤ E t −1 × E E I[h(x)6=y]I[h′(x)6=y] s(x,y)∼Ps(cid:18)Ps(x,y) (cid:19) (h,h′)∼ρ2 (x,y)∼Ps P (x,y) 2 = E t −1 ×e (G ,G ) = χ2 P kP e (G ,G ). s(x,y)∼Ps(cid:18)Ps(x,y) (cid:19) Ps ρ ρ q t s Ps ρ ρ (cid:0) (cid:1) Thisresultindicatesthatλ canbecontrolledbytheterme ,whichcanbeestimatedfromsamples, ρ Ps and the chi-squared divergence between the two distributions that we could try to estimate in an unsupervised way or, maybe more appropriately, use as a constant to tune, expressing a tradeoff between the two distributions. This opens the door to derive new learning algorithmsfor domain adaptationwiththehopeofcontrollinginpartsomenegativetransfer. References [1] P.Germain,A.Habrard,F.Laviolette,andE.Morvant. APAC-Bayesianapproachfordomain adaptationwithspecializationtolinearclassifiers. InICML,pages738–746,2013. [2] S. Ben-David, J. Blitzer, K. Crammer, and F. Pereira. Analysisof representationsfor domain adaptation. InAdvancesinNeuralInformationProcessingSystems,pages137–144,2006. [3] S. Ben-David, J. Blitzer, K. Crammer,A. Kulesza, F. Pereira, andJ.W. Vaughan. A theoryof learningfromdifferentdomains. MachineLearning,79(1-2):151–175,2010. [4] Y.Mansour,M.Mohri,andA.Rostamizadeh. Domainadaptation: Learningboundsandalgo- rithms. InConferenceonLearningTheory,pages19–30,2009. [5] D.A.McAllester. SomePAC-Bayesiantheorems. MachineLearning,37:355–363,1999. [6] DMcAllester. PAC-Bayesianstochasticmodelselection. MachineLearning,51:5–21,2003. [7] M. Seeger. PAC-Bayesian generalizationboundsforgaussian processes. Journalof Machine LearningResearch,3:233–269,2002. [8] O.Catoni. PAC-Bayesiansupervisedclassification:thethermodynamicsofstatisticallearning, volume56. Inst.ofMathematicalStatistic,2007. [9] P. Germain, A. Lacasse, F. Laviolette, and M. Marchand. PAC-Bayesian learning of linear classifiers. InInternationalConferenceonMachineLearning,2009. 4