Table Of Content

A DNF Blocking Scheme Learner for Heterogeneous Datasets ABSTRACT Blocking results in the selection of a small subset of pairs, 5 called the candidate set, which is input to a second step to EntityResolutionconcernsidentifyingco-referententitypa- 1 determineco-referenceusingasophisticatedsimilarityfunc- irsacrossdatasets. Atypicalworkflowcomprisestwosteps. 0 tion. We exclusively address blocking in this work. In the first step, a blocking method uses a one-many func- 2 Blockingmethodsuseablockingscheme toassignentities tion called a blocking scheme to map entities to blocks. In toblocks. Recently,DisjunctiveNormalForm(DNF)Block- n the second step, entities sharing a block are paired and a compared. Current DNF blocking scheme learners (DNF- ing Scheme Learners (BSLs) were proposed to learn DNF J BSLs) apply only to structurally homogeneous tables. We blocking schemes using supervised [3, 21], semi-supervised [6] or unsupervised machine learning techniques [16]. 8 present an unsupervised algorithmic pipeline for learning DNF-BSLsoperateinanexpressivehypothesisspaceand DNF blocking schemes on RDF graph datasets, as well as ] structurally heterogeneous tables. Previous DNF-BSLs are have shown excellent empirical performance [3]. Despite B theiradvantages,currentDNF-BSLsassumethatinputdat- admitted as special cases. We evaluate the pipeline on six D real-worlddatasetpairs. Unsupervisedresultsareshownto asetsaretabular andhavethesameschemas. Thelatteras- sumption is often denoted as structural homogeneity [12]. . be competitive with supervised and semi-supervised base- s Theassumptionoftabularstructuralhomogeneityrestricts lines. Tothebestofourknowledge,thisisthefirstunsuper- c application of DNF-BSLs to other data models. Recent [ vised DNF-BSL that admits RDF graphs and structurally heterogeneous tables as inputs. growth of graph datasets and Linked Open Data (LOD) [5] 1 motivatesthedevelopmentofaDNF-BSLfortheRDF1data v model. These graphs are often published by independent CategoriesandSubjectDescriptors 4 sources and are heterogeneous [27]. 9 I.2.6[Artificial Intelligence]: Learning;H.2.8[Database Inthispaper,wepresentagenericalgorithmicpipelinefor 6 Management]: Database applications—Data Mining learning DNF blocking schemes on pairs of RDF datasets. 1 The formalism of DNF blocking schemes relies on the ex- 0 GeneralTerms istence of a schema. RDF datasets on LOD may not have . accompanying schemas [5]. Instead of relying on possibly 1 Algorithms, Experimentation, Performance unavailablemetadata,webuildadynamicschema usingthe 0 5 properties in the RDF dataset. The RDF dataset can then 1 Keywords be logically represented as a property table, which may be : populatedatrun-time. Previously,propertytableswerede- v Heterogeneity, Entity Resolution, Unsupervised Learning finedasphysical datastructuresusedintheimplementation i X of triplestores [28]. Using a logical property table represen- 1. INTRODUCTION tation admits application of a DNF-BSL to RDF datasets. r a Entity Resolution (ER)istheidentificationofco-referent Asaspecialcase,thepipelinealsoadmitsstructurallyhet- entities across datasets. Different communities refer to it erogeneous tabular inputs. That is, the pipeline can be ap- as instance matching, record linkage and the merge-purge pliedtotabulardatasetswithdifferent schemas. Thus,pre- problem [12, 13]. Scalability indicates a two-step solution viousDNF-BSLsbecomespecialcasesofthepipeline,since [12]. The first step, blocking, mitigates brute-force pairwise they learn schemes on tables with the same schemas. As a comparisonsonallentitiesbyclusteringentitiesintoblocks secondspecialcase,thepipelineaccommodatesRDF-tabular and then comparing pairs of entities only within blocks [9]. heterogeneity,withoneinput,RDF,andtheother,tabular. RDF-tabularheterogneityapplieswhenlinkingdatasetsbe- tween LOD and the relational Deep Web [15, 5]. Much existing ER research bypasses the issue of structural heterogeneity by assuming that schema matching is Permissiontomakedigitalorhardcopiesofallorpartofthisworkfor physically undertaken prior to executing an ER algorithm personalorclassroomuseisgrantedwithoutfeeprovidedthatcopiesare [12]. Schema matching itself is an active, challenging re- notmadeordistributedforprofitorcommercialadvantageandthatcopies search problem [1, 14]. Instead of relying on the unrealis- bearthisnoticeandthefullcitationonthefirstpage.Tocopyotherwise,to ticassumptionofperfectschemareconciliation,thepipeline republish,topostonserversortoredistributetolists,requirespriorspecific permissionand/orafee. 1Resource Description Framework Copyright20XXACMX-XXXXX-XX-X/XX/XX...$15.00. admits possibly noisy schema mappings from an existing cates generator to perform unsupervised schema matching schema matcher, and directly uses the mappings to learn [4]. We describe Dumas in Section 5. a DNF blocking scheme. If a matcher is unavailable, the The property table representation used in this paper is a pipeline allows for a fall-back option. physically implemented data structure in the Jena triple- Weshowanunsupervisedinstantiationofthegenericpipe- store API2 [28]. In this paper, it is used as a logical data line by strategically employing an existing instance-based structure. We note that the concept of logically represent- schema matcher called Dumas [4]. In addition to schema ing one data model as another has precedent. In particu- mappings,Dumasalsogeneratesnoisyduplicates,whichwe lar, the literature abounds with proposed methods on how recycle and pipe to a new DNF-BSL. The new DNF-BSL to integrate relational databases (RDB) with the Semantic uses only two parameters, which may be robustly tuned for Web. Sahoo et al. extensively surveyed this topic, called good empirical performance across multiple domains. The RDB2RDF [24]. A use-case is the Ultrawrap architecture, BSL is also shown to have a strong theoretical guarantee. whichutilizesRDB2RDFtoenablereal-timeOntology-based We evaluate the full unsupervised pipeline on six real- DataAccessorOBDA[25]. Weeffectivelytackletheinverse world dataset pairs against two extended baselines, one of problem by translating an RDF graph to a logical property whichissupervised [3,16]. Oursystemperformanceisfound table. This is the first application to devise such an inverse to be competitive, despite training samples not being man- translation for heterogeneous ER. ually provided. Given that unsupervised methods exist for thesecond ERstepinboththeSemanticWebandrelational 3. PRELIMINARIES communities [13, 12], this first unsupervised heterogeneous Wepresentdefinitionsandexamplestoplacetheremain- DNF-BSL enables, in principle, fully unsupervised ER in der of the work in context. Consider a pair of datasets R both communities. 1 and R . Each dataset individually conforms to either the Section2describessomerelatedworkinthearea,followed 2 RDFortabulardatamodel. AnRDF datasetmaybevisu- bypreliminariesinSection3. Section4describesthegeneric alized as a directed graph or equivalently, as a set of triples. pipeline, followed by an unsupervised instantiation in Sec- A triple is a 3-tuple of the form (subject, property3, object). tion 5. Section 6 describes the experiments, with results Atabular datasetconformstoatabularschema,whichisthe discussed in Section 7. The paper concludes in Section 8. table name followed by a set of fields. The dataset instance is a set of tuples, with each tuple comprising field values. 2. RELATEDWORK ER was comprehensively surveyed by Elmagarmid et al. Example 1. Dataset1(inFigure1)isanRDFdatasetvi- [12], with a generic approach represented by Swoosh [2]. sualized as a directed graph G=(V,E), and can be equiv- Separately, blocking has witnessed much specific research, alently represented as a set of |E| triples. For example, with Christen surveying blocking methods [9]. Bilenko et (Mickey Beats, hasWife, Joan Beats) would be one such al. [3],andMichelsonandKnoblock[21]independentlypro- triple in the triples representation. Datasets 2 and 3 are posed supervised DNF-BSLs in 2006. Since then, a semi- tabular dataset examples, with the former having schema supervised adaptation of the BSL proposed by Bilenko et EmergencyContact(Name,Contact,Relation). Thefirsttu- al. hasbeenpublished[3,6],aswellasanunsupervisedsys- ple of Dataset 2 has field values Mickey Beats, Joan Beats tem [16]. The four systems assume structural homogeneity. and Spouse respectively. The keyword null is reserved. We discuss their core principles in Section 4.3. AccordingtotheRDFspecification4,subjectsandproper- Heterogeneousblockingmaybeperformedwithoutlearn- tiesmustnecessarilybeUniformResourceIdentifiers(URIs), ingaDNFscheme. OneexampleisLocalitySensitiveHash- while an object node may either be a URI or a literal. URI ing(LSH),employedbytheHarra system,forinstance[17]. elements in RDF files typically have associated names (or LSH is promising but applies only to specific distance mea- labels), obtained through dereference. For ease of exposi- sures,likeJaccardandcosine. TheTypifier systembyMaet tion, we henceforth refer to every URI element in an RDF al. isanotherrecentexamplethatreliesontype inferencing file by its associated name. Note also, that in the most andwasdesignedforWebdatapublishedwithoutsemantic general case, RDF datasets do not have to conform to any types[20]. Incontrast,DNF-BSLscanbeappliedgenerally, schema. Thisiswhytheyarecommonlyvisualizedassemi- withmultiplestudiesshowingstrongempiricalperformance structured datasets, and not as tables. In Section 4.1, we [21,3,6,16]. Finally,althoughrelated,clustering istechni- showhowtodynamicallybuildaproperty schema andlogi- cally treated separately from blocking in the literature [9]. cally represent RDF as a tabular data structure. In the Semantic Web, ER is simultaneously known as in- Anentityisdefinedasasemantically distinctsubject node stance matching and link discovery, and has been surveyed in an RDF dataset, or as a (semantically distinct) tuple in by Ferraram et al. [13]. Existing works restrict inputs to a tabular dataset. The entity referring to Mickey Beats is RDF. Also, most techniques in the Semantic Web do not showninredinalldatasetsinFigure1. EntityResolution is learn schemes, but instead present graph-based blocking theprocessofresolvingsemanticallyequivalent(butpossibly methods, a good example being Silk [27]. syntactically different) entities. As earlier described, ER is Finally,theframeworkinthispaperalsoreliesonschema traditionallyconductedontableswiththesameschemas,or mapping. Schema mapping is an active research area, with onone5 dataset. Anexample wouldbe identifying that the a good survey provided by Bellahsene et al. [1]. Gal notes two highlighted tuples in Dataset 3 are duplicates. that it is a difficult problem [14]. Schema matchers may return 1:1 or n:m mappings (or 2https://jena.apache.org/ even1:nandn:1). Aninstance-based schemamatcherrelies 3Alternatively called predicate; we uniformly use property. on data instances to perform schema matching [1]. A good 4http://www.w3.org/RDF/ exampleisDumas [4], whichreliesonaninexpensivedupli- 5Commonlyjustcalleddeduplication,ifthisisthecase[12]. Figure 1: Three example datasets exhibiting various kinds of heterogeneity In the Semantic Web, ER is operationalized by connect- it was input the Last Name field values from the first and ing two equivalent entities with an owl:sameAs6 property fourth tuples in Dataset 3. Since these field values have a edge. Forexample,thetwonodesreferringtoMickeyBeats token (Beats) in common, the GBP returns True. inDataset1shouldbeconnectedusinganowl:sameAs edge. EasyoperationalizingofER(andmoregenerally,linkspeci- Aspecificblockingpredicate (SBP)explicitlypairsaGBP fication [27])explainsinparttherecentinterestinERinthe to a specific field. SemanticWeb[13]. Intherelationalsetting,ERistradition- Definition 3. Aspecific blocking predicate isapair(p ,f) ally operationalized through joins or mediated schemas. It i where p is a general blocking predicate and f is a field. islessevidenthowtooperationalizeERacrossRDF-tabular i A specific blocking predicate takes two tuples t and t as inputs, such as linking Datasets 1 and 2. We return to this 1 2 arguments and applies p to the appropriate field values f issue in Section 4.1. i 1 and f from both tuples. A tuple pair is said to be covered Tointroducethecurrent notionofDNFblockingschemes, 2 if the specificblocking predicate returns True for thatpair. tabularstructuralhomogeneityisassumedfortheremainder ofthissection. Inlatersections,wegeneralizetheconcepts. Previous DNF research assumed that all available GBPs Themostbasicelementsofablockingschemeareindexing can be applied to all fields of the relation [16, 3, 6, 21] . functions hi(xt) [3]. An indexing function accepts a field Hence, given a relation R with m fields in its schema7, and valuefromatupleasinputandreturnsasetY thatcontains sGBPs,thenumberofSBPsisexactlyms. Finally,aDNF 0 or more blocking key values (BKVs). A BKV identifies a blocking scheme is defined as: blockinwhichthetupleisplaced. Intuitively,onemaythink ofablockasahashbucket,exceptthatblockingisone-many Definition 4. A DNF blocking scheme f is a positive P while hashing is typically many-one [9]. For example, if Y propositionalformulaconstructedinDisjunctiveNormalFo- containsmultipleBKVs,atupleisplacedinmultipleblocks. rm8(DNF),usingagivensetH ofSBPsasthesetofatoms. Additionally,ifeachterm isconstrainedtocompriseatmost Definition 1. An indexing function h : Dom(h ) → U∗ i i one atom, the blocking scheme is referred to as disjunctive. takesasinputafieldvaluex fromsometupletandreturns t asetY thatcontains0ormoreBlockingKeyValues(BKVs) SBPscannotbenegated,sincetheDNFschemeisaposi- from the set of all possible BKVs U∗. tiveformula. Atuplepairissaidtobecoverediftheblocking scheme returns True for that pair. Intuitively, this means The domain Dom(h ) is usually just the string datatype. i that the two constituent tuples share a block. In practice, The range is a set of BKVs that the tuple is assigned to. both duplicate and non-duplicate tuple pairs can end up Each BKV is represented by a string identifier. gettingcovered,sinceblockingisjustapre-processingstep. Example 2. An example of an indexing function is To- Example 4. Considerthedisjunctivescheme(ContainsC- kens. When applied to the Last Name field value of the ommonToken,LastName)∨(SameFirstDigit,Zip),applied fourthtupleinDataset3,theoutputsetY is{W.,Beats,Jr.}. on Dataset 3. While the two tuples referring to Mickey This leads to the notion of a general blocking predicate Beats would share a block (with the BKV Beats), the non- (GBP). Intuitively, a GBP p (x ,x ) takes as input field duplicate tuples referring to Susan and Samuel would also values from two tuples, t andi tt,1antd2 uses the ith indexing shareablock(withtheBKV6). Notealsothatthefirstand 1 2 function to obtain BKV sets Y and Y for the two argu- fourthtuplessharemorethanoneblock,sincetheyalsohave 1 2 ments. The predicate is satisfied iff Y and Y share ele- BKV 7 in common. 1 2 ments,orequivalently,ift andt haveablockincommon. 1 2 Givenablockingscheme,ablockingmethod wouldneedto Definition 2. Ageneralblockingpredicate p :Dom(h )× maptuplestoblocksefficiently. PerDefinition4,ablocking i i Dom(h ) → {True,False} takes as input field values x scheme takes a tuple pair as input. In practice, linear-time i t1 and x from two tuples, t and t , and returns True if hash-based techniques are usually applied. t2 1 2 h (x )∩h (x )(cid:54)=Φ, and returns False otherwise. i t1 i t2 Example 5. To efficiently apply the blocking scheme in EachGBPisalwaysassociatedwithanindexingfunction. the previous example on each individual tuple, tokens from Example 3. Consider the GBP ContainsCommonToken, 7Structural homogeneity implies exactly one input schema, even if there are multiple relational instances. associated with the previously introduced Tokens. Suppose 8A disjunction of terms, where each term is a conjunction 6http://www.w3.org/TR/owl-ref/ of literals. the field value corresponding to field Last Name are extracted, along with the first character from the field value of the Zip field, to obtain the tuple’s set of BKVs. For example, applied to the first tuple of Dataset 3, the BKV set {Beats,7} is extracted. An index is maintained, with the BKVs as keys and tuple pointers as values. With n tuples, traditional blocking computes the blocks in time O(n) [9]. Figure 3: Property Table representation of Dataset Let the set of generated blocks be Π. Π contains sets of 1 in Figure 1 the form B , where B is the block referred to by the BKV v v v. The candidate set of pairs Γ is given below: reserved delimiter (; in Figure 3). Technically, field values (cid:91) Γ= {(r,s)},∀r,s∈Bv|r∈R1,s∈R2 (1) now have set semantics, with null representing the empty Bv∈Π set. Also, the original set of triples can be losslessly recon- structedfromthepropertytable(andviceversa),makingit Γispreciselythesetinputtothesecond stepofER,which an information-preserving logical representation. Although classifieseachpairasaduplicate,non-duplicateorprobable intuitively straightforward, we detail these lossless conver- duplicate [8]. Blocking should produce a small Γ but with sions in the Appendix. highcoverageanddensityofduplicates. Metricsquantifying The physical property table was proposed to eliminate these properties are defined in Section 6. expensive property-property self-joins that occur frequently Finally, schema mapping is utilized in the paper. The in SPARQL9 queries. For ER, the logical data structure formal definitionofamappingisquitetechnical;thesurvey is useful because it allows for a dynamic schema that is by Bellahsene et al. provides a full treatment [1]. In this resolvableatrun-time. TriplestoreslikeJenaallowupdating paper, an intuitive understanding of the mapping as a pair andqueryingofRDFgraphs,despitetheunderneathtabular of field-sets suffices. For example, ({Name},{First Name, representation [28]. If the RDF dataset is already stored in Last Name}) is a 1:n mapping between Datasets 2 and 3. such a triplestore, it would not have to be moved prior to More generally, mappings may be of cardinality n:m. The ER.Thisgivesthepropertytableasystems-level advantage. simplest case is a 1:1 mapping, with singleton components. More importantly, having a schema for an RDF dataset meansthatwecaninvokeasuitableschemamatcherinthe 4. THEGENERICPIPELINE generalizedpipeline. Aswesubsequentlyshow,theextended Figure 2 (a) shows a schematic of the generic pipeline. DNF-BSLinthepipelinerequiresthedatasetstohave(pos- Two heterogeneous datasets are initially provided as input, sibly different) schemas10. Finally, representing RDF as a with either dataset being RDF or tabular. If the dataset is tableallowsustoaddressRDF-tabularheterogeneity,byre- RDF, we logically represent it as a property table. We de- ducing it to tabular structural heterogeneity. Traditional scribethedetailsandadvantagesofthistabular datastruc- ERoperationsbecomewell-definedforRDF-tabularinputs. tureinSection4.1. Thekeypointtonoteisthattheschema Onekeyadvantageofthepropertyschemaisthatitdoes matching moduletakestwotables asinput,regardlessofthe not rely on RDF Schema (RDFS) metadata. In practice, data model, and outputs a set of schema mappings Q. An this allows us to represent any file on Linked Open Data extended DNF-BSLacceptsQandalsoatrainingsetofdu- tabularly, regardless of whether metadata is available. plicatesandnon-duplicatesasinput,andlearnsanextended EveninthesimpleexampleofFigure3, wenotethatthe DNF blocking scheme. The DNF-BSL and the blocking property table is not without its challenges. It is usually scheme need to be extended because tables are now struc- sparse, and it may end up being broad, for RDF datasets turallyheterogeneous,andtheSection3formalismdoesnot with many properties. Furthermore, properties are named natively apply. In Section 4.2, the formalism is extended to using different conventions (for example, the prefix Has oc- admit structural heterogeneity. curs in all the properties in Figure 3) and could be opaque, Figure 2 (b) shows an unsupervised instantiation of the depending on the dataset publisher. We show empirically generic pipeline. We describe the details in Section 5. (Section 6) that the instantiated pipeline (Section 5) can handle these difficulties. 4.1 PropertyTableRepresentation Despite their formal description as sets of triples, RDF 4.2 Extendingtheformalism filesareoftenphysically storedintriplestoresassparseprop- TheformalisminSection3assumedstructuralhomogene- erty tables [28]. In this paper, we adapt this table instead ity. We extend it to accommodate structural heterogeneity. as a logical tabular representation of RDF. To enable the Asinput,considertwodatasetsR andR . Ifeitherdataset 1 2 logical construction on RDF dataset R, define the property isinRDF,weassumethatitisinpropertytableform. Con- schema {subject}∪{p|∃(s,p,o),(s,p,o) ∈ R}. In essence, sider the original definition of SBPs in Definition 3. SBPs we flatten the graph by assigning each distinct property (or are associated with a single GBP and a single field, mak- edge label) a corresponding field in this schema, along with ing them amenable only to a single schema. To account for an extra subject field. Every distinct subject in the triples- heterogeneity, we extend SBPs by replacing the field input set has exactly one corresponding tuple in the table. with a mapping. Denote as A and A the respective sets 1 2 Forexample,Figure3isthepropertytablerepresentation of fields of datasets R and R . We define simple extended 1 2 of Dataset 1 in Figure 1. If a subject does not have a cor- SBPs below: responding object value for a given property, the reserved keyword null is entered. If a subject has multiple object 9http://www.w3.org/TR/rdf-sparql-query/ values for a property, the values are concatenated using a 10Even non-DNF blocking typically assumes this [20]. Figure 2: (a) shows the generic pipeline for learning a DNF blocking scheme on heterogeneous datasets, with (b) showing an unsupervised instantiation Definition 5. Asimpleextendedspecificblockingpredicate isapair(p ,m)wherep isageneralblockingpredicateand i i m = ({f },{f }) is a mapping from a single field f ∈ A 1 2 1 1 to a single field f ∈ A . The predicate takes two tuples 2 2 t and t as arguments and applies p to the field values 1 2 i corresponding to f and f in the two tuples respectively. 1 2 The correspondence in Definition 5 is denoted as simple Figure 4: An example showing how H is formed sinceitusesamappingofthesimplestcardinality(1:1). Def- c inition 3 can be reformulated as a special case of Definition 5, with A =A and f =f . 1 2 1 2 Finally, (simple or complex) extended DNF schemes can The SBP semantics are not evident if the mapping car- be defined in a similar vein as Definition 4, using (simple dinality is n:m, 1:n or n:1, that is, between two arbitrary or complex) extended SBPs as atoms. One key advantage field-subsets,F ⊆A andF ⊆A . Ifweinterpret thetwo 1 1 2 2 ofusingdisjunctionasthecombineoperatorinDefinition6 setsasrepresenting|F ||F |1:1mappings,weobtainaset of 1 2 is that, assuming simple extended SBPs as atoms for both simple extended SBPs {(p ,({f },{f }))|f ∈F ,f ∈F }. i 1 2 1 1 2 2 simpleandcomplexextendedDNFschemes,theschemere- Theinterpretationaboveismotivatedbytherequirement mainsapositivebooleanformula, byvirtueofdistributivity that an SBP should always return a boolean value. We ap- of conjunction and disjunction. proachtheproblembyusingthen:mmappingstoconstruct the set of simple extended SBPs, as shown above. We then 4.3 ExtendingExistingDNF-BSLs usedisjunction asacombine operatoronallelementsofthe ExistingDNF-BSLs[3,21,16,6]relyonsimilarhigh-level constructedsettoyieldasinglebooleanresult. Wecanthen principles,whichistodeviseanapproximationalgorithmfor define complex extended SBPs. solving the NP-hard optimization problem first formalized Definition 6. A complex extended specific blocking predi- by Bilenko et al. [3]. The approximation algorithms are cate is a pair (pi,M) where pi is a general blocking pred- differentinthattheyrequiredifferentparametersandlevels icate and M is a mapping from a set F1 ⊆ A1 to a set of supervision. These are detailed below and summarized F2 ⊆A2. The predicate takes two tuples t1 and t2 as argu- in Table 1. These BSLs were originally designed only for ments and applies on them |F1||F2| simple extended SBPs structurally homogeneous tables, with a single field-set A. (pi,m), where m ranges over all 1:1 mappings in the set We describe their underlying core principles before describ- {({f1},{f2})|f1 ∈ F1,f2 ∈ F2}. The predicate returns the ingextensionsinkeepingwiththeformalisminSection4.2. disjunction of these |F1||F2| values as the final output. Assume a set of GBPs G. The core of all approximation algorithms would first construct a search space of SBPs H Example 6. Consider the mapping between sets {Name} by forming the cross product of G and A. The goal of the inDataset2and{FirstName,LastName}inDataset3,in algorithm is to choose a subset H(cid:48) ⊆H such that the opti- Figure 1. Let the input GBP be ContainsCommonToken. mizationcondition11laidoutbyBilenkoetal. issatisfied,at The complex extended SBP corresponding to these inputs least approximately [3]. The condition assumes that train- would be ContainsCommonToken({Name}, {First Name}) ingsetsofduplicatesD andnon-duplicatesN areprovided. ∨ ContainsCommonToken({Name}, {Last Name}). This Intuitively, the condition states that the disjunctive block- complex extended SBP would yield the same result as the ing scheme formed by taking the disjunction of SBPs in H(cid:48) simpleextendedSBPContainsCommonToken({Name},{Na- covers (seeDefinition4)atleast(cid:15)|D|duplicates,whilecov- me})ifanewfieldcalledName isderived fromthemerging ering the minimum possible non-duplicates [3]. Note that (cid:15) of the First Name and Last Name fields in Dataset 3. is a parameter common to all four systems12 in Table 1. An operator like conjunction is theoretically possible but In order to learn a DNF scheme (as opposed to just dis- mayproverestrictivewhenlearningpracticalschemes. The junctive), a beam search parameter k (also common to all disjunction operator makes complex extended SBPs more 11We describe the condition here intuitively, and formally expressive than simple extended SBPs, but requires more state it in the Appendix. computation (Section 4.3). Evaluating alternate combine 12(cid:15) was designated as min thresh in the original System 1 operators is left for future work. paper [21], and σ in the System 3 paper [6]. Specifically, we build a set Q of all 1:1 mappings, |Q| = Table 1: DNF-BSL Systems |A ||A |. Recall A is the field set of dataset i. We de- 1 2 i note the constructed set H of SBPs as simple exhaustive. ID System Parameters Supervision 1 Michelson and (cid:15), k Supervised Note that for the set of all mappings (≈ 2|A1|2|A2|), the constructedsetH (denotedcomplexexhaustive)isnotcom- Knoblock [21] putationally feasible for non-trivial cases. Even the simple 2 Bilenko et al. [3] (cid:15),η,k Supervised exhaustivecaseisonlyafall-backoption,sinceatrue setof 3 Cao et al. [6] s,(cid:15),τ,α,k Semi-sup. 1:1 mappings Q would be much smaller16 than this set. 4 Kejriwal and Mi- Generator: Unsup. ranker [16] c,ut,lt,d,nd Learner: (cid:15),η,k 5. ANUNSUPERVISEDINSTANTIATION 5 Algorithm 1 herein κ, k Unsup. A key question to ask is whether the generic pipeline can be instantiated in an unsupervised fashion. As we showed earlier, existing DNF-BSLs that can be extended require foursystems)isrequired. Thisparameterisusedtosupple- some form of supervision. An unsupervised heterogeneous ment theoriginalsetH withterms,toobtainanewsetHc. DNF-BSL is important because, in principle, it enables a This combinatorial process is demonstrated in Figure 4. fully unsupervised ER workflow in both the relational and H originallyconsistsoftheSBPsa,bandc. TheseSBPs SemanticWebcommunities. AsthesurveysbyElmagarmid coversometuplepairs(TPs). Supposek=2. Atermofsize et al. and Ferraram et al. note, unsupervised techniques 2 is formed by checking if any TP is covered by (at least) for the second ER step do exist already [12, 13]. A second two SBPs. For example, TP-3 is covered by SBPs a and motivationistheobservationthatexistingunsupervisedand b, and hence, also covered by the term a∧b. For k > 2, semi-supervised homogeneous DNF-BSLs (Systems 3-4) re- terms from size 2 to size k are recursively added to H; the quire considerable parameter tuning. Parameter tuning is final supplemented set is denoted as Hc. Note that for |H| increasingly being cited as an important algorithmic issue, predicates, building Hc takes O(|H|k) time per TP. Given inapplicationsrangingfromschemamatching[18]togeneric the exponential dependence on k and diminishing returns, machine learning [11]. Variety in Big Data implies that al- previous results capped k at 2 [16, 3]. If k=1, Hc =H. gorithm design cannot discount parameter tuning. ThesetH(cid:48) ⊆Hcthatisnowchosenbytheapproximation WeproposeanunsupervisedinstantiationwithanewDNF- scheme would potentially consist of terms and SBPs, with BSLthatrequiresonlytwo parameters. InTable1,onlythe their disjunction yielding a k-DNF scheme13. supervisedSystem1requirestwoparameters. Theschematic While System 1 only requires (cid:15) and k as its parameters, of the unsupervised instantiation (of the generic pipeline in Systems 2 and 4 prune their search spaces by removing all Figure 2(a)) is shown in Figure 2(b). We use the existing SBPs and terms from Hc that cover more than η|N| non- schema matcher, Dumas, in the instantiated pipeline [4]. duplicates. Notethatthisstepheuristically improvesboth14 Dumasoutputs1:1fieldmappingsbyfirstusingaduplicates quality and run-time. It comes at the risk of failure, since generator to locate tuple pairs with high cosine similarity. ifthesearchspaceisprunedexcessively(byhighη),itmay Inthesecondstep,DumasusesSoft-TFIDF tobuildasimi- become impossible to cover at least (cid:15)|D| duplicates. Sys- larity matrix fromeachgeneratedduplicate. Ifnduplicates tems 3 and 4 require less supervision but significantly more are input to the second step, n similarity matrices are built parametertuning,giventheyrelyonmanymoreparameters. and then averaged into a single similarity matrix. The as- Systems1-3relyondifferent15SetCovering (SC)variants signment problem is then solved by invoking the Hungar- [10]. All three systems can be extended by constructing a ian Algorithm on this matrix [19]. This results in exactly search space of complex extended SBPs using G and the min(|A |,|A |)1:1fieldmappings(thesetQ)beingoutput. 1 2 mappings set Q, instead of G and a field set A. The un- In addition to using Q, we recycle the noisy duplicates of derlying SC approximations operate in this abstract search Dumas and pipe them into Algorithm 1. Note that Dumas space to choose the final set H(cid:48). An extended DNF scheme does not generate non-duplicates. We address this issue in is formed by a disjunction of (extended) elements in H(cid:48). a novel way, by permuting the generated duplicates set D. Modifying System 4 is problematic because the system is SupposethatDcontainsntuplepairs{(r ,s ),...(r ,s )}, 1 1 n n unsupervised and runs in two phases. The first phase (de- with each r,s respectively from datasets R ,R . By ran- 1 2 noted as generator) generates a noisy training set, and the domly permuting the pairs in D, we heuristically obtain second phase (denoted learner) performs feature-selection non-duplicatepairsoftheform(r ,s ),i(cid:54)=j. Notethat(at i j on the noisy set to output a DNF blocking scheme. The most) n! distinct permutations are possible. For balanced feature-selectionbasedlearnerissimilartoSystems1-3and supervision, we set |N| = |D|, with N the permutation- can beextended. Unfortunately,thegeneratorexplicitly as- generated set. sumeshomogeneity,andcannotbedirectlyextendedtogen- Empirically,thepermutationisexpectedtoyieldaprecise eratetrainingexamplesforheterogeneousdatasets. Thisim- N becauseofobservedduplicatessparsity inERdatasets[9, pliesthattheDNF-BSLcomponentoftheproposedgeneric 16]. Thissparsityisalsoakeytenetunderlyingtheblocking pipeline in Figure 2(a) cannot be instantiated with an ex- procedure itself. If the datasets were dense in duplicates, isting unsupervised system. blocking would not yield any savings. In the event that a schema matcher (and thus, Q) is Algorithm 1 shows the pseudocode of the extended DNF unavailable in Figure 2(a), we present a fall-back option. BSL.InputstothealgorithmarethepipedDumasoutputs, D and Q. To learn a blocking scheme from these inputs, 13A k-DNF formula has at most k literals in each term. two parameters k and κ need to be specified. Similar to 14Since H now contains only a few, high-quality elements. c 15Surveyed briefly in the Appendix. 16At most min(|A |,|A |) 1 2 Algorithm 1 Learn Extended k-DNF Blocking Scheme ferencebetweenthefractions ofcoveredduplicatesandnon- Input: SetDofduplicatetuplepairs,SetQofmappings duplicates. Athresholdparameter,κ,isusedtoremovethe Parameters : Beamsearchparameterk,SC-thresholdκ SBPs and terms that have low scores. Intuitively, κ tries Output : Extended DNF Blocking Scheme B to balance the conflicting needs of previously described pa- Method : //Step 0: Construct sets N and H rameters, (cid:15) and η, and reduce tuning effort. The range of Permute pairs in D to obtain N, |N|=|D| κ is [-1,1]. An advantage of the parameter is that it has Construct set H of simple extended SBPs using set G of an intuitive interpretation. A value close to 1.0 would in- GBPs and Q dicate that the user is confident about low noise-levels in Supplement set H to get set H using k inputs D and Q, since high κ implies the existence of ele- c //Step 1: Build Multimaps MD(cid:48) and MN(cid:48) ments in Hc that cover many positives and few negatives. Construct M =< X,H >, X is a tuple pair in D, Since many keys in M(cid:48) are removed by high κ, this also D X D H ⊆H contains the elements in H covering X leadstocomputationalsavings. However,settingκtoohigh X c c Repeat previous step to build M for tuple pairs in N (perhaps because of misguided user confidence) could po- N Reverse M and M to respectively get M(cid:48) and M(cid:48) tentially lead to excessive purging of M(cid:48) , and subsequent D N D N D //Step 2: Run approximation algorithm algorithm failure. Experimentally, we show that κ is easily for all X ∈keyset(M(cid:48) ) do tunableandevenhighvaluesofκarerobusttonoisyinputs. D Score X byusingformula|M(cid:48) (X)|/|D|−|M(cid:48) (X)|/|N| Weighted Set Covering (W-SC) is then performed using D N Remove X if score(X)<κ Chvatal’s algorithm19 [10],witheachkeyinM(cid:48) actingasa D end for set andthetuplepairscoveredbyallkeysaselementsofthe Perform W-SC on keys in M(cid:48) using Chvatal’s heuristic, universe set U. For example, assuming all SBPs and terms D weights are negative scores in the keyset of M(cid:48) in Figure 5 have scores above κ, U = D //Step 3: Construct and output DNF blocking scheme {1,2,3,4,5}. Note that only M(cid:48) is pruned (using κ) and D B:= Disjunction of chosen keys also, W-SC is performed only on M(cid:48) . M(cid:48) only aids in the D N Output B scorecalculation(andsubsequentpruningprocess)andmay be safely purged from memory before W-SC commences. W-SCneedstofindasubsetoftheM(cid:48) keyset thatcovers D all of U and with minimum total weight. For this reason, theweightofeachsetisthenegative ofitscalculatedscore. GiventhatsetschosenbyW-SCactuallyrepresentSBPsor terms, their disjunction is the k-DNF blocking scheme. Under plausible20 complexity assumptions, Chvatal’s algorithm is essentially the best-known polynomial-time approximation for W-SC [23]. For example, Bilenko et al. usedPeleg’sapproximationtoRed-BlueSC [22,7],whichis known to have worse bounds [7]. The proposed DNF-BSL hasthestrongesttheoreticalapproximationguaranteeofall Figure 5: Step 1 of Algorithm 1, assuming the in- systems in Table 1. formation in Figure 4 6. EXPERIMENTS (extended) Systems 1-3 in Table 1, G, Q and k are used to 6.1 Metrics construct the search space, H . Note that G is considered c thealgorithm’sfeaturespace,andisnotadataset-dependent The quality and efficiency of blocking is evaluated by the input (or parameter). Designing an expressive G has com- special metrics, Reduction Ratio (RR), Pairs Completeness putationalandqualitativeeffects,asweempiricallydemon- (PC) and Pairs Quality (PQ) [9]. Traditional metrics like strate. We describe the GBPs included in G in Section 6. precisionandrecalldonotapplytoblockingsinceitisapre- Step 0 in Algorithm 1 is the permutation step just de- processing step, and its output is not the final ER output. scribed, to generate the non-duplicates set N. G and Q are TodefineRR,whichintuitivelymeasuresefficiency,consider thenusedtoconstructthesetH ofsimpleextended17 SBPs thefullsetofpairsΩthatwouldbegeneratedintheabsence (Definition5),with|H|=|G||Q|. H issupplemented(using ofblocking. Specifically,Ωistheset{(r,s)|r∈R ,s∈R }. 1 2 parameterk)toyieldHc,asearlierdescribedinSection4.3. RR is then given by the formula, 1−|Γ|/|Ω|. Given the Step1constructsmultimaps18onwhichSetCovering(SC) sparsity of duplicates in real-world datasets [9], RR should is eventually run. As a first logical step, multimaps MD ideally be close to, but not precisely, 1.0 (unless Γ=φ). and MN are constructed. Each tuple pair (TP) in D is a Denote the set of all duplicate pairs, or the ground-truth, key in MD, with the SBPs and terms in Hc covering that as Ωm. Similarly, consider the set of duplicates included in TP comprising the value set. MD is then reversed to yield the candidate set Γm =Γ∩Ωm. The metric PC quantifies M(cid:48) . M(cid:48) is built analogously. Figure 5 demonstrates the theeffectiveness oftheblockingschemebymeasuringcover- D N procedure, assuming D contains TPs 1-5, covered as shown age ofΓ withrespecttoΩ . Specifically,itisgivenbythe m m in Figure 4. The time complexity of building (both) M(cid:48) formula, |Γ |/|Ω |. Low PC implies that recall on overall D m m and M(cid:48) is O(|H|k(|D|+|N|)). ERwillbelow,sincemanyduplicatesdidnotshareablock N In Step 2, each key is first scored by calculating the dif- to begin with. 17Since Dumas only outputs 1:1 mappings 19WeincludeChvatal’salgorithmintheSCAppendixsurvey. 18Multimap keys reference multiple values (or a value set). 20P ⊂NP PC and RR together express an efficiency-effectiveness ters, versus the five of System 3. Finally, the three System tradeoff. ThemetricPQissometimesusedtomeasurehow 4 parameters are comparable to the corresponding System dense theblocksareinduplicates,andisgivenby|Γ |/|Γ|. 2 parameters, and evaluations in the original work showed m PQ has not been reported in recent BSL literature [3],[16]. thatthelearnerisrobusttominorparametervariations[16]. One reason is that PQ can be equivalently expressed as For these reasons, the feature-selection learner of System 4 c.PC/(1−RR), where c is |Ω |/|Ω|. When comparing two was extended to serve as a semi-supervised baseline. m BSLs,PQcanthereforebeexpressedwhollyintermsofPC 6.4 Methodology and RR. We do not consider PQ further in this paper. In this section, we describe experimental methodology. 6.2 Datasets Experimental results will be presented in Section 7. The heterogeneous test suite of six dataset pairs (and 6.4.1 Dumas nine individual datasets) is summarized in Table 2. The suite spans over four domains and the three kinds of het- Giventhatthetestsuiteislargerandmoreheterogeneous erogeneity discussed in the paper. All datasets are from inthispaperthanintheoriginalDumaswork[4],weperform real-world sources. We did not curate these files in any two preliminary experiments to evaluate Dumas. Dumas way, except for serializing RDF datasets as property tables was briefly described in Section 5. (instead of triples-sets). The serializing was found to be Preliminary Experiment 1: We evaluate the perfor- near-instantaneous(<1second)inallcases,withnegligible mance of Dumas’s duplicates generator. The generator uses run-time compared to the rest of the pipeline. TF-IDF to retrieve the highest scoring duplicates in order. Dataset Pairs (DPs) 1 and 2 are the RDF benchmarks Suppose we retrieve t duplicates (as an ordered list). De- inthe2010instance-matching track21 ofOAEI22,anannual note, for any k≤t, the number of true positives in sub-list SemanticWebinitiative. Notethatanearliertabular version [1...k]asd(k). DefinePrecision@k asd(k)/k andRecall@k of DP 1 is also popular in homogeneous ER literature [9]. as d(k)/|Ωm|, where Ωm is the ground-truth set of all du- DPs3,5and6describevideogame information. DP6has plicates. We plot Precision@k against Recall@k for all DPs alreadybeenusedasatestcaseinapreviousschemamatch- todemonstratetheprecision-recalltradeoff. Toobtainafull ingwork[26]. vgchartz isatabulardatasettakenfromarep- setofdatapoints,wesettat50,000forallexperiments,and utable charting website23. DBpedia contains 48,132 triples calculate Precision@k and Recall@k for k∈{1...t}. extractedfromDBpedia24,andhasfour(three25 properties Preliminary Experiment 2: Although t was set to a andsubject)fieldsand16,755tuplesinpropertytableform. highvalueinthepreviousexperiment,Dumasrequiresonly Finally,IBM containsuser-contributeddataextractedfrom a few top pairs (typically 10-50) for the schema matching the Many Eyes page26, maintained by IBM Research. step[4]. Tocomputethesimilaritymatrixfromtpairs,Du- DP4describesUSlibraries. Libraries1 wasfromaPoint masusesSoft TF-IDF,whichrequiresanoptional threshold of Interest website27, and Libraries 2 was taken from a US θ. For a given DP, denote Q(cid:48) as the ground-truth set of government listing of libraries. schema mappings, Q as the set of Dumas mappings, and DPs2and4containn:mground-truthschemamappings, Qm ⊆Qasthesetofcorrect Dumasmappings. Definepre- while the others only contain 1:1 ground-truth mappings. cision (onthistask)as|Qm|/|Q|andrecall as|Qm|/|Q(cid:48)|. In this experiment, we set t and θ to default values of 50 and 6.3 Baselines 0.5 respectively and report the results. We also vary these Table 1 listed existing DNF-BSLs, with Section 4.3 de- parameters and describe performance differences. scribinghowtheextendedversionsfitintothepipeline. We 6.4.2 DNFBSLParameterTuning extend two state-of-the-art systems as baselines: Supervised Baseline: System 2 was chosen as the su- We describe parameter tuning methodology. Note that pervised baseline [3], and favored over System 1 [21] be- thebeamsearchparameterk(seeTable1)isnottechnically causeofbetterreportedempiricalperformanceonacommon tuned,butset,incumbentontheexperiment(Section6.4.3). benchmark employed by both efforts. Tuning the parame- Baseline Parameters: Both baseline parameters are ter η leads to better blocking schemes, making System 2 a tuned in similar ways. We do an exhaustive parameter state-of-the-art supervised baseline. sweep of (cid:15) and η, with values in the range of 0.90−0.95 Semi-supervised Baseline: We adapt System 4 as a and 0.0−0.05 (respectively) typically maximizing baseline semi-supervised baselinebyfeedingitthesamenoisydupli- performance. Lowη maximizesRR,whilehigh(cid:15)maximizes cates generated by Dumas as fed to the proposed learner, PC. Although extreme values, with η = 0 and (cid:15) = 1, are as well as a manually labeled negative training set. The expected to maximize performance, they led to failure28 on learnerofSystem4usesfeature selection andwasshownto alltestcases. Thisdemonstratesthenecessityofproperpa- beempiricallycompetitivewithsupervised baselines[16]. In rameter tuning. Fewer parameters imply less tuning, and contrast, System 3 did not evaluate its results against Sys- faster system deployment. tem2. Also,thelearnerofSystem4requiresthreeparame- SC-thresholdκ: Thesingleparameterκoftheproposed method was tuned on the smallest test case (DP 1) and 21http://oaei.ontologymatching.org/2013/#instance found to work best at 0.9. To test the robustness of κ to 22Ontology Alignment Evaluation Initiative different domains, we fixed it at 0.9 for all experiments. 23vgchartz.com |D| and |N|: We emulate the methodology of Bilenko 24dbpedia.org et al. in setting the numbers of positive and negative sam- 25genre, platform and manufacturer. ples input to the system. Bilenko et al. input 50% of true 26www-958.ibm.com/software/data/cognos/manyeyes/ positives and an equal number of negatives to train their datasets 27http://www.poi-factory.com/poifiles 28(cid:15)|D| duplicates could not be covered (see Section 4.3). Table 2: Details of dataset pairs. The notation, where applicable, is (first dataset) /× (second dataset) ID Dataset Pairs Fields Total Entity Pairs DuplicatePairs Data Model 1 Restaurant1/Restaurant2 8/8 339 × 2256 = 764,784 100 RDF/RDF 2 Persons 1 /Persons 2 15/14 2000 × 1000 = 2 million 500 RDF/RDF 3 IBM/vgchartz 12/11 1904 × 20,000 ≈ 38 million 3933 Tabular/Tabular 4 Libraries 1 /Libraries 2 5/10 17,636 × 26,583 ≈ 469 million 16,789 Tabular/Tabular 5 IBM/DBpedia 12/4 1904 × 16,755 ≈ 32 million 748 Tabular/RDF 6 vgchartz/DBpedia 11/4 20,000 × 16,755 ≈ 335 million 10,000 Tabular/RDF system [3]. Let this number be n (= |D|,|N|) for a given test suite. For example, n = 50 for DP 1. For fairness, we use these numbers for the semi-supervised baseline and proposed system also. We retrieve the top n pairs from the DumasgeneratorandinputthepairstobothsystemsasD. Additionally,weprovidenlabelednon-duplicates(asN)to the semi-supervised baseline. In subsequent experiments, the dependence of the proposed system on n is tested. 6.4.3 ExtendedDNFBSL We describe the evaluation of the extended DNF BSLs. SetGofGBPs: Bilenkoetal. firstproposedasetGthat Figure 6: Dumas duplicates-generation results, for hassincebeenadoptedinfutureworks[3,6,16]. Thisorig- k∈{1,2...50000} inal set contained generic token-based functions, numerical functionsandcharacter-basedfunctions[3]. Norecentwork attemptedtosupplementGwithmoreexpressiveGBPs. In formancevariationsoftheproposedsystemwiththenumber particular,phonetic GBPssuchasSoundex,Metaphone and ofprovidedduplicates,n. Inindustrialsettingswithanun- NYSIIS were not added to G, despite proven performance known ground-truth, n would have to be estimated. An benefits [8]. We make an empirical contribution by supple- important question is if we can rely on getting good results menting G with all nine phonetic features implemented in with constant n, despite DP heterogeneity. an open-source package29. For fairness, the same G is al- Statistical Significance: We conduct experiments in waysinputtoalllearnersintheexperimentsbelow. Results tenruns,and(whererelevant)reportstatisticalsignificance on Experiment 2 will indirectly demonstrate the benefits of levels using the paired sample Student’s t-distribution. On supplementing G. A more detailed description of the origi- blockingmetrics,wereportifresultsarenotsignificant(NS), nal (and supplemented) G is provided in the Appendix. weaklysignificant(WS),significant(SS)orhighlysignificant Experiment1: Toevaluatetheproposedlearner against (HS),basedonwhetherthep-valuefallswithinbrackets[1.0, the baselines, we input the same Q (found in Preliminary 0.1), (0.05,0.1], (0.01, 0.05] and [0.0, 0.01] respectively. As Experiment 2) to all three systems. The systems learn the for the choice of samples, we always individually paired PC DNF scheme by choosing a subset H(cid:48) ⊆ H of SBPs and and RR of the proposed system against the baseline that c supplemented terms, with H constructed from G, Q and k achieved a better average on the metric. c (Section 4.3). We learn only disjunctive schemes by setting Implementation: All programs were implemented in k to 1 in this experiment. Java on a 32-bit Ubuntu virtual machine with 3385 MB of Experiment2: WerepeatExperiment1butwithk=2. RAM and a 2.40 GHz Intel 4700MQ i7 processor. We mentioned in Section 4.3 that complexity is exponential in k for all systems in Table 1. Because of this exponential 7. RESULTSANDDISCUSSION dependence, it was not possible to run the experiment for k =2 on all DPs. We note which DPs presented problems, 7.1 Dumas andwhy. Notethat,ifG,Qandthetrainingsetsarefixed, increasing k seems to be the only feasible way of improving PreliminaryExperiment1: Figure6showstheresults blocking quality. However, G is more expressive in this pa- ofDumas’sduplicates-generator. ExceptforDP3,precision per. Intuitively,weexpectthedifferenceacrossExperiments on all cases seems inadequate even at low recall levels. Al- 1 and 2 to be narrower than in previous work. though recall of 100% is eventually attained on most DPs, Experiment 3: In a third set of experiments, we evalu- thepriceis(near)0%precision. Closerinspectionofthere- ate how blocking performance varies with Q. To the base- sultsshowedthatmanyfalsepositivesgotrankedatthetop. line methods, we input the set of (possibly n:m) ground- We discuss the implications of these noisy results shortly. truth mappings Q(cid:48) while the Dumas output Q is retained Preliminary Experiment 2: Table3showstheschema fortheproposedlearner. Thegoalistoevaluateifextended mapping results retrieved from Dumas with parameters set DNF-BSLsaresensitive,orifnoisy1:1matcherslikeDumas at t = 50 and θ = 0.5. These default values were found to suffice for the end goal. maximize performance inall cases, inagreementwithsimi- Experiment 4: We report on run-times and show per- lar values in the original work [4]. We also varied t from 10 to10,000andθfrom0to0.9. Performanceslightlydeclined 29org.apache.commons.codec.language (by at most 5%) for some DPs when t < 50, but remained cost. With k = 2, only DPs 1 and 5 were found computa- Table 3: Best Results of Dumas Schema-Matcher tionally feasible. On the other DPs, the program either ran out of RAM (DPs 4,6), or did not terminate after a long31 Dataset Pair Precision Recall time (DPs 2,3). The former was observed because of high 1 100% 87.5% n and the latter because of the large number of fields (see 2 92.86% 86.67% Table2). Settingkbeyond2wascomputationallyinfeasible 3 91% 100% evenforDPs1and5. Furthermore, resultsonDPs1and5 4 75% 33.33% showed no statistical difference compared to Experiment 1, 5 100% 100% eventhoughrun-timeswentupbyanapproximatefactorof 6 100% 100% 16 (for both DPs). Experiment 3: We providedthe ground-truthsetQ(cid:48) to baseline methods (and with k again set to 1), while retain- otherwise constant across parameter sweeps. This confirms ing Q for the proposed method. Again, we did not observe thatDumasisquiterobusttotandθ. Onedisadvantage of anystatisticallysignificantdifferenceinPCorRRforeither Dumasisthatitisa1:1matcher. Thisexplainsthelowerre- baseline method. We believe this is because the cases for callnumbersonDPs2and4,whichcontainn:mmappings. which Q(cid:48) would most likely have proved useful (DPs 2 and InExperiment3,wetestifthisisproblematicbyproviding 4,whichcontainn:mmappingsthatDumascannotoutput) theground-truthsetQ(cid:48) tobaselines,andcomparingresults. already perform well with the Dumas-output Q. Discussion: Animportantpointtonotefromthe(mostly) Experiment 4: Theoretically, run-time of Algorithm 1 good results in Table 3 is that generator accuracy is not al- wasshowntobeO(|H|k(|D|+|N|));therun-timesofother ways predictive of schema mapping accuracy. This robust- systems in Table 1 are similar. Empirically, this has never ness to noise is always an important criteria in pipelines. If been demonstrated. For k = 1, we plot the run-time data noiseaccumulates ateachstep,thefinalresultswillbequal- collectedfromExperiment1runsonDPs1-6(Figure7(a)). itativelyweak,possiblymeaningless. SinceDumas’smatch- Thetrendisfairlylinear,withthesupervisedsystemslower ing component is able to compensate for generator noise, it for smaller inputs, but not larger inputs. The dependence isagoodempiricalcandidateforunsupervised1:1matching on|Q|showswhyaschemamatcherisnecessary,sinceinits on typical ER cases. Preliminary Experiment 1 also shows absence, the simple exhaustive set is input (Section 4.3). that on real-world ER problems, a simple measure like TF- As further validation of the theoretical run-time, Figure IDF is not appropriate (by itself) for solving ER. The gen- 7(b) shows the linear dependence of the proposed system erator noise provides an interesting test for the extended on |D|. Again, the trend is linear, but the slope32 depends DNF-BSLs. Since both the mappings-set Q and top n gen- on the schema heterogeneities of the individual DPs. For eratedduplicates(thesetD)outputbyDumasarepipedto example,DPs2and3,thelargestdatasetsintermsoffields the learner, there are two potential sources of noise. (Table 2), do not scale as well as the others. Finally,giventhattheproposedsystem(insubsequentex- Figure7(c)showsanimportantrobustnessresult. Weplot periments)permutes theDumas-outputD (Algorithm1)to the PC-RR f-score33 of the proposed system against |D|. generateN,wetestedtheaccuracyofthepermutationpro- Thefirstobservationisthat,evenforsmall|D|,performance cedureinafollow-upexperiment. With|D|rangingfrom50 isalreadyhighexceptonDP2,whichshowsasteepincrease to 10,000 in increments of 50, we generated N of equal size when |D| ≈ 100. On all cases, maximum performance is andcalculatednon-duplicatesaccuracy30 ofN overtenran- achieved at about |D| ≈ 700 and the f-score curves flatten domtrialspervalue. Inallcases,accuracywas100%,show- subsequently. Thisisqualitativelysimilartotherobustness ing that the permutation heuristic is actually quite strong. of Dumas to small numbers of positive samples. Figure7(c)alsoshowsthattheproposedmethodisrobust 7.2 ExtendedDNFBSL tooverestimatesof|D|. DP1,forexample,onlyhas100true Experiment1: Table4showsBSLresultsonallsixDPs. positives, but it continues to perform well at much higher Thehighoverallperformanceexplainstherecentpopularity |D| (albeit with a slight dip at |D|≈700) . ofDNF-BSLs. UsingtheextendedDNFhypothesisspacefor Discussion: We earlier stated, when discussing the pre- blocking schemes allows the learner to compensate for the liminary experimental results, that an extended DNF-BSL two sources of noise earlier discussed. Overall, considering canonlyintegratewellintothepipelineifitisrobusttonoise statistically significant results, the supervised method typi- fromprevioussteps. Previousresearchhasnotedtheoverall callyachievesbetterRR,butPCis(mostly)equallyhighfor robustness of DNF-BSLs. This led to the recent emergence allmethods,withtheproposedmethodperformingthebest of a homogeneous unsupervised system [16], adapted here onDP4(thelargestDP)andthesupervisedbaselineonDP as a semi-supervised baseline. Experiment 1 results showed 2, with high significance. We believe the former result was thatthisrobustnessalsocarriesovertoextended DNF-BSLs. obtainedbecausetheproposedmethodhasthestrongestap- Highoverallperformanceshowsthatthepipelinecanaccom- proximation bounds out of all three systems, and that this modate heterogeneity, a key goal of this paper. effect would be most apparent on large DPs. Importantly, Experiment 2 results demonstrate the advantage of hav- low standard deviation (often 0) is frequently observed for ing an expressive G, which is evidently more viable than all methods. The DNF-BSLs prove to be quite determinis- increasing k. On DPs 1 and 5 (that the systems succeeded tic,whichcanbeimportantwhenreplicatingresultsinboth research and industrial settings. Experiment 2: Next, we evaluated if k = 2 enhances 31Within a factor of 20 of the average time taken by the system for the k=1 experiment (for that DP). BSL performance and justifies the exponentially increased 32Theslopeisthehiddenconstantintheasympoticnotation. 301−|N ∩Ω |/|N| 332.RR.PC m RR+PC