Alexandre Tachard Passos “Combinatorial algorithms and linear programming for inference in ...

147 Pages·2013·2.49 MB·English

Alexandre Tachard Passos
“Combinatorial algorithms and linear programming
for inference in natural language processing.”
“Algoritmos combinat´orios e programa¸c˜ao linear
para inferˆencia em processamento de linguagem
natural.”
CAMPINAS
2013
i
ii

Ficha catalográfica
Universidade Estadual de Campinas
Biblioteca do Instituto de Matemática, Estatística e Computação Científica
Ana Regina Machado - CRB 8/5467

Passos, Alexandre Tachard, 1986-
P268c PasCombinatorial algorithms and linear programming for inference in natural
language processing / Alexandre Tachard Passos. – Campinas, SP : [s.n.], 2013.

PasOrientador: Jacques Wainer.
PasTese (doutorado) – Universidade Estadual de Campinas, Instituto de
Computação.

Pas1. Aprendizado de máquina. 2. Processamento da linguagem natural
(Computação). 3. Algoritmos. 4. Programação linear. 5. Análise combinatória. I.
Wainer, Jacques,1958-. II. Universidade Estadual de Campinas. Instituto de
Computação. III. Título.
Informações para Biblioteca Digital
Título em outro idioma: Algoritmos combinatórios e de programação linear para inferência em
processamento de linguagem natural
Palavras-chave em inglês:
Machine learning
Natural language processing (Computer science)
Algorithms
Linear programming
Combinatorial analysis
Área de concentração: Ciência da Computação
Titulação: Doutor em Ciência da Computação
Banca examinadora:
Jacques Wainer [Orientador]
Andrew Kachites McCallum
Sebastian Robert Riedel
Siome Klein Goldenstein
Eduardo Alves do Valle Júnior
Data de defesa: 28-08-2013
Programa de Pós-Graduação: Ciência da Computação
iv
Powered by TCPDF (www.tcpdf.org)

Institute of Computing /Instituto de Computa¸c˜ao
University of Campinas /Universidade Estadual de Campinas
Combinatorial algorithms and linear programming
for inference in natural language processing.
Alexandre Tachard Passos
August 28, 2013
Examiner Board/Banca Examinadora:
• Prof. Dr. Jacques Wainer (Supervisor)
• Prof. Dr. Siome Klein Goldenstein
Institute of Computing - UNICAMP
• Prof. Dr. Eduardo Valle
Institute of Computing - UNICAMP
• Dr. Andrew K McCallum
School of Computer Science - UMass Amherst
• Dr. Sebastian Riedel
Department of Computer Science - University College London
vii

Abstract
In natural language processing, and in general machine learning, probabilistic graphical
models (and more generally structured linear models) are commonly used. Although
these models are convenient, allowing the expression of complex relationships between
many random variables one wants to predict given a document or sentence, most learning
and prediction algorithms for general models are inefficient. Hence there has recently
been interest in using linear programming relaxations for the inference tasks necessary
when learning or applying these models.
This thesis presents two contributions to the theory and practice of linear program-
ming relaxations for inference in structured linear models. First we present a new al-
gorithm, based on column generation (a technique which is dual to the cutting planes
method) to accelerate the Viterbi algorithm, the most popular exact inference technique
for linear-chain graphical models. The method is also applicable to tree graphical models
and hypergraph models. Then we present a new linear programming relaxation for the
problem of joint inference, when one has many submodels and wants to predict using all
of them at once. In general joint inference is NP-complete, but algorithms based on dual
decomposition have proven to be efficiently applicable for the case when the joint model
can be expressed as many separate models plus linear equality constraints. This thesis
proposes an extension to dual decomposition which allows also the presence of factors
which score parts that belong in different submodels, improving the expressivity of dual
decomposition at no extra computational cost.
ix