Entropy, search, complexity

264 Pages·2007·1.105 MB·English

by  Csiszár, Imre, Katona, G., Tardos, Gábor (eds.)

BOLYAI SOCIETY 16
MATHEMATICAL STUDIES
BOLYAI SOCIETY MATHEMATICAL STUDIES
Series Editor: Managing Editor:
Gábor Fejes Tóth Dezso˝ Miklós
Publication Board:
László Babai
István Juhász
Gyula O. H. Katona
László Lovász
Domokos Szász
Vilmos Totik
1. Combinatorics, Paul Erd(cid:173)s is Eighty, Vol. 1
D. Miklós, V.T. Sós, T. Sz(cid:173)nyi (Eds.)
2. Combinatorics, Paul Erd(cid:173)s is Eighty, Vol. 2
D. Miklós, V.T. Sós, T. Sz(cid:173)nyi (Eds.)
3. Extremal Problems for Finite Sets
P. Frankl, Z. Füredi, G. Katona, D. Miklós (Eds.)
4. Topology with Applications
A. Császár (Ed.)
5. Approximation Theory and Function Series
P. Vértesi, L. Leindler, Sz. Révész, J. Szabados, V. Totik (Eds.)
6. Intuitive Geometry
I. Bárány, K. Böröczky (Eds.)
7. Graph Theory and Combinatorial Biology
L. Lovász, A. Gyárfás, G. Katona, A. Recski (Eds.)
8. Low Dimensional Topology
K. Böröczky, Jr., W. Neumann, A. Stipsicz (Eds.)
9. Random Walks
P. Révész, B. Tóth (Eds.)
10. Contemporary Combinatorics
B. Bollobás (Ed.)
11. Paul Erd(cid:173)s and His Mathematics I+II
G. Halász, L. Lovász, M. Simonovits, V. T. Sós (Eds.)
12. Higher Dimensional Varieties and Rational Points
K. Böröczky, Jr., J. Kollár, T. Szamuely (Eds.)
13. Surgery on Contact 3-Manifolds and Stein Surfaces
B. Ozbagci, A. I. Stipsicz
14. A Panorama of Hungarian Mathematics in the Twentieth Century, Vol. 1
J. Horváth (Ed.)
15. More Sets, Graphs and Numbers
E. Gy(cid:173)ri, G. Katona, L. Lovász (Eds.)
16. Entropy, Search, Complexity
I. Csiszár, G. Katona, G. Tardos (Eds.)
Imre Csiszár
Gyula O. H. Katona
Gábor Tardos (Eds.)
Entropy, Search,
Complexity
(cid:0)(cid:2) (cid:3)(cid:4)
JÁNOS BOLYAI MATHEMATICAL SOCIETY
Imre Csiszár Managing Editor
Hungarian Academy of Sciences, Gábor Wiener
Alfréd Rényi Institute of Mathematics
Budapest University of Technology
Reáltanoda u. 13–15
and Economics
1053 Budapest, Hungary
Pázmány Péter sétány 1/D
E-mail: [e-mail]
1117 Budapest, Hungary
E-mail: [e-mail]
Gyula O. H. Katona
Hungarian Academy of Sciences,
Alfréd Rényi Institute of Mathematics
Reáltanoda u. 13–15
1053 Budapest, Hungary
E-mail: [e-mail]
Gábor Tardos
Hungarian Academy of Sciences,
Alfréd Rényi Institute of Mathematics
Reáltanoda u. 13–15
1053 Budapest, Hungary
E-mail: [e-mail]
Mathematics Subject Classification (2000): 94A15, 90B40, 68P10
Library of Congress Control Number: 2006938672
ISSN 1217-4696
ISBN 978-3-540-32573-4 Springer Berlin Heidelberg New York
ISBN 978-963-9453-06-7 János Bolyai Mathematical Society, Budapest
This work is subject to copyright. Allrights are reserved, whether the whole or part of the material
is concerned, specifically therights of translation, reprinting, reuse of illustrations, recitation, broad-
casting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of
this publication or parts thereof is permitted only under the provisions of the German Copyright
Law of September 9, 1965, in its current version, and permission for use must always be obtained
from Springer. Violations are liable for prosecution under the German Copyright Law.
Springer is a part of Springer Science+Business Media
springer.com
© 2007 János Bolyai Mathematical Society and Springer-Verlag
Printed in Hungary
The use of general descriptive names, registered names, trademarks, etc. in this publication does not
imply, even in the absence of a specific statement, that such names are exempt from the relevant pro-
tective laws and regulations and therefore free for general use.
Cover design: Erich Kirchner, Heidelberg
Printed on acid-free paper 44/3142/db – 5 4 3 2 1 0
Contents
Contents .......................................................... 5
Preface ............................................................ 7
M. Aigner: Two Colors and More .................................. 9
C. Deppe: Coding with Feedback and Searching with Lies ........... 27
A.G.D’yachkov,A.J.MaculaandP.A.Vilenkin:Nonadap-
tive and Trivial Two-Stage Group Testing with Error-Correcting
de-Disjunct Inclusion Matrices ..................................... 71
S. Ghosh, T. Shirakura and J. N. Srivastava: Model Identifi-
cation Using Search Linear Models and Search Designs ............ 85
P. Harremoe¨s: Information Topologies with Applications ...........113
M. Keane: Reinforced Random Walk ............................... 151
D. Petz: Quantum Source Coding and Data Compression ........... 159
F. Topsøe: Information Theory at the Service of Science ............ 179
P. Vita´nyi: Analysis of Sorting Algorithms by Kolmogorov Com-
plexity ............................................................ 209
G. Wiener: Recognition Problems in Combinatorial Search ......... 233

Preface
The present volume is a collection of survey papers in the fields given in
the title. They summarize the latest developments in their respective areas.
More than half of the papers belong to search theory which lies on the
borderline of mathematics and computer science, information theory and
combinatorics, respectively. The volume is slightly related to the twin
conferences “Search And Communication Complexity” and “Information
Theory In Mathematics” held at Balatonlelle, Hungary in 2000. These
conferences led us to believe that there is a need for such a collection of
papers.
The paper written by Martin Aigner starts with the following relatively
new search problem. Given n boolean variables as input one has to find one
of them whose value is in majority. The goal is to minimize the number
of tests needed for this where one test is to compare two input variables
for equality. The paper surveys the large set of problems and results which
grew out of this one.
In the traditional search model an unknown element is sought in a finite
set, based on the information that the unknown element is or is not in
some (asked) subsets. A variant is when a 0,1 function is given on the
underlying set, and only the values of this function at the unknown element
x is sought rather than x itself. This is called the recognition problem.
Ga´bor Wiener’s paper shows that the recognition problem actually includes
the problem of two-party, deterministic communication complexity. Using
this novel observation it unifies and surveys results in both theories.
The theory of search with lies, or the Ulam–R´enyi game is an exciting
areawithmanyapplications. ThepaperofChristianDeppegivesacomplete
survey on search problems obtained by allowing lies, that is, when the
answer on the question “is the unknown element x in the subset A?” can
be wrong, but the number of lies is limited.
In linear statistics the influence of certain statistical parameters (fac-
tors) and their combinations are to be determined. Traditionally it was
supposed that one knows beforehand a few important variables and we as-
sume that any combination of the remaining variables have negligible influ-
ence. A 20 year old theory does not make this assumption. Instead, the
8
Preface
experiments have two simultaneous goals: 1. determine which combinations
have a non-negligible influence, and 2. find the influence of these combina-
tions. This is why this setting is a generalization of the search problems.
The paper of S. Ghosh, T. Shirakura and J. N. Srivastava gives a strong
survey of the results in this theory. Let us mention the theory was founded
by the last of these authors.
The basic problem of sorting is to find the natural order of a set of
integers by pairwise comparisons. It is easy to see that this also fits in
the search model: the underlying set is the set of all permutations, the
unknownelementistheactualpermutationdeterminedbythenaturalorder.
A relatively new development of the theory that Kolmogorov’s complexity
can be used in proving bounds in sorting problems. This new theory is
surveyed here in the paper of Paul Vita´nyi.
ThepaperofD’yachkov,MaculaandVilenkincontainssomenewresults
in the area of non-adaptive search with more than one unknowns. However
it adds an extensive literature in the given area which helps the reader to
obtain a good view.
Flemming Topsøe surveys the situations where information theory can
be used. It has a novel attitude: the situations are treated as problems of
games. 60 references help the reader to study the details.
D´enes Petz gives a survey of the very modern area of Quantum Source
Coding.
The work written by Michael Keane is not a survey. It poses “only” an
exciting new problem that is both very natural and easy to formulate. On
the other hand the paper makes it clear that it is actually a starting point
of a class of difficult problems. The conclusion is a brief description of the
known results.
PeterHarremo¨es’spaperintroducesnewtopologiesonprobabilitydistri-
butions, that is, information theoretical divergencies. Since they are com-
pared with the traditional divergencies (entropies), the paper contains a
good survey of these information theoretical concepts. The importance of
the new concepts are justified by theorems, too.
The editors
BOLYAISOCIETY Entropy, Search, Complexity, pp. 9–26.
MATHEMATICALSTUDIES,16
Two Colors and More
MARTIN AIGNER
Suppose we are given n balls colored with two colors. How many color-compar-
isonsareneededtoproduceaballofthemajoritycolor? Theanswer(firstgiven
bySaksandWerman)isM(n)=n−B(n),whereB(n)isthenumberof1’sinthe
binaryrepresentationofn. Weconsiderinthispaperseveralgeneralizationsand
variantsofthemajorityproblemsuchasproducingak-majorityball,determining
the color status of all balls, arbitrarily many colors, the plurality problem, and
the closely related liar problem.
1. The majority problem
Suppose we are given n balls colored with two colors, and two players Paul
and Carole playing the following game. At any stage of the game Paul
chooses two balls x and y and asks whether they are of the same color,
whereupon Carole answers “yes” or “no”. The game ends when Paul either
produces a ball z of the majority color (meaning that the number of balls
colored like z exceeds the other color), or when Paul states that there is no
majority. Of course, the latter case can only occur when n is even. How
many questions L(n) does Paul have to ask in the worst case?
This problem was first solved by Saks and Werman [11] and later by
Alonso, Reingold, Schott [4] and Wiener [12] using different methods. The
answer is
(1) L(n) = n−B(n),
where B(n) is the number of 1’s in the binary representation of n. Alonso,
Reingold and Schott [4] also gave the solution for the average case.
10
M. Aigner
As a warm-up let us see how Paul finds an algorithm that uses no more
than n−B(n) questions. The data structure during the game is a list of
buckets B ,...,B and a dump D
1 s
2a1 2a2 ... 2as
B B B D
1 2 s
where the balls in each bucket are colored alike, always numbering a power
of 2. Thus, initially, there are n buckets each containing one ball with the
dumpempty. ForthenexttestPaulchoosestwobucketsB ,B witha = a
i j i j
and compares balls from B and B . If the answer is “yes”, he merges the
i j
buckets into one (of new size 2ai+1), otherwise he empties both buckets into
the dump. Hence D contains at any stage an equal number of either color.
The algorithm stops when either all buckets have different sizes 2b1 >
2b2 > ··· > 2bt, or when all balls are in the dump. In the first case the size
2b1 of the largest bucket exceeds 2b2 +···+2bt, and we conclude that B
1
contains the majority color balls. In the other case there is no majority.
Hence with either alternative the game is finished.
It remains to compute the number L of questions. By induction it is
clear that Paul needs 2bi − 1 questions to produce a bucket of size 2bi.
Similarly, when he throws two buckets of equal size 2ci−1 into D then he
has asked 2ci −1 questions. Hence
L ≤ (2b1 +···+2bt −t)+(2c1 +···+2cr −r)
where
n = 2b1 +···+2bt +2c1 +···+2cr.
Since obviously t+r ≥ B(n), we obtain
L ≤ n−B(n).
It is the aim of this paper to present a survey of several natural general-
izations and variants of the majority problem, including a number of open
questions. Only a few proofs will be given in full detail, the emphasis being
on the common ideas for this appealing part of combinatorial search. For
the general background the reader may consult the books by Aigner [1] or
Du–Hwang [7].