Message passing in random satisfiability problems

0.1 MB·English

by  Marc Mezard

Message passing in random satisfiability
problems
4
0
0
2

n
MarcMe´zard
a
J CNRS,LaboratoiredePhysiqueThe´oriqueetMode`lesStatistiques
Universite´ParisSud
4
1 Baˆtiment100,91405OrsayCedex,France
[email protected]

]
n
n
- Abstract
s
i
d
. Thistalksurveystherecentdevelopmentofmessagepassingprocedures
t
a forsolvingconstraintsatisfactionproblems.Thecavitymethodfromsta-
m tisticalphysicsprovidesageneralizationofthebeliefpropagationstrat-
egythatisabletodealwiththeclusteringofsolutionsintheseproblems.
-
d Itallowstoderiveanalyticresultsontheirphasediagrams,andoffersa
n newalgorithmicframework.
o
c
[
1 Messagepassing

1
v ManyNIPSparticipantscanbeconfrontedtothefollowingtypeofgenericproblem[1]. A
7 systeminvolvesmany’simple’discretevariables,andeachvariablecantakeacertainnum-
3
berofdiscretevalues(whichisnottoolarge).Thesevariablesinteract,andtheinteractions
2
canbewrittenintheformofconstraints,whereeachconstraintinvolvesasmallfractionof
1
allthevariables. Onewantstofindthevaluesofthevariableswhicharecompatiblewith
0
allconstraints(or,whenthisisimpossible,findthevalueswhichminimizethenumberof
4
0 violatedconstraints).
/
t This’constraintsatisfactionproblem’(CSP) isaubiquitoussituationwhichshowsupfor
a
instanceinstatistical physics,combinatorialoptimization,errorcorrectingcodes, statisti-
m
cal inference,... It turns out that, in order to solve CSPs, these various disciplines have
- developed, often independently, some message passing procedures which turn out to be
d
remarkablypowerful.Thegeneralstrategyinvolvestwomainsteps:
n
o
• Organizetheconstraintsandvariablesinagraph
c
v: • Exchangemessages,ofprobabilisticnature,alongthegraph
i
X Weshallillustrateithereusingasanexamplethefamous’satisfiability’problem.
r
a
2 Satisfiability andcomplexity theory
The problem of satisfiability involvesN Boolean variablesx ∈{0,1}. There exist thus
i
2N possible configurations of these variables. The constraints take the special form of
’clauses’,whicharelogical’OR’functionsofthevariables. Forinstancetheclausex ∨
1
x ∨x¯ is satisfied whenever x = 1 or x = 1 or x = 0. Therefore, among the 8
27 3 1 27 3
possible configurations of x ,x ,x , the only one which is forbidden by this clause is
1 27 3
x = x = 0,x = 1. Aninstanceofthesatisfiabilityproblemisgivenbythelistofall
1 27 3
theclauses.
Satisfiabilityisadecisionproblem. Onewantstogiveayes/noanswertothequestion: is
thereachoiceoftheBooleanvariables(calledan’assignment’)suchthatallconstraintsare
satisfied (whenthereexistssuch a choicethe correspondinginstanceis said to be ’SAT’,
otherwiseitis’UNSAT’)?
Thisproblemappearsinmanyfieldsanditsimportanceisaconsequenceofitsgenericity:
aninstanceoftheSATproblemcanbeseenasalogicalformula(suchas(x ∨x ∨x¯ )∧
1 27 3
(x¯ ∨x )∧...),anditturnsoutthatanyformulacanbewritteninthisformofthe’and’
11 2
functionofseveralclauses,called’conjunctivenormalform’.
Satisfiability plays an essential role in the theoryof computationalcomplexity,becauseit
wasthe firstproblemwhichhasbeenshownto be’NP-complete’,in a beautifultheorem
byCookin1971[2]. TheNPproblemsarethosedecisionproblemswherea’yes’answer
canbecheckedinatimewhichispolynomialinN. Thisisavastclassofproblemswhich
containssuchdifficultproblemsasthetravelingsalesman,theproblemoffindingaHamil-
tonian path in a graph (a path which goes once through all vertices), the protein folding
problem,orthespinglassprobleminstatisticalphysics. ThefactthatsatisfiabilityisNP-
completemeansthatanyotherNPproblemcanbemappedtosatisfiability inpolynomial
time. Soifwehappenedtohaveapolynomialalgorithmforsatisfiability(i.e. analgorithm
thatsolvesthedecisionprobleminatimewhichgrowslikeapowerofN),alltheproblems
inNPwouldalsobesolvedinpolynomialtime. Thisisgenerallyconsideredunlikely,but
thecorrespondingmathematicalproblem(whethertheNPclassisdistinctornotfromthe
’P’classofproblemswhicharesolvableinpolynomialtime)isanimportantopenproblem.
The result of Cook is a worst case analysis of the satisfiability problem. However it
is known experimentally that many instances of this problem are easy to solve, and re-
searchershavestartedtostudysomeclassesofinstancesinordertocharacterizethe’typ-
icalcase’complexityofsatisfiabilityproblems. Aclassofinstancesisaprobabilitymea-
sureonthe spaceofinstances. Amuchstudiedprobleminthisframeworkistherandom
’3-SAT’problem. Eachclausecontainsexactlythreevariables;theyarechosenrandomly
in{x ,..,x }withuniformmeasure,andeachvariableisnegatedrandomlywithproba-
1 N
bility 1/2. This problem is particularlyinteresting because its difficulty can be tuned by
varying one single control parameter, the ratio α = M of constraints per variable. One
N
expectsintuitivelythatforsmallαmostinstancesareSAT,whileforlargeαmostofthem
are UNSAT.Numericalexperimentshaveconfirmedthis scenario,buttheyindicateactu-
allya moreinterestingbehavior. TheprobabilitythataninstanceisSAT exhibitsa sharp
crossover,fromavaluecloseto1toavaluecloseto0,atathresholdα whichisaround
c
4.3.WhenthenumberofvariablesN increases,thecrossoverbecomessharperandsharper
[3,4]. IthasbeenshownthatitbecomesastaircasebehavioratlargeN [5]: almostallin-
stancesareSATforα<α ,almostallinstancesareUNSATforα>α ,andsomebounds
c c
onthevalueofα havebeenderived[6,7]. Thisthresholdbehaviorisnothingbutaphase
c
transition as one finds in physics, and has been analyzed using the methodsof statistical
physics[3,8].
A veryinterestingobservationis thatthe algorithmicdifficultyofthe problem,measured
bythetimetakenbythealgorithmtoanswerifatypicalinstanceissatisfiable,alsodepends
stronglyonα: thetypicalproblemiseasywhenαiswellbeloworwellaboveα ,andis
c
much harder when α is close to α . Therefore the region of phase transition is also the
c
regionwhichisinterestingfromthecomputationalpointofview.
3 Statistical physicsofthe random 3-SAT problem
In the last two decades, it has been realized that some concepts and methods developed
instatisticalphysicscanbeusefultostudyoptimizationproblems[9]. Theinterestinthis
type of approachhas been focusingin recentyears on the randomsatisfiability problem,
following the works of colleagues like Kirkpatrick, Monasson and Zecchina [8, 10, 11].
This is indeed a choice problem for a multidisciplinary study, because the SAT-UNSAT
transition is a phase transition in the usual statistical physics sense, and the algorithmic
complexityoftheproblemshowsuppreciselyintheneighborhoodofthisphasetransition.
Ontheanalyticalside,thefirstbreakthroughusedthereplicamethodandcomputedsome
approximations of the phase diagram using either a ’replica symmetric’ approximation,
or some variational approximation to the replica symmetry breaking solution. Hereafter
I will mainlyreview the mostrecentdevelopmentsinitiated in [12, 13]. These have pro-
videdsomeanalyticalinsightintothephasediagramoftheproblem,whichturnsouttobe
morecomplicatedthanoriginallythought,aswellasanewalgorithmicstrategybasedon
messagepassingforthecaseofrandom3-satisfiability.
ThemainanalyticalresultonthephasediagramissummarizedinFig.1. Whenonevaries
thecontrolparameterα(thenumberofconstraintspervariable),thereactuallyexistthree
distinctphases,separatedbytwothresholdsα andα . Thethresholdatα =4.267isthe
d c c
satisfiabilitythresholdwhichseparatestheSATphasesatα < α fromtheUNSATphase
c
atα > α . ButintheSATregion,thereactuallyexisttwodistinctphases. Theydifferby
c
thestructureofthespaceofSATassignments(the’SATspace’). Forα < α ≃ 3.86,the
d
setofSATassignmentsbuildsoneclusterwhichisbasicallyconnected[11,14]. Starting
from a genericSAT assignment(which is a pointon the unitN-dimensionalhypercube),
one can step to another one nearby by flipping a finite number of variables. It turns out
that one can get from any generic SAT assignment to any other one through a sequence
of such steps, always staying in the SAT space. On the other hand, for α > α , this
d
SAT space becomesdisconnected: the SAT assignments are groupedinto many clusters.
Itisimpossibletogetfromoneclustertoanotheronewithouthavingtoflipatsomemo-
mentamacroscopicfractionofthevariables. Simultaneously,thespaceofconfigurations
develops many ’metastable states’: imagine walking along the configuration space (the
N-dimensional hypercube), flipping one variable at a time, but allowing only the moves
whichdecrease,orleaveconstant,thenumberofviolatedclauses. Suchanalgorithmwill
gettrappedintosome’metastableclusters’,whichareconnectedclustersofassignments,
allviolatingthesame (non-zero)numberofclauses. Thisstructurehassomedirectalgo-
rithmicimplication: theseparationofclustersinthephaseα > α makesitverydifficult
d
foralgorithmsbasedonlocalmovestofindaSATassignment. Thephaseα < α isthus
d
calledthe’Easy-SAT’phase,whiletheonebetweenα andα isthe’Hard-SAT’phase.
d c
These results have been obtained using the cavity method. Originally this method was
invented in the study of spin glass theory, in order to solve the Sherrington Kirkpatrick
model [15]. It is only recently that new developments in this method allowed to solve
’finiteconnectivity’problemswhereeachvariableinteractswith asmallnumberofother
variables[16]. Thankstothesedevelopment,ithasbecomepossibletostudyconstraintsat-
isfactionproblems. Althoughthemethodisnotfullyrigorous,theself-consistencyofthe
mainunderlyinghypothesishasbeenchecked[17, 18, 19], andthereforetheresultscon-
cerningthesatisfiabilitythreshold,aswellascrucialfeaturesofthephasediagramlikethe
existenceoftheintermediateHard-SATphase,areconjecturedtobeexactresults,notap-
proximations.Ofcourse,itisveryimportanttodeveloprigorousmathematicalapproaches
inordertoconfirmorinfirmthem.
It turns out that the cavity method when applied to one given instance of the problem
amountstoamessagepassingprocedurewhichcanalsobeusedasanalgorithm.Therefore
onesurprisingoffspringofthetheoreticaldevelopmentofthecavitymethodinrecentyears,
SAT (E = 0 ) UNSAT (E >0)
0 0
1 state Many states Many states
E=0 E>0 E>0
α α = 4.267 α =M/N
d c
Figure1:Thephasediagramoftherandom3-SATproblem:whenoneincreasestheratioα
ofconstraintstovariables,onefindssuccessivelyanEasy-SATphase,aHard-SATphase,
andanUNSATphase
whichaimedatansweringveryfundamentalquestionsconcerningthephasediagram,has
been the finding of a new class of algorithms, which turn out to be very efficient in the
Easy-SATphase.
Thenextsectionsaim atdescribingthemainideasofthecavitymethod,takingthepoint
ofviewofthemessagepassingalgorithm. Itisimpossibletoexplainallthedetails,orto
givethejustification,forallthesesteps. Theinterestedreaderisreferredtorefs[16]forthe
basic conceptsofthe cavity methodin finite connectivitydisorderedsystems, and to refs
[12,13,20,18]forthesolutionoftherandomsatisfiabilityproblem.
4 Factorgraphs and warningpropagation
Thefirststepconsistsinagraphicalrepresentationofthesatisfiabilityproblem.Eachclause
isrepresentedbya functionnode, connectedto thevariousvariableswhichappearin the
clause,asdescribedinFig.2.
Factorgraphsturnouttobeveryusefulinvariouscontexts,includingstatisticalinference
[21]anderrorcorrectingcodes[22]. Thegeneralformalismisdescribedin[23].
Inarandom3-SATproblemwithN variablesandM = αN clauses,inthelargeN limit,
itiseasytoseethatthefactorgraphisarandombipartitegraph,wherethefunctionnodes
have a fixed connectivity equal to 3, and the variable nodes have a Poisson distributed
connectivitywithamean3α.
Thesimplestcaseofmessagepassingisthewarningpropagation. Alongalltheedgesof
thegraph,onepassesmessages. Ifclauseainvolvesvariablei,themessageu ∈{0,1}
a→i
sent from clause a to variable i is a way for clause a to inform variable i: A warning
u = 1 means: “Accordingto the messagesI havereceived(fromtheothervariables
a→i
whichareconnectedtome),youshouldtakethevaluewhichsatisfiesme!”.Whenu =
a→i
0,thismeansthatnowarningispassed,whichmeansamessagesaying:“Accordingtothe
messagesIhavereceived,thereisnoproblem,youcantakeanyvalue!”
Itcanbeshownthatwarningpropagationconvergesandgivesthecorrectanswerinasat-
isfiability problem described by a tree factor graph (such a problem can be nontrivial if
c
4
2
a
1 2 b
d
a 1
5
e
3 3
Figure2: Factorgraphrepresentationof satisfiability: A variableis representedby a cir-
cle. A clause is represented by a square, connected with a full (resp. dashed) line to a
variablewhenthisvariableappearsassuch(resp. negated)in theclause. Lefthandside:
The clause x¯ ∨ x ∨ x¯ . Right hand side: the factor graph representing the formula:
1 2 3
(x¯ ∨x¯ ∨x¯ )∧(x ∨x¯ )∧(x ∨x ∨x )∧(x ∨x ∨x¯ )∧(x ∨x¯ ∨x )
1 2 4 1 2 2 4 5 1 2 5 1 3 5
there are, among the clauses, some ’unit clauses’, involving a single variable, and forc-
ing this variable): the problemis SAT if and onlyif, on each variablenode, there are no
contradictorymessages.
Thissimplewarningpropagationcorrespondstosomelimitingcaseofthecelebratedbelief
propagation(“BP”)algorithm.ThebasicideaofBPistostudytheprobabilityspaceofall
SATassignmentswithuniformmeasure. ConsideragaintheclauseinFig.3. Imagineone
knowsthejoint’cavity’probabilityP(a)(x ,x )forthevariablesx ,x whentheclause
2 3 2 3
aisabsent.Thenonecandeducethefollowingestimatefromaconcerningthestateof1:
Pa→1(x1)= X Ca(x1,x2,x3)P(a)(x2,x3), (1)
x2,x3
where C is the indicator function of clause a, equal to one if and only if the clause is
a
satisfied. Theprobabilitydistributionofx intheabsenceofaclausebisgivenby:
1
P(b)(x1)=C Y Pa→1(x1) (2)
a∈V(1)\b
whereC isanormalizationconstant.
These exact equations are useless as such, but they become very useful if one adds the
hypothesisthat
P(a)(x ,x )∼P(a)(x )P(a)(x ). (3)
2 3 2 3
Then eqs (1,2) close onto a set of self-consistent equations for the cavity probabilities
P(a)(x). These equations can be interpreted as a message passing procedure which is
the BPalgorithm; theyalso carrythe nameofBethe approximationin statistical physics.
ThewarningpropagationequationsaresomeversionofBPinwhichonefocusesontothe
variables which are forced to take one given value in all the SAT configurations (in the
statisticalphysicsapproachtheycorrespondtowritingtheBetheequationsdirectlyatzero
temperature).Theonsetofclusteringatthiszerotemperaturelevelsignalstheexistenceof
clustersinwhichagivenvariableisfrozen(ittakesthesamevalueinallconfigurationsof
thecluster):ittakesplaceatavalueα∼3.91.
0 0 1 0 0 0
1 0 1 0
0 0 0 1
0 0
1 0
2 3 2 3
1 1 1 0
a a
u = 1 u = 0
a 1 a 1
1 1
Figure3: Two examplesof warningpropagationin a clause a. In orderto determinethe
messagewhichitpassestovariable1,clauseaconsidersallthemessagesreceivedbythe
othervariablestowhichitisconnected,herethevariables2and3. Onthelefthandside,
themessagesreceivedby2tellittotakethevaluex = 1, whichdoesnotsatisfy clause
2
a, andthemessagesreceivedby3tellittotakethevaluex = 0,whichdoesnotsatisfy
3
clausea. Therefore,inordertobesatisfied,clauseamustrelyonvariable1. Itthussends
a warning u = 1. Right hand side: an example where no warning is sent, because
a→1
variable3istoldbyitsenvironmenttotakethevaluex =1,whichsatisfiesclausea.
3
Themain questionis whetherthe factorizationapproximation(3)is correct. Ifthe factor
graphisatree, itisobviouslycorrect. Inthecaseofa random3-SAT,orrandomK-SAT
problem, one may notice that generically the factor graph is locally tree-like: the loops
appearonlyatlargedistances(oforderlogN). Therefore,intheabsenceofclausea,the
twovariablesx ,x arefarapart. Insuchasituationonemayhopethatthefactorization
2 3
willhold,inthelargeN limit. Howeverthisistrueonlyinthecasewherethereisasingle
purestateinthesystem(i.e. asingleclusterofsolutions). Ifthereareseveralpurestates,
it is well knownin statistical physicsthat the correlationsof the variables decay at large
distancesonlywhentheprobabilitymeasureisrestrictedtoonepurestate.
5 Proliferationofstates: survey propagation
Thereforeone can expectsthe BP, or the warningpropagation,to be correctin the Easy-
SATphase. IntheHard-SATphasewherepurestatesproliferate,theBPwouldbecorrect
if we had a way to restrict the measure to the SAT configurations in one given cluster.
Howeverthereisnowaytoachievethisglobally.BPisalocalmessagepassingprocedure.
Locally it will tend to find equilibrated configurations corresponding to one cluster, but
thereisnowaytoselectthesameclusterindistantpartsofthefactorgraph.
In order to handle such a situation the cavity method introduces generalized messages:
Along each edge a−i, a message is sentfrom clause a to variablei. Thismessage is a
surveyoftheelementarymessagesinthevariousclustersofSATconfigurations. Because
warningsaresosimple(thisiswhereitisusefultousewarningsratherthanstandardBP),
thesurveyischaracterizedbyasinglerealnumberη ,whichgivestheprobabilityofa
a→i
warningbeingsentfromconstraintatovariablei,whenaclusterispickedupatrandom
inthesetofallclustersofSATassignments.
Thesurveypropagation(SP) equationsareeasily written. Lookingagainattheclause of
Fig.3, let us denote by U the set of function nodes, distinct from a, connected through
a full line to variable x , and V the set connected through a dashed line. Introduce the
2
probabilitiesthatno warningsare sentfromU and V, givenby π2 = (1−η )
+ Qb∈U b→2
andπ2 = (1−η ).Similarquantitiesπ3 areintroducedforthevariablex .Then
− Qb∈V b→2 ± 3
theSPequationsread:
π2(1−π2) π3(1−π3)
η = − + + − (4)
a→1 π2 +π2 −π2π2 π3 +π3 −π3π3
+ − + − + − + −
Itturnsoutthatthe propagationof these surveysalongthe graphconvergesfora generic
probleminthewholeHard-SATphase.
These SP equations can be used in two ways. When iterated on one given sample, they
provide very interesting information concerning each variable: From the set of surveys,
one can compute for instance the probability that one variable is constrained to 0, when
one choosesa SAT cluster randomly. The surveyinspired decimationalgorithm[12, 13]
uses this information as follows: it identifies the most biased variable (the one which is
polarized to 1, or to 0, with the largest probability), and fixes it. Then the problem is
reducedto a new satisfiability problemwith one variableless. One can runagainthe SP
algorithmonthissmallerproblemanditeratetheprocedure.ThissimplestrategyfindsSAT
assignmentintheregionα<4.252(quiteclosetotheUNSATthresholdα =4.267),ina
c
timeoforderNlogN whichallowstoreachlargesamplesizesofafewmillionvariables.
Somebacktrackinginthewayoneperformsthedecimation[24]allowstogetclosertothe
threshold,butuntilnowitisnotknownifthesesurveybasedalgorithmswillallowtosolve
thealgorithmicprobleminthewholeHard-SATphaseornot.
OntheotherhandonecanalsousetheSPequationsinordertofindanalyticalresults. The
ideaistoperformastatisticalanalysisoftheSPequations: oneintroducestheprobability
densityP(η),whenonepicksupanedgea−irandomlyinalargerandom3-SATproblem,
that the survey η takes a given value η. The SP equations (4) allow to write a self-
a→i
consistentnon-linearintegralequationforP(η). Solvingtheseequations,onecandeduce
allthefeaturesofthephasediagramofFig.1. ThethresholdsfoundforrandomK-sat,for
K =3,4,5,6,10,arerespectivelyequalto4.267, 9.931, 21.12, 43.37, 708.9(see[18]).
TheequationscanalsobegeneralizedtostudytheUNSATphase,inwhichonecandeter-
mine the minimal numberof violated clauses, both analytically, and also algorithmically
ononegivensample,butthisgoesbeyondthepresentpaper.
Thisgeneralcavitymethodstrategycanalsobeusedinotherconstraintsatisfactionprob-
lems [25], and has foundnon-trivialresults for instance on the coloring problem[26]. A
particularlyinterestingcaseistheXOR-SATproblem[27],wherethesametypeofphase
diagramis found,and the cavity methodresultcan be checkedversusrigorouscomputa-
tions. It is also worth mentioning that for K even, the SAT-UNSAT threshold α found
c
withthecavitymethodhasbeenshowntobeanupperboundtothecorrectresult[28]. Our
conjectureisthatthisboundisactuallytight.
To summarize, we have considered sparse network of many interacting elements, with
interactions taking the form of constraints on their relative values. In this situation, one
often finds a “Hard-SAT” phase, with an exponential number of well separated solution
clusters, togetherwith an exponentiallylargernumberof’metastable states’. Itturnsout
that the local exchange of probabilistic messages between the elements provides a very
detailed statistical informationon the varioussolutions. One can expectthat this general
frameworkwillalsofindapplicationsinneuralnetworkstudies.
Acknowledgments
The results presented here have been obtained in various collaborations, mainly with G.
ParisiandR.Zecchina,andalsowithA.Braunstein,S.Mertens,M.Weigt.
References
[1] M.Me´zard,Science301(2003)1685.
[2] S.Cook,Proc.3rdAnn.ACMSymp.onTheoryofComputing,Assoc.Comput.Mach.,
NewYork,1971,p.151.
[3] S.Kirkpatrick,B.Selman,Science264,1297(1994).
[4] J.A.CrawfordandL.D.Auton,Artif.Intell.81,31-57(1996).
[5] E.Friedgut,J.Amer.Math.Soc.12(1999)1017.
[6] O. Dubois, Y. Boufkhad and J. Mandler, Proc. 11th ACM-SIAM Symp. on Discrete
Algorithms,124(SanFrancisco,CA,2000).
[7] D. Achlioptas and C. Moore, http://arXiv.org/abs/cond-mat/0310227; D. Achlioptas
andY.Peres,http://arXiv.org/abs/cs.CC/0305009.
[8] R.Monasson,R.Zecchina,S.Kirkpatrick,B.SelmanandL.Troyansky,Nature(Lon-
don)400,133(1999).
[9] O.Dubois,R.Monasson,B.SelmanandR.Zecchina(Eds.),Theoret.Comp.Sci.265
(2001).
[10] R.MonassonandR.Zecchina, Phys.Rev. E561357–1361(1997).
[11] G.Biroli,R.MonassonandM.Weigt, Euro.Phys.J. B14551(2000).
[12] M.Me´zard,G.ParisiandR.Zecchina,Science297,812(2002)(Sciencexpresspub-
lishedon-line27-June-2002;10.1126/science.1073287).
[13] M.Me´zardandR.Zecchina,Phys.Rev.E66(2002)056126.
[14] G.Parisi,cs.CC0301015.
[15] M.Me´zard,G.ParisiandM.A.Virasoro,SpinGlassTheoryandBeyond,WorldSci-
entific,Singapore(1987).
[16] M. Me´zard, and G. Parisi, Eur.Phys. J. B 20, 217–233 (2001). M. Me´zard and G.
Parisi,J.Stat.Phys.111(2003)1.
[17] A.MontanariandF.Ricci-Tersenghi,Eur.Phys.J.B33(2003)339.
[18] S.Mertens,M.Me´zardandR.Zecchina,http://arXiv.org/abs/cs.CC/0309020.
[19] A. Montanari, G. Parisi and F. Ricci-Tersenghi,
http://arXiv.org/abs/cond-mat/0308147.
[20] A.Braunstein,M.Me´zard,R.Zecchina,http://fr.arXiv.org/abs/cs.CC/0212002.
[21] J.Pearl,ProbabilisticReasoninginIntelligentSystems,2nded.(SanFrancisco,Mor-
ganKaufmann,1988).
[22] Specialissue onCodes, GraphsandIterativeAlgorithms,IEEE Trans.Info.Theory
vol.47,no2(2001).
[23] F.R.Kschischang,B.J.FreyandH.-A.Loeliger,IEEETrans.Info.Theory47(2001)
498.
[24] G.Parisi,cond-mat/0308510.
[25] A.Braunstein,M.Me´zard,M.WeigtandR.Zecchina,cond-mat/0212451.
[26] R.Mulet,A.Pagnani,M.Weigt,R.Zecchina,Phys.Rev.Lett.89(2002)268701.
[27] M.Me´zard,F.Ricci-Tersenghi,R.Zecchina,J.Stat.Phys111(2003)505.S.Cocco,
O.Dubois,J.Mandler,R.Monasson,http://arXiv.org/abs/cond-mat/0206239.
[28] S.FranzandM.Leone,http://arXiv.org/abs/cond-mat/0208280.