Norms and Value Based Reasoning: Justifying Compliance and Violation

33 Pages·2017·0.28 MB·English

ArtificialIntelligencemanuscriptNo.
(willbeinsertedbytheeditor)
Norms and Value Based Reasoning: Justifying Compliance
and Violation
TrevorBench-Capon · SanjayModgil
February11,2017
Abstract There is an increasing need for norms to be embedded in technology as the
widespread deployment of applications such as autonomous driving, warfare and big data
analysisforcrimefightingandcounter-terrorismbecomesevercloser.Currentapproaches
tonormsinmulti-agentsystemstendeithertosimplymakeprohibitedactionsunavailable,
ortoprovideasetofrules(principles)whichtheagentisobligedtofollow,eitheraspart
ofitsdesignortoavoidsanctionsandpunishments.Inthispaper1wearguefortheposition
thatagentsshouldbeequippedwiththeabilitytoreasonaboutasystem’snorms,byreason-
ingaboutthesocialandmoralvaluesthatnormsaredesignedtoserve;thatis,performthe
sortofmoralreasoningweexpectofhumans.Inparticularwehighlighttheneedforsuch
reasoning when circumstances are such that the rules should arguably be broken, so that
thereasoningcanguideagentsindecidingwhethertocomplywiththenormsand,ifviola-
tionisdesirable,howbesttoviolatethem.Oneapproachtoenablingthisistomakeuseof
anargumentationschemebasedonvaluesanddesignedforpracticalreasoning:arguments
forandagainstactionsaregeneratedusingthisschemeandagentschoosebetweenactions
basedontheirpreferencesoverthesevalues.Moralreasoningthenrequiresthatagentshave
anacceptablesetofvaluesandanacceptableorderingontheirvalues.Wefirstdiscusshow
thisapproachcanbeusedtothinkaboutandjustifynormsingeneral,andthendiscusshow
thisreasoningcanbeusedtothinkaboutwhennormsshouldbeviolated,andtheformthis
violationshouldtake.Weillustratehowvaluebasedreasoningcanbeusedtodecidewhen
andhowtoviolateanormusingaroadtrafficexample.Wealsobrieflyconsiderwhatmakes
anorderingonvaluesacceptable,andhowsuchanorderingmightbedetermined.
1 Introduction
Normsareatopicofconsiderableinterestinagentsystems(WalkerandWooldridge(1995),
ShohamandTennenholtz(1997),WooldridgeandvanderHoek(2005),Moor(2006),vander
TrevorBench-Capon
DepartmentofComputerScience.UniversityofLiverpool.E-mail:[email protected]
SanjayModgil
DepartmentofInformatics.King’sCollege,London.
1 Thispaperisarevised,extendedandconsolidatedversionofseveralpreviouspapers:Bench-Capon
(2016a),Bench-Capon(2016b),Bench-CaponandModgil(2016a)andBench-CaponandModgil(2016b).
2 TrevorBench-Capon,SanjayModgil
Hoeketal.(2007),SenandAiriau(2007),Savarimuthuetal.(2008),ÅgotnesandWooldridge
(2010),Sugawara(2011),Mahmoudetal.(2015)).Anearlypositionpaper,Moor(2006),
set out to argue for the importance (and difficulty) of making machines behave ethically.
Moordistinguishedbetweenimplicitethicalsystems(nowusuallytermedregimented sys-
tems)andexplicitethicalsystems,basedonrulesandprinciples,andoftenemployingde-
onticlogics.Thisdistinctionstillappliestomostapproachestoethicalornormativeagents
systems today. The importance of ethical systems, he argued, would grow as “future ma-
chineswilllikelyhaveincreasedcontrolandautonomy....Morepowerfulmachinesneed
more powerful machine ethics”. Indeed this is now the case, as in the decade since Moor
was writing, autonomous systems acting in the real world have become more and more a
partofourreality,andrecentsuccessesinartificiallyintelligenttechnologieshaveprompted
prominentresearcherstoarguefortheurgencyofresearchintoethicalsystems(Russelletal.
(2016)).
Moor suggested that implicit ethical systems could be regarded as moral agents, even
though “Of course, such machine virtues are task specific and rather limited. Computers
don’thavethepracticalwisdomthatAristotlethoughtweusewhenapplyingourvirtues.”
Butnotallphilosopherswouldagreethatimplicitethicalagentswereethicalatall:Kant,
forexample,arguesinKant(1785)thatactionsdoneinaccordancewiththelawonlyhave
moral worth if they are also done for the sake of the law. This seems correct: acting in
accordancewithwhatisrightdeservesnopraiseifitisimpossibletoactotherwise.None
theless,perhaps,ifweareunleashingagentsintherealworld,itisenoughthattheyactin
conformitywiththelaw:moralworthisperhapsnotanissuewithmachines.
However,regimented(Moor’s‘implicitethical’)agentsaredesignedtoactinalimited,
predictable environment, whereas the real world is not predictable, and unforeseen situa-
tionswillarise.Intheseopensystems,regimentationofbehaviourbyrestrictingagentsto
performingonlypermissibleactions(e.g.Estevaetal.(2002),vanderHoeketal.(2007),
Ågotnes et al. (2009)) does not allow agents to adapt to such unforeseen circumstances.
Moreover,unlikenormsfoundinlegalandmoralsystems,suchnormscannotbeviolated,
andsoitcanbeargued(e.g.JonesandSergot(1992),Governatori(2015))thattheyshould
not be seen as norms all, because the agents have no choice beyond compliance or non-
participation.Suchrulesarethusmoreliketherulesofagame,thanmoralandlegalnorms.
Wethereforearguethatagentsinopensystemsshouldhavethecapacitytoexplicitlyrea-
sonabouttheactionstheyperform,andinparticularwhetherornotbehavingaccordingto
normativeprescriptionsisinaccordancewithethicalcriteriathatthesenormsaredesignedto
serve,sothatifnecessaryagentsmaychoosetoviolatenorms.Again,asarguedbyMoor:
“more powerful machines need more powerful machine ethics”. However, this raises the
questionastowhataretheethicalcriteriabywhich‘explicitethicalagents’shouldchoose
which norms to adhere to. While quantitative utilitarian calculations may suffice in sim-
plescenarios2,morecomplexscenarioscharacteristicofrealworldmoraldilemmasarenot
typicallyresolvablethroughsuchcalculations(Dennet(1995)).Whatisrequiredisacom-
putationalaccountofpracticalreasoningthatequateswithwhatMoorcalledthe“practical
wisdomofthetypethatAristotlethoughtweusewhenapplyingourvirtues”.Practicalrea-
soninghasbeeninvestigatedbothwithinthestandardmulti-agentsystemcontextofbeliefs,
desiresandintentions(BDI,e.g.RaoandGeorgeff(1991)),inworksuchasBroersenetal.
(2001),and,withethicalconsiderationsattheforefront,throughthecombinationofdeontic,
epistemicandactionlogics(vanDenHovenandLokhorst(2002)).Theseapproachesdonot
2 Seeforexampleavariationofthetrolleyproblemsetup(J.F.Bonnefon(2016))inwhichautilitarian
autonomousvehiclewillchoosetodirectitselfsoastominimisethenumberofcasualties.
NormsandValueBasedReasoning:JustifyingComplianceandViolation 3
directlyaddresstheissueofhowtorationallychooseamongstoptionsforactionbyappeal-
ingtoethicalcriteria.However,since2006whenMoorwaswriting,therehasbeengrowing
interestintheuseofargumentationtheoryinthecontextofpracticalreasoninginorderto
resolve such conflicts (e.g., Rahwan and Amgoud (2006)), and in particular Value-Based
ReasoningandArgumenationasinAtkinsonandBench-Capon(2007)andVerheij(2016),
in which the ethical criteria appealed to take the form of values promoted by actions and
therelativeimportance(ranking)ofthesevalues.WewillargueforthepositionthatValue-
BasedReasoning,especiallyinthemannerofworkbasedontheapproachofAtkinsonand
Bench-Capon (2007), is one way to provide current and coming agents whose autonomy
andoperationwillplacetheminunforeseensituations.wheretheymustrecognisetheneed
toviolatetheirnorms,andtoviolatetheminthemostappropriateway,withthecapacityto
makesuchchoices3.
Ofcourse,wedonotadvocatethatvaluebasedreasoningandargumentationsufficeson
itsownasaformalismforexplicitethicalagents;comprehensiveformalmodelsmaywell
requiretheintegrationofsuchreasoningwithBDIanddeonticlogics,andpossiblyalsothe
integrationofquantitativeandqualitativereasoning.Rather,thepurposeofthispaperisto
proposethatvaluebasedreasoningandargumentationisabletoplayanimportantrolein
equippingagentswiththeabilitytoreasonaboutnormsandrelatedethicalconsiderations,
andtothereforealsobeabletodealwithunforeseensituations.Wewillsupportourposition
by illustrating use of value based formalisms for reasoning about norms using associated
valuescharacteristicofsocietiesofincreasingcomplexity,andhowagentsmaythenneedto
justifyviolationofnormsinordertoensurepromotionofthevaluesthatthesenormswere
intendedtoserve.
Wenowfurtherelaborateonsomeoftheapproachesmentionedabove,bywayofthen
introducing value based reasoning and argumentation. We will consider how value based
reasoningcanbeusedtojustifynormsgivencertainscenarios,andhowdifferencesinthe
values,preferencesandstatedescriptionsaffectthenormsthatcanbejustified.
AnexcellentstartingpointforconsideringreasoningaboutnormsisUllmann-Margalit
(1977),whichdoes,ofcourse,considerablypre-datemultiagentsystems,butnonetheless
contains many relevant considerations. In that work, Ullmann-Margalit uses as scenarios
onwhichtobaseherdiscussionofnorms,simpletwoplayergames,suchastheprisoner’s
dilemma (PD, Rapoport and Chammah (1965)). In such games there are two players and
each can cooperate or defect, and their choices determine the payoffs. In PD as used in
Ullmann-Margalit (1977), mutual cooperation gives a payoff of three to each player and
mutualdefectiononetoeachplayer,whileiftheactionsdifferthedefectorreceivesfiveand
thecooperatorreceiveszero.SomekeyresultsconcerningPDarethattheNashEquilibrium
(RothandMurnighan(1978))iswherebothdefect(sincedefectionisthedominantaction,
andwillreceivethebetterpayoffwhatevertheotherplayerdoes)andthatasuccessfulstrat-
egyiniteratedPD(wheretheplayersplayoneanotherrepeatedly)isTit-For-Tat(Axelrod
(1987),butseeBinmore(1998)).UsingTit-For-Tatanagentwillcooperateinthefirstround,
3 Oneoftheanonymousreviewersstated“formalworksinthisfieldoftenaimtoprovidemathematical
theorems,e.g.,torelatetheexpressivepowerofalogicwiththequestionofcomputationalcomplexitythat
arisefromthislogic....Itisbynomeansobviousthatmathematicallyorientedresearchersinthefieldare
committedtotheclaimthatvaluesshouldbeleftout.Thefactthatthefocushasbeenonnorms(inthe
crudesenseofforbiddingtransitions)mighthavemoretodowiththepresentstateofourformalmodelling
capabilities.”Suchformalconcerns,andtheneedtoovercomesomeofthelimitations,arethebasisofthe
REINSprojectdescribedinBroersen(2014).Wedonotchallengethis,butarguethatthemovetowards
Moor’sfullethicalagents,forexamplebytheinclusionoftheabilitytoreasontowithvalues,andtoenable
themtoviolatenorms,willbeneededifautonomousagentssuchasdriverlesscarsaretobedeployedin
practice.
4 TrevorBench-Capon,SanjayModgil
andthencopyitsopponent’spreviousmoveineverysubsequentround.ImportantlyPDis
anon-zerosumgame:theaggregateutilityofmutualcooperationisgreaterthananyother
payoff,andtheequilibriuminfactyieldsthelowestcollectiveutility.Thus,itwouldinfact
bemutuallybeneficialifoneofferedapaymenttotheotheriftheycooperated:thiscould
secureapayoffofthreeandtwo,sothatbothwouldgainovermutualdefection.Suchagree-
ments are, however, not possible in normal versions of the game, which do not allow for
priornegotiations.
Publicgoodsgameshaveformedthebasisofseveralstudiesoftheemergenceofnorms
in multi-agent systems such as Shoham and Tennenholtz (1997), Sen and Airiau (2007),
Skyrms(2014),Bicchieri(2005),Sugawara(2011)andMahmoudetal.(2015).Empirical
studiessuggest,however,thatapplyingstandardgametheory4 topublicgoodsgamesdoes
notprovideaveryrealisticmodelofactualhumanbehaviour.Experimentsusingsuchgames
are very common and have formed the subject of a number of metastudies. For example
Engel(2011)examined131examplesoftheDictatorGameandOosterbeeketal.(2004)was
basedon37papersreportingUltimatumGameexperiments.Innoneofthesemanystudies
wasthecanonicalmodelofgametheoryandeconomics5 followed.Althoughthestudyof
Henrichetal.(2001)lookedatfewersocieties(fifteen),itisparticularlyinterestinginthat
thestudiesconsideredhighlyhomogeneoussocieties,andusedthesamemethodologyfor
eachexperiment.Againnoneofthesocietiesfollowedthecanonicalmodel.Consequently
itisnotplausibletoseethecanonicalgametheoreticmodels,andtheircriteriasuchasthe
NashEquilibrium,asjustificationforthenormsencounteredinsuchsocieties.
Another approach is to model scenarios as State Transition Diagrams (STD), and to
investigatehownormscanbedesignedinsuchsituationstoavoidunwantedstatesofaffairs
as in Wooldridge and van der Hoek (2005) and Ågotnes and Wooldridge (2010). In these
approaches,agentsaretypicallyrepresentedusingtheBelief-Desire-Intention(BDI)model
(RaoandGeorgeff(1991),Wooldridge(2009)),inspiredbyBratman(1999).TheBDImodel
supposes agents to have a set of beliefs and a set of dispositional goals (desires). Actions
arechosenbyidentifyingthedesiresthatcanberealisedinthecurrentsituation(candidate
intentions),andthencommittingtooneormoreoftheseintentions,andchoosingacourse
ofactionintendedtorealisetheassociatedgoals.This,however,leavesopenthequestionof
wherethedesirescomefrominthefirstplace.ThisinturnmeansthatinBDIsystems,there
isnoexplanationofwheregoalscomefrom.Oftentheyarecompletelyfixed,andevenin
systemswheretheycanbederivedfromthecurrentstate(e.g.RahwanandAmgoud(2006)),
thereisafixedsetofpotentialdesiressomeofwhichareactiveinagivensituation,which
meansthatwecansaywhythedesireisactive,butnotwhythereissuchadesiretoactivate.
Thisinabilitytojustifydesiresgreatlylimitstheabilitytoengageinmoralreasoningina
transparentfashion.
Thispaperproposesuseofanalternativeapproachtoactionselection,oftencalledprac-
ticalreasoning(Raz(1979))andformalisedasValue-BasedReasoning,asameanstoen-
ableagentstojustifynormsbyreasoningaboutthesocialandmoralvaluesthatnormsare
designed to serve; that is, the kinds of moral reasoning we expect of humans. Agents are
associated with a set of social values, the aspirations or the purposes an agent might pur-
sue,suchasliberty,equality,fraternity,wealth,healthandhappiness.Thesevaluesprovide
reasonswhycertainsituationsareconsideredgoalsbytheagent,andsoallowsforthejus-
tificationof“desires”,asexplainedinAtkinsonandBench-Capon(2016).Thebasicideais
4 Inwhich,asinclassicaleconomics,playersareallrational,self-interestedandperfectlyinformedand
actsoastomaximisetheirownutility.
5 In which, in the example of the Ultimatuatum Game, the proposer would offer the smallest amount
possible,andtherecipientwouldacceptanyoffer,nomatterhowsmall.
NormsandValueBasedReasoning:JustifyingComplianceandViolation 5
thatagentshaveasetofsuchvaluesandtheiraspirationsandpreferencesarecharacterised
bytheirorderingofthesesocialvalues.Attemptingtopromote(oravoidthedemotionof)
preferredvaluesprovidesreasonsfordesiringsomestatesofaffairs,andforchoosingpar-
ticularactions.Inthecontextofargumentationbasedmodelsofreasoning,anorderingon
valuescanbeconceptualisedasanaudiencetowhichtheargumentsareaddressed(Perel-
man (1971)), and acceptance of an argument as to what to do depends not only on the
argumentitself-foritmust,ofcourse,beasoundargument-butalsoontheaudiencead-
dressed. This notion of audience as an ordering on values was computationally modelled
inGrassoetal.(2000)andmademoreformalinValue-BasedArgumentationFrameworks
(VAFs,Bench-Capon(2003)).VAFsareanextensionoftheabstractArgumentationFrame-
works(AFs)introducedinDung(1995),butwhereasinanAFanargumentisdefeatedby
anyattackingargument,inaVAFanargumentisdefeatedfor anaudiencebyanattacker
onlyifthevalueassociatedwiththeattackingargumentisrankedatleastashighlyasthe
attackedargumentbythataudience.Inthiswaydifferentaudienceswillacceptdifferentsets
ofarguments(preferredsemantics(Dung(1995))isusedtodetermineacceptance),and,as
isshowninBench-Capon(2003),providedtheVAFcontainsnocyclesinthesamevalue,
therewillbeauniquenon-emptypreferredextension.
UseofVAFsprovidesawayofexplaining(andcomputing)thedifferentargumentsac-
ceptedbydifferentaudiences.Value-BasedReasoninghasbeenusedasthebasisofpractical
reasoningin,amongstothers,Garcezetal.(2005),AtkinsonandBench-Capon(2007),and
vanderWeideetal.(2011),andappliedinparticularareasincludinglaw(Bench-Caponetal.
(2005)),e-democracy(CartwrightandAtkinson(2009)),policyanalysis(Tremblay(2016)),
medicine(Atkinsonetal.(2006)),experimentaleconomics(Bench-Caponetal.(2012)),rule
compliance(Burgemeestreetal.(2011)),decisionsupport(Nawwabetal.(2008))andeven
ontologyalignment (Trojahnet al.(2008),Payne andTamma (2015)).Complexityresults
forVAFswereestablishedinDunne(2010)andNofaletal.(2014).Herewewilldiscuss
normsandtheirdesignandjustificationintermsoftheValue-Basedapproachtopractical
reasoning.
InSection2wereviewbackgroundworkontheformalisationofValueBasedArgumen-
tationinAlternateActionBasedTransitionSystems(AATS)(WooldridgeandvanderHoek
(2005)).Thesesystemsmodelopenagentsystems(quamodelsof‘worlds’inwhichagents
engageinjointactionstobringaboutdesiredstatesofaffairs).Section3thenshowshow
thisapproachenablesreasoningaboutandjustifyingthenormsthatservevaluesandvalue
orderingscharacteristicofsocietiesofincreasingcomplexity.Section4furtherexploresthe
use of such reasoning by agents who may need to justify violation of norms in order to
ensure promotion of the values that these norms were intended to serve. We also briefly
considerwhatmakesanorderingonvaluesacceptable,andhowsuchanorderingmightbe
determined,inSection5.Weconcludewithsomereflectivediscussionandpointtofuture
workinSections6and7.
2 Background
Inthissectionweprovidesomeessentialbackground:thestructurewhichweusetomodel
our “world”, Alternate Action Based Transition Systems (AATS); the value-based argu-
mentsthatagentscangeneratefromsuchstructuresandusetojustifytheiractionsinthis
environment,and;therunningexamplewewillusetoillustrateourmodel.
6 TrevorBench-Capon,SanjayModgil
2.1 AlternateActionBasedTransitionSystems
Inopenagentsystems,anindividual’schoicedoesnotnecessarilydeterminethestatethat
will be reached. To account for this, open agent systems should model transitions as the
jointactions6 composedoftheindividualactionsofalltheagentsrelevanttothesituation.
AsuitablevariantofstatetransitiondiagramsforuseinopenagentsystemsisAction-based
AlternatingTransitionSystems(AATS),introducedinWooldridgeandvanderHoek(2005),
sincetheirtransitionsarethejointactionsofalltheagentsrelevanttothesituation.AATSare
formallybasedonAlternating-timeTemporalLogic(Aluretal.(2002)).ThebasicAATS
wasaugmentedinAtkinsonandBench-Capon(2007)toallowthelabellingofthetransitions
withthevaluespromotedanddemotedbythattransition.AATSslabelledinthiswaywere
termedAction-basedAlternatingTransitionSystemswithValues(AATS+V)andAATS+Vs
were used to provide the underpinning semantical structure for the approach to practical
reasoningsetoutinthatpaper.Theformaldefinitionsaregiveninthefollowingsubsection.
2.1.1 FormalDefinitions
Definition1:AATS(WooldridgeandvanderHoek(2005)).AnAction-basedAlternating
TransitionSystem(AATS)isan(n+7)-tupleS=(cid:104)Q,q0,Ag,Ac1,...,Acn,ρ,τ,Φ,π(cid:105),where:
– Qisafinite,non-emptysetofstates;
– q ∈Qistheinitialstate;
0
– Ag={1,...,n}isafinite,non-emptysetofagents;
– Ac isafinite,non-emptysetofactions,foreachag ∈AgwhereAc ∩Ac =∅forall
i i i j
ag (cid:54)=ag ∈Ag;
i j
Q
– ρ : Acag → 2 is an action pre-condition function, which for each action α ∈ Acag
definesthesetofstatesρ(α)fromwhichα maybeexecuted;
– τ :Q×J →Qisapartialsystemtransitionfunction,whichdefinesthestateτ(q,j)
Ag
thatwouldresultbytheperformanceofjfromstateq.Thisfunctionispartialasnotall
jointactionsarepossibleinallstates;
– Φ isafinite,non-emptysetofatomicpropositions;and
– π :Q→2Φ isaninterpretationfunction,whichgivesthesetofprimitivepropositions
satisfiedineachstate:ifp∈π(q),thenthismeansthatthepropositionalvariablepis
satisfied(equivalently,true)instateq.
AATSsareparticularlyconcernedwiththejointactionsofthesetofagentsAg.j is
Ag
thejointactionofthesetofnagentsthatmakeupAg,andisatuple(cid:104)α1,...,αn(cid:105),wherefor
eachαj(wherej≤n)thereissomeagi∈Agsuchthatαj∈Aci.Moreover,therearenotwo
differentactionsαjandαj(cid:48) injAgthatbelongtothesameAci.Thesetofalljointactionsfor
thesetofagentsAgisdenotedbyJAg,soJAg=∏i∈AgAci.GivenanelementjofJAgandan
agentag ∈Ag,ag’sactioninjisdenotedbyji.ThisdefinitionwasextendedinAtkinson
i i
andBench-Capon(2007)toallowthetransitionstobelabelledwiththevaluestheypromote.
Definition2:AATS+V(AtkinsonandBench-Capon(2007)).GivenanAATS,anAATS+V
isdefinedbyaddingtwoadditionalelementsasfollows:
6 Here,asinAluretal.(2002)andWooldridgeandvanderHoek(2005),byjointactionnoimplicationof
theagentsactinginconcertisintended.Ajointactionissimplyanactioncomposedofactionsperformedby
asetofagentsatthesametime,withoutanysuggestionofcoordination,orcommonpurpose.Thiscontrasts
withthenotionofjointactionine.g.Levesqueetal.(1990),whichconcernsactinginteams.
NormsandValueBasedReasoning:JustifyingComplianceandViolation 7
– V isafinite,non-emptysetofvalues.
– δ :Q×Q×V →{+,–,=}isavaluationfunctionwhichdefinesthestatus(promoted
(+),demoted(–)orneutral(=))ofavaluev ∈V ascribedtothetransitionbetweentwo
u
states:δ(qx,qy,vu)labelsthetransitionbetweenqx andqy withoneof{+,–,=}with
respecttothevaluev ∈V.
u
AnAction-basedAlternatingTransitionSystemwithValues(AATS+V)isthusdefined
asa(n+9)tupleS=(cid:104)Q,q0,Ag,Ac1,...,Acn,ρ,τ,Φ,π,V,δ(cid:105).Thevaluemaybeascribed
onthebasisofthesourceandtargetstates,orinvirtueofanactioninthejointaction,where
thatactionhasintrinsicvalue7.
2.2 ReasonsforAction
Thevaluesgiveagentsreasonstoperformornottoperformthevariousactions,basedonthe
argumentationschemeproposedinAtkinsonandBench-Capon(2007).Anumberofsuch
reasonsaregiveninAtkinsonandBench-Capon(2014)(the“N”suffixdenotesreasonsnot
toperformtheaction: jisajointactioninwhichagentagperforms jag,φ isagoal,which
holds or fails to hold in a given state, and which agents may attempt to realise, maintain,
avoidorremove).
R1Weshouldparticipatein jinqinwhichφ holdstomaintainφ andsopromotev.
R2NWeshouldnotparticipatein jinqinwhichφ holdssinceitwouldremoveφ and
sodemotev.
R3Weshouldparticipatein jinqinwhich¬φ holdstoachieveφ andsopromotev.
R4NWeshouldnotparticipatein jinqinwhich¬φ holdssinceitwouldavoidφ and
sofailtopromotev.
R5Weshouldparticipatein j inqtoensureφ andsopromotev.Notethatφ maybe
contingently realised or unrealised in q and that, in some variants, the promotion of v
mightnotbeimmediate,orpermanent.ThisalsoappliestoR5NandR6.
R5NWeshouldnotparticipatein jinqwhichwouldensure¬φ andsodemotev.
R6Weshouldparticipatein jinqtoprevent¬φ andsopromotev.Notethat¬φ may
becontingentlyrealisedorunrealisedinq.
R6NWeshouldnotparticipatein jinqwhichwouldpreventφ andsofailtopromote
v.Wesuggestthattomakethereasonworthconsiderationweshouldonlyusevariants
whichpreventφ immediatelyandpermanently.
R7Weshouldparticipatein jinqinwhich¬φ toenableφ tobeachievedandvtobe
promotedonthenextmove.
R8NWeshouldnotparticipatein jinqinwhichφ whichwillriskφ beingremovedon
thenextmovewhichwoulddemotev.
R9Weshouldparticipatein jinqbecauseperforming jagpromotesv.
R9NWeshouldnotparticipatein jinqbecauseperforming jagdemotesv.
Objections to these arguments can be formed by questioning whether the state is as
claimed,theconsequencesoftheactionwillbeasspecified,whetherthegoalisrealisedand
whetherthevalueisindeedpromoted.Thearguments,attacksandrebuttalsoftheattacksare
thenorganisedinaValue-BasedArgumentationframework(VAF)asdescribedinBench-
Capon(2003)andevaluatedaccordingtoanorderingonthevalues.Thesevalueorderings
7 AlthoughthelabellingsaretakenasgivensinAtkinsonandBench-Capon(2007),itispossibletomake
theascriptionofvaluestotransitionstransparent,andtoenableagentstoargueaboutwhatvaluesshould
labelagiventransition.ThiswasfullydiscussedinAtkinsonandBench-Capon(2016).
8 TrevorBench-Capon,SanjayModgil
willdependonthesubjectivepreferencesoftheparticularaudience,andsodifferentagents
mayquiterationallychoosedifferentactions.
Tosummarise:threestagesinpracticalreasoningareidentifiedinAtkinsonandBench-
Capon(2007):
– Problem formulation: essentially the construction of an AATS+V for the particular
problemsituation.TheAATS+Vwillreflecttheviewsoftheagentengagedintherea-
soning,andsowillembodythatagent’scausalmodel(todeterminethetransitions)and
its values (to enable the labelling of transitions), as is demonstrated in Atkinson and
Bench-Capon(2016).AsAtkinsonandBench-Capon(2016)indicatestherecanbear-
gumentsjustifyingtheformulationoftheproblem,beforetheAATS+Visusedinthe
subsequentstages.
– Epistemicstage:thisinvolvesdeterminationofwhattheagentengagedinthereasoning
believes(orchoosestoassume)aboutthecurrentstateandthejointactionthatwillresult
fromthechoiceofaparticularindividualactionbytheagentconcerned;
– Option selection: the arguments generated from the AATS+V based on the reasons
givenabove,andobjectionsandcounterexamplestothesearguments,areformedintoa
VAFandtheiracceptabilitystatusdeterminedaccordingtothepreferencesoftheagent
engagedinthereasoning.Theacceptabilitystatusdetermineswhichactions,according
toitsbeliefs,valuesandpreferences,theagentcanjustifiablychoosetoperform.
Intheworkmentionedabove,andintheexamplesbelow,valuepreferencesareconsid-
eredpairwise.Infacttransitionsmaybelabelledwithseveralvalues,sothatthecomparison
should be between sets of values: our approach is just to use the most preferred of these
values.UseofsetsofvalueswasconsideredinPrakken(2002),ChorleyandBench-Capon
(2005) and Bench-Capon et al. (2011). Also it might be desirable to consider degrees of
promotionofvalues,asadvocatedinSartor(2010).Suchextensions,however,stillrequire
furtherworkifaconsensusastothebestapproachistobeachieved,and,sincetheirusedoes
notaffectthegeneralprincipleofprovidingagentswiththecapacitytoperformvalue-based
reasoning,wewillcontinuewiththesimplepairwisecomparisonsinthispaper.
2.3 Example
An AATS+V was used in Bex et al. (2014) to model the states and actions found in both
thefableofTheAntandtheGrasshopper(Aesop(1909))andtheparableofTheProdigal
Son (Luke 15:11-32). Fables and parables are suitable examples for us because they are
stories with a moral point, and are frequently used in the moral education of children. In
TheAntandtheGrasshopper,thestoryisthatduringthesummerthegrasshoppersingsand
playswhiletheantworkshardstoringupfoodforthewinter.Whenthewintercomes,the
grasshopperhasnofood;norwilltheantgiveawayanyofitsstore,andsothegrasshopper
dies.InTheProdigalSontheprodigalwasteshisinheritanceonidleplaybutwhendestitute
askshisfatherforforgiveness;thefatherdoesforgiveandtakeshimbackintohishousehold.
AnAATS+VbasedonthemodelofBexetal.(2014)isshowninFigure1.Inourex-
ample, food is sufficiently abundant in summer that one can gather food and eat without
effort.Growingfoodforthewinteris,however,afulltimeeffort(digging,planting,weed-
ing,reaping,storing)andproducesasurplus,butthenatureoftheactivityisthatitiseither
doneornot:theamountproducedisnotproportionaltotheeffort.Thefoodcannotbestored
overthesummer:thereforethewinterendswithaperiodofcarnival(q5,q8andq12)when
thesurplusisconsumedwithfeasting.Thestatehasfivepropositions.Thefirsttwoindicate
NormsandValueBasedReasoning:JustifyingComplianceandViolation 9
Fig.1 AATS+VfortheExample:w=work,p=play,a=ask,g=give,r=refuse,e=eat,f=feastd=die.
ThesameAATS+Visusedforboththefableandtheparable.Jointactionsgivethatactionoftheant/father,
followedbythatofthegrasshopper/son.Statesarevectorsoffivepropositionsrepresenting:ant/fatheralive,
grasshopper/sonalive,ant/fatherhasfood.grasshopper/sonhasfood,seasonissummer,respectively.
whethertheant(father)andthegrasshopper(son)arealive,thethirdrepresentswhetherthe
ant(father)hasnofood(0),enough(1)orabundantfood(2),thefourthsimilarlyrepresents
whetherthegrasshopper(son)hasno,enoughorabundantfood,andthefifthwhetheritis
summer(1)orwinter(0).Thustheinitialstate,q is11001,indicatingthatbotharealive,
1
neitherhasfoodandthatitissummer.Now,ifbothworkandthestatebecomesq ,11220
3
indicatesthatbotharealive,bothhaveabundantfoodandthatitiswinter.Thekeydecisions
are in the initial state (q1) where the grasshopper and the prodigal choose to play (action
p) while the ant and the father choose to work (action w), thus transitioning to q6, and in
q6wheretheantrefusesthegrasshopper(actionr),sotransitioningtoq7,whilethefather
givestotheprodigal(actiong),sotransitioningtoq5.Intheotherstatestherearenochoices
tobemade.
WehavelabelledthediagraminFigure1withjustfourvalues.Lifefortheant(father)
andgrasshopper(son)(L andL )andPleasurefortheant(father)andthegrasshopper(son)
a g
(P andP).Additionalvaluelabelswillbeidentifiedasourdiscussionproceeds.
a g
3 JustifyingNormswithValueBasedArgumentation
Thebasisofourapproachisthatagentswillbefacedwithsituationsrequiringachoiceof
action. They will normally choose the action which has the most favorable consequences
for themselves; in our terms the action which promotes their most preferred value or set
ofvalues.Sincedifferentagentswillhavedifferentpreferencestheywill,quitereasonably,
makedifferentchoices.Butagentsdonotexistinisolation:theirchoiceswillimpactonother
agents,andwhatotheragentschoosewillaffecttheconsequencesoftheirownactions.Ina
10 TrevorBench-Capon,SanjayModgil
socialgroupnormswilldevelop,orbeimposed.Theseareimportantwherethechoicesof
individualsmaygoagainsttheinterestsofthegroup,orwhereadditionalinformationabout
howothersmaybeexpectedtoactisuseful.Normsmaybesocialconventionsorlaws.In
boththesecasestheyaretypicallyreinforcedbysanctions(seeBench-Capon(2015)).Legal
norms are typically backed by formal sanctions such as fines and imprisonment, whereas
social norms are backed by informal sanctions such as ostracism, ridicule or perhaps just
disapprovinglooks.Therearealsomoralnorms.Conventionsandnormsmayembodymoral
norms, and should at least be consistent with them, but unless so embodied, moral norms
are not usually associated with sanctions. Conventions and laws, introducing as they do
thenotionsofsanctionsandpunishments,willoftenbeassociatedwithapurpose,suchas
topromotethesustainabilityofthegrouporsociety.SimulationssuchasMahmoudetal.
(2015) have shown that punishments are essential if normative collapse is to be avoided:
morethanthat,stabilityrequiresthatfailuretopunishbeitselfpunished.
Inthissectionweconsiderhowasetofnormsmightdevelopfromaconsiderationof
value-basedpracticalreasoningintheexamplescenario.
3.1 Argumentsinq
1
Wefirstconsidertheargumentsastowhattodo,availabletoanagentinq ,basedonthe
1
valuesofpleasureandlife.Theagent’sownpleasureandlifewillbedenotedP andL ,the
s s
pleasureandlifeoftheotherasP andL .Ourargumentsarederivedfromthereasonsof
o o
section2.2,buthereexpressedintermsofonlytheagent’sownactionandthevalue,e.g.
youshouldperformα sinceitwillpromotev,whereα istheagent’sactioninthejustified
jointaction,andvisthevaluepromoted.
A YoushouldnotplaysinceitwillriskL beingdemoted(R4N)
s
B YoushouldworksinceitwillenableP tobepromoted(R7)
s
C YoushouldplaytopromoteP (R9)
s
D YoushouldnotworksinceitwilldemoteP(R9N)
s
Thuswehavereasonsproandconworking:theproreasonisthefuturepleasureiten-
ables(B),andtheconreasonistheimmediatelossofpleasurewhichaccompaniesit(D).
Playincontrastaffordsimmediatepleasure(C),butrisksthelossoflife(A).Theriskasso-
ciatedwithargumentAissubstantial:avoidingdeathrequiresboththattheotherworks,and
thattheotherwillforgoitsownpleasureinordertosaveone’slife.Therefore(assuminglife
ispreferredtopleasure)onlythemostrisktakingindividualswillchoosetoplayinq .
1
In the following subsection we will show how this norm can also be explained and
justifiedintermsofvaluebasedreasoning.
3.2 Afirstsetofnorms
Soletussupposethatthesocietyhasthenorm:
SN18:Itisforbiddentoplay
8 WeuseSNforsocialnorms,whichcanbemoralnorms,butmayalsobefound,backedwithsanctions,
associalandlegalnorms.WeuseMNforthosenormsbetterseenaspurelymoralnorms,thosetypicallynot
incorporatedinasocialorlegalcode,andnottypicallyassociatedwithsanctions.