T11. Normative multi-agent systems

Guido Boella & Leon van der Torre

Tutorial NormativeReasoning and Decision Making

Agreement Technologies

• WG1: “Semantics” • WG2: “Norms” • WG3: “Organizations” • WG4: “Argumentation and Negotiation” • WG5: “Trust”

2

Some Norms• You are obliged to return the book in 2 weeks• Role X is permitted to access resource Y• Actions of agent X count as actions of agent Y• Advocates should act in the interest of their clients• Violations should be sanctioned• Don’t do to them what you don’t want them to do to you• People should know the law• Every adult has the right to vote• Make your homework!• We start at 09:00

3

Merriam-Webster Online Dictionary• ‘normative’: ‘conforming to or based on norms’,

– as in normative behavior, normative judgments– not: ‘of, relating to, or determining norms or standards’, as in

normative tests, or ‘prescribing norms’, as in normative rules of ethics or normative grammar.

• ‘norm’: ‘a principle of right action binding upon the members of a group and serving to guide, control, or regulate proper and acceptable behavior’.

– Not: ‘an authoritative standard or model’, ‘an average like a standard, typical pattern, widespread practice or rule in a group’, and various definitions used in mathematics.

4

Normative Systems (1971)• “When a deductive correlation is such that the

first sentence of the ordered pair is a case and the second is a solution, it will be called normative. If among the deductive correlations of the set there is at least one normative correlation, we shall say that the set has normative consequences. A system of sentences which has some normative consequences will be called a normative system.”

• C. Alchourron and E. Bulygin. Normative Systems. Springer, 1971.

5

Normative Systems (1993)• Normative systems are “systems in the behavior

of which norms play a role and which need normative concepts in order to be described or specified”

J.-J. Meyer and R. Wieringa. Deontic Logic in Computer Science: Normative System Specification. John Wiley & Sons, 1993.

• Many distinct notions of “normative systems”– Social expectation, legal law, linguistic imperative…

• Role of norms in computer science is changing – Solutions based on multiagent systems increasing

6

Layout• Challenges due to uncertainty and imprecision

in normative reasoning and decision making

1. Normative reasoning in computer science2. Imprecision & uncertainty normative systems3. Decision making in normative systems

7

1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems

8

AgentLink RoadMap (2003)

9

2005: Normative MultiAgent Systems

• are multiagent systems with normative systems • in which agents can decide whether to follow the

explicitly represented norms, and • the normative systems specify how and in which

extent the agents can modify the norms.’’

G. Boella, L. van der Torre, H. Verhagen, Introduction to normative multiagent systems. Computational and Mathematical Organization Theory, double special issue on normative multiagent systems, 2006.

“The normchange definition”


10

2007: Normative MultiAgent System

• is a multiagent system organized by means of• mechanisms to represent, communicate,

distribute, detect, create, modify, and enforce norms, and

• mechanisms to deliberate about norms and detect norm violation and fulfillment.”

G. Boella, L. van der Torre, and H. Verhagen, Introduction to the special issue on normative multiagent systems. Autonomous Agents and MultiAgent Systems, Aug. 2008.

“The mechanism design definition”





11


12

Norms: Certain and Precise?• Norms are rules to distinguish right from wrong.

• Dilemmas: when may we shoot down a hijacked plane? 1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems

13

Violations: Certain and Precise?• Norms describe what counts as a violation.

• Budget deficit > 3 % of GDP is a violation?1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems

14

Design: Uncertain and Imprecise• Norms are rules to guide, control or regulate behavior.

• What are the effects of new norms on behavior? 1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems

Guide, Control, Regulate Behavior

What’s new with respect to constraints?

15


Guide, Control, Regulate Behavior Ways to deal with violations, representation of permissive norms,

the evolution of norms over time (in deontic logic), the relation between the cognitive abilities of agents and the global properties of norms, how agents can acquire norms, how agents can violate norms, how an agent can be autonomous (in normative agent architectures and decision making), how norms are created by legislator, emerge spontaneously or are negotiated among agents, how norms are enforced, how constitutive or counts-as norms are used to describe institutions, how norms are related to social and legal concepts, how norms structure organizations, how norms coordinate groups and societies, how contracts are related to contract frames & contract law, how legal courts are related, how normative systems interact, …

16


17

Normative Reasoning

Norms are rules to guide, control or regulate behavior.

Infrastructure for open communities.Reasoning in open environments.

Trust and reputation.

More than distinguishing right from wrong.More than describing what counts as a violation.

More than constraints.Uncertain and imprecise.


18

Security and Reliability• Norms make systems more secure & reliable.

• How to prevent over-regulation?1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems

19

Trust• Norms build up trust in international trade.

• Which mechanisms for electronic commerce?

ESCROW


20

Example: ESCROW• Norms describing coordination

– Agent communication: sales agreement– Agent transaction: payment for shipment

• Norms describing fees and compensation– ESCROW gets percentage (from seller or buyer)– Return merchandize within 5 days

• Norms ensure fulfillment of some obligations– Seller is paid for goods, or goods will be returned– (under assumption of trustworthiness of ESCROW)

• Some obligations can still be violated:– Seller does not ship goods, shipper unreliable, …

2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems4. Norm change

21

Virtual Communities• Social norms are emerging in Second Life.

• How to prevent exclusion from communities?1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems

22

Autonomous Systems• aut- + nomos: making ones own norms.

• How to define global policies about local ones?1. Normative reasoning in computer science2. Uncertainty and imprecision in normative reasoning3. Decision making in normative systems

23

Applications Normative Reasoning

Norms are rules used to control or regulate behavior.

Risk management for computer security.Designing trust mechanisms for electronic commerce.Recognition of emerging social norms in Second Life.

Local and global policies for autonomous systems.





24


Examples: How to decide …• … whether to fulfill or violate a norm?• … whether behavior counts as a violation?• … whether an excuse for a violation is acceptable?• … whether a violation should be sanctioned?• … whether to accept a norm / contract / … ?• ... whether to accept sanctions of a norm?• … which contract to propose / norm to create?• … whether an agent conforms to the norm?• … whether a procedure is compliant to the norm?• … on a constitution for Europe?

25


26

Basis: Decision Making With Norms• Introducing obligations in agent architectures•

obligations as normative goals

• Introducing obligations in agent programming

BDIO

Pobs actal tl

hl

tltl cl


Exercise: When Violate a Norm?• When you are not able to fulfill it• When the sanction is too low• When the probability to be caught is too low• …

27


Exercise: When Violate a Norm?• When you are not able to fulfill it• When the sanction is too low• When the probability to be caught is too low• When you don’t know the norm• When there is another more important desire• When there is another more important obligation• When you have a good excuse• When you like to take risks• When other agents violate the norm• When the norm does not make sense

28


29

When to Violate/Fulfill: Challenge

• Multiagent structure of a normative system– Many agents involved in agent interaction– Interaction is highly structured

• Normative system regulates their roles– Powers of roles described by counts-as norms, and– Their behavior is regulated by procedural norms.


30

Norm Compliance• Attacker = agent trying to profit from violation

– Violation undetected, no sanction, or less than profit

• Anticipate rather than regiment (mechanism design)

Attacker / adversary

$$

normative system


31

Profile NS of Attacker A (for Op)1. NS desires p (“your wish is my command”). 2. Absence of p is considered as violation of A.

! Anderson’s reduction of deontic logic to modal logic.3. NS desires that there are no violations. 4. If violation, then NS is motivated to sanction. 5. NS does not like to sanction. 6. A does not like being sanctioned.7. NS has the power to count absence of p as violation.8. NS has the power to enforce sanction.Example: You are obliged to return the book in 2 weeksQuestion: Why do we need all of them? Is it complete?


32

When to Accept a Contract?

• Decision making of trustor taking profiles of trustee and normative system into account

trustee

trustor$$

normative system


33

Challenge: When to Accept a Norm

• “Act only according to that maxim whereby you can at the same time will that it should become a universal law.”

•Immanuel Kant, Metaphysics of Morals, 1785


34

Emergence of Norms• Don’t do to them, what you don’t want them to

do to you• Social delegation cycle

– Agent desires– Merging: Social goal– Planning: Norm– Acceptance: Agent desires

• I accept if you fulfill the norm on theassumption that all others fulfill it


35

Autonomous Systems

• Legal institutions in context of legal institutions– Norm dynamics described by counts-as norms

• Strong permissions in norm creation (Bulygin)• Global policies about local policies

– Von Wright: transmission of will by nested norms


36

Hierarchical Normative Systems

• Decision making of norm. system 1 taking profiles of norm. system 2 and global authority into account

Norm. 2

Norm. 1$$

Global authority


37

Decision Making in NMAS

Norms are rules used to control or regulate behavior.

Norm conformance and compliance.Multiagent structure of normative systems.Norm acceptance for emergence of norms.

Hierarchical normative systems for autonomy.

Concluding Remarks• Challenges due to uncertainty and imprecision



38

http:\\deonticlogic.org

39

40

Further Reading• Normative multiagent systems proceedings:

– Computational and Mathematical Organization Theory, double special issue on normative multiagent systems, 2006.

– Normative Multiagent Systems, 18.03. - 23.03.2007, Schloss Dagstuhl, Germany, 2007.

– Autonomous Agents and Multiagent Systems Journal, special issue on normative multiagent systems, August 2008.

– 4rd International Workshop on Normative Multiagent Systems (NORMAS2009), Dagstuhl, March 2009

• Introductions to the area and its challenges– Ten philosophical problems in deontic logic– Ten challenges for normative multiagent systems– Ten guidelines for norms in computer science

Ten Philosophical Problems1. How can deontic logic be reconstructed in accord with the philosophical position that

norms are neither true nor false?2. When is a set of norms to be termed `coherent'?3. How can deontic logic accommodate possible conflicts of norms? How can the

resolution of apparent conflicts be semantically modeled?4. How do we reason with contrary-to-duty obligations which are in force only in case of

norm violations?5. How to define dyadic deontic operators with regard to given sets of norms and facts?6. How to distinguish various kinds of permissions and relate them to obligations?7. How can meaning postulates and intermediate terms be modeled in semantics for

deontic logic reasoning?8. How to define counts-as conditionals and relate them to obligations and permissions?9. How to revise a set of regulations or obligations? Does belief revision offer a

satisfactory framework for norm revision?10.Can the belief merging framework deal with the problem of merging sets of norms?

41

J. Hansen, G. Pigozzi and L. van der Torre, Ten philosophical problems in deontic logic, Dagstuhl DROPS proceedings 07122, 2007.

Ten Challenges for NorMAS1. Tools for agents supporting communities in their task of recognizing, creating, and

communicating norms to agents. 2. Tools for agents to simplify normative systems, recognize when norms have

become redundant, and to remove norms. 3. Tools for agents to enforce norms. In a distributed approach, roles should be

defined for agents in charge of monitoring and sanctioning. 4. Tools for agents to preserve their autonomy. 5. Tools for agents to construct organizations. 6. Tools for agents to create intermediate concepts and normative ontology, for

example to decide about normative gaps. 7. Tools for agents to decide about norm conflicts. 8. Tools for agents to voluntarily give up some norm autonomy by allowing automated

norm processing in agent acting and decision making9. Tools for conviviality. 10.Tools for legal responsibility of the agents and their principals.

42

G. Boella, L. van der Torre, H. Verhagen, Introduction to the special issue on normative multiagent systems. Autonomous Agents and MultiAgent Systems, Aug 2008.

Ten Guidelines

43

Guideline 11. Motivate which definition of normative

multiagent system is used.

• Norms explicitly represented in system – (the ‘strong’ interpretation: too strict?)

• Norms explicitly represented in specification– (the ‘weak’ interpretation: too general?)– Norm compliance, norm implementation, ...

• Something else

44

Guideline 22. Make explicit why norms are a kind of (soft)

constraints deserving special analysis.

• Ways to deal with violations, representation of permissive norms, the evolution of norms over time (in deontic logic), the relation between the cognitive abilities of agents and the global properties of norms, how agents can acquire norms, how agents can violate norms, how an agent can be autonomous (in normative agent architectures and decision making), how norms are created by legislator, emerge spontaneously or are negotiated among agents, how norms are enforced, how constitutive or counts-as norms are used to describe institutions, how norms are related to social and legal concepts, how norms structure organizations, how norms coordinate groups and societies, how contracts are related to contract frames & contract law, how legal courts are related, how normative systems interact?

45

Guideline 33. Explain why and how norms can be changed at

runtime.

• E.g., legislators and voting on acceptance, observe behavior and violations to modify…

46

Guideline 44. Discuss the use and role of norms as a

mechanism in a game-theoretic setting.

• D. Lewis “master and slave” game• E. Bulygin “rex, minister and subject” game• G. Boella c.s.: violation games, institutionalized

games, negotiation games, norm creation games, control games

47

Norms are rules specifying violation games.

Guideline 55. Clarify role of norms in the multiagent system.

• Norms guide (…) desired system behavior

• Norms are incentives to motivate agents– Gneezy and Rustichini’s daycare example

• Norms organize systems– Modularity, abstractions

48

Guideline 66. Relate the notion of “norm” to the legal,

social, or moral literature.

Five phases in normas design:1.off-line norm design2.norm representation3.norm manipulation4.social reality5.moral reality

49

Guideline 77. Use norms not only to distinguish right from

wrong, but also to resolve dilemmas, and use norms not only describe violations, but in general to coordinate, organize, guide, regulate or control interaction among agents.

50

Guideline 88. Distinguish norms from obligations, prohibitions

and permissions.

• Deontic logic: logical relations obligations, etc– normative system is typically left implicit

• Two distinct philosophical traditions– Von Wright: norms and normative propositions– Alchourron: prescriptive and descriptive obligations

51

J. Hansen. Imperatives and Deontic Logic. PhD thesis, University of Leipzig, 2008.

Guideline 99. Use the deontic paradoxes only to illustrate the

normative multiagent system.

1.A certain man should go to the assistance of his neighbors,

2. If he goes, he should tell them he is coming3. If he does not go, he should not tell them that he is

coming4.He does not go. {Oa,O(a ! t),¬a ! O(¬t),¬a} {Oa, a ! O(t),¬a ! O(¬t),¬a}

52

Guideline 1010. Consider regulative norms in relation to other

kinds of norms and concepts.

• in relation to permissive norms, constitutive norms, procedural norms, agents, roles, groups, societies, rights, duties, obligations, time, beliefs, desires, intentions, goals, roles, and other kinds of norms and other social-cognitive computer science concepts.

53

The Question• Could (or should) “norms” play a similar role in

computer science like “service”, “contract” or “trust”? – Since the use of norms is a key element of human social

intelligence, norms may be essential too for artificial agents that collaborate with humans, or that are to display behavior comparable to human intelligent behavior.

– Norms are thought to ensure efficiency at the level of the multiagent system whilst respecting individual autonomy.

• We have to build more flexible normative multiagent systems, and test them in practice, before we know where they can be used best.

54

Roles in conceptual modelling

Objects, relationships, and roles

Each modelling language has its own reference ontology.

The duality of objects and relationships is present in all mod-elling languages. E.g., variables and constants vs predicatesand relations.

But there is another basic ontological distinction: roles.

Roles, first appearance

Bachman and Daya 1977’s role DB data model:

“Records and n-tuples are role oriented. These deal withemployees, customers, patients, or students: role types. Thisis in contrast with DB theory: each record should representall aspects of some one entity in the real world. The reasonfor confusion is that neither the roles of the real world northe entities of the real world are a subset of the other”

“If members of a set are records of various types, large piecesof redundant code had to be written in order to addressthe semantically equivalent, but syntactically distinct fields(items) of the members when iterating through a set.

By letting all the entity types involved play the same role andby defining the fields as fields of the role, not of the types, itbecame possible to process all members with the same pieceof code.”

Natural types vs roles

Sowa 1984’s conceptual structures distinguish between natu-ral types “that relate to the essence of the entities” and roletypes “that depend on an accidental relationship to someother entity”: child depends on parent, pet on owner, etc.

Sowa: role types are subtypes of natural types. Child, Pet,Quotient and Food are subtypes of the natural types Person,Animal, Number and Physical-Substance.

Properties of roles in Object Orientation

from Steimann 2000

1. A role comes with its own properties and behavior. Hence,it is a type. E.g., a director of a department commandsother members and makes buy-orders.

2. Roles depend on relationships: e.g., an employee is re-lated to an employer, a student to a school, etc.

3. Objects of unrelated types can play the same role. E.g.,both a person and an organization are customers.

4. An object may play different roles. E.g., a person can bea student and an employee.

5. An object may play the same role several times. E.g., aperson can hold several employments.

6. An object may acquire and abandon roles dynamically.E.g., a person can acquire the role of student or of em-ployee.

7. The sequence in which roles may be acquired and relin-quished can be subject to restrictions. E.g., a personbecomes a teaching assistant only if it is a student.

8. Roles can play roles. E.g., an employee can be a projectleader, a role of the employee.

9. A role can be transferred from one object to another.E.g., the salary of an open position may be specifiedindependently of the person that will be employed.

10. The state of an object can vary depending on the role inwhich it is being addressed: this should be viewed as aseparate instance of the object. E.g., an employee hasan address per job and also a private one.

11. If an object plays several of roles simultaneously, it re-sponds according to the role in which it is being ad-dressed. E.g., an person gives the address of the em-ployee it is playing.

12. Roles restrict access. This corresponds to an object hav-ing different perspectives, facets, or aspects. E.g., theprivate phone number of an employee can be invisiblewhen the person is playing the employee role.

13. Different roles may share structure and behavior: roledefinitions inherit from each other. E.g., the role firstyear student can inherit the behavior of giving exams fromthe role student.

14. An object and its roles share identity. Since roles do notexist by themselves they cannot have an identity.

15. An object and its roles have different identities. E.g., thenumber of passengers of an airline can be greater thanthe number of person who travelled with it.

Possible views of roles

1. Roles as named places in relationships.

2. Roles as specifications or generalizations of natural types.

3. Roles as adjunct instances.

Role as a named place in a relationship

If two places of a relationship are of the same type, thenthe type would not suffice to distinguish them. Role namesshould serve to identify the places.

Every place of a relationship has a role name in the ER model(and UML).

For example, the is-parent-of (or is-child-of ) relationship issufficiently characterized by the role names parent and child.

However, it fails to account for the fact that roles come withtheir own properties and behavior.

Roles as specializations

If roles are types, how are the roles and the natural typesrelated?

A role type is more specific than the natural type of its roleplayers: if Customer and Supplier are subtypes of Person.But, not only persons, but also organizations like companies,etc., can be customers and suppliers. However, declaringCustomer and Supplier as subtypes of both Person and Or-ganization is a problem.

Roles as generalizations

If roles are not subtypes, could they be supertypes?

No, if roles are more specific than the types of their players.

But this is false, anyway: a person has many properties notrequired of a customer or supplier. Rather, being a customeror supplier imposes its required properties on persons andorganizations, making the former supertypes of the latter.

From the extensional point of view, roles are supertypes stat-ically, while dynamically they are subtypes.

Roles as adjunct instances

Roles as independent types the instances of which are carriersof role specific state and behavior, but not identity.

An object and its roles are related by a special played-by rela-tion, and an object and its roles form an aggregate (subject).

Picking up a role corresponds to the creation of a new in-stance of the corresponding role type and its integration intothe compound.

Ontological properties of roles

Following Sowa, Guarino and Welty 2002 presented an onto-logical distinction between role and natural types.

A role is a concept that is:

• founded: the individuals of a role stand in relation toother individuals,

• lacks semantic rigidity: they can enter and leave the ex-tent of the concept without losing their identity.

A natural type, on the other hand, is characterized by:

• semantic rigidity: an individual of a natural type cannotdrop this type without losing its identity,

• lack of foundation: for an individual to belong to thattype it is not required that it stands in relationship toothers.

E.g., Person is a natural type, Student is a role.

Adolescent and Adult are states or phases, lacking both se-mantic rigidity and foundation.

Dependence of roles

Fine 1995’s notion of dependence: “to say that an object x

depends upon an F is to say that an F will be ineliminablyinvolved in any definition of x”.

Masolo et al. 2004’s definitional dependence: e.g., the def-inition of the concept of student makes reference not to aspecific school but to the concept of school, etc.

Handbook Norms Change Permission Institution Agency

Deontic Logic as Normative Reasoning

Leon van der Torre

Individual and Collective Reasoning GroupComputer Science and Communication

University of Luxembourg

EASSS 2012 (Valencia)

May 2012

Leon van der Torre Deontic Logic as Normative Reasoning


1 Handbook

2 Problem 1 - In what sense are obligations different from norms?

3 Problem 2 - How do norms change?

4 Problem 3 - How to relate various kinds of permissions?

5 Problem 4 - What is the role of constitutive norms?

6 Problem 5 - Where does deontic logic meet game theory?



Volume 1: Deontic logic and imperatives

A. BACKGROUND1. R. Hilpinen and P. McNamara, Deontic logic: history, overview.2. J. Hansen, Imperative logic: history, overview.B. DEONTIC PROBLEMS3. S. Hansson, Varieties of permission.4. M. Sergot, Contrary-to-duty norms.5. L. Goble, Prima facie norms, normative conflicts and dilemmas.6. M. Sergot, The theory of normative positions.7. A.J.I. Jones and D. Grossi, Constitutive norms and counts as conditionals.8. P. McNamara, Logics for supererogation and allied normative concepts.C. CANDIDATES FOR A NEW STANDARD9. S. Hansson, Alternative semantics for deontic logic.10. X. Parent and L. van der Torre, Input/output logic.11. J. Hansen, Imperativist approach to deontic logic.

12. L. Lindahl and J. Odelstad, TJS. Normative systems with intermediaries.



Volume 2: Norms in systems, and interactions

D. DYNAMICS1 ?, Time and deontic logic. 2. J. Broersen, J. Horty and J.-J. Ch. Meyer,Actions and deontic logic. 3. G. Boella, G. Pigozzi and L. van der Torre,Normative system change. 4. J. van Benthem and F. Liu, Deontic logic andchanging preferences. 5. D. Gabbay, Reactive approaches.E. INTERACTIONS6 R. Thomason, The formalization of practical reasoning: problems andprospects. 7 B. Kooi and A. Tamminga, Deontic logic and game theory. 8 K.van Fintel and T. Gillies, Deontic logic and natural language. 9 G. Sartor, G.Governatori, and A. Rotolo. Deontic logic and legal reasoning.F. INTEGRATION INTO SYSTEMS

10. D. Garg, Modalities for access control. 11. J. Lobo, E. Lupu and A. Russo,

Distributed systems management. 12. F. Cuppens and N. Cuppens-Boulahia,

Obligations for usage control. 13. G. Aucher and G. Boella, Privacy and

epistemic obligations. 14. R. Parikh, L. Oldeloohuisl, and C. Baskent,

Epistemic norms. 15. G. Governatori, Rule, Norms and compliance.



In what sense are obligations different from norms?

BQ Can norms have truth values? (Jorgensen, 1938) Imperatives,promises, legal statutes, and moral standards are usually notviewed as being true or false. E.g.: “John, leave the room!”

BQ Is a logic of norms possible? Without truth, norms cannot:be premise or conclusion in an inference,be termed consistent or contradictory, orbe compounded by truth-functional operators.

BQ Can deontic logic be built on descriptive obligations only?

(Makinson, 1998) turns the Bad Questions into a Good Question:

Problem 1. How can deontic logic be reconstructed in accord withthe philosophical position that norms are neither true nor false?

Explained in following slides in three steps.



Step 1: Traditional Deontic Logic, axiomatic (1951)


SDL = normal modal logic of type KDextends the propositional tautologies with the axiomsK : O(x → y) → (Ox → Oy) and D : ¬(Ox ∧ O¬x), closedunder modus ponens x , x → y/y and Necessitation x/OxProhibition & permission defined Fx = O¬x and Px = ¬O¬x .

BQ Which representation for conditional norms O(x |a)?O(x |a) = O(a → x), orO(x |a) = a → Ox , or something else

BQ Factual or deontic detachment?Factual detachment: (O(x |a) ∧ a) → Ox

Deontic detachment: (O(x |a) ∧ Oa) → Ox



Step 2: Traditional Deontic Logic, possible worlds (1969)


DSDL3 = dyadic modal logicKripke models �W ,≥� where ≥ is total pre-orderO(x |a) iff x holds in all the most preferred a worldsAxiomatized by Lewis (counterfactuals, 1973) and Spohn

BQ Which representation for conditional norms O(x |a)?BQ Which logical relations hold among conditional norms?

BQ Factual or deontic detachment?Factual detachment: (O(x |a) ∧ a) → Ox

Deontic detachment: (O(x |a) ∧ Oa) → Ox

BQ How are logical relations and detachment related?For example, transitivity and deontic detachment



Step 3: Modern Deontic Logic, iterative (1998)


Makinson’s iterative detachment approachNormative code N is pairs of propositional formulas (a, x)Gross output: out(N,A) = apply N as rules on A

E.g., E0 = ∅,En+1 = En ∪ {x |(a, x) ∈ N,En ∪ A implies a}Net output: consistency constraint for output

E.g., En+1 = En ∪ {x |(a, x) ∈ N,En ∪ A implies a} if cons.En otherwise

BQ Which representation for conditional norms O(x |a)?GQ How to detach obligations from conditional norms?

Factual detachment: ((a, x) ∧ a) → Ox

Deontic detachment: ((a, x) ∧ Oa) → Ox

BQ How are logical relations and detachment related?



Candidates for a New Standard

Makinson’s iterative detachment approachGQ Which approaches do not fit into this general framework?GQ What are the alternatives to this iterative framework?

Reactive

Algebraic

Dynamic

q /\ ¬V(n) ! p

Diagnostic

Non-Monotonic

Programming

Op!p,Oq!q ! O(p/\q)!p,!q

Labeled

Iterative

Imperativistic

Input/Output

K O

!p,!q ! O(p/\q)

a:b/Oc

a in out(C,b)



Problem 1 - Some Good Questions

Makinson’s iterative detachment approach

GQ When is a normative system coherent?

GQ When is a norm redundant?

GQ When are two normative systems equivalent?

GQ How are the three questions above related?

GQ Which meta-logical properties can or should hold?Law2case principleStrong case2law principle

GQ How to prevent the pragmatic odditywithout creating the drowning problem?



Problem 2 - How do norms change?




Problem 1 - In what sense are obligations different from norms?

GQ How is obligation change different from norm change?How to formalize the relation between them?

Obligations can change while normative system remains sameDetachment of obligation, delegation, etcA relatively clear and well studied subject (in 70s and 80s)

Norm change: the new challenge (workshops 2007, 2010)Norm revision and contraction, e.g. change of legal codeNorm evolution, e.g. change of social normsMerging normative systems, e.g. merger of companies




Problem 2.1. How to revise a set of norms?

Alchourron & Makinson [1981] study changes of legal code:additionamendment: revisionderogation: contraction

Legal code is a non-empty and finite set of propositions.

Starting point of belief revision (with Gardenfors), a.k.a. AGM

AGM 1985: minimality represented by recovery postulateMore advanced variants, e.g. for iterated belief revisionAlso some issues with prioritized norms we do not consider

Defeasible deontic logic derived from non-monotonic logic



AGM 85

C. E. Alchourrón, P. Gärdenfors, and D. Makinson. On the logic of theory change: Partial

meet contraction and revision functions. J. Symb. Log., 50(2):510–530, 1985. Leon van der Torre Deontic Logic as Normative Reasoning


AGM 85

C. E. Alchourrón, P. Gärdenfors, and D. Makinson. On the logic of theory change: Partial

meet contraction and revision functions. J. Symb. Log., 50(2):510–530, 1985.



Problem 2.1 - How to revise a set of norms?

BQ Do the AGM postulates hold for norm revision?E.g., success postulate criticized in belief revision

GQ Does AGM offer a satisfactory framework for norm revision?Role of minimal change in norm changeRevision of conditional norms

Example

If we have {(�, a), (a, b)} and we have that c is an exception tothe obligation to do b, then we need to have b �∈ out(N, c). Twosolutions seem to be {(¬c , a), (a, b)} or {(�, a), (a ∧ ¬c , b)}.

GQ What triggers the change of a norm?AGM says a new or removed norm

GQ Do general patterns in the revision of norms exist?If so, how to formalize them?

GQ Derogation is contraction? How about annulment?




GQ How to formalize the evolution of (social) norms?E.g., norm about phone calls during meetingsNo persons who change the norm like in legal codeChange of social norm triggered by norm violationsSocial delegation cycle: relates agent desires and social goals

GQ How to merge sets of norms?Problem only recently addressed in the literature.

GQ What is norm merging?1 Merging norms that belong to the same normative system2 Merging norms that belong to different systems



Merging norms: same normative system

Example (Cholvy & Cuppens 1999)

An organization that works with secret documents has two rules.R1 = It is obligatory that any document containing some secretinformation is kept in a safe, when nobody is using this document.R2 = If nobody has used a given document for five years, then it isobligatory to destroy this document by burning it.

To derive a contradiction we need to add an IC that keeping adocument and destroying it are contradictory actions

⇒ The notion of coherence between norms can involveinformation other than norms.

Is this a genuine problem of merging norms?



Merging norms: different systems

Example (Gregoire, 2004)

The Belgian-French bilateral agreement preventing double taxation:Resident in B ∧¬ France State Worker ⇒ Taxable in BResident in B ∧¬ Work in France ⇒ Taxable in B

Belief merging would simply give a set containing the two initialrules. But if someone is not a State worker in France (but works inFrance), we would get contradictory requirements from theBelgium and the France parties.



Problem 2.3 - How to merge sets of norms?

Belief merging (a.k.a. belief fusion) is a ‘generalization’ of beliefrevision. Belief merging dedicated to the combination ofinformation coming from different (conflicting) sources.

GQ Can belief merging offer a satisfactory framework for norms?

GQ Do distinctions in belief merging correspond to something?Belief merging distinguishes majoritarian and egalitarianoperators.They try to capture the intuitions that often guide theaggregation of individual preferences into a social one.These intuitions seem to have nothing to say when we try tomodel the merging of sets of norms.

GQ What are alternative frameworks for norm merging?



Problem 3 - Permissive norms

BQ How to solve the following problem with permissions:

Example (Ross’ paradox)

Px implies P(x ∨ y), e.g., if a person is permitted to smoke, he isalso permitted to smoke or kill.

BQ How to formalize free choice permissions:

Example (Free choice permission)

P(x ∨ y) implies Px ∧ Py , e.g., “You may either sleep on thesofa-bed or sleep on the guest room bed” implies “You may sleepon the sofa-bed and you may sleep on the guest room bed”.




BQ Is permission unilateral (Op → Pp) or bilateral (Pp → ¬Op)?BQ What is difference between weak (negative, tacit) and strong

(positive, implicit and explicit) permission?von Wright (1951): Permission is dual of obligation

Px =def ¬O¬x (weak permission)

Negative permission defined without permissive norms

Px =def ¬x �∈ out(N,A) (weak permission)

BQ Are permissive norms necessary for normative systems?

Example (Bulygin 1986)

Permissive norms are needed when there multiple authorities. If ahigher authority permits you, a lower authority cannot prohibit it.




GQ What is the difference between permission and authorization?

Example (Makinson1986)

A priest may be authorized to marry people, in the sense that hisactions may have that effect, without being permitted to do so.

GQ What is a right?

Example

If you have the right to enter, then a guard must let you in.

Concept of permission is multi-faceted.

Problem 3. How to relate various kinds of permissions and relatethem to obligations?




GQ How to formalize relation permissions and permissive norms?

Something is positively permitted iff the code explicitlypresents it. However, permissions follow from explicit ones.

GQ How do permissive norms influence obligations?Obligations imply permissions, but how about the other way?

GQ What is role of permissive norms in norm change / merging?




Example

Consider a set G consisting of the norm (work, tax), and a set Pconsisting of (18y, vote). Does it follow from our mini-code thatvoting is permitted on condition of being employed?

In one sense yes: there is nothing in the code that forbids it(negative permission).In another sense no: a person may be employed at the age of17 and not covered by the explicit permission (positive staticpermission).But in another sense, yes again. For if we were to forbid aperson to vote on the condition of being employed we wouldbe creating an incoherence in the code, in its application topeople who are both employed and 18 or over (dynamicpositive permission).




Kinds of permissions used in different circumstances:

Negative permissions seem to answer to the needs of thecitizen, who needs to now that his action is not forbidden.

Since it’s difficult to establish these kinds of permissions (alsobecause of ambiguities in the code), positive staticpermissions seem to suit better the need to know what isexplicitly permitted.

Lastly, dynamic permissions are the answers to the need of thelegislator, who need to anticipate the effect of changing anexisting corpus of norms by adding a prohibition.



Problem 4 - What is the role of constitutive norms?

Normative system contains definitions, expressions of purposes,conceptual rules, etc.: we call them constitutive norms.

Example

“An act of theft is punished by a prison sentence not exceeding 5years or a fine.”

GQ What is the role of constitutive norms? (What can’t we do ifwe have only regulative and permissive norms)

Meaning postulates and intermediate conceptsCreation of social reality, counts-as conditionalsLegal and institutional powers of agentsDefining the way normative systems can changeInterpretation issues. . .



Example meaning postulates and intermediate concepts

“An act of theft is punished by a prison sentence not exceeding 5years or a fine.”

“Someone commits an act of theft if that person has taken amovable object from the possession of another person into hisown possession with the intention to own it, and if the actoccurred without the consent of the other person or someother legal authorization.”

A person in the sense of the law is a human being that hasbeen born.A movable object is any physical object that is not a person ora piece of land.A movable object is in the possession of a person if thatperson is able to control its uses and location.The owner of an object is within the limits of the law entitledto do with it whatever he wants, namely keep it, use it,transfer possession or ownership of the object to anotherperson, and destroy or abandon it.



Meaning postulates and intermediate concepts

Intermediate concepts link legal terms to terms that describenatural facts.

GQ Why are there intermediate concepts?Wedberg & Ross (1950s): a legal term like ’ownership’ (x isthe owner of y at time t) serves the purpose of economy ofexpression of a set of legal rules.To regulate normative change and legal interpretation by judgeTo resolve conflicts



Problem 4 - Meaning postulates and intermediate concepts

Example (Intermediates to resolve conflicts)

Suppose that (�,¬dog) forbids dogs on the premisesThere is also a higher order principle that no blind person may berequired to give up her guide dog. The judge cannot change thestatute. What he can do is to conclude that the statute does notapply to guide dogs. So the statute must be re-interpreted asreading (�,¬tdog) with the additional intermediate(dog ∧ ¬guidedog, tdog) ∈ T , and thus no conflict arises for thecase of blind persons that want to keep their guide dog.

GQ What is the exact process of creating and modifyingtheoretical terms in order to resolve conflicts?



Problem 4 - Constitutive norms

GQ How do regulative, permissive and constitutive norms interact?



Problem 4 - Constitutive norms

There is no consensus on the logic of counts-as conditionals,probably due to the fact that the concept is not studied indepth yet.

For example, the adoption of the transitivity rule T for Jones& Sergot’s logic is criticized by Artosi, Rotolo & Vida: Theyclaim that non-monotonicity is a crucial feature of count-asconditionals:

Example: Suppose that in an auction if the agent x raises onehand, this may count as making a bid. It is clear that thisdoes not hold if x raises one hand and scratches his own head.



Problem 5 - Where does deontic logic meet game theory?


Where Does Deontic Logic Meet Game Theory?

Leon van der Torre EASSS 2012

1

Why do we need deontic detachment?

•  Thomason 1980: – Context of justification (e.g. a judge, looking back) – Context of deliberation (normative decision making)

•  Consider Chisholm’s paradox without DD – Day after tomorrow you must help neighbors – Tomorrow you must tell

•  S.O. Hansson: If you don’t tell & you help, then you never received “moral cue for action” to tell

2 R. Thomason, “Deontic logic as founded on tense logic." In New Studies in Deontic Logic, R. Hilpinen, ed., D. Reidel, Dordrecht, 1981, pp. 165-176.

Quick History Practical Reasoning •  Aristotle: theoretical & practical reasoning •  40s Practical Reas. splits into two branches

– Classical decision and game theory in economics – Bounded rationality and means-end reasoning in AI

•  90s unification in multi-agent systems – Russell and Norvig’s “modern approach” to AI – Shoham’s DTGT foundations of MAS

•  Deontic logic: obligations treated as goals in AI – Distinguished from theory of normative systems

3

NORMAS •  Deontic logic and normative reasoning are one •  They should be studied both from a bounded

rationality or AI perspective, and from a DTGT or economic perspective: they should be studied from a multi-agent perspective – The DL handbook: “a modern approach”

•  Consequences: – Action and time: DTGT model – Normative decision theory: BOID

4

NORMAS •  State of art in action and time Deontic Logic •  Same situation as in preference-based DL:

– Semantics is relatively clear and intuitive – Languages and proof systems are to be improved

•  All you need to know action and time in DL is in Horty’s “Agency and Deontic Logic” (2001)

5 J. Broersen, L. van der Torre: What an Agent Ought To Do. Artificial Intelligence and Law 11(1): 45-61 (2003)

DTGT vs Deontic Logic •  Models are the same, choices over branching

time, with only one distinction: – DTGT: each agent has its own utility – DL: there is a single global utility

•  Savage’s sure thing principle plays central role •  There exist more detailed DTGT models and

we can use them as more detailed DL models

6

The Gambling Problem •  Obligation to be and obligation to do

– Not interdefinable •  Granularity of indeterministic actions

7

The Driving Example •  Assume a collision •  Dominance act utilitarianism

– There was no dominant action •  Orthodox perspective on the agent's ought

– You should have changed lanes •  The issue of moral luck (Horty, 2001, p.121).

– Orthodox ought: agent looks back in time – Dominant: agent may regret, but not be blamed

•  What if utility of 5 and 3? (Whiff and Poof) 8

Procrastinate's Choice •  Field = agent’s reasoning is limited to subtree

–  “Intention” in BDI agent theory? •  Possibilist interpretation

–  It is possible to write the review, so he accept it •  Strategic interpretation

– He will not write it, so he should not accept it

9

A Challenge •  The bystander effect:

– One person watches a child in water and saves it – One hundred people watch child and it drowns

•  How to reason about this in deontic logic? –  If deontic logic is going to play a role in ethics, it has

to say something about examples like this

10

A Challenge •  The bystander effect:

– One person watches a child in water and saves it – One hundred people watch child and it drowns

•  How to reason about this in deontic logic? –  If deontic logic is going to play a role in ethics, it has

to say something about examples like this •  Introduce normative systems in DTGT model

– Needs an explicit mechanism of norm creation

11

Summary •  How to combine deontic logic & game theory?

– Mechanism design, norm is a mechanism –  Introduce norms in Horty’s model

•  Computational methods for deontic logic – Algorithms, complexity, proof systems

12

T11. Normative multi-agent systems

Education

Transcript of T11. Normative multi-agent systems