ABM: Decision making

Dr Andy Evans

Thinking in AI

Agent based systems and other AI can contain standard maths etc.

But their real power comes from replicating how we act in the real world: assessing situations, reasoning about them, making decisions, and then applying rules.

Reasoning: “if a café contains food, and food removes hunger, a café removes hunger”Rules: “if my hunger is high, I should go to a café”

Reasoning

Reasoning is the remit of “brain in a box” AI.Useful for:

Developing rulesets in AI.Interpreting requests from people (Natural Language Processing).Developing new knowledge and replicating sentience.

ReasoningProgramming languages developed in the late 60’s / early 70’s offered the promise of logical reasoning (Planner; Prolog).These allow the manipulation of assertions about the world:

“man is mortal” and “Socrates is a man” leads to “Socrates is mortal”

Assertions are usually “triples” of subject-predicate [relationship]-object.

There are interfaces for connecting Prolog and Javahttp://en.wikipedia.org/wiki/Prolog#Interfaces_to_other_languages

Reasoning

This led to SHRDLU (MIT), which could do basic natural language parsing and responses, based on a limited knowledge of the world.SHRDLU, however, was very limited.

Need more knowledge about the world.Need, therefore, to get everyone involved in uploading knowledge.Led to projects like Cyc / OpenCyc (1984+; 306,000 facts) and Open Mind Common Sense (1999+; 1,000,000+ facts).

Reasoning

An obvious source for information is the web.

However, parsing natural language is not easy – a lot of the semantic content (meaning) relies on context.

It would be easier if language was embedded in an ontology (structured collection of knowledge). For example “death” [the event] was marked up as different from “Death” [the medieval figure]

Obvious similarities to metadata – standards for which (eg Dublin Core) led to work on structuring knowledge.

Semantic Web

1990’s Tim Berners-Lee and others pushed for a “Semantic Web” marked up for meaning in context.

1999 Resource Description Framework (RDF) – way of marking up web content so it relates to a framework of triples defining the content. Very general.

Used to identify eg. initial Really Simple Syndication (RSS) feeds (now not RDF).

But, not very specific, needs another language to extend it.

DARPA Agent Markup Language

1999 DARPA (US Military Research) wanted a machine-readable knowledge format for agents, that could be used over the web.

Developed a RDF extension for this which led to the Web Ontology Language (OWL).

This can be used to build more specific RDF ontologies.http://www.schemaweb.info/schema/BrowseSchema.aspx

Example: Friend of a Friend (FOAF)

Issues

Easy to build self-contained ontologies, but what happens when the same term is used by different groups. How do we resolve conflicts?

Automated reasoning about knowledge is still not great because ontologies don’t capture the estimation and flexibility involved in real reasoning (eg. accepting paradoxes).

By concentrating on formal definitions rather than experience they tend to be circular. Alternative: Folksonomies, but difficult in modelling.

Hard to move from knowledge to action. For this we need rules for when to act, and behaviours.

Thinking for agents

Building up rulesets is somewhat easier.

Though it is harder to represent them in a coherent fashion as there’s an infinite number of things one might do.

While many standards for representing agents’ states, few for representing rules, and most very very limited.

Usual for these to just be encoded into a programming language.

Rulesets

Most rules are condition-state-action like:

“if hunger is high go to café”

Normally there’d be a hunger state variable, given some value, and a series of thresholds.

A simple agent would look at the state variable and implement or not-implement the associated rule.

How do we decide actions?

Ok to have condition-state-action rules like:“if hunger is high go to café”

And“if tiredness is high go to bed”

But how do we decide which rule should be enacted if we have both?How do real people choose?

Picking rulesOne simple decision making process is to randomly choose.

Another is to weight the rules and pick the rule with the highest weight.

Roulette Wheel picking weights rules then picks probabilistically based on the weights using Monte Carlo sampling.

How do we pick the weights? Calibration? Do we adjust them with experience? For example, with a GA?

We may try and model specific cognitive biases:http://en.wikipedia.org/wiki/List_of_cognitive_biases

Anchoring and adjustment: pick an educated or constrained guess at likelihoods or behaviour and adjust from that based on evidence.

Reality is fuzzy

Alternatively we may wish to hedge our bets and run several rules.This is especially the case as rules tend to be binary (run / don’t run) yet the world isn’t always like this.

Say we have two rules:if hot open windowif cold close window

How is “hot”? 30 degrees? 40 degrees?

Language isn’t usually precise…We often mix rules (e.g. open the window slightly).

Fuzzy Sets and Logic

Fuzzy Sets let us say something is 90% “one thing” and 10% “another”, without being illogical.

Fuzzy Logic then lets us use this in rules:E.g. it’s 90% “right” to do something, so I’ll do it 90% - opening a window, for example.

Fuzzy SetsWe give things a degree of membership between 0 and 1 in several sets (to a combined total of 1). We then label these sets using human terms.Encapsulates terms with no consensus definition, but we might use surveys to define them.

Degrees

Degre

e o

f m

em

bers

hip

0

1

40

Cold Hot0.5

20

Membershipfunction

17° = 15% cold + 85% hot

Fuzzy Logic models

We give our variables membership functions, and express the variables as nouns (“length”, “temperature”) or adjectives (“long”, “hot”).

We can then build up linguistic equations (“IF length long, AND temperature hot, THEN openWindow”).

Actions then based on conversion schemes for converting from fuzzy percentages of inputs to membership functions of outputs.

Bayesian Networks

Of course, it may be that we see people in one state, and their actions, but have no way of replicating the rulesets in human language. In this case, we can generate a Bayesian Network. These gives probabilities that states will occur together.This can be interpreted as “if A then B”.They allow you to update the probabilities on new evidence.They allow you to chain these “rules” together to make inferences.

Bayesian Networks

In a Bayesian Network the states are linked by probabilities, so:

If A then B; if B then C; if C then D

Not only this, but this can be updated when an event A happens, propagating the new probabilities by using the new final probability of B to recalculate the probability of C, etc.

Expert Systems

All these elements may be brought together in an “Expert System”.These are decision trees, in which rules and probabilities link states. Forward chaining: you input states and the system runs through the rules to suggest a most useful scenario of action.Backward chaining: you input goals, and the system tells you the states you need to achieve to get there.Don’t have to use Fuzzy Sets or Bayesian probabilities, but often do.

How do we have confidence in our reasoning?

Expert systems may allow you to assess how confident you are that a rule should be applied, though it isn’t always clear how confidences add up.For example:

“Man is mortal” (confidence 0.99)“Socrates is a man” (confidence 0.5)

Final confidence that “Socrates is mortal” = ?

Dempster-Shafer theory allows us to deal with the confidence in an event is happening (assigns a confidence to each potential event totalling one for all possible events, and allows us to assess multiple groups of evidence.)

Picking rules

However, ideally we want a cognitive framework to embed rule-choice within.

Something that embeds decision making within a wider model of thought and existence.

Belief-Desire-Intention

We need some kind of reasoning architecture that allows the agents to decide or be driven to decisions.Most famous is the Belief-Desire-Intention model.

Beliefs – facts about the world (which can be rules).Desires – things the agent wants to do / happen.Intentions – actions the agent has chosen, usually from a set of plans.

Driven by Events, which cascade a series of changes.

Decision making

BDI decisions are usually made by assuming a utility function. This might include:

whichever desire is most important wins;whichever plan achieves most desires;whichever plan is most likely to succeed;whichever plan does the above, after testing in multiple

situations;whichever a community of agents decide on (eg

by voting)

Desires are goals, rather than more dynamic drivers.

The PECS model

Similar model is PECS – more sophisticated as it includes internal drivers:Physis – physical states Emotional – emotional statesCognitive – facts about the world Social status – position within society etc.On the basis of these, the agent plans and picks a behaviour.Ultimately, though, these are decided between by a weighted utility function.

Thinking for agents

Ultimately we have to trade off complexity of reasoning against speed of processing.

On the one hand, behaviour developed by a GA/GP would be dumb, but fast (which is why it is used to control agents in games).

On the other, full cognitive architecture systems like Soar, CLARION, and Adaptive Control of Thought—Rational (ACT-R) are still not perfect, and take a great deal of time to set up.

Further reading

Michael Wooldridge (2009) “An Introduction to MultiAgent Systems” Wiley (2nd Edition)

MAS architectures, BDI etc.

Stuart Russell and Peter Norvig (2010) “Artificial Intelligence: A Modern Approach” Prentice Hall (3rd Edition)

Neural nets, Language Processing, etc.