Intelligent Agents – state of the art – Materials collected at: Americas School on Agents and...

Intelligent Agents– state of the art –

Materials collected at:

Americas School on Agents and Multi-agent Systems(University of Southern California)

06.01. – 11.01.2002

Aleksander Pivk 05.02.2002

69th Solomon Seminar

2/58

What is an agent?

• The main point about agents is that they are autonomous: capable of acting independently, exhibiting control over their internal state.

• Thus: an agent is a computer system capable of autonomous action in some environment.

SYSTEM

ENVIRONMENT

outputinput


3/58

What is an agent?

• Trivial (non-interesting) agents: – thermostat

• An intelligent agent is a computer system capable of flexible autonomous action in some environment.

By flexible, we mean:– reactive;– pro-active;– social.


4/58

Intelligent Agents and AI

• – A little intelligence goes a long way! – • Oren Etzioni, speaking of commercial experience of

NETBOT, Inc.:

• Microsoft Office Assistant

We made our agents dumber and dumber…until finally they made some money.


5/58

Purely Reactive Agents

• Some agents decide what to do without reference to their history – they base their making decision entirely on the present, with no reference at all to the past.

• We call such agents purely reactive:action: S A

• A thermostat is a purely reactive agent:

action(s)= off; if s=temperature OKon; otherwise


6/58

Perception

• Introduce the perception system:

• The see function is the agent’s ability to observe its env., whereas the action function represents the agent’s decision making process.

• New functions:see: S P {maps environment states to percepts}

action: P* A {maps sequences of percepts to actions}

ENVIRONMENT

outputinput AGENT

see action


7/58

Agents with State

• Lets consider agents that maintain state:

• Have some internal data structure, used to record inf. about the env. state and history.

• Let I be the set of all internal states of the agent.• Functions:

see: S P {maps environment states to percepts}action: I A {maps from internal states to actions}next: I P I {maps an internal states and percept to

IS}

ENVIRONMENT

outputinput

AGENTsee action

next state


8/58

Application & Research Domains

• Learning Agents

• Embodied Agents

• Logics for Agents

• Coordination, Cooperation, Collaboration

• Market-based Multi-agent Systems


9/58

Learning Agents• Why should agents learn?

– Learning user and world models, action-to-utility mappings, problem solving

• Learning modalities– from users (observation, interaction, being told) – from other agents (collaborative filtering, from experts)– from experience (supervised, reinforcement, probabilistic models)

• Learning techniques– neural/decision networks, decision trees,reinforcement learning,

instance based learning

• Assistant agents (work effort, productivity)


10/58

Learning for Information Agents

• Information agents – Access information from a variety of data sources

– Integrate the data from these sources

– Monitor and provide notifications

• Technical challenges– Turning semi-structured data into structured data

– Ensuring continued access to the data

– Resolving naming inconsistencies across sources

– Building agents that efficiently execute their tasks


11/58

Agent

WorldGovernments

NATO Members

19951996

CIA World Factbook

1997

Country Information Agent


12/58

Learned Flight Delay Predictor

Historical FlightData

Historical WeatherData

Yahoo Weather

Prediction

Agent

Flight Delay Prediction Agent


13/58

New Listing: 3br 2bath200K

Send EmailNotification

Real Estate Notification Agent


14/58

Travel Planning Agent


15/58

Wrapper Induction

Problem description:• Web sources present data in human-readable format

– take user query– apply it to data base– present results in “template” HTML page

• To integrate data from multiple sources, one must first extract relevant information from Web pages

• Task: learn extraction rules based on labeled examples– Hand-writing rules is tedious, error prone, and time consuming


16/58

Example of Extraction Task

NAME Casablanca Restaurant

STREET 220 Lincoln Boulevard

CITY Venice

PHONE (310) 392-5751

17/58

WIEN [Kushmerick et al ‘97, ‘00]

• Assumes items are always in fixed, known order– … Name: J. Doe; 1 Main; 111-1111. Name: E. Poe; …

• Introduces several types of wrappers

– LR:

• Advantages:– Fast to learn & extract

• Drawbacks: – Cannot handle permutations and missing items

– Must label entire page

PhoneName AddrName: ; : . ; :


18/58

SoftMealy [Hsu & Dung, ‘98]

• Learns a transducer

• Advantages:– Also learns order of items

– Allows item permutations & missing items

– Uses wildcards (eg, Number, AllCaps, etc)

• Drawback: – Must “see” all possible permutations

Phone

Addr

NameName:

Phone: Phone:

Addr:

;

;

;

.


19/58

WHIRL Wrappers [Cohen ’99]

• Learns underlying HTML template– WHIRL “soft logic” to measure document similarity

• Name: html_table_tr_td

• Address: html_table_tr_td_td

• Advantages:– Learns from unlabeled data

– Explicitly exploits HTML structure

• Disadvantage:– Not as expressive as previous ones

– Works only at the level of “HTML nodes”


20/58

STALKER [Muslea et al, ’98 ’99 ’01]

• Hierarchical wrapper induction– Decomposes a hard problem in several easier ones

– Extracts items independently of each other

– Each rule is a finite automaton

• Advantages:– Powerful extraction language (eg, embedded list)

– One hard-to-extract item does not affect others

• Disadvantage:– Does not exploit item order (sometimes may help)

21/58

Extraction Rules

Extraction rule: sequence of landmarks

Name: Joel’s Phone: (310) 777-1111 Review: …

SkipTo(Phone) SkipTo() SkipTo()

22/58

Start: EITHER SkipTo( Phone : ) OR SkipTo( Phone ) SkipTo(: )

More about Extraction Rules

Name: Kim’s Phone (toll free) : (800) 757-1111 …

Name: Joel’s Phone: (310) 777-1111 Review: …

Name: Kim’s Phone: (888) 111-1111 Review: …


23/58

InductiveLearningSystem

ExtractionRules

EC Tree

Labeled Pages

Learning the Extraction Rules

GUI

24/58

Training Examples: Name: Del Taco Phone (toll free) : ( 800 ) 123-4567 Cuisine ...

Name: Burger King Phone : ( 310 ) 987-9876 Cuisine: …

Initial candidate: SkipTo( ( )

Example of Rule Induction

25/58

Initial candidate: SkipTo( ( )

… SkipTo(Phone) SkipTo(:) SkipTo( ( ) ...

SkipTo( ( ) ... SkipTo(Phone) SkipTo( ( ) ... SkipTo(:) SkipTo(()

Training Examples: Name: Del Taco Phone (toll free) : ( 800 ) 123-4567 Cuisine ...

Name: Burger King Phone : ( 310 ) 987-9876 Cuisine: …

Example of Rule Induction

26/58

Multi-view Learning

Name: KFC Phone: (310) 111-1111 Review: Fried chicken …

SkipTo( Phone: ) BackTo( ( Number ) )

Two ways to find start of the phone number:


27/58

RULE 1 RULE 2

+-

Unlabeled data

++-

-

+-

Labeled data

Co-Testing

28/58

Name: Joel’s Phone: (310) 777-1111 Review: ...

SkipTo( Phone: )

Name: Kim’s Phone: (213) 757-1111 Review: ...

Name: Chez Jean Phone: (310) 666-1111 Review: ...

Name: Burger King Phone: (818) 789-1211 Review: ...

Name: Café del Rey Phone: (310) 111-1111 Review: ...

Name: KFC Phone: (800) 111-7171 Review:...

BackTo( (Number) )

Co-Testing for Wrapper Induction

Embodied Agents


30/58

Embodied Agents

• Animated agent research integrates:– Artistic animation– Computer graphics– Intelligent agents

• Why build animated agents?– For more effective communication– For artistic effect– As models for robotic or human agents– When behavior cannot be scripted

• e.g., due to interactions with people and other agents

• In agents, we begin to see dynamic models of thought and action


31/58

Animated Pedagogical Agents

• Animated characters that:– Interact with students in learning envs.– Help keep learning on track– Act as guides, tutors, teammates– Engage in instructional dialog– Enhance motivation and interest

• APA’s require:– Realistic, lifelike behavior– A rich set of cognitive and social

abilities for effective learning


32/58

Steve: An Embodied Intelligent Agent for Virtual Environments

• J. Rickel, L. Johnson, M. Thiebaux, et al.

• 3D agent that interacts with students in virtual environments

• Can work together with multiple students and multiple users


33/58

Steve’s Architecture (detailed)

Cognition

MotorControl

Perception

General capabilities

Domain knowledge

Event notifications

Current state

Motor commands

Virtual Environment

Monitor events

Filter, assembleinto coherent view

Translate into movements, speech

Broadcast to environment

Commands to environment

Steve


34/58

Mission Rehearsal Exercise


35/58

Co. & Co. & Co.

As a process As an outcomeCoordinati

on

Decisionmaking takes others into account

Individuals are notthwarted by others

Cooperation Individual decisionsfurther the collective welfare

Individuals appear to be“working together”

Collaboration

Individuals worktoward shared goals

Common goalsachieved well and/orefficiently


36/58

Example: Hidden Pictures

• Simple (visual) search task

• How would YOU work as a part of a team to solve it ?


37/58

Market-based MAS

• Marketspace: class of agent interaction env.• What you need to know:

– Economic foundations and principles:• Game theory• Price system (general equlibrium)• Auction theory

– Design issues and experience• Market models and mechanisms• Trading Agents


38/58

Business games

• http://www.cmu.edu/comlabgames-----------------------------------------------------------At the comlabgames website, www.cmu.edu/comlabgames, there are three modules for designing, playing and analyzing the experimental outcomes of games: two person strategic form games, multi-person extensive form games, and auctions and markets. The original comlabgames website is visited on average 50,000 of time each week, and is linked to hundreds of other sites. UCLA mirrors the original site. Comlabgames is very easy to use, and the students just bring their laptops to class, design the games, run them and then analyze the data.

• Vesna Prasnikar, Marko Grobelnik

http://www.cmu.edu/comlabgames


39/58

Logics for Agents

• Symbolic Reasoning– an agent contains an explicitly represented, symbolic

model of the world– makes decisions (action to take) via symbolic reasoning– problems:

• transduction: how to translate the real world in accurate, adequate symbolic description (speech understanding)

• representation/reasoning: how to symbolically represent inf., and how to get agents to reason with this inf. (planning)

– Theorem Proving Agents: agent decides what to do by using logic to encode the theory stating the best action in any situation (predicates + rules)

– Agent oriented programming: AGENT0 and PLACA


40/58

Logics for Agents

• Practical Reasoning (BDI Logic)– is a matter of weighting conflicting considerations for and

against competing options, where the relevant considerations are provided by the agent desires/values about and what the agent believes (Bratman).

– directed towards actions (theoretical towards belief)

– consists of two activities:• deliberation: deciding what state of affairs we want to achieve

• means-end reasoning: deciding how to achieve them

– implemented BDI agents: IRMA, PRS, Desiderata, LORA


41/58

Recent conferences & workshops

• IJCAI-95 workshop on Adaptation and Learning in Multiagent Systems

• ML-95 workshop on Agents that learn from other agents

• AAAI-96 Spring Symposium on Adaptation, Co-evolution and Learning in Multiagent Systems

• ECAI-96 workshop on Multiagent Learning• ICMAS-96 workshop on Learning, Interaction,

and Organization in Multiagent Environments


42/58

Recent conferences & workshops

• AAAI-97 workshop on Multiagent Learning (AAAI Technical Report WS-97-03)

• IJCAI-99 workshop on Agents Learning About, From and With Other Agents

• AGENTS’2000-ECML’2000 workshop on Learning Agents

• ML’2000 workshop on Multiagent Learning• IJCAI’01 workshop on Agent-based Systems• Autonomous Agents and MAS 2002 Conference• AAAI-2002 Spring Symposium on Collaborating Agent


43/58

Journal Special Issues

• Machine Learning: Multiagent Learning

• Journal of Experimental & Theoretical Artificial Intelligence: Learning in Distributed AI Systems

• International Journal of Human-Computer Studies: Learning and Evolution in Multiagent Systems


44/58

Additional Information

• ASAMAS 2002 – http://ai.ijs.si/Sandi/asamas.html– http://agents.usc.edu/agents/school/main.html

• AgentNews– http://agents.umbc.edu/

• AgentLink– http://www.AgentLink.org/

Intelligent Agents – state of the art – Materials collected at: Americas School on Agents and...

Documents

Transcript of Intelligent Agents – state of the art – Materials collected at: Americas School on Agents and...