Generative Models of Group Members as Support forGroup Collaboration

Anthony Jameson�, Stephan Baldes

�, and Thomas Kleinbauer

���

�DFKI / International University in Germany�

DFKI, the German Research Center for Artificial Intelligence

Abstract. Collaborating groups sometimes have to resolve conflicts among groupmembers in terms of their preferences and values. When face-to-face or other syn-chronous communication is not possible, this type of conflict resolution is maderelatively difficult by the low bandwidth of communication. We present an ap-proach to this problem that is based on the idea of generative models of groupmembers who are not currently available for communication. This approach isillustrated with reference to a prototype system for planning joint vacations.

1 Introduction

User models have been used in various ways to support group collaboration. Themost common way is to help people to identify potential collaborators in the first place(see, e.g., McDonald & Ackerman, 2000). Another approach is for a system to providespecific support during the collaboration process itself. For example, the AGENTSALONsystem (see, e.g., Sumi & Mase, 2001) helps visitors to a conference to find topics ofcommon interest and to exchange ideas about them.

In the present paper, we propose a relatively novel form of collaboration support ofthe second type. The approach is applicable in situations in which a group of personsaims to arrive at a joint decision with regard to some question (e.g., “Where shall wego on vacation together?”). In contrast to systems like AGENTSALON, or to systemssupporting synchronous communication via audio, video, and/or text, our approach isapplicable to cases where the group members communicate asynchronously via theweb. The basic idea is that, with asynchronous communication, group members in somerespects have a relatively poor awareness of each others’ reactions to the problemsthat they jointly face. Even though each member may be able to formulate her overallattitudes and ideas in great detail, it is difficult for other members to elicit her reactionsto a large number of specific proposals or issues. Moreover, there are some aspectsof a member’s relevant beliefs and attitudes that are inherently difficult to convey viathe media typically available for asynchronous communication:her specific emotionalresponses and their overall motivation and orientation.

�The research described here was supported by the German Ministry of Education and Research(BMB � F) under grant 01 IW 001 (project MIAU). We thank the anonymous reviewers forhelpful comments.

Fig. 1. Visualization of the basic idea of using representatives of absent group membersfor the support of group collaboration.(Explanation in text.)

We therefore see a role for generative models of individual group members thatother members can interact with. Figure 1 illustrates the basic idea.1 The left-handside of the figure shows the situation in which group members can communicate witheach other in real time. In the situation on the right, such synchronous communicationis not possible. Instead, for each member there is a computational model of her rel-evant beliefs, preferences, motivation and other relevant properties, with which othermembers can interact. This model will be referred to as the member’s representative.2

Each such model is generative in the sense that it can simulate the real group member’sresponses to inputs (e.g., proposed solutions) that the real member has never actuallybeen confronted with. It is possible (though not necessary), that each member givesher representative some degree of authority to make partial decisions—for example, toaccept proposals that the real member would accept if she were present.

Since each representative will necessarily be an incomplete and simplified modelof the corresponding real group member, interaction with the representatives can beonly an approximation of direct interaction with the real group members. Still, suchinteraction can be helpful in various ways:

Reaching agreement on relatively easy aspects of a decisionIf the representatives have some authority to make decisions, the currently activemember may be able to arrive at partial solutions through interaction with therepresentatives. In this way, the attention of the group members can be focused asmuch as possible on the more difficult aspects of the decision.Enhancing awareness of the other members’ beliefs, preferences, and motivation.Even if each group member has specified her beliefs, preferences, and motivationin detail, it may be difficult for other members to understand and appreciate any

1 The collaborative learning environment I-HELP (see, e.g., Vassileva et al., 1999) similarlyincludes software representatives of real learners, though their functions are different fromthose described here.

2 The word agent could also be used, but since this word has so many different senses, we willavoid using it.

Fig. 2. Basic layout of the Travel Decision Forum.

static representation fully. A generative model that actually responds to particularproposals can create a more vivid impression.

In the main part of this paper, we illustrate the ideas just introduced with reference aspecific prototype.3 The implications for some of the central questions of the workshopwill be discussed in the final section.

2 Overview of the Prototype

The prototype supports users in the following scenario: Three friends want to plan ajoint vacation. They are not able to get together or to use synchronous communicationmedia to discuss their plans. Consequently, at any moment only one group member willbe interacting with the web-based system, which will make use of stored informationabout the other group members.

Figure 2 shows the setting in which the interaction takes place: An animated charac-ter, the mediator, sits in front of a screen on which he can display possible solutions andproposals. On the right, we see two animated characters that represent the two absentgroup members (here: “Ritchie” and “Tina”). In the front, we see the back of a characterthat represents the currently active group member (here: “Claudia”). Unlike the other

3 This brief description of the prototype is intended to support and complement the discussionat the workshop, at which the system will be demonstrated. More detailed descriptions anddiscussions will be found in papers that are currently in press (Jameson, Baldes, & Kleinbauer,2003), under review (Jameson, Hackl, & Kleinbauer, 2003), or in preparation.

Fig. 3. Dialog box for the collaborative specification of preferences.(The currently active group member is Claudia. The preferences of each member are representedby a uniquely colored letter; Claudia cannot change the position of the letters “T” and “R”. Theinput cells marked with white show the proposal currently under consideration; see Section 4.2below.)

two characters, this character does not act independently but rather serves as a meansof making the current member visible on the screen.

Using two of the buttons at the bottom of the screen, the current user can at almostany point change her preference specifications or modify the parameters that determinethe behavior of the characters (see Figure 7 below). In each of these cases, the form inquestion appears in front of the background image, as though the user were holding itup and looking around it to view the scene (see, for example, Figure 4).

3 Phase 1: Specification and Refinement of Preferences

3.1 Collaborative Preference SpecificationIn the first of the two main phases of the interaction with the system, each member

specifies her preferences concerning the vacation via a collaborative preference speci-fication form (Figure 3). This form is in many ways similar to preference specificationforms that are familiar from systems like the ACTIVE BUYER’s GUIDE4 and the once-popular system PERSONALOGIC (which is no longer available): For each of severalvalue dimensions (shown on the tabs in the form), a number of attributes are shown.For each attribute, the user can give a rating between �� (“Don’t want it”) and (“Want it”). These ratings are interpreted in terms of multiattribute utility theory (cf.Jameson, Schafer, Simons, & Weis, 1995); we omit details concerning their meaningand processing, since these are not required for an understanding of the central points

4 http://www.activebuyerguide.com/

Fig. 4. The current member tests her preference specifications by examining examplesolutions generated on the basis of the preferences of all group members.

being made here. On the basis of the preference specifications of each member, the sys-tem can evaluate a specific vacation solution from the point of view of that member.Similarly, after aggregating the preferences of the three members according to any ofvarious possible methods, the system can evaluate any specific solution from the pointof view of the group as a whole.

The novel aspect of this preference specification form is the way in which it allowsthe previously specified preferences of other group members to be viewed (optionally)by the current user. This feature can be seen as a simple way of enhancing awareness ofthe preferences of other group members (cf. the discussion of an earlier version by Plua& Jameson, 2002). Despite its simplicity, this collaborative preference specificationappears to yield considerable benefits. For example, in a brief study that we conductedwith 22 subjects, 14 stated that they preferred being able to see the preferences of an-other group member while specifying their own preference, while only 3 preferred notto have them shown. In the former condition, there was a tendency for users’ specifiedpreferences to be more similar to the visible preferences of the other group member;subjects’ comments indicated that they wanted to minimize unnecessary differences inpreference specifications so as to facilitate the reaching of agreement.

Though useful, the reflection of the preferences of other members in the input formdoes not involve a generative model that produces new behavior; generative models areonly used in Phase 2 (see below). As can be seen in Figure 4, the representatives of theother members are not yet present as characters.

3.2 Testing Preference Specifications by Requesting ExampleSolutions

During this first phase, at any point the current member can perform a “realitycheck” on her specified preferences by referring to the offerings in the relevant databaseof possible solutions. (This procedure involves the invocation of the MAUT MACHINE,which is discussed by Schmitt, Dengler, & Bauer, 2003.) In the simplest case, the cur-rent member asks the mediator to show the solution from the database that fits best withher own preferences as they have been specified so far—that is, without regard to thepreferences of the other members. It is well known from experience with recommendersystems that users often want to check intermittently what kind of solutions are likelyto emerge, even after they have input just a few preferences or ratings of their own(cf. Burke, Hammond, & Young, 1997). Not only does this type of intermittent testingmake the interaction more interesting; it also gives the user important feedback abouther preferences—for example, concerning unrealistic requirements or neglected aspectsof the problem.

This type of incremental preference elicitation is supported in many recommendersystems (e.g., ACTIVE BUYER’s GUIDE, the AUTOMATED TRAVEL ASSISTANT ofLinden, Hanks, & Lesh, 1997, and the VEIL system of Blythe, 2002). Our prototypeonce again introduces a somewhat novel group-related element: The current user canask to see example solutions that are based on the specified preferences of all of thegroup members.5 For example, in Figure 4, the mediator shows Claudia the most im-portant attributes of an example solution that was chosen on the basis of the preferencesof all three group members. Claudia can see that her main requirement concerning thedimension Sports Facilities—the possibility of bike riding—is satisfied by the exam-ple solution. A glance at the relevant part of the preference specification form indicateswhy this desire is realistic in this situation: Ritchie has likewise expressed a strong pref-erence for bike riding, and he has assigned to the dimension Sports Facilities an evenhigher weight than Claudia. It might have happened, however, that Claudia noticed thather preferences concerning Sports Facilities were having no impact on the proposedexample solutions—for example, because the other members expressed strong prefer-ences for countries in which bike riding is rarely available. In that case, Claudia mightconsider revising her preferences: She might increase or decrease the importance sheassigns to Sports Facilities, or she might consider whether it would be a good time totry out a new sport. This type of preference adaptation by individual members can initself facilitate the finding of an appropriate solution.

This use of the absent members’ stored preference models for the generation ofexample solutions is a further step in the direction of the use of generative models ofthese members; but as we will see in the next section, this idea can be carried muchfarther.

This first phase continues for the current member until that member has indicatedthat she is not interested in continuing to specify or adapt her preferences in this way.

5 This functionality is also available in POLYLENS (O’Connor, Cosley, Konstan, & Riedl, 2001);but in the context of that system it does not have the function of supporting the incrementalrefinement of specified preferences.

4 Phase 2: Arriving at a Joint Preference Model

4.1 The Goal of This PhaseIn this phase, the group members’ goal is to agree on a joint preference model—that

is, a single way of filling out the preference specification form that can be used as arepresentation of the preferences of the group as a whole. When it comes to searchingfor specific solutions (i.e., specific hotels in particular countries), it is not obvious that asingle joint preference model is more helpful than a set of individual preference models(cf. the discussion of this issue by O’Connor et al., 2001). In fact, there will in generalbe some loss of information when a set of preference models is replaced by a single one.One advantage of a single common model, though, is that it can serve as a convenientfocal point for discussion and negotiation. Note that there are many cases in face-to-face collaboration where people find it useful to agree on the criteria to be used inmaking a decision before they discuss specific solutions—especially when the numberof potential specific solutions is great. One advantage of doing so is that, once thecriteria have been agreed upon, it may be possible to eliminate a large proportion of thepossible solutions without close examination. And even if the members have difficultyin agreeing on the criteria, the process of discussing them can help each member to seewhere the key disagreements lie and what adjustments in her own requirements can bestfacilitate agreement.

Once a joint preference model has been agreed upon—and has been found throughthe inspection of example solutions to be reasonably realistic—it may be relativelystraightforward to use this preference model for the identification of specific solutionsthat are acceptable to all of the group members.

4.2 Structure of the Interaction in This PhaseIn this second phase, animated characters representing the two absent group mem-

bers appear on the screen (see Figure 5). For each value dimension (e.g., Sports Facili-ties), the mediator moderates a simple form of negotiation between the current memberand the two representatives.6

For each dimension, the mediator proceeds as follows:Step 1. On the basis of the specified preferences of all members, he computes a

proposal—a particular way of filling out the preference specification form with respectto one dimension. The mediator then displays the proposal on the screen behind him.The screen in Figure 5 shows a proposal for the dimension Health Facilities. The pro-totype provides several mechanisms for generating proposals, the choice of which canbe influenced by the mediator and/or the current user:7

The random-choice mechanism: For each attribute, one of the three ratings sup-plied is chosen randomly. This is the mechanism that was used to generate theproposal shown in Figure 5.

6 The order in which the value dimensions are dealt with depends on the particular mediationstrategy chosen by the mediator.

7 The question of the conditions under which each of these mechanisms is likely to generate asuitable proposal exceeds the scope of this paper; see Jameson et al. (2003).

Fig. 5. The current member adapts the proposal made by the mediator so as to take intoaccount the preferences and expressed reactions of the representatives of the absentgroup members.

The median mechanism: For each attribute, the median of the three ratings ischosen.The averaging mechanism: For each attribute, the average of the three ratings ischosen, with the result being rounded off where necessary.An automatically generated nonmanipulable mechanism: With the method de-vised by Conitzer and Sandholm (2002), a mechanism is generated on the flywhich is nonmanipulable: The mechanism ensures that it is in the interest of eachgroup member to specify her preferences accurately. (By contrast, with the av-eraging mechanism, a member can sometimes be tempted to give an inaccurate,extreme rating in order to ensure that the proposals generated correspond with herown true preference.) These automatically generated mechanisms are optimizedto take into account various parameters, for example, whether the proposal is sup-posed to optimize only the total utility for all members or also equity: the extentto which it is equally satisfactory to all members.

This proposal is also displayed in the preference specification form, which the currentmember has opened: For each attribute, the proposed joint rating is highlighted with awhite background.

Step 2. The mediator gives each of the two representatives an opportunity to sum-marize the reaction to this proposal of the corresponding real group member. The natureof these summaries will be described in 4.3; suffice it to say here that they always end

with a statement of acceptance or rejection of the proposal. The representative accepts aproposal when its deviation from the stored preferences of the represented group mem-ber does not exceed a certain threshold, specified in advance by that group member (cf.Figure 7). In Figure 5, Tina’s representative has just accepted the proposal—which isindeed quite close to Tina’s stored preferences—whereas Ritchie’s representative hasrejected it.

Step 3. The mediator asks the current member to respond to the proposal. Thisresponse can take various forms:

1. The current member accepts the proposal.If the other representatives have likewise accepted the proposal, the mediatordeclares the discussion of this dimension to be completed, and the proposal isstored as part of the joint preference model.8

2. The current member adapts her preferences.The comments of the two representatives may, for various reasons, have inducedthe current member to reconsider and change her own preferences. In this case,the mediator will present in the next step a new proposal which is based on thenew set of individual preference models.

3. The current member changes the proposal.For example, in Figure 5, Claudia might change her entry for Massage from++ (“Want it”) to � (“Don’t Care”), thereby eliminating the largest deviationfrom the preferences of Ritchie (and herself), in the hope that the new proposalwill be acceptable to both of the representatives. The mediator will present thecounterproposal to both representatives in the next step. This response makessense especially when the current member can use her knowledge of the domainor of the other members to generate a proposal that is more likely to be successfulthan a proposal generated automatically by the mediator.

4. The current member rejects the proposalThis response makes sense only when the current member sees no likelihoodthat further discussion of this dimension might lead to a mutually acceptablesolution. In this case, the mediator puts the current dimension aside for the timebeing. He will bring it up again in the next interaction with one of the othergroup members; perhaps this member will see some way of achieving consensusthrough a reaction of type 2 or 3.

Step 1 again. The mediator goes back to Step 1, presenting a new proposal fordiscussion. Depending on what has just happened in Step 3, this proposal will be either acounterproposal offered by the current member or a proposal generated by the mediatorconcerning either the current dimension or the next dimension.

4.3 Performances of the RepresentativesThe brief performances of the animated characters are designed to place the current

member in a position that is more or less similar to that of a person who, in face-to-face communication with another person, tries to recognize that person’s attitudes and

8 If one or both of the representatives has rejected the proposal, it makes little sense for thecurrent member to accept it; one of the next two responses listed is more appropriate.

Fig. 6. The representative of Ritchie makes positive (left) and negative (right) commentson aspects of a proposal.(More explicit and verbose formulations are used at the beginning of the interaction, e.g., “Frommy point of view, the rating that you propose for ’Sailing’ is much too low”.)

motivation. In particular, the current member is supposed to realize that not only shebut also the other members are in general making sacrifices if they agree to a compro-mise proposal. The current member’s own sacrifices tend to be relatively salient to her,and they may even evoke emotional responses—for example, when she agrees to dowithout her favorite sports activity or (as a smoker) to stay in a nonsmoking room. Itis much less obvious to her, without explicit visualization, that the other members haveto make similar sacrifices: These sacrifices can in principle be recognized in the prefer-ence specification form (cf. Figures 3 and 5) as cases where the letter for the memberin question lies (far) away from the highlighted cell representing the proposal. But thisrepresentation is much more abstract than the current member’s own responses; andbecause of the large amount of information in the form, relevant information can easilybe overlooked.

To reduce this asymmetry, each representative responds to a proposal by (a) pickingout the aspects of the proposal that are most important for the group member that it rep-resents and (b) verbalizing a plausible response of this group member to this proposal,using gestures and facial expressions for visual emphasis (see Figure 6).

The current member can set parameters (see Figure 7) to determine the level ofdetail of these short performances; for example, she can ensure that the representativecomments only on the most negative aspects of a proposal. She can also control thespeed of the representatives’ speech and have particular performances skipped entirelyby clicking on the Skip Comment button. These interface details are intended to ensurethat users do not get prematurely bored with the animated characters, whose perfor-mances are inevitably more redundant and predictable than those of human speakers.More importantly, they offer a way for the user to fade the characters out gradually ifshe so desires, relying more and more on the information presented compactly in thepreference specification form, once she has recognized how to interpret it appropriately.

Fig. 7. Dialog box for the setting of parameters for the control of the animated charac-ters.

4.4 Specifying the Motivation of the Representatives

So far, our discussion of the prototype has presupposed the default settings for themotivation of the group members: As is done in most relevant systems and studies, wehave been assuming that each member has the goal of maximizing the fulfillment of herown requirements.

Especially in a group whose members are acquainted with each other, we often findother motivations as well. For example, a member may be cooperatively motivated—i.e., view the interests of the other members as being just as important as her own—perhaps because she knows that a joint vacation is not enjoyable for one member ifthe other members are dissatisfied. It is even possible for one member to assign moreimportance to the interests of other members than to her own interests (e.g., when aparent is prepared to put up with just about anything as long as the children are happy).

To support interaction in situations like this, the system allows each member to setparameters for her own representative (see Figure 7) that specify which group members’preferences the representative should take into account when responding to a proposal.(Each representative has access to all of the preferences that have been specified sofar.) For example, the real person Tina may have chosen “Yes” for each of the options“Really cares about X”, thereby specifying that she wants her representative to takethe interests of all three group members into account .9 In this case, the Tina character,when commenting on proposals, will make comments that refer to the preferences of theother two members (e.g., “For Claudia, the rating you propose for Sailing is much toolow”) in addition to comments that reflect her own preferences; and the representative’sdecision as to whether accept a proposal will be based on the weighted sum of theevaluations of all members.

9 The options of the form “Ostensibly cares about X” are relevant only in special situations thatwill not be discussed here.

It is important for the current member to be able to assess the motivation of the othermembers. For example, if the current member is convinced that the other two membersare cooperatively motivated, it may make sense for her to adopt a cooperative attitudeherself. She may, for example, set the tolerance level of her representative for the ac-ceptance of proposals (cf. Figure 3) relatively high, thereby accelerating the processesof finding a joint preference model—without having to fear that the other members willtake advantage of her cooperative attitude. More importantly, when thinking of counter-proposals, she can look for creative solutions that aim to satisfy the group as a whole,instead of just thinking about how to maximize the benefit for herself. In this case, es-pecially effective use is being made of valuable resources: the knowledge and creativityof the only human who is involved in the current interaction.

5 Ultimate Results of the InteractionThe style of interaction described in the previous section continues until the cur-

rent user has either (a) agreed (to the extent possible) with the representatives of theother members on a joint preference model for each dimension or (b) run out of time orinterest. At this point, even if there are still some dimensions for which no joint pref-erence model has been agreed upon, there may have been considerable movement inthe direction of an agreement: The current user may have adapted her preference spec-ifications in a way that reduces the amount of conflict; she may have adopted a morecooperative orientation toward the evaluation of proposals; and she may have increasedthe leeway granted to her representative in accepting compromise proposals. When thenext group member interacts with the system (seeing, of course, the animated characterrepresentative of the previous current user instead of his own representative), there willbe opportunities for further convergence.

Once a point has been reached at which none of the group members desires furtherinteraction with the system, the mediator can use the joint preference model (and, ifnecessary, the individual preference models) to select one or more specific solutions.There are various ways in which the final selection of a specific solution can be orga-nized, depending, among other things, on whether all of the possible specific solutionsare already available or new solutions will become available in the future. In any case,our hypothesis is that the convergence and mutual awareness brought about by the in-teractions described above will make this final step much more straightforward than itwould otherwise be.

6 Implications for the Questions of the WorkshopThis work suggests partial answers to the following central questions of this work-

shop:Should there be an explicit group model or a dynamic combination of individualmodels as needed?Our prototype emphasizes one advantage of explicit group models that is appli-cable in cases where there is a focus on reaching agreement: Arriving at a single

group model can become one of the goals of the interaction, and the model canserve as a focal point for discussion.How do we use multi-user models to enhance collaboration?Despite the central role of a group model, the individual models also play animportant role our prototype, because they serve as the basis for the generativemodels of the individual group members.How to mediate between conflicting user interests?The strategies used by our mediator for generating new proposals have been sum-marized only briefly here because of the space limitation, but during the workshopdiscussion they can be compared with other mediation mechanisms. The empha-sis in this paper has been on ways of helping group members to make good use oftheir capacities for social perception and for creative thinking about the problemat hand.How to take into account social aspects within a group?The prototype provides for the explicit modeling of some important social param-eters that are usually left implicit.

During the discussion at the workshop itself, the implications of this work for thecentral questions will be worked out in more detail and integrated with the contributionsof the other participants.

ReferencesBlythe, J. (2002). Visual exploration and incremental utility elicitation. In R. Dechter, M. Kearns,

& R. S. Sutton (Eds.), Proceedings of the Eighteenth National Conference on ArtificialIntelligence (pp. 526–532). Menlo Park, CA / Cambridge, MA: AAAI Press / MIT Press.

Burke, R. D., Hammond, K. J., & Young, B. C. (1997). The FindMe approach to assistedbrowsing. IEEE Expert, 12(4), 32–40.

Conitzer, V., & Sandholm, T. (2002). Complexity of mechanism design. In A. Darwiche &N. Friedman (Eds.), Uncertainty in Artificial Intelligence: Proceedings of the EighteenthConference (pp. 103–110). San Francisco: Morgan Kaufmann.

Jameson, A., Baldes, S., & Kleinbauer, T. (2003). Enhancing mutual awareness in group recom-mender systems. In B. Mobasher & S. S. Anand (Eds.), Proceedings of the IJCAI 2003Workshop on Intelligent Techniques for Web Personalization.

Jameson, A., Hackl, C., & Kleinbauer, T. (2003). Evaluation of automatically designed mecha-nisms. (Manuscript submitted for publication)

Jameson, A., Schafer, R., Simons, J., & Weis, T. (1995). Adaptive provision of evaluation-oriented information: Tasks and techniques. In C. S. Mellish (Ed.), Proceedings of theFourteenth International Joint Conference on Artificial Intelligence (pp. 1886–1893). SanMateo, CA: Morgan Kaufmann.

Linden, G., Hanks, S., & Lesh, N. (1997). Interactive assessment of user preference models: Theautomated travel assistant. In A. Jameson, C. Paris, & C. Tasso (Eds.), User modeling:Proceedings of the Sixth International Conference, UM97 (pp. 67–78). Vienna: SpringerWien New York.

McDonald, D. W., & Ackerman, M. S. (2000). Expertise Recommender: A flexible recommen-dation system and architecture. In Proceedings of the 2000 Conference on Computer-Supported Cooperative Work (pp. 231–240). Philadelphia, PA.

O’Connor, M., Cosley, D., Konstan, J., & Riedl, J. (2001). PolyLens:A recommender systemfor groups of users. In Proceedings of the European Conference on Computer-SupportedCooperative Work.

Plua, C., & Jameson, A. (2002). Collaborative preference elicitation in a group travel recom-mender system. In F. Ricci & B. Smyth (Eds.), Proceedings of the AH 2002 Workshopon Recommendation and Personalization in eCommerce (pp. 148–154). Malaga, Spain:Universidad de Malaga, Departamento de Lenguajes y Ciencias de la Computacion.

Schmitt, C., Dengler, D., & Bauer, M. (2003). Multivariate preference models and decisionmaking with the MAUT Machine. In P. Brusilovsky, A. Corbett, & F. de Rosis (Eds.),UM2003, User Modeling: Proceedings of the Ninth International Conference. Berlin:Springer.

Sumi, Y., & Mase, K. (2001). Collecting, visualizing, and exchanging personal interests andexperiences in communities. In N. Zhong, Y. Yao, J. Liu, & S. Ohsuga (Eds.), Web intelli-gence: Research and development (WI 2001 proceedings) (pp. 163–174). Berlin: Springer.

Vassileva, J., Greer, J., McCalla, G., Deters, R., Zapata, D., Mudgal, C., & Grant, S. (1999). Amulti-agent design of a peer-help environment. In S. P. Lajoie & M. Vivet (Eds.), Artificialintelligence in education: Open learning environments: New computational technologiesto support learning, exploration, and collaboration (pp. 38–45). Amsterdam: IOI Press.