User Testing of the Brooklyn Museum’s Posse

Allison Hall, Iris Bierlein,Carolyn Li-Madeo, Steph DiOrio

Pratt Institute of Library Information Science

Executive Summary

Posse is the Brooklyn Museum’s innovative and exciting new community portal that enables users from around the world to engage with the museum’s collections. Posse members perform import-ant tasks through two crowdsourcing games and are rewarded with video prizes and a game rank-ing system. The work of Posse members is a thread that runs throughout the Brooklyn Museum’s website, tags created by the Posse community are used to increase the visibility and findability of artwork in the collections, and member curated exhibits have even been featured in the physical museum. In order to ensure the engagement of current and potential users in the creation of high quality folksonomy tags the understandability of the tasks, mission and navigation of the Posse features is vital to the success of the larger project.

The findings of this report came from usability testing of the Posse games “Tag You’re It” and “Freeze Tag”. Eight individuals, with diverse backgrounds, physical locations and ages who all fit into the user profile of potential Posse members took part in usability testing of the Brooklyn Museum website, with a focus on Posse’s gaming features. The users performed four tasks related to Posse under the observation of usability researchers. Participants were asked questions before, during and after the usability testing in order to gauge the intuitiveness of the games and the web-site’s navigation, as well as their engagement level. The findings of this user test were then analyzed by the research team. From this analysis the usability team focused on three and four aspects of the website were highlighted for modifications to increase the usability of the website and the engagement factor of the Posse games:

1. Increase the visibility of the Posse community and activities through the use of drop-down menus on the homepage and a welcome message to all logged in users across the website. 2. Clear directions, options to see a help page with examples of how to properly tag and the implementation of clear, simple language consistently throughout the Posse games will help to increase the understandability of the tasks and game purpose to users.3. Enable users to be able to return to the image they were working with in a Posse game if they are to follow a link within the game out into the museum collection. 4. Change the wording from gaming language to a direct appeal in order to better engage and motivate users in their Posse tasks. Make the gamification and scoring features of the tasks clearer and perhaps add actual incentives (exhibit admission) rewards to high scoring users. These recommendations were created to increase the usability, flexibility and navigation of the user interface for first time and returning users. User engagement and understanding of the Posse games and activities is another concern addressed in this report.

Table of Contents

Introduction

Methodology

Findings and Recommendations Conclusion

Appendix

1

2

3

11

12

Introduction

Posse is community portal into the Brooklyn Museum’s online collections. Through Posse users from around the globe can curate their own personal collections, comment on works of art, and discuss their thoughts with other community members. Posse members also perform important tasks through their participation in two crowdsourcing games: “Tag You’re It!” a tagging creation game, and “Freeze Tag” a tag maintenance game. Through playing these two games, users can share their personal insights on works of art located in the Brooklyn Museum’s collection and help to make art more discoverable for other website visitors.

The success of this innovative and exciting community portal relies heavily on the understandability of the games’ purposes to potential users who range in age, gender and location. Additional challenges that these games must face, are how to work within the navigation and ‘look and feel’ of the larger Brooklyn Museum website while striking the balance between ease of use and gaming pleasure. Finally, the Posse games must also encourage users to both have fun and to create useful and high-quality folksonomy tags.

In order to best serve the Posse community, a group of usability experts performed eight usability tests of the Posse games. Testers engaged with a series of tasks related to game play under the observance of usability researchers. Participants were asked questions before, during and after the usability testing in order to gauge the intuitiveness of the games and the website’s navigation, as well as their engagement level. The findings of these usability studies were analyzed by the UX researchers who focused on three key findings from the user testing. In addressing these findings the researchers have developed seven recommendations for the Brooklyn Museum website. The goal of these recommendations is to make the Brooklyn Museum’s website and Posse features easier to use for first time and returning visitors as well as to ensure that the goals of the Posse community are supported.

User Testing | 1

Methodology

For the user testing of the Brooklyn Museum’s Posse feature on its website, the research team recruited eight different individuals with varying degrees of internet familiarity and asked them to complete tasks related to the feature. The research team recorded the users completing the tasks using the programs Morae and Silverback (with two researchers using each). The two researchers using Morae had the pre-test questionnaire and post-test questionnaire contained within Morae, whilst the two researchers using Silverback created the pre-test questionnaire and post-test ques-tionnaire on a Google Survey.

Users signed a consent form and completed the pre-test questionnaire first and were then encour-aged to think aloud as they completed the four tasks designed to have them navigate the website (and the Posse feature in particular). After completing the assigned tasks, the users completed the post-test questionnaire and selected a point on an emotion matrix measuring their reaction to the website’s attractiveness and engagement. This concluded the test, and the users were permitted to leave afterwards. The research team then proceeded to analyze the data from the recorded tests and drew conclusions from their data. The following suggestions for improving the Posse feature are based on these conclusions.

User Testing | 2

Findings

The Brooklyn Museum’s Posse feature on its website is a unique portal for users to engage and interact as a community. By using games to crowdsource tags for objects within the museum, users get a chance to feel like they are making an impact. Based on this user study, several findings have been found that affect the overall usability of these one of a kind community games.

Visibility: Posse, Instructions, Icons

The most common issue found among all users was related to poor visibility. When asked to find Posse from the Home page of the Brooklyn Museum, most users resulted to using the Search box. Users expected to find a link to Posse under one of the Menu headings but many were unsuccessful searching this way. A community engagement tool as unique as these games should be easy to find, as well as stumbleupon. As of right now the placement of Posse within the site is not intuitive for first time users to find. It would be in the Brooklyn Museum’s favor to create better visibility or placement of the Posse portal on the homepage.

Another visibility issue on the site is related to the pages of the two games. Users reported during the tests that they were unclear as of what to do and how to play the games. Once the users identified the instructions for the game, many noted that they were not big enough for them to see. The instruc-tions blend in with the rest of the site. Additionally many users reported that the large video on the Tag! You’re It! main page confused them. Users watched the video thinking that it would be a fun explanation of how to play the game, however there was no explanation. Once they moved on to the next page to play the game they were doubly confused because they couldn’t find the instructions. The game itself did not seem be intuitive for users so without being able to find instructions this was a frustrating experience for them.

Lastly, users had a very difficult time locating common buttons and icons. Many of the icons on the game pages are small and placed in locations that users would not expect to find them like toward the bottom of the page. Additionally between the two games, icons are not consistently placed. After users had played Tag! You’re It! and moved on to Freeze Tag, they expected to see icons and buttons in the same place. Due to the lack of consistency, poor placement of icons, and icon sizing, the user experience outcome was not fun. One user aborted task three (locating and playing Freeze Tag) due to these issues.

User Testing | 3

Lack of Breadcrumbs

Both Tag! You’re It! and Freeze Tag give the user the chance to view objects from multiple views with alternate image angles. This can be very helpful to a user who is not able to physically see the object. However there is no point in providing alternative views if the user can not retrace their steps to get back to the original view of the object they are looking at. During the study it was discovered that users who view alternate image angles link to unrelated objects and are directed to another different

User Testing | 4

object when they attempt to go back using the browser. The larger solution to this issue may be tech-nical. However based on the fact that there are no breadcrumbs for users to retrace their steps makes this feature not only useless but very disappointing.

Gamification of features was confusing and unengaging

Although all moderators ensured the user that they were testing the website, not them, users felt there were right and wrong answers when playing the games. Furthermore many users wondered what the point of the games were. Users noted that they didn’t understand how points were awarded and thus the game felt unappealing to them. The use of the word “Play” and the emphasis on games confused the user. The games on Posse are more like educational and learning tools, reported users. Users certainly expected more recognizable game-like features. The games were described as not being fun, confusing, and unengaging. The Brooklyn Museum should decide whether they want to fully gamify these powerful tools with more common gameplay rules or relabel them under educational resources. These tools have the potential to create strong community within the museum but as of right now users do not enjoy using or interacting with them.

User Testing | 5

Recommendations

In light of the findings there are several quick and cost-effective suggestions which directly address some of the usability issues identified during the user study of Posse and its games. Prioritizing a positive online user experience can help the Brooklyn Museum to build stronger community ties, enrich collections, and encourage users to become donors. The the implementation of the recommendations laid out in this report is strongly suggested as they are all relatively small and simple to carry out yet all designed to provide maximum return on investment.

Issue: Make the Posse feature more visibleThere are three main ways to access posse: through the “collections”, “community” and “search” tabs. Having multiple routes to a feature is useful as it increases the likelihood casual browsers will en-counter the feature, those browsing the collections landing page and those browsing the community landing page. While a casual encounter with Posse is possible via these tabs users on the museum homepage found it difficult to find as it was buried on subpages. These users resorted to the search bar, with varying success.

Recommendation: Use drop-down menu tabs on the homepage

The museum website had a huge amount of wonderful features, yet they are not quickly obvious via the homepage. In order to help direct users to features they are interested in, such as Posse, the employ of a drop-down style menu may be helpful. Drop-down menus can help to inform users about the content of each tab without causing them to click and then backtrack if it turns out not to be the content they are interested in. The example below shows a mockup of the homepage with drop-down menus, a feature which would make Posse and other elements of the site visible from the homepage without having to add new content.

Fig 1: Illustration of a drop-down menu deisgn.

User Testing | 6

Recommendation: Include a user welcome message on login

Place a welcome message to users with a link to their profile on all pages of the museum once a user has logged in. The message and link to a user profile should be prominent and in an easy to find location, such as the upper right hand corner (as seen in the example below). Making the font color and style consistent across all content that relates to Posse profile information would also help users quickly recognize which content is directly related to them.

Fig 2: Illustration of a user greeting.

Issue: Users had difficulty understanding the directions for both Tag! You’re It! and Freeze TagTagging artwork in a social context has the potential to be a fun and engaging activity for users of all ages, yet it the tagging system does have a learning curve. The low user engagement ratings may result partially from lack understanding of how to use the Tag! You’re It! and Freeze Tag! features. By explaining, in simple and direct language, why and how users can interact with art and each other, via social tagging, user frustration levels can be reduced and engagement increased. Explain the issues of understanding the directions, and the error message for the video (which happened in Safari)

User Testing | 7

Fig 3: By providing a static illustrated example of how to play the game on the landing page users get a better sense of how tags are

written and why.

Recommendation: Implement clear, simple language and consistent use of high visibility icons.While using jargon and slang can sense of comfort and familiarity with your users, especially with social tools, ensuring that the meaning of the slang is fully understood is challenging. Trying to be inclusive for your users is a wonderful motivation, yet terms and phrases are applied inconsistently and can be confusing to users, which ultimately creates an alienating experience. Phrases like “Nah, skip this one” (in Tag! You’re It!) and even the game titles themselves may be too strongly geared for familiarity rather than conveying information to a user.

Along with simplifying icon language to something most users are familiar with, such as “next”, “done”, and/or “end”, consistently placed larger icons would help users feel more confident in their actions on the website.

User Testing | 8

Recommendation: Require that users see the “Need Help?” pop-up before starting the games.The “Need Help?” section of Freeze Tag! is well done, clearly worded and informative. The use of simple language and illustrations ensures it is relatable to a wide range of users. A similar instructional tool would undoubtedly benefit users of Tag! You’re It!. Despite the valuable content, the “Need Help?” link is easily overlooked or purposefully ignored by users. Even users who were confused about what to do and read the “Need Help?” sentence did not click on it. By creating a lightbox style pop-up before users can start playing either game would ensure undivided user attention and not require that users realize or admit they do not understand the games. This could be an alternative for adding a landing page for the games, or could be included along with.

Fig 4: Examples of potential Posse game interfaces that welcome and guide the user through their task using consistent icons, language

and navigation.

User Testing | 9

Issue: If users clicked on an alternative view of an image, in either Tag! You’re It! or Freeze Tag they were taken to a different image, and were unable to get back to the image they were working on

Recommendation: Correct links to alternative views, or correct the code which automatically generates a new object when an image is clicked, or the back or forwards buttons is usedTo act on this recommendation may require expert assistance to locate and correct the programming error on the site, which will likely have a higher monetary cost than the other suggestions. It was included in this report because it is a major usability issue that has the potential to cause serious frustration and distrust among users and disables users from fully participating in or finishing their work in either game.

Fig 5: Ensure that the links for the alternative views do not generate new, unrelated content.

Issue: Gamification of crowdsourcing features was confusing and unengaging for usersThe social tagging of art by members of Posse is helping to make art more approachable to the general public, and as one tester commented these tags and activities “would be a wonderful teaching tool for children, to get them thinking about art.” The mission behind Tag! You’re It! and Freeze Tag, is excellent, yet for a majority of the users tested the game element of tagging was lost on them.

Recommendation: Changing the wording from gaming language to a direct appeal to users to contribute to a positive and important project. The game element made some users feel that there was a right and wrong way to contribute which was intimidating and lessened their desire to use the feature at all.

Recommendation: Give clearer explanation of the scoring system, as many felt it was arbitrary. Perhaps add actual incentives like free admission to ticketed shows, or other rewards for high quality and quantity participation.

User Testing | 10

Conclusion

The Brooklyn Museum is an outstanding resource for art and education. The museum has thoughtfully spent time creating ways for visitors of their website to engage with each others as well as items within the museum. Based on this user study evaluating how users find and use Posse, some usability issues impacted the overall experience of the user. One of the largest issues found was visibility. Users found it hard to locate Posse without using the site’s search box. While playing Tag! You’re It! and Freeze Tag users reported that they were unable to find and understand directions due to poor placement. Furthermore many icons were not large enough or where users would expect to find them. Visibility should be a high priority for a community tool like the ones on this site. Another issue frustrating to users was that if they clicked an alternate view of an image, the user was then unable to get back to the original image they were looking at. This problem discouraged users from playing the game as it seriously affected the functionality. The final important issue users were confused by was the gamification of the Posse tools. The game element made users feel that there were right and wrong answers which prevented them from wanting to contribute to the tags. Users also felt unsure of the purpose of the game.

These problems on the Brooklyn Museum Posse site are important to fix to improve the user experience of this unique new community. The recommendations laid out in this paper show simple yet effective improvements that will allow users to find and use Posse more easily. The museum must provide users with help and instructions the moment that they get to the site; a user should not feel confused. Providing tagging examples, creating consistent and logically placed icons, and placing Help options on the site will ensure users can find answers if they are confused. It should also be a quick fix for the museum to correct the alternative views links on each image. A user should not be thrown off and taken to a different image. Finally the museum should consider providing museum-related incentives for playing the Posse games. Using direct and simple language in the instructions may also ease users’ hesitations to play the games. Once these improvements have been implemented, the Brooklyn Museum will have a strong community that will be seen on and offline.

User Testing | 11

Appendix & References

ScriptSET UP: 2 pens (one for testers to sign consent form with)water on hand and cupOpen Website: on the Mac’s adjust the website window to be smaller to reduce the white edges. then minimize the page.Set up Morae Recorder

Welcome the participant -- with a friendly greeting, ask them if they would like a glass of water. Tell the participant to make themselves comfortable.

Once the participant is settled in, introduce yourself and any other researchers in the room, and thank the par-ticipant for agreeing to engage in the user test today.

Explain that you will be reading from as script, which is not intended to be impersonal, but rather to maintain consistency across all the tests which are being carried out by multiple testers.

START SCRIPT:Thank you again for being here today. Our team is conducting usability research on a feature of the Brooklyn Museum website calle “Posse”. Your voluntary participation in this user study will enable our research team to more accurately assess usability issues and improve overall user experience of the Brooklyn Museum website. We will be testing the website, NOT you!

We will be using a video software that records your image while you work through the testing, as well as recording the movements of the mouse on the computer screen. Recording the testing sessions allows our research group to go back and review specific issues. These videos are for research purposes only and will never be released.

The test will be conducted in this order:First we will ask you to complete a pre-test questionnaire which gathers demographic information.

Then you will be asked to complete 4 tasks using the Brooklyn Museum website. At the end of each of these 4 tasks there will be 1 question about your opinion of the task.

After all 4 tasks have been completed there will be a post-test questionnaire.

As mentioned before your participation is completely voluntary, and you are welcome to stop the test at any-time, for any reason. “Do you have any questions so far?”Before we start the test you will need to read and sign our participant consent form. If for any reason you are uncomfortable with any elements of the consent form please let us know and we can end the session.

HAND TESTER 2 CONSENT FORMs WITH YOUR NAME AND EMAIL ALREADY FILLED INlet them read it and sign both copies and keep one for their records, return the other to you.

Thank you for your consent, now we can move on to the pre-test questionnaire. Please answer as accurately as you can, these are for our records only and there is no right or wrong answer.

User Testing | 12

User Testing | 13

Before we start the first task here is a little background information:Your friend told you about an interactive feature on the Brooklyn Museum website called Posse that includes games related to the museum’s collections. You are interested in trying out the feature for yourself.

Please read each task outloud and then complete them at a comfortable pace. During the testing please think out loud, as it helps our research. Participate in the games as if you were a member of the community. Don’t worry we’re testing the website, not you.

Do you have any questions?

PULL UP THE BROOKLYN MUSEUM HOMEPAGE

INTRODUCE THE FIRST TASK

For this first task please find Posse and sign in to Posse with the user name: usability_tester and password: lis644. PROVIDE THEM WITH A SHEET WITH THIS INFORMATION WRITTEN ON IT

When participant signs in and is navigated to the profile page ask them if this “Is this the page you expected to be taken to?”

THEN END THE TASK

For the 2nd task please play “Tag You’re It”. Write tags for more than one object, playing until you feel ready to move on.

While the participant is on the “Tag You’re It” page ask “How do you feel about this page?”

THEN END THE TASKFor the 3rd task play “Freeze Tag”. Edit tags for more than one object, playing until you feel ready to move on.

When user locates “Freeze Tag” ask them if they thought that navigating to the new game was hard or easy.

While the participant is on the “Freeze Tag” page ask them how they feel about the page.

THEN END THE TASK

For the final task go to your profile and review your tags.

While the participant is in the Profile page ask them how they feel about the page and what information they expected to see in the profile.

END TASK

Now that all the tasks have been completed please fill out the post-test questionnaire.

Before we are completely done with this session if you could indicate how you feel now that you have complet-ed the tasks using this emotion matrix

User Testing | 14

PROVIDE EMOTION MATRIX

Do you have any questions about the study or the website?

Thank you so much for taking the time to participate in our study. Your contribution today will help our team to make suggestions with the most benefit.

If you have any questions please don’t hesitate to contact us using the email on the consent form.

ESCORTE TESTER OUT

Testing QuestionsPre-Test QuestionsWhat gender do you identify with? Male Female Other

In which age range do you fall?18-2425-2930-3536-4041-5051-6565+

What is your highest level of education completed?High School Some CollegeBachelor’s DegreeSome Graduate workMaster’s DegreePost Masters

How would you define your internet capabilities?Novice (little to no experience)Intermediate (everyday user)Expert (basic understand of web design)

Do you have an interest in art?YesNoNot Sure

Have you been to a museum in the last 6 months?Yes No

Do you browse museum websites?YesNo

User Testing | 15

Are you familiar with social tagging (hastags, folksonomies, etc.)?Yes NoNot Sure

Task 1Find Posse and sign in to Posse.Use the login provided:User name: ibierlei@pratt.eduPassword: lis644

Task 1 - Please indicate how easy or difficult it was to complete Task 1Overall this task was?v bad = 1 v good = 5

Task 2 Play “Tag You’re It”.Write tags for more than one object, playing until you feel ready to move on.

Task 2 - Please indicate how easy or difficult it was to complete Task 2Overall this task was?v bad = 1 v good = 5

Task 3Play “Freeze Tag”. Edit tags for more than one object, playing until you feel ready to move on

Task 3 - Please indicate how easy or difficult it was to complete Task 2Overall this task was?v bad = 1 v good = 5

Task 4 Description: Go to your profile and review you tags.Instructions: Go to your profile and review you tags.

Post-Test QuestionnaireComplete the following questionnaire based on your overall experience completing the tasks.

SUS1. I think that i would like to use this website frequently 2. I found the website unnecessarily complex3. I thought that the website was easy to use.4. I think that I would need the support of a technical person to be able to use this website.5. I found the various functions in this website were well integrated.6. I thought there was too much inconsistency in this website.7. I would imagine that most people would learn to use this website very quickly.8. I found the website very cumbersome to use9. I felt very confident using the system.10. I needed to learn a lot of things before I could get going with this system.

User Testing | 16

Notes from Test 1• Both participants (my parents, in their early 50s) used the search bar to find Posse, Tag, You’re It and Freeze Tag,

indicating that the site is not straightforward enough with regards to Posse• Mom pointed out the difference between ‘done with this one’ and ‘end task’ on Tag, You’re It is unclear and

should be worded differently• Dad pointed out that ‘I’m done with this one’ isn’t a good color and needs to be more differentiated from the

rest of the text to be revealed as a link• Dad also pointed out that the ‘switch games’ bar is located in a bad place and needs to be made more visible • There is no immediate link back to the user’s profile after Freeze Tag is completed, which makes it difficult for

the user to navigate back

Suggestions based on my data: breadcrumbs, links to the profile on all pages, increased promotion of the Posse fea-ture to make it more visible, move the switch games tab to a more obvious location, adjust the color and phrasing of ‘I’m done with this one’

Notes from Test 2• Two participants (a 23yr old everyday user and a 50yr old novice user)• Both participants thought they would find Posse under the Education tab thinking that is where educational

games would be. They both resorted to using the search box to find Posse.• Novice User had a very hard time with navigating the homepage: “This is a weird looking webiste”• Both users did not expect to land on the tags page when they logged in to Posse - they expected to be in a games

page.• Both users watched the video on Tag You’re It (only the novice user watched the full video, thinking there would

be game instructions).• Everyday User on Tag You’re It: “This does not intuitively makes sense to me”, “For an interactive game, this isn’t

very fun”, “This seems like a beta version”.• Novice User on Tag You’re it: “I don’t understand this game and the directions are not clear enough to me”.• The End button on Tag You’re It is small and hidden. • Both users were confused with I’m Done, Skip this One, etc - they had a hard time understanding how to sub-

mit tags for an object. • Novice User was not able to locate Freeze Tag, went back to home page and then education again - did not

attempt searching it in search box. • Everyday User on Profile page: “I don’t know what I’m looking at”• Everyday User found the Collections: Play page and said that she wish she had known about this sooner because

it was much easier to use. • Both users were very frustrated using the site.

Biggest Problems: Icons and Instructions are not placed prominently or are not large enough for users to see. Game icons for both games were not similar which caused confusion for the user. Posse is not intuitive under Community label. Profile area/page is very disorienting to the user. Large video on Tag You’re It is misleading. The games do not have enough positive feedback/interesting interaction to encourage a user to keep playing.

User Testing | 17

Notes from Test 3• Two participants (a 24 year old expert and a 65 year old novice)• Both participants utilized the search bar to locate Posse. • The older user went directly to the search bar and did not mind being redirected to Google.• The younger user looked first around the website and was surprised to not find a log-in area, she then looked in

education, collection and finally community. In community she did not see the Posse link in the navigation and turned finally to the search bar in order to locate Posse.

• Both users found the concept of the game to have not been well presented• The older user assumed that tagging was related to ‘pinning’ (like as in Pinterest) and did not understand initially

what information should be put into the text box. She also was put off by the idea of tagging art that she did not want appearing on her profile, as she had seen other users’ profiles which consisted mainly of tags and thumb-nails of tagged art. The instructions on “Tag you’re it” were confusing to the user as she thought that the infor-mation stated was most likely available in the the tombstone information, and that it seemed strange to copy this information in tags. This user did not watch the video and stated that she would probably not watch it as she expects to be able to jump into a game.

• The younger user understood what tags are but noted that it was odd how little instruction was available within the game.

• The older user found “Freeze tag” quickly, but did not find the instructions to be helpful. She thought felt that she didn’t know enough context about the art or other players to judge many of the tags and also that it seemed strange that so many of the flagged tags were blatantly wrong (“this is just math?”) She thought to make it a game some of the tags presented should be right.

• The younger user had a lot of difficulty finding “Freeze Tag” as she did not see the small link to it in the game in “Tag you’re it” she had to retrace her steps and ended up just using the search bar again, which she stated was “stupid and disappointing”

• The younger user found “Freeze Tag” easier to use but was less engaged. “This isn’t very fun” and was also sur-prised to find “blatantly wrong” tags being presented to users in the game.

• The older had a very hard time locating her profile. The younger user also had a hard time and stated that it was strange that there was no indication on each page that you are logged in.

• When the older user clicked on a tag and saw all of the images that had been tagged with that word she was incredibly impressed and thought that it was amazing and a great way to look at a museum website.

• The younger user did not like her experience on the site at all and found it hard to navigate and boring. She assumed that the games were prototypes.

• The younger user also noticed that the justification changed between both games. “Tag you’re it” is center aligned and “Freeze Tag” is left justified.

• The older user stated that she did not enjoy the games and found them very hard to use but she ultimately judged the website lightly as she thought that users who really wanted to utilize the site would figure out how to.

• Both users found Posse to be a strange name for the community.

Biggest Problems: Posse navigation is not well integrated into the navigation of the larger website. Users had a hard time locating login/logout information, and the games themselves. The games also lacked explicet descriptions and guides. Users had a hard time understanding the concept of a tag and also often felt unsure of how they were contributing.

User Testing | 18

Notes from Test 4• Two participants (a 30 year old intermediate and a 51-65 yr old expert users)• Both participants utilized the search bar in an attempt to locate Posse almost immediately.• Expert user was thrown off by Google style result of search, went back to home page thinking there had been

an error and tried all menu options saying “I really feel like a moron.” User gave up before locating posse. Intermediate user reported that task 1 was easy, Expert user reported it to be difficult.

Tag! You’re It!• For Task 2 Intermediate user looked under photos and ended up on the BK museum’s filckr site. Then tried

collections comments before finding the “Tag You’re It” on the right side of the posse landing page. • The Tag You’re It video failed to load for both users. The intermediate user reported “I don’t understand what

I’m supposed to say [when tagging]” when provided only with the directions on the Tag! You’re it! landing page.

• Both users found the directions on the Tag You’re It! landing page unclear, with the example of tag formatting confusing/misleading.

• Intermediate user found the directions on the Tag You’re It! page unclear to the point of being too uncomfort-able to tag. “I just don’t understand why you would want to tag something or where it is actually tagging it” “I found it [tag you’re it] but I don’t know what to do with it.”

• Expert user accomplished tagging but had to use trial and error to understand the formatting of the tags, first attempted to use semicolons instead of spaces to separate tags and encountered several error messages. “If I got an error message too many times I’d just skip the whole thing” but “after I got through the first [tagging] it wasn’t so difficult.” Expert never mastered the use of quotes in order to make phrases.

Freeze Tag• both users found this feature easily• Both users had uncertainties about how to play the game but neither used the “Need Help?” button.• Intermediate user was confused about why certain tags were listed in the first place• Expert user initially thought the challenged tags for an object were user names since they didn’t appear rele-

vant to the art object. • When clicking on an alternative view while playing the game Expert user was taken to a different piece of art

entirely and was unable to find the original item by using the back.• Expert user “I really do like this concept, it’s a really cool game.”• Find Your Profile• Expert user was unable to find full profile from Freeze Tag. Tried “stats and standings” and clicked on “change

my settings” and then gave up• Intermediate user went to community/posse/profile Overall• Both users commented on the high tagging number user “SKroft”, this seemed distracting• Intermediate user expressed confusion over the motivation of why someone would participate in these ele-

ments of the website in the first place. “I don’t understand why it’s a game” referring to both Tag! You’re It! and Freeze Tag.

• The scoring element of the games was unclear to both users.• Intermediate would not use the posse feature frequently as they would be more likely to use a collections sec-

tion rather than playing games. User commented that the names were gimmicky and didn’t give any indica-tion of what they were about. Overall the user found the feature unengaging.

• Expert user reported would like to use posse frequently but did not feel confident using the website and felt it was very difficult to use. Felt you didn’t need to learn a lot to get going on the website “just need to have the patience to go through the website.” They found the website very engaging and a 6 in terms of beauty.

• Biggest Issues: Visibility of posse, community is not an obvious menu heading to find it. Directions for both games are too vague, and Tag! You’re it! in particular is too dependent on the video, which did not always load, leaving users little to no direction. If user click on the “alternative views” of art in either game they are taken to an entirely new piece of art, this is a catastrophic error.

User Testing | 19

User Demographics

The graphs below were created using the data collected from users during the pre-test questionnaire.

User Testing | 20

References

Nørgaard, M., & Hornbæk, K. (2006, June). What do usability evaluators do in practice?: an explorative study of think-aloud testing. In Proceedings of the 6th conference on Designing Inter-active systems (pp. 209-218). ACM.