A Usability Analysis / Comparison of Fraternity...

A Usability Analysis / Comparison of Fraternity Websites

College of Information Sciences and Technology The Pennsylvania State University

University Park, PA 16802 December 17th, 2007

Team 7

Sean Smith [email protected] Andrew Burgdorf [email protected] Kevin Ahrens [email protected] Kim Buchter [email protected]

Usability Analysis 1

Abstract The purposes of the Phi Sigma Pi and Theta Chi fraternity websites are the same, to provide information to users; however, the differences in usability between the two websites cause their effectiveness in fulfilling this purpose to differ. This document attempts to evaluate the usability of the two sites through multiple comparison analyses. The following feature’s usability will be compared: link color, color scheme, contacting executives, navigation and the way users visually search the sites. The end results of this evaluation will be improvement recommendations to the webmasters of each site on the basis of the following usability criteria: visual clarity, consistency, informative feedback, explicitness, appropriate functionality, flexibility and control, error prevention and control, and user guidance and support.


Table of Contents

1. Introduction 3

2. Background of Websites 4

3. Comparison Analysis 5 - 25

3.1. Task Analysis 5 - 13

3.1.1. Task Analysis – National History / Contacting Executives 6 - 7 3.1.1.1. Participants 6 3.1.1.2. Methodology 7 3.1.1.3. GOMS / KLM Analysis 7

3.1.2. Results Analysis 13

3.2. Aesthetics 15 - 16

3.2.1. Survey Questionnaire 15 - 16 3.2.1.1. Participants 15 3.2.1.2. Methodology 15 3.2.1.3. Survey Results 16


3.3. Color Scheme 16 - 22

3.3.1. Visual Search 17 - 22 3.3.1.1. Participants 17 3.3.1.2. Methodology 17 3.3.1.3. Perceptual Interaction Results 19


3.4. Navigation 23 - 25

3.4.1. Observational Experiment 23 - 24 3.4.1.1. Participants 23 3.4.1.2. Methodology 23 3.4.1.3. Interview Results 24


4. Recommendations for Improving Usability 26 - 28

4.1. Task Analysis 26

4.2. Aesthetics 27

4.3. Color Scheme 28

4.4. Navigation 28

5. Conclusion 29

6. References 30


1. Introduction At The Pennsylvania State University, students have the opportunity to join several different

social or academic groups. Because of the widespread use of the internet, all of these groups

have websites that are used by either their respective members or other university students. In

this paper, two Penn State fraternity chapter’s websites will be evaluated through comparison:

Phi Sigma Pi, a national academic fraternity, and Theta Chi, a national social fraternity. The

purpose of this evaluation is to determine usability, that is, how easy the websites are to use.

This document will analyze important tasks, aesthetics, color scheme, and navigation of both

websites in order to provide usability recommendations based on the criteria of visual clarity,

consistency, informative feedback, explicitness, appropriate functionality, flexibility and control,

error prevention and control, and user guidance and support (Ritter & Churchill, 2007, p. 11).

The websites can be found online at:

• Theta Chi - http://www.greeks.psu.edu/ifc/tx/

• Phi Sigma Pi - http://www.clubs.psu.edu/up/PhiSimgaPi/


2. Background of Fraternities Phi Sigma Pi is a co-ed, honors fraternity which is composed of males and females. Theta Chi, a

social fraternity, is a male-only organization. The users of these sites are predominantly brothers

of the fraternities, students interested in joining the fraternity, as well as alumni of the fraternity.

Having a website is in the best interest of the fraternities because they provide information about

the chapter, information about the national fraternities, the location of the houses, and

information for pledging.


3. Comparison Analysis The subsequent sections will be studying the usability of the Theta Chi and Phi Sigma Pi

website’s important tasks, aesthetics, color scheme, and navigation. These comparison studies

will be done through experiments and the results of the experiments will lead to usability

improvement recommendations.

3.1 Task Analysis In an attempt to better understand how users complete a task, a Task Analysis is performed and

studied. From a Task Analysis, it is possible to discover and forecast how long users will take to

learn the task, the length of time required to complete the task, and the consistency of the

interface (Ritter & Churchill, 2007, p. 170). By reviewing two fraternity websites, we hope to

gain a better understanding of how users (members, potential members, or alumni) interact with

these websites.

3.1.1 Task Analysis – National History / Contacting Executives Two common tasks on each website were analyzed to understand how users use the sites. First,

participants were asked to find the National History of each fraternity, as well as contact the

executive members of the fraternity. Both a GOMS analysis and KLM analysis were performed.

A hierarchical task analysis, which is a way to show tasks divided by subtasks through showing

their order, is included below for each of the experiments.

When contacting the executive board, the participants were asked to discover how to send a

message to the President of the Chapter. This included entering their email addresses, names,

and a brief message (“Hello”).

Table 1: Theta Chi Task Analysis for Contacting the Executive Board 1 Click Splash Screen 2 Choose Contact (on side navigation bar) 3 Choose Name Field Enter Name 4 Choose Email Address Field Enter Email Address


5 Choose Message Field Write Message 6 Choose Send Button

Table 2: Phi Sigma Pi Task Analysis for Contacting the Executive Board

1 Choose Contact Us Navigation Button (at top of screen) 2 Choose Email Field Enter Email Address 3 Choose Name Field Enter Name 4 Choose Body Field Write Message 5 Choose Submit Button

For the National History Task Analysis, the participants were asked to find out about the

National History of the fraternity by navigating through the site.

Table 3: Theta Chi Task Analysis for Finding National History 1 Click Splash Screen 2 Choose History Button (on side navigation bar)

Table 4: Pi Sigma Phi Task Analysis for Finding National History

1 Click History Navigation Button (at top of screen) 2 Choose National History Tab

These tasks were chosen because they are potential tasks that could be performed by a brother in

the fraternity, potential brothers, or possibly even alumni.

3.1.1.1. Participants All participants were male IST sophomores, juniors, and seniors who were willing to be involved

in our experiment after being approached within the IST building. While they had never used

either of the fraternity websites, they were expert users of the internet and were very comfortable

with computers. The participants were placed in front of a desktop computer with a standard Dell

Keyboard and Optical Dell Mouse (with a scroll wheel).


3.1.1.2. Methodology To complete the GOMS and KLM analyses, the index page of each of the fraternities’ websites

in Mozilla Firefox was opened. The user was timed after they clearly understood what task they

needed to complete. An online Stopwatch tool was used to measure the time that it took to

complete each task.

In addition to recording the participant’s time, we also observed the user’s behavior to

understand how they used the site. Before we started recording the time of the user’s actions, we

made sure that the mouse pointer was hovering in the Address Bar of the web browser. This

provided a more “realistic” approach, because it could be very likely that the user had just

clicked on the Address Bar and typed in the web address. The first experiment tested how easy it

was for a person to contact the president of the fraternity. The second experiment tested how

easy it was for a person to navigate and find the National History of the fraternity.

3.1.1.3. GOMS/KLM Results To review these websites, two types of Task Analysis will be completed. First, a GOMS

analysis was completed, followed by a KLM analysis. The GOMS model, first introduced by

Card, Moran, and Newell in 1983, describes the knowledge required to complete a task on a

system. It is an acronym standing for Goals, Operators, Methods, and Selection Rules. (Kieras,

2006, p. 2).

When performing a GOMS analysis, we reduced the user’s interaction with the websites down to

their elementary actions. This allows us to categorize the goals, methods, operations, and

selection rules of the tasks. The websites were laid out similarly; therefore, comparing the

GOMS analysis between the two websites produces a similar result.

In our project GOMS allowed us to break our goal down into multiple goals. Table 5, Table 6,

Table 7, and Table 8 show the GOMS models derived for the tasks analyzed for both fraternities,

Phi Sigma Pi and Theta Chi.


Table 5: GOMS Analysis for Theta Chi when emailing the president.

Task item: T1 Name is T1. Click Splash Screen. Next is T2. Task item: T2 Name is T2. Choose Contact. Next is T3. Task item: T3 Name is T3. Choose Name Field. Next is T4. Task item: T4 Name is T4.

Enter Name. Next is T5. Task item: T5 Name is T5.

Choose Email Address Field. Next is T6. Task item: T6 Name is T6. Enter Email Address. Next is T7. Task item: T7 Name is T7. Choose Message Field. Next is T8. Task item: T8 Name is T8. Write Message. Next is T9. Task item: T9 Name is T9. Choose Send Button. Next is T10. Task item: T10

Name is T10. Done.

Method for goal: Contact President Step 1. Store T1 under <current task name>. Step 2. Check for done. Decide. If <current task name> is T10,


Then Delete <current task>; Delete <current task name>; Return with goal accomplished. Step 3. Get task item whose Name is <current task name> And store under <current task> Step 4. Accomplish goal: Contact President Step 5. Store Next of <current task> under current task name> Goto Check for done.

Table 6: GOMS Analysis for Phi Sigma Pi when emailing the president.

Task item: T1 Name is T1. Choose Contact Us Navigation Button. Next is T2. Task item: T2 Name is T2. Choose Email Field. Next is T3. Task item: T3 Name is T3. Enter Email Address. Next is T4. Task item: T4 Name is T4.

Choose Name Field. Next is T5. Task item: T5 Name is T5. Enter Name. Next is T6. Task item: T6 Name is T6. Choose Body Field. Next is T7. Task item: T7 Name is T7. Write Message. Next is T8. Task item: T8 Name is T8. Choose Submit Button. Next is T9.


Task item: T9 Name is T9. Done. Method for goal: Contact President Step. Store T1 under <current task name>. Step. Check for done. Decide. If <current task name> is T9, Then Delete <current task>; Delete <current task name>; Return with goal accomplished. Step. Get task item whose Name is <current task name> And store under <current task> Step. Accomplish goal: Contact President Step. Store Next of <current task> under current task name> Goto Check for done.

Table 7: GOMS Analysis for Theta Chi when navigating to the National History.

Task item: T1 Name is T1 Click Splash Screen Next is T2 Task item: T2 Name is T2 Choose History Button

Next is T3 Task item: T3 Name is T3 Done Method for goal: Find History Step. Store T1 under <current task name> Step Check for done Decide: If <current task name> is T3, Then Delete <current task> Delete <current task name> Return with goal accomplished Step. Get task whose Name is <current task name> and store under <current task> Step. Accomplish goal: Find History Step. Store Next of <current task> under <current


task name> Goto Check for done

Table 8: GOMS Analysis for Phi Sigma Pi when navigating to the National History.

Task item: T1 Name is T1 Click History Button Next is T2 Task item: T2 Name is T2 Click National History Tab

Next is T3 Task item: T3 Name is T3 Done Method for goal: Find History Step. Store T1 under <current task name> Step Check for done Decide: If <current task name> is T3, Then Delete <current task> Delete <current task name> Return with goal accomplished Step. Get task whose Name is <current task name> and store under <current task> Step. Accomplish goal: Find History Step. Store Next of <current task> under <current task name> Goto Check for done

Keystroke Level Model A simpler form of the GOMS model exists that allows researchers to compute the time required

to complete a task. This is called the Keystroke Level Model. In order to perform a Keystroke

Level Analysis, we first had to identify the keystrokes used in the task analysis. We utilized

Fitt’s law to determine the time it takes for the user to move the mouse to specific places on the

screen. The end results between the two fraternity websites were nearly the same because the

tasks were extremely similar between the two sites. A problem we ran into was the times


produced by Fitt’s Law were small, prompting us to use a default time when performing our

calculations. (Ritter & Churchill, 2007, p. 182).

The following tables provide the KLM models for each experiment for both Theta Chi and Phi

Sigma Pi as well as the computed times. Please note that all [mouse] functions are a left click,

all times are in seconds, and Fitt’s Law was used to calculate the time for pointing the Mouse

(P).

Table 9: KLM model for Theta Chi when emailing the president.

1. Click Splash Screen. H[mouse] P K[mouse]

2. Choose Contact Button. M P K[mouse] 3. Choose Name Text Box. M P K[mouse]

4. Type Name. H [keyboard] M K[keyboard] 5. Choose Email Text Box. H[mouse] M P K[mouse]

6. Type Email Address. H [keyboard] M K[keyboard] 7. Choose Message Text Box. H[mouse] M P K[mouse]

8. Type Message. H [keyboard] M K[keyboard] 9. Click Send Button H[mouse] P K[mouse]

Time Predictions Texecute = [6 tK1 + 1 tkname + 1 tkemail +1 tkmessage +6 tP + 7 tH] + 7 tM

= 6(0.15) + 4.004 + 4.48 + 1.68 + 6(1.03) + 7(0.57) + 7(1.35) = 30.684 seconds

Table 10: KLM model for Phi Sigma Pi when emailing the president.

1. Choose Contact Button. H[mouse] M P K[mouse] 2. Choose Email Text Box. M P K[mouse]

3. Type Email Address. H [keyboard] M K[keyboard] 4. Choose Name Text Box. H [mouse] M P K[mouse]

5. Type Name. H [keyboard] M K[keyboard] 6. Choose Message Text Box. H[mouse] M P K[mouse]

7. Type Message. H [keyboard] M K[keyboard] 8. Click Submit Button. H[mouse] P K[mouse]


Time Predictions Texecute = [5 tK1 + 1 tkname + 1 tkemail +1 tkmessage + 5 tP + 7 tH] + 7 tM

= 5(0.15) + 4.004 + 4.48 + 1.68 + 5(1.03) + 7(0.57) + 7(1.35) = 29.504 seconds

Table 11: KLM model for Theta Chi when navigating to the National History Page.

1. Click Splash Screen. H[mouse] P K[mouse]

2. Choose History Button. M P K[mouse]

Time Predictions

tP = Texecute = [2 tK + 2 tP + 1 tH] + 1 tM

= 2(0.15) + 2(1.03) + 1(0.57) + 1(1.35) = 4.28 seconds

Table 12: KLM model for Phi Sigma Pi when navigating to the National History Page.

1. Click History Navigation Button. H[mouse] P K[mouse] 2. Choose National History Tab. M P K[mouse]

Time Predictions Texecute = [2 tK + 2 tP + 1 tH] + 1 tM

= 2(0.15) + 2(1.03) + 1(0.57) + 1(1.35) = 4.28 seconds

3.1.2 Results Analysis

By using the Keystroke Level Model, we were able to predict the time it would take users to

complete the task. This was accomplished by dividing the overall task into many smaller tasks.

We computed a predicted time for each small task and added them up to find a predicted time for

the overall task. We predicted that it would take 4.28 seconds to find the national history for


Theta Chi. We predicted it would it would also take 4.28 seconds to find the national history for

Phi Sigma Pi. The time similarities occur because each task involved very similar steps. In total,

our group predicted it would take 30.68 seconds to contact the president of Theta Chi and 29.50

seconds to contact the president of Phi Sigma Pi. The KLM does not take many factors into

consideration (color scheme, fonts, etc…); however, it gives us an objective analysis comparing

the two websites.

The four subject’s data is listed below in Table 13. The average time it took the subjects to find

Theta Chi’s national history was 5.15 seconds and 4.25 seconds for Phi Sigma Pi. The average

time it took the subjects to contact the president of Theta Chi was 17.71 seconds and 17.43 for

Phi Sigma Pi. The main reason why the predicted times for contacting the president were longer

than the test results is because the users all used the tab key to transition to the next text box,

while the predicted time theorized that users would use the mouse to click the next text box. The

test results for finding the national history was very close to the predicted time of 4.28 seconds,

because the task was very quick and straightforward.

Table 13: Results of all four subjects.

Emailing the President Theta Chi

Subject 1 – 19.68s Subject 2 – 15.00s Subject 3 - 12.46s Subject 4 – 23.72s Average – 17.71s

Phi Sigma Pi Subject 1 – 19.82s Subject 2 – 21.51s Subject 3 - 16.04s Subject 4 – 12.35s Average – 17.43s

National History Theta Chi

Subject 1 – 6.23s Subject 2 – 4.51s Subject 3 - 5.88s


Subject 4 – 3.98s Average – 5.15s

Phi Sigma Pi Subject 1 – 3.18s Subject 2 – 3.52s Subject 3 - 6.31s Subject 4 – 4.02s Average – 4.25s

3.2. Aesthetics Many expertise believe that the aesthetics, or how pleasant something looks, of an interface is

extremely important to the usability of that interface (Ritter & Churchill, 2007, 12). Therefore,

when evaluating the two fraternity websites, it is critical that we take aesthetics into account.

The problem with aesthetics is that it is subjective, meaning it is a matter of opinion that changes

from user to user. In order to correctly gauge the aesthetics of the two websites, we administered

a verbal survey.

3.2.1. Survey Questionnaire This survey consisted of one general question, which was verbally administered.

3.2.1.1. Participants

The verbal survey was administered to a group of 41 students in an Interface Design class. The

participants were both male and female IST majors which are very experienced internet users.

3.2.1.2. Methodology

The survey was administered by our group when we had the class’s attention. Using the

projector, we brought up both websites simultaneously and pared them side-by-side on the

screen. This allowed both websites to be viewed simultaneously by all students (Theta Chi on

Left, Phi Sigma Pi on Right). After allowing the students ample time to view the sites, the

question was posed to them. To gather the results, we first asked for the students who found the

left site more visually appealing to raise their hand. After counting and recording the number,

we asked them to lower their hands. Next, we asked the students who favored the right site to

raise their hand; the results were then counted and recorded.


3.2.1.3. Survey Results

We verbally asked the class the following question: “Which website is more visually appealing

to you?”

3.2.2. Results Analysis The results of the survey were very clear-cut; students preferred the look of Theta Chi’s website

over Phi Sigma Pi. Multiple students justified their position stating that Theta Chi’s website had

side navigation that they were accustomed to, and looked more professional and clean cut. A

problem with our surveyed users is that because all the participants were educated in the area of

user interface design, they could have a bias toward certain aesthetics. This being the case, it

would not be a representative sample of normal fraternity website users and the results would be

less meaningful.

Table 14: Students Response to Aesthetically Pleasing Website Question

For Theta Chi For Phi Sigma Pi Number of Students 32 9

3.3 Color Scheme The choice of the colors used for a website is important, because it can create visual clarity,

consistency, and raise the aesthetic appearance of the site. Color is a result of perception and can

change under differing conditions of lighting or contrast, but it can provide information,

warming, and change the mood of a display. Although a user can identify up to 8,000 shades of

colors when compared side-by-side, it is only possible to recognize 8 to 10 distinct colors when

the colors are separated (Ritter & Churchill, 2007, pg. 73).

To understand how users perceive, a brief overview of visual perception will be given. Visual

receptors within the eye allow the user to see electromagnetic energy from 400 to 700

nanometers. The place where the image is rendered from the lens, the retina, has different light

sensitivity throughout. In addition to the lens and retina, the rods and cones are the receptors that

distinguish between alternating light waves (Ritter & Churchill, 2006).


3.3.1. Visual Search The capacity to quickly scan and acquire information from a website is critical to the usability of

that site. When a user visits the home page of a website, the information that he or she is looking

for may not be on the site. However, the home page contains many links that will enable the

user to find the specific information they are looking for.

If you are visiting a website for the purpose of finding a specific hyperlink, you would glance

over the entire website, while searching for the familiar blue hyperlink color. Only after you

have located the hyperlink’s color, would you determine if it is the correct link. Therefore, it is

very important that the visual cues for hyperlinks stand out when dispersed throughout the

website’s distracting text and pictures. The purpose of this experiment was to determine which

color hyperlinks, when dispersed throughout the two different websites, the user is able to

identify the least amount of time.

3.3.1.1 Participants

Four volunteer subjects, all male IST students found in the IST building, were used for this

experiment.

3.3.1.2 Methodology

Our experiment had two parts; each part consisted of two phases. Using the Visual Search tool

created by Dr. Tom Busey from Indiana University - Bloomington, we created distracters based

on the colors used in the two websites. The Visual Search Tool is used for searching within an

image for a specific target and recording the completion time to find this target. It is available

online at <http://cognitrn.psych.indiana.edu/CogsciSoftware/VisualSearch/index.html>.


Figure 1: This image shows the visual search field for Experiment 4. There is a blue target with white, black, dark yellow, and dark red distracters.

Figure 2: This image shows the visual search field for Experiment 2. There is a red target with purple, grey, yellow, and black distracters.

We ran four experiments on each of these subjects, and they reviewed 8 images in each

experiment. Four images had the target present, and four had the target absent. If the subject

saw the target present, they pressed “f” on the keyboard. If they saw that the target was absent,

they pressed “j”. There were 10 rows and 10 columns, so there were 100 dashes in each trial.


By running this experiment on four subjects, we were able to tell if the four different colored

distracters allowed a person to easily search for a target: blue and red. Blue was chosen because

it is both the color of links on the Phi Sigma Pi site and because it is the default for links in

HTML. Red was chosen for comparison, and besides providing a contrasting color, it is the

color of links on Theta Chi’s site. Distracters and targets all had the same shape: they were a

dash (-). The distracter colors were the color schemes from two fraternity websites, and the RGB

values are listed below.

Table 15: The distracter colors used based off of Phi Sigma Pi website.

RGB Value Distracter Color Red Green Blue Purple 102 0 102 Grey 153 153 153 Yellow 255 255 0 Black 0 0 0

Table 16: The target colors used based off of Theta Chi’s website.

RGB Value Distracter Color Red Green Blue White 255 255 255 Dark Yellow 204 204 0 Dark Red 153 0 0 Black 0 0 0

Table 17: The target colors used.

RGB Value Target Color Red Green Blue Blue 0 51 255 Red 255 0 51

3.3.1.3 Perceptual Interaction Results

After running the experiments, the results on which color links work better for each of the two

fraternity’s websites became evident. Below are the results for the two experiments measuring

the time it takes a person to find a link with the colors from the Phi Sigma Pi website.


Table 18: Target Present with purple, grey, yellow and black distracters

Red (ms) Blue (ms) Subject 1 918 1137 Subject 2 1094 891 Subject 3 1422 711 Subject 4 1410 680 Average 1211 854.75

Table 19: Target Absent with purple, grey, yellow and black distracters

Red (ms) Blue (ms) Subject 1 1320 1586 Subject 2 1410 1632 Subject 3 2090 1410 Subject 4 2102 832 Average 1730.5 1365

Below are the results for the two experiments measuring the time it takes a person to find a link

with the colors from the Theta Chi website.

Table 20: Target Present with white, dark yellow, dark red and black distracters

Red (ms) Blue (ms) Subject 1 1629 774 Subject 2 973 562 Subject 3 754 594 Subject 4 1328 758 Average 1171 672

Table 21: Target Absent with white, dark yellow, dark red and black distracters

Red (ms) Blue (ms) Subject 1 2207 1113 Subject 2 1418 890 Subject 3 1348 742 Subject 4 1969 804 Average 1735.5 887.25


As exemplified in the tables, participants had were able to identify more quickly when the target

was present or absent when the target color was blue, regardless of whether the purple, grey,

yellow and black distracters (Phi Sigma Pi) were used, or if white, dark yellow, dark red and

black distracters (Theta Chi) were used.


3.3.2 Results Analysis

Understanding how users visually search a webpage is a critical aspect of usability design

because, on the internet, vision is the most important sense. The visual outputs of a website can

allow a user to easily find information, or conversely, frustrate him or her.

In our experiments to determine if the two websites used easily identifiable colors for links, we

recorded the amount of time to find the target color. Ideally, users of a website should want to

be able to find information quickly, and be able to distinguish plain text from hypertext. The

outcome of this experiment shows whether the color scheme of the two websites allows the users

to easily find links or targeted information or if the colors could be modified to better suit users.

This experiment was particularly designed for hyperlinks; we utilized the color scheme of the

website for the distracters and then used the target color as the link color. It is important for

links to have a different color than the rest of the text because laws of visual organization suggest

that it is important to maintain similarity and a common fate (Ritter & Churchill, 2006). This

means that text that maintains the same color will be seen as grouped together, so the user will

know that this similarity leads to the common fate of linking to a new page.

Knowledge about visual search, which is the time-consuming procedure of looking for a certain

object within the visual scene (Busey, p. 3), was the outcome of this experiment. By measuring

the reaction time, we are able to conclude how quickly the subject was able to finish his visual

search. Our results show that there can be considerable differences in the reaction time to

finding the target object. For example, when distracter colors prevalent on the Theta Chi website

were used, it is obvious that the subjects found the blue target much easier to locate, because the

reaction times were much quicker than the red target with similar distracter colors. By reviewing

the results from the experiments, which use the color scheme from the Phi Sigma Pi website, it is

obvious to tell that the blue target is easier find than the red target.

Two reasons may explain why blue was easier to find in the experiments. First, blue is the

default color for links, so users (especially experts, as used in our study) are tuned through


experience to find blue links. In addition, complementary colors, which are the opposite color on

the color wheel, provide a clear distinction between link color and distracter color. When the

distracter colors were white, dark yellow, dark red, and black the target was easier to identify

because blue is complementary to two of these distracter colors – yellow and red – both which

are opposite the color wheel to blue.

3.4. Navigation Navigation is a term used to describe the method of way finding on a website and the visual

elements and areas that direct users to content (Texas Tech). Ensuring easy-to-use navigation is

an important usability aspect of any website. If a website does not provide a navigational system

which allows users to find desired information; the users will be reluctant to utilize the website,

or they will not use it at all. To identify the relative strength of each site’s navigation, we

observed users navigating each site.

3.4.1. Observational Experiment Our group had users attempt to navigate to each website’s national history and contact executive

page. While navigating, users would verbally dictate everything they were thinking/performing

throughout their navigation. The purpose of this was to specifically identify strengths and

weaknesses of each site by means of observing real users.

3.4.1.1. Participants

The participants of the studies were two IST students (male user 1, female user 2) located in an

IST computer lab. The participants were experienced internet users and voluntarily performed

the required tasks.

3.4.1.2. Methodology

For this observational experiment, we asked User 1 to perform the task of finding the national

history of Theta Chi from the fraternity homepage. User 1 then performed the task to the best of

his/her ability, noting what they were trying to do and the problems they were running in to.

Next, we had User 1 navigate back to the home page and attempt to navigate to the contact


executive portion. User 1 then performed the task to the best of his/her ability, noting what they

were trying to do and the problems they were running in to. Following this we performed the

same experiment with User 1, except with the Phi Sigma Pi fraternity website. We repeated this

process for User 2.

3.4.1.3. Interview Results User 1 Find Theta Chi National History

Began by stating that he was searching for history in the left navigation menu. He found it within seconds and was finished with the task.

Find Theta Chi Contact Executive

Began by stating that he was searching for something like contact in the left navigation menu. He found it quickly and clicked it which brought up a contact form. He was not sure if what he opened was the contact executive page. We were required to verify to him that it was indeed the contact executive page.

Find Phi Sigma Pi National History

Began by stating that he was looking for a history link. He noted that he was having trouble reading the “block links” because the letters were not in line and were difficult to read. He found the link for history and clicked it. He thought he had found it, but we had to inform him that it was not the correct page. He visually searched the page while until he found the national history tab, which he noted did not pop out to him as a link should.

Find Phi Sigma Pi Contact Executive

Began by stating that he was looking for a contact link. He again noted that the blocks were hard to read, but quickly found the contact us link. This link took him to a form which he noted had marked that you were emailing specified executive members.

User 2 Find Theta Chi National History

Began by stating that she was searching for some sort of history link in the left navigation menu. She found it within seconds and was finished with the task.

Find Theta Chi Contact Executive

Began by stating that she was looking for a contact link. She found it quickly and clicked it which brought up a contact form. She was not sure if what he opened was the contact executive page. We were required to verify to her that it was indeed the contact executive page.

Find Phi Sigma Pi National History

Began by stating that she was looking for a national history link. She found the link for history and clicked it. She searched the page and quickly clicked the National History tab.

Find Phi Sigma Pi Contact Executive

Began by stating that she was looking for a contact link. She found the contact us link and clicked it. This link took her to a form and she asked which executive she should contact.


3.4.2. Results Analysis

The results of the experiment show that the navigation for Theta Chi is excellent for national

history. Users had no problem finding this information in a very short amount of time. For

contact executive, users could easily and quickly navigate to the contact page, but they were not

certain who they were contacting. When trying to locate Phi Sigma Pi’s national history, User 1

found the top navigational links hard to read as well as was could not locate the national history

tab quickly. Again, when attempting to contact the executives of Phi Sigma Pi, User 1 had a

difficult time reading the links.


4. Recommendations for Improving Usability We will be providing usability recommendations using the criteria of visual clarity, consistency,

informative feedback, explicitness, appropriate functionality, flexibility and control, error

prevention and control, and user guidance and support based on the knowledge we have gained

during our experiments.

4.1. Task Analysis By analyzing the tasks that users will frequently complete, we determined what site is more

usable and ways to improve tasks. In addition to qualitative analysis, by completing the

Keystroke Level Model, we were able to determine the time intervals to complete tasks so we are

able to provide a quantitative analysis as well.

Contacting the executive board is very straightforward as we saw in our results, so it fulfills the

explicitness criteria. Both the Phi Sigma Pi and Theta Chi websites, contacting the President is

easy because the visual clarity of the contact page ensures that users know where to type in their

name, email, and the body of their message. At Phi Sigma Pi, the user is given the opportunity

to contact other officers; this information is provided with each form, so consistency is

maintained. As was aforementioned, a task analysis provides a way to demonstrate consistency

within the site (Ritter & Churchill, 2007, p. 170).

Although we did not test the functionality of the contact form to ensure that messages were

delivered to the officers, a screen displays stating that the message was sent without a problem.

We are confident that the web script is working at Phi Sigma Pi because it provided some error

prevention features. If the email field was left blank, the message would not send, an error

message would display stating that a problem had occurred. However, at the Theta Chi site, only

the confirmation page is shown regardless of what fields were left blank.

Finding the national history at each of the chapters was straight forward as well and was

completed much quicker than emailing the president. By providing the history tab with the rest

of the navigation tab, the user was guided to the correct area and would intuitively know which

link to select.


We recommend that Phi Sigma Pi eliminate the “Type your text here” in the message text box.

This would decrease the amount of time to type the message by eliminating the need to clear the

box before typing. In addition, the contact us page could provide the name of the officers, so one

would know who to address the message too. For contacting the president of Theta Chi, the text

in the text boxes showed up as white on a white background. This made the words typed unable

to be seen unless you highlight the words. This is a very simple change that could be made to

greatly improve the efficiency of the emailing function of the website. By breaking down the

steps of each task we were able to see that moving a hand from the mouse to the computer and

vice versa took time. By making the buttons larger, this would reduce the time to complete these

tasks.

Upon entering the site, Theta Chi uses a splash screen which welcomes the user to the site along

with a few pictures. However, this screen takes time to navigate and serves no purpose, so the

organization could remove the splash screen.

4.2. Aesthetics When viewing the results of our aesthetics survey, it is evident that the Theta Chi fraternity has a

more visually pleasing website than Phi Sigma Pi. This is the case because Theta Chi’s website

looks more consistent to the users; the navigation style, organization, and color scheme has the

same feel as many other websites which the users are accustomed to working with. Users are

more familiar with Theta Chi’s side navigation that is high contrast plain text as opposed to Phi

Sigma Pi’s top navigation composed of hard-to-read, blocky pictures. Additionally, users prefer

the organizational style of Theta Chi; they dislike the vast amounts of white space present in Phi

Sigma Pi’s website which causes much more scrolling. Lastly, the users prefer the visual clarity

provided by the high-contrast black, red, and white color scheme of Theta Chi rather than the

orange, purple and white of Phi Sigma Pi.

We recommend that Theta Chi keep the same color scheme, organization, and navigation style

because it satisfies the usability criteria and the users prefer it. On the other hand, Phi Sigma Pi

can make many improvements in the aesthetics area. Our group recommends that Phi Sigma Pi

change their website’s color scheme to be more familiar and visually clear; the webmaster can

implement this with the help of the ColorJack Sphere utility (ColorJack). Next, we recommend


that Phi Sigma Pi close up the white space on their site. Lastly, we recommend that Phi Sigma

Pi adopt a side navigational system similar to that of Theta Chi to improve consistency for the

users.

4.3. Color Scheme The color scheme was maintained throughout the entire site, so consistency was maintained as

well. By providing these visual cues, visual clarity and explicitness was provided so the user

knew that he or she would travel to a new page if a link was clicked. All links worked as

expected, although a few links on Theta Chi’s website (Pictures, Alumni, and Theta Chi) led to

external websites that opened in a new browser, or tab. This activity was slightly unexpected.

Participants in the Perceptual Interaction and visual search experiment were able to identify blue

links easier than identifying red links, we recommend that both sites use blue links because it

enables users to more easily locate hyperlinks. In addition, more analysis may have to be done

to determine which color is the best for each site in allowing users to find the information he or

she is looking for. Also, links to external sites can be denoted so that the user is not surprised if

a new window pops up.

4.4. Navigation The results of the navigation experiments suggest that Theta Chi has an excellent navigational

system for locating the national history. Theta Chi also makes it very fast and easy to contact the

executive of the fraternity; however, they do not make it evident exactly who will be contacted

when you fill out the contact form. Phi Sigma Pi’s navigation was excellent for locating the

national history and contacting executives; however, the navigational links were hard to read,

and the national history tab did not stand out as much as it could.

Our group recommends to Theta Chi that they specify who will be contacted when the contact

form is filled out; this will improve the explicitness of the contact page. We recommend that Phi

Sigma Pi change their top navigation to side, plain text navigation in order to improve visual

clarity of the website.


5. Conclusion Both Theta Chi and Phi Sigma Pi fraternities have improvements they should implement

concerning the usability of their website. The improvements we have suggested have been

thoroughly researched and will greatly enhance each website’s usability when implemented. The

changes we have advised are the beginning of increased usability; however, there are many

aspects that have yet to be considered, such as increasing usability for the colorblind. The

webmaster of both sites should always keep the users in mind, by doing this one can always

ensure website usability.


6. References

Busey, Tom. “Visual Search: A Tutorial on the VisualSearch Experiment Tool.” Indiana

University – Bloomington.

http://cognitrn.psych.indiana.edu/CogsciSoftware/VisualSearch/images/VisualSearch.pdf

Card, Stuart et al. (1980). “The Keystroke-Level Model for User Performance Time with

Interactive Systems”. Communications of the ACM. Pg 396-410. Accessed Online:

http://reserve.libraries.psu.edu/ist/501/50132.pdf

ColorJack: Color Theory. Retrieved December 14, 2007 from http://www.colorjack.com/sphere/

Kieras, David. (2006). “A Guide to GOMS Model Usability Evaluation using GOMSL and

GLEAN4”. Accessed Online:

ftp://www.eecs.umich.edu/people/kieras/GOMS/GOMSL_Guide.pdf

Ritter, F.E., & Churchill, E. (2007). The ABCS of Humans: Guidance for Interactive System

Designers. MIT Press, 112-115.

Texas Tech. (2006). “Texas Tech Identity Guidelines”. University Office of Communications

and Marketing. Retrieved December 14, 2007 from

http://www.texastech.edu/identityguidelines/glossary.php

A Usability Analysis / Comparison of Fraternity...

Documents

Transcript of A Usability Analysis / Comparison of Fraternity...