FULL STUDY - JULY 2016€¦ · Now, we are proud Belgians, so we couldn’t leave Brussels Airlines...

23
Low-cost airlines: 3 websites tested FULL STUDY - JULY 2016

Transcript of FULL STUDY - JULY 2016€¦ · Now, we are proud Belgians, so we couldn’t leave Brussels Airlines...

Page 1: FULL STUDY - JULY 2016€¦ · Now, we are proud Belgians, so we couldn’t leave Brussels Airlines out of the picture . This was also an interesting opportunity to test if ordering

Low-cost airlines: 3 websites testedFULL STUDY - JULY 2016

Page 2: FULL STUDY - JULY 2016€¦ · Now, we are proud Belgians, so we couldn’t leave Brussels Airlines out of the picture . This was also an interesting opportunity to test if ordering

LOW-COST AIRLINES: 3 WEBSITES TESTED - FULL STUDY 2

Table of content

Up, up and away! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Usability issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

So which airline offers the best prices? . . . . . . . . . . . . . . . . . . . . . . . . 12

Users’ comments on the low-cost airlines . . . . . . . . . . . . . . . . . . . . . . 13

Airlines can always do better! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Appendices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Contact info . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Tests were conducted between 22nd of June & 1st of July 2016 on 9 participants . Participants were chosen randomly . UXprobe is an independent company and did not perform these tests under the commission of Brussels Airlines, Ryanair or Easyjet .

Acknowledgement for iconsIcon made by Freepik from www .flaticon .com

Page 3: FULL STUDY - JULY 2016€¦ · Now, we are proud Belgians, so we couldn’t leave Brussels Airlines out of the picture . This was also an interesting opportunity to test if ordering

LOW-COST AIRLINES: 3 WEBSITES TESTED - FULL STUDY 3

With the start of the summer, we are getting into a vacation mood at UXprobe . Low-cost airlines are all the rage right now, so we are set on flying with them to our next tropical destination . But which one should we choose?

To help us find an answer to that question, we performed some remote UX tests (or call it web testing) with our new tool to see how people would order flight tickets online . And we have some interesting results to share .

Through our testing, it became clear to us why Ryanair is such a strong performer, while also having recommendations to improve website usability for all three tested airlines (Ryanair, Brussels Airlines & Easyjet) .

Remote UX testing – video recordings of people visiting your website and talking about their experience – cannot only help finding out which low-cost airline is the most user friendly . It also provides companies with a powerful tool that they can use to gather insights, find usability issues, easily compare their websites against the competition and help them get greater online revenue.

Let’s see how airline website can reach new heights (pun intended) in usability!

Page 4: FULL STUDY - JULY 2016€¦ · Now, we are proud Belgians, so we couldn’t leave Brussels Airlines out of the picture . This was also an interesting opportunity to test if ordering

LOW-COST AIRLINES: 3 WEBSITES TESTED - FULL STUDY 4

Up, up and away!The popularity of low-cost airlines is rising quickly . Right now, already 35 % of all passengers in European airflight are flying with low-cost .

And from that 35 %, there are 2 clear leaders: Ryanair & Easyjet.

Now, we are proud Belgians, so we couldn’t leave Brussels Airlines out of the picture . This was also an interesting opportunity to test if ordering a ticket from an airline’s headquarter (Brussels Airport) would be easier .

Ryanair is the biggest in terms of marketshare (11%) . But it’s also big in other categories: 200 destinations in 33 countries, more than 110 million passengers each year & a fleet of over 340 Boeing 737 aircraft . Founded in 1985, Ryanair has become a big player in European airflight .

Easyjet comes in as a close second (7% overall marketshare), but are growing bigger each year . Right now, they are active in more than 30 countries, with more than 70 million passengers each year and a fleet of over 20 Airbus Aircraft . Founded in 1995 with the specific goal of offering low-fares flight in Europe, Easyjet has now also expanded their offerings to business flights .

Brussels Airlines is a distant third in this comparison (1% overall marketshare), but does hold the title of the largest airline of Belgium . It operated to over 90 destinations (some of them also outside of Europe) and following a recent expansion, Brussels Airlines now has a fleet size of 48 (different types of aircraft) . Founded in 2006, the company was created from a merge between Virgin Express & SN Brussels Airlines .

Page 5: FULL STUDY - JULY 2016€¦ · Now, we are proud Belgians, so we couldn’t leave Brussels Airlines out of the picture . This was also an interesting opportunity to test if ordering

LOW-COST AIRLINES: 3 WEBSITES TESTED - FULL STUDY 5

How were the tests performed?Tests were conducted from June 22nd through 1st of July, 2016 . We used both the Dutch and English versions of the website, based on each tester’s personal preference .

Testers were asked to order a return trip plane ticket for 1 person from Brussels Airport (Zaventem) to either Rome (Ryanair, Brussels Airlines) or Napels (Easyjet) . On top of the standard hand luggage, we also asked them to order 1 extra piece of luggage (lowest weight possible) .

After each task, testers were asked to rate their experience on a 5 point scale, ranging from “extremely easy” to “extremely difficult” . We also asked some additional questions based on their reactions during the test .

Usability testing done easily with UXprobeThese tests were all performed using our new usability test tool that allows us to quickly set up a remote UX test . A link is created for each test, which can be shared through social media or email .

That way, we could choose for ourselves if we wanted to target specific users, limit our user base, or go for as many testers as possible . It also allows us to do tests without a moderator .

Nevertheless, for this particular test, we chose to have a moderator accompany our testers each time . Each test lasted only about half of an hour, which meant that we could easily test multiple people in one day .

Both the screen as the webcam and microphone of each tester was recorded . Results, such as the time needed to perform each task, were stored immediately together with the video and audio .

Page 6: FULL STUDY - JULY 2016€¦ · Now, we are proud Belgians, so we couldn’t leave Brussels Airlines out of the picture . This was also an interesting opportunity to test if ordering

LOW-COST AIRLINES: 3 WEBSITES TESTED - FULL STUDY 6

Afterwards, by writing notes next to the video of a tester, we got statistics on what kind of feedback our testers gave us, which tests failed, how long each test took and what kind of errors our testers encountered . From there, we can start sharing these insights with you!

ResultsOnce again, before we start going into the specific issues, let’s first go over some general findings from these tests:• Fastest website overall: Ryanair

› Average time to order a flight ticket on Ryanair was around 7 minutes .

• In user satisfaction, Ryanair is also number one in our tests › Ryanair was the only company that got evaluated

“extremely easy” after a test . Just as with our

previous tests with banks, speed correlates nicely with user satisfaction .

› It’s important to mention that only one task was used to determine these two specific ranks of accomplishment . There were no other measures for additional features of the website or the business .

• Most users were able to complete the task successfully › There were no website breaking bugs to be

spotted this time, good job! › For most users, the website was straightforward

enough to guide them through the steps laid out by the airline .

› But what caused the failures at Brussels Airlines? You can find the answer at “Loyalty Programs” (next page) .

Page 7: FULL STUDY - JULY 2016€¦ · Now, we are proud Belgians, so we couldn’t leave Brussels Airlines out of the picture . This was also an interesting opportunity to test if ordering

LOW-COST AIRLINES: 3 WEBSITES TESTED - FULL STUDY 7

Usability issuesThrough our UX remote testing, we found usability issues in the following areas:

Loyalty programsDon’t get us wrong, loyalty programs are a good thing! They help companies increase sales and can provide more value for an existing customer . But for new customers, if you don’t provide adequate information or don’t use terms that are straightforward, a loyalty program can kill a sale.

Brussels Airlines is proud of its’ loyalty program called Loops and therefore (rightfully) puts it as one of the options on the front page, after choosing a departure/arrival date . But most of our testers became confused by the option and decided to leave it unchecked when ordering a ticket (instead of, for example, looking for more information about it) .

If you doubt that option would ever lead to a lost sale, then look at the reaction of someone who becomes frustrated after checking the loyalty program option because he can’t find a good way to revert back to Euro . He eventually quits the task .

You need to consider every option when testing your website and how that could affect your customer, both new and old .

Language useAs we already mentioned in our previous blog post (“Which Banking website is the most user-friendly?”), your website’s language use can either cause joy or confusion with customers .

Ryanair is a bit more playful than others, showing a “Daar gaan we!” (Here we go!) button to start the ordering process . It might seem trivial, but our testers noticed, and liked it.

Page 8: FULL STUDY - JULY 2016€¦ · Now, we are proud Belgians, so we couldn’t leave Brussels Airlines out of the picture . This was also an interesting opportunity to test if ordering

LOW-COST AIRLINES: 3 WEBSITES TESTED - FULL STUDY 8

Using such terms can have a positive effect on your website’s satisfaction rate . It is this kind of language, combined with other factors, that probably put Ryanair on top in our satisfaction survey, performed after each test .

On the other hand, confusing terms prevents users from achieving their goals, increasing dissatisfaction . It can even lead a company to lose extra revenue.

Easyjet uses “Ruimbagage” (“Hold luggage”) for the extra luggage you can order (that isn’t hand luggage) . But for our testers, that term was confusing .

Watch as our Dutch tester gets confused when first reading the term, then selects “ruimbagage” hesitantly .

For our English tester, she thought she needed “Cabin Baggage” (the term being used for hand luggage) and eventually did not select anything .

Page 9: FULL STUDY - JULY 2016€¦ · Now, we are proud Belgians, so we couldn’t leave Brussels Airlines out of the picture . This was also an interesting opportunity to test if ordering

LOW-COST AIRLINES: 3 WEBSITES TESTED - FULL STUDY 9

Loading timesOrdering a flight ticket might seem like an easy task, but there is a lot of work being done in the background: checking seat & flight availability, creating new accounts, order processing, etc .

Speed is key if you want to increase your customer satisfaction . If you have a slow website, chances are that your customers won’t come back .To avoid confusion, you can make it clear to your customers that they need to wait while a request is being handled, create some kind of distraction . But it won’t magically solve your slow website .

Brussels Airlines uses a splash screen, but by showing it at almost every step, our testers started to notice,

defeating the purpose of creating a distraction during loading times .

In the questions presented afterwards by our tool, where they could fill out both positive & negative points about the website, over half of our testers still found the website to be slow and the splash screen distracting.

Product optionsYou might have experienced it before: you order a return trip flight, select your seat, but the site suddenly says you selected two! Ah yes of course, 1 seat for the flight there and another one to come back .Airlines have struggled with wording this correctly, sometimes you pay extra for your seats in low-cost, so

Page 10: FULL STUDY - JULY 2016€¦ · Now, we are proud Belgians, so we couldn’t leave Brussels Airlines out of the picture . This was also an interesting opportunity to test if ordering

LOW-COST AIRLINES: 3 WEBSITES TESTED - FULL STUDY 10

the customer should be notified about it . But sometimes, airline companies can’t get the message through .

In the video below, you can see a tester struggling to find out why she ordered two seats, even suggesting starting the task all over again .

If she would indeed have restarted the task, you would have noticed it in your analytics tools’ task flow (see Appendix E for an example of task flow graph) . But only with usability testing you would have found out exactly why she restarted .

PricingLow-cost airlines provide low fares . For everything else, you pay an extra fee . That’s not a bad thing, it has become a great business model for some companies .However, great power comes with great responsibility: if you must charge an extra fee, you should be transparent about it . But what exactly does that mean? When do you show that you are charging an extra fee?

For some of our testers (you can watch the video for one of them below), the notification of a surcharge (8 €) for paying with credit card came way too late.

Page 11: FULL STUDY - JULY 2016€¦ · Now, we are proud Belgians, so we couldn’t leave Brussels Airlines out of the picture . This was also an interesting opportunity to test if ordering

LOW-COST AIRLINES: 3 WEBSITES TESTED - FULL STUDY 11

In her answer to the questions presented after the test, she wrote this:

“The 8 € processing fee at the end is super annoying. I hate feeling like companies, especially airlines, are adding on tons of little extra charges. I already expect more to come when I arrive for my flight, which is going to be annoying... “

If she would have switched payment options you would have noticed it in your analytics . But was it because she pressed the wrong button? Or because your website did not have the right credit card options?

With usability testing, you would know the exact reason and make meaningful changes because of it .

Interface & Pop-UpsWe all know it: pop-ups are annoying, but sometimes they are used to show useful information .

In the case of airlines, popups were used to explain the different (luggage/pricing) options . But a bad implementation could cause confusion or even give the impression to your customers that your site is not working correctly.

The tester below (and most others) becomes increasingly annoyed about a pop-up that gets activated automatically when hovering over a luggage option, while he actually just wants to scroll downward on the page .

Page 12: FULL STUDY - JULY 2016€¦ · Now, we are proud Belgians, so we couldn’t leave Brussels Airlines out of the picture . This was also an interesting opportunity to test if ordering

LOW-COST AIRLINES: 3 WEBSITES TESTED - FULL STUDY 12

So which airline offers the best prices?We graphed the airline ticket prices against the dates in which the tests were conducted .Prices remained relatively constant for Brussels Airlines

Easyjet prices peaked during the middle of the week and are the lowest on Fridays .

For Ryanair, the trends are slightly different, with a steady rise in prices towards the weekend and a drop during the midweek .

Putting them all in one chart and bearing in mind the different dates and destinations, we observe something interesting about the overall price trends . Brussels Airlines prices remain relatively constant while the prices of Ryanair and Easyjet move in opposite directions . That is to say when Ryanair prices fall, Easyjet prices climbs and vice versa .

Page 13: FULL STUDY - JULY 2016€¦ · Now, we are proud Belgians, so we couldn’t leave Brussels Airlines out of the picture . This was also an interesting opportunity to test if ordering

LOW-COST AIRLINES: 3 WEBSITES TESTED - FULL STUDY 13

Users comments on the low-cost airlinesAfter looking at the results and usability issues, we asked our users for a few positive, negative and general comments about the website and recorded them . We noted that there are usually more areas of improvement given rather than compliments about the website . This means that our users already have a certain expectation when navigating the website, and it is a big mistake to underdeliver on the airline experience with poor usability .

Brussels AirlinesWhile navigating the website, we realised that each user had identified a different issue that if addressed, would have helped in their experience of purchasing their flight tickets .

Extra charges, extra steps and errors are areas that will really frustrate the users . Though we understand that ordering tickets online happens in real time, it might help if the availability was presented first before the price (or presented together) . The airline had revealed the prices first, then the user realised afterwards that the seat wasn’t available . Also, when the user switched to a different payment method (credit card), they incurred

extra charges. There was also an 8 € hidden transaction fee implemented only at the end of the process . The users remarked that they had to fill in their details twice during the process, and that one user was upset that the address was asked for at the end of the process .

The structure of the matrix was confusing for the users, and it has presented them with too many options. The Loops loyalty system was also confusing and the users have difficulty navigating some links and buttons because it did not feel very intuitive to know where to click next . The seat selection comes late in the process, as some users wanted to choose their seats earlier . Putting the advertising above the search bar also made it more difficult to locate it .

While ordering the baggage options and flight times together is intended to be a seamless experience, it became confusing for the users, as one of them was not sure at the end of the transaction the amount of extra luggage allowance given . Also, the wait screen with the flight attendants and the rollover popups have been described as “annoying” by some users .

With this feedback, Brussels Airlines could learn and analyse where the customer clicked, why they think it should be done that way, and better organize the

Page 14: FULL STUDY - JULY 2016€¦ · Now, we are proud Belgians, so we couldn’t leave Brussels Airlines out of the picture . This was also an interesting opportunity to test if ordering

LOW-COST AIRLINES: 3 WEBSITES TESTED - FULL STUDY 14

information and website layout to make the experience more pleasant for the user . However, the airline did not deviate from the primary function of their website, as one user also remarked that the steps were logical and familiar as a whole process . Hopefully this would make Brussels Airlines’ website easier to navigate for future users .

EasyjetFor Easyjet, the biggest issues were that of the lack of customisation options. The users were not free to pick the dates that they wanted because of the automatic date picker . For the tariff options, it was not easy to switch among the two tariffs because they had to fill in all their details again, making it difficult to compare among the choices. One user was very upset at the request from Easyjet to have to open a new account with them .

Easyjet had a few areas of their website where too much information and choices were presented. Some users did not appreciate the accommodations and car rental options, though it might be helpful to another group of users . Even when choosing their age category, it had too many options . Some of them also felt that the user assistance and pop ups were so frequent that it became unnecessary and a source of distraction instead . These users desire that the information such as the hold

luggage options for the flexi option is clearer and easier to understand .

For improvement, Easyjet can consider an infographic to compare all the different baggage options . Users feel that they had to read a lot of text in order to fully understand the different options available, their prices, and if there are any hidden costs to look out for . An infographic would make the experience and journey easier for these users.

There was also feedback given to suggest that when filling out their details, the country and phone number should match automatically. The place, street and postal code could be given the autofill option as well, as the user had expressed the convenience of having that addition .

On overall, the users appreciated the seat placement and seat selection processes as well as the straightforward navigation of the website. The division in the website where the total amount due was always clearly presented on the right side was beneficial for the users as well .

Page 15: FULL STUDY - JULY 2016€¦ · Now, we are proud Belgians, so we couldn’t leave Brussels Airlines out of the picture . This was also an interesting opportunity to test if ordering

LOW-COST AIRLINES: 3 WEBSITES TESTED - FULL STUDY 15

RyanairGenerally the airline which the users had the most positive experiences in . Users reflected that the process went pretty fast, and that as far as prices were concerned, they were always clearly displayed . Clear overview and easily scannable with minimal amount of text were some of the good points given . The users appreciated the layout, and they reported that they clicked the fewest number of times to reach their goal, and encountered the fewest loading screens . The extra charges were clear when making selections . The users loved the little animations, feeling that it served as a guide instead of a distraction . The date picking function is easy to use and the users liked the overview on top without having to go through all the extra options . Having Belgium on the top of the country list was a plus for some users .

The users reported a few usability issues for Ryanair . The main source of frustration was confusion caused by having too many options and too much information. The seat selection process was confusing to many of the users, with mistakes made such as closing the seat selection button after selecting the seat thinking it would save the seat choice . Seat button card was a confusing term, and the users were not satisfied with too many

extra offers . The users also encountered advertising after upgrading their selected seats .

Ryanair uses airport names to identify destinations . This has confused at least one of our users . There was difficulty in finding information pertaining to hand luggage, and the user reported that maybe the information was indeed there but they didn’t care so much to go actively looking for it. The airline can consider to organize that information in a clearer manner so that the process can become more passive when looking up information .

During the flight selection screen, there was a user who suggested to leave out the price on top, beneath the date . Also, a suggestion was to place the cheapest option as the default in the flight selection screen . Ryanair could look into some of these suggestions to iterate on their website to further delight their customers .

Page 16: FULL STUDY - JULY 2016€¦ · Now, we are proud Belgians, so we couldn’t leave Brussels Airlines out of the picture . This was also an interesting opportunity to test if ordering

LOW-COST AIRLINES: 3 WEBSITES TESTED - FULL STUDY 16

Airlines can always do better!The users had a generally positive experience in ordering their plane tickets in light of the coming vacation time . However, from our usability test and interviews, we realise that these airlines do have room for improvement in automating their services to serve their customers better . Using usability testing, we found out the areas to consider for improving their websites, and it is indeed a good start to better treat and retain their valued customers .

Convinced you need to test your website as well?Website testing on different screens, different OS’es & different browsers is a must for good UX design . Our tool shows all of that information for every test . That way, a company could quickly check if this problem is platform/browser specific and fix it immediately .

UXprobe video review (see screenshot 1) shows you the test user’s video and you are able to write feedback on the right side .UXprobe Tasks (see screenshot 2) allows you to see the analytics behind the tests .UXprobe session overview (see screenshot 3) compiles all the test users data in one list .

After reading these findings, do you want to know if your website can be improved as well? Are you curious to know if we have a free trial? Do you want to experience a test for yourself? Or do you just want to jump right in and take a look at our pricing?

Well, all of that is possible at UXprobe! You can get in touch with us and we’ll be happy to answer any question you might have .

Screenshot 1: UXprobe Video Review panel Screenshot 3: UXprobe Task Metrics panelScreenshot 2: UXprobe Sessions panel

Page 17: FULL STUDY - JULY 2016€¦ · Now, we are proud Belgians, so we couldn’t leave Brussels Airlines out of the picture . This was also an interesting opportunity to test if ordering

LOW-COST AIRLINES: 3 WEBSITES TESTED - FULL STUDY 17

Appendices

Appendix AFeedback for Brussels Airlines(part 1/2)

8 back to index

Brussels Airlines

Negative • Splash Screen, Flow• Slow, while the info was clear I did not get a result for my request• More euros, less loops• I was not sure how much extra luggage I got . The tariff system was not easy to grasp . Why should I type

name and details more than once?• Clarity in tarification, too much information at once (needs to be better divided), certain steps have too

many choices at once (for example with insurance), forward flight has a specific tariff that isn’t prepositioned at the return flight, advantages of air miles should be separate

• The “Loading, please wait” screen with 2 stewardesses flashed between all screens, that’s distracting . Too small text

• Rollover pop-ups when selecting prices just come out of nothing + the waiting screen isn’t professional . The first time, you barely see what the image was

• I needed to re-enter the address at the end of the process, while I already had done that in a previous step• Choosing baggage options and flight times at the same time felt confusing . The information about allowed

baggage was pretty clear, but seeing different prices below them threw me off a bit, until I realised they corresponded to the flight time . However, since I started with looking at baggage info, that stuck in my head, and I couldn’t shake the impression that I was being charged different prices for bringing the same baggage . . .

• Putting advertising above the search bar makes it more difficult to find it .• On the first page, you present a graph with the lowest retour prices for each week, but it’s unclear if those

are already prices for my specific flight + graphs aren’t clickable• The difference between several flight options/details aren’t clear• The matrix view of the lowest prices is unclear• The choice of selecting your seat while you are already in the payment flow is a little odd• In one specific case, Brussels Airlines showed certain prices, but when I selected them, the website told me

those prices weren’t available anymore . That’s why I decided to stop my task• Lowest price selection matrix is only without a suitcase option, which could be confusing• No automatic fill-in of country of residence based on where you are departing

Positive • I like having different screens for each step (better than other website where everything is on one page)• Actually no specific feature . It looks nice but is not as easy to navigate as it should be• Quick overview of available tariffs, you can book directly with seat reservation

Page 18: FULL STUDY - JULY 2016€¦ · Now, we are proud Belgians, so we couldn’t leave Brussels Airlines out of the picture . This was also an interesting opportunity to test if ordering

LOW-COST AIRLINES: 3 WEBSITES TESTED - FULL STUDY 18

Appendices

Appendix AFeedback for Brussels Airlines(part 2/2)

8 back to index

Brussels Airlines

Positive • The steps seemed logical and mostly familiar . There were some links and buttons that didn’t really look like links or buttons . The €8 processing fee at the end is super annoying . I hate feeling like companies, especially airlines, are adding on tons of little extra charges . I already expect more to come when I arrive for my flight, which is going to be annoying . . .

Easyjet

Negative • Too much promotion, complex information about fares, why do I need to open an account?• Too much extra options (accomodation/car rental/etc…) during process• If you change tariff/options, you need to reinsert everything again, flexi tariff is better for luggage, but I

can’t see both tariffs in one view .• Too small text and too much of it . You need to read everything before you’re sure that there won’t be any

extra costs . The accomodation/transport options are distracting .• The continue button is on a weird spot (in the right sidebar) . I would also expect it after the big block of

text .• I had to read and compare a lot of text to figure out the difference in baggage types . I didn’t realize that I

had to specifically request “hold” baggage, and then I didn’t notice there was a charge for it until the end . I was also unfamiliar with the term “hold” baggage, but that could have been a UK vs US English issue .

• Too much user assistance and pop-ups that aren’t necessary .• Icon for travel insurance is a suitcase, confusing for the function of ordering luggage .• Dates aren’t easy to change because of automatic date picker• The choice of country and your phone number area code does not automatically match .• Too much options for age on the dropdown list• The hold luggage options for the flexi option could have been more clear (the fact that you already get 1

extra suitcase with the flexi option)• Most of the popups that clarified certain options only came up after I had already filled in the information .

Appendix BFeedback for Easyjet(part 1/2)

Page 19: FULL STUDY - JULY 2016€¦ · Now, we are proud Belgians, so we couldn’t leave Brussels Airlines out of the picture . This was also an interesting opportunity to test if ordering

LOW-COST AIRLINES: 3 WEBSITES TESTED - FULL STUDY 19

Appendices

Appendix BFeedback for Easyjet(part 2/2)

8 back to index

Appendix CFeedback for Ryanair(part 1/2)

Easyjet

Positive • Good division in website, total amount due was always on the right side • A lot of information, but it was clear for me .• List of luggage options was short, without any extra icons or distractions & I was able to choose it fast .• I was familiar with the patterns they used, and everything felt pretty straight-forward . It’s not an “enjoyable”

site, and it felt a little cheap, but I also kind of expect that from a cheap airline . Especially after the baggage fee, I expect the experience of flying with EasyJet will be pretty cheap-feeling too .

• Selecting seat place is more logical than with the other airline websites

General • Autofill on place/street/postal code would be a nice addition• Showing your seat place again in the payment flow is nice, but unnecessary

Ryanair

Negative • Information about seat selection & bagage should have been more clear• Many extra offers (which I do not want) e .g . seats The button card on the payment site was not so clear,

what does card mean• You first get a date and a price and only afterwards a specific option -> confusing . I would leave out the

price on top (beneath the date) during flight selection screen • I couldn’t find any info about hand luggage, so I had to add checked baggage . I’m not sure if I can bring

anything on the plane at all, since it wasn’t listed . The upgrades from “Regular” service were not clear at all, I couldn’t read the icons and didn’t see any way to get more info . Maybe it was there, but I didn’t care so much to go hunting for it .

• You can’t scroll through calendar• Confused with seat picking -> too much options• Ryanair uses the names of the airport, but that’s confusing• I though the close button on the seat selection screen also confirmed my action of selecting that seat, but it

didn’t -> was confusing• Annoying to have advertising after you selected your seats to upgrade

Page 20: FULL STUDY - JULY 2016€¦ · Now, we are proud Belgians, so we couldn’t leave Brussels Airlines out of the picture . This was also an interesting opportunity to test if ordering

LOW-COST AIRLINES: 3 WEBSITES TESTED - FULL STUDY 20

Appendices

Appendix CFeedback for Ryanair(part 2/2)

8 back to index

Ryanair

Positive • It actually went pretty fast, the fact that we could select our own seat was positive, but not very clear• Quite fast, clear price infornation as far as the price was concerned• Clear overview, easily scannable with minimal amount of text, steps are as expected• Low amounts of clicks to reach my goal! Low amount of loading screens & there is no push to sell me

something . All prices are clear & nice lay-out .• I already filled in a Belgian phone number, so I didn’t need to fill out a specific country code• The extra charges were clearer when I was making selections . Overall, the site felt pleasant to use and there

were helpful little animations to keep things going . I felt a bit more like I was being guided, compared to the other sites .

• You never have to say no to something (except if you didn’t pick a seat)• Date picker is more easy to use than other websites, symbols & interface is clear .• Possibility to chat with someone is nice• Belgium on top of the country list • The fact that you get an overview on top and don’t need to go through the extra options is very good!• Automatic correction of spaces in credit card number is nice

General • Confirmation button was on top, but I expected it on the button, maybe a more standout color?• Would do a format check on the credit card number• Default to the cheapest option when on flight selection screen?

Brussels Airlines Easyjet Ryanair

User 1 11:35 10:43 10:16

User 2 13:38 15:34 08:04

User 3 15:08 16:00 15:32

User 4 08:11 11:49 10:04

User 5 11:52 10:25 08:39

User 6 07:01 09:05 09:59

User 7 13:43 10:25 13:46

User 8 06:06 12:01 06:30

User 9 16:25 18:42 14:25

Time taken overall

Brussels Airlines Easyjet Ryanair

User 1 Difficult nor easy Easy Easy

User 2 Difficult nor easy Easy Extremely easy

User 3 Extremely difficult Extremely difficult Easy

User 4 Difficult Difficult Difficult

User 5 Difficult Difficult Easy

User 6 Easy Difficult nor easy Easy

User 7 Difficult Difficult Easy

User 8 Easy Difficult nor easy Extremely easy

User 9 Difficult nor easy Difficult nor easy Easy

Task feedback “Order ticket online” Appendix DTables of raw data(part 1/2)

Numbers in red are durations for failed tasks

Page 21: FULL STUDY - JULY 2016€¦ · Now, we are proud Belgians, so we couldn’t leave Brussels Airlines out of the picture . This was also an interesting opportunity to test if ordering

LOW-COST AIRLINES: 3 WEBSITES TESTED - FULL STUDY 21

Appendices

Appendix DTables of raw data(part 2/2)

Numbers in red are durations for failed tasks

8 back to index

Brussels Airlines Easyjet Ryanair

User 1 07:29 06:53 05:56

User 2 09:36 12:35 05:45

User 3 10:33 11:09 09:51

User 4 06:33 09:02 07:56

User 5 10:59 09:39 06:35

User 6 05:17 06:18 07:01

User 7 10:27 09:34 06:35

User 8 04:45 08:10 04:31

User 9 14:49 17:37 08:25

MAX 14:49 17:37 09:51

AVG 08:26 10:06 06:57

MIN 04:45 06:18 04:31

Task duration

Brussels Airlines Easyjet Ryanair

User 1 Difficult nor easy Easy Easy

User 2 Difficult nor easy Easy Extremely easy

User 3 Difficult Extremely difficult Easy

User 4 Difficult Difficult Difficult

User 5 Difficult Difficult Easy

User 6 Easy Difficult nor easy Easy

User 7 Difficult Difficult Easy

User 8 Easy Difficult nor easy Extremely easy

User 9 Difficult nor easy Difficult nor easy Easy

Website feedback

Brussels Airlines Easyjet Ryanair

User 1 € 232 € 305 € 153

User 2 € 230 € 275 € 196

User 3 € 282 N/A € 213

User 4 € 242 € 318 € 162

User 5 N/A € 353 € 172

User 6 € 232 € 329 € 143

User 7 N/A € 300 € 185

User 8 € 232 € 303 € 163

User 9 € 244 € 253 € 183

Ticket price

Page 22: FULL STUDY - JULY 2016€¦ · Now, we are proud Belgians, so we couldn’t leave Brussels Airlines out of the picture . This was also an interesting opportunity to test if ordering

LOW-COST AIRLINES: 3 WEBSITES TESTED - FULL STUDY 22

Appendices

Appendix EExample of task flow graph

8 back to index

Task flow graph showing the path that users go through while performing a task

Page 23: FULL STUDY - JULY 2016€¦ · Now, we are proud Belgians, so we couldn’t leave Brussels Airlines out of the picture . This was also an interesting opportunity to test if ordering

LOW-COST AIRLINES: 3 WEBSITES TESTED - FULL STUDY 23

Super-powered usability tests

Like and follow us on

6 Facebook

6 Twitter

6 LinkedIn

6 YouTube

www.uxpro.be

UXprobe is providing an online tool to create and perform usability tests.

How much does it cost?Have a look at our Pricing Page

Can I get a live demo?We will demo for you with great pleasure . A short demo by Skype takes 15-20 minutes: let’s schedule one!

Is there anyone I can talk or write to?Sure, we are at your disposal for any request and to provide further information . Feel free to contact us:

Jan Moons, our co-founder, UX expert

6 jan@uxpro .be / +32 485 69 78 35

Isabelle Dro, our communications manager

6 isabelle@uxpro .be / +32 485 71 48 36

Paul Davies, our co-founder, engineer

6 paul@uxpro .be / +32 478 96 39 17

We are based in Antwerp (Belgium); you are welcome to visit us!