SACHDEVA_SHALEEN_1437822_Evaluation Report on IRCTC.com_CI7700

Evaluation Report on - IRCTC.com

[email protected] 1

Evaluation ReportOn

IRCTC.com

Shaleen Sachdeva (K1437822)

Kingston University

CI7700 User Experience Design (System)


[email protected] 2

Consistency is one of the most powerful usability principles: when things always behave the same, users don’t have to worry about what will happen. Instead, they know what will happen based on

earlier experience.

— JAKOB NIELSEN


[email protected] 3

Project Brief & Project Aim

Introduction & Background

Analysis

Aim of Usability Test

Method

Results

Redesign Recommendations

Conclusion

Appendices

References

Content

4

5

15

17

18

26

38

40

41

49


[email protected] 4

Project Brief & Project Aim

Project Brief

The coursework is to analyse, usability test, redesign and prototype a web &

mobile based website and provide a multi channel user experience adopting a

user-centered approach.

Project Aim

The aim of the project is to evaluate the current IRCTC website (desktop &

mobile) and compare it with it’s competitor website ( Cleartrip ) and accordingly

improve its user experience and user interface.


[email protected] 5

Introduction& Background

About IRCTC

Indian Railway Catering & Tourism Corporation (IRCTC) - it is a collateral of

Indian Railways which handles tourism and online ticket booking for railways

and flights.

It offers easy rail and flight ticket booking.

It has schemes like Shubh Yatra where you can pay for the whole year and enjoy

the discounts the entire year and also offers adventure tourism packages.

IRCTC Official website: http://www.irctc.co.in


[email protected] 6

IRCTC Current Trends

The current trends carried out by alexa.com shows that IRCTC is ranked 48th

travel booking website in India and 517th on global chart.

The engagement rate of visitors visiting the site is 5.94 which has dropped by

3.57%. The average time user stay on the website is 10:26


[email protected] 7

Project Scope

The Scope of the project consist of analysing the IRCTC website, usability

testing and then redesign the website for multi-platform (desktop, tablet

and mobile).

The project will be complete by 19th of December 2014.

It will include a report, prototype of a website (low fidelity and hight fidelity)

and a multimedia presentation.

Target Users

Everyone who uses IRCTC to travel around India.

User Group

The users are divided into 3 groups

• Indian• NRI’s• Foreigners

Project Scope, Target Users

& User Group


[email protected] 8

The main focus of the usability test was to identify problems faced by the

users while checking the train availability or booking the tickets.

The areas of where participants can face problems are

• Journey arrival and destination column• Train Time-Table• Ticket Fare • Train Coaches• Seat Availability • Book now • Travel summary

Overall Problems


[email protected] 9

IRCTC - Current Website Interaction

The current IRCTC website is simple and can be navigated easily however there

are major issues with the layout, readability and eye movement which distracts

the user which results in delay in ticket booking process due to which user get

irritated and annoyed.

Login PageThe user is thrown by lots of information and adverts as soon as they land on the landing

page of the website.

The Login box has a Captcha code which irritates users every time they login which delays

the booking process. Also every time user need to login to check train availability or book

tickets which increase steps towards final output.


[email protected] 10

Plan My JourneyThe main attention of this page is taken by the adverts and secondary information because

of which it is difficult to locate the primary information - Plan My Journey.

Also the Text field and buttons used for Plan My Journey are very small which decreases the

readability.




Train Time-TableThe table used, to display train timings and other related information is small and cluttered

which leads to readability issue and confuses the users.





Train Details




Summary



Expected Usability IssueEven tough users may be able to book a train ticket but due to the poor

interface design, information architecture and inconsistent layout, a user may

get confused and frustrated and might just end up using any other website.

The website is aesthetically simple but because of bad user experience

design it may be difficult for users to memories or learn to use the website.



Hierarchical Task Analysis

Analysis

PersonasThe main use of persona is to create a realistic representation of you key users for your reference.These personas are base on the user group defined.



Karan is a Business Man living

in Ahmedabad. He travels to

New Delhi three to four time in

a month via train for his business

meetings. He uses IRCTC to book

his train tickets.

Anu is a Non Residential Indian.

She lives and studies in London

and visits India every 1 or 2 years.

She loves to travel around India

via trains and uses IRCTC to book

train tickets.

Anu Chitkara

Uses IRCTC Website: • To book train tickets

• Check train availability

Uses IRCTC Website: • To book train tickets


Age

Gender

Age

Gender

Train User

Train Usage

Internet Savvy

Train User

Train Usage

Internet Savvy

34

Male

22

Female

YesYes

Wants from IRCTC Website: • Easy to use

• Remember journey history

• Round trip option

• Quick links

Wants from IRCTC Website:• Easy to use

• Check train tickets without

signing in


Technology Skills: • Uses internet at work

• Owns a smart-phone

Technology Skills: • Uses internet for more than

20 hrs. /day

• Owns a smart-phone and an

iPad.

Dan is a traveller. He loves

travelling and exploring history

and culture of different places.

He wants to visit India and wants

to travel around India via trains.

He wants to use IRCTC for his

train tickets.

Uses IRCTC Website: • Never used IRCTC

• To book train tickets


Age

Gender

Train User

Train Usage

Internet Savvy

28

Male

No

Wants from IRCTC Website:• Easy to navigate and under-

stand

• Find train routes on map

• Find trains without signing in

Technology Skills: • Owns a smart-phone and uses

it to do his daily work

Dan BrownKaran Kapoor



Aim Of The Usability Test

The main purpose of the Usability Test is to understand the strength and

weakness of how easily and efficiently a user can book a train ticket on the

desktop and mobile website.

The main aim of the redesign was to provide user a flawless user experience

where they can book a train ticket in few steps without any difficulty.

Multiphase approach was used in the usability test.

Phase1-TestingcompetitorwebsiteA competitor website was selected for the Usability Test in this case Cleartrip.

The same tasks were conducted on the website to understand the users

reaction.

Phase2-Testingwithdifferentdevices.Desktop and mobile websites was tested to understand the usability, the

responsiveness and ease of use on both the devices.



Experimental Design

To achieve the goals outlined in the aims, the usability study was conducted

with 6 participants form each user groups. The participants were told to

complete 4 tasks for each website - 3 for desktop website and 1 for mobile

website.

A Common Industry Format (CIF) usability test was carried out on both IRCTC

and Cleartrip to evaluate and compare the two websites.

During the task the session was recorded to analyse and identify potential

area of improvement to the website like

• Navigation choice

• Task completion rate

• Errors

• Comments

• Overall satisfaction ratings

Method



Participants

In total 6 participants were choose from each user groups for the usability

test, the participants were contacted through verbal and email contact.

2 participants from each type of persona were selected for the usability test.

There were 2 females and 4 male participants and were aged between 18-60.

Participants-Table



Tasks /Scenarios

Following task were given to the participants for completion.

Task1-Find3Trainswhichtakeminimumtraveltime.

Imagine you are traveling from New Delhi (NDLS) to Ahmedabad (ADI).

Using IRCTC find 3 trains which takes the minimum time from New Delhi to

Ahmedabad.

* Participants will face problem while viewing the train time-table as the table is badly designed and have readability issues.



Task2-Checktheavailabilityoftheseats.Once you have figured out the least time taking train from New Delhi to

Ahmedabad find out the availability of seats on 2nd, 3rd and 4th January

2015 for AC 2 tire (2A) coach and AC first Class (1A) coach for that particular

train.

* Participants will be lost and confused while doing this task as the main information is not shown upfront.



Task3-Booktrainticketsfor3people.Imagine you are going back from Ahmedabad to New Delhi with 2 of your

friends. Using IRCTC website book train tickets for AC 2 tire(2A) coach for

your friends and yourself for 31st December 2014.

* Participants will be annoyed as the layout of passenger details form has a bad layout and is time taking.



Task2-Checktheavailabilityoftheseats.(MobileWebsite)Once you have figured out the least time taking train from New Delhi to

Ahmedabad find out the availability of seats on 2nd, 3rd and 4th January

2015 for AC 2 tire (2A) coach and AC first Class (1A) coach for that particular

train.

* Participants will be lost and confused while doing this task as the main information is not shown upfront.

* Also the mobile website is not responsive.



Usability Metrics

During the tasks the nominal data, ordinal data and ratio data was collected.

Nominal Data - for task success and error

Ordinal Data - CSUQ likert scale data, levels of task completion attempts.

Ratio data - Time-on-Task, mouse clicks, Assists.

Performance:Task Success: the success level was examined during the task execution.

Number of Errors (effectiveness): number of errors committed was recorded

when the participants were working through the task.

Task Time (efficiency): time for each task was monitored to see how much time

a participant take to complete a particular task.

Time to Proficiency (learn-ability): the experience of the participant was

recorded as they worked through the tasks.

Assists(Efficiency):An assist was given to the participants if they faced any problem while going

through the tasks or begin to stray off the tasks.

Satisfaction:A post-task questionnaire was prepared to measure participants satisfaction.

Computer System Usability Questionnaire (CUSQ) was used to measure the

response.

Post-test probing questions regarding the user interface and functionality of

the website was also asked.

Observation:An observation form was also prepared to collect participants behavior and

performance based on errors, assists and time per tasks.



Procedure

• A single moderator interacted with the participant and the tasks lasted

for approximate 45 minutes.

• Laptop and mobile device was set up for participants.

• Silverback and iGlasses was used for recording.

• Participants was talked through and asked to sign a consent form.

• www.irctc.co.in and www.cleartrip.com was opened on Safari.

• The participant then was provided with the tasks and was told to think

aloud during the task.

• During the tasks the moderator was tracking the time and recording the

task progress.

• After the tasks were completed post-questionnaire link was given to

participants.

In addition the moderator also asked the participants general questions

regarding the website.

Equipments Used & Location

EquipmentsUsed:• Apple Mac Book Pro - Retina 15inch

• Apple Iphone 4

• Silverback application

• IGlasses application

• Quick Time Player

• Safari

• Stopwatch on Moto X

Locations:Usability test took place at 2 locations -

• Kingston University, Surrey - KT1 2EE

• Seething Wells Halls of Residence, Surrey - KT6 5PJ



All the participants attempted the 4 tasks given to them. The tasks were

completed with 100% success rate but overall the participants found the

website to be frustrating and confusing.

On the other hand the cleartrip website had a better interface and user

experience. Based on the qualitative information, the website was simple,

easy to use and was consistent and responsive across multichannel.

ParticipantscommentswhenusingIRCTCwebsite.“The homepage is too cluttered.” - Participant 2

“Website is very confusing.” - Participant 4

“There is so much of information on one single page. It’s really confusing” -

Participant 3

“I don’t feel like using the site. It’s is not trustworthy at all” - Participant 1

Highlighted Issues“ ‘Plan my Journey’ area should be located in the center because that’s the

important thing” - Participant 1

“I never realised clicking on the class will show me the

train details - Participant 2

“ The train time table is too small and cluttered” - Participant 3

“I am having trouble finding the train availability” - Participant 5

“After using Cleartrip, I would choose Cleartrip over IRCTC.

It’s much easier to use” - Participant 6

Results



Comparison Between IRCTC &

Cleartrip

The result of tasks designed for IRCTC & Cleartrip were analyised and it

demonstrated that the Task on time (mean) for IRCTC website took more time

as compared to the Cleartrip website. Participant found the 2nd Task most

challenging. This shows that they were confused about the train detail page

and where to look for the train availability and thus got lost on the website.

According to the table below, the errors and the assists for IRCTC website

was also more compared to the Cleartrip website. This showed that because

of poor user experience and user interface of IRCTC the participants got

confused and frustrated when going through the website and required help.

On the other hand user experience and user interface of Cleartrip was better

which results in less errors and assists.

Comparison IRCTC & Cleartrip - Table



Finding per Task

Mesurable results were gathered through time per task and post-task

Questionnaire.

Time per Task

The time per task for all the participants was recorded and measured. The

table below shows the time taken (in seconds) by each participants for each

given tasks. The time on task show interesting results.

Task 1 - 3 out of 6 participants were able to complete the task within 2 minutes

the other 3 took a longer time as they were new to the website ans found the

train time table and website confusing.

Task 2 - Participants spend the maximum time on this task as it was the most

challenging task. The user was confused with the class link and the availability

table. Participant 6 was lost and almost gave up.

Time per Task - Table



Overall the participants were not happy and satisfied while doing the task.

They were mostly confused and faced lots of usability issues.

Task 3 - 3rd Task was sort of a continuation of Task 2 therefore it took less

time to complete than task 2. The maximum time was taken while entering

the passenger details and captcha code.

Task 4 - This task was replica of Task 2 but for mobile website. Due to the non

responsiveness of the website the task took a little more time than expected.



Mean & Median - Graph

Mean & Median - Table



Errors & Assists

While conducting each task the number of errors and assists were also

recorded. The result shows that the participant faced the maximum problem

when working on Task 2 as it was the challenging task. Total of 10 errors

were made and 8 assists were given

Task 4 was replica of Task 2 but for mobile website so the number of errors

and assists were comparatively less.

Task 1 was the simplest therefore the number of errors and assists were the

least.

Task 3 was almost similar to task 2 so there were less errors and assists.



Error&Assist-Graph

Error&Assist-Table



Overall Metrics

After the task completion the participants were provided with a post-task

questionnaire where they were told to rate the IRCTC website using ‘Likert

Scales’ ranging from ‘Completely Disagree’ to ‘Completely Agree’.

Post-TaskQuestionnaire-Table



Post-TaskQuestionnaire-Table

The questionnaire result shows that the participants

• believed that the home page was not at all attractive.

• the website was difficult to navigate.

• the user interface and the layout was not simple and consistent.

• was not satisfied with the way the train information was displayed.

• was not satisfied by the train summary information.

• would not recommend the website to anyone else

• would rather user other websites over IRCTC.



Overall Rating

On the basis of Usability, Efficiency, Attractiveness and Learnability the

participants rated the IRCTC website. The Table and Graph shows the overall

rating.

UsabilityThe participants were not satisfied with the usability of the website as all the

rating was below 4 out of 7.

AttractivenessThe rating for the attractiveness of the IRCTC website was below average.

The score was approximately 2 out of 7.

EfficiencyThe participants were not happy with the efficiency of the website they rated

the website 2 out of 7 which is again below average.

LearnabilityParticipants found the website easy to learn because of the similar tasks.



OverallRating-Table

OverallRating-Graph



Probed Question Response

The qualitative feedback from the Usability test was very useful as it showed

the proposed changes of IRCTC website by the participants. The participants

gave their opinion about the website while working on the tasks and after the

tasks.



The table below shows the recommended changes that have been determined

by participants views, behavior, success rate, errors and assists made during

the usability test. The recommendation table includes the ‘Usability Problem’,

‘Redesign Recommendation’ and ‘Severity’ level - High, Medium and Low.

It is believed that the following recommendation will improve the overall

experience and will address the problem faced by participants.

Redesign Recommendations



The Usability Test conducted for IRCTC website have underlined various usability issues which have interfered users in achieving information or delayed in completing tasks. The satisfactory rate of all the participants was average or below average and had a bad experience on the site.

Most of the participants found IRCTC website to be messy and cluttered. They found the website working but felt that it needs resonable improvement to provide a user friendly experience.

Even though participants were able to finish the tasks but they got confused and frustrated while doing the tasks or after completing the tasks.Also the mobile website was not at all responsive.

Thus, if the above design recommendation are carried out the user interface and user experience of the website will improve. The redesign will focus on improving the website‘s usability across multichannel and will focus on website’s main functions -• Plan my Journey• Train Time Table• Train Details• Summary

I believe by making the changes the user experience and user interface of the website would be improved thus making thw website easy to use.

Conclusion



Appendices



Tasks(IRCTC)



Competitive Intelligence (no date). irctc.co.in Site Overview. Available at: http://www.alexa.com/siteinfo/irctc.co.in (Accessed: 24 November 2014).

IRCTC site analysis (no date). IRCTC site analysis. Available at: https://www.scribd.com/doc/157869310/IRCTC-site-analysis (Accessed: 12 November 2014).

Krug, S. (2009) Rocket surgery made easy: the do-it-yourself guide to finding and fixing usability problems. United States: New Riders Publishing.

Nielsen Norman Group (no date). Nielsen Norman Group: UX Training, Consulting, & Research. Available at: http://www.nngroup.com (Accessed: 12 November 2014).

Quotes on Design (no date). Quotes on Design. Available at: http://quotesondesign.com (Accessed: 2 January 2015).

Search flights (no date). Cal. Available at: http://cleartrip.com (Accessed: 5 November 2014).

Tullis, T. and Albert, B. (2008) Measuring the user experience: collecting, analyzing, and presenting usability metrics. United States: Morgan Kaufmann Publishers In.

References



(no date a). : : Usability.org | Solving Problems That Matter | Contact Us @ +1.443.831.1351 or [email protected] : : Available at: http://usability.org (Accessed: 14 November 2014).

(no date b). IRCTC Next Generation eTicketing System. Available at: http://irctc.co.in (Accessed: 4 November 2014).

Citation

(Tullis and Albert, 2008, pp. 65 – 97)




(IRCTC site analysis, no date)

(Competitive Intelligence, no date)

(Search flights, no date)

(Krug, 2009)

(Nielsen Norman Group, no date)

(Quotes on Design, no date)

SACHDEVA_SHALEEN_1437822_Evaluation Report on IRCTC.com_CI7700

Documents

Transcript of SACHDEVA_SHALEEN_1437822_Evaluation Report on IRCTC.com_CI7700