1

Information Technology Service Quality – How does yours measure up?Jane McGuireStrategic PlannerOffice of the CIOUNM

2

Today’s Topics

Quality Service Quality Measurement ServQual TechQual+

3

What is Quality?

ISO Quality Control / Quality Assurance Six Sigma Lean Manufacturing, JIT production Balanced Scorecard Malcolm Baldrige Quality Award

Process

4

Malcolm Baldrige Quality Award Leadership Strategic Planning Customer and Market Knowledge Measurement Workforce Process Management Results / Outcomes

5

Quality criteria

“The only criteria that count in the evaluation of service quality are defined by the customers.” Valarie Zeithaml, Ph.D., Delivering Quality Service

6

What the customer wantsper Horst Schultze, CEO, Ritz Carlton, 2006

Defect-free serviceCorrect, consistent, reliable

Timely serviceNow, according to created expectation

To feel goodIndividual treatment, knowledge

Result = LoyaltyResult = Loyalty

7

How do we measure service quality?

Self-Assessment checklist Anecdotal, individual feedback Focus Groups and one-time surveying -

qualitative & quantitative data ECAR, Soochow & University of Hong Kong survey,

University of Wisconsin/ECAR survey

Polling Microtrends (Mark J Penn)

Statistics Logs, time measures, ratios, control charts, variation

8

Service Quality (SERVQUAL) Model – Published 1990

Comprehensive customer view of Quality Service over time addresses:

Tangibles Facilities, equipment, personnel, materials

Reliability Delivering what’s promised

Responsiveness Helping customers with immediate needs

Assurance Competence, skills, knowledge, credibility, courtesy,

security Empathy

Access, communication, understanding the customer

9

Evolution: SERVQUAL, LibQual+®,

ServQual 1990Measure five service areas

LibQual+® mid 90sMeasure place, service affect,

information control

10

Higher Education TechQual+

2006 -Tim Chester, CIO, TAMU/Qatar then Pepperdine University

3-year research project – multiple rounds of qualitative and quantitative data collection

Statistically reliable, valid and universal instrument for technology service assessment

12 institutions participating in development

11

TechQual+ Survey topics

Instrument Content Analysis

12

TechQual+ Survey The Instrument

Standardized, outcomes-based ‘zone-of-tolerance’ analysis assessment tool

Customer, not service-provider, perspective In-house comparison across years National comparison across institutions Large numbers of customer data to support data-driven

decisions Useful to set internal priorities and allocate resources

The Tool Web-delivered, remotely hosted on SQL server, institution-defined loading categories www.techqual.org

13

TechQual+ Content 6 areas of questions

Inclusive planning Access Campus information systems Web Support Classroom Technology

5 questions per area 3 answers per question

Desired level of service Minimum level of service Perceived or experience level

14

Content related to SERVQUAL Customer values

Empathy: Inclusive planningReliability: Access, Campus

information systemsAssurance: Campus information

systems, WebResponsiveness: Support Tangibles: Classroom Technology,

Support

15

Statistics in TechQual+ Number of respondents, mean and

standard deviationService adequacy gapService superiority gap

Zone of tolerance identified

16

For Example:

17

*Qual+ Analysis By groups for each question

18

TechQual+ RefinementUNM’s involvement

CIO/ITS and HSLIC co-sponsoring UNM HRRB approval of research project using human subjects Principal Investigators: Holly Buchanan, Barney Maccabe Investigators: Jane McGuire, Sally Bowler-Hill

Focus Groups – April 2007 - faculty, staff, student and 2 mixed groups Participants looking for Collaboration, Consistency and

Communication – True in every school, large and small! Need for personal control: if no control, it better work

Pilot Surveying Internal central IT on main and north campuses – June 2007 5 Colleges & admin groups – September 2007

19

Why Pilot ITS/HSLIC?

Gather opinions on most & least important questions to ask UNM community

Establish a baseline for internal perception of IT services

Learn the survey instrument Tests the interfaces and administrative

functions of this instrument

20

ITS/HSLIC Results

HSLIC ITS Total

Invited 90 241 331

Respondents 42 29 71

Response Rate 47% 12% 21%

Incomplete surveys 35

Faculty respondents 8

Staff respondents 63

Student respondents 0

21

ITS/HSLIC Vertical Bar Chart

Incl

usiv

e

Incl

usiv

e

Plann

ing

Plann

ing In

fo.

Info

. Sy

stem

s

Syst

ems

Acce

ss

Acce

ss

Cla

ssro

om

Cla

ssro

om

Tech

nolo

gy

Tech

nolo

gy

Web

Web Su

ppor

t

Supp

ort

Mean Score (1-9)

Question #

22

ITS/HSLIC Radar Chart

Questions

1-5 Incl Planning

6-10 Info Systems

11-15 Access

16-20 Classroom Tech

21-25 Web

26-30 Support

23

Campus Pilot

September 2007 Preliminary list of divisions to survey

Anderson School of Management College of Education College of Nursing University College University Libraries Administrative groups: Budget Office, Extended

University, Research & Economic Development, Continuing Ed

24

Demographic data loaded Status: faculty/staff/student Age:

<=18, 19-22, 23-30, 31-45, 46-65, >65 UNM ‘Age’

< 6mo, 6-24mo, 25mo-5 yr, >5 yr Gender Pure student & Faculty/staff Division/College groups

25

Consider for next time at UNM…

Fewer Questions or categories Motivate a higher open rate

Punchier invitations & reminders More remindersSender should have direct relationshipDepartmental competition?Better rewards? (iPods?)

26

Using survey results Work with departmental IT service providers

Share planning Share results

Use data to base decisions, set priorities and make requests

Organize for improvement, as suggested by ServQual developers: Formalize improvement processes State clear direction & priorities Involve many, emphasize teamwork Incremental steps, not all at once Empower staff, flatten the organization

27

ServQual: service quality gaps

2. Management perceptions

and service expectations

/ standards gap

3. Service quality specifications

and service delivery gap

4. Service delivery and external

communication gap

1. Customerservice

expectations& management perception gap

28

Close the service gaps

2. Define affordable

performance standards, train staff

3. Measure performance only against

the standards

4. Create appropriate expectations

with the customer

1. Understand customer expectations

& perceptions

29

Quality service tenets Constant, incremental improvement Add strategic value - don’t only provide a

commodity service Reliably flawless Right now! Individualize / personalize service

Right the first time or VERY right the second time

30

Questions / Discussion