Drivers of Satisfaction for Online Public Services

Email: [email protected]: www.umr.co.nzFacebook: www.facebook.com/umr.research

WELLINGTON3 Collina Terrace

ThorndonWELLINGTON 6011

NEW ZEALANDTel: +64 4 473 1061Fax: +64 4 472 3501

AUCKLAND11 Earle Street

ParnellAUCKLAND 1052NEW ZEALAND

Tel: +64 9 373 8700Fax: +64 9 373 8704

SYDNEYLevel One, Suite 105

332-342 Oxford StreetSYDNEY NSW 2022

AUSTRALIATel: +61 2 9386 1622Fax: +61 2 9386 1633

Drivers of Satisfactionfor Online Public Services

[November 2010]

Page | 2

Report on research carried out for the State Services Commission by UMR Research Published by the State Services Commission March 2011 ISBN: 978‐0‐478‐36132‐2 Crown copyright

This work is licensed under the Creative Commons Attribution 3.0 New Zealand licence. In essence, you are free to copy, distribute and adapt the work, as long as you attribute the work to the Crown and abide by the other licence terms. To view a copy of this licence, visit http://creativecommons.org/licenses/by/3.0/nz/. Please note that no departmental or governmental emblem, logo or Coat of Arms may be used in any way that infringes any provision of the Flags, Emblems, and Names Protection Act 1981. (www.legislation.govt.nz/act/public/1981/0047/latest/whole.html#dlm52216) Attribution to the Crown should be in written form and not by reproduction of any such emblem, logo or Coat of Arms. Note: This report, together with FAQs, can also be downloaded/printed from: www.ssc.govt.nz/online‐drivers‐of‐satisfaction

Page | 3

Table of contents

1. EXECUTIVE SUMMARY ......................................................................................................... 4

2. INTRODUCTION ................................................................................................................... 6

3. OVERVIEW OF APPROACH ................................................................................................... 7

3.1 Methodology ............................................................................................................. 7 3.2 Sampling .................................................................................................................... 7 3.3 Sample demographics ............................................................................................... 8

4. OBTAINING INFORMATION ................................................................................................. 9

4.1 Websites visited ........................................................................................................ 9 4.2 Key drivers ............................................................................................................... 10 4.3 Interpretation of drivers based on Qualitative Research ........................................ 11

5. TRANSACTING ................................................................................................................... 13

5.1 Websites visited ...................................................................................................... 13 5.2 Key drivers ............................................................................................................... 14 5.3 Interpretation of drivers based on Qualitative Research ........................................ 15

6. ADDITIONAL ANALYSIS ...................................................................................................... 18

6.1 Comparison with Kiwis Count 2009 ........................................................................ 18 6.2 Comparison with How New Zealanders Access Public Services .............................. 18 6.3 Demographic Analysis ............................................................................................. 19 6.4 Preferred way of accessing public services ............................................................. 19

APPENDIX A: FREQUENCY DISTRIBUTION OF RATINGS ............................................................. 20

A.1 Obtaining information ............................................................................................. 20 A.2 Transacting .............................................................................................................. 23

APPENDIX B: METHODOLOGY .................................................................................................. 26

B.1 Sample frame .......................................................................................................... 26 B.2 Quota management and sample sizes .................................................................... 28 B.3 Response rates ........................................................................................................ 29 B.3 Statistical analysis .................................................................................................... 30 B.3.1 Univariate analysis................................................................................................... 30

B.3.2 Bivariate analysis: Correlation ................................................................................. 30

B.3.3 Multiple Linear Regression ...................................................................................... 35

Page | 4

1. Executive summary The State Services Commission commissioned research to identify the key drivers of satisfaction with public services delivered online, whether it was for obtaining information or carrying out a transaction. The results showed that two of the five drivers in each of these categories were the same. These are highlighted in the table below with their relative contributions to overall satisfaction with the website. Key Drivers for Obtaining Information Key Drivers for Transacting

The service experience met my expectations (30%) The service experience met your expectations (28%)

I was satisfied with the time it took to do what I wanted to do (23%)

Overall, I was satisfied with the accessibility of the website (23%)

It was easy to find my way around the site (17%) It’s an example of good value for tax dollars spent (21%)

It’s an example of good value for tax dollars spent (16%)

When I got to the site it was easy to find what I was looking for (14%)

The lay‐out was clear (13%) The process was straightforward and easy to understand (14%)

The most important driver for both ‘Obtaining information’ and ‘Transacting’ on websites that offer public services is “The service experience met my expectations”. This is also the key driver for overall satisfaction with public services as reported in the Kiwis Count 2009 report. Qualitative research helped to understand how respondents interpreted this driver in the two scenarios of Obtaining Information and Transacting.

Obtaining Information Transacting

– Information sought being easy and quick to obtain – Hoping to be able to do what they wanted to do as

quickly as possible to save time, but also being apprehensive that it would be difficult and time‐consuming

– Concerns about not being able to get to the right site to do what they wanted or feeling the need to ask someone for help

– The website was expected to be factual and serious as opposed to ‘fun’

– Transaction forms being easy to access, understand and complete

– Confirmation of the completion of the transaction – Need for the information provided to be confidential

and that their privacy is assured, that is, their information is not shared. Government websites tended to be implicitly trusted to meet those expectations.

“It’s an example of good value for tax dollars spent” is a driver for both ‘Obtaining information’ and ‘Transacting’ on websites that deliver public services. It is also one of the drivers for overall satisfaction with public services as reported in the Kiwis Count 2009 report. Respondents in the qualitative research did not specifically mention this driver.

Page | 5

The most commonly visited websites for both Obtaining Information and Transaction are shown below. Obtaining Information Transacting

98% of all respondents visited public service websites in the last 12 months to obtain information.

68% of all respondents visited public service websites in the last 12 months for any transaction purposes.

The most popular websites visited were: ‐ Inland Revenue ‐ Local library ‐ Local council ‐ New Zealand Transport Agency

The most popular websites visited were: ‐ Inland Revenue ‐ New Zealand Transport Agency ‐ Local library ‐ Local council

68% of the respondents mentioned that they had most recently visited the website for personal reasons.

78% of the respondents mentioned that they had most recently visited the website for personal reasons.

Page | 6

2. Introduction The State Service Commission’s (SSC) New Zealanders’ Experience (NZE) Research programme explores New Zealanders’ satisfaction with the quality of public service delivery. The SSC has adapted the Common Measurements Tool (CMT) for use by New Zealand State Services agencies. The tool was originally developed in Canada by the Institute of Citizen Centred Service as a standard means of measuring customer satisfaction with public services. In 2007, the SSC commissioned a survey, known as the Drivers Survey, to determine the key drivers that influence satisfaction with service delivery in New Zealand. This identified a set of six drivers of satisfaction with public services. The drivers of satisfaction with service quality vary depending on the channel through which those services are delivered. Kiwis Count 2007 identified that there were five drivers that have the greatest impact on satisfaction with telephone services, only one of which matched the overall set of drivers. As more State Services are becoming available online, the SSC commissioned research to identify the key factors that drive satisfaction with public services delivered online. The SSC will produce a set of questions to be used as part of the CMT. The research focused on two different ways in which people experience public services online. Those who go online to websites only to obtain information and those who do a transaction, for example, requesting information to be sent to them, ordering or applying or paying for something from a public service organisation. The main objectives of the research were: ‐ to identify the key drivers of satisfaction for online services, and ‐ develop a set of questions for inclusion in future CMT surveys of online public services.

The structure for the rest of the report is explained below: Section 3: This explains the broad methodology in which the project was conducted and the final sample sizes achieved. Section 4: This section details the websites visited for obtaining information, whether visited for work or for personal reasons and focuses on the key drivers of satisfaction for obtaining information from public services websites. Section 5: This section details the websites visited for transaction purposes, whether visited for work or for personal reasons and focuses on the key drivers of satisfaction for transacting on websites that offer public services. Section 6: This section gives further information on the drivers by drawing comparisons with the Kiwis Count report and based on demographic differences, if any. The Appendices show the frequency distributions of key drivers, along with overall measures and provides a detailed explanation of the methodology.

Page | 7

3. Overview of approach

3.1 Methodology A six step approach was adopted that included workshops with SSC’s experts in the area of citizen satisfaction and a limited amount of qualitative research. These provided inputs into a final questionnaire for a survey that was conducted online. The qualitative research comprised of four focus groups; two of which were conducted with those who had obtained information and two with those who had transacted on a website that offered a public service in the past year. One of each type of groups was held in Napier and Auckland. 1. Initial workshop 2. Small scale qualitative research 3. Second workshop 4. Pre‐fieldwork testing including cognitive testing and pilot testing 5. Quantitative online survey data collection 6. Data processing, analysis and reporting This report gives a detailed account of the results from the quantitative survey, with results from the qualitative research woven in where appropriate. An online survey methodology was used for its various advantages. The strongest reason for conducting an online survey was that it involved using the media that is being explored. Additional cognitive testing of the core drivers of online satisfaction was carried out to test the statements for inclusion in the CMT. The results are included in a separate report that also gives a more detailed account of the qualitative research.

3.2 Sampling The survey sample comprised of n=1,644 New Zealanders that have visited a New Zealand website that offers a public service in the past 12 months. The sample targeted was 1,500. For a sample of n=1,644 the margin of error for a 50% figure at the 95% confidence level is plus or minus 2.4%. The final sample sizes for visiting websites for obtaining information was 1,608 and for visiting websites for transaction purposes was 1,119. The sampling frame was compiled based on data collected in UMR’s Omnibus survey on those who said that they had visited a public service website in the past 12 months. The sample frame allowed us to select a representative sample based on gender, age and region.

Page | 8

3.3 Sample demographics A breakdown of all the respondents across various demographics who participated in this survey is shown below. The sample frame was weighted by age, gender and region in order to ensure that the sample was representative of New Zealanders who had visited a government website.

SAMPLE INFORMATION/ DEMOGRAPHICS ‐ WEIGHTED

All Obtained

Information Transacted

n=1,644 n=1,608 n=1,119

% % %

Sex

Male 49 49 50

Female 51 51 50

Age

15‐24 11 11 11

25‐44 50 51 49

45‐64 28 28 30

65 plus 10 10 11

Ethnicity

Māori 12 12 12

Pacific 4 4 4

Asian 8 8 9

Base: All respondents

Page | 9

4. Obtaining Information

4.1 Websites visited Ninety eight percent (1,608) of all respondents (1,644) visited a website to obtain information in the last twelve months. Respondents were shown a list of different websites and were asked to check all the websites that they had visited in the last 12 months to obtain information for work or for personal reasons. Inland Revenue was the website that was most frequently visited in the last 12 months (63%), followed by New Zealand Transport Agency (37%), local libraries (36%) and local councils (36%) websites. Respondents were also asked to select which website they had visited most recently to obtain information. The top four websites visited most recently were the same, though the order was different: Inland Revenue (23%), followed by local libraries (13%), local councils (10%) and New Zealand Transport Agency (8%).

Visited in last 12 months Visited most recently

Base 1,608 1,605

% %

Inland Revenue 63 23

Local library 36 13

Local council 36 10

New Zealand Transport Agency 37 8

Companies office 19 5

StudyLink 19 4

Work and Income 26 4

Department of Internal Affairs 17 3

Sorted.co.nz 25 3

Regional council 21 3

Jobs.govt.nz 15 2

Immigration Services 10 2

Statistics New Zealand 17 2

New Zealand Qualifications Authority 14 2

Department of Labour 14 2

Police 17 2

Ministry of Health 13 2

Department of Building and Housing 7 1

Ministry of Economic Development 8 1

Department of Conservation 11 1

Housing New Zealand Corporation 5 1

Career Services 11 1

Ministry of Justice 13 1

Land Information New Zealand 9 ‐

Parliamentary Counsel Office 2 ‐

Other public service website 7 3

Page | 10

A majority of the respondents said that they had most recently visited the website for personal reasons (68%).

Purpose of obtaining information from website

Base: 1,605 %

Work reasons 31Personal reasons 68

4.2 Key drivers New Zealanders rated 34 specific service attributes1 in relation to the overall satisfaction with the public service website that they had visited most recently. Regression analysis was then undertaken to identify the drivers of satisfaction with public service websites when obtaining information. The drivers of satisfaction for obtaining information from the public service websites are shown in the graph below. All these drivers were found to be significant at a 95% confidence level2. Together, these drivers explained 82% of the variation in overall satisfaction ratings with the public service website visited for obtaining information. The percentage given alongside each driver indicates the relative impact that this factor has on driving up satisfaction with the website. The total number of observations for this model was 1,483.

13%

16%

17%

23%

30%

0% 5% 10% 15% 20% 25% 30% 35%

The lay‐out was clear

It’s an example of good value for tax dollars spent

It was easy to find my way around the site

I was satisfied with the time it took to do what I wanted to do

The service experience met my expectations

DRIVERS OF WEBSITE SATISFACTION (FOR OBTAINING INFORMATION)

1 A list of all the 34 attributes is shown in the appendix. 2 A 95% significance or a p value of 5% means that there is only a 5% chance that results of this model would have compe up in a random distribution. So we can say with a 95% probability of being correct that the variable is having some effect, assuming the model is specified correctly.

Page | 11

4.3 Interpretation of drivers based on Qualitative Research

The service experience met my expectations The qualitative research found that expectations were best met if the information sought was easy and quick to obtain. Before going to a public service website, respondents said that they hoped to be able to do what they wanted to do as quickly as possible to save time, but were also apprehensive that it would be difficult and time‐consuming. Some thought they would not be able to get to the right site or felt that they would need to ask someone for help. The website was expected to be factual and serious as opposed to ‘fun.’ There was also an expectation by some that more information would be available on‐line. To make information easy and quick to obtain, the following features were important to users. ‐ having, straightforward, clear, logical pathways to information ‐ sites that are easy to navigate and search ‐ sites that tell people what they have to do in a simple and easily understood way ‐ well laid out websites ‐ provision of easy steps to most requested information ‐ quick loading of information ‐ providing the information that is required and which can be easily understood Quality of information was also important to satisfaction. This meant the information had to be accurate and up‐to‐date. When you get the information it’s got to be up to date. There’s no use getting information

only to find out it hasn’t been updated. (Napier, transactors, male) Those who preferred to obtain information online did so primarily for the convenience. I would use the website first and see how far I go there. It means I don’t have to leave

home, it’s a time thing and a convenience thing. (Auckland, obtained information, male) Convenience included the ability to access information at a time that suited, that it was quicker than trying to carry out the same task either by phone or by visiting an office in person.

I was satisfied with the time it took to do what I wanted to do The qualitative research found that while it was important to overall satisfaction to be able to obtain the information quickly, the time taken varied between individuals and was not described as a specific number of seconds or minutes. Instead, the speed of obtaining information was described in terms of the few clicks of a mouse it took. If you’re searching for information, if you can get the information within a few clicks.

(Napier, transactors, male) One click away depending on the situation but mainly one click away. Then you are gone.

[So we are still talking about speed and ease of getting the information.] Yes. (Auckland, transactors, male)

What you are looking for is a progression, big tab, smaller tab, a flow. (Napier, obtained

information, male)

Page | 12

It was easy to find my way around the site / Lay‐out was clear There was also considerable overlap between the speed of obtaining information and being able to do things easily. If it’s easy to follow then it should be speedy. If it’s hard to direct yourself around a

website, then it’s not easy to follow and it’s going to be time-consuming. (Napier, transactors, female)

I have struggled with websites which are poorly defined for clarity, the fonts are wrong,

the colour background is wrong, takes forever to load. So website design. (Auckland, transactors, male)

Some respondents identified ease of navigation, which in turn was closely linked to how clearly the website was laid out. Ease of navigation. [What makes something easy to navigate?] Clear menus and drop

down menus that actually make sense. (Napier, obtained information, male) Others talked about a logical progression to the information sought, which also made it quick to access. Some described ‘logical’ as being ‘easy’ to access information. Logical and linear path to the information that makes me feel satisfied (Auckland,

transactors, male) Logical and easy way of getting around the website. [What is the best way to do that?] A

link as well as text and you know if you hit that link you are going in the right direction. (Auckland, obtained information, male)

Some spoke about the need for good search engines to ensure the information was obtained quickly. Some stated that they would be satisfied if it was easy to understand what to do and if contact details were available for assistance if problems were encountered. Easy to follow and easy to understand. [And a contact person and a phone number for if

you don’t. (Interjection.)] (Napier, transactors, female) Make sure you aim for the normal person instead of some of the language they use can be

confusing for us souls. (Auckland, obtained information, male)

Good value for tax dollars spent Respondents did not specifically mention this driver in the qualitative research. However, in the pre‐fieldwork

testing, most respondents related the statement back to whether they had had a good or bad experience and

rated accordingly; a good experience is usually seen as good value.

Page | 13

5. Transacting

5.1 Websites visited Sixty‐eight percent (1,119) of all respondents (1,644) visited a website to obtain information in the last twelve months. Respondents were shown a list of different websites and were asked to check all the public service websites that they had visited in the last 12 months for transaction purposes for work or for personal reasons. Inland Revenue was the website that was most frequently visited in the last 12 months (39%), followed by New Zealand Transport Agency (32%), local libraries (22%) and local councils (14%). Respondents were then asked to select which websites they had visited most recently for transaction purposes. The top three websites visited most recently were the same: Inland Revenue (23%), New Zealand Transport Agency (23%) and local libraries (12%). StudyLink (7%) and local council (6%) were the next most recently visited websites.

Visited in last 12 months Visited most recently

Base: 1,119 1,091

% %

Inland Revenue 39 23

New Zealand Transport Agency 32 23

Local library 22 12

StudyLink 12 7

Local council 14 6

Department of Internal Affairs 8 4

Work and Income 11 3

Companies office 9 3

Police 7 3

Regional council 7 2

Jobs.govt.nz 5 2

Immigration Services 5 2

Ministry of Justice 5 2

Other public service website 3 2

Sorted.co.nz 4 1

Statistics New Zealand 4 1

New Zealand Qualifications Authority 4 1

Ministry of Health 3 1

Department of Conservation 3 1

Land Information New Zealand 3 1

Department of Labour 3 1

Department of Building and Housing 3 1

Ministry of Economic Development 3 1

Career Services 3 0

Housing New Zealand Corporation 2 0

Parliamentary Counsel Office 0 0

Unsure 3 0

Page | 14

A majority of respondents stated that they had most recently visited the website for transacting for personal reasons (78%). Visiting websites for personal reasons was higher for respondents who were transacting (78%) compared to those who were looking for information (68%).

Purpose of transacting on website

Base: 1,091 %

Work reasons 22 Personal reasons 78

5.2 Key drivers New Zealanders rated 44 specific service attributes3 in relation to the overall satisfaction with the website that they had visited most recently. Regression analysis was then undertaken to identify the drivers of satisfaction with public service websites when carrying out transactions. The drivers of satisfaction for transacting on public service websites are shown in the graph below. All these drivers were found to be significant at 95% confidence level4. Together, these drivers explain 82% of the variation in overall satisfaction ratings with the public service website visited for transaction purposes most recently. The percentage given alongside each driver indicates the relative impact that this has on driving up satisfaction with the website. The total number of observations for this model was 1,004.

14%

14%

21%

23%

28%

0% 5% 10% 15% 20% 25% 30%

The process was straightforward and easy to understand

When I got to the site it was easy to find what I was looking for


Overall, I was satisfied with the accessibility of the website

The service experience met your expectations

DRIVERS OF WEBSITE SATISFACTION (FOR TRANSACTION)

3 A list of all the 44 attributes is shown in the appendix. 4 A 95% significance or a p value of 5% means that there is only a 5% chance that results of this model would have come up in a random distribution. So we can say with a 95% probability of being correct that the variable is having some effect, assuming the model is specified correctly.

Page | 15

5.3 Interpretation of drivers based on Qualitative Research

The service experience met your expectations The qualitative research found that expectations were best met for those transacting online if transaction forms were easy to access, understand and complete. Confirmation of the completion of the transaction was also critical. In contrast to obtaining information, respondents expressed the need for the information they provided to be confidential and that their privacy was assured, that is, their information was not shared. Government websites tended to be implicitly trusted to meet those expectations. I am not too worried if I am on a government website. If I was on a non-government

website I would be a bit more. Depending if it was a bank or something like that. Because the government I would have thought has pretty good security systems put in place stopping people hacking and stuff. (Auckland, transactors, male)

The following features are important to meet the expectations of those transacting: ‐ easy to access forms that are easy to fill in/ have clear explanations about what to do ‐ the need to provide only essential information ‐ immediate receipt to confirm the transaction has been successfully completed ‐ visual confirmation that the site is secure ‐ a means of being able to track the progress of the transaction ‐ information on what happens next once the transaction is complete ‐ forms load quickly.

Overall, I was satisfied with the accessibility of the website The qualitative research found that accessibility was an issue. Some respondents find it difficult to locate the right government website to go to do what they want to in order to transact. A central directory that could direct people to where they needed to go was suggested.

I think the first thing is finding the government department and the website ... There is no single newzealandgovernment.com, I don’t know if there is or not I haven’t looked for it. [Would it be useful to have one?] I think it possibly should be an overriding front end to all the other government websites. I am dealing with the government, this is my starting point ... It just gives you an idea also of the full breadth and width of what the government is doing. Here are all the government websites we have got. (Auckland, transactors, male)

I typed in nzgovt or whatever it was and up popped this site and I just looked and thought “where do I go to from there?” and I had to sit there and read and read and read and I couldn’t find what I wanted on that. I think I actually found another link that took me to another site so I had to read all of that to find out where I was going. (Napier, transactors, male)

It’s an example of good value for tax dollars spent As noted, respondents did not specifically mention this driver in the qualitative research. However, what has been found in previous research, and in the pre‐fieldwork testing, is that this statement is usually related to whether people have had a good or bad experience. A good experience is usually seen as good value.

Page | 16

When I got to the site it was easy to find what I was looking for The ability to direct people straight to the right website and the right part of the site to transact was a strong influence on satisfying people. [What is the most important if you are transacting on a website, two most important things

for you to make you satisfied with the experience?] Logical way of getting where you need and getting information. (Auckland, obtained information, male)

Obviously satisfied we can get online and can do the transaction. (Napier, transactors,

male) The importance of being directed quickly to where to transact was illustrated by examples given by two respondents talking about making payments online. I would say my best experience ever with a government website was paying my parking

ticket. They made that so quick and easy it wasn’t funny. [What was so good about that?] I can’t remember how the ticket came, whether it was in the post but it was basically enter this number into this website, bang it is there and it takes you off to the secure site you put your credit card details in and the money is gone and that is that. [So it was quite simple to do?] Yes. (Auckland, obtained information, male)

I have to agree that is how mine was, gave me the website and everything you needed, took

you straight there. [What is the alternative to not paying it online?] Cheque. [Would that be a lot more difficult?] Yes. (Auckland, obtained information, male)

The process was straightforward and easy to understand The ease of transaction was important and this largely came down to forms that are easy to access and fill out, as well as language and instructions that are easy to understand. Maybe the forms that you’re filling in, if you’re required to fill in forms, are easy to follow.

And access so you don’t have to find them. (Napier, transactors, female)

You can tell that the government sites have been written by somebody who is in government because it’s not in everyday English. It’s probably taken them weeks or months to write and it takes us twice as long to understand it. (Napier, transactors, female)

Satisfaction is linked to being able to provide the minimum amount of information necessary to do the transaction. For example, only having to give information once.

Easy I suppose is providing the minimum of information that they need. It doesn’t come back and say required information for the address, you go I gave it to you last month and the one before that. (Auckland, transactors, male)

Sometimes the type of questions they ask and the sheer number of questions can lead to either a satisfied or dissatisfied state. (Napier, transactors, male)

For me simple is good, fewer questions. The less information I have to give the better, not overly detailed questions, again more streamlined. If I feel they are asking me for unnecessary information I might get annoyed so I want a really streamlined process, the fewer questions the better only asking me for the essentials. (Auckland, transactors, male)

Page | 17

Some find it easier if the website shows how far through a transaction they are as it provides a visual clue to how things are progressing and how long it will be likely to take.

The bar across the top shows where you are in the transaction, that is really good, Step One, Step Two and you know you are nearly at the end. [So why is that important?] You know you haven’t got long to go, it is a time thing, it always comes back to time. (Napier, transactors, male)

A critical part of the transaction process was instant confirmation of its completion. I want that final assurance that the process is finished and been submitted. You get an

assurance back ideally on the website that says thank you and that completion is clear rather than getting to the end of the process and no clear signal. (Auckland, transactors, male)

A few still wanted the reassurance of someone contacting them to confirm completion. I would rather have a phone call than an email, bureaucratic receipts I would rather have

someone talk to me about it than having to read it. (Auckland, transactors, male) Twenty‐four hours was the expected maximum time for a response on‐line to a query. People did not want to be left ‘dangling’ and unsure what would happen next and when. That the confirmation gives us an expected time frame for an answer. [What do you

expect as a decent time frame?] I expect a day. If you are just asking a straight forward question for example how big does my fence have to be for the dog then straight back. You expect a day or less. (Auckland, obtained information, male)

If we’re making a transaction we expect a reply within a certain timeframe. (Napier,

transactors, female) Some talked about the need for tracking numbers on transactions and the ability to print transactions so they could be satisfied that they had tangible evidence that the transaction had occurred.

Page | 18

6. Additional Analysis Further analysis was carried out by comparing the drivers with the overall drivers identified in the Kiwis Count report and also testing for demographic differences, if any.

6.1 Comparison with Kiwis Count 2009 Two of the drivers of satisfaction with public services delivered online are the same as the overall drivers of satisfaction with public services, as reported in the Kiwis Count 2009 report. The following table gives a comparison of the relative contribution of these two drivers from the two studies. Key Driver of Satisfaction (Relative Contribution)

Obtaining Information from public service websites

Transacting in public service websites

With public Services (Kiwis Count 2009)


30% 28% 31%


16% 21% 11%

6.2 Comparison with How New Zealanders Access Public Services Comparisons have also been drawn with the report on service channels based on Kiwis Count 2009 (How New Zealanders Access Public Services). New Zealanders were asked about the key aspects that would encourage or increase their use of public services over the internet. The most important aspect stated was “Make it simple and user‐friendly”, chosen by 63% of the respondents. This matches some of the drivers that were found to improve the satisfaction with public services websites for both obtaining information as well as transacting.

Public Services (Kiwis Count 2009)

Satisfaction with obtaining information from public service websites (Drivers)

Satisfaction with transacting in public service websites (Drivers)

Make it simple and user‐friendly


When I got to the site it was easy to find what I was looking for

The lay‐out was clear The process was straightforward and easy to understand

Other aspects mentioned in the Channels Report related to privacy and security, with “Ensure my privacy is protected” chosen by 48% of respondents and “Improve online security” selected by 35% of respondents. However “privacy issues” or “security concerns” did not feature as a key driver for satisfaction for public service websites visited for transaction purposes (especially). The qualitative research found that respondents generally trusted public service websites to be secure and to keep information provided by individuals as confidential.

Page | 19

6.3 Demographic Analysis Driver Analysis was carried out separately on the two groups of respondents who had visited any public service website in the last twelve months, either for obtaining information or for transaction purposes. The overall drivers were tested out by key demographics such as age, ethnicity and income using indicator variables in the regression model. There were no differences seen across these demographics at the 95% significance level for those who visited public service websites for obtaining information. For transaction, while Māori and Asian came out significant at the 95% significance level, separate analysis results was not reported on them due to very small sample sizes.

6.4 Preferred way of accessing public services Respondents were asked what their preferred way of accessing public services was. As expected from an online panel, a majority of the respondents visiting the websites, both for obtaining information as well as for transaction purposes, stated that internet was their preferred mode of access. There was no difference in the preferred method of accessing public services, whether obtaining information or transacting.

PREFERRED METHOD OF ACCESSING PUBLIC SERVICES

Thinking about accessing public services, generally speaking which is your preferred way of accessing services?

Obtained Information Transacted

n=1,608 n=1,119

% %

Internet 67 70

Phone 17 16

Face to face 14 12

Postal Mail 1 1

Unsure 1 1

Base: All respondents

Page | 20

Appendix A: Frequency distribution of ratings

A.1 Obtaining information Respondents were first asked to rate their overall experience when they obtained information from the website that they visited most recently. Almost half the respondents (48%) mentioned 8‐10 rating on the scale of 0 to 10, where 0 means very poor and 10 means very good.

5%

6%

4%

7%

7%

7%

9%

7%

11%

25%

26%

30%

53%

53%

48%

1%

1%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Overall Satisfaction

Overall Quality of Service Delivery

Overall Experience

0‐2 3‐4 5 6‐7 8‐10 Unsure

RATINGS ONOVERALL MEASURES

Then respondents were asked to rate 34 attributes (considered for driver analysis). Then they were asked two overall measures: Satisfaction with the overall quality of service delivery Overall satisfaction with the website Ratings on both these measures were similar and slightly more positive than the first overall measure, which was asked prior to the rating of the individual attributes. More than half the respondents gave a high satisfaction rating (8‐10) on both these measures.

Page | 21

The individual attributes tested are given below: Statements tested as drivers

1) It was easy to find this website

2) When I got to the site it was easy to find what I was looking for

3) I got the service I wanted after an acceptable number of steps

4) It was easy to find my way around the site

5) I was satisfied with the time it took to do what I wanted to do

6) I always knew where I was on the site

7) The process was straightforward and easy to understand

8) Overall, I was satisfied with the accessibility of the website

9) The search function/engine worked well

10) The site provided features and suggestions that were helpful

11) I feel confident that my privacy was fully protected on this website

12) I feel confident that my information was not shared with anyone else

13) I was able to do everything I needed to do online

14) I understood what to do every step of the way

15) In the end, I got what I needed from the website

16) The amount of time it took to get the overall service was acceptable

17) The service experience met my expectations

18) I was able to do what I wanted when I wanted

19) The site had all the information I needed

20) The information I needed on the site was up‐to‐date

21) The information on the site was accurate

22) There were useful links to other websites

23) The tone of the language on the site was appropriate

24) The language on this site was easy to understand

25) The information on the website was easy to understand

26) I was informed of everything I had to do to get what I wanted

27) I was not overloaded with irrelevant information

28) The information on the site was consistent

29) The site was visually appealing

30) The lay‐out was clear

31) I trust the information on this site

32) It’s an example of good value for tax dollars spent

33) My individual circumstances were taken into account

34) The site was tailored to meet my needs

Page | 22

The following five statements came out as the key drivers from the regression analysis. All the five drivers had a high proportion of respondents who were very satisfied (more than 70% rated 8‐10 across all the drivers) on each of the aspects. There were a higher percentage of respondents who were unsure (5%) and mentioned not applicable (2%) for “It’s an example of good value for tax dollar spent”.

6%

6%

6%

7%

6%

10%

8%

10%

8%

7%

10%

10%

11%

9%

9%

26%

23%

23%

23%

23%

47%

47%

49%

52%

54%

5%

1%

1%

2%

1%

1%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

The lay‐out was clear



I was satisfied with the time it took to do what I wanted to do


0‐2 3‐4 5 6‐7 8‐10 Unsure Not applicable

DRIVERSOF SATISFACTION WITH WEBSITE ‐ RATINGS

Page | 23

A.2 Transacting Similar to the previous section, respondents were first asked to rate their overall experience when they had visited the website for transaction purposes most recently. More than half the respondents (56%) mentioned 8‐10 rating on the scale of 0 to 10, where 0 means very poor and 10 means very good.

5%

6%

5%

5%

5%

6%

7%

7%

7%

24%

22%

25%

57%

60%

56%

1%

1%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Overall Satisfaction

Overall Quality of Service Delivery

Overall Experience

0‐2 3‐4 5 6‐7 8‐10 Unsure

RATINGS ONOVERALL MEASURES

Then respondents were asked to rate 44 individual attributes. Then they were asked two overall measures: Satisfaction with the overall quality of service delivery Overall satisfaction with the website Ratings on both these measures were similar to the first overall measure, which was asked prior to the rating of the individual attributes.

Page | 24

The individual attributes tested are given below:

Statements tested as drivers

1) It was easy to find this website

2) When I got to the site it was easy to find what I was looking for

3) I got the service I wanted after an acceptable number of steps

4) It was easy to find your way around the site

5) I was satisfied with the time it took to complete the transaction

6) I was satisfied with the time it took to do what I wanted to do

7) I always knew where I was on the site

8) The process was straightforward and easy to understand

9) The amount of information I had to provide was acceptable

10) The type of information I had to provide was acceptable

11) Overall, I was satisfied with the accessibility of the website

12) The search function/engine worked well

13) The site provided features and suggestions that were helpful

14) I feel comfortable sending my credit card number over the Internet to pay for a government service

15) I feel confident that my privacy was fully protected on this website

16) I feel confident that my information was not shared with anyone else

17) I received confirmation in an acceptable time that I had completed the transaction

18) I knew when the service would be delivered

19) I was able to do everything I needed to do online

20) I understood what to do every step of the way

21) I understood what would happen next

22) In the end, I got what I needed from the website

23) The amount of time it took to get the overall service was acceptable

24) The service experience met your expectations

25) I was able to do what I wanted when I wanted

26) The site had all the information I needed

27) The information I needed on the site was up‐to‐date

28) The information on the site was accurate

29) There were useful links to other websites

30) The tone of the language on the site was appropriate

31) The language on this site was easy to understand

32) The information on the website was easy to understand

33) I was informed of everything I had to do to get what I wanted

34) Forms were easy to understand and fill out

35) I was not overloaded with irrelevant information

36) The information on the site was consistent

37) The site was visually appealing

38) The lay‐out was clear

39) I was able to review the information I was providing before submitting it

40) I was able to see all my previous transactions

41) I would get fast support if I forgot my password

42) It’s an example of good value for tax dollars spent

43) My individual circumstances were taken into account

44) The site was tailored to meet my needs

Page | 25

The following five statements came out as the key drivers from the regression analysis. All the five drivers had a high proportion of respondents who were very satisfied. More than 60% rated 8‐10 across three of the drivers while 55% rated 8‐10 on the remaining two. There were a higher percentage of respondents who were unsure (5%) and mentioned not applicable (2%) for “It’s an example of good value for tax dollar spent”, similar to that in the obtaining information section.

6%

7%

5%

4%

6%

5%

7%

6%

5%

5%

7%

7%

8%

7%

6%

21%

23%

19%

20%

20%

61%

55%

55%

64%

61%

1%

5% 2%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

The process was straightforward and easy to

understand

When I got to the site it was easy to find what I was looking

for


Overall, I was satisfied with the accessibility of the website

The service experience met your expectations

0‐2 3‐4 5 6‐7 8‐10 Unsure Not applicable

DRIVERSOF SATISFACTION WITH WEBSITE ‐ RATINGS

Page | 26

Appendix B: Methodology

B.1 Sample frame The sampling frame was compiled based on data collected in UMR’s Omnibus survey5 on those that claimed to have visited a government website in the past 12 months. The sample breakup was designed by gender, age and region. Fifty five percent of the respondents said that they had used the internet to look for information about public services and/or have used the internet to carry out transactions or dealings with the public sector (40%). The omnibus data was extrapolated out across the 2006 Census population. Assumptions have been made for the 15‐17 year old population as the Omnibus survey only includes respondents 18 years of age or older. We have assumed that visitation would be at least as high as that of 18 or 19 year olds given the likely usage of NCEA and NZTA websites.

The table below shows the proposed sampling regime of 1,500 by region, age and sex based on

visitation recorded in the UMR Omnibus survey: SAMPLING REGIME

Male

15‐29

Male

30‐44

Male

45‐59

Male

60 Plus

Female

15‐29

Female

30‐44

Female

45‐59

Female

60 Plus

Northland 5 4 5 4 9 13 11 3

Auckland 84 97 64 33 65 96 52 32

Waikato 14 16 10 17 9 33 20 11

Bay of Plenty 17 10 15 10 15 16 13 15

Gisborne/ Hawkes

Bay 7 8 5 4 8 11 4 5

Manawatu‐

Wanganui/ Taranaki 15 27 9 3 13 7 14 11

Wellington+

Wairarapa 18 29 19 3 27 35 20 10

Marlborough/

Tasman/ Nelson 4 6 2 3 6 8 12 3

Christchurch+ Timaru 38 16 31 18 31 32 32 15

Otago/ Southland/

West Coast 18 20 14 11 20 22 3 5

TOTAL 220 233 174 106 203 273 181 110

5 The Omnibus survey is a fortnightly telephone survey of a nationally representative sample of 750 New Zealanders aged 18 and over conducted by UMR Research.

Page | 27

The following tables outline the actual population according to the 2006 Census and the extrapolated target population (those that have visited a government website in the past 12 months) based on the UMR Omnibus survey.

ACTUAL POPULATION NUMBERS ‐ CENSUS 2006

Male

15‐17

Male

18‐29

Male

30‐44

Male

45‐59

Male 60

Plus

Female

15‐17

Female

18‐29

Female

30‐44

Female

45‐59

Female 60

Plus

Northland 3,708 8,049 13,791 15,288 14,250 3,489 8,232 15,570 15,966 15,336

Auckland 31,083 111,822 148,785 117,897 82,731 30,135 116,814 164,082 123,417 99,087

Waikato 9,240 28,983 38,349 36,036 30,474 9,114 28,611 42,183 37,443 34,761

Bay of Plenty 6,327 15,429 24,567 24,459 23,586 6,081 15,900 28,164 26,343 27,252

Gisborne/

Hawkes Bay 4,851 11,883 18,522 18,918 15,945 4,770 12,183 20,835 19,761 18,981

Manawatu‐

Wanganui/

Taranaki

8,025 23,523 31,695 31,323 28,548 7,857 23,316 34,680 32,442 34,047

Wellington+

Wairarapa 9,903 36,879 50,130 41,691 31,974 9,738 39,237 55,014 43,470 38,472

Marlborough/

Tasman/ Nelson 3,000 7,797 13,296 14,391 12,351 2,862 7,326 14,490 14,745 14,010

Christchurch+

Timaru 11,391 40,575 55,197 51,702 43,584 10,926 39,546 59,868 53,349 53,220

Otago/

Southland/ West

Coast

6,966 26,139 32,205 32,589 27,069 6,612 26,940 34,128 31,881 31,641

TARGET POPULATION

(Those that have visited a government website in last twelve months)

Estimates based on UMR Omnibus question cross‐referenced with Census 2006 data

Male

15‐17

Male

18‐29

Male

30‐44

Male

45‐59

Male 60

Plus

Female

15‐17

Female

18‐29

Female

30‐44

Female

45‐59

Female 60

Plus

Northland 2,596 4,636 4,597 6,115 4,750 2,442 8,232 15,570 12,773 3,834

Auckland 21,758 77,415 114,450 75,454 39,567 21,095 56,523 113,295 61,709 38,110

Waikato 6,468 9,661 19,175 12,012 20,316 6,380 4,769 38,668 23,402 13,035

Bay of Plenty 4,429 15,429 12,284 17,471 11,793 4,257 13,250 18,776 15,053 18,168

Gisborne/

Hawkes Bay 3,396 4,753 9,261 6,306 5,125 802 8,122 12,501 4,391 5,998

Manawatu‐

Wanganui/

Taranaki

5,618 11,762 31,695 10,441 3,569 2,537 13,323 8,670 16,221 13,619

Wellington+

Wairarapa 6,932 14,752 34,464 22,741 3,553 3,728 28,026 42,070 23,711 11,542

Marlborough/

Tasman/ Nelson 2,100 2,599 6,648 2,878 3,970 1,772 4,934 9,660 14,745 3,503

Christchurch+

Timaru 7,974 36,518 18,399 36,496 21,792 6,817 29,660 37,418 37,344 17,740

Otago/

Southland/ West

Coast

4,876 16,337 24,154 16,295 13,535 599 23,573 26,544 3,542 6,328

Page | 28

B.2 Quota management and sample sizes Using the sampling frame a randomised list of contacts was drawn from the panel taking into account their age, gender and region. The number of respondents selected exceeded the targeted sample but was in the proportion of the sample frame. As respondents took part in the survey, questions confirming the respondent’s demographics were asked. This allowed for management of quotas and would not allow respondents to complete the survey if their demographic quotas, as outlined in the sampling regime, were full. During the course of the online survey, quotas were monitored so that reminders could be sent and if necessary, increase the sample in certain demographics to increase the amount of responses. This was especially true for the age group 15‐29 which had a low response rate and were targeted with reminders and additional sample was sent from the panel. This survey contained two sections, measuring drivers of satisfaction with respondents obtaining information and transacting from a public service website. The survey’s quotas were originally set so that n=1,500 respondents, matching the sampling frame, would have answered at least one of the sections. As expected, a higher proportion of respondents met the criteria to undertake the obtaining information section than the transacting section. The overall quotas were increased, while still matching the proportions of the sampling regime, so that at least n=1,000 would have completed the transacting section. Despite sending reminders and increasing the number of invites, the sample still fell short in certain demographies such as the lower age group (response rate of 15.7%). This was corrected by weighting it back to the sample frame by age and gender. A few responses were removed where respondents mentioned visiting websites that were not public services in New Zealand.

Page | 29

B.3 Response rates The response rates and completion rates for the survey are reported in the following table. It is based on the definitions of ESOMAR6 and IMRO7.

Metrics Definition No. of respondents

Response Rate = 26.2%

Based on people who have accepted the invitation to the survey and started to complete the survey (or % questionnaire opened)

3,508

Completion rate = 12.4%

Based on the proportion of those who have started, qualified, and then completed the survey (or % validated)

1,654

6 Note: According to ESOMAR (2005), response rates for online panels should be reported as follows: “Response based on the total amount of invites (% of full numbers) per sample drawn (country,

questionnaire); % questionnaire opened; % questionnaire completed (including screen‐out); % in target group (based on quotas);

% validated (the balance is cleaned out, if applicable)” (p. 20). 7 Note2: IMRO (2006): Response rate is “based on the people who have accepted the invitation to the survey and started to complete the survey. Even if they are disqualified during screening, the attempt qualifies as a response” (p. 13). Completion rate “is calculated as the proportion of those who have started, qualified, and then completed the survey” (p. 13).

Page | 30

B.3 Statistical analysis

B.3.1 Univariate analysis Univariate analysis is the method for analysing data on a single variable at a time. An examination of each critical variable in terms of proportion of respondents who mentioned that they were unsure, or that it was not applicable to them, the average and distribution exposed potential problems that could affect the subsequent multivariate analysis. The distribution for each potential driver along with their means and standard deviation was examined to understand the pattern of responses for each variable and whether there were any exceptional cases. Most of the attributes had very low proportion of respondents who selected either unsure or not applicable. Statements with large proportion of ‘unsure’ or ‘not applicable’ did not come up in the driver analysis.

B.3.2 Bivariate analysis: Correlation Correlation is a statistical technique that can show whether and how strongly pairs of variables are related. While some correlations are fairly obvious in the data, it may contain unsuspected correlations, or correlations may be suspected, yet it is not known which are the strongest. In our analysis we have performed two sets of correlation analyses: 1. Correlation of each of the possible drivers with each other 2. Correlation of each of the possible drivers with the outcome variable With more than 70% of all correlation cases being more than 0.6, most of the variables were to a great extent related to each other. Due to the large sample size, all correlations were significant. The different correlation tables are shown below.

Page | 31

OBTAINING INFORMATION

TABLE SHOWING CORRELATIONS: Between explanatory variables 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34

1 1.0 0.4 0.5 0.5 0.5 0.5 0.5 0.5 0.4 0.4 0.5 0.5 0.4 0.4 0.4 0.5 0.5 0.4 0.4 0.5 0.5 0.4 0.6 0.5 0.5 0.4 0.4 0.5 0.4 0.4 0.6 0.5 0.4 0.42 1.0 0.8 0.9 0.8 0.8 0.9 0.8 0.7 0.7 0.4 0.4 0.6 0.7 0.6 0.7 0.7 0.7 0.7 0.5 0.5 0.5 0.5 0.6 0.6 0.7 0.6 0.6 0.6 0.8 0.5 0.7 0.6 0.73 1.0 0.8 0.9 0.8 0.8 0.8 0.7 0.7 0.5 0.5 0.7 0.8 0.7 0.8 0.8 0.8 0.7 0.6 0.6 0.6 0.6 0.6 0.7 0.8 0.7 0.6 0.6 0.7 0.5 0.7 0.7 0.84 1.0 0.8 0.8 0.9 0.8 0.7 0.7 0.4 0.4 0.6 0.8 0.6 0.7 0.7 0.7 0.6 0.5 0.5 0.6 0.6 0.6 0.7 0.7 0.7 0.6 0.6 0.8 0.5 0.7 0.6 0.75 1.0 0.8 0.9 0.9 0.7 0.7 0.5 0.5 0.7 0.8 0.7 0.8 0.8 0.8 0.7 0.6 0.6 0.6 0.6 0.6 0.7 0.8 0.7 0.7 0.6 0.7 0.6 0.7 0.7 0.86 1.0 0.8 0.8 0.7 0.6 0.5 0.4 0.6 0.7 0.6 0.7 0.7 0.6 0.6 0.5 0.5 0.5 0.6 0.6 0.6 0.7 0.6 0.6 0.6 0.7 0.5 0.6 0.6 0.77 1.0 0.9 0.7 0.7 0.5 0.5 0.7 0.8 0.7 0.8 0.8 0.7 0.7 0.6 0.6 0.5 0.6 0.6 0.7 0.7 0.7 0.7 0.6 0.8 0.5 0.7 0.6 0.88 1.0 0.7 0.7 0.5 0.5 0.7 0.7 0.7 0.8 0.7 0.7 0.7 0.6 0.6 0.5 0.6 0.6 0.7 0.7 0.7 0.7 0.6 0.8 0.6 0.7 0.6 0.79 1.0 0.7 0.5 0.5 0.7 0.7 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.7 0.6 0.6 0.6 0.7 0.5 0.6 0.6 0.7

10 1.0 0.5 0.5 0.7 0.7 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.6 0.5 0.5 0.6 0.7 0.6 0.6 0.6 0.7 0.5 0.7 0.7 0.711 1.0 0.9 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.4 0.6 0.5 0.5 0.5 0.5 0.6 0.4 0.4 0.6 0.5 0.5 0.412 1.0 0.5 0.5 0.5 0.5 0.5 0.4 0.5 0.5 0.5 0.4 0.6 0.5 0.5 0.4 0.4 0.5 0.4 0.4 0.5 0.5 0.4 0.413 1.0 0.7 0.8 0.7 0.7 0.8 0.8 0.6 0.6 0.5 0.5 0.6 0.6 0.7 0.6 0.6 0.5 0.6 0.5 0.6 0.6 0.714 1.0 0.7 0.8 0.7 0.7 0.6 0.5 0.6 0.6 0.6 0.6 0.7 0.7 0.6 0.6 0.6 0.7 0.5 0.7 0.6 0.715 1.0 0.7 0.8 0.7 0.8 0.6 0.6 0.6 0.5 0.6 0.6 0.7 0.6 0.6 0.4 0.6 0.6 0.6 0.6 0.716 1.0 0.8 0.7 0.7 0.6 0.6 0.6 0.6 0.6 0.7 0.7 0.7 0.7 0.6 0.7 0.6 0.7 0.6 0.717 1.0 0.8 0.8 0.7 0.7 0.6 0.6 0.6 0.7 0.8 0.7 0.7 0.6 0.7 0.6 0.7 0.7 0.718 1.0 0.8 0.7 0.7 0.6 0.6 0.6 0.7 0.7 0.6 0.6 0.5 0.7 0.6 0.7 0.6 0.719 1.0 0.8 0.7 0.6 0.6 0.6 0.7 0.7 0.6 0.7 0.5 0.6 0.6 0.7 0.7 0.720 1.0 0.8 0.5 0.7 0.6 0.6 0.6 0.5 0.7 0.5 0.6 0.6 0.6 0.6 0.621 1.0 0.5 0.7 0.7 0.7 0.6 0.5 0.7 0.4 0.6 0.7 0.6 0.5 0.622 1.0 0.5 0.5 0.6 0.6 0.5 0.5 0.5 0.6 0.5 0.6 0.6 0.623 1.0 0.9 0.8 0.6 0.6 0.7 0.5 0.6 0.7 0.6 0.5 0.624 1.0 0.9 0.6 0.6 0.7 0.5 0.6 0.6 0.6 0.5 0.625 1.0 0.7 0.6 0.7 0.5 0.7 0.6 0.6 0.6 0.726 1.0 0.7 0.7 0.6 0.7 0.6 0.7 0.7 0.827 1.0 0.7 0.6 0.7 0.6 0.7 0.6 0.728 1.0 0.6 0.7 0.7 0.7 0.6 0.729 1.0 0.7 0.5 0.6 0.6 0.730 1.0 0.6 0.7 0.6 0.831 1.0 0.6 0.5 0.632 1.0 0.7 0.833 1.0 0.834 1.0

0.6 0.7 0.8 ‐ 0.9

Correlation

Page | 32


TABLE SHOWING CORRELATIONS: Between overall measures q7a ... please rate the

overall experience when you obtained

information from it

q12a_1 How satisfied were you with the overall

quality of service delivery?

q12a_2 Overall, how satisfied were you with

this website? q7a ... please rate the overall experience when you obtained information from it

1.00 0.78 0.77

q12a_1 How satisfied were you with the overall quality of service delivery?

1.00 0.93

q12a_2 Overall, how satisfied were you with this website? 1.00


TABLE SHOWING CORRELATIONS: Between with dependent variable

Correlation coefficient1 It was easy to find this website (1) .5

When I got to the site it was easy to find what I was looking for (2) .8

I got the service I wanted after an acceptable number of steps (3) .8

It was easy to find my way around the site (4) .8

I was satisfied with the time it took to do what I wanted to do (5) .8

I always knew where I was on the site (6) .7

The process was straightforward and easy to understand (7) .8

Overall, I was satisfied with the accessibility of the website (8) .8

The search function/engine worked well (9) .8

The site provided features and suggestions that were helpful (10) .8

I feel confident that my privacy was fully protected on this website (11) .5

I feel confident that my information was not shared with anyone else (12) .5

I was able to do everything I needed to do online (13) .7

I understood what to do every step of the way (14) .8

In the end, I got what I needed from the website (15) .7

The amount of time it took to get the overall service was acceptable (16) .8

The service experience met my expectations (17) .8

I was able to do what I wanted when I wanted (18) .8

The site had all the information I needed (19) .8

The information I needed on the site was up‐to‐date (20) .6

The information on the site was accurate (21) .6

There were useful links to other websites (22) .6

The tone of the language on the site was appropriate (23) .6

The language on this site was easy to understand (24) .7

The information on the website was easy to understand (25) .7

I was informed of everything I had to do to get what I wanted (26) .8

I was not overloaded with irrelevant information (27) .7

The information on the site was consistent (28) .7

The site was visually appealing (29) .6

The lay‐out was clear (30) .8

I trust the information on this site (31) .6

It’s an example of good value for tax dollars spent (32) .8

My individual circumstances were taken into account (33) .7

The site was tailored to meet my needs (34) .8

Note1: The value of ‐1.00 represents a perfect negative correlation while a value of +1.00 represents a perfect positive correlation. A

value of 0.00

0.6 0.7 0.8 ‐ 0.9

Correlation

Page | 33

TRANSACTING

TABLE SHOWING CORRELATIONS (between outcome variables) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44

1 1.0 0.6 0.5 0.6 0.5 0.5 0.6 0.6 0.7 0.7 0.6 0.6 0.5 0.4 0.6 0.6 0.5 0.5 0.5 0.6 0.6 0.5 0.5 0.5 0.5 0.6 0.6 0.6 0.6 0.7 0.7 0.6 0.6 0.6 0.5 0.7 0.5 0.6 0.6 0.4 0.5 0.5 0.5 0.52 1.0 0.8 0.9 0.8 0.8 0.8 0.8 0.7 0.7 0.8 0.7 0.7 0.4 0.4 0.5 0.6 0.7 0.7 0.8 0.7 0.6 0.8 0.8 0.8 0.7 0.6 0.6 0.7 0.6 0.7 0.8 0.7 0.7 0.7 0.7 0.6 0.8 0.6 0.4 0.6 0.7 0.6 0.73 1.0 0.8 0.9 0.9 0.8 0.9 0.7 0.8 0.8 0.7 0.7 0.4 0.5 0.5 0.8 0.8 0.8 0.8 0.8 0.7 0.9 0.8 0.8 0.8 0.7 0.7 0.6 0.6 0.7 0.7 0.7 0.8 0.7 0.7 0.6 0.8 0.6 0.4 0.6 0.7 0.7 0.84 1.0 0.8 0.8 0.9 0.9 0.7 0.7 0.9 0.8 0.7 0.4 0.5 0.5 0.6 0.7 0.7 0.8 0.7 0.6 0.8 0.7 0.8 0.7 0.6 0.6 0.7 0.7 0.7 0.8 0.7 0.8 0.7 0.7 0.6 0.8 0.6 0.4 0.6 0.7 0.6 0.75 1.0 0.9 0.8 0.8 0.7 0.8 0.8 0.7 0.7 0.5 0.5 0.5 0.8 0.8 0.8 0.8 0.8 0.7 0.9 0.9 0.8 0.8 0.7 0.7 0.7 0.6 0.7 0.7 0.7 0.8 0.7 0.7 0.6 0.8 0.6 0.4 0.6 0.8 0.6 0.76 1.0 0.8 0.9 0.7 0.7 0.8 0.7 0.7 0.4 0.5 0.5 0.7 0.8 0.8 0.8 0.8 0.7 0.9 0.9 0.8 0.8 0.7 0.6 0.7 0.6 0.7 0.7 0.8 0.8 0.7 0.7 0.6 0.8 0.6 0.4 0.6 0.8 0.6 0.77 1.0 0.8 0.7 0.7 0.8 0.8 0.7 0.4 0.5 0.5 0.6 0.7 0.7 0.8 0.7 0.6 0.7 0.7 0.7 0.7 0.6 0.6 0.7 0.7 0.7 0.8 0.7 0.8 0.7 0.7 0.6 0.8 0.6 0.4 0.6 0.7 0.6 0.78 1.0 0.8 0.8 0.8 0.7 0.8 0.4 0.5 0.5 0.7 0.7 0.7 0.8 0.8 0.7 0.8 0.8 0.8 0.7 0.7 0.6 0.7 0.7 0.7 0.8 0.8 0.8 0.7 0.7 0.6 0.8 0.6 0.4 0.6 0.7 0.6 0.79 1.0 0.9 0.8 0.7 0.7 0.5 0.6 0.6 0.6 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.8 0.7 0.7 0.6 0.7 0.8 0.8 0.7 0.8 0.7 0.8 0.6 0.7 0.6 0.4 0.6 0.6 0.6 0.7

10 1.0 0.8 0.7 0.7 0.5 0.6 0.6 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.6 0.7 0.7 0.7 0.7 0.8 0.7 0.8 0.6 0.7 0.7 0.4 0.6 0.7 0.6 0.711 1.0 0.8 0.7 0.4 0.6 0.5 0.7 0.7 0.7 0.7 0.8 0.7 0.8 0.8 0.8 0.7 0.7 0.7 0.7 0.7 0.7 0.8 0.8 0.7 0.7 0.8 0.6 0.8 0.6 0.4 0.7 0.7 0.7 0.712 1.0 0.7 0.4 0.6 0.6 0.7 0.7 0.7 0.8 0.7 0.6 0.7 0.7 0.7 0.7 0.7 0.6 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.8 0.7 0.5 0.7 0.7 0.7 0.713 1.0 0.4 0.6 0.6 0.7 0.7 0.7 0.8 0.7 0.6 0.7 0.7 0.7 0.7 0.7 0.7 0.8 0.7 0.7 0.7 0.7 0.7 0.6 0.7 0.7 0.7 0.6 0.6 0.7 0.7 0.7 0.714 1.0 0.7 0.6 0.5 0.5 0.4 0.5 0.4 0.4 0.4 0.4 0.4 0.4 0.5 0.5 0.4 0.4 0.5 0.4 0.5 0.5 0.4 0.5 0.4 0.4 0.4 0.3 0.4 0.5 0.4 0.415 1.0 0.9 0.6 0.6 0.5 0.6 0.6 0.5 0.5 0.5 0.5 0.5 0.6 0.6 0.5 0.6 0.6 0.6 0.6 0.5 0.5 0.7 0.5 0.5 0.6 0.4 0.5 0.6 0.5 0.516 1.0 0.6 0.6 0.5 0.6 0.6 0.5 0.5 0.5 0.5 0.5 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.5 0.5 0.6 0.5 0.5 0.6 0.4 0.5 0.6 0.5 0.517 1.0 0.8 0.8 0.7 0.8 0.7 0.8 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.7 0.7 0.7 0.7 0.6 0.7 0.5 0.6 0.7 0.4 0.7 0.7 0.6 0.718 1.0 0.7 0.7 0.8 0.7 0.8 0.8 0.8 0.7 0.7 0.7 0.7 0.7 0.6 0.7 0.7 0.7 0.6 0.7 0.6 0.7 0.6 0.5 0.7 0.7 0.6 0.719 1.0 0.8 0.7 0.8 0.7 0.8 0.8 0.8 0.7 0.6 0.6 0.6 0.6 0.7 0.7 0.7 0.6 0.7 0.5 0.6 0.6 0.4 0.6 0.7 0.6 0.720 1.0 0.8 0.7 0.7 0.8 0.8 0.8 0.7 0.7 0.7 0.7 0.8 0.8 0.8 0.8 0.7 0.7 0.6 0.8 0.7 0.4 0.7 0.7 0.6 0.721 1.0 0.8 0.8 0.8 0.8 0.8 0.7 0.7 0.7 0.7 0.7 0.8 0.8 0.7 0.7 0.8 0.6 0.7 0.7 0.5 0.7 0.7 0.7 0.722 1.0 0.7 0.8 0.8 0.8 0.7 0.7 0.6 0.6 0.6 0.7 0.7 0.7 0.6 0.7 0.5 0.6 0.6 0.4 0.6 0.7 0.6 0.723 1.0 0.9 0.8 0.8 0.7 0.7 0.7 0.6 0.7 0.7 0.7 0.7 0.7 0.7 0.6 0.7 0.6 0.4 0.6 0.7 0.7 0.724 1.0 0.9 0.8 0.7 0.7 0.7 0.7 0.7 0.8 0.8 0.7 0.7 0.7 0.6 0.8 0.7 0.4 0.7 0.8 0.7 0.825 1.0 0.8 0.7 0.7 0.7 0.7 0.7 0.8 0.8 0.8 0.7 0.7 0.6 0.7 0.7 0.5 0.7 0.7 0.6 0.726 1.0 0.8 0.8 0.7 0.7 0.7 0.7 0.8 0.8 0.7 0.7 0.5 0.7 0.7 0.5 0.6 0.7 0.7 0.727 1.0 0.8 0.7 0.7 0.7 0.7 0.7 0.7 0.6 0.8 0.5 0.7 0.6 0.4 0.6 0.7 0.6 0.628 1.0 0.6 0.7 0.7 0.7 0.7 0.7 0.6 0.8 0.5 0.6 0.7 0.4 0.5 0.6 0.6 0.629 1.0 0.6 0.6 0.7 0.7 0.6 0.6 0.7 0.7 0.7 0.6 0.6 0.6 0.7 0.7 0.730 1.0 0.9 0.8 0.7 0.7 0.7 0.8 0.6 0.7 0.7 0.4 0.6 0.7 0.6 0.631 1.0 0.9 0.8 0.7 0.7 0.8 0.6 0.7 0.7 0.4 0.6 0.7 0.6 0.632 1.0 0.8 0.8 0.7 0.8 0.6 0.8 0.6 0.4 0.6 0.7 0.6 0.733 1.0 0.8 0.7 0.8 0.6 0.8 0.7 0.4 0.7 0.7 0.7 0.734 1.0 0.8 0.8 0.6 0.7 0.6 0.4 0.6 0.7 0.6 0.735 1.0 0.8 0.7 0.7 0.6 0.4 0.6 0.7 0.6 0.736 1.0 0.6 0.7 0.7 0.4 0.7 0.7 0.6 0.737 1.0 0.7 0.6 0.5 0.6 0.7 0.6 0.738 1.0 0.7 0.5 0.7 0.8 0.7 0.839 1.0 0.5 0.7 0.7 0.6 0.740 1.0 0.6 0.5 0.6 0.541 1.0 0.7 0.7 0.742 1.0 0.7 0.843 1.0 0.844 1.0

0.6 0.7 0.8 ‐ 0.9

Correlation

Page | 34

TRANSACTING

TABLE SHOWING CORRELATIONS: Between overall measures

...please rate the overall

experience when you carried out a transaction

How satisfied were you with the overall quality of

service delivery?

Overall, how satisfied were you with this

website? ...please rate the overall experience when you carried out a transaction

1.00 0.82 0.78

How satisfied were you with the overall quality of service delivery?

1.00 0.88

Overall, how satisfied were you with this website?

1.00

TRANSACTING

TABLE SHOWING CORRELATIONS (independent variables with ‘Satisfaction with this website’) Correlation coefficient

1

It was easy to find this website (1) .5

When I got to the site it was easy to find what I was looking for (2) .8

I got the service I wanted after an acceptable number of steps (3) .8

It was easy to find your way around the site (4) .8

I was satisfied with the time it took to complete the transaction (5) .8

I was satisfied with the time it took to do what I wanted to do (6) .8

I always knew where I was on the site (7) .8

The process was straightforward and easy to understand (8) .8

The amount of information I had to provide was acceptable (9) .7

The type of information I had to provide was acceptable (10) .7

Overall, I was satisfied with the accessibility of the website (11) .8

The search function/engine worked well (12) .8

The site provided features and suggestions that were helpful (13) .8

I feel comfortable sending my credit card number over the Internet to pay for a government service (14) .5

I feel confident that my privacy was fully protected on this website (15) .6

I feel confident that my information was not shared with anyoneelse (16) .6

I received confirmation in an acceptable time that I had completedthe transaction (17) .7

I knew when the service would be delivered (18) .8

I was able to do everything I needed to do online (19) .7

I understood what to do every step of the way (20) .8

I understood what would happen next (21) .8

In the end, I got what I needed from the website (22) .7

The amount of time it took to get the overall service was acceptable (23) .8

The service experience met your expectations (24) .8

I was able to do what I wanted when I wanted (25) .8

The site had all the information I needed (26) .8

The information I needed on the site was up‐to‐date (27) .7

The information on the site was accurate (28) .7

There were useful links to other websites (29) .7

The tone of the language on the site was appropriate (30) .7

The language on this site was easy to understand (31) .7

The information on the website was easy to understand (32) .8

I was informed of everything I had to do to get what I wanted (33) .8

Forms were easy to understand and fill out (34) .7

I was not overloaded with irrelevant information (35) .7

The information on the site was consistent (36) .8

The site was visually appealing (37) .7

The lay‐out was clear (38) .8

I was able to review the information I was providing before submitting it (39) .6

I was able to see all my previous transactions (40) .5

I would get fast support if I forgot my password (41) .7

It’s an example of good value for tax dollars spent (42) .8

My individual circumstances were taken into account (43) .7

The site was tailored to meet my needs (44) .8

Note1: The value of ‐1.00 represents a perfect negative correlation while a value of +1.00 represents a perfect positive correlation. A value of 0.00

0.6 0.7 0.8 ‐ 0.9

Correlation

Page | 35

B.3.3 Multiple Linear Regression To conduct driver analysis, “Overall satisfaction with the website” was used as the dependent variable which was explained using multiple independent variables. The scale for both the dependent and independent variables ranged from 0 to 10 points. The data used for the driver analysis excluded responses that had ‘Not Applicable’ and ‘Unsure’ for any of the ratings. This was not a problem as the percentages of such responses were not very high for the majority of the potential drivers. The resulting sample size for the driver analysis consisted of all the observations that had a rating of 0‐10 across all attributes appearing in the model. The two main objectives of interest for a good driver model were the overall fit of the model and the relative importance. R2 is the total variance in the dependent variable that can be explained by the independent variables in the model. R2 values range from 0 to 1, with higher values indicating better ability to predict (or explain) the dependent variable. R2 can be used as an indicator of the reliability of a relationship identified by regression analysis. An R2 of 0.8 indicates that 80% of the change in the dependent variable is explained by a change in the explanatory variables in the model. Relative importance of the independent variables is expressed in the form of coefficients or beta weights. A weight of 0.4 associated with a variable indicates that a unit change in that variable can lead to a 0.4 unit change in the dependent variable. Thus, beta weights are used to identify the variables that have the most impact on the dependent variable. The steps followed for the driver analysis were as follows: Initially, two different methods of regression analysis were used with all the explanatory variables ‐ stepwise forwards and stepwise backwards. All the variables that appeared from each of these methods were put into a forced entry regression model. The significance of the drivers was reviewed and any insignificant variable was dropped from the model. Several iterations were run and the final model was selected based on the optimum R square and the no. of drivers. Forced regression: All independent variables (inputs) are used in the equation to predict the dependent variable (output), whether or not the independent variables have a statistically significant impact. Forward regression: Independent variables (inputs) are added to the equation to predict the dependent variable (output) one at a time, starting with the one that makes the most significant impact, then adding the next most significant and so on until there are no more variables that make a significant difference. Backward regression: All independent variables (inputs) are put in to the equation to predict the dependent variable (output). The variable that makes the least significant impact is removed and then the next least significant and so on until there are no more non‐significant variables.

Drivers of Satisfaction for Online Public Services

Documents

Transcript of Drivers of Satisfaction for Online Public Services