The CMT in an Electronic Environment

The CMT in an Electronic Environment

Guy GordonA/ADM

Service ManitobaCMT Day Alberta Federal Council

Feb. 12 2004

The CMT in an Electronic Environment

Overview of Presentation

• Role of Service Manitoba re: CMT• Electronic Surveying• Challenges and Issues

The CMT in an Electronic Environment 1. Role of Service Manitoba

• Provide assistance with survey design, administration, and analysis

• Build and deploy on-line surveys using Inquisite (I.e. direct email, static link, and random pop-up)

• Run reports and deliver results to clients• Serve as a quality control point &

Coordination point (ManWeb, CSU,CSM, FLS)

• Develop capacity and work to roll use of Inquiste tool out to departments

The CMT in an Electronic Environment

How Manitoba Has Used the CMT

• Manitoba has used the CMT survey extensively with many different organizations.

• Surveys have been administered over the phone, mail, and on-line.

• How you administer the survey depends on the population you are surveying.

• The current trend has been to conduct surveys on-line as a static link on a Website, a pop-up box invite, or by email.

The CMT in an Electronic Environment

• Government of Manitoba’s Website• STEP services• Career Start• Stevenson Aviation Services• Agriculture & Food• Office of the Auditor General• Family Services and Housing (4)• Bilingual Service Centres• MB4Youth• Student Aid• Government Services (Internal)• Conservation

Organizations That Have Used the CMT in Manitoba

The CMT in an Electronic Environment

The CMT in an Electronic EnvironmentHow the Common Measurement Toolis promoted and used in MB.

• CMT continues to be voluntary.• Service Manitoba promotes CMT as a

proven “industry standard”. • Service Manitoba will assist with

design, administration and reporting.• Strongly recommend use of “Core

Questions”• Enhanced CMT rolled out in January• CMT now being used to evaluate

internal service delivery (e.g. Gov’t Services)

The CMT in an Electronic Environment

2. Electronic Surveying

• The demand for surveys across the Manitoba Public Service was increasing.

• Each year Service Manitoba was seeing an increase in survey requests.

• In order to meet this demand, we needed a tool that would allow us to turn around surveys quickly from administration, to data entry to analysis.

• We determined that conducting surveys electronically met all of these requirements.

Background...

The CMT in an Electronic Environment

• First started by building our own survey software in-house.

• At this time, electronic and on-line surveying was just beginning to emerge.

• We wanted to create a program that was flexible and could meet the needs of different organizations, that was easy to use, and that could conduct basic data analysis.

• We did succeed in creating a software program that could assist us in conducing electronic surveys but...

The CMT in an Electronic Environment

• A lot of time was invested in the process.• A lot of resources were invested in the

project.• Technical issues - the program was not

compatible with all end-user’s desktop.• Technical Support was needed to assist end-

users but there were no resources provided.• The software program was great for simple,

short surveys, but more flexibility was required for customized more comprehensive surveys.

The CMT in an Electronic Environment

Software Evaluation

• There were many valuable lessons learned from our software building experience.

• We knew what was required to meet all of the different survey needs of the Manitoba Public Service:

- Flexibility - Language - Bilingual - Look and feel - Ability to administer in numerous

ways - Technical requirements and support - Ease of use

The CMT in an Electronic Environment

• In 2001, Service Manitoba decided to go with an off-the-shelf survey software package.

• We evaluated 5 different packages to determine which would best meet our survey needs.

• As a result of our previous experience, the software would need to offer:

- Ability to customize design of questionnaires

- Ability to deploy unlimited number of surveys simultaneously

- Sophisticated data analysis - Report generation capabilities

The CMT in an Electronic Environment - Proven stability

- Ability to imbed survey in existing on-line applications

- Network and/or enterprise versions

• As a result of our evaluation, we decided to purchase Inquisite.

The CMT in an Electronic Environment

Inquisitewww.inquisite.com

• Since the purchase of Inquisite, we have seen overwhelming benefits in speed, flexibility, and cost.

• Flexibility - we administer any type of survey quickly and easily (CMT, employee).

• Deploy numerous surveys for different organizations at the same time.

• Surveys administered in many ways - in email to known recipients, imbedded on Website, pop-up box on Website.

The CMT in an Electronic Environment

• Data analysis - can give you real time results and publish results online, exports to SPSS and Excel for more sophisticated data analysis.

• Easy to use - very intuitive for MS Office users.

• Responsive and competent technical support.• Security of data - Data resides on Manitoba’s

servers, ensures that data collected is safe and secure.

The CMT in an Electronic Environment

The CMT in an Electronic Environment

The CMT in an Electronic Environment

The CMT in an Electronic Environment

The CMT in an Electronic Environment

The CMT in an Electronic Environment

The CMT in an Electronic Environment

Practical Issues to Think About• Cost: cost of 3 users can run approximately

$13,000 Canadian, plus $3,000 for technical support.

• Requires assistance of IT staff to install: IT staff were vital to help with configuration, deployment, and maintenance. This does require some time and resources when first using the software.

• Licensing: restricted licensing can be a hindrance as the use of surveys increase. In Manitoba, this has imposed some strains on staff as popularity of electronic surveys increases.

The CMT in an Electronic Environment

3. Challenges and Lessons Learned

• Take up of electronic surveying has been very successful. Every month we are getting an increase in requests.

• For many organizations, this has been their first time collecting any type of data.

• Turnaround time has improved - can usually run a survey and have report ready for management in a month and a half.

• Has proven to be a very cost effective solution for Manitoba - during the first year we administered over 20 surveys.

What is working well...

The CMT in an Electronic Environment

• The demand for this service has exceeded the resources available.

• Electronic administration has increased client expectations, staff resources have not increased.

• Important to clarify role of everyone involved at beginning of the project - who is responsible for what, what are the expectations, what will be provided.

• Need to try different ways of administrating online surveys - pop-up, static link, etc.

• Difficult to determine response rates.

Some lessons learned...

The CMT in an Electronic Environment

4. Next Steps

• Working with Departments to support and encourage the use of the CMT.

• Build organizational capacity by providing training opportunities for Department staff and assist with Inquiste licensing.

• Assist in coordinating online service delivery.• ICCS has guide to online surveying on their Website:

http://www.iccs-isac.org/eng/WebSurveyIntro.htm

• Working with other jurisdictions in exploring the link between employee and client satisfaction.

The CMT in an Electronic Environment

Questions??