OIRE Systems Projects

Department Heads Meeting, 3/11/2013

SURESH NAIR, Ph.D.Interim Associate Vice Provost for Institutional EffectivenessProfessor, School of BusinessUniversity of Connecticut, Storrs

1

Overview Online Student Evaluation of Teaching

(SET) HuskyDM enhancements HuskyAL – Assessment of Learning OIRE Self Service DataMart

2

Online SET – Student Evaluation of Teaching The new 5 point form required by the

Senate is being implemented It will be an online system Students in courses ending after March

22 will receive online SET links by e-mail.

3

4

History of the SET We have been using a paper-based 10-point SET

Average of ratings is reported In Fall 2010, a new 5-point SET was approved by

Senate after trials—strong reliability and validity http://senate.uconn.edu/evaluations/SEoT.guidelines.pdf Medians of ratings are to be reported

eXplorance Blue was chosen as the vendor for the new SET—the goal was to use paper SET’s with a quick turn-around and electronic reporting

A pilot was run in Fall 2012 on a subset of courses scored using both the 10-point and 5-point SET

5

Problems with Paper SET Multiple Problems with Paper SETs scoring

Printing of paper forms is a logistical problem due to manual nature of process

Resolution of scanning is on new form poor, requiring constant redos. New scanning software yields 10% uncertainty of student responses requiring manual intervention and “guessing”

Possible inaccuracy when instructor changes after 10th day of class and paper SETs already printed with old name.

Only one vendor was available in RFP process, since almost every major university in the country schools has moved to completely online systems

Online SET – Student Interface6

Online SET – Instructor Report7

Online SET – Instructor Report (contd.)8

9

The path forward Rollout of the 5-point fully online scoring and

reporting system planned for Spring 2013 (opt outs available)

Student scoring Evaluation and comments to be entered online Students may use computers, tablets or smart

phones Strategies to improve student response rate

Push green/sustainability angle Use promotional videos to promote online

scoring Early access to grades (say, two days before

others)

10

The path forward (contd.) Instructor and DH reporting

Will be able to see reports online much sooner than the paper system

Instructors will be able to see comments online Research has shown that the quantity and

quality of text comments online is much better than comments on paper

Opt-out: Faculty may request paper scoring after obtaining consent from DH

HuskyDM status update The Provost Annual Report (PAR) for last

year was created by HuskyDM ~95% faculty entered their information We have noted all the feedback and are

making improvements this year – discussed next

Additionally, HuskyDM can this year be used for Creating PTR form drafts Merit forms

11

Publications upload

Demo

12

HuskyDM enhancements (contd.) Some of the most common complaints were

“data not appearing in proper PAR item cell” Reason – faculty may not choose to enter all the

fields required by the logic for creating PAR Resolution – we are redoing most screens to now

directly ask for the PAR# via a drop-down menu

13

HuskyDM enhancements (contd.)

Similar to Publications Upload, we are planning to upload the following information directly into HuskyDM this year (this work is not complete) Grants and Contracts HR data Advisee data

PTR forms via HuskyDM This has been tested with ~20 TT faculty

this year This is a convenience available to TT

faculty, not a requirement

14

PAR Redesign

The Provost Annual Report (PAR) has remained unchanged for about 20 years.

A committee of DHs and users is currently looking at making it less confusing and more useful

15

HuskyDM enhancements planned (contd.) Merit forms via HuskyDM

We have contacted each school to collect their Merit forms

BUS has been using HuskyDM for merit for 5 years now

Again, a convenience, not a requirement

16

HuskyAL – Assessment of Learning

We are working on an online tool to replace OATS

DHs and Program Directors are responsible to continue to Assessing their programs All programs should have Mission and

Purpose, Goals, Objectives/Outcomes and Methods, as before

All programs should track results , and specify follow-up actions, as before

The only difference is, instead of posting into OATS manually, you will upload into HuskyLM using Excel.

17

OIRE DataMart We are working on a self service DataMart for

OIRE data Departments and Schools should be able to

access routine data using a WebFocus online interface.

Will have some functionality to manipulate data somewhat like Pivot tables in Excel

18

Contact Information Suresh Nair, Interim Associate Vice

Provost for Institutional Effectiveness, Suresh.Nair@uconn.edu

HuskyDM Alexander del Campo, School of Business,

IT Services, Alexander.delCampo@uconn.edu

Liming Liu, OIR, Liming.Liu@uconn.edu Online SET

Cheryl Williams, OIR, cheryl.williams@uconn.edu Valorie Elwell, UITS, valorie.elwell@uconn.edu Georgianne Copley, UITS,

georgianne.copley@uconn.edu

19

20

Comparison between 10 and 5 point scales In Fall 2012, 39 courses were scored on

both the 10-point and 5-point scales The 5-point median scale actually

resulted in higher SETs than the old 10-point scale (9% higher)Average of

10-point scale

Twice median of 5-point scale

Count of courses

5-6 73% 5%6-7 118% 5%7-8 103% 13%8-9 112% 77%Total 109% 100%