WOW, I Did Not Know I Could Do That!Tricks for configuring and
analyzing SmarterMeasure Data
Cynthia Pascal, Northern Virginia Community CollegeLou Ann Martin, Tri-County Technical CollegeCourtney Tillett, Tri-County Technical CollegeDr. Mac Adkins, President, SmarterServices
All attendee lines are muted. Please submit questions via chat. Q & A at the end. The session is being recorded and an archive
will be available.
Housekeeping
Introduction – Making Sense of your Data Northern Virginia Community College Tri-County Technical College Five C’s to Maximize SmarterMeasure
Agenda
Level One – Students learn about themselves• Aha moment• The tiger looking in the mirror
Level Two – Schools learn how to best support students • Integration into orientation courses, first year experience
programs• Recommendation of appropriate remedial / support resourcesLevel Three – Leaders learn how to maximize data• Computing retention gains, comparing populations• Fulfilling accreditation requirements, justifying costs
Levels of Benefit
Level One – Students learn about themselves• Aha moment• The tiger looking in the mirror
Level Two – Schools learn how to best support students • Integration into orientation courses, first year experience
programs• Recommendation of appropriate remedial / support resourcesLevel Three – Leaders learn how to maximize data• Computing retention gains, comparing populations• Fulfilling accreditation requirements, justifying costs
Levels of Benefit
Which level would you say best describes the highest level of how your school is using SmarterMeasue?
Level One – Using it as a student serviceLevel Two – Integrated into your software and processesLevel Three – Conducting statistical analysis of your data
Poll
Progression of SmarterMeasure Data Utilization
Predictive
Correlation
Comparison
Descriptive
Student Service Metrics
Approaches to Research Projects
Internally
Conducted
Company
Assisted
Professionall
y Assist
ed
Cynthia Pascal Coordinator of Academic Advising & Retention
Northern Virginia Community College
Northern Virginia Community College
Northern Virginia Community College’s Extended Learning Institute
Current Use of
SmarterMeasure
• Website• Orientation• Welcome
Messages• Student Planners• Embedded in Bb• Freshman
Seminar/ College Skills Assignment
• Advising partner colleges of students’ “goodness of fit”
Northern Virginia Community College
Tool to assess incoming students
Northern Virginia Community College’s Extended Learning
Institute
Northern Virginia Community College
Northern Virginia Community College’s Extended Learning Institute
Data used to have conversations with students
who are on academic probation or taking a class
for the third time.
Data used to have conversations with
students who are on academic probation or
repeating a class for the third time.
Northern Virginia Community College
Northern Virginia Community College’s Extended Learning Institute
Lou Ann Martin – Title III Activity Director for the Academic Support Network
Courtney Tillett – Information Tech-instruction-applications Analyst
Tri-County Technical College
Current use Completed voluntarily by incoming students
Completion encouraged through acceptance letter Results captured in Banner Results reviewed briefly with students by advisors
Students enrolled in Freshman Seminar/College Skills classes use results as springboard for discussions and journal entries
Goal Would like to be able to identify students at-risk due to
non-academic issues Initial research found no specific correlations Tried different approach after consulting with
SmarterMeasure Preliminary results are promising and suggest the need
to tweak our readiness ranges
SmarterMeasure Results at TCTC
Acceptance Letter
Results in Banner
Readiness Ranges Analysis
Default Cut
Points
N
p-value
F
Adjusted Cut Points
p-
value
F
MeanGPA
PersonalAttribute
s
70 / 85 1,457 0.2723 1.30 80 /90 0.2327
1.46 Fail: 2.15Question: 2.30Pass: 2.34
LifeFactors
70 / 85 1,482 0.2453 1.41 80 / 90 0.1217
2.11 Fail: 2.21Question: 2.24Pass: 2.56
Reading Recall
30 / 65 1,380 0.1237 2.09 70 / 90 0.0002
8.52 Fail: 2.21 Question: 2.24Pass: 2.56
Technical
Competency
80 / 90 1,362 0.0011 6.82 90 / 100 0.0367
3.31 Fail: 1.84Question: 2.26Pass: 2.34
Technical
Knowledge
50 / 75 1,352 0.0039 5.56 70 / 80 0.0057
5.18 Fail: 2.03Question: 2.26Pass: 2.45
TypingRate
21 / 31 1,346 0.0303 3.51 25 / 40 0.1226
2.10 Fail: 2.17Question: 2.36Pass: 2.36
Notes: Based on “Making SmarterMeasure a Strong Predictor of
Academic Success” Used ANOVA to determine the strength of the relationships
between the readiness ranges and first-semester GPA 6,931 FTPS students from the fall and spring semesters of the last
three academic years (2011-2012 – 2013-2014) 1,593 matched to SmarterMeasure results Any test taken after the first day of class for each semester was
excluded from the analyses
Readiness Ranges Analysis
How can the ranges for Personal Attributes and Life Factors be adjusted? Would any of the individual subscales in either
section be more predictive? What is the practical significance of the
findings?
Questions Raised
Configure the assessment engine Control for self-selection bias Consider adding your own questions Classify using your own demographic factors Create a sanitized data set
Five C’s to Maximize SmarterMeasure
Your Account Manager is eager to train you on any of these configuration options.
Configure the Assessment Engine
Julie Owen Tara McLaughlin
Login Tutorial Video
Add Google Analytics
Integrate into LMS or SIS
Require student email
Turn on/off demographics
School Level
More demographics
Adjust time zone
Custom Remedial Resources
Custom Remedial Text
Require student email
Renewal alert
School Level
Integrate into third party software such as an LMS, SIS or Advising suite.
Single Sign On
Automated data transfer
Gradebook integration
School Level
Re-order sections
Skip sections
Add Math and Writing
Adjust Readiness Ranges
Select Versions
Group Level
Change scoring type – 4 point or percentage
Assessment Mode – Sequential or Disaggregate
Show National Averages
Results options – Report, Custom Text or URL
Group Level
Custom Interface
Text
Select each of the ways that your school has configured SmarterMeasure.Re-order or delete sections LMS integrationAdding custom questionsCustom introductory or score report textAdjusting Readiness Ranges
Poll
Control for Self-Selection Bias
Consider Adding Your Own Questions
Classify Using Your Own Demographic Factors
Create a Sanitized Data Set
Based on the webinar today how likely is your school to enhance the way it uses SmarterMeasure?Very likely – I learned some actionable ideas.Likely – This has me thinking.Not likely – This is interesting but there is no need to change what we are doing.
Poll
Please Give Us Your Feedback
You will receive a post-webinar email with a link to the survey.
Top Related