General Education Assessment Results & Analyses 2019-2020 ...€¦ · General Education Committee...
Transcript of General Education Assessment Results & Analyses 2019-2020 ...€¦ · General Education Committee...
![Page 1: General Education Assessment Results & Analyses 2019-2020 ...€¦ · General Education Committee ... and greater life-experience has led Senior students to assimilate lessons similar](https://reader033.fdocuments.in/reader033/viewer/2022060605/605a9792c2493612004703fb/html5/thumbnails/1.jpg)
General Education Assessment
Results & Analyses
2019-2020 Academic Year
![Page 2: General Education Assessment Results & Analyses 2019-2020 ...€¦ · General Education Committee ... and greater life-experience has led Senior students to assimilate lessons similar](https://reader033.fdocuments.in/reader033/viewer/2022060605/605a9792c2493612004703fb/html5/thumbnails/2.jpg)
Be seen. Be heard. BE INSPIRED.
General Education Committee (2019–2020 AY)College of Arts & Sciences:
Paul Lepp (20), ChairJean-Francois Mondon (22)Bishnu Sedai (21)Scott Kast (20)College of Business:
Lori Willoughby (21)
College of Education & Health Sciences:
Pamela Ondracek (22)
Student Representative:
Brett Hlebechuk (20)
Director of Academic Assessment:
Michael Brooks (life-time member; like the mob)
![Page 3: General Education Assessment Results & Analyses 2019-2020 ...€¦ · General Education Committee ... and greater life-experience has led Senior students to assimilate lessons similar](https://reader033.fdocuments.in/reader033/viewer/2022060605/605a9792c2493612004703fb/html5/thumbnails/3.jpg)
Be seen. Be heard. BE INSPIRED.
Gen Ed Changes and Updates• Added Course removal application form (https://www.minotstateu.edu/ge/GE-FacultyInfo.shtml)
• Updated CCS 4 (Quantitative Literacy) and CCS 5 (Oral and Written Communication) definitions – these were duplicate definitions.
![Page 4: General Education Assessment Results & Analyses 2019-2020 ...€¦ · General Education Committee ... and greater life-experience has led Senior students to assimilate lessons similar](https://reader033.fdocuments.in/reader033/viewer/2022060605/605a9792c2493612004703fb/html5/thumbnails/4.jpg)
• Data from each Development Content area collected every third semester.• Added data to all categories except CCS3, CCS 6 and PSR 3, which will not be reassessed until the
end of spring 2020.
• Construed as a snap shot of the effectiveness of the Gen Ed program.
• Data reported as means ± SD and significant difference between classes (Ie, Freshman v. Seniors) reported. • Results very consistent between reports.
• Assumed that each student will be exposed to each content area multiple times during the course of their career.
Be seen. Be heard. BE INSPIRED.
General Education – Data Analyses in the recent past
![Page 5: General Education Assessment Results & Analyses 2019-2020 ...€¦ · General Education Committee ... and greater life-experience has led Senior students to assimilate lessons similar](https://reader033.fdocuments.in/reader033/viewer/2022060605/605a9792c2493612004703fb/html5/thumbnails/5.jpg)
Be seen. Be heard. BE INSPIRED.
Most Students receive only a single exposure to a content area
![Page 6: General Education Assessment Results & Analyses 2019-2020 ...€¦ · General Education Committee ... and greater life-experience has led Senior students to assimilate lessons similar](https://reader033.fdocuments.in/reader033/viewer/2022060605/605a9792c2493612004703fb/html5/thumbnails/6.jpg)
Be seen. Be heard. BE INSPIRED.
• Data consists of two populations: those exposed once the content area and those exposed two or more times
• This confounds analyses based upon class status (I.e., freshman v. seniors)
• Collecting longitudinal data would allow us to distinguish between student with multiple exposure to a content area and those without.
![Page 7: General Education Assessment Results & Analyses 2019-2020 ...€¦ · General Education Committee ... and greater life-experience has led Senior students to assimilate lessons similar](https://reader033.fdocuments.in/reader033/viewer/2022060605/605a9792c2493612004703fb/html5/thumbnails/7.jpg)
Be seen. Be heard. BE INSPIRED.
Ways to use Data
• Quality control check• Arbitrarily define ”success” level and attempt to improve it. I.e., 80% of seniors should score a 3 or above
• Hypothesis Testing• Statistical tests are tests of hypotheses
Rubric Ratings
1 – Insufficient2 – Basic3 – Sufficient4 – Advanced
![Page 8: General Education Assessment Results & Analyses 2019-2020 ...€¦ · General Education Committee ... and greater life-experience has led Senior students to assimilate lessons similar](https://reader033.fdocuments.in/reader033/viewer/2022060605/605a9792c2493612004703fb/html5/thumbnails/8.jpg)
Be seen. Be heard. BE INSPIRED.
𝛘2 analysis of CCS 2.1CCS 2.1 – Information Literacy; Determine the nature and extent of information needed
0
20
40
60
80
100
120
1 2 3 4
Observed & Expected Distributions of Freshman Scores
Observed Expected
P-value = 0.00036• 25 of 32 subcategories exhibited this same pattern.
• Freshman scores were significantly lower than expected.
• Comparison based on model of data distribution.
• Does the model reflect reality? Pre and post tests would be better.
n = 269
![Page 9: General Education Assessment Results & Analyses 2019-2020 ...€¦ · General Education Committee ... and greater life-experience has led Senior students to assimilate lessons similar](https://reader033.fdocuments.in/reader033/viewer/2022060605/605a9792c2493612004703fb/html5/thumbnails/9.jpg)
Be seen. Be heard. BE INSPIRED.
𝛘2 analysis of CCS 2.1CCS 2.1 – Information Literacy; Determine the nature and extent of information needed
• Null hypothesis: The intervention/class did not change freshmen, sophomores, juniors and seniors ability to determine the nature and extent of the information needed because the intervention/class was ineffective. -falsified
• Alternative hypothesis #1: The intervention/class increased freshmen, sophomores, juniors and seniors ability to determine the nature and extent of the information needed because the the intervention/class was effective. - falsified
• Alternative hypothesis #2: Freshman, sophomores, juniors and seniors scores were below expectation because they failed to determine the nature and extent of the information needed. - supported
n = 269
0
20
40
60
80
100
120
1 2 3 4
Observed & Expected Distributions of Freshman Scores
Observed Expected
P-value = 0.00036
![Page 10: General Education Assessment Results & Analyses 2019-2020 ...€¦ · General Education Committee ... and greater life-experience has led Senior students to assimilate lessons similar](https://reader033.fdocuments.in/reader033/viewer/2022060605/605a9792c2493612004703fb/html5/thumbnails/10.jpg)
Be seen. Be heard. BE INSPIRED.
𝛘2 analysis of CCS 2.1CCS 2.1 – Information Literacy; Determine the nature and extent of information needed
0
50
100
150
200
1 2 3 4
Observed & Expected Distributions of Sophomores's scores
Sophomore Observed Sophomore Expected
P-value = 0.0065
0
10
20
30
40
50
60
70
1 2 3 4
Observed & Expected Distributions of Junior's scores
Junior Observed Junior Expected
P-value = 0.054
• Null hypothesis: The intervention/class did not change freshmen, sophomores, juniors and seniors ability to determine the nature and extent of the information needed relative to peers. - falsified
• Alternative hypothesis #1: The intervention/class increased freshmen, sophomores, juniors and seniors ability to determine the nature and extent of the information needed relative to peers. - falsified
• Alternative hypothesis #2: Freshman, sophomores, juniors and seniors scores were below expectation because they failed to determine the nature and extent of the information needed relative to peers. -supported
• Not enough statistical power
n = 296
n = 163
![Page 11: General Education Assessment Results & Analyses 2019-2020 ...€¦ · General Education Committee ... and greater life-experience has led Senior students to assimilate lessons similar](https://reader033.fdocuments.in/reader033/viewer/2022060605/605a9792c2493612004703fb/html5/thumbnails/11.jpg)
Be seen. Be heard. BE INSPIRED.
𝛘2 analysis of CCS 2.1CCS 2.1 – Information Literacy; Determine the nature and extent of information needed
• Null hypothesis: The intervention/class did not change freshmen, sophomores, juniors and seniors ability to determine the nature and extent of the information needed relative to peers. - falsified
• Alternative hypothesis #1: The intervention/class increased freshmen, sophomores, juniors and seniors ability to determine the nature and extent of the information relative to peers. - supported
• Alternative hypothesis #2: Freshman, sophomores, juniors and seniors scores were below expectation relative to peers. - falsified
n = 136
P-value =1.8e-7
![Page 12: General Education Assessment Results & Analyses 2019-2020 ...€¦ · General Education Committee ... and greater life-experience has led Senior students to assimilate lessons similar](https://reader033.fdocuments.in/reader033/viewer/2022060605/605a9792c2493612004703fb/html5/thumbnails/12.jpg)
Be seen. Be heard. BE INSPIRED.
Mann-Whitney analysis of CCS 2.1CCS 2.1 – Information Literacy; Determine the nature and extent of information needed
Mann-Whitney P-value: 5.843e-11
• Null hypothesis – The distribution of scores between Freshmen and Seniors did not differ significantly (a=0.05, b=0.2) because the course instruction had little to no impact on the students’ learning outcomes relative to each other.
• Alternative hypothesis #1 – The distribution of scores was significantly higher for Seniors than Freshman because exposure to similar course content in multiple courses during a Senior student’s college career has resulted in greater assimilation of learning outcomes compared to the single exposure among Freshmen. - supported
• Alternative hypothesis #2 - – The distribution of scores was significantly higher for Seniors than Freshman because undefined forces such maturity and greater life-experience has led Senior students to assimilate lessons similar to those taught during an instructional course. -supported
0
20
40
60
80
100
120
1 2 3 4
Freshman Distribution v. Senior Distributiion
Freshman Senior
n = 269 n = 136
![Page 13: General Education Assessment Results & Analyses 2019-2020 ...€¦ · General Education Committee ... and greater life-experience has led Senior students to assimilate lessons similar](https://reader033.fdocuments.in/reader033/viewer/2022060605/605a9792c2493612004703fb/html5/thumbnails/13.jpg)
Be seen. Be heard. BE INSPIRED.
Frequency Distribution of CCS 2.1 Scores by ClassCCS 2.1 – Information Literacy; Determine the nature and extent of information needed
0
0.1
0.2
0.3
0.4
0.5
0.6
1 2 3 4
Normalized Class Distribution
Freshman Sophomore Junior Senior
• Alternative hypothesis #1 – The distribution of scores was significantly higher for Seniors than Freshman because exposure to similar course content in multiple courses during a Senior student’s college career has resulted in greater assimilation of learning outcomes compared to the single exposure among Freshmen. - supported
• Alternative hypothesis #2 - – The distribution of scores was significantly higher for Seniors than Freshman because undefined forces such maturity and greater life-experience has led Senior students to assimilate lessons similar to those taught during an instructional course. -supported
• Longitudinal tracking could distinguish between these two hypotheses.
![Page 14: General Education Assessment Results & Analyses 2019-2020 ...€¦ · General Education Committee ... and greater life-experience has led Senior students to assimilate lessons similar](https://reader033.fdocuments.in/reader033/viewer/2022060605/605a9792c2493612004703fb/html5/thumbnails/14.jpg)
Be seen. Be heard. BE INSPIRED.
Frequency Distribution of CCS 2.1 Scores by ClassCCS 2.1 – Information Literacy; Determine the nature and extent of information needed
0
0.1
0.2
0.3
0.4
0.5
0.6
1 2 3 4
Normalized Class Distribution
Freshman Sophomore Junior Senior
• The majority of freshman (>50%), garnered ratings of sufficient (3) or advanced (4), despite the fact that freshman tended to have a higher percentage of ‘insufficient’ (1) and ‘basic’ (2) ratings compared to those of more advanced academic standing.
![Page 15: General Education Assessment Results & Analyses 2019-2020 ...€¦ · General Education Committee ... and greater life-experience has led Senior students to assimilate lessons similar](https://reader033.fdocuments.in/reader033/viewer/2022060605/605a9792c2493612004703fb/html5/thumbnails/15.jpg)
Be seen. Be heard. BE INSPIRED.
Frequency Distribution of CCS 4.1 Scores by ClassCCS 4.1 – Quantitative Literacy; Interpretation
• No statistically significant difference between academic levels.• (281 freshmen, 365 sophomores, 219 juniors, 245 seniors)• Lack of statistical power (CCS 4.1 b=0.61, CCS 4.2 b=0.67)• Estimate n=800/group for b=0.2
• The majority of freshman, in excess of 70%, garnered ratings of sufficient (3) or advanced (4), despite the fact that freshman tended to have a higher percentage of ‘insufficient’ (1) ratings compared to those of more advanced academic standing.
![Page 16: General Education Assessment Results & Analyses 2019-2020 ...€¦ · General Education Committee ... and greater life-experience has led Senior students to assimilate lessons similar](https://reader033.fdocuments.in/reader033/viewer/2022060605/605a9792c2493612004703fb/html5/thumbnails/16.jpg)
Be seen. Be heard. BE INSPIRED.
Frequency Distribution of IP 2.1 Scores by ClassIP 2.1 – Experience; Knowledge of Cultural Worldview Frameworks
• No statistically significant difference between freshmen and seniors
![Page 17: General Education Assessment Results & Analyses 2019-2020 ...€¦ · General Education Committee ... and greater life-experience has led Senior students to assimilate lessons similar](https://reader033.fdocuments.in/reader033/viewer/2022060605/605a9792c2493612004703fb/html5/thumbnails/17.jpg)
* Maybe some developmental categories require multiple exposures while others require only a single intervention.
Conclusions & Recommendations:
• Most General Education categories show a marked progression in competency during a student’s college career.
• Longitudinal data would be relatively easy to collect and helpful in addressing hypotheses• Assessment Input form
• Get rid of year in school box• Permits analysis of demographic data
• Pilot pre- and post-assessment assignments in PSR 1 or IP 2
![Page 18: General Education Assessment Results & Analyses 2019-2020 ...€¦ · General Education Committee ... and greater life-experience has led Senior students to assimilate lessons similar](https://reader033.fdocuments.in/reader033/viewer/2022060605/605a9792c2493612004703fb/html5/thumbnails/18.jpg)
Be seen. Be heard. BE INSPIRED.
![Page 19: General Education Assessment Results & Analyses 2019-2020 ...€¦ · General Education Committee ... and greater life-experience has led Senior students to assimilate lessons similar](https://reader033.fdocuments.in/reader033/viewer/2022060605/605a9792c2493612004703fb/html5/thumbnails/19.jpg)
Be seen. Be heard. BE INSPIRED.
Parametric v non-parametric analysis• T-test assumes a normal distribution of data
0
50
100
150
200
250
1 2 3 4
CCS 5.1 Oral & Written Communication - TargetingFreshman Distribution v. Senior Distributiion
Freshman Senior
Freshman mean±SD: 3.28±0.82n=473
Seniors mean±SD: 3.48±0.73n=252
T-test p-value Freshman v Seniors: 0.001278144
Mann-Whitney p-value: 0.00097969