Spring 2009 Evaluation Highlights:North Carolina Virtual Public School

Kevin Oliver, Assistant Professor of Instructional Technology

NC State University

Background

• Friday Institute has received annual contracts from DPI to assist with the evaluation of the North Carolina Virtual Public School (NCVPS)

• recurring surveys of stakeholder groups (i.e., students, teachers, distance learning advisors, and high school principals) (Sum 07, 09; Spring 08, 09)

• also, focused studies of NCVPS pilot initiatives:– efforts to design and develop course modules for

elementary-middle school

– student success course to transition 8th graders

Spring 2009 Surveys

• questions about student learning, barriers to success, teacher preparation and quality, advisor preparation and quality, curriculum and teaching quality, and leadership support

• similar questions to spring '08, allowing for comparison of cohorts

• spring 2009 response rates:

Are Students Successful in NCVPS Courses?

• more than 75% of students, teachers, and advisors agreed or strongly agreed that students were succeeding in NCVPS courses

• AP teachers were the most likely to agree students were succeeding at 100% while only 55.6% of credit recovery teachers reported the same

Are Students Learning More Online Compared to Face-to-Face?

• when asked if students were learning less or learning more online, compared to similar face-to-face courses, 43% of teachers reported students were learning more online significant increase from < 25%

in 2008

Are Students Gaining Tangential Skills from Participation in NCVPS Courses?

• 97% of students agreed courses taught them learning and innovation skills (e.g., creativity, critical thinking, problem solving) significant increase from 88% in 2008

• 92% of students agreed courses taught them technology literacy skills using such tools as Word, Excel, Powerpoint, Web development software, audio/video editors, photo editors significant increase from 87% in 2008

Are Students Gaining Tangential Skills from Participation in NCVPS Courses?

• 79% of students agreed courses taught them information literacy skills (e.g., online research skills) significant increase from 68% in 2008

• nearly two-thirds of students and more than three-fourths of teachers and advisors agreed that NCVPS courses were supporting student attainment of five of the seven 21st century skill areas (exceptions: civic literacy, global understanding)

Are Stakeholders Satisfied with Curriculum Quality?

• 95.2% of teachers agreed course content was sufficiently rigorous significant increase from 90.6% in 2008

• 92.9% of teachers agreed course assignments were sufficiently rigorous significant increase from 88.4% in 2008

• 90.9% of teachers agreed that they provided differentiated content and assignments significant increase from 79% in 2008

Are Stakeholders Satisfied with Curriculum Quality?

• 88% of teachers agreed they leveraged 21st century tools such as wikis, blogs, and podcasts in their courses significant increase from

54% in 2008

Are Stakeholders Satisfied with the Availability of Courses Offered by NCVPS?

• only 10% of students reported problems getting into courses they wanted to take

• only 11% of students reported NCVPS was not offering the courses they needed to take

Are Stakeholders Satisfied with Teaching Quality?

• 76% of students and 81% of advisors agreed that teachers were appropriately prepared to teach an online course

• 100% of teachers agreed that they participated in NCVPS professional development in the past year

• 50.2% of students agreed that teachers posted their own audio-visual presentations to explain content; such elaborations were requested by many students in written comments

Are Stakeholders Satisfied with Advising Quality?

• 26.6% of students agreed that a lack of distance learning advisor support was a barrier to taking NCVPS courses

• 85.3% of teachers agreed it was a barrier for at least some students

Are Stakeholders Satisfied with Advising Quality?

• DLA role is only full-time for 27% of advisors (typically media coordinators or counselors)

• advisors report four time-consuming tasks: marketing courses, registering students, monitoring student grades, and counseling students to stay on task

• some comment it can be difficult to manage regular job role and advising role simultaneously (mean of 73 advisees per advisor)

Do Stakeholders Encounter Technical Barriers When Using NCVPS Courses?

• 41% of students agreed that technical problems affected their experiences taking online courses significant decrease from 52% in 2008

• 52% of teachers agreed that technical problems affected their experiences teaching online courses significant decrease from 77% in 2008

Do Stakeholders Encounter Technical Barriers When Using NCVPS Courses?

• credit recovery students were significantly more likely to agree at 27.7% that a lack of technical expertise was a barrier to taking NCVPS courses compared to the three other course levels between 11-17%

• some credit recovery students may require more technical support than students at other course levels

Do Stakeholders Encounter Infrastructure Barriers When Taking NCVPS Courses?

• more than 80% of students indicated they had appropriate access to computers and the Internet to take NCVPS courses, so adequate infrastructure appears to be available to most students

• school-based computers are the primary source of access for more than 47% of students; only 34% primarily use home-owned computers

Do Stakeholders Encounter Infrastructure Barriers When Taking NCVPS Courses?

• 26.3% of credit recovery students agreed that a lack of access to computers at school was a barrier--significantly more than students at other course levels (general, honors, AP)

• credit recovery students were significantly more likely than students at other levels to report Internet connectivity as a barrier to using NCVPS courses, both at school (27.1%) and in the home (30.3%)

Are School and LEA Administrators Strong Advocates for NCVPS?

• only about one-half of students and teachers agreed that school-level and LEA administration were enthusiastic and supportive of online course opportunities

• more than 88% of principals surveyed indicated they were enthusiastic about offering online course opportunities to students (n = 47)

Do Stakeholders Personally Advocatefor NCVPS?

• percent who agree/strongly agree they would recommend more students take NCVPS courses

Subject Area Close-Up:Foreign Language

• significantly fewer foreign language students agreed they were successful in NCVPS courses at 68.3%

• significantly fewer foreign language students agreed they were learning more online compared to face-to-face at 18.9%

• significantly fewer foreign language students agreed that their courses supported the seven 21st century skill areas

Subject Area Close-Up:Foreign Language

• significantly fewer foreign language students agreed that their teachers were adequately prepared at 71.7%

• significantly fewer foreign language students agreed with a number of teacher quality variables (e.g., differentiation, using 21st century tools, encouraging study strategies)

Subject Area Close-Up:Math

• significantly fewer math students would recommend more students take NCVPS courses at 47.5%

• significantly fewer math students agreed that their courses supported the seven 21st century skill areas

• significantly fewer math students agreed with a number of teacher quality variables (e.g., differentiation, using 21st century tools, encouraging study strategies)

Subject Area Close-Ups:Discussion

• follow-up summer survey inquiring into subject area discrepancies

• English/Language Arts courses among the most highly rated

• could be problematic teaching foreign languages and math online, supporting synchronous dialogue and teachers visually working out problems

• OR, students may just like these subjects less overall• results might be the same if we compared foreign

language and math to other subjects in face-to-face environments