Interdisciplinary Coaching As a Nexus for transforming how … · 2018-06-18 · Vermont that is...

iCANNSF#1505202AnnualReport

1

Interdisciplinary Coaching As a Nexus for transforming how institutions support undergraduates in STEM (iCAN)

Annual Report NSF #1505202

PI Matthew Marino, University of Central Florida Co-PI Eleazar Vasquez, University of Central Florida

Co-PI Brian Moore, University of Central Florida Co-PI Manju Banerjee, Landmark College

Section I. Accomplishments

Major goals of the project The iCAN Project is a two-year exploratory project consisting of five phases. Each is designed to allow us to answer two fundamental research questions:

1. Which attributes of the Landmark College model are most salient at a large urban institution?

2. What technological and community supports add value to the model? Measurable innovative outcomes associated with iCAN include:

1. The development of a cost-effective model of institutional supports that enhance undergraduate STEM learning and persistence.

2. The use of interdisciplinary support teams to provide undergraduate STEM majors with content knowledge, study strategies, and career-specific social skill acquisition.

3. The development of a continuum of intervention supports that help secondary STEM teachers prepare their students for success in undergraduate STEM courses.

4. The identification of technological and community supports that add value to the model. Major Activities Fall 2015 – Conduct semi-structured group interviews at Landmark college with STEM majors who are successful, those that have failed to persist in STEM at other institutions, and academic coaches. Spring 2016 – Develop, implement, and test a model of supports for undergraduate STEM majors at the University of Central Florida. Specific Objectives The following objectives are associated with the first year of the iCAN Project: Objective 1 - The purpose of this objective was to examine and identify the essential elements of a successful STEM-support model at Landmark College in order to generalize knowledge and services to larger models such as the University of Central Florida. The research question was as follows: What are the essential attributes of an undergraduate support model for STEM learning and persistence in students with disabilities (SWDs)? Objective 2 - Collaboratively (i.e. using data from Landmark College, including undergraduates, graduates, the PIs, and advisory board) design a hybrid model of supports for undergraduates in STEM majors that can be used at UCF and other large institutions Objective 3 - Iteratively implement, test, measure, and revise the model at UCF


2

Significant Results Objective 1 – Landmark College (LC) is a small, private college in southeastern

Vermont that is well-reputed for their support model for students with disabilities. Executive Function (EF) skills training is a prize-winning service in their academic coaching model included in the tuition (Parker & Boutelle, 2009). LC is noted for being the first undergraduate institution to fully accommodate and support students with dyslexia. Decades later, they are developing and implementing programs that cooperatively foster academic, social, and emotional excellence in their students. There are numerous stories from Landmark that consist of disabilities being overcome through the institution’s innovation and supports. Landmark College agreed to share their students and faculty with the principal investigators (PIs) of this study in an effort to gather results for analysis and publication.

The interview sample comprised both students and staff from Landmark College. 30 individuals participated in total, including 20 undergraduate STEM majors who reported having EF deficits comorbid with their disability; five academic coaches, and five academic advisors. IRB informed consent was obtained from all participants. The researchers elected to conduct purposive sampling of the participants to maximize access to the richest information possible across age range, gender, and ethnic background. Use of purposive sampling permitted selection of a broad range of students.

A qualitative research design was implemented to examine faculty and student perspectives via semi-structured interviews. Semi-structured interviews are the primary source of data for many qualitative research studies (DiCicco-Bloom & Crabtree, 2006; Kvale, 1983; 2007) and were implemented in this study to extrapolate themes most indicative of a successful STEM coaching model. The interview protocol is a popular style of experimentation (Turner, 2010) that accesses themes or categories that emerge and are common to all participating interviewees (Kvale, 2007).

In these semi-structured interviews, the general line of inquiry oriented around academic coaching, technology, virtual coaching, and strengths and weaknesses of the current Landmark College model. “Sequential analysis” occurred during collection of data with the interview participants (Pope, Ziebland, & Mays, 2000, p. 114), so that as data was being collected, the line of inquiry was also being refined and coordinated by the interviewers with better clarity and potency. The interviews were recorded with audio and video, yielding more data for further examination. Interviews were transcribed by an independent contractor with proven reliability figures. The transcriptions are not isolated to words alone but sometimes included pauses, facial expressions, and tone of voice.

Objective 1 - Analysis A qualitative research software program named NVivo (version 11.2.1) was used to

process, analyze, and interpret the interview transcripts. Although software such as NVivo should never be seen as a “shortcut” to data analysis (Pope, Ziebland, & Mays, 2000, p. 114), NVivo is an excellent tool for management and categorization of large bodies of data (i.e., interview transcription). Typical qualitative designs neglect to represent the data with statistical means. Oftentimes frequency counts are utilized as a reference point but the truest expression of these data are often indices (i.e., an index) that yield a hierarchy of categories (Pope, Ziebland, & Mays, 2000).

NVivo works to systematically highlight main themes (or “nodes”) as the reviewer “codes” the interview transcripts. Each node nests other subcategories beneath it for further examination. Tools for further inquiry (e.g., “cases”) provide further cross-cutting and analysis


3

of the transcript data. NVivo’s process of coding at nodes, nesting, and driving case inquiry is an ideal tool for transcript analysis in order to implement the constant comparative method. According to the developers and practitioners of NVivo, the best qualitative data analysis is approached iteratively (QSR, 2015). That is, numerous readings of a given transcript afford the most refinement of themes and categories pertinent to the objective. According to scholars (Haney et al., 1998; Stemler, 2001; Miles & Huberman, 1984), this iterative process is consistent with the emergent coding process in qualitative literature. Numerous readings, codings, and re-codings provided refinement and clarification of themes sought in this research design, consistent with the constant comparative and emergent coding processes (Glaser & Strauss, 1967).

Objective 2 – An revised model conceptualization was developed in early spring of 2016 and shared with advisory board members during a January annual meeting. In addition, the model was presented at an American Education Research Association meeting. Critique from stakeholders led to further model revision. A convenience sample was utilized during a spring 2016 pilot study. Eight certified special education teachers seeking graduate degrees in special education and eight undergraduate STEM majors with disabilities participated in the iterative design, implementation, and assessment of a new graduate course titled, “College and career readiness for adolescents with high incidence disabilities.”

Objective 3 – The model was iteratively implemented, analyzed and refined during the

spring of 2016. Participants - Undergraduate STEM majors with disabilities

An attrition rate of 20% was anticipated based on previous studies by the PIs. This proved correct as two students withdrew before completing 20% of the intervention. All remaining undergraduate STEM majors (n=8) participated in 7 to 10 hours of academic coaching with a certified special education teacher who was pursuing a graduate degree. Six of the eight were female. Ages ranged from 18 – 22. STEM majors included computer science, nursing, mechanical and aerospace engineering, chemistry, biology, and criminal justice. All of the undergraduates were freshman or sophomores. Disabilities identified by the undergraduate students during initial interviews included Autism Spectrum Disorder, Attention Deficit Hyperactivity Disorder, learning disabilities, and processing disorders. Graduate students

Seven graduate students and the principal investigator served as coaches for undergraduates with disabilities during the pilot study. Graduate students were volunteers who participated as part of a graduate course on secondary special education methods. The goal of this research was to identify a scalable model of supports for the undergraduates. To achieve this, the research team developed a graduate course scope and sequence that would enable participants to develop individual learning plans for secondary students with disabilities using Bronfenbrenner’s Ecological Systems Theory (1979). The researchers hypothesized graduate special education teachers would gain knowledge and skills from coaching undergraduate STEM majors with disabilities that could be transferred to K-12 students.

IRB informed consent was obtained from all participants. Each participant completed the SuccessNavigator Pretest, the Barkley Deficits in Executive Functioning Scale (BDEFS), and a 43 item qualtrics survey. Each undergraduate STEM major was assigned to a graduate student coach. The coaches facilitated weekly 30 – 60 minute meetings with the undergraduates. Meetings were held using multiple modalities including online (i.e., Adobe Connect synchronous


4

meetings), face to face, and mixed modes. Some graduate students were at a distance and only utilized the online platform. Students participated in 7 – 10 coaching sessions during the semester. Each student had one coach for the duration of the intervention. Each coaching session was recorded using Adobe Connect even when the participants met face to face. Each graduate student coach had their own virtual classroom. All recordings were reviewed to ensure the graduate students were implementing course constructs with fidelity. For example, coaching (Parker, Field Hoffman, Sawilowski, & Rolands, 2011), growth mindset (Yeager et al., 2016), college and career readiness (Lombardi, Kowitt, & Staples, 2015), and mentoring (Bozeman & Feeny, 2007) were constructs of focus. Feedback was provided to graduate students during weekly 60-minute coaches’ meetings using the Adobe Connect platform. The meetings provided opportunities to develop a community of learners, share tips and tricks, and discuss undergraduate students’ strengths, challenges, and progress. Most importantly, they served as a forum to analyze the model. Each week model revisions were critiqued with the undergraduate and graduate students.

Graduate students submitted a draft case study about their students following spring break. These were revised based on instructor feedback and resubmitted at the end of the semester. Following final exams, undergraduate STEM majors repeated the SuccessNavigator and completed a 13-item Qualtrics post-intervention survey. Undergraduate STEM majors received a $100 gift certificate for completing all aspects of the pilot study.

Key outcomes or other achievements

Objective 1 - The essential attributes to Landmark’s STEM-support model have been identified as corresponding themes that emerged from the interview data. Themes were assigned as nodes within the analysis software. As these nodes withstood multiple iterations of coding and assessment, larger findings began to emerge. Upon further iterations of coding the data, three key findings surfaced as over-arching themes. These include: (1) Academic counseling is the core support. Technology facilitates, not supplements, this relationship; (2) Persistence comes through self-efficacy; and (3) Learning comes through metacognition.

Virtually every interviewee commented at great length about the influence the academic counseling support provides in the persistence and learning. Testimony emphasized a relational component to the support, providing a platform by which other supports could be rendered. Frequent attention was paid to ancillary supports such as technology and psycho-educational models (i.e., metacognition, goals, time management, and self-efficacy). Students and staff seemed to intuit how an ancillary support such as technology was a partial support, with academic counseling being the comprehensive support throughout.

The majority of interview testimony consisted of the academic advisors’ unequivocal role as the primary resource for support and growth of the students. This conclusion was unanimous across all participants’ data from both students and staff alike. This theme within LC’s support model tended toward the relational aspects of coaching and counseling. Even when interview questions prompted lines of inquiry regarding technological supports, the students noted the role the advisors played, not the technology.

The academic counseling at Landmark College focuses on weekly meetings, accountability, prioritization of goals, and practice of self-advocacy and metacognition skill sets. The successful blending of psychology and education provide a rich platform for growth in academic, social, and emotional domains. This cognitive-behavioral therapy (CBT) approach emphasizes self-growth, determination, grit, persistence, resilience, to name only a few research


5

constructs that empower the persistence and learning of the student. The advisors, whether realizing it or not, were implementing many of these concepts into practice, in attempt to increase the persistence and learning of their students. A more detailed discussion of this aspect of the project is included in Koch’s (2016) dissertation. Objective 2 – A progression of model development is noted in figures 1 – 2.

Figure 1. Initial iCAN model of supports based on success at Landmark College.

Figure 2. Revised iCAN model


6

Figure 3. Revised implementation model for fall quasi-experimental study.

Objective 3 – Analysis of the qualtrics post-intervention survey indicates undergraduate stem majors benefitted from the project. For example, when asked whether the students intended to persist in their STEM majors 75% responded “definitely yes” with an additional 25% responding “probably yes”. Other choices with no responses included “might or might not”, “probably not”, and “definitely not. When students were asked to rank order the types of support they valued coaching from the graduate students ranked first followed by student accessibility services, career mentors, peer mentors, student resource center sessions (i.e., tutoring), and assistive technology respectively.

When asked how long the weekly coaching sessions should last student responses varied from 15 to 60 minutes. Others stated the time should be variable based on the week. The coaches felt 30 – 45 minutes was the ideal time and that future coaches should plan for 60 minutes to allow time for post-meeting documentation and resource acquisition.

Undergraduate students were asked to identify technologies they utilized during the semester. The overlap of assistive and instructional technology was identified by Marino, Sameshima, and Beecher (2009) as critical in a revised Technological Pedagogical Content Knowledge (TPACK) model for students with disabilities. This point is especially relevant given the ubiquitous nature of technology in STEM disciplines. Surprisingly, none of the students identified text-to-speech software or Read and Write literacy software. One student used speech-to-text software. Two students used note taking software. Two utilized LiveScribe Pens. Nearly all of the participants used digital reminders and calendars. These have proven beneficial for secondary students in the past (Marino, Tsuruski, & Basham, 2011). Half of the students reported they always use technology when completing assignments or studying. Two students used technology some of the time and one student said she never used technology.

There were highly variable responses to questions about the responsibility of peer and career mentors in the model. For example, students (n=8) were asked to identify the amount of time that should be dedicated to: 1) social skills related to my career, 2) organization, 3) time


7

management, 4) political strategies, and 5) job acquisition skills. One students felt peer mentors should spend 62.5% of their time focused on job acquisition skills while several students did not think peer mentors should spend any time on the topic. The same variable pattern emerged for career mentors. Several students felt peer mentors should focus on political strategies for furthering one’s career while 40% of the students felt no time should be allotted for the construct. Opportunities for training and professional development There have been six formal professional development and training opportunities during this project year. Informal training for undergraduate and graduate students who participated in the project occurred on a weekly basis during the spring semester. Dissemination to communities of interest Widespread dissemination is ahead of schedule based on our initial proposal. A project website is operational at http://projectican.net. There have been six conference presentations, a dissertation, and an article in review as a result of this year’s activities. Plan for next reporting period to accomplish goals We will continue to follow the proposed plan for project implementation.

Figure 4. Project Timeline. Section II. Products

Journals or juried conference papers Manuscript in review – Marino, M. T., Vasquez, E., *Koch, A., *Fisher, K., & Banerjee, M. (in

review). Preparing special educators to promote college and career readiness in STEM: The iCAN project. Teacher Education and Special Education.

Note: *Denotes doctoral candidate or graduate from the Exceptional Education Ph.D. program. Books – nothing to report Book Chapters – nothing to report Dissertation - Koch, A. (June, 2016). Project iCAN: A STEM learning and persistence model for postsecondary students with disabilities.


8

Other conference presentations - Marino, M. T. (2016, January). Presentation. Bringing a model of STEM supports for students

with disabilities to scale: The iCAN project. American Educational Research Association special education STEM meeting. University of California, Santa Barbara, CA.

Marino, M. T., Vasquez, E., & *Donehower, C. (2016, March). Presentation. iCAN: An exploratory study of UDL principles for college students with and without disabilities. Universal Design for Learning Implementation Research Network annual convention. Towson University, MD.

Marino, M. T., Vasquez, E., Hines, R. & Holbrook, J. (2016, April). Technology Innovations at the University of Central Florida. Workshop for Florida Technology Leadership Consortium.

Marino, M. T. (2016, April). Presentation. iCAN: A collaborative study to promote STEM performance and persistence for college students with disabilities. Landmark College, Putney VT.

Coy, C. & Marino, M. T. (2016, April). Presentation. Applying UDL in digital learning environments. Council for Exceptional Children Annual Convention, St. Louis, MO.

Marino, M. T., Banerjee, M., & Vasquez, E. (2016, June). Presentation. Interdisciplinary Coaching As a Nexus for transforming how institutions support undergraduates in STEM (iCAN). Postsecondary Disability Training Institute, Philadelphia, PA.

Other publications – nothing to report Technologies or Techniques – nothing to report Patents – nothing to report Inventions – nothing to report Licenses – nothing to report Project website – http://projectican.net Other Products – nothing to report Section III. Participants

Individuals who have worked on the project Names, Most Senior Project Role, Nearest Person Month Worked Matthew Marino, Principal Investigator, 1 month Eleazar Vasquez, Co-PI, 1 month Brian Moore, Co-PI, 1 month Manju Banerjee, Co-PI, .5 months Christine Parsons, Project Assistant, 12 months Aaron Koch, Doctoral Research Assistant, 6 months Rebecca Hopkins, Doctoral Research Assistant, 3 months Faith Ezekiel-Wilder, Doctoral Research Assistant, 3 months Partner Organizations Name, location, contribution to project Manju Banerjee, Co-PI, Landmark College, Putney, Vermont.


9

Section IV. Impacts

Impact on the development of principle disciplines of the project Impact on other disciplines – nothing to report Impact on the development of human resources – nothing to report Impact on institutional resources that form infrastructure – nothing to report Impact on information resources that form infrastructure – nothing to report Impact on technology transfer – nothing to report Impact on society beyond science and technology – nothing to report Section V. Changes / Problems

Changes in approach and reason for change. - We may modify our pretest measures based on the pilot study this spring. There was a lack of consistency across the two measures (i.e., Success Navigator and Barkley deficits in Executive Functioning Scale). We are still analyzing the data and plan to have a recommendation by the end of July. Actual or anticipated problems or delays and actions or plans to resolve them – nothing to report Changes that have a significant impact on expenditures – nothing to report Significant changes in use or care of human subjects – nothing to report Significant changes in use or care of biohazards – nothing to report

References Adobe Connect (2016). Retrieved from http://www.adobe.com/products/adobeconnect.html Barkley Deficits in Executive Function Scale (2011). Retrieved from

http://www.guilford.com/books/Barkley-Deficits-Executive-Functioning-Scale-BDEFS-Adults/Russell-Barkley/9781606239346

Barkley, R. A. (2012). Executive functions: what they are, how they work, and why they evolved. New York: Guilford Press.

Bozeman, B., & Feeney, M. K. (2007). Toward a useful theory of mentoring: A conceptual analysis and critique. (6), 719-739.

Bronfenbrenner, U. (1979) Contexts of child rearing: Problems and prospects. American Psychologist, 34(10), 844.

DiCicco-Bloom, B., & Crabtree, B. F. (2006). The qualitative research interview. Medical Education, 40(4), 314-321.

Glasser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory. Chicago: Aldine. Haney, W., Russell, M., Gulek, C., & Fierros, E. (1998). Drawing on education: Using student

drawings to promote middle school improvement. Schools in the Middle, 7(3), 38-43. Koch, A. (2016). Project iCAN: A STEM learning and persistence model for postsecondary

students with disabilities. Orlando, University of Central Florida. Dissertation Abstracts. Kvale, S. (1983). The qualitative research interview: A phenomenological and a hermeneutical

mode of understanding. Journal of Phenomenological Psychology, 14(2), 171. Kvale, S. (2007). Doing interviews. Thousand Oaks, CA: Sage. Lombardi, A. R., Kowitt, J. S., Staples, F. E. (2015). Correlates of critical thinking and college

and career readiness for students with and without disabilities. Career development and transition for exceptional individuals 38(3), 142-151.


10

Marino, M. T., Tsuruski, B. K., & Basham, J. D. (2011). Selecting science software for students with learning disabilities and other special needs. The Science Teacher, 78(3), 70-72.

Marino M. T., Sameshima, P., & Beecher, C. C. (2009). Integrating TPACK in pre-service teacher education: Frameworks for promoting inclusive educational practice. Contemporary Issues in Technology and Teacher Education. 9(2), 186-207.

Miles, M. B., & Huberman, A. M. (1984). Qualitative data analysis: A sourcebook of new methods. California: SAGE.

NVivo for Mac. Version 11.2.1 (2016). Downloaded from http://www.qsrinternational.com/ Parker, D. R., Field Hoffman, S., Sawilowsky, S., Rolands, L. (2011). An examination of the

effects of ADHD coaching on university students’ executive functioning. Journal of Postsecondary Education and Disability 24(2), 115-132.

Parker, D. R., & Boutelle, K. (2009). Executive Function Coaching for College Students with Learning Disabilities and ADHD: A New Approach for Fostering Self-Determination. Learning Disabilities Research & Practice, 24(4), 204–215

Pope, C., Ziebland, S., & Mays, N. (2000). Analyzing qualitative data. BMJ, 320(7227), 114-116.

SuccessNavigator (2016). Retrieved from https://www.ets.org/successnavigator Stemler, S. (2001). An overview of content analysis. Practical Assessment, Research &

Evaluation, 7(17), 137-146. Turner, D. W. (2010). Qualitative interview design: A practical guide for novice investigators.

The Qualitative Report, 15(3), 754. Yeager, D. S., et al., (2016). Using design thinking to improve psychological interventions: The

case for growth mindset during transition to high school. Journal of Educational Psychology, 108(3), 374-391.

Interdisciplinary Coaching As a Nexus for transforming how … · 2018-06-18 · Vermont that is...

Documents

Transcript of Interdisciplinary Coaching As a Nexus for transforming how … · 2018-06-18 · Vermont that is...