INCREDIBLEMINDSAFTERSCHOOL CAREER ROLEPLAYING PROGRAM

AUTHORSTodd Borghesani, Esq.

Bruce Milligan

Keenan Grenell, Ph.D.Roderick Weldon Woodruff

WHITE PAPER

1

IncredibleMinds addresses three critical educational needs: strengthening the STEM pipeline, targeting under-served locations and offering parental career guidance. We achieve this through the innovative use of

video games, modeling and simulation tools and an extensible career roleplaying framework. Using NASA content and subject matter expertise, IncredibleMinds offers experiential and immersive STEM career education to minority youth. We use the afterschool environment to serve minority high school students, parents, guardians, teachers, and counselors.

Research continues to suggest that career decision-making is one of the major challenges young people face, and that parents need assistance in helping their child choose a career path (Otto, 1996). A survey by the National Association of Colleges and Employers (1996) confirmed that parents most heavily influence career choices. Yet, few resources are available to help parents understand and fulfill their roles in their children’s career decision-making process.

In our initial areas of operation, Portsmouth, Virginia and Washington, D.C., there is a cogent need for an innovative solution that bridges high school to college; and that bridges the needs of industry and education. Southeastern Virginia has a regional economy in need of transformation. A shortage of skilled workers threatens to halt progress on emerging and expanding industries. This region is home to one of our nation’s critical national technologies, modeling and simulation.

Washington, D.C. is one of the worst school districts in America. Only 29%

What is the Educational Need?

INCREDIBLEMINDSAFTERSCHOOL CAREER ROLEPLAYING PROGRAM

2

percent of D.C high-school graduates enroll in college within 18 months of graduation verses 48% nation-wide, and only 9% of them graduate from college within five years. Education Week, in its annual grading of the 50 states and the District of Columbia. ranked the District last, giving it a D-plus overall and an F for K12 student achievement and college readiness.

The IncredibleMinds Career Roleplaying Program initially addresses the need for youth to pursue STEM careers in NASA aerospace and aviation. Our extensible framework supports broad application across the diverse interests of youth and needs of NASA mission

directorates, Federal agencies and industry. IncredibleMinds can address the nation's need for “in-demand” careers, including: aerospace engineering, behavioral sciences, biosciences, chemical engineering, chemistry, civil engineering, clinical medicine, cognitive and neural science, computational sciences, ecology and environment, electrical engineering, geosciences, immunology, materials science, mathematics, mechanical engineering, microbiology, molecular biology and genetics, naval architecture, nanotechnology, neuroscience, ocean engineering, oceanography, pharmacology, plant and animal science, physics, and space and planetary sciences (Thompson, 2005).

IncredibleMinds is built from the authentic stories and career competencies of the engineers, mathematicians, scientists and technologists behind NASA Langley Research Center’s world-class aerospace and aviation research. Across the areas of aerospace and aviation, we will identify and draw resources from the following NASA Langley teams: Flight Simulation and Software Branch (Computer, Electrical, Mechanical and Aerospace Engineer), Structures and Materials Facilities (Materials Engineer/Scientist) and Scientific and General Purpose Computing Complex (Modeling and Simulation Engineer).

Based on NASA’s Workforce Plan released in March 2004, out of 148 critical competencies identified in the plan, 30 key competencies were classified as “critical needs” for NASA. Computer, electrical, mechanical, aerospace and general engineers are all occupations that represent competencies in critical need.

Meeting NASA’s Education Priorities

3

To support NASA’s education priorities and workforce needs, the IncredibleMinds Program strengthens the STEM pipeline. This program is designed to engage high school students and their parents to advance science and technology literacy in under-served and under-represented communities. The program helps youth identify and begin experiencing their career ambitions through active participation in compelling, interactive educational activities. Using authentic career “experiences” as the foundation of our strategy, youth begin to understand the value and context of STEM education. They are encouraged to pursue educational disciplines critical to NASA’s future engineering, scientific, and technical needs. As youth roleplay NASA careers, they develop the critical skills and capabilities needed to ensure achievement of NASA mission priorities. They become NASA’s workforce of the future.

IncredibleMinds directly supports Outcome Two of the NASA Education portfolio. NASA aerospace and aviation themes, content and authentic data will be used to inform high school students about STEM career opportunities; enhance student skills and proficiency in STEM disciplines; and communicate information about NASA’s

mission activities. The afterschool Career Roleplaying Program bridges high school to college education and our outreach program reaches community stakeholders. To support high school student achievement, we provide short-duration professional development for educators. Additionally, the program provides opportunities for parent and family involvement in STEM K-12 student learning. Our development strategy uses an evidence-based approach grounded in cognitive, education and psychology research as well as industry best practices.

Supporting NASA education outcomes and objectives

4

The IncredibleMinds research and development team’s composition offers a strong combination of complementary expertise. Michelle Roper and Bruce Milligan of the Federation of American Scientists’ (FAS) Learning Technologies Program focus upon strategies to harness

the potential of emerging information technologies to improve how we teach and learn. Todd Borghesani specializes in serious games and simulations for K12 STEM education and has a background in NASA K12 education. Our doctoral student researcher from George Mason University’s Department of Psychology will offer expertise in developmental, human factors and industrial psychology as it relates to virtual environments for training and education. Bill Waite, president of Aegis, leads a professional modeling and simulation firm committed to the education of our next generation. Dr. Ralph Coppola of Parametric Technology Corporation’s Global Design and Engineering Program is part of a team that has taught simulation-based engineering to over two million youth and 15,000 teachers. Chuchun Tsai is a Fairfax, Virginia, career and technical education teacher who directs afterschool programs, which use modeling and simulation tools. Sylvia Gray is a veteran career management consultant who works with major corporate clients as well as youth and parents in under-served and under-represented communities. Neal Ashby of Ashby Design works with clients such as National Geographic, and transforms their ideas and concepts into visual designs. Joyce Malyn-Smith and Costanza Eggers-Piérola of Education Development Center are the external evaluators. EDC is a global nonprofit that develops, delivers, and evaluates innovative programs to solve some of the world’s most urgent challenges in education. Roderick Weldon Woodruff, founder of the Urban Video Game Academy, prepares youth in under-served areas for post-secondary education and technology careers by teaching them the fundamentals of video game design and development using advanced modeling and simulation tools.

The IncredibleMinds Board of Advisors assures cross-industry and inter-agency replicability. The Board includes Deborah A. Boehm-Davis, Ph.D., University Professor and Chair, Department of Psychology, George Mason University; Keenan Grenell, Ph.D., Vice President and Dean of Diversity, Colgate University, and Global Fellow at the Institute of Innovation, Creativity and Capital, University of Texas at Austin; Robert Pavlik, Ph.D., Director, School Development Center, Institute for the Transformation of Learning at Marquette University; James

A World Class Research and Development Team

5

Cowgill, Producer, Destineer Studios; William V. Tucker, Modeling and Simulation Chief Scientist, Boeing Integrated Defense Systems; John P Grant, Netcentric Integration and Experimentation Center-Tidewater, Raytheon Network Centric Systems; James G. Batterson, NASA Aerospace Technologist, NASA Langley Research Center; James Brough, National Aviation and Space Education Manager, Federal Aviation Administration; Julia Fallon, Technology Integration Program Manager,  Educat ional Technology,  Research and Educator Development, Washington State Office of Superintendent of Public Instruction; Andre J. Peltier, Ph.D., STEM Teacher Specialist and Microbiologist, Portsmouth Public Schools; Jeff Aron, Senior Director for Corporate, Foundation and Public Outreach, Federation of American Scientists; Debbie Mayer, Real World Design Challenge Program Officer, Department of Energy.

The NASA IncredibleMinds Institutes’ pilot program will reach students, parents and c o m m u n i t y l e a d e r s i n Portsmouth, Virginia and Washington, D.C. Our target audiences are high school students, parents/guardians, teachers, school administrators,

community and spiritual leaders from under-served and under-represented communities.

Our twin goals are product development and community-based program development. Our evidence-based, product development goal is to develop a scalable and replicable STEM career roleplaying system. In service to communities, our outreach program offers parent workshops that help parents play a critical role in the career exploration process of their children.

The concept of play is central to human exploration of life. Playing video games is remarkably relevant for productive citizenship in the 21st century. These immersive worlds offer youth a place to learn how to act within dynamic systems; to think procedurally within computationally rich spaces; to build and navigate complex information networks, and to engage in collaborative peer-to-peer learning (Institute of Play, 2008). STEM education and careers are not interesting to today’s youth because their most fascinating aspects cannot be brought to life using conventional media. The

Our Target Audience, Goals and Objectives

6

IncredibleMinds approach leverages youths’ fascination with video games. To convey the vast and interesting career opportunities available to youth we combine commercial video games with industry modeling and simulation technologies.

Tomorrow’s engineering and scientific workforce must possess 21st century skills and be able to work with industry modeling and simulation tools. Modeling and simulation tools offer a rich, e n g a g i n g a n d s c a l a b l e environment where NASA STEM

careers can be visualized and experienced. No matter what activities a NASA scientist or engineer chiefly engages in, there is an element of puzzle-solving to his or her work. It is this puzzle-solving that makes video games and modeling and simulation tools excellent “props” for experiencing NASA careers.

IncredibleMinds goes beyond college guidance and occupational information to address the entire process of STEM career assessment, exploration and action planning necessary to help minority youth find STEM careers that fit their skills, interests, personalities, and developmental needs. New educational solutions must be relevant and valuable to industry as well as relevant and valuable to youth (National Association of Governors, 2006). The key to the future of education is leveraging current investments in entertainment software and industry technologies.

Our evidence-based approach is developed from the Federation of American Scientists’ Learning Science and Technology Research and Development Roadmap Project. Microsoft Research, Hewlett Packard, Carnegie Corporation of New York, National Science Foundation, Department of Defense and the Hewlett Foundation financed this seminal project in 2003. The Roadmap integrates the advice of approximately 100 researchers to define the state of the art in learning science and technology, and to identify key research challenges. Based upon this roadmap, the areas of research, theories and pedagogy that inform our approach are career development; cognitive research; cognitive apprenticeship; instructional scaffolding and inquiry-based learning. From a best practices perspective, we will be adapting the electronic entertainment industry’s roleplaying game systems; NASA’s Workforce Plan; the Department of Labor’s career competency

Our Rationale for the Approach and Best Practices

7

models and the Department of Education’s State Career Clusters.

Virtually all teens play games. Nearly all American teens — 97% — aged 12-17 play computer, Internet, console, or mobile games. Teens are also playing these games with relative frequency and duration. Nearly one-third (31%) of teen gamers play games every day, and another one in five (21%) play games three to five days a week. According to the Pew MacArthur Foundation study, “Teens, Video Games and Civics,” 49% of teens play video games such as flight simulations. Social gameplay is thought to offer the possibility for youth to have collaborative and interactive experiences; experiences that potentially parallel many real world political and civic activities. Some scholars have suggested that play in groups, particularly when working collaboratively toward a common goal, lays the groundwork for learning how to work with groups toward a common goal in other facets of life, particularly within the workplace and community (Jenkins, 2008).

Set in an authentic NASA context, each roleplaying challenge is designed as a combination of individual inquiries demanding teamwork to complete (Chinn, 2000). We design each NASA roleplaying challenge from representations of authentic career pursuits. The roles are built from NASA’s Workforce Plan and its articulation of critical competencies, the Department of Labor authentic career competency models (DOL Career Infonet) and the career linkages to STEM education (States' Career Clusters Initiative, 2007).

The roleplaying of real-life situations promotes the development of critical thinking skills, and humanizes science by discerning its importance to everyday life. It can also develop importance skills needed in the real world, e.g., teamwork, collaborative learning and effective communication (Fadali, 2001). According to Piaget, students accommodate and assimilate new information better when they have concrete educational experiences. Video game, modeling and simulation technologies offer powerful, hands-on opportunities to recreate reality; visualize engineered systems and scientific phenomena; model complex systems, and engage youth in the practice of science and engineering. The use of roleplaying as an instructional strategy has been part of the repertoire of teaching methodology for many years.

Recent cognitive research has begun to differentiate the types of knowledge required for expertise. In particular, researchers have begun to distinguish among the concepts, facts, and procedures associated with expertise and various types of strategic knowledge. Strategic knowledge refers to the usually tacit knowledge that underlies an expert's ability to make use of concepts, facts, and procedures as necessary to solve problems and accomplish tasks. The attributes that experts use to generate new knowledge

8

are very similar to the qualities essential for the effective transmission of knowledge within the learners' environment.

For centuries, apprenticeship was the vehicle for transmitting the knowledge required for expert practice in fields from painting and sculpting to medicine and law. It was the natural way to learn. Cognitive apprenticeship is a model of instruction that works to make thinking visible (Collins, 1991). We use this model to inform how we support the virtual mentor’s interactions with the roleplaying team during a challenge. As such, we identify which processes of an advanced modeling and simulation task should be made visible. The virtual mentors assist in revealing concepts, facts and procedures as necessary to solve problems and accomplish tasks. In a landmark series of studies, Bloom and colleagues demonstrated that one-on-one mentoring improved student achievement by two standard deviations over group instruction (Bloom, 1984). Mentoring can directly affect self-worth, school value, parental relationships and academic achievement (Rhodes, 2003). As Sternberg (1985) explains, “What seems to be critical is not the sheer amount of real world experience a youth gains, but rather what they can do with that experience.” Using mentors within the context of career roleplaying offers youth examples of natural patterns of learning (Gartner, 1995).

In order to make a virtual mentor's thinking visible, we use authentic modeling and simulation tools. In a case study in Massachusetts, 3-D design software was shown to have a positive impact on math and science achievement with at-risk students, as it enhanced motivation and it helped students become technologically literate. In 2002, 70 Shawsheen Valley Technical High School students who were at risk of failing the math portion of the Massachusetts Comprehensive Assessment System (MCAS) attended a special summer program designed to help them learn the math portion of the required MCAS tests. The students were exposed to design modules, using 3-D design software, to help them learn 32 math concepts. There was significant increase in Stanford Diagnostic math scores after participation in the program. Some 88.5% of the students showed overall growth (62 out of 70); and the performance of 15 % increased by at least four grade levels.

Career development theory informs our use of technology and design of career coaching, virtual mentoring and roleplaying. According to Super and Knasel (1981) career adaptability is considered to be a major predictor of success. Research shows that certain students who take career education courses perform as well or better than students not in career education programs (Castellano, 2004); have lower dropout rates (Plank, 2005); and earn more money in the labor market (Kemple, 2004). Other high-quality

9

research supported by the U.S. Department of Education has shown that students who take math-enhanced career education perform at higher levels than students who do not do so (Stone, 2006). This growing research base helps to build the case that career education can improve student outcomes and serve a wide range of students.

Most students struggle with vocational decisions. This has been cited by numerous studies as the top concern of high school students and young adults (Montross, 2004). Adolescents’ search for identity translates in large part to how they will define themselves in terms of vocational

roles they intend to play (Vondracek 1993; Blustein, Devinis & Kidney 1989, Marcia 1980). As part of a 10-year, $25 million initiative, the Kauffman Foundation reported that parents and students both said they would be most motivated to change STEM education by arguments that relate to future STEM opportunities for young people in the job market. The Association for Counselor Education and Supervision has expressed concerns about the current state of career education. Pope and Minor (2000) have challenged the field to examine what strategies are being used to excite students.

Studies conducted by organizations such as ACT state that only 34% of students graduate from high school ready for college -- and that number is even smaller for minorities. In the United States, approximately 50% of prospective engineers are “weeded out” in large lecture courses their very first year in college. The work of Seymour and Hewett shows that this “weeding out” is because these students are not sufficiently engaged in the content.

For our national aerospace and aviation industries, inspiring and preparing minority youth to pursue STEM careers is critical. Fifty-four percent of the aerospace science and technology workforce is over 45 years of age, and 33% are eligible to retire today. At the same time, NASA faces significant human capital challenges that threaten the Agency’s ability to accomplish its mission.

Academic achievement is influenced by real world experiences. To bring the

Evidence of Customer Need and Addressing that Need

10

real world to youth we leverage current investments in commercial video games and industry modeling and simulation tools. While the combination of these technologies is powerful, it is our specialized social processes that distinguish the Career Roleplaying Program. Using these commercial technologies with career coaching, virtual mentoring and roleplaying brings STEM careers to life.

The afterschool environment offers a key opportunity to expose students to ideas and teach them skills that can unlock doors to future career prospects. Career education continues to be a tested strategy to engage students in their learning and prepare them for post-secondary education and the complex world of the 21st century. The afterschool hours are particularly important for children in under-served communities. Corporate Voices for Working Families and the U.S. Chamber of Commerce have both identified afterschool programs for all students as a “unique venue in which young people can develop the range of skills they need to enter the 21st century workforce.”

From a technology perspective, the key to the future of education is leveraging current investments in entertainment software and industry technologies. There is a natural technical synergy between video game, modeling and simulation technologies (Smith, 2007, Williams, 2006). The video game industry, estimated at $9 billion in the U.S., has captured the imagination of youth and adults alike. Including more than 90 million people, the "gamer generation" already dominates the workforce. 

We use commercial video games as the conceptual bridge to “playing” with industry simulation and modeling tools. Modeling and simulation tools are the immersive environments that our next-generation engineers and scientists must master. These tools provide the means to design and build engineered systems as well as to investigate and visualize complex scientific phenomena. The modeling and simulation industry is a $6 billion U.S. industry that is central to innovation in high-growth industries and the advancement of scientific discovery. Today, Congressional and industry leaders consider the modeling and simulation industry a "national critical technology.” According to Dr. Edwin Crues, NASA Constellation Program Modeling and Simulation Architect, “We cannot have an active and vibrant space program without an active and vibrant modeling and simulation community to support it.”

11

Our goals and objectives are a c h i e v e d t h r o u g h t h e development and operation of three program components: (1) IncredibleMinds Student Career Roleplaying, (2) IncredibleMinds Educator Career Coaching, and the (3) IncredibleMinds Parent Career Coaching Workshops.

How The Student Career Roleplaying Program Works. The Student Career Roleplaying Program is a three-step process. First, youth Explore commercial video games to gain career insight. Second, they Discover how to use industry modeling and simulation tools within the context of these game worlds as they engage in their character’s creation. Third, they experience what it means to Become “professionals” as they roleplay in team-based, real world engineering and scientific challenges.

The Career Roleplaying Program begins with Explore, which consists of initiating coaching conversations, interest assessment and gameplay. The content of the conversation includes NASA career opportunities. Central to this process is the interest assessment, which uses Hoffman’s Self-Directed Search to help youth choose careers and fields of study that best match their self-reported skills and interests.

Gameplay is essential to inspiring youth to think differently about exploring their futures. Youth play aviation-related games and simulations, such as Ace Combat 6 and Microsoft Flight Simulator X. These games offer team play, competition and the opportunity to control numerous aircraft. Each also offers a rich and immersive context for conversations about careers in aerospace, aviation, flight simulation, structures and materials science, and scientific computing.

During Discover and Become, youth are introduced to authentic careers using experiential roleplaying with authentic simulation and modeling tools. They must craft their “professional” characters to meet the team-based challenge. Playing a roleplaying game is much like acting out a scene from a play, movie or book, only without a predefined script. Youth learn engineering design, scientific inquiry and background knowledge as interdisciplinary teams (Marzano, 2004). Taking on the roles of an elite NASA Team, youth solve challenges that are faced by NASA engineers, scientists and mathematicians. The use of simulation and modeling tools are scaffolded with 3-D elements so that they are developmentally appropriate

Our Innovative Approach, Tools and Techniques

12

(Vygotski, 1962). Youth work in real world contexts designed from NASA Langley Flight Simulation and Software Branch, Structures and Materials Facilities and the Data Visualization and Animation Laboratory. It is the pursuit and acquisition of “expertise” that drives gameplay (Kolb, 1984). Youth ascend through NASA’s ranks, from New Hire, to Seasoned, to Established and then Veteran, and ultimately, to Legend.

Practicing engineers and scientists -- the mentors -- understand the power of modeling and simulation as well as the specific industry domain that the roleplaying challenge relates. They are the advisors in the challenge. The mentor is the virtual, and on occasion, in person, “cameo” character that changes the roleplaying group’s dynamic. By offering advice, increasing the challenge’s complexity for one of the players, or sending the team on a quest, they increase the level of randomness, reflecting reality. The mentor further opens the window to the real world.

How The IncredibleMinds Educator Career Coaching Program Works. The educator professional development program combines training in the Student Career Roleplaying Program with a career coaching process. In general, the career coach is the inspirational guide. The career coach is on-site as a generalist in STEM career development. Career coaching provides each youth with a framework with which to organize his or her thinking about career options and what it would take to get there. The career coaching process is designed to carry youth from interest assessment and technological literacy through contextual college and career skills.

During the Explore, Discover and Become process, the coach works to inspire curiosity, connecting “gamer intelligence” to professional careers. Coaches inspire youth to think about NASA Careers. What it is like to build spacecraft or top-secret aerospace systems as NASA researchers? What is it like to be a space entrepreneur designing futuristic, suborbital aircraft? These are the questions that flow from the challenge narrative. The answers unfold across each roleplaying challenge.

How IncredibleMinds Parent Career Coaching Workshops Function. Parent workshops serve the dual purpose of advancing STEM literacy among adults and building parent-child career exploration partnerships (Montross, 2004, Schescke, 2008). Everyone needs to acquire some degree of science, technology, engineering and mathematics literacy to succeed, even to function, in today’s increasingly sophisticated world (ITEA, 2002, Reach Higher, 2008).

Parental career coaching workshops are structured as short sessions. We

13

open with the future world of work and the power of having a career -- a future inspired by NASA. We tell a story about this future: "This is a day in the life of your child, 10 years from now.” Then we broaden our message and relate this success story to the community. During this part of the conversation, we use a visual backdrop of NASA’s achievements. Parents learn how to help their children develop a career support network; map careers to STEM courses, and understand NASA student internships.

The IncredibleMinds Career Roleplaying Program brings a wealth of real world experiences to minority youth. By combining video games with modeling and simulation tools, we offer these youth the environment to explore, discover and become our future. Career decision-making begins with trying to understand our potential in the early stages of development, when there are only inklings of what we might become (Montross, 2004). When the learning context is the “career pursuit,” we can use it to inspire and motivate youth to see value in pursuing STEM academic studies, build 21st century skills, achieve STEM literacy and understand higher education’s role in the world of work.

Career development theory is applied through the use of an authentic competency model. We transform NASA career competencies into roleplaying characters, which youth can discover and become during play. A competency is the capability to apply or use a set of related knowledge, skills, and abilities required to successfully perform "critical work functions" or tasks in a defined work setting. Competencies serve as the basis for skill standards that specify the level of knowledge, skills, and abilities required for success in the workplace as well as potential measurement criteria for assessing competency attainment (Malyn-Smith, 2000).

Central to exploring NASA careers is the motivational effect of commercial video games (Garris, 2002, Cordula, 2003). Implicit in the research literature is the notion that if we pair instructional content with certain game features, we can harness the power of games to engage users and achieve desired instructional goals. Rather than trying to build video games that do this, we pair commercial game titles and simulation and modeling tools with the social processes of career coaching, virtual mentoring and roleplaying.

14

The EDC team will design the evaluation plan and instruments; collaborate with the George Mason University (GMU) team in data collection; and apply qualitative and quantitative analytic methods to determine the program components’ impact and inform the development of

each program component. GMU’s research assistants will assist in coordinating, scheduling, and conducting data collection (including recruitment for interviews and focus groups); participate and provide feedback in instrument revisions and analysis; and transcribe, code, and input data. See Exhibit 1 for the Evaluation Logic Model.

The research team will gather data from multiple sources to address internal validity and triangulate findings. This “pattern-matching” (Yin, R.. 1984). Case Study Research: Design and Methods. Beverly Hills, California: Sage Publications) – typically used in case study methodologies — will ensure that our resulting theories are confirmed through multiple lines of inquiry as well as facilitate our research team’s openness to identify unexpected directions to pursue in the analysis. This plan will employ multiple evaluation techniques to systematically examine the pilot program’s written materials and training structure, as well as participant satisfaction.

In order to evaluate each of these facets of the program we will develop a series of evaluation questions that examine the inputs, activities, and outputs of the logic model. EDC will triangulate coding; analyze data, and develop recommendations for program improvement and roll-out. EDC will also monitor the progress of each activity towards completion of the overall goals and provide feedback to FAS and partners, as well as produce a brief formative evaluation report and a summative evaluation report.

From February to June 2009, the EDC team will work with project partners to review and revise the evaluation logic model (See Budget Narrative) to align with NASA PART outcomes and will design a suite of tools specifically constructed to address the program outcomes and reflect the program’s characteristics. The team will draft a codebook to reflect the framework and PART outcomes. Instruments will include online pre- and post-surveys (for students, coaches, and mentors); questionnaires for program staff; training and parent workshop evaluation forms, and interview and focus group question guides (parents, students, mentors).

Our Evaluation Plan and Instrumentation

15

Preliminary and ongoing data collection will occur during the planning and implementation stages of the summer and school-year institutes (June 2009 – December 2010) as well as during the training of mentors and at parent workshops. EDC will work closely with GMU to guide their collection of program and participant information (including baseline data), transcription of interviews, and data entry. EDC will maintain communication with GMU and their research assistants to update data. EDC will lead onsite focus groups and conduct phone interviews; and send and gather online surveys. EDC will continue to work with GMU to supervise the collection of data during the rollout of Incredible Minds in schools between June and October 2010 and the cleaning and entering of data into the evaluation database.

EDC will analyze data and develop formative and summative reports that describe the program’s progress and achievement of expected outcomes. Construct validity will be addressed by integrating measures and foci that highlight best practices aligned with the PART, and by engaging in a grounded theory approach with an iterative coding process that evolves with the analysis (Strauss, A. and Corbin, J. (1990). Basics of Qualitative Research: Grounded Theory Procedures and Techniques. London: Sage Publications). In this way, results will reflect viewpoints of participants as well as the concepts of best practices elicited from PART and other sources. The formative evaluation report will be delivered to partners in December 2009 and include recommendations for program improvement, rollout and expansion. EDC will analyze results against intended outcomes and submit a summative report on findings to partners by the end of the project period.

EDC has sophisticated quantitative and qualitative research and evaluation staff that employ a range of research designs responsive to various educational contexts. In their work, staff collaborate with universities, colleges, and other institutions to conduct complex national studies. EDC’s partners include Harvard University, Boston College, University of California Berkeley, Frank Porter Graham Child Development Institute at the University of North Carolina, Wheelock College, University of Illinois, and the Center for Applied Linguistics. Dr. Eggers-Piérola has carried out program evaluation of Title III and Title V initiatives in Massachusetts and regional STEM reform initiatives in NY State.

16

Our work afforded by NASA’s investment builds the foundation for regional and national expansion. Our growth strategy focuses on (1) advancing our relationship with NASA, (2) using our extensible career roleplaying framework to design solutions for

a wide range of Federal and industry sponsors, (3) integrating our educational effort into regional economic and workforce efforts, and (4) building capacity for the Department of Energy Real World Design Challenge.

We will advance the NASA Langley relationship using a Space Act Agreement. Under this partnership we will cultivate our relationship to further serve the career education, outreach and workforce development needs of Langley. Using this model relationship, we can serve the needs of other NASA Field Centers, as well.

IncredibleMinds integrates into the regional workforce and economic development of Southeastern Virginia. We have chosen to launch IncredibleMinds in the Hampton Roads region because of this area’s modeling and simulation regional cluster. This cluster also affords us easy access to modeling and simulation companies; federal research and development centers; and community colleges and universities that provide classes in modeling and simulation. We can leverage and transfer this expertise to the advantage of youth trapped in the worst school system in the nation: Washington, D.C.

On a national scale, IncredibleMinds is designed to become the persistent, afterschool capacity-building program for the Department of Energy’s Real World Design Challenge. Our goal is to prepare youth and teachers to excel in this annual competition. If we are to enhance the skills of youth so that we can advance industry, then each year students will need to solve more complex challenges. They practice and perfect their skills, knowledge, and abilities solving real problems using real tools in the IncredibleMinds Program. Then, during the annual Real World Design Challenge, they compete as well-honed design teams. During the 2008-09 school year the Real World Design Challenge is being rolled out in California, Connecticut, Hawaii, Kansas, Massachusetts, Minnesota, Oklahoma, Pennsylvania, Vermont, Virginia and Washington.

Expanding Regionally and Nationally

17

This strategy provides the foundation for pursuing short- and long-term funding opportunities with NASA and the federal laboratories of the DOD, DOE, DHS, EPA, NIH and NOAA. We will use a sponsorship model to recruit major corporations in verticals such as advanced manufacturing; aerospace and aviation; automotive and transportation; defense; biotechnology and medical devices; chemical; electronics and computer technologies; entertainment; petroleum and energy; pharmaceuticals and life sciences; and telecommunications.