Academy of Management Panel Symposium Proposal #15313

Academy of Management Panel Symposium Proposal

title:

Does Entrepreneurship Teaching/Training Actually Work?

A Symposium on Developing Metrics

Sponsors:

Lead: ENT Division

Also relevant: Teaching (TTC), Practice (PTC), TIM

Chair/organizer: Norris Krueger

Participating countries/programs: [I = invited]

Finland: Aalto University

Sweden: Chalmers University of Technology

Global: Entrepreneurs Organization (EO):

Netherlands: University of Twente

Denmark: Danish Foundation for Entrepreneurship

Denmark/EU: PACE/Aarhus University

Malaysia/USA: Stevens Institute of Technology/UKM

Discussant/Provocateur: Jon Potter, OECD Office of Entrepreneurship [I]

Discussant/Provocateur: Thom Ruhe, Ewing Marion Kauffman Foundation [I]

Discussant/Provocateur: Dianne HB Welsh, UNC-Greensboro, Technology Transfer

Society

Abstract

Does entrepreneurship education work? More important, how can we make it work? And how

can we measure the impact? This symposium focuses on measuring rigorously the impact of

entrepreneurial education in ways that we have rarely seen in single papers, let alone bringing

together some of the best programs in the world. These speakers are all from deeply experiential

programs and almost all from programs whose experiential activities center intensively around

technology commercialization. If scholars and educators want to know how to “move the needle”

1

Academy of Management Panel Symposium Proposal #15313

in turning ideas into reality, this panel will show both a set of programs doing enviable work and

some fascinating, robust tools for measuring the impact of experiential entrepreneurship

education.

Creating viable new businesses AND creating deep entrepreneurial thinkers? These

panelists will show you the state of the art of what we know about doing that and the immense

potential for future research on how do we grow the expert entrepreneurial mindset and measure

that growth.

2

Academy of Management Panel Symposium Proposal #15313

Overview

Research into entrepreneurial learning and entrepreneurial education has increasingly

recognized that to be effective requires transformative learning at a very deep level. The

literature on how we learn to become experts (e.g., Gladwell’s recent book, Outliers) reinforces

that one cannot learn to think like an expert without dramatic changes in how we organize what

we know. Knowing more is simply not enough; experts organize their knowledge far differently,

often in ways that are clearly not obvious to novices. Entrepreneurship education and training is

no different.

Why Assess Impact? In a world that increasingly recognizes the need for increases in

entrepreneurial thinking to support economic growth, it seems imperative that we identify what

methods are most effective at this transformational learning. It is equally imperative that we

identify what methods are less effective, even counterproductive. We propose here to integrate

existing theoretical and especially empirical field work to help us to (a) understand better why

and how our best pedagogies work, (b) why and how other pedagogies do not, and (c) create

mechanisms to allow us to map pedagogical methods to corresponding deep cognitive changes.

But we need good metrics.

The implications from great metrics? Imagine the ability to match pedagogical

exercises to learning needs –at a deep transformative level. If education research in general is

any predictor, we will eventually be able to not only identify which of our tools serve to make

which specific cognitive changes. For example, if trainees or students are weak at counterfactual

thinking (per Gaglio) what exercises are most effective at providing the needed cognitive scripts

and maps.

We all want to believe that entrepreneurship teaching and training matters. Matters

deeply. However, metrics have been in short supply, especially when we move past tests of

knowledge and skills. Several programs around the globe are tackling this with rigorous research

projects that simultaneously provide great practical value. We will be able to improve

entrepreneurial training but we first need to take a rigorous look at the cognitive impact of

entrepreneurship training methods. This PDW begins that effort in earnest with four relatively

new projects from the Netherlands (with tech entrepreneurs in an incubator), Denmark (youth

3

Academy of Management Panel Symposium Proposal #15313

entrepreneurs), Sweden (technology commercialization students) and globally (the renowned

Entrepreneurs Organization's peer learning groups.

We offer here short examples of the research projects attached to each initiative, however,

the PDW presentations will focus on these as opportunities:

Opportunities to identify tools that the audience can take home

Opportunities for the audience to offer advice to the participants

Opportunities to identify additional collaborations and possible extensions.

To this end, we have brought together several of the best initiatives existing today from well-

established programs like Chalmers to a brand-new project in Malaysia. We have invited expert

discussants from OECD and the Kauffman Foundation.

We fully intend for this to become a community of practice – we also want to learn what others

are doing as well as share the very best of what is out there.

4

Academy of Management Panel Symposium Proposal #15313

Sponsor Rationales

Why MED?

Does entrepreneurship education work? More important, how can we make it work? And how

can we measure the inpact? This symposium focuses on measuring rigorously the impact of

entrepreneurial education in ways that we have rarely seen in single papers, let alone bringing

together some of the best programs in the world. These speakers are all from deeply experiential

programs and almost all from programs whose experiential activities center intensively around

technology commercialization. If scholars and educators want to know how to “move the needle”

in turning ideas into reality, this panel will show both a set of programs doing enviable work and

some fascinating, robust tools for measuring the impact of experiential entrepreneurship

education.

Why ENT?

Does entrepreneurship education work? More important, how can we make it work? And how

can we measure the inpact? This symposium focuses on measuring rigorously the impact of

entrepreneurial education in ways that we have rarely seen in single papers, let alone bringing

together some of the best programs in the world. These speakers are all all from deeply

experiential programs and almost all from programs whose experiential activities center around

technology commercialization. If scholars and educators want to know how to “move the needle”

in turning ideas into reality, this panel will show both a set of programs doing enviable work and

some fascinating, robust tools for measuring the impact of experiential entrepreneurship

education.

Why TIM?

Technology commercialization-based entrepreneurial learning – we are looking deeply here.This

symposium focuses on measuring rigorously the impact of entrepreneurial education, all from

deeply experiential programs and almost all from programs whose experiential activities center

around technology commercialization. If scholars and educators want to know how to “move the

needle” in turning ideas into reality, this panel will show both a set of programs doing enviable

work and some fascinating, robust tools for measuring the impact of experiential

entrepreneurship education.

5

Academy of Management Panel Symposium Proposal #15313

Why PTC?

This symposium focuses on measuring rigorously the impact of entrepreneurial education in

ways that we have rarely seen in single papers, let alone bringing together some of the best

programs in the world. These speakers are all from deeply experiential programs and almost all

from programs whose experiential activities center around technology commercialization. If

scholars and educators want to know how to “move the needle” in turning ideas into reality, this

panel will show both a set of programs doing enviable work and some fascinating, robust tools

for measuring the impact of experiential entrepreneurship education.

Why Strategic Doing Initiative?

Creating viable new businesses AND creating deep entrepreneurial thinkers? This symposium

focuses on measuring rigorously the impact of entrepreneurial education, all from deeply

experiential programs and almost all from programs whose experiential activities center around

technology commercialization. If scholars and educators want to know how to “move the needle”

in turning ideas into reality, this panel will show both a set of programs doing enviable work and

some fascinating, robust tools for measuring the impact of experiential entrepreneurship

education.

6

Academy of Management Panel Symposium Proposal #15313

Key Presenters

University of Twente VentureLab -Aard Groen [presenter], Jeroen Kraajibrink, Gabi Kaffka

Measuring impact of entrepreneurship support in an integrated high-tech pre-incubator

Initiated and executed by the Nikos Institute of the University of Twente, VentureLab is a

pre-incubator facility at which entrepreneurs – even without business idea – are facilitated in

starting up their high-tech venture. It offers an intensive and integrated training and coaching

program, aimed at developing the necessary competencies to create a high-technology, high-

growth company. It also provides entrepreneurs office facilities and access to newly developed

technologies, venture capitalists and relevant networks of (international) companies and

scientists.

Measuring the impact of an integrated program such as VentureLab comes with at least

three major challenges. The first is that there is a distinction between the impact on the

entrepreneur and the impact on the venture. Hence, in measuring impact there are at least two

dependent variables. Second, the most important impact may only become visible after a delay of

a number of years – when we can see to what extent the entrepreneurs and ventures have grown

successfully. Third, because entrepreneurs do not operate in isolation and because they receive an

integrated offer, it is challenging to measure the individual impact of parts of the program.

To address these three challenges, VentureLab Twente is set up as a quasi-experiment

with extremely rich forms of data collection throughout and after the program. The following

data are collected:

Dependent variable: entrepreneur’s and venture performance:

1. Three four-monthly panel presentations in which an independent business panel assesses

the quality of the entrepreneurs and their venture.

2. Three four-monthly surveys in which entrepreneurs report about their individual progress

and the progress of their venture.

3. Yearly follow-up surveys and Chamber of Commerce data to assess venture growth and

performance.

Independent variables: attributing impact to elements of the program:

4. Attendance of trainings and coach meetings in order to assess the extent to which

entrepreneurs have actually participated.

7

Academy of Management Panel Symposium Proposal #15313

5. Training evaluations in which entrepreneurs give their opinion on a particular training in

terms of quality and usefulness.

6. Weekly diary in which entrepreneurs mention what they have learned in the past week,

and the results achieved in that week.

7. Exit interviews and focus groups in which entrepreneurs reflect on the quality and

usefulness of the program.

Control variables:

8. Intake interview and survey in which entrepreneurs report about their experience,

ambitions, capabilities and personality.

Since its inception in 2009, 200 entrepreneurs have entered the VentureLab program. This has

resulted in a voluminous and rich data set that forms the basis for various PhD projects. Research

is currently ongoing and the first results on impact are expected to be available shortly.

Danish Foundation for Entrepreneurship – Young Enterprise, Kare Moberg

Impact of Experiential Learning on Young Entrepreneurs – from ABC to PhD

Different educational levels have different learning goals. This must obviously be taken into

account when it comes to evaluation projects. Still, there is a need for comparability between

evaluation projects that target these different levels. The Danish Foundation for Entrepreneurship

– Young Enterprise has launched a major research project which measures the effects of

entrepreneurship education at basic, secondary as well as tertiary level.

At basic level the focus is on the learning process, in particular how entrepreneurship education

affects if the students internalize and take responsibility for their learning process, and how this

in turn affects their connectedness to school, classmates, educators and society. The survey is

based on two rounds of data collection. In all 1312 randomly selected 10th graders (574) and 9th

graders (738), are included in the analysis. The effects of two approaches to entrepreneurship

education are compared: the cognitive skill oriented “Entrepreneurship as a tradesmanship”

approach and the non-cognitive skill oriented “Entrepreneurship as a method” approach.

8

Academy of Management Panel Symposium Proposal #15313

The first has a positive effect of entrepreneurial intentions but its effects on school attachment

are negative; the opposite is true for the latter. At secondary and tertiary levels the evaluation

projects are mainly based on the effect that entrepreneurship education has on entrepreneurial

self-efficacy (ESE), and how this affects entrepreneurial behaviours. The evaluations are set up

as quasi experiments with tests before, during and after the education. At university level, 15

master-programmes are included in the sample, 7 in the control group and 8 with a strong focus

on entrepreneurship. Two rounds of data have been collected with a total of 1256 responses. This

allows us to analyze the effects that different educational designs have on different ESE

dimensions, in particular regarding their focus on the creation and discovery approaches.

In order to do this, however, we had to develop a new ESE scale which is not jargon biased

(possible to understand for students in the control group), and which covers five different skill-

sets: creativity, planning, marshaling of resources, financial literacy and managing ambiguity.

Chalmers University of Technology Martin Lackeus [presenter], co-authors: Karen Middleton

Williams, Mats Lundqvist

Measuring Entrepreneurship Education Performance

The School of Entrepreneurship at Chalmers University of Technology in Gothenburg,

Sweden has since 1997 championed an action-based approach to technology entrepreneurship.

Classes of about 30 selected student per year (approximately 50% from technology/science and

50% from business) work with real venture creation during their second year in teams of three,

trying to develop IP from technology transfer into a viable start-up.

  Over the years, performance measurement of this and other educational venture creation

programs has focused on startup outcome or on individual level learning (through questionnaires

or more interactive evaluations). Although there arguably are significant effects coming from real

ventures being created – learning-wise and other-wise, you could also argue that outcome-based

measurement in combination with traditional individual-level assessments could and should be

complemented by other learning categories, such as team-level based and context-based (i.e. how

much students interacts with and draws from context). On the individual level, there is also

9

Academy of Management Panel Symposium Proposal #15313

reason to not only measure traditional know-how or know-what factors in venture creation

programs. Rather, know-why can arguably be the most important learning outcome for nascent

entrepreneurs to figure out – a concept related as much to emotionality and action as it is to

traditional cognitive learning.

This session draws upon 15 years of experience from and research around action-based

entrepreneurship education that among other things have produced a quarter of all the revenue

that university incubated technology ventures generated 2010 in Swedish. Measurements in four

categories will be addressed – the individual-based, group-based, context-based and outcome-

based. Focus will be on evaluating the development of entrepreneurial competence and

entrepreneurial ecosystems and on the appreciation of “know-why” in entrepreneurship.

At Chalmers, we investigate links between strong emotions and entrepreneurial learning

outcomes in a formal learning environment consisting of an action-based entrepreneurship

education program. Students’ own experiences were quantitatively and qualitatively assessed

during their participation in an entrepreneurship program where they were expected to start a real

venture as formal part of curriculum. This kind of learning environment has previously been

characterised as an emotional roller coaster with transformative learning experiences frequently

reported, due to the learning environment’s capacity to approximate the process of starting a real

venture quite accurately. It thus represents a rare opportunity to conduct laboratory studies on

nascent entrepreneurs. We continue to build on previous work on these so-called venture

creation programs (e.g., Babson paper in 2011). We now also ask: How are emotional amplitude

and entrepreneurial learning outcomes linked?

Entrepreneurs Organization Lesley Hayes, EO & University of Athabasca

Entrepreneur – Peer Learning Groups.

A number of entrepreneurial organizations offer international entrepreneurial support

systems – i.e. the Entrepreneur’s Organization with over 8,000 members globally running

businesses over $1MM USD in annual revenue. Their primary entrepreneurial learning structure

is a version of a peer-learning network popularized by Napoleon Hill - a ‘mastermind group’.

Underlying many current efforts to ‘educate’ entrepreneurs is the belief that

entrepreneurial cognition – or decisions - can be improved with training, skills & knowledge, and

these improvements lead to improvements in marketplace performance (Matlay, 2008).

10

Academy of Management Panel Symposium Proposal #15313

Knowledge is conceptualized as a transferrable, retainable and re-useable asset (Macpherson and

Holt, 2007). The emerging field of entrepreneurship research is littered with disagreements

regarding the source of entrepreneurial skills and successes as being primarily environmental,

political, biological, personality, skill or uniquely individual.

Traditional models of teaching are often a expert lecturer controlling a highly structured

learning environment; determining where and when learning occurs. However, research

indicates entrepreneurs learn best from experience, reflection, problem solving, and peer

interaction (Gibb, 1993; Deakins and Freel, 1998; Politis and Gabrielsson, 2007). EO & other

organizations offer programs that support entrepreneurs to learn from their own experiences.

This discussion will delve into the structures, language protocols, demographic and psychometric

variables which may be controlled for or influenced which can increase the potential for

generative (double-loop) entrepreneurial learning to occur.

There is minimal data collection from within these current groups, although one study by

Joakim Tell, “The emergent nature of learning networks’ (2008), did examine a learning network.

What are the best practices from the ‘60s and ‘70’s regarding optimum number of participants,

topics, structures, and other practices that influence these groups usefulness? Does group

dynamics, team cultures, social identity, social networks, or weak and strong ties, provide

insight, co-vary or predict into variables such as trust, cohesion, leadership and confidence.

What are the ways that we can support and measure Smilor’s statement “Effective entrepreneurs

are exceptional learners…they learn from other entrepreneurs, they learn from experience and

they learn by doing” (Smilor, 1997: 344)

Format:

As we are sharing here multiple overlapping but unique projects (at different stages of

development) we opt for the panel symposium format. We want to ensure that the audiences gets

enough time to:

(a) offer advice to one or more of the projects,

(b) identify ways to adapt one or more of these to their own interests and

c) share their own research opportunities around this theme.

It is clear that universities are eager to assess impact and the process of doing that has grown

immensely in quality… but it can only get better!

11