Joe ValacichWashington State UniversityAugust 3, 2003

jsv@wsu.edu

Developing a Research Stream

Agenda

Some General Career AdviceThe Research ProcessDeveloping a Research StreamDeveloping TheoryDesigning Studies – Getting the Most from

Your Data Collection Effort!Writing It UpTop-Ten Ways to be RejectedSome words to live by

Some General Career Advice

Lessons I learned…

Attend conferences, be proactive, get to know the “players” – IS disciple is dominated by a small number of people

Hit the ground running!!! – tenure is a 4 to 5 year process, not 7!

Have a sense of urgency!!! Become “known” for something – focus Always work on the paper closest to being

published Make it a family effort – celebrate success, find

balance! Research should be fun!

How to Become Valuable to Your School

Work hard / be availableFocus your research Become a good teacher / advisor / colleagueVolunteer for hard assignmentsVolunteer to review Involve yourself in curricula developmentWork with senior colleagues (take the lead)Develop / Talk / Listen / Involve Industry

Contacts

How to Become Valuable to Some Other School

DO EVERYTHING LISTED ON THE PRIOR SLIDE!!!

In general…

Work hardFind a balanceJuggle multiple projects at the same

timeBe proactiveDELIVER

The Research Process

The Research Process

… a series of interlocking choices in which we try simultaneously to maximize several

conflicting choices…

Key choices generalizability with respect to populations realism for the participants precision in control and measurement of variables

McGrath, 1982

The Research Process

Unfortunately, the very choices used to maximize generalizability, realism, or control will reduce the other two…

In other words, all research strategies and methods are seriously flawed…

Thus, every research strategy is a three-horned dilemma…

The Three-Horned Dilemma

…every research strategy either avoids two horns by an uneasy compromise but gets impaled, to the hilt, on the third horn...

or it grabs the dilemma boldly by one horn, maximizing on it, but at the same time “sitting down” (with some pain) on the

other two horns...

Eg., Laboratory Experiments and the Three-Horned Dilemma

Generalizability Very low (typically) controlled and selected population

Realism Very low (typically) deliberately contrived setting

Precision high (the whole idea behind a lab experiment!)

Play to a method’s strength

Laboratory experiment: precisionSurvey: generalizabilityField study: realism

Developing a Research Stream

Key Skills

Research requires three skills: Conceptualizing Theory Research Design Writing

Build a Team with all Three

What is a Good Research Project?

Asks new questions or old questions in a new way or in a new situation leading to different answers

Has a story with a message that is interesting to the AE, reviewers, and readers

Is the first step/next step in a research stream

Fits your research risk/return portfolio

What is a Good Research Project?

Do research that is funDo research to be publishedDo research that addresses a

fundamental issueDo research that is interesting

whatever the result

From a Personal Level:

What is a Good Research Project?

Write to be read

Write to be cited

Write to change practice

Publishing the Results:

What is a Good Research Project?

The definition of a Nobel prize in Physics: “Oh shit, why didn’t I think of that?”

Simple ideas are ideal

Scientific discovery does not start with the word “Eureka.” It starts with the words “That’s funny.”

Investigate anomalies

Sources of Research Projects(Garbage Can Model of Research)

Theory

Methods

Personal Experience

Resources

Previous Research

Developing Theory

What is a Good Theory

Theory is the why of the study

Theory is not a summary of prior research

What is a Good Theory

A good theory explains the relationships among a set of constructs

And explains why those relationships exist

Big “T” theory vs. Little “t” theory

Classic Approach

Theory is conceptual, abstract and tentative; based on prior theory and empirical results.

Three steps… define concepts and write propositions that

state the relationship between them devise testable hypotheses and methods to

measure concepts gather and analyze data in an attempt to

verify hypotheses

Designing Studies

Getting the Most from Your Data Collection Effort

Think 1, 2, … n studies aheadStick to a theme (don’t change

topics until you’ve played it out) reusability: theory, measures, learning

Share Cells Example...

Study 1: Individuals vs. Face-to-Face Groups

IndividualFace-to-Face Group

Baseline

ObjectiveDevil’s Adv.

DialecticalInquiry

Study 2: Face-to-Face vs. CMC Groups

Face-to-Face Group

CMC Group

Baseline

ObjectiveDevil’s Adv.

DialecticalInquiry

Study 3: Objective vs. Carping Devil’s Advocacy

IndividualFace-to-Face Group

CMC Group

ObjectiveDevil’s Adv.

CarpingDevil’s Adv.

Sharing Cells

1 1, 2 2

IndividualFace-to-Face Group

CMC Group

Baseline

ObjectiveDevil’s Adv.

1 1, 2 2

1, 3 1, 2, 3 2, 3

3 3 3

DialecticalInquiry

CarpingDevil’s Adv.

Designing for Statistical Significance (MaxMinCon)

Maximize the difference of the treatment means

(Kerlinger, 1986)

t = X1 - X2

s / n

Minimize the error variance

Control systematic variance

Executing Studies

Pilot Test, Pilot Test, Pilot Test Measures Manipulations

Recruiting Subjects Incentives Overbooking

Random Assignment to TreatmentsConfiguring the Environment

Double check everything

Writing It Up

Standard Structure

(be innovative in content not style)

Introduction: 1-2 pagesTheory: 7-10 pagesMethod: 3-5 pagesResults: 2-3 pagesDiscussion: 7-12 pages

50-60%

40-50%

Spin

(the data never speak for themselves)

Find the message (“unique selling

proposition”)Find new theory if necessaryFind metaphors for key ideas

Presentation

(writing sells ideas)

Find exemplar paper(s)Know your audience (reviewers)Stay “on point” (discard good ideas

that are not essential)No surprises (you’re not writing a

mystery novel)Avoid amateur style (quotes,

strawmen, exaggeration, negativism, old papers, dissertations)

Implications

(go beyond your data)

Present implications for practice (what would you tell your class to do

or not do?)Speculate for future research

(why did these results occur, boundary conditions, next studies needed)

Cultivation

(no paper is final until it is in print)

Solicit opinions widely“Test market” and refine the

message at conferencesWork with the reviewers and AEs

to improve the paper

Top-Ten Ways to Be Rejected

Top-Ten Ways to be Rejected(Theory Development)

1. Avoid theory in favor of empirical results

2. Miss key papers in your literature review and theory development

3. Include many red herrings4. Plagiarize from the reviewers’

articles

Top-Ten Ways to be Rejected(Study Design)

5. Openly and directly criticize the work of others.

6. Theorize one set of constructs and measure another

7. Describe your methodology in vague terms8. Fail to admit the faults of your research.9. Forget the lessons of the three-horned

dilemma! In other words, try to design one study that has high realism, generalizability, and control… This cannot be done!

Top-Ten Ways to be Rejected(Writing it Up)

10. Draw conclusions that differ from your results11. Submit a paper longer than 35 pages12. Include numerous typographical errors and

fail to format your paper to the journal’s editorial standards

13. Write obscurely and repetitively14. Avoid sharing your ideas with colleagues15. Respond to the reviewers with a one page “We did what you said”16. Have 16 items in your top ten list

Words to Live (work) By

“Your career is a marathon, not a sprint race.”

Terry Connolly, 1988

“In this career, it is OUTCOMES, not PROCESS, that matters.”

E.W. Martin, 1990

“It is far easier to MAINTAIN a reputation than it is to GAIN a reputation.”

J.F. Nunamaker, 1989

Words to Live (work) By

“For tenure decisions… if it is close… it is NO!”

Dan Dalton, 1994

“Your publication record is you primary PORTABLE wealth…”

Paul Gray, 1995

“Every paper that is not fundamentally flawed can find a home.”

Joe Valacich, 2000

Conclusion

Focus, focus, focusGet the theory right, first!Pilot testReuse, reuse, reuse…Have Fun!

For more information…