ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The...

30

Transcript of ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The...

Page 1: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd
Page 2: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

ffirs.indd iiffirs.indd ii 07/06/12 2:13 PM07/06/12 2:13 PM

Page 3: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

THE HANDBOOK OF INSTITUTIONAL RESEARCH

ffirs.indd iffirs.indd i 07/06/12 2:13 PM07/06/12 2:13 PM

Page 4: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

ffirs.indd iiffirs.indd ii 07/06/12 2:13 PM07/06/12 2:13 PM

Page 5: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

THE HANDBOOK OF INSTITUTIONAL RESEARCH

Richard D. Howard, Gerald W. McLaughlin, William E. Knight, and Associates

ffirs.indd iiiffirs.indd iii 07/06/12 2:13 PM07/06/12 2:13 PM

Page 6: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

Copyright © 2012 by John Wiley & Sons, Inc. All rights reserved.

Published by Jossey-Bass

A Wiley Imprint

One Montgomery Street,

Suite 1200

San Francisco, CA 94104-4594

www.josseybass.com

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any

means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under

Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the

publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center,

Inc., 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400, fax 978-646-8600, or on the Web at www.copyright

.com. Requests to the publisher for permission should be addressed to the Permissions Department, John Wiley &

Sons, Inc., 111 River Street, Hoboken, NJ 07030, 201-748-6011, fax 201-748-6008, or online at www.wiley.com/go/

permissions.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in

preparing this book, they make no representations or warranties with respect to the accuracy or completeness

of the contents of this book and specifi cally disclaim any implied warranties of merchantability or fi tness for a

particular purpose. No warranty may be created or extended by sales representatives or written sales materials.

The advice and strategies contained herein may not be suitable for your situation. You should consult with a

professional where appropriate. Neither the publisher nor author shall be liable for any loss of profi t or any other

commercial damages, including but not limited to special, incidental, consequential, or other damages. Readers

should be aware that Internet Web sites offered as citations and/or sources for further information may have

changed or disappeared between the time this was written and when it is read.

Jossey-Bass books and products are available through most bookstores. To contact Jossey-Bass directly call our

Customer Care Department within the U.S. at 800-956-7739, outside the U.S. at 317-572-3986, or fax 317-572-4002.

Wiley also publishes its books in a variety of electronic formats and by print-on-demand. Some material included

with standard print versions of this book may not be included in e-books or in print-on-demand. If the version of

this book that you purchased references media such as CD or DVD that was not included in your purchase, you

may download this material at http://booksupport.wiley.com. For more information about Wiley products, visit

www.wiley.com.

Library of Congress Cataloging-in-Publication Data

The handbook of institutional research / [edited by] Richard D. Howard, Gerald W. McLaughlin,

William E. Knight. — First edition.

pages cm. — (The Jossey-Bass higher and adult education series)

Includes bibliographical references and index.

ISBN 978-0-470-60953-8; ISBN 978-1-118-25901-6 (mobipocket);

ISBN 978-1-118-23451-8 (epub); ISBN 978-1-118-22074-0 (pdf)

1. Education, Higher—Research—Handbooks, manuals, etc. I. Howard, Richard D. II. McLaughlin,

Gerald W. III. Knight, William E., 1965–

LB2326.3.H36 2012

378.007—dc23

2012016806

Printed in the United States of America

first edition

HB Printing 10 9 8 7 6 5 4 3 2 1

ffirs.indd ivffirs.indd iv 07/06/12 2:13 PM07/06/12 2:13 PM

Page 7: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

The Jossey-Bass Higher and Adult Education Series

ffirs.indd vffirs.indd v 07/06/12 2:13 PM07/06/12 2:13 PM

Page 8: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

ffirs.indd viffirs.indd vi 07/06/12 2:13 PM07/06/12 2:13 PM

Page 9: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

vii

CONTENTS

Tables, Figures, and Exhibits xi

Appendices xvii

Preface xix

Acknowledgments xxiii

About the Editors xxv

List of Contributors xxvii

The Association for Institutional Research xxxi

PART 1: THE HISTORY, THEORY, AND PRACTICE OF INSTITUTIONAL RESEARCH 1

1 The History of Institutional Research 3

Donald J. Reichard

2 The Structure and Functions of Institutional Research Offi ces 22

J. Fredericks Volkwein, Ying (Jessie) Liu, and James Woodell

3 Practicing Institutional Research 40

Mardy T. Eimers, Jang Wan Ko, and Denise Gardner

4 The Role of Institutional Research in International Universities 57

Ann S. Ferren and Martha C. Merrill

ftoc.indd viiftoc.indd vii 28/07/12 3:45 PM28/07/12 3:45 PM

Page 10: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

5 Out of the Box and Out of the Offi ce: Institutional Research for Changing Times 73

Richard A. Voorhees and Teri Hinds

6 Institutional Research and Collaborative Organizational Learning 86

Victor M. H. Borden and Adrianna Kezar

7 Developing Institutional Adaptability Using Change Management Processes 107

Kim Bender

PART 2: SUPPORTING CAMPUS LEADERSHIP AND MANAGEMENT 131

8 Supporting Institutional Governance 133

James Purcell, Charles Harrington, and Beverly King

9 Supporting the Provost and Academic Vice President 145

James T. Posey and Gita Wijesinghe Pitter

10 Examining Faculty Recruitment, Retention, Promotion, and Retirement 165

Daniel Teodorescu

11 Institutional Planning and Resource Management 183

Michael J. Dooris and Jerome S. Rackoff

12 Building Cost Models 203

John Milam and Paul Brinkman

13 Managing College Enrollments 221

John Cheslock and Rick Kroc

14 Refocusing Student Success: Toward a Comprehensive Model 237

Gary A. Rice and Alene Bycer Russell

15 Academic Space Management and the Role of Institutional Research 256

Catherine Watt

16 Managing Sustainability 268

Josetta S. McLaughlin and Lisa M. Amoroso

PART 3: BRIDGING INTERNAL AND EXTERNAL REQUIREMENTS FOR IR 295

17 Challenges in Meeting Demands for Accountability 299

Carol Fuller, Cathy Lebo, and John Muffo

viii Contents

ftoc.indd viiiftoc.indd viii 28/07/12 3:45 PM28/07/12 3:45 PM

Page 11: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

Contents ix

18 Accreditation and the Changing Role of the Institutional Researcher 310

Paula S. Krist, Elizabeth A. Jones, and Kimberly Thompson

19 Regulated Ethics: Institutional Research Compliance with IRBs and FERPA 325

Rachel Dykstra Boon

20 Data, Discrimination, and the Law 340

Andrew L. Luna

21 Federal Higher Education Reporting Databases and Tools 354

Gayle M. Fink and Chad Muntz

22 Collective Responses to a New Era of Accountability in Higher Education 371

Christine M. Keller

23 System- and State-Level Data Collection Issues and Practices 386

Marsha V. Krotseng

24 Developing K–20+ State Databases 404

Maryann S. Ruddock

25 Data Exchange Consortia: Characteristics, Current Examples, and Developing a New Exchange 420

Julie Carpenter-Hubin, Rebecca Carr, and Rosemary Hayes

26 Business Intelligence and Analytics: The IR Vision for Data Administration, Reporting, Data Marts, and Data Warehousing 434

John Milam, John Porter, and John Rome

PART 4: INSTITUTIONAL RESEARCH TOOLS AND TECHNIQUES 455

27 Analytic Approaches to Creating Planning and Decision Support Information 459

Gerald McLaughlin, Richard Howard, and Daniel Jones-White

28 Exploring and Mining Data 478

Jing Luan, Thulasi Kumar, Sutee Sujitparapitaya, and Tom Bohannon

29 Measuring Opinion and Behavior 502

Sean Simone, Corbin M. Campbell, and Daniel W. Newhart

30 Institutional Research with Published Instruments 523

Julie Noble and Richard Sawyer

ftoc.indd ixftoc.indd ix 28/07/12 3:45 PM28/07/12 3:45 PM

Page 12: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

x Contents

31 Measuring and Evaluating Faculty Workload 550

Heather A. Kelly, Jeffrey A. Seybert, Patrick M. Rossol, and Allison M. Walters

32 Analyzing Equity in Faculty Compensation 573

Robert K. Toutkoushian and Dennis A. Kramer II

33 Effective Reporting 594

Liz Sanders and Joe Filkins

34 Tools for Setting Strategy 611

Jan W. Lyddon, Bruce E. McComb, and J. Patrick Mizak

35 Tools for Executing Strategy 625

Jan W. Lyddon, Bruce E. McComb, and J. Patrick Mizak

36 Developing Institutional Comparisons 644

Glenn W. James

37 Tools for Improving Institutional Effectiveness 656

Jonathan D. Fife and Stephen D. Spangehl

38 Tools for Measuring the Effectiveness of Institutional Research 673

Sharron Ronco, Sandra Archer, and Patricia Ryan

Epilogue 689

Name Index 695

Subject Index 707

ftoc.indd xftoc.indd x 28/07/12 3:45 PM28/07/12 3:45 PM

Page 13: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

xi

TABLES, FIGURES, AND EXHIBITS

List of Tables

Table 2.1 Highest Degree and Years Experience

Table 2.2 The Four Faces of Institutional Research

Table 6.1 The Tenets of Institutional Research: Shifting from a Decision Support to a Collaborative Organizational Learning Ethos

Table 7.1 Summary of Change Management Characteristics, Theories, and Models

Table 7.2 Planning and Evaluation Effectiveness: Annual Report for 2003–2004

Table 10.1 Tenure Track Faculty Recruitments and Success Rates, Fall 1999–2009

Table 10.2 Largest Gaps Between Importance and Satisfaction for Infl uences on Faculty Careers

Table 10.3 Sample Template for Tracking Tenure/Promotion Denials

Table 10.4 Age Distribution for Full-Time Faculty at Four-Year Institutions, 2003

Table 10.5 Projected Proportions of Faculty: Retired and Eligible to Retire

Table 12.1 Step 1: Get SCH Breakout with ICLM

Table 12.2 Step 2: Convert to Percentages

Table 12.3 Step 3: Allocate Budget Data Based on SCH

ftoc.indd xiftoc.indd xi 28/07/12 3:45 PM28/07/12 3:45 PM

Page 14: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

xii Tables, Figures, and Exhibits

Table 12.4 Step 4: Sum Breakout Cells for Subtotals

Table 14.1 Learning Goal Status After Ten Years: UAA First-Time Undergraduates, Fall 1998 Entry Cohort

Table 16.1 GRI Indicators

Table 16.2 STARS Ratings and Recognition System

Table 16.3 STARS 1.2 Table of Credits

Table 16.4 Example of LEED Point System for Commercial Interiors

Table 24.1 National Perspectives on Elements of Statewide Longitudinal Databases

Table 24.2 Role and Sophistication of Stakeholders as Determinants of Data Access

Table 27.1 Key Questions: Constructivist Versus Positivist

Table 28.1 Cluster Means of Student Engagement Typology

Table 28.2 Predictors Included in This Research Model

Table 28.3 Predictors Selected for Neural Network Model

Table 29.1 Questions an Institutional Researcher Should Address When Designing a Study

Table 29.2 Best Practices for Developing Questionnaire Items, with Examples

Table 29.3 Response Scales

Table 29.4 Survey Design Map

Table 29.5 Types of Sampling Procedures

Table 30.1 Undergraduate Admission Tests

Table 30.2 Graduate/Professional School Admission Tests

Table 30.3 Undergraduate Course Placement Tests

Table 30.4 Undergraduate ESL Course Placement Tests

Table 30.5 Undergraduate Honors Tests and Credit-by-Examination

Table 30.6 Psychosocial Instruments

Table 31.1 Kansas Study: Percent Student Credit Hours Taught by Faculty Type and Academic Discipline, National Refined Means, Fall 2002

Table 31.2 Kansas Study: Instructional Cost Per Student Credit Hour—Most Expensive Disciplines

Table 31.3 Kansas Study: Instructional Cost Per Student Credit Hour—Least Expensive Disciplines

Table 32.1 Comparison of Average Faculty Salaries for the University of Georgia and Its Peers, 2006

ftoc.indd xiiftoc.indd xii 28/07/12 3:45 PM28/07/12 3:45 PM

Page 15: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

Tables, Figures, and Exhibits xiii

Table 32.2 Comparison of Average Faculty Salaries by Rank for the University of Georgia and Its Peers, 2006

Table 32.3 Change in Relative Salary Positions for Public Institutions in the University System of Georgia with Designated Peers, 2001–2006

Table 32.4 Cost of Raising Average Faculty Salaries for Selected Public Institutions in Georgia to the Median for Peers, 2006

Table 32.5 Selected Comparisons of Average Faculty Salaries by Gender, 2008–2009

Table 32.6 Illustration of Single-Equation Method for Measuring the Unexplained Wage Gap

Table 32.7 Illustration of Total Wage Gap Decomposition Using Multiple-Equation Methods

Table 34.1 PESTEL Analysis Table

Table 34.2 Competitive Analysis Questions

Table 34.3 Standard SWOT Matrix

Table 34.4 SWOT Matrix Organized by Balanced Scorecard Perspectives

Table 34.5 TOWS Matrix

Table 34.6 Ansoff Matrix

Table 34.7 Examples of Benchmarking Sources in Higher Education

Table 35.1 Balanced Scorecard Spreadsheet

Table 35.2 Sample Balanced Scorecard Display Using Desktop Software

List of Figures

Figure 2.1 IR Offi ce Size: FTE Professionals

Figure 2.2 The Golden Triangle of Institutional Research

Figure 2.3 IR Offi ce Organizational Location and Reporting Level

Figure 2.4 IR Organized by Major Function, Typical for the Professional Bureaucracy Version of IR

Figure 2.5 Calculation of IR Maturity

Figure 2.6 Calculation of Task Hierarchy Score

Figure 2.7 Average Task Hierarchy Scores for the Three Levels of IR Offi ce Maturity

Figure 6.1 Constructs and Processes Associated with Organizational Learning

Figure 7.1 The AQF Concept Model for Change Management Process

Figure 7.2 The Ten-Part AQF Process and Information Flow

ftoc.indd xiiiftoc.indd xiii 28/07/12 3:45 PM28/07/12 3:45 PM

Page 16: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

xiv Tables, Figures, and Exhibits

Figure 11.1 Import-Export Model: The Orienting Role of the Environment

Figure 13.1 Arizona Hispanic Population Pipeline: 1,880,000 Total Hispanics in 2008

Figure 13.2 Price Response and Student Interest

Figure 14.1 Ten-Year Map of the Fall 1998 First-Time Undergraduate Entry Cohort

Figure 15.1 Example of Code Use in a Space Database

Figure 15.2 Institutional Knowledge Phases

Figure 15.3 Individual Faculty Member’s Space Assignments in a Space Database

Figure 15.4 Individual Faculty Member’s Research Awards, Including Funding per Square Foot

Figure 15.5 Departmental Summary of Space

Figure 15.6 Elements of a Basic Web-Based Space Database

Figure 16.1 Sustainability Reporting Process

Figure 25.1 The Data Exchange Lifecycle

Figure 28.1 Drop Line of Cluster Means

Figure 28.2 Academic Advisement

Figure 28.3 Educational Experiences: Infl uence on Return to the Same Institution

Figure 28.4 Type Node and Data Audit Output

Figure 28.5 Rules with Instance and Confidence Figures and List of Variables Importance

Figure 28.6 Analysis Output of C5.0 and C&R Tree

Figure 28.7 Example of an MLP

Figure 28.8 Flow Process for Neural Network Analysis

Figure 28.9 ROC Curves for Decision Tree and Neural Network Models

Figure 29.1 Equations for Determining the Size of a Sample

Figure 30.1 Effectiveness of an Instrument for Identifying At-Risk Students

Figure 31.1 2010 Delaware Study Data Collection Form

Figure 31.2 Total Student Credit Hours Taught per FTE Tenured and Tenure Track Faculty for a Science Department

Figure 31.3 Faculty Activity Study Summary Form

Figure 31.4 Out-of-Classroom Faculty Activity Related to Service for a Science Department

Figure 34.1 SWOT and Ansoff Matrix

ftoc.indd xivftoc.indd xiv 28/07/12 3:45 PM28/07/12 3:45 PM

Page 17: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

Tables, Figures, and Exhibits xv

Figure 35.1 Perspectives for an Educational Institution Model

Figure 35.2 Strategy Map for a University

Figure 35.3 Cascading Alignment of Balanced Scorecards to Dashboards

Figure 37.1 Basic Elements of an Effective Organization

Figure 37.2 Cycle of Continuous Adaptation

Figure 37.3 Plan-Do-Study-Act

List of Exhibits

Exhibit 3.1 Terenzini’s Three Organizational Intelligences of Insti tutional Research

Exhibit 3.2 Myers-Briggs Type Indicator Dimensions: An Institutional Research Perspective

Exhibit 10.1 Typical Questions Related to Faculty Retention Analysis

Exhibit 10.2 Correlates of Job Satisfaction

Exhibit 15.1 Questions to Ask in Space Management Implementation

Exhibit 16.1 Sustainability-Related Codes for Use by Collegiate Institutions

Exhibit 35.1 IR Professionals’ Roles in Balanced Scorecards

Exhibit 38.1 Suggested Topic List for IR Self-Study

ftoc.indd xvftoc.indd xv 28/07/12 3:45 PM28/07/12 3:45 PM

Page 18: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

ftoc.indd xviftoc.indd xvi 28/07/12 3:45 PM28/07/12 3:45 PM

Page 19: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

xvii

APPENDICES

14.1 UAA Decision Rules for Student Status Determination

16.1 The Talloires Declaration

16.2 Calculating a STARS Score

16.3 Survey Questions for the Princeton Review Green Rating of Colleges

18.1 Accrediting Agencies’ Websites

19.1 FERPA Concerns to Address in Agreements with Third Parties for Disclosure of Identifi able Data for Educational Research

21.1 NCES Sample Surveys

21.2 IPEDS Surveys

21.3 NSF Surveys

22.1 Initiative Websites, Resources, and Reports

37.1 Resources

ftoc.indd xviiftoc.indd xvii 28/07/12 3:45 PM28/07/12 3:45 PM

Page 20: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

ftoc.indd xviiiftoc.indd xviii 28/07/12 3:45 PM28/07/12 3:45 PM

Page 21: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

xix

PREFACE

The purpose of this Handbook is to give practitioners, administrators, faculty

members, and students a single source that provides an overview of the

core aspects of institutional research—its history and theoretical foundations;

its role in support of planning and decision making; its role in responding to

external accountability mandates; the data sources built by, and available to,

institutional research professionals; and the tools and techniques used to study

our institutions and to inform all members of the academy.

The last comprehensive discussion of the theory and practice of institu-

tional research in higher education was more than 40 years ago. In 1971 Paul

Dressel, a professor at Michigan State University, worked with his associates to

write Institutional Research in the University: A Handbook, a volume that described

a burgeoning practice that was much more of an art than a profession, and a

newcomer to academic institutions. Since that time, the art described by

Dressel has turned into a profession, practiced in virtually every postsecond-

ary institution in the United States and at many others around the world. As a

profession, it has a published set of ethics and standards, organizations of prac-

titioners, an evolving language, and a unique and expanding knowledge base.

Over the years, as the profession has grown, there have been many publications

addressing various aspects of the practice of institutional research. However,

since Dressel’s publication in 1971 there has been no attempt to comprehen-

sively describe and discuss the concepts, processes, and methodologies used

in the practice of institutional research. That is why this Handbook was written.

Our fi rst task was to try to answer the perplexing question: What exactly is

institutional research? Although a number of defi nitions have been put forward

to describe the function of institutional research, Joe Saupe’s (1990) defi nition

fpref.indd xixfpref.indd xix 07/06/12 2:14 PM07/06/12 2:14 PM

Page 22: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

xx Preface

clearly and succinctly defi nes the profession: “Institutional research is research

conducted within an institution of higher education to provide information

which supports institutional planning, policy formation and decision making”

(p. 1). Earlier, in Asa Knowles’s Handbook of College and University Administration

(1971), John Stecklein outlined three fundamental purposes of institutional

research that we believe are still relevant today. Paraphrasing Stecklein, the

purposes of institutional research are: (1) provide service to faculty members,

(2) provide service to the administration, and (3) provide services to coordinat-

ing groups and other outside agencies (pp. 4–125). He also notes a wide range

of activities by which we provide these services, such as

• Providing a research basis for critical examinations of teaching procedures

and practices

• Creating a better understanding of the purpose of a course or curriculum

• Determining a basis for comparative judgments concerning instruction and

curriculum building

• Obtaining a better understanding of admissions practices, examinations

procedures, grading practices, and workloads

• Obtaining a better understanding of the role of the faculty member in the

administration of a college or university

• Developing a better understanding of the factors that infl uence costs

• Obtaining a better understanding of the way in which curricular decisions

can impact the use of resources such as space utilization, building costs, and

other routine operations

• Providing up-to-date statistics on the characteristics of the institution, iden-

tifying trends in any of these characteristics, providing data and information

useful in obtaining fi nancial support, and providing data useful in explain-

ing the mission and achievements of the institution

It is an amazingly broad list of activities and responsibilities. It is also very

similar to the list developed by Chambers and Gerek (2007). Adding to this

list, Randy Swing (2009), executive director of the Association for Institutional

Research, recently stated that institutional research professionals also need

to support and manage change on their campuses; IR practitioners need to

develop their technical skills, critical thinking and reasoning skills, understand-

ing of campus systems and divisional cultures, and management skills. A short

description of the Association is on page xxxi.

Overview and Organization of the Handbook

The Handbook is made up of four parts that describe the breadth of the core

knowledge and techniques that make up the profession of institutional research.

In organizing the topics covered in this volume, we used Pat Terenzini’s (1993)

conceptual model, which defi nes three tiers, or major types, of knowledge as a

fpref.indd xxfpref.indd xx 07/06/12 2:14 PM07/06/12 2:14 PM

Page 23: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

Preface xxi

framework for ordering the chapters. In the fi rst part, the authors set a context

for the practice of the profession of institutional research (Terenzini’s third tier

of knowledge: contextual intelligence or institutional wisdom), looking at the devel-

opment of the profession and its current practice, and offering some thinking

about future directions on our campuses. In the second part, Terenzini’s sec-

ond tier of knowledge, issues intelligence, is refl ected in a series of chapters that

address specifi c planning and decision support activities on our campuses. In

the third part, the focus is on the development and use of different data sources

that support both external accountability and institutional studies. In this part,

Terenzini’s aspects of issues intelligence are combined with his first tier of

knowledge—technical and analytical intelligence—and refl ected in the activities

and processes discussed as IR bridges the internal and external environments.

Chapters in the fi nal part focus on technical and analytical topics that refl ect

Terenzini’s fi rst tier of intelligence. Topics include research and management

tools and techniques typically used by institutional research professionals.

The following is a short description of each part:

• In Part One—The History, Theory, and Practice of Institutional Research—the

reader is introduced to the profession, its history, how the function is organized

and practiced, and new roles and expectations that are emerging on our campuses.

• In Part Two—Supporting Campus Leadership and Management—the focus

of the chapters is the support of institutional leadership and the processes that

institutional research is often asked to inform and assess. Topics include sup-

port of executive and academic leadership and governance, monitoring and

analysis of faculty and student success, strategic and operational planning and

management, facilities management, and campus sustainability.

• The third part—Bridging Internal and External Requirements for IR—includes

chapters in which various data sources (federal, state, specially created, and

campus) and their use by institutional researcher professionals are discussed.

• The focus of the fourth part—Institutional Research Tools and Techniques—includes discussions of the analytic tools, techniques, and methodologies used by

institutional research professionals in the practice of institutional research. Topics

include research activities such as statistical applications; comparative analyses;

quality control systems; measuring student, faculty, and staff opinions; and man-

agement activities such as being strategic in improving organizational effectiveness.

Handbook Development

This Handbook was developed in several steps. Initially, the editors developed

a proposed outline of topics to be covered. This outline was then reviewed by

two sets of IR professionals—the fi rst set made up of individuals representing

different sectors of higher education, the second set including experts in vari-

ous aspects of institutional research. Based on comments and suggestions from

fpref.indd xxifpref.indd xxi 07/06/12 2:14 PM07/06/12 2:14 PM

Page 24: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

xxii Preface

these professionals, topics were identifi ed. The resulting outline was then sent

in a call for volunteer authors to all members of the Association for Institutional

Research. In response to the call, some two hundred individuals volunteered

to write—some volunteering to write on a specifi c topic, others volunteering to

write wherever they might be needed. Lead authors for each chapter were

selected by the editors. These authors were then given a list of individuals who

had volunteered to write on their topics, as possible coauthors. Some authors

chose to write their chapters by themselves; others selected colleagues for coau-

thors and recruited coauthors from the list of other volunteers. Through this

process, we believe this volume can accurately be characterized as a discussion

of the roles, knowledge, and methods of institutional research, written by prac-

ticing institutional research professionals and scholars.

Summary

In summary, this volume covers a broad array of topics refl ecting the diverse

nature and multitude of tasks in which institutional research practitioners

fi nd themselves engaged. Some of the discussions are conceptual and give

extensive references to other works. Other discussions are more technical, but

they also provide a substantive list of supporting references. As was the case in

Institutional Research in the University: A Handbook (Dressel & Associates, 1971),

“Chapters overlap somewhat, but differences in focus and in points of view

justify this. Institutional research simply cannot be divided into a completely

independent set of discrete topics” (p. xii). This Handbook is not, however, a

fi nal authority on the practice of institutional research, nor a compendium of

all institutional research activities; rather, it is intended to be a starting place

in which the interested person will fi nd a road map to more topics, advanced

discussions, and examples of how we can do what we do.

References

Chambers, S., & Gerek, M. L. (2007, February 26). IR activities, IR applications, vol. 12.

Association for Institutional Research.

Dressel, P. L., & Associates. (1971). Institutional research in the university: A handbook. San Francisco: Jossey-Bass.

Knowles, A. S. (Ed.). (1971). Handbook of college and university administration. New York:

McGraw-Hill.

Saupe, J. P. (1990). The functions of institutional research. Tallahassee, FL: Association of

Institutional Research.

Stecklein, J. E. (1971). Institutional research. In A. S. Knowles (Ed.), Handbook of college and university administration (pp. 4-123–4-134). New York: McGraw-Hill.

Swing, R. (2009). Institutional researchers as change agents. In C. Leimer (Ed.), Imagining the future of institutional research (pp. 5–16). New Directions for Institutional Research,

no. 143. San Francisco: Jossey-Bass.

Terenzini, P. T. (1993). On the nature of institutional research and the knowledge and

skills it requires. Research in Higher Education, 34(1), 1–10.

fpref.indd xxiifpref.indd xxii 07/06/12 2:14 PM07/06/12 2:14 PM

Page 25: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

xxiii

ACKNOWLEDGMENTS

First, we would like to thank our wives, Josetta, Adriene, and especially Pat,

for their patience and support as we worked to make this Handbook a reality.

The Handbook reminds us of how important networking is for our profes-

sion. Over the years we have profi ted from the wisdom and guidance of many

mentors. We are thankful that their doors were always open, with special thanks

to Jim Montgomery, Chuck Elton, and Cameron Fincher.

We would also like to express our appreciation to those authors who

wrote chapters, contributing their wisdom and energy to the creation of this

Handbook. In addition, we would like to thank Jossey-Bass and the Association

for Institutional Research for the support and advice provided during the

development of this volume.

Finally, we would like to recognize Jean Chulak, who, as AIR’s administra-

tive director from 1974 to 1991, was the heart and soul of the Association in

its formative years. We cannot imagine where we would have been without her

energy and dedication.

Richard D. Howard

Gerald W. McLaughlin

William E. Knight

flast.indd xxiiiflast.indd xxiii 07/06/12 2:14 PM07/06/12 2:14 PM

Page 26: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

flast.indd xxivflast.indd xxiv 07/06/12 2:14 PM07/06/12 2:14 PM

Page 27: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

xxv

ABOUT THE EDITORS

R ichard D. Howard currently serves as a consultant in higher education.

He recently retired from his positions as director of institutional research

and professor of educational policy and administration at the University of

Minnesota. Before this he served as director of institutional research and held

faculty positions at West Virginia University, North Carolina State University,

and the University of Arizona. Howard also was a tenured professor of higher

education at Montana State University, teaching graduate courses and direct-

ing student research, and served as chair of the University’s faculty council. For

the Association for Institutional Research, he is a past president and served as

chair of the professional development committee, forum chair, and editor of

the Resources in Institutional Research monograph series, and has taught continu-

ously at the Foundations Institute since its inception. Howard has also taught in

the Statistics Institute and the Data and Decisions Institutes and has presented

papers and workshops at the annual meetings of various professional associa-

tions, including the Association for Institutional Research (AIR), the Society

for College and University Planning (SCUP), EDUCAUSE, the Southern

Association of Colleges and Schools (SACS), and the American Educational

Research Association (AERA). He has also been awarded the Outstanding

Service Award and is a Distinguished Member of the Association. In addition,

Howard is a past president and Distinguished Member of SAIR. His profes-

sional interests include strategic planning, higher education administration,

data administration, and mixed methods methodologies.

Gerald W. McLaughlin is an associate vice president for enrollment manage-

ment and marketing at DePaul University. He was formerly director of institu-

tional planning and research at DePaul and also director of institutional research

flast.indd xxvflast.indd xxv 07/06/12 2:14 PM07/06/12 2:14 PM

Page 28: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

xxvi About the Editors

and planning analysis at Virginia Tech. McLaughlin has taught courses in man-

agement and education and was a professor at Virginia Tech, where he also

sat on master’s and doctoral committees. For the Association for Institutional

Research, he has served as president, forum chair, and chair of the publications

board. McLaughlin was forum chair and president and a founding member of

the Southern Association for Institutional Research (SAIR). He has served as

editor of the AIR Professional fi le and IR Applications. McLaughlin has taught in

the Foundations Institute since its inception and has also taught in the Statistics

Institute and the Data and Decisions Institutes and has given papers and work-

shops at various professional associations, including AIR, SCUP, EDUCAUSE,

SACS, SAIR, and the Association of Governing Boards. He has received the

Sidney Suslow Award as well as the Outstanding Service and Distinguished

Member Awards from AIR and SAIR. McLaughlin’s areas of professional inter-

est include methodology, strategic management, and data management.

William E. Knight is executive director of institutional effectiveness and an

adjunct faculty member at Ball State University. He leads and teaches in Ball

State’s institutional research certifi cate program. Knight was previously an

institutional researcher and faculty member at Bowling Green State University

and Georgia Southern University and an institutional researcher at Kent State

University. He received the BG Best Award in 1998 and the BGSU Timothy D.

King Friend of Student Affairs Award in 2009. Knight is past president, past

forum chair, and a past member of the board of directors of the Association

for Institutional Research. His scholarly interests include the impact of college

on students and effectiveness in institutional research. Knight has authored

or coauthored twenty-fi ve peer-reviewed and invited publications, edited the

Primer for Institutional Research, served as associate editor of the Resources in Institutional Research monograph series, delivered one hundred peer-reviewed

and invited conference presentations and workshops, and served as a member

of fi fty-fi ve dissertation and thesis committees. He also serves as a consultant-

evaluator, team chair, and member of the Accreditation Review Council for the

North Central Association Higher Learning Commission.

flast.indd xxviflast.indd xxvi 07/06/12 2:14 PM07/06/12 2:14 PM

Page 29: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

xxvii

LIST OF CONTRIBUTORS

Lisa M. Amoroso, Associate Professor of Management, Dominican

University

Sandra J. Archer, Director for University Analysis and Planning Support,

University of Central Florida

Kim K. Bender, Director of Assessment, Colorado State University

Tom R. Bohannon, Analytical Consultant, SAS Institute Inc.

Rachel D. Boon, Associate Director, Higher Education Data Sharing

Consortium, Wabash College

Victor M. H. Borden, Professor, Educational Leadership and Policy Studies,

Indiana University Bloomington

Paul T. Brinkman, Associate Vice President for Budget and Planning,

University of Utah

Corbin M. Campbell, Graduate Assistant, Offi ce of Institutional Research,

University of Maryland—College Park

Julie W. Carpenter-Hubin, Director of Institutional Research and Planning,

The Ohio State University-Main Campus

Rebecca E. Carr, National Coordinator, Association of American Universities

Data Exchange

John J. Cheslock, Associate Professor, Dept. of Ed. Policy Studies and

Senior Research Associate, Center for the Study of Higher Education, The

Pennsylvania State University

Michael J. Dooris, Director of Planning Research and Assessment, The

Pennsylvania State University

flast.indd xxviiflast.indd xxvii 07/06/12 2:14 PM07/06/12 2:14 PM

Page 30: ffirs.indd ii 07/06/12 2:13 PM...HB Printing 10 9 8 7 6 5 4 3 2 1 ffirs.indd iv 07/06/12 2:13 PM The Jossey-Bass Higher and Adult Education Series ffirs.indd v 07/06/12 2:13 PM ffirs.indd

xxviii List of Contributors

Mardy T. Eimers, Director of Institutional Research, University of Missouri-

Columbia

Ann S. Ferren, Senior Fellow, Association of American Colleges and

Universities

Jonathan D. Fife, Visiting Professor, Educational Leadership and Policy

Studies, Virginia Polytechnic Institute and State University

Joseph W. Filkins, Senior Research Associate, Institutional Research and

Marketing Analytics, DePaul University

Gayle M. Fink, Assistant Vice President for Institutional Effectiveness,

Bowie State University

Carol H. Fuller, Consultant

Denise C. Gardner, Assistant Provost and Director, Offi ce of Institutional

Research & Assessment, The University of Tennessee

Charles F. Harrington, Professor of Management, The University of North

Carolina at Pembroke

Rosemary Q. Hayes, Director of Consortia for Student Retention Data

Exchange, Consortium of Student Retention Data Exchange

Teri L. Hinds, Director of Institutional Planning, Assessment & Research,

Winona State University

Richard D. Howard, Consultant

Glenn W. James, Director of Institutional Research, Tennessee Technological

University

Elizabeth A. Jones, Director of the Doctoral Program and Professor, School

of Education, Holy Family University

Daniel R. Jones-White, Analyst, Offi ce of Institutional Research, University

of Minnesota

Christine M. Keller, Voluntary System of Accountability Executive Director

and Director of Research and Policy Analysis, Association of Public and

Land-grant Universities

Heather A. Kelly, Director of the Offi ce of Institutional Research, University

of Delaware

Adrianna J. Kezar, Associate Professor, Rossier School of Education,

University of Southern California

Beverly R. King, Assistant Vice Chancellor for Institutional Effectiveness,

The University of North Carolina at Pembroke

Jang W. Ko, Assistant Professor, Department of Education, SungKyunKwan

University, Korea

Dennis A. Kramer II, Graduate Assistant, Institute of Higher Education,

University of Georgia

Paula S. Krist, Assistant Dean, Assessment Support, University of San Diego

flast.indd xxviiiflast.indd xxviii 07/06/12 2:14 PM07/06/12 2:14 PM