Qualitatively Excavating below the Quantitative Surface · revitalize the profession and as well, a...

Qualitatively Excavating below the Quantitative Surface: An action-oriented, Case-based application of the Meta-professional Model

Michael Theall, Ph.D. Associate Professor, Teacher Education Youngstown State University [email protected]

Bonnie B. Mullinix, Ed.D. Senior Consultant, Faculty and Educational Development Teaching Learning and Technology Group (TLT Group) [email protected]

Raoul A. Arreola, Ph.D. Professor & Assistant Dean for Assessment College of Pharmacy The University of Tennessee Health Science Center [email protected]

Abstract Previous research on the “Meta-Profession” of the professoriate has explored the model’s conceptual basis and examined data from an international survey of faculty, academic administrators, and academic support professionals, who provided opinions about faculty expertise across 20 Meta-professional skill sets and the need for each skill set in four professorial roles. The ultimate power of the Meta-Profession model lies in its potential for organizational development and facilitating improvements in faculty evaluation and development. This study considers how survey and qualitative data from a single institution were mixed to explore issues more deeply and to support an organizational development agenda, namely to guide faculty evaluation and professional development. The focus of this study is thus as much on the process of using the meta-professional model as on the themes that emerged from the qualitative explorations. Nonetheless, examples of qualitative findings will be presented in this report.

The Foundation: Since 2001, there has been focused examination of the work of the professoriate in terms of the professional roles faculty fill and the skills that faculty need for success in those roles. The emphasis of this work is fourfold: 1) to illuminate the nature of the professoriate as a profession; 2) to identify more precisely than previously, the nature of professorial work and the skills it requires; 3) to connect this investigation to policy and practice in faculty evaluation and professional development; and 4) to develop tools and processes that can be used at the institutional level in an organizational development framework. This work has been disseminated in several venues under the general title “The Meta-Profession Project.” (Arreola, 2007, 2005a, 2005b, 2004, 2000; Arreola, Aleamoni, & Theall, 2001; Arreola, Theall, & Aleamoni, 2003; Theall, 2006, 2001; Theall & Arreola, 2006, 2004, 2001; Theall, et. al., 2008, 2007; Theall & Cox, 2008). The Meta-Profession concept, research, and activities were recognized by AERA through it’s” Relating Research to Practice: Integrative Scholarship Award” in 2005. The broader research and literature landscape provides many insights regarding relevant higher education organizational issues, as well as information about leadership and faculty

Qualitative-quantitative exploration of the Meta-Profession

Theall, Mullinix, & Arreola AERA 2009, San Diego, April 15

2

roles, work, and rewards (e.g., Bergquist, 1992; Birnbaum, 2001, 1992, 1988; Boyer, 1990; Diamond & Adam, 2002, 2000, 1995; Finnegan et. al., 1996; Gappa, Austin, & Trice, 2007; Middaugh, 2001; O’Meara & Rice, 2005; Schuster & Finkelstein, 2006; Wergin, 2007). Distilled, this literature demonstrates that major changes have occurred and continue to occur in higher education as a result of external pressures such as the demand for accountability, technology growth, and societal expectations and demands, and internal issues such as changing demographics of the professoriate, changing demographics of the student body, implications and applications of new technologies, new strategies for teaching and learning, and the impact of all these factors on institutional budgets. At a time when demands for “reform” and complaints about higher education and the faculty are increasing (e.g., Campbell, 2000; Hersh & Merrow, 2005), funding for professional development and enhancing the quality of teaching and learning are declining. The professoriate as a profession is suffering (Burgan, 2006; Katz, 2006; Schuster, 1991-92; Trower et. al., 2001) yet despite this evidence and demonstrations of the effectiveness of faculty/professional development programs, (e.g., Sorcinelli, Austin, Eddy, & Beach, 2006) resources are limited or removed entirely. These problems have been attributed, in part, to a lack of leadership within the professoriate, and the Meta-Profession model has been proposed as a way to revitalize the profession and as well, a way to return some stability to higher education (Theall, 2006). Literature about faculty evaluation and development is voluminous (e.g., Arreola, 2007: Braskamp & Ory, 1994; Centra, 1993; Chism, 2007; Perry & Smart, 2007; Rice, Sorcinelli, & Austin, 2000; Theall & Franklin, 1991, 1990), and it is clear that effective evaluation is not only possible, but that it provides opportunities for significant benefits when combined with development programs, assessment activities, accreditation efforts, and instructional design/development projects. The synergy of these activities becomes obvious when one considers how much and what kinds of complementary and useful information are regularly collected in higher education. Admissions data, registration/grades data, assessment results, evaluation data, institutional research studies, and alumni data when combined, could provide a detailed picture of institutional performance. These data could also contribute to better policies and practices in faculty evaluation and development and ultimately result in improvements in teaching and learning by virtue of maximizing the working conditions and effectiveness of the faculty. A Meta-professional plan for institutional improvement: The ability to capitalize on the potential to enhance higher education by enhancing the professoriate, requires that the professoriate must be better understood as a profession and better organized to meet its obligations. Thus, organizational development that focuses on creating and sustaining conditions for success is critical. Faculty and academic administrators play a key role in this effort but a mechanism is needed to support their work. The Meta-Profession concept can provide data and processes that serve this need. However, a generic model is insufficient. There must be relevant information about local conditions, priorities, attitudes, and opportunities to acknowledge the contextual factors that should influence thoughtful practice. Initially, the Meta-Profession Project model was based on existing literature and practice, feedback from professionals in faculty evaluation and development, and small sample data



3

collection in venues such as professional conferences. Table 1 below, describes the generic “skill sets” necessary to fulfill faculty role responsibilities and Table 2 provides initial estimates of the need for each skill set in each role. One finding that has continually been identified as immediately apparent in the matrix, is that graduate school training focuses extensively on the “base profession”, but rarely provides preparation for the many other requirements of the “meta-profession”. The 20 Meta-Professional skill sets identified serve as a summary overview of faculty responsibilities. Additional matrices that examine subsets or application of skills within each faculty role may be seen at the Meta-Profession Project website. One example: the need for various skill sets associated with seven different teaching situations from large classes to on-line instruction to tutoring. These additional matrices can be viewed at: http://www.cedanet.com/meta.

Table 1: Meta-Professional Skill Set Definitions* [Note: The term ‘skill set’ is used to indicate a combination of knowledge, experience, proficiency, and skill in a specific area.]

SKILL SET Brief Definition/Description

BASE PROFESSION SKILL SETS Content Expertise

The formally recognized knowledge, skills and abilities a faculty member possesses in a chosen field by virtue of advanced training, education, and/or experience.

Practice/Clinical Skills

Those skills in translating content expertise into actions so as to carry out a process, produce a product, and/or provide a service.

Research Techniques

Those skills in acquiring existing knowledge, and/or creating or discovering new knowledge, within one's area of content expertise.

META-PROFESSION SKILL SETS

Instructional Design

Those technical skills in designing, sequencing, and presenting experiences which induce learning. Requires knowledge and skill in task analysis, the psychology of learning, the conditions of learning, and the development of performance objectives.

Instructional Delivery

Those human interactive skills that promote or facilitate learning in face-to-face instruction, as well as those skills in using various forms of instructional delivery mechanisms.

Instructional Assessment

Those skills in developing and using tools and procedures for assessing student learning (including test construction, questionnaire and survey construction, grading practices, and grading procedures).

Course Management

Those organizational and bureaucratic skills involved in maintaining and operating a course.

Instructional Research

Those technical skills and techniques associated with the scholarly inquiry into all aspects of instruction, teaching, and education.

Psychometrics/Statistics

Psychometrics/statistics is concerned with the measurement of human characteristics and the design and analysis of research based on those measurements.

Epistemology

That branch of philosophy that studies the nature and limits of knowledge as well as examining the structure, origin, and criteria of knowledge. Its application can often be seen in course or curriculum design in which the structure of the knowledge to be acquired by the student is taken into account in the design of instructional events or experiences.

Learning Theory Learning Theory deals with various models to explain how learning takes place and to provide a frame of reference for designing, developing, and delivering instruction.

Human Development

Theories and models of human intellectual, ethical, social, cultural, and physical development. Knowledge and expertise in the theories of human development is often required in the design and development of the entire educational experience.



4

SKILL SET Brief Definition/Description

Information Technology

Information technology (IT) is a term that encompasses all forms of technology used to create, store, exchange, and use information in its various forms (business data, voice conversations, still images, motion pictures, multimedia presentations, etc.).

Technical Writing

The delivery of technical information to readers (or listeners or viewers) in a manner that is adapted to their needs, level of understanding, and background. The primary skill is to write about highly technical subjects in such a way that a beginner (learner) or a non-specialist can understand.

Graphic Design Graphic design is the process and art of combining text and graphics to produce an efficient and effective means of visually communicating information or concepts.

Public Speaking Public Speaking is generally defined as speaking to large group of individuals, in a formal setting, for the purpose of imparting information and/or persuading others to a particular point of view.

Communication Styles

Individuals have various preferences for both communicating with others and interpreting the communications from others. Numerous models have been developed which describe how to recognize people’s preferred style of communicating and what strategy to use in communicating most effectively with them.

Conflict Management

The practice of identifying and handling conflict in a sensible, fair, and efficient manner. Conflict management requires such skills as effective communicating, problem solving, and negotiating with a focus on interests.

Group Process/Team Building

Groups of individuals gathered together to achieve a goal or objective, either as a committee or some other grouping, go through several predictable stages before useful work can be done.

Resource Management

The management of material resources so as to ensure their effective and efficient use in meeting specific purposes. Involves skills associated with inventory control procedures, replacement and maintenance scheduling, cost control, etc.

Personnel Management

Skills in communicating effectively, developing teams, managing diversity, managing conflict, delegating responsibility, coaching and training, giving and receiving constructive feedback, and motivating and guiding either individuals or groups to achieve specific goals

Financial/Budget Development

Requires an understanding of a variety of economic and monetary concepts including cash flow, direct and indirect costs, debt management, depreciation, etc., the ability to read and understand financial reports, as well as the ability to interpret and respond appropriately to federal, state, and/or local regulations and policies affecting the expenditure of funds.

Policy Analysis & Development

Those skills necessary for understanding the political constraints faced by policy makers, assessing the performance of alternative approaches to policy implementation, evaluating the effectiveness of policies, and the role conflicts in values have on the development of policies.

* Theall, Mullinix & Arreola (in press for 2009).



5

Table 2: An Overview of the Meta-Professional Model: Estimated General Frequency of Need for each Skill Set in each of Four Traditional Faculty Roles

In 2007-08 this exploration was expanded to encompass an international study co-sponsored by the AERA SIG in Faculty Teaching, Evaluation, and Development and the Professional and Organizational Development Network in Higher Education designed to gather data from colleges and universities in the USA and other English-speaking nations (primarily Canada). A web-based survey invited respondents (faculty, academic administrators, and staff in units such as teaching centers) to rate faculty expertise in each of the 20 generic “skill sets” and also, to indicate the frequency of need for each skill set in four roles: teaching, scholarship, service, and administration. A subset of the international study included two institutions (one USA, one Canadian) that promoted the survey internally in order to maximize the number of responses from their institutions. This was undertaken to ensure that their data could be analyzed separately for organizational development and related purposes. An on-line survey format was used and data from one institution were included in Theall et al. (2008). Methodological approach and focus for this portion of the study: Given the international and institutional data at hand, quantitatively based profiles were available for statistical and graphical comparison and explored for commonalities and discrepancies. Missing from this what the “so what now” aspect that was central to our efforts. It is fair to say that quantitative data often provides only a surface level view of “what is”. In order to understand how these findings relate to a particular context it is



6

necessary to excavate; to dig below the surface using qualitative methods. Such additional information allows more accurate interpretation of results and provides insights into the institutional context. It was decided that the a qualitative exploration was necessary and that the obvious place to begin would be the US institution. We determined that we would focus on both the process and the additional findings (with data from the second Canadian institution held separate until further investigation could take place on-site at that institution). The methodology included individual and group interviews with faculty and administrators. This paper will present and discuss our qualitative findings within the following five focal arenas:

1) the relationship between the model, the general survey findings, and the development of the qualitative design to explore discrepant and consistent patterns;

2) the quantitative results from the individual campus survey in relation to results from the international dataset and as stand-alone data;

3) the qualitative results as corroborating or conflicting evidence of campus opinions and attitudes;

4) the use of the Meta-professional framework and data to determine points of maximum impact for organizational development interventions in evaluation and development practice on the campus; and

5) the implications for future campus activities and efforts to improve faculty evaluation and development.

In addition, we will consider what was learned about the process of facilitating a dialogue aimed ultimately at promoting action, and effective faculty, educational and organizational development within an institution. Reflecting on this action research (Berg, 2001) design and undertaking has involved individuals broader than the core researcher-authors. Informing this process and approach was information collected during two conference proceedings in 2008: 1) an interactive presentation at a conference of professional faculty developers (Theall, Mullinix, & Arreola, 2008) that served to both inform the current study and validate the potential of the model and process for its successful campus use; and 2) a related presentation at a similar professional conference (Theall & Cox, 2008) that discussed the use of a specific mechanism (faculty learning communities) as a vehicle for promoting dialogue on meta-profession-related topics. This paper extracts insights from those events and the experience of facilitating cascading interviews to proffer ideas related to the following:

• Strategies for engaging faculty and key administrative stakeholders on campus in a data-driven dialogue on faculty roles and responsibilities and support;

• Suggestions for soliciting a sufficient sample of data and the effective use of the data collection process as a starting point for ongoing dialogue about faculty work;

• The role of qualitative and process-oriented interactions to add to and enhance the quantitative data and promote action-oriented discussions;

• Strategies to create faculty community and promote the exchange of ideas aimed at devising evidence-based interventions that could improve local faculty evaluation and development practices and effectiveness;

Scanning the Landscape: Quantitative findings from the International Study:



7

In 2007, the Professional and Organizational Development Network in Higher Education (POD) and the Special Interest Group in Faculty Teaching, Evaluation, and Development of the American Educational Research Association (AERA – SIGFTED) agreed to co-sponsor a research project intended to validate the Meta-Profession conceptual model and to explore several issues surrounding its application in faculty development and evaluation . The 2007-2008 co-sponsored survey validating the Meta-Profession conceptual model and mentioned above was international in scope and provided a broad base of responses. Circulated by professional organizations to their members, by some of those members to faculty and administrators at their individual campuses, and by individuals interested in the research, the request for participation were disseminated throughout the USA and several other countries. 415 responses were received from the USA and 114 from other countries, primarily Canada. Additional data were collected at the US institution that became to focus of this qualitative study (with 74 persons responding). One of the first questions addressed was the extent to which the original model was accurate in its estimates of the need for the skill sets. The data essentially validated the summary matrix representation of the need for the skill sets, and preliminary analyses were reported by Theall et. al. (2008). However, an equally important question was the extent to which faculty possessed expertise in the skill sets. Thus, in the same survey, faculty were asked to self-report their own expertise, and administrators were asked to report their estimates of the skills of the faculty with whom they worked. Since “base- profession” skills (content expertise, clinical/practical skills and research techniques) are the focus of graduate education and prime criteria for entry into the professoriate, high levels of expertise were expected and the assumption was made that faculty would regularly need these skills. Thus, further analysis concentrated only on the Meta-Professional skill sets. Expert i se – Frequency o f Need Ratings Table 3 presents aggregated data from the survey. The skill sets are arrayed on the left with the second column showing expertise ratings and the next four columns showing the rated frequency of need for each skill set in each role. The ratings scales used four points as follows.

• Expertise was rated as “Advanced” (1.0), “Moderate” (2.0), “Basic” (3.0), “Minimal-none” (4.0).

• Frequency of need was rated as “Almost Always” (1.0), “Frequently” (2.0), “Occasionally” (3.0), “Almost Never” (4.0).

Colors and lettering were used to emphasize patterns. Similar graphic profiles displayed in this and subsequent Tables have been designed to prompt discussion and dialogue around the findings. In each Table letters indicate the variable that was strongest and shaded cells show where the difference between variables was found to be significant.



8

Table 3: Expertise and Frequency of Need – International Survey Data

To determine the scores for the above profile, “Need” and “Expertise” ratings from faculty and administrators all respondents were combined and a mean score is shown in column two for faculty expertise. The means range from the strongest rating of 1.72 (for Public Speaking) to the weakest rating of 2.80 (for Policy Analysis and Development). An important question concerns the extent to which there are matches between expertise and need. Table 3 reflects the differences using color coding and letter abbreviations in the matrix cells. Color swiftly indicates where a significant difference was found: red cells indicate that the expertise rating was significantly stronger than the frequency of need rating; blue cells that the frequency of need rating was significantly stronger. In cells with no background color, there were no significant differences. Letters are used only to identify the direction of difference. The letter “E” in a cell indicates a stronger frequency of need rating while the letter “N” indicates a higher expertise rating. The overall pattern reveals two areas where faculty expertise in certain skill sets is rated significantly weaker than the need for those skill sets, namely, in the “Teaching” and “Administration” roles. This is not unexpected as “base profession” training in graduate schools does not concentrate on these skill sets except in a few disciplines (e.g., education, business, psychology). Means associated with expertise in each skill set means were compared to those from ratings of the frequency of need for each skill set in each role using ‘t tests’. These were used because each respondent provided a pair of ratings. All significance findings were at alpha levels more stringent than .05 and most were well below alpha .01.



9

Similar graphic profiles are displayed in this and subsequent Tables for consistency and ease of user readability. Tables have been designed to prompt discussion and dialogue around the findings. However, since the two concepts are different and there was no prior research to indicate how expertise and frequency of need might differ, it was difficult to frame hypotheses about the direction of the difference. One could assume that high need would be matched by advanced skill level. Alternatively, acknowledging the distance between “base-profession’ training and “meta-professional” skills needs, one could assume that (especially for new faculty) there would be discrepancies between skills and need. The data above include responses from a variety of faculty and administrators and reasons for differences might be covered by individual differences. Demographic data were also collected about rank and years of experience, and using these data, further analysis might be able to isolate differences resulting from these variables. Use of ‘t’ tests alone is thus, probably insufficient and further analyses will be done using correlations or non-parametric tests such as Chi Square to cross-check the results. In any case, if there are mismatches, then caution is required in the evaluation of that skill and it would be wise to allocate development resources to enhancing faculty expertise. In general, expertise ratings suggest “Moderate” to “Basic” expertise in the skill sets and these skill levels may or may not be sufficient depending upon the nature of the role and task at hand. What these data suggest with respect to faculty development is the possibility and importance of using data to drive faculty development activities. Given that resources are often limited (this being an almost universal case in current times), these resources should target areas of greatest need. When data are collected and analyzed at the institutional level and patterns are found (either similar to or discrepant from the international/generalized findings shown above), then the institution would be well advised to support professional enrichment activities that focus on the teaching role in certain skill sets, and possibly in the administration role for other skill sets. The data also make clear that faculty evaluation of certain kinds of skills must be tempered by the realization that high levels of expertise can not be expected across all faculty, particularly at entry into the profession. Refined expertise can not be assumed, and comparisons of individuals must thus avoid pitting experienced faculty who may have had the opportunity for training and skill development against those who have not had such opportunities. In other words, evaluation using inappropriate criteria/standards for performance is both poor methodology and unfair practice. Perhaps more important, if these kinds of data are collected at the institutional level, their first and most important use would be as information for open dialogue and discussion about institutional context and realities related to the expectations for faculty work, existing policies and practices in evaluation and development, and the identification of improvement needs. Significance testing becomes less important in this context.



10

Faculty – Administrator rat ings : The international survey also explored possible differences in the ratings provided by faculty (54% of the respondents) and administrators (46% of the respondents). Table 4 displays these findings in ways similar to those in Table 3. T-tests were used since different groups provided ratings of the two concepts, and the hypothesis was that there would be no cross-group differences. Each pair of ratings was independent and t-tests could be used without fear of increasing the chance of Type II error. The same rationale applies to Tables 5 through 7. All significant differences in Table 4 were at alpha .05 and most were well below alpha .01. The most striking results are: 1) that in no case did administrators rate faculty expertise in any skill sets significantly stronger than did faculty; and 2) that in every skill set but one in the administrative role, administrator ratings of need were significantly higher than those of the faculty. The graphic dramatically displays this dichotomous result. A possible explanation for the result is that the demands of the roles are so different that one’s perspectives about the importance of skills changes as one moves from one role to another. However, this is a tentative explanation at best and further investigation is clearly required. It is clear, however, that if an evaluator (a department chair or dean, for example) perceives certain skills to be almost always needed while evaluatees (faculty members) do not, and if the administrator rates faculty skills lower than do the faculty, these differences can have a major impact on the evaluation of the faculty. Mismatched priorities do not form a sound basis for accurate or fair evaluation.

Table 4: Administrator and Faculty Ratings – International Survey Data



11

Discipl inary Dif f erence Ratings The international survey included respondents from more than 24 disciplinary areas but the two largest groups of related disciplines were Science-Technology-Engineering-Math (STEM, 30% of the sample) and education and social science (ED/SO 40% of the sample). These two groups were chosen for additional analysis. In Table 5 green shaded cells indicate significantly stronger ED/SO ratings and orange cells indicate significantly stronger STEM ratings. Table 5 uses the same graphic patterns as previous tables. Significant differences are indicated by the presence of words in the cells and by shading, and blank cells indicate no significant differences. Alpha levels are all at .05 or lower and most differences are at alpha levels below .01. T-tests were used in this analysis since two groups were rating the same concepts. One result is somewhat predictable: that ED/SO faculty would rate their expertise in teaching-related skill sets higher than would STEM faculty. However, the noticeable differences extend beyond those skill sets into many other skill sets and almost 30 of the frequency of need ratings. Only in two cases (need for technical writing and public speaking in the teaching role) were STEM ratings significantly stronger than ED/SO ratings. There is no apparent explanation for the striking differences except the obvious possibility that disciplines have very different perspectives, values, and criteria for performance. This reasoning is certainly borne out by other work on disciplinary differences (e.g. Biglan, 1973; Franklin & Theall, 1992; Smart, Feldman, & Ethington, 2000). Two important implications for evaluation arise: 1) that the values and performance criteria of one discipline should not be used to judge performance of a person in a different discipline; and 2) that exploration of disciplinary differences should be part of good evaluation practice on every campus.



12

Table 5: Disciplinary Differences in Ratings – International Survey Data

Institutional Data and Perspectives As suggested above, the most effective use of the Meta-Profession model is at the institutional level where the concept and tools can be applied in a search for information that can form the basis for better understanding of unique institutional identity and dynamics. This understanding can directly result in improvements in policy and practice in faculty development and evaluation as well as secondary improvements in teaching and learning, organizational effectiveness, and institutional performance. To explore the application of the model, an additional data collection effort was carried out at two institutions, one in CANADA, the other in the USA. The focus of the next section is on the USA institution and an ongoing extension of the survey research. Beginning to Dig: a Quanti tat ive Prof i l e o f an Inst i tut ional Context A local data collection effort essentially identical to the international survey was conducted concurrently with the international survey. Seventy-four responses were received (approximately 25% of full-time faculty), and the data were analyzed in the same manner as those from the large survey. The first analysis compared expertise and need ratings. Table 6 presents the local data in the same manner as did Table 2, using coloring and lettering. These results present a very close match to the international data with respect to frequency of need for the skill sets across the four roles. T-tests were used as before with alpha levels at .05 or less and most below .01. The number of red cells (significantly stronger expertise



13

than needs ratings) differs by only one. The number of blue cells showing significantly stronger needs ratings is smaller, but only by a few. This reflects some differences in the expertise mean scores. The expertise ratings are stronger in the local data and as a result, there are fewer differences with need ratings in the teaching and administration roles. The mean scores for local vs. international were not analyzed for significant differences, but in some cases (e.g., Psychometrics/Statistics and Resource Management) the numeric differences were large (.91 and .74 weaker respectively, in the international sample). This might be expected, however, as previous analysis (Theall et. al., 2008) found significant differences across four institutional types based on Carnegie classification. The international sample would have homogenized the results from all types of institutions and thus, a mean expertise score at any one institution might differ considerably from the overall sample score in any skill set and role combination. Further, the respondent pool changes within this context, and the potential for increased faculty respondents as compared to administrator/faculty developer responses may also shift the findings. In all cases, the local data becomes critical and the comparison with the international profiles prompts reflections and dialogue.

Table 6: Expertise and Frequency of Need Ratings – Institutional Data



14

A local inst i tut ional s tudy: disc ip l inary di f f erence rat ings : Data from the local institution were analyzed for disciplinary differences using the same disciplinary groups as in the international survey. Fortunately the local sample contained disciplinary responses equivalent to (albeit smaller than) the international sample. Of the 74 respondents, 10 were from STEM disciplines and 13 from ED/SO disciplines. Data from the analysis are presented in Table 7 in the same manner as before. Analysis was similar and significant results were at the same alpha levels. As in the expertise-need analysis, results from the local institution were very similar to those in the international survey, though not as pronounced in terms in the number of significant differences. ED/SO expertise and need ratings were significantly stronger in some cases and in similar cells. The more coherent nature of the local sample and regression toward the mean in the larger sample may account for this. STEM ratings were not significantly stronger in any case, though they were numerically higher in many of the same cells as in Table 5. As before, established disciplinary differences may account for the results, but more investigation is necessary before reaching conclusions.

Table 7: Disciplinary Differences in Education/Social Sciences and STEM: Institutional Data

Comparison of administrator and faculty ratings was not possible with the local sample due to the small number of administrator respondents.



15

Moving to new depths: Why Qualitative? Ultimately, the interest in this study is in its ability to serve as a lever for change: to be able to provide a mechanism for reflection and dialogue that can help to unveil the professoriate and reveal context-specific issues that impact faculty success. This implies a process of qualitative, participatory research that begins by referencing relevant data, such as that collected and presented above. Further, while the results of the local quantitative study were very interesting in their own right and informative when considered in relation to the international results, the local data analysis raised several questions not directly answerable through examination of quantitative data. Review of the local results coupled with prior review of the literature (e.g., Birnbaum, 1988 on institutional types; Hativa & Marincovich, 1995 on disciplinary differences) suggested that there would be unique characteristics, but that a quantitative survey methodology would likely prove insufficient to unearth the reasons for the differences (Berg, 2001; Huberman & Miles, 2002). Excavating Deeper: Building Collective Awareness through Qualitative Exploration To dig more deeply, a qualitative follow-up study and process was initiated with the intention of using the quantitative survey findings as a prompt for active exploration of a contextually-grounded interpretation of the Meta-Profession model vis à vis local institutional realities. An interview process was chosen and a further limitation was imposed: to restrict the study to faculty and administrators within a school of education. The reasons were: 1) that narrowing the focus guaranteed a more representative sample distribution; 2) that the narrow focus would allow more detailed exploration of one disciplinary/college view; 3) that cross-department opinions could be more thoroughly examined by keeping the sample within one college; and 4) that the much larger sample required to examine and interpret differences across disciplines was untenable given the available timeframe and resources. Further, the primary researcher, situated within this institutional context, was presented with the opportunity and challenge to propose actionable change based on this process. Individual and small group interviews were conducted over a six month period. The college has three departments and thirty-six full-time faculty. Of these, sixteen faculty (44% of faculty), three chairs, and three deans (100% of administrators) were interviewed. The faculty were from all three departments, group interviews included only faculty from the same departments, and all three Deans a joint interview. In all, twelve interviews were conducted and recorded. Three people (two chairs and one faculty member) were interviewed near the end of 2008 with preliminary results from these earliest interviews reported in Theall, Mullinix, & Arreola (2008). The remaining interviews were conducted in the first two months of 2009. Three early interviews were recorded using an analog audiotape recorder while the remaining nine interviews were recorded with a digital recorder (resulting in improved quality, the ability to accurately identify the location of comments to the second; and the ability to share MP3 audio files for triangulated analysis by co-researchers). Data was reviewed and emergent themes identified through review of notes, multiple listenings, and transcriptions of key portions of interviews identified as illustrative of salient themes.



16

Artifacts Unearthed: Implications for Organizational and Faculty Development Emerging from the Interviews The specific responses of faculty and administrators are particularly valuable to understanding local context, to engendering ongoing local dialogue about faculty work and outcomes, and to furthering best practice and associated action in a variety of ways. Given that these results are highly context-specific (and occasionally sensitive), we recognize their limitations and do not propose that they necessarily provide broad generalizable insights about colleges of education or all the processes under investigation here. Even with these limitations, we believe that a sampling of these results does help to illuminate the general conclusions about the main focus of this paper: the data gathering and application processes and implications of both the process and what is unearthed, for organizational and faculty development from the practitioner’s (facilitator or faculty developer) view as well as from the client’s (faculty and administrators) views. Directly related to this, our analysis shows that key findings at this point in the local process can be seen to cluster around three key areas that have specific implications for organizational and/or faculty development. These areas of importance are: 1) identity; 2) role; and 3) attention to language. The importance o f ident i ty One’s membership in a discipline, a college and a department can connect with one’s perceptions about work within the discipline, college, or department. While the quantitative data revealed consistent and powerful disciplinary differences with respect to the importance of various skills in various roles at the department or college level (e.g., biology vs. literature or a STEM college vs. a college of education), this exploration revealed something further: considerable differences of opinion about roles and skills within one coherent college. A “coherent college” as used here, means a college (such as education, business, nursing, or engineering) with common professional interests and mission, as opposed to a college with a wide variety of disciplinary departments and less commonality (e.g., liberal arts and sciences or fine arts and humanities). This focused, qualitative exploration revealed both differences between departmental focus (e.g., in intended outcomes) and practical differences in the nature and requirements of faculty work in these departments (e.g., variation of supervision expectations/hours across departments). These perceived organizational affiliations and identities and differentiated work environments might provoke conflict between organizational units, with or without the explicit awareness of the key players. One issue of concern arose around departmental cooperation on co-listed courses and a degree of proprietary faculty feeling about the courses. Cooperation to resolve such discrepancies at the college level is sometimes hindered by such “turf” issues. One faculty member connected this issue to professional development barriers, noting: Until those little individual professor’s kingdoms get put aside, it’s hard to have real professional development - really changing what I do. This person’s comment suggests that where there is little collaboration at the department or college level or dismissal of each other’s work, opportunities to interact with and learn from colleagues are limited and professional development is hindered. This helps to underscore



17

the interplay between disciplinary and departmental identities that influence perceptions and actions that serve to create or dismantle barriers for faculty success. Another identified issue is related to differences in faculty assignments and allocation of release time within or across departments and how, dependent upon transparency, these assignments might be perceived by faculty and thus, might impact on organizational relations. A faculty member shared: Now, you might occasionally get reassigned hours for a project you’re working on. And then, dependent on who you are, there will be other times you get reassigned hours and other faculty don’t know for what reason. And that’s causing some issues for some faculty in this college. All the differences noted above led to variable perceptions among faculty and to possible causes of conflict with respect to performance expectations. One case of particular and recurring common interest was related to clinical supervision. [in comparison]…to their five [supervisions] we’re doing eighteen in the field, and to their three visits, we’re doing [ours] every other week. Yeah, there are some issues that are unique. Faculty in teacher education or special education may supervise 15 or more students in school placements each term. Each student requires a minimum of three visits from the supervising faculty member. Thus, when preparation, travel, observation, debriefing, dialogue with the on-site supervising teacher, and report preparation are considered, the workload can easily amount to 200 hours per semester. Contrast this with no clinical supervision requirements in other courses, or with limited supervision in other areas (school counseling faculty can have no more than five students to supervise per semester, per regulation of the accrediting professional association for school counseling). If equal expectations exist in all traditional faculty roles (teaching, research/scholarship and service), while little or no explicit consideration is given to differential work expectations (such as supervision), certain faculty are seriously handicapped and can understandably develop negative attitudes with respect to the equity of the evaluation process or the distribution of rewards for certain kinds of work. This can play out in many ways. For example, several individuals expressed their frustration at external accreditors for having such very different limitations on supervision and thus perpetuating this ongoing faculty challenge. At the same time, they acknowledged the importance of supervision for effective teacher preparation and the challenges such overloaded expectations present. As one person put it, …if you don’t do it, it’s going to blow up in the field. And then you also have the issue…you’re serving children. You can’t have nincompoops and ill-prepared students. They’re also ambassadors of the university. So, we have not respected supervision. Differential expectations and low transparency can also lead individuals to feel so unsure and



18

pressured that they move into defensive stances and end up reflecting negatively rather than supportively on their colleague’s achievements. For example, when discussing attitudes about the kinds of scholarship done by faculty in the college, one person said, I have to say that here and other places I’ve heard comments such as, “Ah, he or she just throws a couple surveys together and bangs them out and sends them to a friend who edits a publication that is peer reviewed and gets them published…”; and it’s as though … it’s not acceptable, it’s not a good as … I’ve heard that a lot….You don’t know if that’s professional jealousy or what’s behind all that. But I would hope that there would be an understanding of what the expectations are in your specific area and a respect for that when it comes to looking at that research and the kinds of things that are judged as good or useful or scholarly to the profession. In effect, the differential expectations for faculty across departments can lead to perceptions of inequality in status or treatment. Where such perceptions of inequality are strong and unanimous within a department (not a conclusion reached in this particular study) the potential for low morale, conflict, and inter-collegial discord is high. One purpose of using the Meta-Profession model is to open dialogue and candid exploration of the differences so that inequities can be surfaced and resolved, thus helping to avoid internal conflict. The importance o f ro le Perceptions of organizational and personal identity differences also connect with opinions about roles, and a powerful role difference is that between faculty and administrators. This is clearly supported by quantitative data showing differences of opinion about skills and the frequency of need for those skills, but once again, qualitative data shed additional light on the reasons for these differences. One faculty member (who had been a department chair) said, If we’re looking for administrators who can lead, and who can nurture and help faculty grow and develop, we need to develop some criteria for department chairs and we need to do some meaningful assessment to determine who we want those people to be. We don’t do anything like that. Agreement came from another person with similar background who said, We’re in a really sad state because no one wants to be a chair. No one wants to be a chair. The second repetition of the “No one wants to be a chair” was stated in a reflective and distressed tone, revealing this as an observation of a critical challenge. The first person contributed, with even greater emphasis, When they [administrators] are unprincipled, when they don’t operate on principles of equity and fairness and justice, everybody suffers. Some suffer more than others….If you have an unprincipled administrator, the faculty who suffer the most are the principled



19

faculty – because those faculty will try their best to do what’s right for students or for the program... Role differences can have differential implications and variable outcomes. For example, with respect to evaluation, another faculty member noted the importance of the chair knowing the faculty in a deeper and more personal way. That kind of brings us to this idea of evaluation. If the person doing the evaluating truly knows the person that they are observing…the evaluation will of course, be much more fair. But if you take a new administrator and put him in a classroom with a new instructor, neither of those individuals knows the other very well. They both come in with preconceived notions of what success means to them by definition, but the definitions might not even be close. If the definitions are not close, the evaluation will most likely not be fair, and the person being evaluated won’t understand the context in which some of the helpful or constructive criticisms are given, but with the administrator’s point of view that they were only trying to help. And I think that sometimes the classroom teacher…will take it the wrong way. View it as a personal attack in some instances. That’s a problem of not knowing each other in that kind of setting. Another difference is the extent to which administrators feel (or specifically are) compelled to respond to external agencies such as state, national, and specialized governing or accrediting bodies. While administrators place high degrees of importance on compliance that preserves the reputation, accreditation, and support of their colleges and departments, faculty often look upon the work of compliance as excessive paperwork with no direct application to their primary tasks: teaching, research, and service. One faculty member said of Specialized Professional Association (SPA) reports: It’s hourly work. What do you get for a SPA report? You get a three hour release time – which I’m totally happy with. It’s the first time in almost twenty years of writing SPA reports that I’ve gotten anything – at all.” As several of the faculty interviewees indicated, there is a feeling that preparing a major report of program performance as part of a seven-year cycle for a professional association accreditor, is a form of scholarship that should be recognized and rewarded as such. It’s hourly work but the result is kind of a piece of scholarship, so why can’t it count? Why on earth isn’t the SPA report a peer… – I can’t think of anything more peer reviewed than that SPA report. In contrast, a dean views the preparation of such a report as a service activity, adding that should faculty take their work and revise it for submission and publication, then it would count as scholarship. I don’t see any problem with that….Absolutely, in fact, one of our professors is doing something with that process - to submit in the future a journal article. I don’t see any problem with that do you? [at which point another dean agreed]



20

The faculty who noted this issue, responded that they have little interest in trying to turn what may be considered an onerous report into a publishable work, even if there was a venue for publishing such material (that in itself, being a questionable assumption). I sure don’t want to write about something like the damn SPA report that I don’t care about. What’s inside of it – I’m only doing because I’m the [content] person and I have to do it. That’s part of my responsibility. Even across departments, role perception differed. Some faculty felt more oriented to teaching or scholarship or service. Though teaching is the most commonly shared role, those who do not feel they have the time to do scholarship, expressed some degree of envy that others have that “luxury.” One person said, And from my perspective…the thing that I don’t have time to do is my scholarship. I love to write. I don’t have time. While another noted, When I went through and was hired here, teaching and service were the two main routes and that, of course, fit me quite comfortably….I honestly have not published. But to me, to write something and go through that process, and to keep going back to that, is directing my attention away from the work that – oh dear, I have to say it – the work that I prefer to do. There were several references to what is often called “differential staffing” or the ability to change the emphasis of one’s work from one role to another over time. A dean’s direct comment was clear: I don’t have any problem with that. And prompted another dean to follow with, As part of the induction process it might be nice to say to a new person, ‘You know you have to do scholarship, service, and whatever….’In your development plan what would be your first area of concentration?’, and then just let them move and develop in that area… The general notion of differentiated staffing was supported by all groups (faculty, chairs, deans) but operationally, it can pose burdens on chairs who are responsible for offering specific sets of courses on a regular basis. When a department has but one faculty member who teaches two or three unique courses and these courses are required within a program, it is difficult to change that person’s workload to scholarship or service because that necessitates transferring the teaching responsibility to another faculty member or hiring another person to teach. Unless everyone is in agreement about role changes, personnel issues can arise. Chairs are most aware of these kinds of issues and this connects with chairs comments about building and sustaining a balance within the departments: hiring people with complimentary skills, supporting collaboration among faculty, and keeping the



21

department in the best position to respond to external stakeholder needs (students, upper level administrators, trustees, legislators, accreditors, disciplinary/curricular associations). This emphasis on balance was succinctly stated by a chair who said, I try to assemble a group of people who meet all the needs of the department. In their roles, chairs feel particular pressures that sometimes conflict with the broad definitions of academic freedom and faculty autonomy that are part of academic tradition. In responding to these pressures, chairs may seem to faculty as having abandoned their responsibility to protect faculty from externally imposed demands. They are viewed as having abandoned their faculty sensibilities in favor of acceding to requirements imposed from the “top down.” The importance o f language A major and unanticipated finding from the interviews (and indeed, something that could come only from a qualitative process) was that while the Meta-profession model applies to a discussion about faculty roles and skills, its necessarily generic language is not always the same as the language used by faculty and administrators as they describe their roles, experiences and perspectives. Their initial frames of reference were more targeted to the traditional roles they fill, namely teaching – scholarship – service (they tend not to include administration as does the model), rather than to the skills involved in these roles. For example, their responses to questions about the skills critical to faculty success often began with a comment like, “Well, teaching is important, of course….” Here are three quotes presenting very different opening responses to a question about “…skills faculty need to succeed.” A dean went directly to the Meta-profession model and said, Well, I certainly think that all of the three areas under “base profession” are important: almost always. Do you agree with that? (a second dean agreed: “Oh sure”; and the first continued) When we go down to the next group, I want to say communication styles, conflict management, group process, instructional design, delivery, and assessment… and went on to read several other skill set names. A faculty member began with a personal reflection, saying, I think after my very first year here, as far as what I was looking for, was confidence that I fit into the university; fit into the college of education; understood all the protocols and policy that goes with it. That first year was very much a juggle….because no one really sits down and talks with you within your own college. Another faculty member was pragmatic with specific reference to a situation that is unique to college of education faculty.



22

I think you have to be really open minded to a whole different type of situation because most of us came from public schools and the philosophy of the school is the teachers all band together and we’re in it for the students and so forth and we make a team and that type of thing whereas, at least at this institution, I didn’t see that. I don’t see that, “Hey, the kids come first.” ….That was the biggest shock. But while interviewees agreed that the instructional skill sets in the model (design, delivery, assessment, research) were appropriately related to teaching, when asked to define the skills involved in effective teaching, they spoke in more relational and operational terms rather than referencing skills sets. They used language like, “understanding students”, “getting along with students”, and “working with adult learners”. Once the natural terms are in play, it becomes possible to not only map them on top of the MetaProfession model, but also to engage participants in either affirming these connections or actively displaying these in new arrays that reveal these relationships. It is reasonably clear, for example, that an issue such as “working with adult learners” could be part of “Instructional Design” and “Human Development” and “Communication Styles”; likewise, an issue like “sensitivity to diversity and individual differences” could also fall under the same headings. Such reflection might lead us to additional questions that would further clarify whether “Learning Theory” or even “Instructional Delivery” were involved in their definitions of these areas. Ultimately however, the core question is whether such a mapping activity furthers the development of the participants and their community. A critical important insight common to qualitative endeavors was revealed in this study: that it is important to listen closely and to allow participants to offer their opinions in their own words. The Meta-profession model may offer a point of departure and referential framework to promote discussion, but it cannot and should not become a tool that constrains dialogue. Terms and language generated by participants affirm the model in the broadest sense, but, more importantly, offer a grounded, ethnographic fabric that reflects the realities of the context. The local dialogue must, so to speak, be in the local dialect. It is the responsibility of the facilitator-researcher to translate and extrapolate back towards the model (and avoid forcing the language of the model into the local dialogue). With organizational development as the ultimate goal, it makes far more sense to keep the model as a reflective framework for the facilitator-researcher to consider, than it does to foist it onto participants and risk stifling communication, connections and insights. Observations on Intersecting Strata These three umbrella themes of role, identity and language have independent coherence, but they also have intersections. Individuals’ identities affect the way they perceive and interact with their roles, and influence and affect their choice of language. As we excavated the qualitative data, we found clear evidence of stratification that situated respondents, providing telltale signs of their roles and identities in the content, phrasing and emotional affect of their comments. Yet there were also times where these strata intersected and offered potential points of interest, and possible leverage for further exploration and for moving dialogue forward.



23

It was relatively easy to differentiate interview comments originating from faculty, chairs and deans -- both in the language used and the understanding evident in their comments. Most dynamic and pivotal were chairs. Chairs seem to find themselves caught particularly in the middle, with comments referencing their work as faculty and their work as administrators (both accurately reflecting the realities of their context). And yet, in talking about either administrators or faculty, they referenced them as “others”, not themselves. “Administrators” seemed to be those people, other than themselves, who were above and removed from their realities. While there was an understanding of administrative vision and challenges, they felt more closely connected to and responsible for advocating for faculty. And yet “faculty” were also “other” in that some faculty (usually not all) were viewed as not sharing the commitment to the well-being of the community/department, or a vision beyond themselves. Chairs were caught between roles and caught between identity and their language reflected the conflicted space they occupy. In reflecting on the language markers associated with these role and identity strata, the work of George Mild (1982) comes to mind. Mild referenced the image of a squat pear to describe career progression in the corporate world. Individuals enter at the base of the pear, and this is where they develop competence, skills and abilities that for most, will serve as the sweet spot in their career path: the place where they are happiest and most fulfilled. For the few who choose to move up the career ladder into the narrowing administrative neck of the pear, they begin to encounter new tasks and the need to develop new skills. They also move farther away from what had been their chosen profession. As they balance the excitement of new challenges and understanding against the comfort and competence of their previous roles, a disconnect can occur. As the quantitative data in this study suggest, perceptions of what is important, change as well. Chairs are often drawn in both directions. Some find the challenges of administration invigorating; others find them oppressive and distracting. Disaffection with the requirements and responsibilities of the administrative roles and angst about no longer being a productive teacher-scholar may be reasons for the frequent turnover in academic administrative positions, especially the chair’s role. Those who are most intrigued by administrative challenges, may move farther still up the narrowing neck and find a comfortable spot. Ultimately, those few who find a fit in this area and manage to effectively combine ambition, personal style, and the necessary skill sets, may reach the pinnacle of the pear and the highest levels of academe: provost or president. Many more may happily return to their sweet spot, and never look up again. An understanding of the complexity of strata associated with roles and identity can help to inform the facilitation of discussions, the interpretation of the comments, and the points of intersection where action and change may be possible. It becomes possible, through these indicators, to identify individuals who are positioned to help move an agenda forward and promote productive changes. Whether newer faculty who are passionate and inspired to contribute; seasoned faculty with political savvy as past chairs and fierce loyalty to their colleagues; chairs who are both dedicated advocates for faculty and ethical, visionary leaders; or deans willing to listen and use their position, resources and connections to support common agendas, each individual can contribute to the advancement of the academic community and the professoriate. Promoting and facilitating dialogue across these strata can help to identify these change agents and carefully facilitation of shared vision and discrepant challenges can help a community come together, learn from each other, utilize strengths and move forward.



24

Approach to the Excavation Process As noted, a significant portion of this qualitative exploration was aimed at determining the most productive process for facilitating discussion within an academic community to help reveal both the existing understanding of the professoriate and strategies for supporting faculty success. The initial idea was to make this a data driven model: introduce the findings from the international survey, compare these with the local findings and use the Meta-profession model to locate similarities or discrepancies in findings that can provoke reflection and discussion. In implementing the interviews, we learned that the initial plan was neither most appropriate nor most productive. We learned the following, and adjusted our practice accordingly:

1. The model, while interesting and relevant, was too complex and time consuming in most interviews to delve into.

2. The most productive starting point was a single question: What skills do faculty need in order to succeed?

3. A secondary question generally followed as a secondary prompt during the interview: What should be done to support faculty success?

4. Emergent ideas were identified and carried forward into subsequent interviews. Moving these points and threads forward into new discussions served to connect the community and build the ideas forward.

One of the repeating threads that naturally emerged during discussions focused on the barriers that exist in the academy that impede faculty success. Since these issues can have serious consequences with respect to faculty (and thus department and college) success, additional investigation is appropriate. Future research and development: Further explorat ion Collaboration with institutions wishing to use a Meta-Professional approach may add to the data set of the international survey. Accumulation of survey data will allow more precise analysis of individual and institutional variables to better reflect opinions and beliefs on faculty roles, responsibilities and preparation. As the data increases and the model’s related institutional profiles become increasingly robust, it’s usefulness as a guide to context-specific qualitative exploration and community dialogue around predictive ‘mirror’ findings or discrepant divergent findings increases. For example, analysis on the basis of disciplinary area has the potential to reveal differences in attitudes and priorities that can affect both faculty evaluation and development, and may point to differential strategies for institutional departments. In all cases, a large database is necessary in order to productively conduct such analysis. In this model, qualitative and longitudinal data are intended to be situated at the institutional level to promote improvement and use of the locally-generated evidence to guide programs and practices. As such, departments and/or institutions must have both the desire and the resources for ongoing exploration of the issues: a situation that is not always the case. That



25

said, the possibility does exist that collaborations with and between institutions might provide informative case reports of the application of the model and processes and the impact of this work on practice. If so, the case reported in this paper will provide a referential framework for applying this model, using international and local quantitative data to project productive qualitative exploration and community dialogue. Additional cases would potentially inform and validate this approach and provide a basis for generalizations about the professoriate and enhancement of the profession. Ultimately however, the greatest contribution of individual campus research and dialogue will be any local changes and improvement offered to faculty, and enhancement of the local teaching and learning environment. Linking the evaluat ion-deve lopment potent ia l o f the Meta-profess ion to assessment and accredi tat ion. Formalizing a process of institutional, college, and departmental dialogue about faculty work, roles, and skills can directly support improvements in policy and practice. It can also be linked to other important activities and ongoing needs, particularly assessment and accreditation. Literally every institution, college, and department must respond to demands for accountability in terms of its qualifications, operations, and performance. Assessment data play a major role in such reports and faculty often report a felt need to improve their learning outcomes assessment skills. In fact, such was one result of an internal, unpublished study at the college where the current data were collected as well as another unpublished study asking similar questions at the local institutional level. Use of the Meta-profession model and process is one way to gather more institutional/college data, and to use those data as the basis for clarification of needs, expectations, and responses to accreditor requirements. Assessment practices and needs can be better understood and responses can thus be more precise. An example follows. This qualitative exploration focused on a college of education. Most US colleges of education are accredited by the National Council for Accreditation of Teacher Education (NCATE). NCATE has several accreditation standards, but its Standard 5 pertains explicitly to “Faculty Qualifications, Performance, and Evaluation.” The standard has six elements which are arrayed against Base-profession and Meta-profession skill sets in Table 8 below. The relationships of the skill sets with the NCATE Standard 5 criteria are clearly demonstrated and bring home the importance of addressing the accreditation criteria through an exploration of faculty skills and roles. Since the Meta-profession model deals with four primary faculty roles and with skill sets that cut across these roles, it can provide a practical way of identifying critical skills-roles combinations so that emphasis can be placed on these in terms of evaluation and development. For example, the “Professional Enrichment” criterion in Standard 5 can be addressed by locating skill sets most in need of attention either because they are “Almost always” needed in a given role, or because the level of faculty expertise is found to be in need of improvement. While NCATE has identified its standards, it does not have a standardized process, rubrics, or criteria that easily allow colleges of education and their individual faculty to respond to the standard. What determines how a college of education identifies, evaluates, and supports faculty qualifications, best practices, evaluation process, and development initiatives? At the moment, there is no standardization that can be used to make accreditation compliance data



26

more consistent, reliable, or comparable. Thinking Meta-professionally, and applying the model across colleges of education could provide data that identify more clearly, those skills and practices that are hallmarks of high-performing units. These data could help NCATE to improve its own accreditation process and lead to a more complete, simple and standardized means of reporting performance.

Table 8: NCATE Standard 5 Components and Meta-profession skills

Summary:

The Meta-profession model was originally conceived as a general representation of faculty work, skills, and roles that had some explanatory power, but more importantly, as a tool that could be used to support and improve organizational development, faculty development, and faculty evaluation. Additional benefits would accrue as teaching and learning improvements resulted from skill development, collegial dialogue, and ongoing examination of the issues and factors that affect faculty, program, department, and college performance. While the model and process described can certainly be used for formal research, a potentially more productive



27

path is to support internal explorations that focus on operational details, building consensus, and creating environments that maximize faculty effectiveness. For the moment, the series of Meta- studies and the combination of quantitative and qualitative data continue to validate both model and process as a useful addition to the repertoires of faculty-organizational development staff, faculty evaluation practitioners, and faculty and administrators desiring to reinvigorate collegial dialogue about important aspects of their professional lives and their membership in the professoriate. References: Arreola, R. A. (2007). Developing a Comprehensive Faculty Evaluation System. (3nd ed.) Bolton, MA: Anker

Publishing Company. Arreola, R. A. (2005a). Crossing over to the Dark Side: Translating Research in Faculty Evaluation

into Academic Policy and Practice. Invited address at the 86th annual meeting of the American Educational Research Association. April 22. Available at: http://www.cedanet.com/meta

Arreola, R. A. (2005b). “Monster at the foot of the bed: surviving the challenge of marketplace forces on higher education.” In S. Chadwick-Blossey & D. R. Robertson (Eds.) To Improve the Academy 24. Bolton, MA: Anker Publishing Company. Available at: http://www.cedanet.com/meta

Arreola, R. A. (2004). Scholarship reconceptualized: an evaluative perspective. Paper presented at the 85th annual meeting of the American Educational Research Association. April 12. Available at: http://www.cedanet.com/meta

Arreola, R. A. (2000). Higher education’s meta profession. The Department Chair 11(2), 4-5, Fall. Available at: http://www.cedanet.com/meta

Arreola, R. A., Aleamoni, L. M., & Theall, M. (2001) Reconceptualizing the scholarship of teaching: college teaching as a ‘meta-profession. Presentation at the Fourth Annual Faculty Roles and Rewards Conference of the American Association for Higher Education. Tampa, FL: February 4. Available at: http://www.cedanet.com/meta

Arreola, R. A., Theall, M. & Aleamoni, L. M. (2003). “Beyond Scholarship: Recognizing the Multiple Roles of the Professoriate.” Paper presented at the 84th annual meeting of the American Educational Research Association. Chicago: April 22. Available at: http://www.cedanet.com/meta

Berg, B. L. (2001). Qualitative research methods for the social sciences. Needham Heights, MA: Allyn & Bacon.

Bergquist, W. H. (1992). The four cultures of the Academy. San Francisco: Jossey Bass Birnbaum, R. (2001). Management fads in higher education. Where they come from, what they do, and why they

fail. San Francisco: Jossey Bass. Birnbaum, R (1992). How academic leadership works. Understanding success and failure in the college presidency.

San Francisco: Jossey Bass. Birnbaum, R (1988). How colleges work: the cybernetics of academic organization and leadership. San Francisco:

Jossey Bass. Boyer, E. L. (1990). Scholarship reconsidered. San Francisco: Jossey Bass. Braskamp, L. A. & Ory, J. C. (1994). Assessing Faculty Work. San Francisco: Jossey-Bass. Burgan, M. (2006). Whatever happened to the faculty. Drift and decision in higher education. Baltimore, MD:

Johns Hopkins Press. Campbell, J. R. (2000). Dry rot in the ivory tower. New York: University Press of America. Centra, J. A. (1993). Reflective Faculty Evaluation: Enhancing Teaching and Determining Faculty Effectiveness.

San Francisco: Jossey-Bass. Chism, N.V. (2007) (2nd ed) Peer review of teaching. Bolton, MA: Anker Publishing.



28

Diamond, R. M. & Adam, B. E. (2002). (Eds.) A field guide to academic leadership. San Francisco: Jossey Bass.

Diamond, R. M. & Adam B. E. (2000). The disciplines speak II. More statements on rewarding the scholarly, professional, and creative work of faculty. Washington, DC: American Association for Higher Education. Diamond, R. M. & Adam B. E. (1995). The disciplines speak. Rewarding the scholarly, professional, and creative work of faculty. Washington, DC: American Association for Higher Education. Finnegan, D. E., Webster, D., & Gamson, Z. F. (Eds.) (1996). (2nd ed.) Faculty and faculty issues in

colleges and universities. In B Townsend (Series Ed.) ASHE reader Series. Boston: Pearson Custom Publishing.

Gappa, J. M., Austin, A.E., & Trice, A.G. (2007). Rethinking faculty work: higher education’s strategic imperative. San Francisco: Jossey Bass.

Hersh, S. H. & Merrow, J. (2005). Declining by degrees. New York: Palgrave. Huberman, A. M. & Miles, M. B. (Eds.) (2002). The Qualitative Researcher’s Companion. Thousand

Oaks, CA: Sage Publications. Katz, S.N. (2006). “What has happened to the professoriate?” The Chronicle of Higher Education.

October 6, B8-B11. Middaugh, M. F. (2001). Understanding faculty productivity. Standards and benchmarks for colleges and

universities. San Francisco: Jossey Bass. Mild. G. L. (1982). The squat pear principle: Why managers rise or fall. Chicago, IL: Contemporary Books.

O’Meara, K. A. & Rice, R. E. (2005). Faculty priorities reconsidered. San Francisco: Jossey Bass.

Rice, R. E., Sorcinelli, M. D. & Austin, A.E. (2000). Heeding new voices: Academic careers for a new generation. Washington DC: American Association for Higher Education. Schuster, J. H. (1991-1992). “Whatever happened to THE faculty?” Teaching Excellence 3 (5). Schuster, J. H., & Finkelstein, M. (2006). The American faculty. The restructuring of academic work and

careers. Baltimore: Johns Hopkins University Press. Sorcinelli, M. D., Austin, A. E., Eddy, P. L., & Beach, A. L (2006) Bolton, MA: Anker Publishing. Theall, M. (2006). Leadership in faculty evaluation and development: some thoughts on why and

how the meta-profession can control its own destiny. In V. R. K. Prasad (Ed.) Education sector HR perspectives. Nagarjuna Hills, Punjagutta, India: Icfai University Press. Available at: http://www.cedanet.com/meta.

Theall, M. (2001). Beyond the scholarship of teaching to a unified view of the profession. Paper presented at the 82nd annual meeting of the American Educational Research Association. Seattle: April 13. Available at: http://www.cedanet.com/meta.

Theall, M. & Arreola, R. A. (2006). The meta-profession of teaching. NEA Higher Education Advocate, 22 (5), 5-8, June. Available at: http://www.cedanet.com/meta.

Theall, M. & Arreola, R.. A. (2004). The real work of the faculty: a collaborative POD study. Presentation at the 25th annual meeting of the POD Network. Montreal, Canada, November 5.

Theall, M. & Arreola, R. A. (2001). Multiple roles, multiple responsibilities: conceptualizing the professoriate as a meta-profession. Presentation at the 22nd Annual Conference of the Professional and Organizational Development Network. St. Louis, October 13.

Theall, M., Arreola, R. A., Franklin, J, Mullinix, B. B., Svinicki, M., & Chism, N. (2008). The roles and skills of faculty in US and UK higher education institutions: What faculty need to succeed. Paper presented at the 89th annual meeting of the American Educational Research Association. New York: March 25.

Available at: http://www.cedanet.com/meta Theall, M. & Cox. M. D. (2008). Faculty learning communities as a leadership vehicles for exploring

the Meta-Profession of the professoriate and revitalizing collegial campus dialogue.



29

Presentation at the 28th annual Lilly Conference on College teaching. Miami University, Oxford, OH: November 23.

Theall, M. & Franklin, J. (Eds.) (1991). "Effective Practices for Improving Teaching." New Directions for Teaching and Learning. # 48. San Francisco: Jossey Bass.

Theall, M., & Franklin, J. (1990). Student ratings of instruction: Issues for improving practice (Eds.). New Directions for Teaching and Learning #43.

Theall, M., Mullinix, B. B., & Arreola, R. A. (2008). Implications/applications of an international study of faculty skills and roles. Presentation at the 32nd annual meeting of the Professional and Organizational Development Network. Reno, NV. October 23.

Theall, M., Mullinix, B. B., & Arreola, R. A. (in press for 2009). Promoting Dialogue and Action on “Meta-Professional” Skills, Roles and Responsibilities. In L. Nilson & J. Miller (Eds.) To Improve the Academy 28. San Francisco: Jossey Bass.

Trower, C. A., Austin, A. E., & Sorcinelli, M. D. (2001). “Paradise lost. How the academy converts enthusiastic recruits into early-career doubters.“ AAHE Bulletin, 53 (9), 3-6.

Wergin, J. F. (2007). (Ed.) Leadership in place. Bolton, MA: Anker Publishing.

Qualitatively Excavating below the Quantitative Surface · revitalize the profession and as well, a...

Documents

Transcript of Qualitatively Excavating below the Quantitative Surface · revitalize the profession and as well, a...