AFBE · PDF fileFollowing a review of the literature discussion is made and then conclusions...

AFBE JOURNAL

Volume 9, No.2, December, 2016

ISSN2071-7873

AFBE Journal Vol.9, no. 2

TABLE OF CONTENTS

ACADEMIC PAPERS

Jamnean Joungtrakul, “IMPROVING THE RESPONSE RATE OF QUESTIONNAIRES IN CONDUCTING QUANTITATIVE RESEARCH”

53

Altynay Nazarbay, Paul J Davis, “LEADERSHIP AND POST-MERGER ORGANISATIONAL IDENTITY IN AN EMERGING MARKET: THE CASE OF KAZAKHSTAN”

65

Penpak Pheunpha, Suchada Bowarnkittiwong, “VALUE-ADDED MODELS FOR IMPROVING UNIVERSITY SCHOOL EFFECTIVENESS IN THAILAND”

89

AFBE Journal Vol.9, no. 2 53

IMPROVING THE RESPONSE RATE OF QUESTIONNAIRES IN CONDUCTING

QUANTITATIVE RESEARCH

Jamnean Joungtrakul, DBA. Distinguished Professor, Far East University, South Korea

University Advisor, North-Chiang Mai University, Thailand

Email: [email protected]

ABSTRACT

One of the most important challenges faced by quantitative researchers nowadays is how to get their questionnaires back from the sample groups with a number that is large enough to be sufficient for data analysis. At present the rate of response in many cases is only about 10-15%. This low rate of response leads to doubt of the quality or rigour of such quantitative research projects both in terms of validity and reliability. This review paper aims to ascertain: (1) what is an acceptable rate of response to a questionnaire in quantitative research? (2) How to improve the rate ofresponse to a questionnaire in quantitative research? To respond to the two objectives of this paper a review was made of the concepts of (1) the population in quantitative research; (2) sampling in quantitative research (3) methods of distribution and collection of answered questionnaires; (4) the acceptable response rate of questionnaires; (5) how to improve the response rate of questionnaires.

Following a review of the literature discussion is made and then conclusions and recommendations are presented.

Keywords: response rate, questionnaire, quantitative research

INTRODUCTION

The most important factor in conducting any kind of research is how to get quality and sufficient data for analysis to answer the research questions and research objectives. It could be argued that if there is no data there is no research(J. Joungtrakul, Sheehan, & Aticomsuwan, 2013). It could be also argued that quality and sufficient data bring quality and rigorous research results.

Quantitative researchers are now facing one major problem of how to get their satisfactorily answered questionnaires back from the sample groups with a sufficient number for data analysis. At present the rate of response in many cases is very low. In such cases it is only about 10-15%. This low rate of response leads to the question of the quality or rigour of such quantitative research projects both in terms of validity and reliability.


The objectives of this review paper are to try to ascertain: (1) what is an acceptable rate of response of questionnaires in quantitative research? (2) How to improve the rate ofresponse of questionnaires in quantitative research?

To answer these two objectives the paper reviews the following concepts: (1) the population in quantitative research; (2) sampling in quantitative research (3) methods of distribution and collection of questionnaires; (4) the acceptable response rate of questionnaires; and (5) how to improve the response rate of satisfactorilycompleted questionnaires.

Following the review of the literature discussion is be made and then conclusions and recommendations are presented.

LITERATURE REVIEW

The concept of population in quantitative research.

A population is “a group of individualunits with some commonality. For example, a researcher may want to study characteristics of female smokers in the United States. This would bethe population being analyzed in the study…”(Online available at: cirt.gcu.edu, retrieved 1 December 2016).The other example is that a researcher conducting a research to ascertain the readiness to cope with the free flow of skilled labor in the ASEAN Economic Community of engineers in electronics and computer companies. She contacted the Electronics and Computer Employers’ Association in Thailand to obtain the list of the companies who are members of the Association and found that there are 47 companies being members. She then contacted those 47 companies and there were 28 companies willing to participate in her research project. She found that there were7,000 engineers working in these 28 companies. She used 7,000 engineers as her population in her research project (N. Joungtrakul, 2013a, 2013b).

Therefore, a population can be defined as “an entire group of individuals, events or objects with some observable characteristics” (Mugenda&Mugenda, 2003, cited inWambui& Gichuho, 2013, p. 388). It is a specific group of people or organizations where the researcher hopes to obtain the desired datain order to analyze this data to answer the research questions and objectives. It can be seen as “all members of a specific group… The larger the population to which the researcher ultimately wants to generalize the results” is called the “target population”(Online available at: www2southeastern.edu, retrieved 5 December 2016).

In quantitative survey research the data is mostly obtained via questionnaires. The number of the population is very important and must be specific to ensure that the data to be obtained reflects the needed information to be analyzed to answer the research questions and objectives. It also forms the basis for determining the number of individual samples or sample size to be used in the research project. So, in quantitative research design the population in the study must be identified (Creswell, 2009). In addition, a population is an abstract concept so that the researcher must provide an operational definitionsimilar to developing operational definitions for constructs that are measured. This could include the unit being sampled; the geographical location and the temporal boundaries (Online available at: csun.edu, retrieved 5 December 2016) similar to the second example demonstrated above(N. Joungtrakul, 2013a, 2013b).


One additional thing that the researcher has to do is to determine the target population size. According to Draugalis and Plaza (2009) one way of determining the target population size is a census where the researcher collects the needed data from all members of the population. This is required in an extremely small population. The other way is to use a certain number as a percentage of the population or use a statistical table provided for example, by Krejcie and Morgan (1970, cited in Draugalis & Plaza, 2009) or Yamane (1973). The target population size is essential as it would be difficult or impossible to collect data from all members of the population or by undertaking a census in every case. As stated by Dillman (2007, cited in Draugalis & Plaza, 2009) that “there is nothing to be gained by surveying all 1000 members of a population in a way that produces only 350 responses (a 35% response rate) versus surveying a sample of only 500 in a way that produces the same number of responses (70% response rate)” (p. 4).

Finally, from what has been discussed there are a few terms which apply and we have to make sure that we understand them correctly before we go further to the next section. To clarify what has been described earlier in this paper the terms “population” and “target population” aredefined and operationalized.

Firstly, a population is the total number of people from whom the researcher will obtain the data. Secondly, the target population is the specific pool of people who the researcher wants to study and has a working sampling frame(Online available at: csun.edu, retrieved 5 December 2016).

The concept of sampling in quantitative research.

Once the population and target population have been identified it is necessary to determine the number of samples for the research project. A sample is “a selection of respondents chosen in such a way that they represent the total population as good as possible” (Online available at: checkmarket.com, retrieved 1 December 2016).Unless we want to survey using a census which is rare and the researcher normally applies a small group of the population as we normally work with a large size of population. Since the size of the population is normally large it would be impossible to collect information from all the people who are the population of the study. Therefore, we would normally “select individuals from which to collect the data. This process is called sampling. The group from which the data is drawn is a representative sample of the population so the results of the study can be generalized to the population as a whole” (Online available at: cirt.gcu.edu, retrieved 1 December 2016).So samples are the number of people or organizations representing the population in requesting to provide information to be the data for analysis to answer the research questions and objectives.

An excellent explanation of sampling is as follows: “The primary goal of sampling is to get a representative sample, or a small collection of units or cases from a much larger collection or population, such that the researcher can study the smaller group and produce accurate generalizations about the larger group. Researchers focus on the specific techniques that will yield highly representative samples (i.e., samples that are very much like the population). Quantitative researchers tend to use a type of sampling based on theories of probability from mathematics, called probability sampling”(Online available at: csun.edu, retrieved 5 December 2016, p. 22).In general there are two types of sampling: nonprobability and probability sampling techniques.


Firstly, nonprobability sampling is “a sampling technique in which each unit in a population does not have a specifiable probability of being selected. In other words, nonprobability sampling does not select their units from the population in a mathematically random way. As a result, nonrandom samples typically produce samples that are not representative of the population. This also means that are ability to generalize from them is very limited” (Online available at: csun.edu, retrieved 5 December 2016, p. 22). There are four major types of this kind of sampling; (1)Haphazard, Accidental, or Convenience sampling. (2)Quota sampling (3) Purposive or Judgmental sample; (4)Snowball sampling(Online available at: csun.edu, retrieved 5 December 2016).

Secondly, probability sampling is “a sampling technique in which each unit in a population has a specifiable chance of being selected. The motivation behind using probability sampling is to generate a sample that it is representative of the population from which it was drawn. Random sampling does not guarantee that every random sample perfectly represents the population. Instead, it means that most random samples will be close to the population most of the time, and that one can calculate the probability of a particular sample being accurate” (Online available at: csun.edu, retrieved 5 December 2016, pp. 22-23).In general, there are four types of probability sampling techniques: (1) Simple Random; (2) Systematic Sampling; (3) Stratified Sampling; (4) Cluster Sampling(Online available at: csun.edu, retrieved 5 December 2016).

The next item we have to consider is the sampling element. A sampling element is the unit of analysis or case in a population that is being measured. Then a sampling ratio is determined. The sampling ratio is “determined by dividing the sample size by the total population. For example, if a population has 50,000 people, and a researcher draws 5,000 people for the sample, the sample ratio would be .10 (5,000/50,000)” (Online available at: csun.edu, retrieved 5 December 2016, p. 23). Following the sampling ratio we then determine a sampling frame which means that we operationalize“a population by developing a specific list that closely approximates all the elements in the population. This is a sampling frame. The researcher can choose from many types of sampling frames: Telephone directories, etc. A mismatch between the sampling frame and the conceptually defined population can be a major source of error. Just as a mismatch between the theoretical and operational definitions of a variable creates invalid measurement, so a mismatch between the sampling frame and the population causes invalid sampling” (Online available at: csun.edu, retrieved 5 December 2016, p. 23).

The next item we should consider is the parameter. Parameters are normally“determined when all the elements in a population are measured…. The population parameter is never known with absolute accuracy for large populations, so researchers must estimate it on the basis of samples. In other words, they use information from the samples to infer things about the population”(Online available at: csun.edu, retrieved 5 December 2016, pp. 23-24). Then the use of statistics should be considered. In general, a statistic is any characteristic of a sample that may be used to infer about a parameter of a population(Online available at: csun.edu, retrieved 5 December 2016).

The most important thing we have to consider at this point is how large should a sample be? The sample size depends largely on: (1) The kind of data analysis the researcher plans (descriptive, multiple regression); (2) how accurate the sample has to be for the researcher’s purposes (acceptable sampling error); (3) population characteristics (homogenous or heterogeneous, large or small); (4) principle for sample sizes is, the smaller the population,


the bigger the sampling ratio has to be for an accurate sample (Online available at: csun.edu, retrieved 5 December 2016). It should be noted that “Larger populations permit smaller sampling ratios for equally good samples. This is because as the population size grows, the returns in accuracy for sample size shrink. For small populations (under 1,000), a researcher needs a large sampling ratio (about 30%). For moderately large populations (10,000), a smaller sampling ratio (about 10%) is needed to be equally accurate. For large populations (over 150,000), smaller sampling ratios (about 1%) are possible to be very accurate. To sample from very large populations (over 10,000,000), one can achieve accuracy using tiny sampling ratios (0.025%)” (Online available at: csun.edu, retrieved 5 December 2016, p. 25).For more details explanations see Babbie (2008).

Having determined the number of sample the next important step is to determine the sample by using various sampling methods.

The methods of distribution and collection of questionnaire.

There are three major methods in distributing and collecting questionnaires: (1) personal delivery and collection of completed questionnaires; (2) sending out questionnaires by post with prepaid stampedenvelopes for respondent to return the completed questionnaires; (3) sending out questionnaires by email and the respondents complete the questionnaires and return via email(N. Joungtrakul, 2013a, 2013b).

In addition, the University of Iowa (Online available at: its.uiowa.edu, retrieved 10 December 2016) suggests several methods to distribute a survey questionnaire to respondents: (1) anonymous Survey Link; (2) email Customized Links; (3) purchase Respondents; and (4) additional distribution methods such as SMS, Social Media, In-Page Pop-up, and QR Code.

The acceptable response rate of a questionnaire.

The response rate is the percentage of completed acceptable questionnaires returned to the researcher compared to the number of questionnaires distributed to the respondents. It is the percentage of people who respond fully and correctly to a survey questionnaire (Online available at: facultyinnovate.utexas.edu, retrieved 12 December 2016, p. 1). It is also known as the completion rate or return rate. It is the number of people who answer the survey divided by the number of people in the sample. For example, if 1,000 questionnaires were sent by mail, and 257 of them were successfully completed (entirely) and returned, then the response rate would be 25.7 percent (Online available at: en.m.wikipedia.org, retrieved 11 December 2016). High response rates help ensuring that the survey results are “representative of the target population. A survey must have a good response rate in order to produce accurate, useful results” (Online available at: facultyinnovate.utexas.edu, retrieved 12 December 2016). We can obtain response rate by “dividing the number of people who submitted a completed survey (80% or more of questions answered) by the number of people” we “attempt to contact” (Online available at: facultyinnovate.utexas.edu, retrieved 12 December 2016, p. 1). For example, if we asked 185 participants to complete our questionnaire and 107 responded, the response rate is 107/185 or 58% (Online available at: facultyinnovate.utexas.edu, retrieved 12 December 2016).

This is important because it reflects the representativeness of the sample to the population of the research project. It affects the quality and rigourof the results of the research. As “a low response rate can give rise to sampling bias if the nonresponse is unequal among the


participants regarding exposure and/or outcomes. Such bias is known as nonresponse bias”(Online available at: en.m.wikipedia.org, retrieved 11 December 2016). Draugalis, Coons, and Plaza (2008) assert that “response rate and potential nonresponse bias require critical consideration because they ultimately affect the validity of the results.” In addition it affects the confidence and generalizability of the results of the research project. At present there is no consensus on what is the acceptable response rate of questionnaires. Some expert opinionson what is an acceptable response rate were compiled by Poole (2014. Online available at: socialnorms.org, retrieved 11 December 2016) is shown in Table 1.

TABLE 1: SURVEY RESPONSE RATE

Response

Rate (%)

Author and Description

25% Dr. Norman Hertz when asked by the Supreme Court of Arizona

30% R. Allen Reese, manager of the Graduate Research Institute of Hull U. in the United Kingdom

36% H. W. Vanderleest (1996) response rate achieved after a reminder

38% in Slovenia where surveys are uncommon

50% Babbie (1990, 1998)

60% Kiess&Bloomquist (1985) to avoid bias by the most happy/unhappy respondents only

60% AAPOR study looking at minimum standards for publishability in key journals

70% Don A. Dillman (1974; 2000)

75% Bailey (1987, cited in Hager et al., 2003 in Nonprofit and Voluntary Sector Quarterly, pp. 252-267)

Note: In addition, various studies described their response rate as “acceptable” at 10%, 54%, and 65%, while others on the American Psychological Association website reported caveats regarding non-responder differences for studies with 38.9%, 40% and 42% response rates Source: Poole, 2014. Online available at: socialnorms.org, retrieved 11 December 2016. In addition, acceptable response rates vary by how the survey is administered: (1) mail: 50% adequate, 60% good, 70% very good; (2) phone: 80% good; (3) email: 40% average, 50% good, 60% very good; (4) online: 30% average; (5) classroom paper: >50% = good; (6) face-to-face: 80-85% good (Online available at: facultyinnovate.utexas.edu, retrieved 12 December 2016).

While the experts’ opinion on response rate and the practices of researchers vary some journals, for example, the American Journal of Pharmaceutical Education, states that the


questionnaire response rate in general should be at 60 percent or above in order to have a quality research result. At the same time the response rate for research in the field of pharmacy must be at 80 percent or above(Draugalis et al., 2008; Fincham, 2008).Mugenda and Mugenda(1999, cited in Oduma& Were, 2014) proposed that the response rate of 50 percent and above is rated as fair and is acceptable while the response rate of 60 percent is rated as good and the response rate of 70 percent and above is rated as excellent. Although there are some arguments that surveys with lower response rates yield more accurate measurements than surveys with higher response rates (Visser, Krosnick, Marquette &Curtin, 1996: Keeter et al., 2006, cited in Poole, 2014. Online available at: socialnorms.org, retrieved 11 December 2016).Draugalis and Plaza (2009) assert that “a response rate of 50%-60% or greater is optimal because nonresponse bias is thought to be minimal with that high of a response rate.” Therefore, researchers should generally aim for a 50% or higher response rate in any research project. A higher response rate of 80% or greater should be aimed for in “a small random sample rather than a low response rate from a larger pool of potential respondents” (Online available at: surveygizmo.com, retrieved 11 December 2016).

How to improve the response rate of questionnaire.

Survey response rate at present is rather low. In general, internal survey yield approximately 30-40% while external survey yield approximately 10-15% only (Online available at: surveygizmo.com, retrieved 11 December 2016). Factors that affect survey response rate include: customer loyalty; brand recognition; perceived benefits; demographics; and survey distribution. It was argued that “an important participation incentive to survey is that their opinions will be heard and that action will be taken based on their feedback. If respondents believe that participating in a survey will result in real improvements the response rate may increase, as will the quality of the feedback” (Online available at: surveygizmo.com, retrieved 11 December 2016). In addition, the research purpose, type of statistical analysis, how the survey is administered, and how close we are to respondents also affect the response rate(Online available at: facultyinnovate.utexas.edu, retrieved 12 December 2016).

In order to improve the survey response ratethe following actions should be taken: make a proper survey design; provide clear value; and send reminders (Online available at: surveygizmo.com, retrieved 11 December 2016).

Four more strategies for increasing the response rate are proposed as follows: choose an appropriate survey length for your audience; make sure the survey is easy to take and return; contact participants multiple times; and choose the right delivery method (Online available at: scantron.com, retrieved 11 December 2016). In addition, the following strategies should be considered to help increase the survey response rate: use agency letterhead; sign a cover letter; use recognizable graphics; handwrite addresses; personalize the mailing address on the envelope and provide a salutation in a cover letter rather than using a generic approach (i.e., Dear Colleague….); avoiding traditionally busy periods such as holidays, districtwide testing days, and summer breaks; ending the questionnaire with a “thankyou”; folding the questionnaire forms into a booklet; using lightly shaded background colors; and provide adequate space for participants to answer each question completely (Online available at: cdc.gov, retrieved 11 December 2016). In addition, to increase our response rate we should (1) give sufficient information about the project i.e. purpose of the survey and how the results will be used including the terms of anonymity and confidentiality; (2) give enough amount of time to complete the questionnaire; (3) provide clear instruction on how to complete the questionnaire and how to submit it; (4) design the questionnaire to make it easy to read and


follow; (5) send reminders; (6) offer an incentive for participating our survey (Online available at: facultyinnovate.utexas.edu, retrieved 12 December 2016).

In general there are four major approaches that could be applied to improve the response rate. The first approach is to make personal delivery and collection of the completed questionnaires(N. Joungtrakul, 2013a, 2013b). The second approach is to appoint a focal point of contact for delivering and collecting the completed questionnaires in organizations(N. Joungtrakul, 2013a, 2013b). The third approach is to apply the reciprocity concept to respondents and the focal point of contact person (Christians, 2000; J. Joungtrakul, 2009; N. Joungtrakul, 2013a, 2013b). The fourth approach is to have a follow-up plan (Draugalis et al., 2008).

DISCUSSION

Discussion of this paper follows the two questions posed as the objectives of this paper: (1) what is an acceptable rate of response of questionnaires in quantitative research? (2) How to improve the rate ofresponse of questionnaires in quantitative research?

Objective 1: what is the acceptable rate of response of questionnaires in quantitative research? To respond to the first objective it was found that there is no consensus agreement on this issue. Based on Fincham (2008) the questionnaire response rate should be at 60% or above in order to have quality research results. At the same time the response rate for research in the field of pharmacy must be at 80 % or above. In general, as response rate affects the validity and reliability of the results of the research so the researchers should generally aim for 50% or higher response rate in any research project.

In practice the response rate of quantitative projects is varied. In some project the rate is 66%(Sadaengharn, Ingard, & Worakulratana, 2016); 84.16% (Tanoamchard & Promsuwan, 2016) and in some cases the rate is as high as 93.2% (Oduma & Were, 2014). However, most of the research projects conducted by Thai researchers obtained 100 % rate of return. In fact it is a requirement from several government organizations funding the research that the return rate must be at 100 %. Some Thai universities also required the same for their students' theses and dissertations. This sometimes creates a malpractice of data collection. In many cases the questionnaires were sent out much more than the number of samples originally determined according to the sampling methods and procedures. For example if 400 is the determined number of the sample group the researcher would send out 800 questionnaires and if 600 questionnaires were returned the researcher would choose only 400 out of the 600 returned questionnaires and claimed a 100 percent response rate.! It could be argued that this practice destroys the original sampling methods and procedures and could causeproblems of representativeness that could lead to poor quality of the results of the research especially in terms of validity (Draugalis et al., 2008).

Objective 2: How to improve the rate ofresponse of questionnaires in quantitative research? To respond to the second objective it is found that there are four major approaches that could be applied to improve the response rate.

The first approach is to make personal delivery and collection of the completed questionnaires. This could be done by the researcher and research assistant. It can take place at the respondents' workplaces or residences or, if students, at their place of study. In case of delivering and collecting the questionnaires at the workplace, researchers may obtain


approval and cooperation from the management of the organization to have a meeting with respondents and explain the objectives of the project and request for cooperation in completing the questionnaires. A similar approach may apply when seeking student questionnaires. Completeness of the questionnaires can be checked at that time to ensure the usable of the questionnaires. Response rate can also be assured(N. Joungtrakul, 2013a, 2013b).

The second approach is to appoint a focal point of contact for delivering and collecting the completed questionnaires in organizations. Researchers make a contact and request for cooperation from organization to appoint a focal point of contact to coordinate with the researcher and all concerned including the respondents. This person may be the Human Resource Manager or another person as designated by the organization. She or he will deliver and collect and check for the completeness of the questionnaires for the researcher(N. Joungtrakul, 2013a, 2013b).

The third approach is to apply the reciprocity concept to respondents and the focal point of contact person (Christians, 2000). Respondents and the contact person devote their time and effort to help the researcher to accomplish the task without any compensation (Marshall & Rossman, 1999). The researcher gets everything i.e. the data, the research report, monetary and non-monetary compensation such as funding, publishing of paper and promotion to higher levels. The respondents and the contact person should be given recognition in some ways. It could be a piece of token with a minimal cost such as keychain or towel etc. The contact person may be given a copy of the report of the research covered by a thank you letter from the research or by the authority of the affiliate organization or university. The organization should be treated in the same as the contact person. For respondents the token is normally given out when the questionnaire is returned and found to be a complete one. The thank you letter and the research report should be delivered as soon as possible upon the completion of the report (J. Joungtrakul, 2009).

The fourth approach is to have a follow-up plan (Draugalis et al., 2008; Fincham, 2008). Researcher should have a follow-up plan and make follow-up regularly with respondents. A follow-up may be by letters, emails, telephone calls, or personal contacts (N. Joungtrakul, 2013a, 2013b). This approach may be utilized along with the three approaches indicated above.

Each approach has its own strengths and weaknesses. The pros and cons of each approach are presented in Table 2.

TABLE 2: PROS AND CONS OF THE FOUR APPROACHES TO IMPROVING RESPONSE RATE

No. Approach Pros Cons

1 Make Personal delivery and collection of the completed questionnaires.

• Ensure the completeness of the questionnaires.

• Ensure the high rate of response.

• Time consuming • Expensive

2 Appoint a focal point of contact for delivering and collecting the completed

• Same as 1 • Less time consuming • Less expensive

• Difficult to get support and cooperation.


questionnaires. 3 Apply the reciprocity

concept to respondents and the focal point of contact person.

• Same as 1 • Time consuming • More expensive

4 Make a follow up plan and follow up with respondents regularly.

• Same as 1 • Time consuming • Less effective than 1-3

From the pros and cons of each approach presented in Table 2 the first approach of making personal delivery and collection of completed questionnaires seems to be the most effective one. However, the combination of all four approaches in one research project would enhance the rate of response of questionnaires.

CONCLUSIONS

It could be concluded that there is no universal consensus on the acceptable rate of response of questionnaires. Some scholars propose a minimum of 50 %- 70 % rate of response (Mugenda&Mugenda, 1999, cited in Oduma & Were, 2014) while others proposea minimum of 60 % for general research and 80 % or greater for the field of pharmacy (Fincham, 2008).

To improve the response rate of questionnaires there are four major approaches that could be applied: (1) The researcher makes personal delivery and collection of the completed questionnaires; (2) The researcher appoints a focal point of contact for delivering and collecting the completed questionnaires in organizations; (3) The researcher applies the reciprocity concept to respondents and the focal point of the contact person; (4) The researcher has a follow-up plan and makesfollow-up regularly with respondents(Christians, 2000; J. Joungtrakul, 2009; N. Joungtrakul, 2013a, 2013b).

RECOMMENDATIONS

To achieve the highest quality or rigorous of each research project both in terms of validity and reliability all efforts should be made to obtain the highest rate of response. The researcher should aim for at least 60% rate of response in any research project.

To improve the response rate of questionnaires the researcher should apply the four major approaches separately or in combination of all four of them in any research project. The practice of sending out the questionnaires much greater in number of the determined sample size by original sampling methods and procedures should be avoided or discontinued.

REFERENCES

Babbie, E. (2008). The Basics of Social Research (4 ed.). Belmont, CA: Thomson Wadsworth.

Chapter 5: Subjects, Participants, and Sampling by Jeffrey Oescher. (Online available at www2southeastern.edu, retrieved 5 December 2016). Chapter 8: Quantitative Sampling. (Online available at csun.edu, retrieved 5 December 2016).


Christians, C. G. (2000). Ethics and Politics in Qualitative Research. In N. K. Denzin & Y. S.

Lincoln (Eds.), Handbook of Qualitative Research (2 ed., pp. 133-155). Thousand Oaks, California: Sage.

Creswell, J. W. (2009). Research Design: Qualitative, Quantitative, and Mixed Methods

Approaches (3 ed.). Los Angeles: SAGE. Draugalis, J. R., Coons, S. J., & Plaza, C. M. (2008). Best Practices for Survey Research

Reports: A Synopsis for Authors and Reviewers. American Journal of Pharmaceutical Education, 72(1), 1-51.

Draugalis, J. R., & Plaza, C. M. (2009). Best Practices for Survey Research Reports

Revisited: Implications of Target Population, Pfobability Sampling, and Response Rate. American Journal of Pharmaceutical Education, 73(8), 142.

Fincham, J. E. (2008). Response Rates and Responsiveness for Surveys, Standards, and th

Journal. American Journal of Pharmaceutical Education, 72(2), 1-3.

How do I distribute a Survey to respondents? (Online available at: its.uiowa.edu, retrieved 10 December 2016).

How to determine population and survey sample size? (Online available at checkmarket.com, retrieved 1 December 2016).

Improving Survey Response Rates: Four Tactics to Increase Participation. (Online available at: scantron.com, retrieved 11 December 2016). Increasing Questionnaire Response Rate. (Online available at: cdc.gov, retrieved 11 December 2016). Joungtrakul, J. (2009). Industrial Democracy and Best Practice in Thailand. Saarbrucken,

Germany: LAP Lambert Academic Publishing AG & Co. KG. Joungtrakul, J., Sheehan, B., & Aticomsuwan, A. (2013). Qualitative Data Collection Tool: A

New Approach to Developing an Interview Guide. AFBE Journal, 6(2), inpress. Joungtrakul, N. (2013a). Readiness to Cope with the Free Flow of Skilled Labour in the AEC

of Engineers in the Electronics and Computer Companies (in Thai) Thailand HR Journal, 5(1), 64-88.

Joungtrakul, N. (2013b). Thai Engineers Readiness to Cope with the Free Flow of Skilled

Labour in the ASEAN Economic Community. Human Resource Development, 4 (1 ), 6-21.

Marshall, C., & Rossman, G. B. (1999). Designing Qualitative Research (3rd ed.). Thousand

Oaks, California: Sage.


Oduma, C., & Were, S. (2014). Influence of Career Development on Employee Performance in the Public University, A Case of Kenyatta University. International Journal of Social Sciences Management and Entrepreneurship, 1(2), 1-16.

Poole, 2014. What is an acceptable survey response rate? National Social Norms Center, Michigan State University. (Online available at: socialnorms.org, retrieved 11 December 2016).

Response rates. (Online available at: facultyinnovate.utexas.edu, retrieved 12 December 2016).

Response rate (survey). (Online available at: en.m.wikipedia.org, retrieved 11 December 2016).

Sadaengharn, P., Ingard, A., & Worakulratana, P. (2016). A Research Report on the Study of Sucess Indicators for Happy Workplace Implementation. Cholburi, Thailand: Faculty of Management and Tourism, Burapha University.

Sampling Methods. (Online available at cirt.gcu.edu, retrieved 1 December 2016).

Survey Response Rats. (Online available at: surveygizmo.com, retrieved 11 December 2016).

Tanoamchard, W., & Promsuwan, N. (2016). A Research Report on an Opinion Survey of

Unions' Leaders on Happy Workplace in the Eastern Thailand. Cholburi, Thaialnd: Faculty of Management and Tourism, Burapha University.

Wambui, K. M., & Gichuho, C. M. (2013). Determining Influence of Management Commitment on Development of Emotional Intellignce in Public Service - A survey of Ministries in Kenya. International Journal of Science and Research, 2(5), 387-389.

Yamane, T. . (1973). Statistics an Introductory Analysis (3 ed.). New York: Harper & Row

Publishers.


LEADERSHIP AND POST-MERGER ORGANISATIONAL IDENTITY IN AN EMERGING

MARKET: THE CASE OF KAZAKHSTAN

Altynay Nazarbay

London School of Economics and Political Science, London, UK.

[email protected]

Paul J Davis

Kazakh British Technical University, Almaty, Kazakhstan

[email protected]

ABSTRACT

Purpose: To identify leadership practices and contextual factors that influence a managers’ ability to

foster shared organisational identity during the integration process of mergers and acquisitions

(M&A).

Methods: Data encompasses semi-structured interview of nine middle-level managers of two

multinational companies in oil and gas industry of Kazakhstan, who were directly involved in M&A

integration process.

Findings: Thematic analysis of the interview data revealed leadership strategies that helped these

managers foster mutually beneficial relationship between organisational groups which were involved

in the merger, thereby laying foundation for shared organisational identity.

Originality. The present study corroborated and extended the existing research and theoretical

propositions and translated these into a list of leadership practices and contextual factors, which can

be further tested and refined in future research.

Keywords: leadership; organisational identity; mergers and acquisitions; oil industry; Kazakhstan

INTRODUCTION

Merger and acquisition (M&A) activities around the world costed approximately 4.1 trillion US

dollars in 2015 (Cristerna & Ventresca, 2015). Despite these large financial investments, 50–80% of

M&As fail to achieve their expected economic synergies (King, Dalton, Daily, & Covin, 2004;

Thanos & Papadakis, 2013). This is partly explained by social and psychological aspects that are often

neglected in the planning and implementation stages of the integration process in M&As (Birkinshaw,

Bresman, & Hakanson, 2000; Cartwright, 2005). Therefore, the focus of this research is to understand


how can organisations ensure smooth integration of merger partners by unifying their separate

organisational identities.

Employees’ psychological reaction to M&As has received considerable attention from scholars

(Cartwright, 2005). One influential approach that has been applied in studying employee reactions to

M&As is the Social Identity Approach (SIA). SIA is an overarching theoretical framework that

constitutes a number of theories (Tajfel & Turner, 1986; Turner, Hogg, Oakes, Reicher, & Wetherell,

1987). The core idea of the SIA is that people define themselves not only on the basis of distinctive

individual features but also on the basis of their group membership. There are five main motives that

underlie an individual’s tendencies to identify himself/herself with social groups (Brewer, 1991;

Baumeister & Leary, 1995; Hogg, 2000; Vignoles, 2011). These are - the need to protect and enhance

one’s self-esteem (the self-esteem motive); the need to reduce uncertainty (the meaning motive); the

need to feel included and accepted within one’s social context (the need to belong); the need to

distinguish oneself from others (the distinctiveness motive); and the need to feel connected with one’s

past, present, and future identities despite the occurrence of significant changes in one’s life (the

continuity motive).

Organisational identity (OI) forms a significant part of an employees’ self-concept (Haslam, 2004).

According to the original formulation of the concept, OI refers to the features of an organisation that

are perceived as central and distinctive when compared to another organisation (Albert & Whetten,

1985). These features can be manifested in organisational culture, strategy and practices. Research

indicates that organisational identification is associated with favourable attitudes and behaviours, such

as job satisfaction, perceived organisational justice, organisational commitment, extra-role behaviour,

and collaborative behaviour (Meal & Ashforth, 1992; Riketta, 2005; Riketta & van Dick, 2005; Chan,

2006; Bartels, Pruyn, & de Jong, 2009). Although organisational identity should be one of the

priorities during M&A integration, many organisations undermanage this issue (Giessner, 2016).

Therefore, it is important to develop an understanding of how organisational leaders can better

manage organisational identity during M&As.

Drawing on the SIA, the present study has explored the role of leaders in accomplishing post-merger

organisational identity. It sought to identify leadership practices and contextual factors that influence

managers’ ability to engage in effective organisational identity management activities during

integration process of M&A. The study also explored the role of culture in post-merger identity

formation.

This study sought to answer three questions relating to identity management in M&As:

(1) What practices do leaders use to foster post-merger organisational identification?


(2) What factors influence effectiveness of middle managers’ attempt to foster post-

merger identification?

(3) How does cultural context influence leaders’ effectiveness in managing post-merger

identity?

LITERATURE REVIEW

Group-prototypicality and group-orientedness

Although leaders can play a critical role in establishing post-merger identification, the research on this

issue is relatively sparse and there are no ready answers in the literature about what makes effective

identity management in M&A context. Nevertheless, the SIA has provided some insights. From the

perspective of SIA, there are two key elements of leadership effectiveness: group-prototypicality and

group-orientedness (van Knippenberg & van Knippenberg 2005).

Group-prototypicality refers to the extent to which a leader embodies the core characteristics of an

organisation. Prototypical leaders were shown to be more trusted and supported by other group

members than non-prototypical leaders (van Knippenberg, van Knippenberg & Bobbio, 2008).

Bobbio, van Knippenberg and van Knippenberg (2005) examined leader group-prototypicality in an

experimental merger scenario. They found that in a high leader prototypicality condition, participants

were more willing to support the change than in a low leader prototypicality condition. The

researchers argued that highly prototypical leaders were able to provide a sense of continuity to

employees during merger integration because they represent the pre-merger characteristics of the

group.

However, one problem is that leaders are unlikely to be prototypical of all organisations involved in a

merger (Giessner, Ullrich, & van Dick, 2011). At the beginning of a merger, a leader cannot be

representative of all pre-merger organisational identities and, as a result, the leader may be perceived

as “one of them” rather than “one of us”. The SIA suggests that leaders can engage in group-oriented

behaviour (e.g. fair distribution of resources, fair procedures, supportive attitudes etc.) to demonstrate

his/her commitment and dedication to the group, thereby compensating for the lack of prototypicality.

To test this assumption, van Knippenberg and van Knippenber (2005) conducted a laboratory

experiment and distributed a survey. In their study they found that when a leader’s prototypicality was

low, leader group-orientedness had strong positive impact on productivity levels and effectiveness

ratings. More specifically, Knippenberg and van Knippenber (2005) tested self-sacrificing as an

example of a leader’s group-oriented behaviour. Similar results were found for different group-

oriented behaviours of leaders, such as the appeal to collective interests (vs. self-interest) and fair

allocation of resources (Platow & van Knippenberg, 2001; Haslam, van Knippenberg, & Spears,


2006). Therefore, the less a leader is representative of all merger partners, the more he/she needs to

display group-orientedness.

Observable and projected continuity

The SIA research has also sought to identify the strategies that leaders can use to increase employees’

sense of continuity during M&As. Based on data gathered from in-depth interviews of managers in a

newly merged industrial company in Germany, Ullrich, Wieseke, and van Dick (2005) developed a

model, according to which the sense of continuity consists of observable and projected continuity.

Observable continuity refers to stability of identity and job contents between past and present. Given

that observable continuity is often relatively low for employees of a low status organisation, projected

continuity can compensate for negative effects of observable discontinuity on post-merger

identification (Ullrich et al., 2005; Giessner, 2011). Projected continuity focuses on stability of the

present identity and future identity. The authors define it as employees’ awareness of the merged

firms’ goals and the ways of achieving these goals. Projected continuity is especially important during

merger, because it can be established for employees of all of the organisations involved in the merger.

Leaders can ensure projected continuity by providing a clear and positive vision. For example, Venus

(2013) found that when CEO’s speech was perceived to advocate a vision of continuity, employees

were more likely to support the merger. The effect was even stronger when employees perceived high

levels of uncertainty. It is important to note that, in addition to a shared vision, leaders need to be

specific about the steps necessary to achieve the vision (Ullrich et al., 2005). Thus, projected

continuity can be a useful tool for leaders for increasing post-merger OI (Lupina-Wegner, Drzensky,

Ullrich, & van Dick, 2014).

Leadership in context

According to the existing literature, an effective leader displays group-orientedness and clarifies the

projected continuity for employees to establish post-merger identity. Despite these research insights,

our knowledge about effective identity management during M&A is still limited and fragmented

(Giessner et al., 2016). Most of the SIA research on leadership has been set by a strong theoretical and

experimental tradition (Steffens et al., 2014). This has led to the application of SIA research in

organisations very slow and unsystematic, and this is because the research has not provided clear and

applicable guidelines for practitioners. Therefore, the present study aims to collaborate and extend the

existing research findings and theoretical propositions to translate them into concrete set of leadership

practices that can be tested and refined further by future research.

Because SIA provides a very broad framework, a more specific theory within the SIA was chosen to

guide the study - The Intergroup Relational Identity theory (IRIT). It was developed by Hogg, van

Knippenberg, and Rast (2012) to address leadership for inter-group collaboration. The theory posits


out that rather than promoting a shared collective identity, which fails to satisfy the social identity

motives, leaders should attempt to build an inter-group relational identity, which is a social identity

defined in terms of mutually beneficial relationships between subgroups. And to achieve that, Hogg et

al. (2012; 2015) proposed three relevant strategies for building collaborative relationships between

subgroups:

(1) Identity rhetoric – communicate a message which emphasizes benefits of intergroup

collaboration and appreciation of distinctive features of each group;

(2) Boundary spanning - behaviourally exemplify collaborative interaction with subgroup

members;

(3) Leadership coalition - form a leadership coalition that includes leaders from each

subgroup.

Hogg et al. (2012) predicted that, over time, organisational members would internalise these

collaborative relationships as part of their social identities. The authors also argued that the

development of IRI could be the first step toward establishing a collective identity. The IRIT appears

to have potential implications for the M&A research and practice, because it could help leaders to

overcome disruptive social identity processes linked to psychological motives such as continuity and

distinctiveness and establish collaborative relationships between organisational groups. However, no

study, to date, has examined this theory in the M&A context. The present study will apply the IRIT to

examine effective leadership practices for post-merger identity management.

The second objective of this study is to explore challenges that are encountered by middle-level

managers when they try to establish the post-merger OI. Neither research nor practice has devoted

enough attention to provide comprehensive fit-for-purpose guidelines and resources for middle

mangers to excel in identity management (Allan & Cianni, 2011; Giessner, 2016). Middle-level

managers play a pivotal role in the M&A integration because they can directly influence and

immediately respond to employees’ concerns (Huy, 2002; Giessner, Dawson, & West, 2013).

However, they have to manage the dual pressure of adjusting to new organisation and helping others

to accept the new organisational identity (van Dijk & van Dick, 2009). The study aims to identify

factors that impact middle-level managers’ effectiveness in establishing post-merger identity.

The third objective of the study is to explore the role of national culture context in shaping post-

merger identity. This topic falls largely beyond the scope of SIA research. SIA assumes that the

motives are universal without considering the possibility that the strength of these motives can vary

depending on culture context. Research indicates that cultural norms and values can influence the


relative strength of social identity motives (Markus & Kitayama, 1991; Vignoles & Moncaster, 2007;

Feitosa, Salas, & Salazar, 2012). Therefore, the present study draws on Hofstede’s (2001) cultural

dimension framework to speculate on how post-merger identity formation may differ across various

cultural contexts. Specifically the potential effects of a strong collectivist value orientation will be

examined. Collectivist cultures appreciate the sense of belonging to a greater extent than

individualistic cultures (Feitosa et al., 2012). Thus, because of the propensity to form close

relationships with their workgroups, individuals from collectivist cultures tend to display stronger

organisational identity and greater preference towards in-groups (Gundlach, Ziynuska, & Stoner,

2006; Vignoles & Moncaster, 2007; Baker, Carson, & Carson, 2009). This may hinder leaders’

attempts to create a new post-merger identity. On the other hand, individuals from collectivist

countries also tend to score high on normative commitment (Felfe, Yan, & Six, 2008). This means

that, in collectivist culture, employees will feel strong obligation to fulfil the internalised

organisational goals and to align with the managers. This may facilitate leaders’ effectiveness in

building post-merger identity. Given these contradicting predictions, the present study will examine

whether the collectivist values facilitate or hinder the emergence of the post-merger identity. It is the

first study to examine how cultural context may impact the effectiveness of leadership in establishing

the post-merger identity.

KAZAKHSTAN

According to recent reports, the number of M&A transactions in emerging markets, like Kazakhstan,

is increasing rapidly (Cogman, Jaslowitzer, & Rapp, 2015). Human aspects of M&A are especially

important to understand in emerging economies and research suggests that local managers are not

fully aware of modern employee-management techniques (Davis, 2012). Therefore, research such as

this one, has the potential to equip leaders with relevant knowledge about how to effectively integrate

different organisational groups, which are a part of M&As, especially in emerging markets. This is the

first study to provide insights into M&A leadership practices in Kazakhstan.

RESEARCH DESIGN

The present study aimed to examine how the identity management was executed by middle managers,

who have been interviewed and what were the challenges they encountered during M&A integration

process. A qualitative method was deemed more suitable to examine the research questions because it

focuses on understanding the nature of the research problem, versus the quantitative method which

focuses more on the “quantity of observed characteristics” (Jackson, 2008). Furthermore, semi-

structured interviews allowed for greater understanding of the middle managers’ experience and their

viewpoint on the identity management during the M&A integration process.


Open-ended questions were used in interviews to provide rich content for the analysis of this study’s

research questions. The interview guideline consisted of six sections:

(1) Organisational identity. This section looked at how leaders conceptualised the term

‘organisational identity’ and how much importance they placed on it.

(2) Pre-merger identity and employee responses. To gain a greater understanding of M&A

settings, participants were first asked to describe distinctive organisational features that

constituted the pre-merger identities and then how employees responded to the differences

between such identities.

(3) Leadership practices. Participants were asked to recall their actions to foster post-merger

identification.

(4) Challenges. In this section, participants were probed about problems they encountered during

the integration process and how they dealt with them.

(5) Possible improvements. Participants were then asked to describe the ideal leadership

strategies and practices, as per them, for accomplishing post-merger identification. Finally,

they were asked about what they would have done differently had they been in charge of a

new merger integration process.

(6) Culture. Managers who had previously worked in different countries were asked an additional

question on whether they observed a cultural difference that could possibly influence the

integration.

A total of nine middle-level managers were interviewed, across various departments and belonging to

two companies. All participants were directly involved in a post-merger integration process. Six

managers belonged to Company A and three from Company B. There were five males and four

females, three local and six expatriate workers. Participants were selected through a combination of

snowball sampling and opportunity sampling. This limited access was largely due to the busy

schedule of the middle managers associated with the merger and integration activities.

The interview data was collected retrospectively, approximately 4 months after the commencement of

the integration process. Although the retrospective methods have limitations, it is less problematic in

M&A situation, because sudden and disruptive change creates sharp ‘flashbulb’ type memories

(Cartwright & Cooper, 1990). Therefore, during the data collection all integration-related issues were

salient to the participants and they were expected to be able to recall them with high level of accuracy.

The interviews were conducted at two different company offices in Kazakhstan. At the beginning of

the interview session, all participants were informed about the scope of the study. They were assured

that their responses would be anonymous and confidential. Permission to audiotape the interview was


taken from all participants. In six out of nine cases, the permission was obtained. The notes were

taken for the other three interviews. Before the interview started, participants read the Information

Sheet and signed the Consent Form. Duration of the interviews ranged from 30 to 60 minutes

depending on the participants’ availability. An average interview length was approximately 45

minutes. All audiotaped interviews were transcribed verbatim.

DISCUSSION AND FINDINGS

Leadership practices

The first objective was to gain insight into which practices were deemed effective by managers to

achieve post-merger organisational identity. The most commonly reported practices included: using

identity shaping language, communicating positive vision, displaying supportive attitudes, “walk the

talk”, treating employees fairly, and facilitating the intergroup contact. The findings on the identity

management practices resonate closely with three strategies proposed by IRIT: identity rhetoric,

boundary spanning, and leadership-coalition. The following paragraphs will consider each strategy in

detail.

Identity rhetoric

Several managers touched upon ‘language as a tool’ for post-merger identity management. They used

the identity language of inclusiveness and similarity that helped them to craft a sense of “us” for

members of two organisations. For example, leaders employed inclusive language, such as “we”,

“together”, “our company”, and “our values”, thereby emphasizing a shared group membership. They

also stressed the similarities between groups such as common values of safety and respect. The

following quotation is an example of how one respondent used language to minimise differentiation

between two organisations: “We have two buildings: Company B has a building that is called ‘X’ and

Company A is in ‘Y’ building. I stopped calling them the Company B office and the Company A

office. Instead, I call the buildings by their names. The language is very important here, because that

is already signals how you view them as one group, instead of emphasising the two separate groups”

(Participant 4).

Managers stressed the necessity of establishing and promoting a vision to create a sense of common

group membership. The vision for the merged company was to set the company apart from other

energy companies as an innovation leader. The following language was used by leaders to describe

the vision of the merged company: “exceptional organisation” and “word class portfolio”. Such

depiction of the vision provides a sense of projected continuity, reduces uncertainty, maintains

distinctiveness, and enhances self-esteem of organisational members (Ullrich et al., 2005; Lupina-

Wegner et al, 2013). This created a basis for the post-merger identity formation process.


Next, leaders explained that the vision could only be achieved through inter-organisational

collaboration and by combining the unique qualities of both companies. For example, Company A

leaders highlighted the strongest qualities of each organisation, such as Company A’s operational

excellence and Company B’s agility and strong position in trading and shipping. Stressing positive

aspects of subgroup identity and integrating them as part of the common identity can increase

perceived sense of continuity and self-esteem among group members (Seyranion, 2014). Furthermore,

the leaders’ messages conveyed that intergroup collaboration is essential to create a highly

competitive company in a low oil price environment, which is an outcome valued by both

organisations.

Finally, it was pointed out several times by many respondents that it is necessary for a leader to be

open, forthright, and provide clear information about merger processes (e.g. merger goals, objectives

and plans) to reduce uncertainty and provide meaning for employees: “I think we were struggling at

the beginning at least sharing information while we did not have much. But the realisation you come

to is that whatever we know is more than what others know. Even sharing something even if it is not a

full answer, it is important to constantly update people on what’s happening.” (Participant 2).

Boundary spanning

To solidify the positive perception of inter-organisational relationships, leaders enacted the

collaborative intergroup behaviour themselves. Most of the managers referred to trust as a critical

element of effective leadership during merger integration. To gain trust and support from employees,

leaders demonstrated consistency between their words and actions. One respondent described how he

tried to make the best of both approaches visible to employees from the early stage of integration

(Participant 6). When the best practices where identified, he implemented them immediately. These

“quick wins” also helped Company A managers to gain Company B employees’ trust and compensate

for the lack of prototypicality. When non-prototypical leaders succeed on behalf of the group, they

gain endorsement of this group (Giessner et al., 2009).

Company A leaders attempted to build a positive relationship with Company B employees by

exhibiting group-oriented behaviour. For example, selection and resourcing was considered a critical

part of integration process and had to be executed in a fair manner. Participant 6 commented that to

ensure unbiased selection, leaders from both Company A and Company B were involved in the

decision-making processes. Participant 5 described that to ensure transparency and objectivity, she

made selection decisions by thoroughly analysing competencies and activities.


Managers also recognised the centrality of employee concerns and highlighted the necessity to reach

out and listen to them. Availability of a leader can help eliminate boundaries between organisational

groups that exist in early stages of a merger: “It is very much about communicating a lot, reaching out

to people a lot, walking through the corridor, join lunches, walking to coffee corner. That is what you

need leaders to do, you need them to be there, you need to be accessible, and approachable. That is

very powerful, because that’s what people will recognise and that’s will remove boundaries between

leaders and employees” (Participant 4).

Research has shown that behaviour of proximal authority figures, such as middle managers, can guide

employees in an ambiguous merger situation (Labiance et al., 2000; Melkonian, Monin, &

Noorderhaven, 2011). By offering direct behavioural cues, leaders may increase employees’

willingness to cooperate during merger integration. A longitudinal study by Melkonion et al. (2011)

showed that this is especially true at early stages of merger and for those employees, belonging to the

lower status merger partner, who may feel more uncertainty and the need to reduce it.

It is also important to note that leadership behaviours would not directly shape the identity, but rather

underscore the rhetoric. Hogg et al. (2012) suggested that behavioural exemplarity and identity

rhetoric reinforce one another, i.e. behaviour adds to credibility of leadership rhetoric and rhetoric

helps to form an understanding of a leader’s behaviour. It is also possible that undertaking these

leadership actions at early stages of merger may lay ground for subsequent leadership effectiveness

(Hogg et al., 2012). These dynamics deserve close attention from researchers.

Leadership coalition

The third strategy, according to the IRIT, which can be used to promote shared identity in the merged

organisation, is to create a leadership coalition that includes leaders from both of the pre-merger

organisations, all of whom can effectively engage in behavioural exemplarity. Prior to the merger

integration process, the joint leadership team was created. Leaders of both organisations were

involved in the planning of the integration process, participated in the merger-related decision-making

thereby representing the collaborative relationships between groups.

There are four ways in which the joint leadership team could benefit the formation of the shared OI.

First, by emphasising the valued partnership leaders could attenuate negative consequences associated

with status differences (Wageman, Nunes, Burruss, & Hackman, 2008). Joint leadership team can

promote collaborative climate, mutual trust, and respect, thereby fostering positive inter-

organisational relationships (Hogg et al., 2012).


Second, joint leadership teams could indirectly influence employees’ attitudes towards the other

organisation. This is known as an “extended contact” effect, whereby knowing that an in-group leader

has prolonged positive relationships with an outgroup leader can lead to improved attitudes towards

that out-group (Wright, Aron, & Tropp, 2002). Thus, leadership coalitions may create a favourable

environment in the new organisation.

Third, joint leadership team provide avenues for peer interaction (i.e. with other middle managers).

Research indicates that middle managers describe peer interaction as a helpful tool during a transition,

because these managers share similar experience and can exchange information (Herzig & Jimmieson,

2006). Peer interaction reduced uncertainty and helped middle managers adjust to the new

organisation. This is a critical aspect of a merger because if a leader does not identify with the new

organisation, he/she will not be able to enact the identity management practices and increase the post-

merger identification in others (Van Dijk & Van Dick, 2009).

Forth, joint leadership teams create alignment and consistency between leaders’ actions, which

reduces ambiguities in the employees eyes and ensure transparent and structured integration process.

Intergroup contact

There is another leadership strategy that is not addressed in IRIT but appeared to be critical in the

present study. All managers emphasised the importance of creating an opportunity for interaction

between members of two organisational groups. Participant 9 referred to employees as “co-creators of

new organisational identities”. Thus, organisational identity is not only directed and imposed by

leaders, but also shaped through interaction between employees. Both formal and informal

interactions (e.g. joint work streams, get-to-know sessions, Town-hall meetings, weekly team

meetings) between members of the merging organisations could be beneficial for a number of reasons.

According to the contact hypothesis, originally developed for inter-racial encounters, contact can lead

to positive attitudes, but only under certain circumstances, which are: equal status between the groups,

cooperative relationships between the groups, opportunities for personal acquaintance between the

members, common goals, and support of authorities (Allport, 1954). Research has provided

supporting evidence for the importance of appropriate intergroup contact in reducing negative

attitudes towards out-groups (Dovido, Gaertner, & Kawakami, 2003; Pettigrew & Tropp, 2006).

More favourable attitudes are produced as a result of learning about others and creating a more

individualised representation of out-group members. For example, one may learn about other groups’

agendas and working styles through interaction, thereby reducing misunderstandings and improving


coordination. Frequent inter-group contact can lead to effective inter-group relationships (Richter,

West, van Dick, & Dawson, 2006).

The SIA cannot fully account for the beneficial effects on inter-group contact. The social identity

research focuses more on how people perceive groups in certain context and not so much on the

interaction between members of different groups (Swaab, Postmes, & Spears, 2008). In the SIA

research, scholars often use the definition of OI that views it as a relatively stable notion (Albert &

Whetten, 1985). In contrast, the interpretivist approach posits that OI is a dynamic notion that is

developed through the interaction between organisational members (Ashforth & Mael, 1989; Weick,

1995; Gioia, Schultz, & Carley, 2000; Hatch & Schultz, 2002). During M&As, organisational

members develop their shared understanding about what the organisation is becoming through

interaction with other organisational members. This is called as a sensemaking processes (Weick,

1995). A case study on an acquisition within the pharmaceutical industry found that two firms were

able to integrate successfully and preserve their pre-merger identities through the sensemaking

process, which involved negotiations of identity claims and developing mutual conceptions of the

merged organisation among employees (Bernarsid & Giustinioano, 2015). As a result of interaction,

employees transformed their representation of post-merger identity towards a more positive one and

appreciated the differences between the identities. Thus, inter-group interaction is an essential tool

that can translate the shared experience into a shared meaning (Larsson & Lubatkin, 2001). This, in

turn, facilitates the emergence of post-merger OI.

Top-down and bottom-up identity formation

The present study shows that both top-down and bottom-up approaches can work in combination to

effectively shape post-merger identity. The leaders developed a broad scheme for employees’

interpretation and actions by expressing that the desired future identity would be derived from the best

features of both organisations and mobilised routines that directed employees’ attention to achieve

this identity, such as get-to-know sessions and weekly meetings. The desired identity was then

interpreted and negotiated between two organisational groups, and the best practices of the pre-merger

organisations were imported. Managers built on the set of imported practices to create further

frameworks that defined the post-merger identity. In addition, by accepting and rejecting particular

pre-merger practices, managers reduce ambiguity about the merger objectives and signals what the

new organisation will be like, thereby continuously shaping the post-merger identity (Driori,

Wrzesniewski, & Ellis, 2013).

This suggests that top-down and bottom-up identity shaping processes are mutually reinforcing, i.e.

one gives rise to another. However, previous research focused on top-down process of post-merger

identity formation, with little consideration to bottom-up processes. Future research could explore the


mutual influence between two bottom-up and top-down processes of the post-merger identity

formation.

This integration process implemented by the leadership team is in line with the two-stage model of

identity creation that was developed based on a field study of four sequential mergers of informational

technology firms (Driori et al., 2013). They argued that boundary negotiations act as an engine for

post-merger identity formation.

Barriers to effective post-merger identity management

The second objective was to explore middle-level managers’ perspective on challenges associated

with identity management. Managers pinpointed various characteristics of the merger process design

that hampered the effectiveness of the post-merger identity management: a long temporal lag between

the deal closure and the integration process, high speed of change, late physical consummating of the

merger, and the absence of OI assessment tools.

First, the study shows that the timing of the integration process could be a critical factor. There was a

one-year lag between the deal closure and the start of the integration process. Typically, the level of

expectations among employees is very high right after the deal closure (Ranft & Lord, 2002).

Employees of both organisations expected that the merger would follow the assimilation merger

pattern. However, managers were not able to inform them about a clear integration plan until the

integration process started. As noted by managers, this prolonged period of uncertainty and perceived

identity threat led to increased stress among the acquired organisation employees. When the actual

merger pattern was announced, it did not match any of employees’ expectations. Employees of the

acquiring organisation experienced unexpected shock that threatened their OI. The mismatch between

expected and actual merger patterns had possibly weakened the post-merger identification (Gleibs,

Tauber, Viki, & Giessner, 2013). Perceived permeability of boundaries, i.e. when employees of one

organisation have access to the opportunities that are afforded to the employees of the other

organisation, presented a further threat to employees of a high-status organisation (Terry, Carey, &

Callan, 2001). Overall, the long temporal lag might deteriorate the positive perceptions of the merger

among all employees (Colombo, Conca, Buongiorno, & Ghan, 2007). This suggests that early

planning and managing employees’ expectation (e.g. informing them about the integration plan)

following the first signs of merger is required to minimise the possible negative effects on the post-

merger identification.

The second characteristic of merger design that could influence the managers’ effectiveness in

establishing the post-merger OI is the speed of change. During the merger, changes were undertaken

at a very high speed, which, according to the managers, hindered the effectiveness of the identity


shaping practices. The integration process created workload and time pressure for managers. Some

managers noted that there was an underestimation of challenges, which led to overloading and conflict

of task among employees. Most of the managers mentioned that there was not enough time to engage

in identity management practices and that there were other things that came first as a priority and

required more attention.

A survey of 232 M&As found that the optimal speed of the integration process depends on the degree

of internal (e.g. management style, organisational structure) and external relatedness (e.g.

geographical markets, customers) between the organisations prior to the merger (Homburg &

Bucerius, 2006). Given that pre-merger organisational structures were different (i.e. internal

relatedness was low) and companies were operating in the same industry (i.e. external relatedness was

high), slower speed of change would have been more beneficial.

The third obstacle to effective identity management was late physical consummation. Moving

employees into one office location did not happen until the last phase of integration. As pointed out

by many managers, Participant 1 expressed his concerns his/her views on importance of proximity

between employees to fostering post-merger identity: “This is the killer for identity shaping, an

absolute killer. The sooner you can bring and co-locate two groups of people in one location that will

be a great benefit”. Separate office locations prevented frequent face-to-face interaction between

employees of two organisations, which undermined the formation of the post-merger OI. Therefore,

intergroup proximity should be arranged right from the start of merger.

Finally, several managers pointed out the absence of appropriate tools to define, measure, and make

OI actionable in the merger. As noted by Participant 7: “It is a very difficult topic to talk about

identity. It is a tricky part of the merger, because it is very vague, when people try to make it concrete

and measure it. That is the key challenge to put on the agenda of such a big change programme”.

Despite the existence of various scales for OI measurement, such as the Organisational Identification

Questionnaire designed by Gautam, Van Dick, and Wagner (2004), it appears that Company A was

not aware of these. This suggests that managers were not equipped with necessary tools and

information to effectively manage OI during the merger. It is important that OI metrics are utilised

during the merger integration process to keep a pulse on the strength of the identification within a

newly merged organisation and provide ‘hard’ evidence for the effectiveness of identity management

practices.

The role of national culture

The present study revealed the possibility of cultural factors influencing the effectiveness of identity

management practices. Managers reported that collectivistic orientation of employees facilitated the


post-merger identity formation. Participant 4, for example, observed that compared to their Western

colleagues, employees in Kazakhstan, had a greater tendency to form groups. Another respondent

observed that employees in Kazakhstan were more inclined to build new relationships with employees

of the merging partner through informal interpersonal interaction compared to their more distant

Western counterparts (Participant 1). Given the beneficial effects of the inter-group contact discussed

above, it helped leaders to integrate two organisational groups. Another possible explanation for the

facilitating effect of the collectivism is that individuals possess a more interdependent view of self

(Markus & Kitayama, 1991; Hofstede, 2001). It is possible that employees in Kazakhstan more

readily recognised mutual interdependence and accepted inter-group relational identity.

Power distance (PD) is another cultural dimension that could increase the effectiveness of the top-

down identity formation practices. Although there is no official data on Kazakhstan Hofstede’s

dimensions, it is assumed that the PD index is similar to other post-soviet countries, such as Russian,

which scores 93% on this dimension (Hofstede, 2001). This means that unequal power distribution is

expected and accepted by less powerful organisational members. For example, Participant 7 observed

that, in Kazakhstan, employees were more prone to unequivocally accept a leaders’ identity claims. A

cross-cultural survey examined the relationships between PD and OI, but the researchers found no

significant correlation (Baker et al. 2009). However, they tested a sample of countries with a

relatively low PD index with the highest PD index of 39% (Australia). Their findings were possibly

confounded by the basement effect. This calls for further research on the relationship between PD,

post-merger identification, and leadership effectiveness.

Theoretical implications

The present research advances our theoretical and practical understanding of identity management

during M&As. The study showed that although SIA is useful to predict how an employee will react to

the new OI, it is out of the SIA’s scope to explain how the identity shaping processes are influenced

by contextual forces during M&As. This calls for a more comprehensive theorisation and examination

of leadership practices to create a post-merger identity. OI has been examined from different angles

but there has been little work on integrating and applying these approaches to practical issues, such as

M&As (He & Brown, 2013). Theory and practice would greatly benefit from studies that will explore

the leadership for post-merger identity by using an integrated approach. For example, the interpretivist

approach can complement our knowledge about the interactional process of post-merger identity

construction through sensemaking mechanism. It depicts post-merger identity as less dependent on

leaders’ understanding of OI and more on the employees’ interpretations. The present study also

showed that bringing cross-cultural perspective in to post-merger identity management research could

provide useful insights. Particularly, it may show how post-merger identity evolves, depending on the

cultural context. Because some social identity motives are more pronounced in few cultures than

others, they may require greater consideration when planning the integration process. Cross-cultural


research on leadership practices for post-merger identity could provide practical insights for cross-

border M&As. Overall the study indicates that integrating different approaches to OI can provide

fruitful insights for identity management research and practice.

Practical implications

The present study also outlined the general recommendations for managerial practice derived from the

theoretical and research insights discussed above (see Table One). Managers should tailor their

practices to specific merger context and avoid the “one-size-fits-all” approach. They should also be

aware that establishing a post-merger identity is a long-term process and that the benefits may not be

immediately visible.

Organisations should place more emphasis on interventions that support and prepare middle-level

managers to implement identity management practices (Giessner et al., 2016). HR should orchestrate

its role in providing training sessions to managers prior to the start of the integration process and

supporting them during the merger. A training session prior to the merger could be managers to help

them understand the basic concept of OI, its role in the integration processes, and identity

management practices. Most importantly managers should become aware of the psychological

motives underlying OI that should be addressed to ensure the post-merger identification.

The study also demonstrated that managers’ effectiveness was influenced by the merger design.

Therefore, it is advisable that merging organisations should plan the integration process ahead and

give considerations to how the design of the integration process can influence OI.

TABLE 1. POST-MERGER IDENTITY MANAGEMENT PRACTICES FOR

ORGANISATIONAL LEADERS.

Approach Practices Description

Top-down

Identity rhetoric

• Communicate necessity of the merger

• Use inclusive language

• Communicate compelling vision for

the future organisation

• Communicate the need for

collaborative relationships

• Highlight the strongest aspects of the

merging organisations


Walk the talk

• Embody the identity rhetoric

(behavioural exemplarity)

• Demonstrate group-oriented behaviour

towards employees of both

organisations (e.g. fairness, supportive

attitudes)

Build joint leadership teams

• Involve leaders of both organisations

in decision-making

• Develop positive collaborative

relationships with leaders of the

partner organisation

Bottom-up

Facilitate inter-group contact

• Create opportunity for face-to-face

informal and formal interaction

between employees of the merger

partners (e.g. regular team meetings)

• Allow employees to discover the best

practices of the partner organisation

and to negotiate the boundaries

CONCLUSIONS

The present study showed that an M&A poses unique challenges for establishing a shared OI and

leaders, and middle-level managers, specifically, play a crucial role in this process. The study

contributed to our knowledge of leadership for effective identity management by applying and

extending the IRIT in the M&A context. Consistent with the IRIT, the key message from the

managers was that leaders should facilitate the emergence of mutually beneficial relationships

between employees of the merger partners and this lays ground for post-merger identity. They should

ensure that the integration process does not disengage employees from their pre-merger identities, but

rather they should facilitate the emergence of a shared identity that combines the valuable features of

the pre-merger identities. Thus, homogenous OI is not a necessary outcome, at least in the early stages

of an M&A.

Therefore, the main challenge for leaders is to establish strong inter-organisational relationships while

sustaining pre-merger identities. To ensure the development of such inter-group relationships, leaders


need to promote interaction at all levels of the organisation. However, this approach could possibly

threaten and fail if the merger is not handled right. OI should be carefully managed because it is a

sensitive aspect of individual dynamics. To avoid these problems, the study generated a set of

recommendations, for managers, which addresses the social identity motives and could help managers

to promote social harmony in a newly merged organisation. Both top-down and bottom-up approaches

to the post-merger identity formation were found to be critical during the M&A integration.

The study also indicated that the effectiveness of the post-merger identity management could be

influenced by the context in which this process occurs. The M&A design and cultural context could

greatly influence the managers’ effectiveness in dealing with OI. Giving a careful consideration to the

OI issues and the related cultural factors during the planning and executing the integration process

could lay the foundations for successful post-merger identity formation.

However, the above observations do not constitute a fully developed knowledge of OI in the M&A

context, as the study was not without its limitations. First, the study provides only a snapshot of the

M&A reality. The temporal dimension is largely absent in the M&A identity management research.

Therefore, future research should conduct longitudinal studies to examine how the identity is

developed through the various stages of the merger, how to unfold the identity management strategies,

and aim to develop systematic models of the identity formation processes during M&As.

A second caveat is related to the single merger case. The study may not be representative of other

M&A cases. Future research should investigate the applicability of the leadership practices and the

identity dynamics of M&As in different types of organisational and sociocultural settings.

The third limitation is associated with the subjective nature of the qualitative research methodology.

Although the method provided in-depth insights into the M&A setting, one drawback is that it lacks

objectivity, as the study is based on the participants’ experiences and observations. Furthermore, some

interview data was collected in the unsystematic manner, because of the participants’ concerns over

confidentiality that prevented the researcher to record the interviews and some important information

was possibly missed. It is also possible that participants did not disclose full information about the

merger (e.g. internal conflicts and the sensitivity involved in the integration process). Therefore, more

rigorous research methods are required to test the ideas discussed in the study.

Overall, the present study calls for further research to gain a better understanding of the interplay

between organisational identity, leadership actions, and a local context of M&As.


REFERENCES

Albert, S., & Whetten, D. A. (1985). Organisational identity. Research in Organisational

Behaviour, 7, 263-295.

Allan, S., & Cianni, M. (2011). Middle managers: Their critical and growing role in M&A

transaction. Retrieved online on 10 June 2016: https://www.towerswatson.com/en-

GB/Insights/Newsletters/Global/strategy-at-work/2011/Middle-Managers-Their-Critical-and-

Growing-Role-in-M-and-A-Transactions

Allport, G. W. (1954). The nature of prejudice. Cambridge, MA: Perseus Books.

Ashforth, B. E., & Mael, F. A. (1989). Social Identity Theory and the Organization. Academy of

Management Review, 14, 20–39.

Baker, D. S., Carson, K. D., & Carson, P. P. (2009). An individual-level examination of the

impact of cultural values on organizational identification. Journal of Applied Management and

Entrepreneurship, 14(2), 29–43.

Bartels, J., Pruyn A., & de Jong, M. (2009). Employee identification before and after an

internal merger: a longitudinal analysis. Journal of Occupational and Organizational

Psychology, 82, 113–28.

Baumeister, R. F., & Leary, M. R. (1995). The need to belong: desire for interpersonal

attachments as a fundamental human motive. Psychological Bulletin, 117,497–529.

Birkinshaw, J., Bresman, H., & Håkansson, L. (2000). Managing the post-acquisition

integration process: how the human integration and task integration processes interact to foster

value creation. Journal of Management Studies, 37(3), 395–425.

Bobbio, A., van Knippenberg, D., & van Knippenberg, B. (2005, June). Leading change: Two

empirical studies from a social identity theory of leadership perspective. Poster presented at the

EAESP medium sized meeting ‘Academy colloquium on social identity in organizations’,

Amsterdam, the Netherlands.

Braun, V. & Clarke, V. (2006) Using thematic analysis in psychology. Qualitative Research in

Psychology, 3, 77-101.

Brewer, M. B. (1991). The social self: on being the same and different at the same time.

Personality and Social Psychology Bulletin, 17, 475–82.

Cartwright, S. (2005). Mergers and acquisitions: An update and appraisal. In G. P. Hodgkinson,

& J. K. Ford (eds.), International review of industrial and organizational psychology (Vol. 20,

pp. 1–38). Chichester, UK: Wiley.

Chan S H (2006). Organizational identification and commitment of members of a human

https://www.towerswatson.com/en-GB/Insights/Newsletters/Global/strategy-at-work/2011/Middle-Managers-Their-Critical-and-Growing-Role-in-M-and-A-Transactions




develop organization. Journal of Management, 25(3), 249- 268.

Cogman, D., Jaslowitzer, P., & Rapp, M. S. (2015). Why emerging market companies acquired

abroad. Retrieved on 10 August, 2016: http://www.mckinsey.com/business-

functions/strategy-and-corporate- finance/our-insights/why-emerging-market-companies-

acquire-abroad

Colombo, G., Conca, V., Buongiorno, M., & Ghan, L. (2007). Integrating cross-border

acquisitions: a process-oriented approach. Long-Range Planning, 40, 202–222.

Cristerna, H., & Ventresca, C. (2015). The M&A market is poised for continued growth in 2015: 2015

outlook and expected trends. Retrieved online on 14 July 2016:

https://www.jpmorgan.com/country/US/EN/insights/ma-market-poised-for- growth

Davis, P. J. (2012). HR holds back economic development in Kazakhstan. Human Resource

Management International Digest, 20(6), 43-46.

De Bernardis, L., & Giustiniano, L. (2015). Evolution of multiple organisational identities after an

M&A event. A case study from Europe. Journal of Organizational Change Management, 28(3), 1-40.

Dovidio, J. F., Gaertner, S. L., & Kawakami, K. (2003). Intergroup contact: the past, present, and the

future. Group Processes and Intergroup Relations, 6, 5–22.

Drori, I., Wrzesniewski, A., & Ellis, S. (2013) One out of many? Boundary negotiation and identity

formation in postmerger integration. Organization Science, 24(6), 1717-1741.

Feitosa, J., Salas, E., & Salazar, M. R. (2012). Social identity: Clarifying its dimensions across

culture. Psychological Topics, 21(3), 527-548.

Felfe, J., Yan, W., & Six, B. (2008).The impact of individual collectivism on commitment and its

influence on organizational citizenship behaviour and turnover in three countries. International

Journal of Cross Cultural Management, 8(2), 211–237.

Gautam, T., van Dick, R., & Wagner, U. (2004). Organizational identification and organizational

commitment: distinct aspects of two related concepts. Asian Journal of Social Psychology, 7, 301–

315.

Giessner, S. R. (2016, July 20). Organisational mergers: A behavioural perspective on identity

management. Address presented in Rotterdam School of Management (RSM), Rotterdam.

Giessner, S.R., Dawson, J.F.,& West, M.A. (2013). Job satisfaction and supportive leadership during

organizational merger. Paper presented at the Academy of Management Meeting, August 9-13, in

Lake Buena Vista (Orlando), Florida.

http://www.mckinsey.com/business-functions/strategy-and-corporate-finance/our-insights/why-emerging-market-companies-acquire-abroad



https://www.jpmorgan.com/country/US/EN/insights/ma-market-poised-for-growth


Giessner, S.R., Horton, K. E., & Humborstad, S.I.W. (2016). Identity management during

organizational mergers: Empirical insights and practical advice. Social Issues and Policy Review, 10,

47-81.

Giessner, S.R., Ullrich, J., & van Dick, R. (2011). Social identity and corporate mergers. Social and

Personality Psychology Compass, 5(6), 333-345.

Giessner, S. R., van Knippenberg, D., & Sleebos, E. (2009). License to fail? How leader group

prototypicality moderates the effects of leader performance on perceptions of leadership effectiveness.

The Leadership Quarterly, 20(3), 434–451.

Gioia, D.A., Schultz, M. & Corley, K.G. (2000). Organizational identity, image and adaptive

instability. Academy of Management Review, 25(1), 63–81.

Gleibs, I. H., Ta ̈uber, S., Viki, G. T., & Giessner, S. R. (2013). When what we get is not what we

want: the role of implemented versus desired merger patterns in support for mergers. Social

Psychology, 44(3), 177–190.

Gundlach, M., Zivnuska, S., & Stoner, J. (2006). Understanding the relationship between

individualism-collectivism and team performance through an integration of social identity theory and

the social relations model. Human Relations, 59,1603-32.

Haslam, S. A. (2004).Psychology in organizations: The social identity approach (2nd ed.). London:

Sage.

Haspeslagh, P. C., & Jemison, D. B. (1991). Managing Acquisitions: Creating Value through

Corporate Renewal. New York: The Free Press.

Hatch, M. J., & Schultz, M. (2002).The dynamics of organizational identity. Human Relations, 55,

989–1018.

He, H. & Brown, A.D. (2013) Organizational identity and organizational identification: A review of

the literature and suggestions for future research. Group and Organization Management, 38 (1), 3-35.

Herzig, S. E., & Jimmieson, N. L. (2006). Middle managers' uncertainty management during

organizational change. Leadership & Organization Development Journal, 27(8), 628 – 645.

Hofstede, G. (2001). Culture’s consequences: Comparing values, behaviors, institutions, and

organizations across nations. Thousand Oaks, CA: Sage.

Hogg, M. A. (2000). Subjective uncertainty reduction through self-categorization: A motivational

theory of social identity processes. European Review of Social Psychology, 11, 223–55.

Hogg, M. A. (2007). Uncertainty-identity theory. In M.P. Zanna (Ed.), Advances in Experimental

Social Psychology (vol. 39, pp. 69-126). San Diego, CA: Academic Press.


Hogg, M. A., van Knippenberg, D., & Rast, D. E. (2012). Intergroup leadership in organisations:

Leading across group and intergroup boundaries. Academy of Management Review, 37, 232–255.

Homburg, C., & Bucerius, M. (2006). Is speed of integration really a success factor in mergers and

acquisitions? An analysis of the role of internal and external relatedness. Strategic Management

Journal, 27, 347–67.

Huy, Q. N. (2002). Emotional balancing of organizational continuity and radical change: The

contribution of middle managers. Administrative Science Quarterly, 47, 31-69.

Jackson, S. L. (2008) Research Methods and Statistics: A Critical Thinking Approach, Wadsworth

Cengage Learning.

Jetten, J., Peters, K., & Boen, F. (2014). Leadership as social identity management: introduce the

Identity Leadership Inventory (ILI) to assess and validate a four-dimensional model. The Leadership

Quarterly, 25,1001-1024.

King, D. R., Dalton, D. R., Daily, C. M., & Covin, J. G. (2004). Meta-analyses of post-acquisition

performance: Indications of unidentified moderators. Strategic Management Journal, 25, 187– 200.

Labianca, G., Gray, B., & Brass, D. J. (2000). A grounded model of organizational schema change

during empowerment. Organization Science, 11(2), 235–257.

Larsson, R. & M. Lubatkin (2001). Achieving acculturation in mergers and acquisitions: An

international case study. Human Relations, 54(12), 1573-1599.

Lupina-Wegener, A., Drzensky, F., Ullrich, J., & van Dick, R. (2014). Focusing on the bright

tomorrow? The role of projected continuity for post-merger identification in dominant and

subordinate merging organizations. British Journal of Social Psychology, 53, 752–772.

Markus, H. R., & Kitayama, S. (1991). Culture and the self: Implications for cognition, emotion, and

motivation. Psychological Review, 98(2), 224– 253.

Meal, F. A. & Ashforth, B. E. (1992). Alumni and their alma matter: A partial test of the reformulated

model of organizational identification. Journal of Organisational Behaviour, 13, 103-123.

Melkonian, T., Monin, P., & Niels, N. (2011). Distributive justice, procedural justice, exemplarity,

and employees’ willingness to cooperate in M&A integration process: an analysis of the Air France-

KLM merger. Human Resource Management, 50(6), 809-837.

M&A statistics (2016). Retrieved from https://imaa-institute.org/statistics-mergers-acquisitions/

Olkkonen, M. E.,& Lipponen, J. (2006). Relationship between organizational justice, identification

with organization and work unit and group-related outcome. Organisational Behaviour and Human

Decision Process, 100, 202-215.

https://imaa-institute.org/statistics-mergers-acquisitions/


Pettigrew, T. F., & Tropp, L. (2000). Does intergroup contact reduce prejudice? Recent meta- analytic

findings. In S. Oskamp (Ed.), Reducing prejudice and discrimination (pp. 93–114). Mahwah, NJ:

LEA.

Platow, M.J. & van Knippenberg, D. (2001). A social identity analysis of leadership endorsement:

The effects of leader ingroup prototypicality and distributive intergroup fairness. Personality and

Social Psychology Bulletin, 27, 1508-1519.

Platow, M.J., van Knippenberg, D., Haslam, S.A., van Knippenberg, B. & Spears, R. (2006). A

special gift we bestow on you for being representative of us: Considering leader charisma from a self-

categorization perspective. The British Journal of Social Psychology, 45, 303-320.

Ranft, A. L. & Lord, M. D. (2002). Acquiring new technologies and capabilities: A grounded model

of acquisition implementation. Organization Science, 13(4), 420–441.

Richter, A., West, M. A., van Dick, R., & Dawson, J. F. (2006). Boundary spanners’ identification,

intergroup contact and effective intergroup relations. Academy of Management Journal, 49,1252–

1269.

Riketta M (2005). Organizational identification: A meta-analysis. Journal of Vocational Behavior, 66,

358-384.

Riketta, M., & van Dick, R. (2005). Foci of attachment in organizations: A meta-analytic comparison

of the strength and correlates of workgroup versus organizational identification and commitment.

Journal of Vocational Behavior, 67, 490–510.

Seyranian, V. (2014).Social identity framing communication strategies for mobilizing social change.

The Leadership Quarterly, 25(3), 468-486.

Steffens, N. K., Haslam, S. A., Reicher, S.D., Platow, M. J., Frasen, K., Yang, J., Ryan, M. K.,

Swaab, R. I., Postmes, T., & Spears, R. (2008).Identity formation in multiparty negotiations.Social

Psychology, 47(1), 167-187.

Tajfel, H., & Turner, J. C. (1986).The social identity theory of intergroup behaviour. In S. Worchel, &

W.G. Austin (eds.), Psychology of intergroup relations. Chicago: Nelson-Hall, 7–24.

Terry, D.J., Carey, C.J., & Callan, V.J. (2001). Employee adjustment to an organizational merger: An

intergroup perspective. Personality and Social Psychology Bulletin, 27, 267-280.

Thanos, I. C., & Papadakis, V. M. (2013). Unbundling acquisition performance: How do they perform

and how can this be measured. In D. Faulkner, S. Teerikangas, & R. Joseph (eds.), Handbookof

mergers & acquisitions (pp. 114–147). Oxford: Oxford University Press.

Turner, J. C., Hogg, M. A., Oakes, P. J., Reicher, S. D., &Wetherell, M. S. (1987).Rediscovering

thesocial group: A self-categorization theory. Oxford, England: Basil Blackwell.


Ullrich, J., Wieseke, J., & Van Dick, R. (2005). Continuity and change in mergers and acquisitions: A

social identity case study of a German industrial merger. Journal of Management Studies, 42,1549–

1569.

van Dick, R., Ullrich, J., & Tissington, P. A. (2006). Working under a black cloud: How to sustain

organizational identification after a merger. British Journal of Management, 17, 69–79.

van Dijk, R. L., & van Dick, R. (2009). Navigating organizational change: Change leaders, employee

resistance and work-based identities. Journal of Change Management, 9, 143–163.

van Knippenberg, D. (2003). “Intergroup Relations in Organizations,” in M. A. West, D. Tjosvold, &

K. G. Smith (eds.), International Handbook Of Organizational Teamwork and Cooperative Working.

Chichester, UK: Wiley, 381–400.

van Knippenberg, D. (2011). Embodying who we are: Leader group prototypicality and leadership

effectiveness. The Leadership Quarterly, 22 (6), 1078-1091.

van Knippenberg, B. & van Knippenberg, D. (2005). Leader self-sacrifice and leadership

effectiveness: The moderating role of leader prototypicality. Journal of Applied Psychology, 90 (1),

25-37.

van Knippenberg, D., van Knippenberg, B. & Bobbio, A. (2008). Leaders as agents of continuity: Self

continuity and resistance to collective change. In F. Sani (Ed.), Self-continuity: Individual and

collective perspectives (pp. 175-186). New York: Psychology Press.

Venus, M. (2013). Demystifying visionary leadership: In search of the essence of effective vision

communication. Rotterdam: Erasmus Research Institute of Management (PhD Series 289).

Vignoles, V. L. (2011). Identity motives. In K. Luycke, S. J. Schwartz, & V. L. Vignoles (Eds.),

Handbook of identity theory and research (pp. 403-432). New York, NY: Springer.

Vignoles, L. V., & Moncaster, N. (2007). Identity motives and in-group favouritism: A new approach

to individual differences in inter-group discrimination. British Journal of Social Psychology, 46, 91-

113.

Wageman, R., Nunes, D. A., Burruss, J. A., & Hackman, J. R. (2008). Senior Leadership Teams:

What it takes to make them great. Boston: Harvard Business School Press.

Weick, K.E. (1995), Sensemaking in Organizations, SAGE, Thousand Oaks, CA.

Wright, S. C., Aron, A., & Tropp, L. R. (2002). Including others (and their groups) in the self: Self-

expansion and inter-group relations. In J. P. Forgas & K. Williams (Eds.), The social self: Cognitive,

interpersonal and intergroup perspectives: 343–363. New York: Psychology Press.


VALUE-ADDED MODELS FOR IMPROVING UNIVERSITY SCHOOL EFFECTIVENESS IN THAILAND

Penpak Pheunpha, UbonRatchitani University, and Suchada Bowarnkittiwong Faculty of Education, Chulalongkorn University, Thailand

(Penpak Pheunpha, Associate Dean, Academic, AbonRatchatani University, Facultyl of Management Science, penpheu@ hotmail.com)

ABSTRACT

Educational policy makers are constantly seeking “outcomes-based” performance indicators to accurately measure school effectiveness. Most of these indicators concentrate on student achievement inside the classroom. However, many non-school variables affect student performance, including but not limited to demographics. This study uses hierarchical linear models (HLM) to examine the influence of these variables or school performance. Using data from 49 schools in Thailand, we found that about 70% of the difference between schools’ average GPA could be accounted for by non-school elements. To more accurately measure a school’s effect on student achievement, we therefore propose a “value-added” scoring system for schools that takes variables like student background and previous achievement into account. Such systems help educational stakeholders to see a school’s role in student achievement more clearly and present a more accounts picture of school effectiveness. Used properly, value-added scores are reliable, precise, and consistent measures. Key Words: Value-added models, School effectiveness, School Improvement, Accountability

INTRODUCTION

During the past few decades, administrators, policy-makers and researchers have focused on school effectiveness and school improvement both in Thailand and internationally. Research on educational effectiveness conducted in several countries such as the United States, England, Australia, Netherlands, China, Canada, Japan, and Korea, revealed that classroom and school effects such as teaching quality and school policy are important variables that explain student achievement (Darmarwan & Keeves, 2006; Reynolds, 2006; Secker & Lissitz,1997). Multiple indicators of school effectiveness have potential benefits in the field of school transparency, self-improvement and accountability (Roekel, 2013). School effectiveness refers to the performance of the school (Scheerens, 2000). In other words, a school is seen as ‘effective’ when it achieves what it sets out to achieve (Ninan, 2006). The performance of the school can be expressed as the output of the school, which in this study, is measured in terms of average student achievement of the students in the school at the end of one year of formal schooling. The question of school effectiveness is interesting because it is well known that schools differ in performance; however, researchers have debated the appropriate method to adjust for student background in measuring and reporting school effects (e.g. Dyer, Linn, & Patton, 1969; Macro, 1974 cited from Raudenbush & Willms,1995). According to Hill (1995), raw scores do not provide an accurate indicator of the contribution of differences among schools. Meaningful assessment of school effectiveness relies on a measure of achievement that has been adjusted to take into account the social composition of the student groups attending the schools. Among the numerous methods for measuring school effectiveness, the “value-added analysis” or “value-added model” has often been considered (Manzi et al., 2010; OECD, 2008).

The value-added model is part of a family of statistical models that are employed to make inferences about the effectiveness of educational units, usually schools and/or teachers (Braun & Wainer, 2007). Value-added models are a very special class of growth models that attach meaning to student level change by comparing it to an expected amount (e.g., one-year) of change as well as to change that other students with other educational experiences or backgrounds have established


(Lissitz et al., 2006). As Meyer and Dokumaci (2009) describe, a value-added model is a quasi-experimental statistical model that yields estimates of the contribution of schools, classrooms, teachers, or other educational units to student achievement (or other student outcomes), controlling for contributory (non-school) variables of student growth in achievement, such as students’ prior attainment, student and family demographic variables. The purpose is to clarify valid and fair comparisons of student outcomes across schools, although the schools may serve very different student populations.

Measurement of the value-added models involving multilevel modeling is now widely applied to school effectiveness and school improvement research. Due to the hierarchical structure found in educational data sets, the multilevel generalized linear model was used in this analysis. In statistical parlance, the value-added model is calculated as the predictor of the random school-effect in multilevel regression (Raudenbush & Willms, 1995; Tekwe et al., 2004). The value-added model recognizes that students have different levels of capability and come from different environments, and that these factors will influence a student’s ability to expand their knowledge base (Educational Policy and Research Division, 2007). For the objective of this study, school effectiveness is defined as the value-added score or value-added residual (Secker & Lissitz, 1997; Meyer, 1997; McPherson, 1993) from hierarchical linear models. Value-added residuals are those that measure achievement via GPA over and above what would have been expected from attending a comparable school. Value-added residuals seek to “level the playing field” by assessing school effectiveness after controlling for a set of variables that is beyond the school’s control, such as school environmental characteristics (e.g. school size, school category) and pre-existing student characteristics (e.g. gender, socioeconomic status, expectation, and opportunity to learn). Moreover, the prior attainment of the students is always considered a control variable (Manzi et al., 2010; Sanders, 2000).

In Thailand, there is a public organization called the Office for National Education Standards and Quality Assessment (ONESQA) which aims to develop the criteria and methods for external quality assessment. ONESQA evaluates the quality of educational institutions. External quality assessment of all educational institutions will be conducted at least once every five years. The assessment outcomes will be submitted to the agencies concerned and the general public accordingly. ONESQA also has eight basic indicators for assessing school practice. The total possible score of all indicators for learners is 80, which consists of 1) a score out of 10 for having a healthy body and mind 2) a score out of 10 for having virtue, morality, and desired value 3) a score out of 10 for continuously seeking knowledge 4) a score out of 10 for being a critical thinker/problem solver 5) a score out of 20 for student achievement 6) a score out of 10 for effective student-centered instructions 7) a score out of 5 for efficient administrative and school development, and finally, 8) a score out of 5 for developing internal quality assurances in school and its affiliates. As a result, student achievement is the most heavily weighted portion of ONESQA’s assessment of school performance. After the implementation of ONESQA indicators, some schools developed policies to inflate the grades of their students in an attempt to improve their school performance level.

Additionally, student achievement is a raw score with many confounders related to students’

prior attainment, expectations, and other family background variables. (Rowe, 2000; Sanders, 2000; Evans, 2008; Noell, 2006; Jong et al., 2006). Furthermore, a school’s evaluation depends upon the proportion of its students scoring at or above a threshold on a test in each subject area. Such students are proclaimed “proficient” while students scoring below the threshold are deemed “not proficient.” A common criticism is that this approach is unfair to schools within which students entered with comparatively low levels of cognitive skill. Even if all schools are equally effective at promoting learning, such schools are less likely to display high proficiency rates in the long run than are schools serving students whose initial skills are high (Reardon & Raudenbush, 2008). Some researchers and educators (Ronsiri, 2007; Opadwattana, 2006; Oppkeaw, 2005) recommended that the ONESQA modify its indicators to use other indicators such as national test scores or school value-added scores


in test results, rather than raw school averages. Thus, it is important to measure school effectiveness using the value-added model in Thailand.

Given many issues from using the student achievement indicator, we are interested in measuring school effectiveness by using value-added models concerning non-school variables that cause or contribute to effectiveness or some specific kind of intervention in the school process. We are interested in non-school factors that affect student achievement. This paper examines the residual score of student achievement after controlling non-school variables. It also identifies the change in school ranks when comparing school performance with student achievement (raw score) versus residual score or value-added score. Value-added model is reliable, valid, fair, acceptable, and has more details than other approaches such as using raw scores or input-output measure, because this model will be able to demonstrate non-school variable controls which affect student attainment in different levels. Based on this approach, we will know how to improve school in the future.

RESEARCH QUESTIONS

The four research questions for this study are:

1. Is student achievement associated with student level variables and school level variables? 2. How are inferences about the association between and strength and direction of school

value-added scores and school achievement (GPA: grade point average) affected by differences in student demographic variables, a set of input resource variables and a set of school contextual variables?

3. To what extent are value-added scores sensitive to changes in measures of student achievement?

4. What is the difference between ranking the school by value-added score versus school achievement?

Our study examined variations among schools using hierarchical linear models (HLM) to test whether extracurricular factors that have a theoretical fundamental are capable of capturing the unique contribution of specific school practices such as teaching quality, policy, and leadership affect student achievement. Multilevel value-added models allow us to control for differences due to student demographic, inputs, and contextual external school, as well. In our models, variations associated with students’ prior achievement, gender, age, students’ socioeconomic status, students’ opportunity studies outside school, and student expectations about pursuit of advanced grades or degrees was controlled before school level variables were considered. School level factors consist of average prior attainment per school, student-teacher ratio, school facilities, school size (small, medium, large and very large) and school sector (public or private) were controlled for in this study before value-added residuals were estimated. Dependent variable specification Grade Point Average: GPA is the most important part of the ONESQA assessment of school achievement all students. Our study measured by collecting 9th-grade students about their 8th-grade GPAs. We collected our data in the students’ first semester of 9th-grade and therefore had not yet established their 9th-grade GPAs. Independent variable specification

Student demographics: Our literature review found many possible variables to use in our value-added models. For this study, we selected six of the most commonly-used variables in school effectiveness research for adjusting in HLM models. Based on the literatures, we included gender (GENDER, e.g. Kyriakides, 2004; Levacic & Jenkins, 2006), age (AGE, e.g. Kyriakides, 2004; Levacic & Jenkins, 2006), prior attainment (PRIOR, students’ sixth-grade grade point average e.g. Booker & Isenberg, 2008; Peng, Thomas, Yang & Li, 2006; Levacic & Jenkins, 2006; Keeve et al., 2005; Noell, 2006; Secker & Lissitz, 1997; Meyer, 1996), students expectations to pursue further education (EXPECT, e.g. Secker & Lissitz, 1997; Creemers & Kyriakides, 2009), socio-economic status defined as family income per month (SES, e.g. Maeyer et al., 2010; French, Peevely & Stanley,


2008; Peng, Thomas, Yang & Li, 2006; Secker & Lissitz, 1997), and student’s opportunity to learn defined as the student’s ability to study outside of school (OPPORT, e.g. Kyriakides,2004; Jong et al., 2004). These variables are beyond the schools’ control and represent the “what students bring to school.”

Input variables: We used three variables consisting of the average of students’ prior attainment (MnPRIOR), the student-teacher ratio in each school (RATIO, e.g. Levacic & Jenkins, 2006; Hanushek, 1995), and school facilities (FACILITY). We aggregated student prior attainment to adjust for Level-2 because teachers may have graded different groups of students in their classrooms differently depending upon the preserved abilities or backgrounds of their students (Daring-Hammond, 1995; Bennett, Gottesman, Rock, & Cerullo, 1993 cited from Martinez et al. 2009). School facilities are defined as how well a classroom’s equipped with materials for teaching and learning for instance, computers, overheads, televisions, media, chemicals for experiments in lab, microscopes, etc. (FACILITY, e.g. Fuller and Clarke, 1994 cited from Scheerens, 2000).

School contextual variables: We also used two contextual variables: school size (SIZE, e.g. Keeves, et al., 2005; Kyriakides, 2004), and school sector (SECTOR, e.g. Secker & Lissitz, 1997; Hofman et al., 2002).

Details of all predictors and dependent variable are presented in the Appendix, Table 11. We can diagram all variables in our conceptual framework Figure 1

FIGURE 1: A CONCEPTUAL FRAMEWORK FOR RESEARCH

Feedback

Student demographic - Gender

-SES - Prior attainment

Outputs Achievement of Students’ eighth-grade, (Grade Point Average: GPA)

- Age

- Opportunity to learn - Student’s expectation

Inputs - Student-teacher ratio -Access to facilities or materials M i

School context

• School size • School sector (private, public)

School And

Teacher Practices

- Value-added

Feedback

Analysis with HLM


A conceptual framework for our Hierarchical Linear Models (HLM) study is shown in Figure 2

FIGURE 2: A CONCEPUAL FRAMEWORK FOR R$ESEARCH ANALYSIS

Y.j is mean grade point average for school j Rij is level-1 residual or random effect for student i in school j

U0j is level-2 residual or value added score for school j

METHODOLOGY

Sample Two-staged random sampling was used to select one classroom from each of 50 schools, five academic areas in secondary schools in Bangkok and Nonthaburi Province, Thailand. However, only 49 schools have data completed for analysis. Since there was one school where we were unable to obtain GPA and prior-attainment score for student, and since these variables were important for computing our hierarchical linear equation models, we chose to delete this particular school. All students studied in the third Mutthayomsuksa (equal to ninth-grade in the United States, n = 1,852) in the first semester in 2010. There were 21 to 54 students per classroom. The average number of students per school was 38. The sample was 54.8% female and 45.2% male. 22 (42.9%) of the 49 schools were private schools and 27 (57.1%) were public schools.

Materials The questionnaire was used in this study to ask students about demographic background

information including gender, age, prior attainment (GPA in sixth-grade when they studied in primary school), socioeconomic status (SES), opportunity to study outside school, student expectations about

Level-1 Student demographic - Gender - Age - SES - Prior attainment - Opportunity to learn - Student’s expectation

Level-2

Inputs - Student-teacher ratio

-Access to facilities or materials M i tt i t

School context

- School size - School sector (private,

public)

GPA

(Yij)

Mean GPA (Y.j)

School.j residual ) U0j

Studentij residual )Rij(

Level 2 (Between Schools)

Level 1 (Between Students)


pursuit of advanced grades or degrees (EXPECT), GPA (in eighth grade), total number of students and teachers in school, school facility levels, school sector, and school size. Analytical approach

The first step was to run a model without independent variables, which we called the “unconditional model.” It was fitted to provide estimates of the variance components at each level using a hierarchical linear model, and to present useful preliminary information about how much variation in the outcome lies within and between schools and to offer the reliability of each school’s sample mean as an estimate of its true population mean (Raudenbush and Bryk, 2002). The unconditional model for student and school levels can be stated in the equations as follows: Level-1 model

Yij = B0j + Rij Level-2 model

B0j = G00 + U0j Where: Yij is the grade point average of student i in school j; B0j is the mean score in school j;

and Rij is the deviation of each student score from the mean score in the school (the random student effect). At the school-level model (level-2), school means are a function of a grand mean for all schools in the samples (G00), and a random effect specific to each school (U0j). The mixed unconditional model is this: Yij = G00 + U0j + Rij where: U0j and Rij are assumed normally distributed.

The second step undertaken was to estimate effects in which student demographic variables were added to the level-1 equation in the unconditional model (by controlling for student demographics). At this stage, a step-up approach was followed to examine which of the six student-level variables (listed in Table11 in the Appendix) was significantly influenced by GPA (at p ≤ 0.05). The six variables of GENDER, AGE, PRIOR, SES, OPPORT, and EXPECT were also found to be important and significant in previous studies. Therefore, we included them at this stage. A set of equations we call the “conditional model” is as the same as one-way ANCOVA with random effects in Hierarchical Linear Models of Raudenbush and Bryk (2002). The equations are presented in the Appendix.

The gender, student expectations, opportunity to study outside school were un-centered while other variables (prior attainment, socioeconomic status) were grand-mean-centered in the HLM analysis, so that the intercept term would represent the average GPA score for student characteristics (B0j) (Darmawan and Keeves, 2006). Missing data was replaced by the group mean. Regression coefficients were B1j, B2j, B3j, …, B6j, respectively; hence, intercept and regression coefficients were adjusted for differences among schools on these variables. Grand mean centering reduces estimation bias that might result if other significant predictors were not specified in the models (Bryk & Raudenbush, 2002). We constrained terms of B1j, B2j, B3j, …, B6j to be the same fixed value for each level-2 unit because we would like to test for B0j based on the GPA intercept.

The third step was to estimate school effectiveness by controlling a set of input factors (Mean Prior attainment (MnPrior), RATIO, FACILITY) and a set of school contextual factors (SIZE, SECTOR) affecting GPA. Then we applied an analysis conditional model and took into account 5 factors to calculate mean-GPA equation level-2. A set of new equations we call the “hypothetical model” and a detail of all variables in the models are presented in the Appendix.

RESULTS

The un-standardized means, standard deviations, minimum, and maximum values of all variables in the two levels (student and school levels) are included in Table 1. All of the variables used in the hierarchical linear models were standardized to have a mean of 0 and standard deviation of 1. Coefficients for these variables can be interpreted as the change in student achievement expected for 1 unit (1 unit = 1 standard deviation) change in the variable. Among the 1,852 students in this study, the average GPA is 3.04, standard deviation is 0.74, ranging from 0.21 to 4.00. The average prior attainment of students is 3.34, standard deviation is 0.58, ranging from 1.00 to 4.00. The average students’ ages is 14 years and 8 months, ranging from 12 years and 3 months to 18 years and 6 months, and the average socio-economic status (income per month, 30 Baht = 1 US dollar) is 1120.52


dollars, ranging from $23.33 to $20,000. Opportunity to study outside school, and student expectations mean values are 2.73, 3.59 respectively. At school level variables, the average student-teacher ratio is approximately 19 students per teacher. Students and teachers believed their schools were sufficiently equipped with materials; on average, the schools were given a score of 3.40 from 5 (FACILITY). Descriptive statistics for contributory variables both student and school levels are shown in Table 1.

TABLE 1: DESCRIPTIVE STATISTICS FOR COMNTRIBUTORY VARIABLES IN

STUDENT LEVEL.

Variables Number Mean S. D. Min Max STUDENT LEVEL

AGE (Years) 1852 14.67 0.53 12.25 18.50 PRIOR 1852 3.34 0.58 1.00 4.00 SES (US dollar) 1852 1120.52 1590.95 23.33 20,000 OPPORT 1852 2.73 1.51 1.00 5.00 EXPECT 1852 3.59 0.68 1.00 5.00 GPA (output) 1852 3.04 0.74 0.21 4.00

SCHOOL LEVEL MnPRIOR 49 3.32 0.33 2.58 3.96

SIZE 49 2.82 1.03 1.00 4.00 RATIO 49 18.88 4.22 10.97 29.00 FACILITY 49 3.40 0.39 2.69 4.57

Correlations between predictor variables and outcome variable (GPA) in level-1 (student-level) are presented in Table 2. The highest association is GPA and PRIOR (r = 0.65, p < 0.01). The second strongest relationship is GPA and OPPORT (r = 0.26, p < 0.01), and it is followed by GPA and EXPECT, GPA and GENDER, and GPA and SES. For associations among predictors, correlation between GENDER and PRIOR is a strongest correlation (r = 0.22, p < 0.01), and it is also followed by SES and OPPORT, and OPPORT and PRIOR. TABLE 2: CORRELATIONS BETWEEN PREDICTOR VARIABLES AND GPA IN LEVEL 1

(STUDENT LEVEL)

Variables GPA PRIOR GENDER AGE SES EXPECT OPPORT GPA 1.00

PRIOR 0.65** 1.00 GENDER -0.18** -0.22** 1.00

AGE -0.04 -0.04 -0.02 1.00 SES 0.09** 0.07** 0.06* -0.03 1.00

EXPECT 0.23** 0.14 0.04 0.06* 0.02 1.00 OPPORT 0.26** 0.20** -0.05* -0.07** 0.20** 0.11** 1.00

* Correlation is significant at the 0.05 level, ** Correlation is significant at the 0.01 level.

Correlations between predictor variables and GPA in level-2 (school-level) are presented in Table 3. In the school-level, the highest correlation is between GPA and MnPRIOR (r = 0.84, p < 0.01). The second strongest correlation is between GPA and SIZE (r = 0.45, p < 0.01). For other pairs correlations between predictor variables and GPA are not significant. For associations between predictors, the correlation between SIZE and FACLITY is the strongest one (r = 0.52, p < 0.01). The second strongest correlation is between SIZE and MnPRIOR (r = 0.43, p < 0.01). The third and


the forth correlations are between SIZE and RATIO (r = 0.33, p < 0.05), FACILITY and MnPRIOR (r=.30, p < 0.05), respectively. Other correlation pairs are not significant.

TABLE 3: CORR#ELATIONS BETWEEN VARIABLES IN LEVEL 2 AND SCHOOL LEVEL.

Variables GPA MnPRIO

R SECTOR SIZE RATIO FACILITY GPA 1.00

MnPRIOR 0.84** 1.00

SECTOR -0.04 0.05 1.00

SIZE 0.45** 0.43** 0.04 1.00

RATIO 0.20 0.03 0.19 0.33* 1.00 FACILITY 0.23 0.30* -0.18 0.52** -0.04 1.00

* Correlation is significant at the 0.05 level, ** Correlation is significant at the 0.01 level.

ANALYSIS RESULTS OF HIERARCHICAL LINEAR MODEL We analyzed the data using a combination of statistical methods. First, we started with the unconditional model, which has no adjusting predictors. This model is equivalent to the one-way ANOVA with random effects. In the fixed Effect, from Table 4, we can see that the grand-mean GPA across 49 schools is 3.009. This has a standard error of 0.074 and yields a 95% confident interval of 3.009 ± 1.96(√0.263) = (2.496, 3.522). The t-test in unconditional model is 40.886, and p < 0.01, which indicates that the grand mean, γ00, isn’t null. Table 4 also lists restricted maximum likelihood estimates of the variance components. At the student level, Var� (rij) = 𝜎𝜎�2= 0.287. At the school level, τ00 = 0.263, which is the variance of the true school means, β0j, around the grand mean, γ00. One of the purposes of estimating this unconditional model is to assess the degree of GPA variances between schools. A common metric for these variances is the Intra-Class Correlation (ICC), which measures the proportion of the variance in GPA between schools. ICC can be computed as follows: ICC = τ00/(τ00+σ2) = 0.263/(0.263+0.287) = 0.4817, indicating that there is about 48.17% variance student achievement (GPA) between schools and about 51.83% between students. For the unconditional model output, �̂�𝜆= 0.97, indicating that the sample means tend to be quite reliable as indicators of the true school means (Raudenbush & Bryk, 2002). The value of chi-square (χ2) is 1701.5 with 48 degrees of freedom. The null hypothesis is implausible (p < 0.01) which is indicating significant variation does exist among schools in their achievement (GPA). Why do schools differ? Then we can modify the model by adding Level 1 variables to unconditional model. The result for analysis unconditional model is showed in Table 4.

TABLE 4: FINAL ESTIMATION OF FIXED EFFECTS AND RANDOM EFFECTS FOR UNCONDITIONAL MODEL

Fixed Effect

Coefficient Standard Error T-ratio d.f. p-value

Intercept GPA,G00 3.009 0.074 40.886 48 0.000 Random effect Standard

Deviation Variance

Component Total

variance df Chi-square P-value

INTERCEPT, U0 Level-1, Rij

0.513 0.536

0.263 0.287

0.550 48 1701.484 0.000

The outcome variable is Grade Point Average.

THE HYPOTHETICAL MODEL


Since student demographic variables have such a large impact on school achievement levels, we must find a way to minimize the inequity between schools caused by different student populations. Our hypothetical model controlled for the four meaningful student demographic variables (PRIOR, EXPECT, GENDER, and OPPORT) to see what differences if any still existed between schools.

As previously noted, a student’s prior achievement has a very strong association with his/her GPA-when PRIOR increase by 1 unit, we can expect GPA to increase by 0.313 point on average. Similarly, increase in EXPECT and OPPORT tend to predict higher GPA by 0.079 point and 0.034 point respectively, GENDER, on the other hand, is associated with a decrease in GPA. Because girls tend to outperform boys academically in Thailand, an increase by 1 unit in a school’s male population predicts a 0.06 point drop in school GPA.

When we control for these demographic variables, a different pictures of between school comparisons enlarges. Table 7 shows that the random effect variance component drops noticeably in the hypothetical model, a total unexplained variance of 0.259. On the school level, the only meaningful variables in terms of GPA prediction were MnPRIOR (0.419 point per S.D.) and RATIO (0.009 point per S.D.) Our data show a substantial range of achievement among the 49 school studied.

Since student demographic variables have such a large impact on school achievement levels, we must find a way to minimize the inequity between schools caused by these variables. Our hypothetical model examined five variables at the school level to try to locate the difference between schools. We discovered that mean prior attainment (MnPRIOR) and student-teacher ratio (RATIO) had a significant effect on mean GPA (Table 7). On average, when the MnPRIOR increases by 1 unit (1 unit = 1 S.D.), a school’s mean GPA increased by 0.419 points. When RATIO increases by 1 unit, mean GPA went up by 0.009 points. Other factors such as sector and facilities did not have a significant effect on school GPA.

At the student level, ∧

Var (rij) = σ2 = 0.196. τ00 is the variance of the true school means, βoj which is mean of individual school around the grand mean (the grand mean is the mean of all school

combined), γ00, at the school level. The estimated variability in these school means is ∧

Var (βoj) = ∧

Var (U0j) = τ00= 0.063. To gauge the magnitude of the variation among schools in their mean achievement levels, it is useful to calculate the plausible range values for these means. Under the

normality assumption, 95% of the school means fall in the range, 3.022 ± 1.96( 063.0 ) = (2.53, 3.51). This is considerably smaller than the range of plausible values in the conditional model, which is (2.338, 3.708). This indicates a substantial range in average achievement among 49 schools in our data. The value of τ00 is significantly greater than zero (χ2= 559.248, p < 0.01), indicating that some schools have different means. The hypothetical models results are presented in Table 5.


TABLE 5: FINAL ESTIMATION OF FIXED EFFECTS AND RANDOM EFFECTS FOR THE HYPOTHETICAL MODEL (WITH ROBUST STANDARD ERROR)

Fixed Effect Coefficient Standard error t-ratio Df p-value

GPA –intercept2, G00 3.022 0.035 87.055 43 0.000 ZMnPRIOR, G01 0.426 0.068 6.217 43 0.000 ZSECTOR, G02 -0.065 0.036 -1.819 43 0.075 ZSIZES, G03 0.006 0.059 0.108 43 0.915 ZRATIO, G04 0.092 0.043 2.129 43 0.039 ZFACILITY, G05 -0.022 0.037 -0.594 43 0.556 ZGENDER, G10 -0.060 0.017 -3.625 1840 0.001 ZAGE, G20 0.002 0.014 0.109 1840 0.914 ZPRIOR, G30 0.313 0.030 10.541 1840 0.000 ZSES, G40 -0.011 0.013 -0.851 1840 0.395

ZEXPECT, G50 0.079 0.014 5.643 1840 0.000

ZOPPORT, G60 0.034 0.012 2.924 1840 0.004

Random Effect Standard Deviation

Variance Component

total observed variance df χ2 p-value

GPA-intercept, U0j 0.251 0.063 0.259 43 559.248 0.000

Level-1, R ij 0.442 0.196

As Table 5 illustrates, the hypothetical model accounts for a significant part of the overall

random variance between schools. It also narrows the range of near school GPAs in the study from student demographics, input variables, and contextual outside school variables to student achievement. Although the reliability of the sample as measured by GPA intercept is slightly lower than that of the conditional model (0.920 vs. 0.957), the hypothetical model still seems to be a sound indicator of true school average. In this model, R2 ( the proportion of parameter variance explained by the model) is 0.76, indicating that 76% of between school variance in GPA can be explained by mean prior attainment, student-teacher ratio, prior attainment, student expectations, opportunity to study outside school, and gender. The hypothetical model explains 69.9% of the total variance in our data, 18.6% more than was explained by the conditional model. The correlation between student’s GPA and the mean GPA of his/her school has now dropped to 0.119 (τ00/(τ00+σ2) = 0.063/(0.063+0.196) = 0.119). See Table 6.

TABLE 6 : RELIABILITY AND VARIANCE FOR HYPOTHETICAL MODEL

Hypothetical Model Yij is GPA (grade point average of 9thgrade students) Reliability (Rxx’) (Parameter Var./Total Variance)

B0 0.920

Proportions Parameter Variance: Explained (R2) (τ�00 (uncon)–τ�00 (hypo))/τ�00 (uncon)

Tau U0 0.760 (0.263 – 0.063)/0.263

Total Variance Explained by Model(Rxx’*R2) Tau U0 0.699 (0.920*0.760)

After controlling for student demographics, input we analyzed the hypothetical models by

controlling for student demographics, input variables, and school contextual variables, we can create “ value-added” score for each school Using Empirical Bayes (EB) and the Ordinary Least Square (OLS) estimates (Raudenbush & Bryk, 2002). Value-added school rankings differed significantly from GPA-mean ranking. Some schools move up in rank when non-school variables were taken into account, while other schools dropped significantly. In Table 9, we had chosen 9 representative schools to show the possible changes when using value-added scoring. Schools A – C should the largest


negative change between GPA mean and value-added rankings, moving from the top 20 into the bottom 20. Schools G – I, on the other hand showed the largest positive changes, moving from near the bottom of the schools studied into the top 10. Other schools, such as D – F remain relatively unchanged. 48 of the 49 schools studied, rank did change despite the moderately strong correlation between mean GPA and value-added scores. This is because mean GPA only explains 24% of the value-added score.

The correlation between mean GPA and mean prior attainment is rather high at 0.84. In other words, the best predictor of a student’s current GPA is his/her previous GPA. Further, the relationship between A school’s mean GPA ranking and its mean prior attainment ranking is nearly perfect (0.99); mean GPA appears to measure mostly the prior levels of a school’s students. Value-added scores, on the other hand, have (achievement) no relationship with PRIOR, and thus may reveal other aspects of school effectiveness. See Table 7 and Table 8.

TABLE 7: SCHOOL EFFECTIVENESS RANKS WITH GPS-MEANAND VALUE-ADDED SCORES, EXAMPLE OF 9 SCHOOLS.

School ID

GPA- Mean

Value-added Score

School Ranks with GPA-Mean

School Ranks With Value-added score

School Ranks Changed

A 3.93 -.090 1 33 -32 B 3.51 -.106 11 36 -25 C 3.18 -.254 19 42 -23 D 3.52 .179 10 11 -1

E 3.77 .283 5 5 0

F 2.89 -.051 30 29 1

G 2.31 .182 44 10 34

H 2.72 .372 35 2 33

I 2.75 .253 34 9 25

TABLE 8: CORRELATION BETWEEN GPA RANKS, VAS RANKS,PRIOR

ATTAINMENT,, GPA, AND VAS. Variables GPA

RANKS VAS RANKS PRIOR

ATTAIN GPA VAS

GPA RANKS 1.00 VAS RANKS 0.48** 1.00 PRIOR ATTAIN -0.84** 0.01 1.00 GPA -0.99** -0.47** 0.84** 1.00 VAS -.49** -0.99** 0.00 0.49** 1.00

**. Correlation is significant at 0.01 level, VAS is value-added score

CONCLUSIONS AND DISCUSSIONS

This study reveals that measuring student achievement is far more complicated than it first appears. Although changes in student GPA often provide some information about a school’s effectiveness, other non-academic variables exert a huge influence on student learning outcomes. Therefore, any teacher evaluation system used by a school must take non-academic extracurricular factors into account.

This study uses a multilevel linear regression model to create a “value-added” score for school effectiveness. This model includes non-school information such as prior student attainment (PRIOR), student expectations (EXPECT), and student extracurricular study opportunities (OPPORT) along with GPA to measure effectiveness of schools. Upon examination of the data, we gathered a


few conclusions. On the student level, the strongest predictor of GPA was prior attainment. This reinforces findings by (Martinez et al., 2009; Evans, 2008; Ray, Evans & McCormack, 2008; Fitz-Gibbon, 1997). High student expectations and more opportunities for outside learning also tend to produce higher GPAs. In other words, grade point average appears to be related more strongly to factors beyond a school’s direct control than it is to any given teaching or administrative strategy.

One surprising result of the study was that schools with higher student-to-teacher ratios outperformed schools with lower student-to-teacher ratios. At first, this might seem to contradict conventional wisdom: how can larger class sizes actually benefit students? The larger schools in our data were also some of most prestigious schools in Thailand; presumably, they attracted some of the highest achieving students and most effective teachers. This situation differs greatly from schools with high student to teacher ratios in the United States, which usually come about as the result of cutbacks and overcrowdings.

The influence of non-school factors on school effectiveness is even more pronounced when we compare schools to one another rather than looking at achievement on the student or class levels. The bivariate correlation between value-added school rankings and mean GPA scores alone is not strong (r = 0.47). Given that non-school variables seem to affect student achievement, accounting for up to 22% of GPA disparities between schools, value-added ranking appears to be a more accurate measurement of overall school effectiveness than simply comparing GPAs. Because education data sets are already multilevel (A combination of individual achievement and aggregated school-level statistics), a multi-level value-added model is the most appropriate tool for measuring school performance. At Level 1, non-school variables account for 51.3% of the difference between schools, with an additional 18% at Level 2 for a total of almost 70% of school effectiveness explained by non-school variables. With such a large influence on school performance, it seems clear that non-school factors must be measured along with GPA. Advocates of value-added models argue that measurements take student background into account to provide a more accurate record of student and school achievement. We therefore recommend that ONESQA and other educational assessment institutes use value-added scores instead of raw GPA scores. Using a value-added score would also remove a major incentive for grade inflation in Thai schools.

Value-added measurement is just one tool among many in evaluating a school’s effectiveness (Chester, 2005). Other criteria such as learning rates (Von Hippel, 2009; Chatterji, 2002; Kupermintz, 2002), students and teacher satisfaction, school self-evaluation, and outside reviewers are necessary to form a complete picture of a school. Value-added scores should ,therefore, not be treated as a “magic wand” to evaluate school performance (Thomas, 2001). To prevent “mono-operation bias” (Maeyer et al., 2010), a variety of criteria should be used. Nevertheless, in our study, the random variance in GPA-intercept is still significant, so value-added measurements serve a very important function.

Our research into value-added assessment has opened up many other potential areas of research. This study focuses on the relationship between GPA, school effectiveness, and non-school variables. It would be interesting to explore the relationship between the non-school variables we have and national test scores. Examining longitudinal data over a longer time period would reveal the strength of the correlations between GPA and non-school variables that we have described here. In addition, further research could seek to explain the influence of particular variables such as gender or class size. Finally, our statistical model can be further refined though the inclusion of other variables to account for random effects. The information gathered from this and other value added assessment projects can be used to reform educational policy on both the classroom and legislative levels.

Acknowledgements This research was supported by a scholarship under the Strategic Scholarships for Frontier Research Network, Commission on Higher Education, Thailand. The authors would like to thank Professor Jose Felipe Martinez for editing and sharing his insights and knowledge for this paper. We are also grateful to Professor Dr. Mike Seltzer and our friends in RAC class at UCLA for suggesting and sharing insights and knowledge for this paper, as well. The authors particularly thank Johnny Lin, Marcus Desmond Harmon, Qian Li, Mathew Palmer, Jia Bai, Casey O’Neill, and Heejung Park, as well as the reviewers and editors, for their helpful comments.

REFERENCES


Booker, K., & Isenberg, E. (2008). Measuring school effectiveness in Memphis. New Leaders for New Schools. New York: MPR Reference No.: 6325-530.

Braun, H. & Wainer, H. (2007). Value-added Modeling. In C.R. Rao & S. Sinharay, S. (Eds.) Handbook of statistics 26, Psychometrics (475-501). Amsterdam: Elsevier.

Chester, M. D.(2005). Making valid and consistent inferences about school effectiveness from multiple measure. Ohio Department of Education. Educational Measurement: Issues and Practice.

Chatterji, M. (2002). Models and methods for examining standards-based reforms and accountability initiatives: Have to tools of inquiry answered pressing questions on improving school? Review of Educational Research, 72(3), 345-386.

Creemers, B., & Kyriakides, L. (2009). Situational effects of the school factors included in the dynamic model of educational effectiveness. South African Journal of Education. 29, 293-315.

Darling-Hammond, L. (1995). Equity issues in performance-based assessment. In M. T. Nettles & A.L. Nettles (Eds.), Equity and excellence in in educational testing and assessment (pp. 89-114). Boston: Kluwer.

Darmawan, I. G. N., & Keeves, J. P. (2006). Accountability of teachers and schools: A value-added approach. International Education Journal, 7(2), 174-188.

De Jong, R., Westerhof, K. J., & Kruiter, J. H. (2004). Empirical evidence of a comprehensive model of school effectiveness: A multilevel study in mathematics in the 1th year of junior general education in the Netherlands. School Effectiveness and School Improvement, 15, 3-13.

Education Policy and Research Division, Australian. (2007). Value-added measures for school improvement. Department of Education and Early Childhood Development, 13.

Evans, H. (2008). Value-added in England schools. Department for Children, Schools, Families England. [email protected].

Fitz-Gibbon, C. T. (1997). The value-added national project final report: feasibility studies for a national system of value-added indicators. London: School Curriculum and Assessment Authority.

French, P. E., Peevely, G. L., & Stanley, R. E. (2008). Measuring perceived school board effectiveness in Tennessee: the latest survey results. Intl Journal of Public Administration. 31, 211 – 243.

Hill, P. (1995). Value-added measures of achievement. LARTV seminar Series, No. 44, May, 1995. Hofman, R. H., Hofman, W. H. A., & Guldemond, H. (2002). School governance, culture, and student

achievement. International Journal of Leadership in Education, 5(3), 249 – 272. Keeves, J. P., Hungi, N., & Afrassa. (2005). Measuring value added effects across

schools: should schools be compared in performance?. Studies in Educational Evaluation, 31, 247 – 266.

Kupermintz, H. (2002). Value-added assessment of teachers: The empirical evidence. In A. Molar (Ed.), School reform proposals: The research evidence (pp. 217-234). Greenwich, CT: Information Age Publishing.

Kyriakides, L. (2004). Differential school effectiveness in relative to sex and social class:some implications for policy evaluation. Educational Research and Evaluation, 10(2), 141-161.

Levacic, R., & Jenkins, A. (2006). Evaluating the effectiveness of specialist school in England. School Effectiveness and School Improvement. 17(3), 229 – 254.

Lissitz, R. W., Doran, H., Schafer, W. D., & Willhoft, J. (2006). Growth modeling, value-added modeling and linking: an introduction. Longitudinal Value-Added Models of Student Performance. Maple Grove, Minnesota, Jam Press,1- 46.

Meyer, R. H. (1996). Value-added indicators of school performance. Http://books.nap.edu/openbook.phd?record_id=5143&page=199.

Maeyer, Van den Bergh, H., Rymenans, R., Van Petegem, P., & Rijlaarsdam, G. (2010). Effectiveness criteria in school effectiveness studies: Further research on the choice for multivariate model. Educational Research Review, 5(2010), 81- 69.

Manzi, J., San Martin, E, & Van Bellegem, S. (2010). School System Evaluation by value-added analysis under endogeneity. International Association for Research

mailto:[email protected]

http://books.nap.edu/openbook.phd?record_id=5143&page=199


and Teaching. Economics, Finance, Operations Research, Econometrics and Statistics (ECORE). Ecore discussion paper.

Martinez, J. F., Stecher, B, & Borko, H. (2009). Classroom assessment practices, teacher judgments, and student achievement in mathematics: Evidence from the ECLS. Educational Assessment, 14, 78 – 102.

Meyer, R. H. (1997). Value-added indicators of school performance: a primer. Economics of Education Review, 16. 283 – 301.

McPherson, A. (1993). Measuring added value in school. London: National Commission on Education.

Meyer, R. H., & Dokumaci, E. (2009). Value-added models and the next generation of assessments. Paper present at Exploratory Seminar: Measurement Challenges Within to Race to the Top Agenda December 2009. Download copies at http://www.k12center.org/publication.html

Noell, G. H., (2006). Assessing teacher preparation program effectiveness: a pilot examination of value added approaches. Value Added Teacher Preparation Program Assessment. Department of Psychology, Louisiana University.

Ninan, M. (2006). School climate and its impact on school effectiveness: a case study. Paper presented at the International Congress for School Effectiveness and Improvement at Fort Lauderdale, Florida-USA January 04, 2006.

OECD. (2008). Measuring improvements in learning outcomes. Best Practices to Assess the Value-Added of School. Organization for Economic Co-operation and Development OECD.

Oppkeaw, A. (2005). A comparison of predictive validity of undergraduate achievement, controlling intelligence-free factor using an equate and non equate secondary school grade point average. Department of Educational Research and Psychology, Chulalongkorn University. Thailand.

Opadwattana, W. (2006). A comparison of predictive validity of selected components by using accumulative average score equating and previous components of central university admission system. Department of Educational Research and Psychology, Chulalongkorn University. Thailand.

Peng, W. J., Thomas, S. M., Yang, X., & Li, J. (2006). Development school evaluation methods to improve the quality of schooling in China: a pilot ‘value added’ study. Assessment in Education, 13(2), 135-154.

Ray, A., Evans, H., & McCormack, T. (2008). The use of national value-added models for school improvement in English schools. Department for Children, Schools and Families, London United Kingdom.

Reardon, S. F., & Raudenbush, S. W. (2008). Assumptions of value-added models for estimating school effects. Papered for the National Conference on Value-Added Modeling.

Reynolds, D. (2006). World class schools: Some methodological and substantive finding and implications of the international school effectiveness research project. Educational Research and Evaluation, 12(6), 535 – 560.

Roekel, D. V. (2013). Multiple Indicators of school effectiveness. NEA Education Policy and Practice Department, NEA National Education Association. Washington, D.C.

Ronsiri, S. (2008). An analysis of the state of learning assessment for upper secondary school students. Department of Educational Research and Psychology, Chulalongkorn University. Thailand.

Rowe, K. J. (2000). Assessment, League tables and school effectiveness: considering the issues and ‘let’s get real’!. Journal of Educational Enquiry, 1(1), 73 – 98.

Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and Data analysis methods. (second edition). California: sage Publications, Inc.

Raudenbush, S. W., & Willms, J. D. (1995). The estimation of school effects. Journal of Educational and Behavioral Statistics, 20, 307 – 336.

Sanders, W. L., (2000). Value-added assessment from student achievement data: opportunity and hurdles. Journal of Personnel Evaluation in Education 14(4), 329-339.

Scheerens, J. (2000). Improving school effective. Amsterdam: North-Holland Publishing Company. (Paris, UNESCO).

http://www.k12center.org/publication.html


Tekwe, C.D., Carter, R. L., Ma, C. X., Algina, J., Lucas, M.E., & Roth, J., (2004). An Empirical comparison of statistical models for value-added assessment of school performance. Journal of Educational and Behavioral Statistics, 29, 11-36.

Thomas, S. (2001). Dimensions of secondary school effectiveness: comparative analyses across region. School Effectiveness and School Improvement, 12(3).

The Department of Education and Early Childhood Development. (2007). Value-added measures for school improvement. Education Policy and Research Division. Paper,13.

Von Hipple, P.T. (2009). Achievement, learning, and seasonal impact as measures of school effectiveness: it’s better to be valid than reliable. School Effectiveness and School Improvement, 20(2), 187-213.

Secker. C. E., & Lissitz. R. W. (1997). Estimating school value-added effectiveness: consequence of respecification of hierarchical linear model. Paper presented at the American Educational Research Association Annual Meeting. Chicago.

AFBE · PDF fileFollowing a review of the literature discussion is made and then conclusions...

Documents

Transcript of AFBE · PDF fileFollowing a review of the literature discussion is made and then conclusions...