12. Left-right party ideology and government policies: A meta-analysis

Kurs Public Choice

WS 2012/13

Prof. Dr. Friedrich Schneider Johannes Kepler Universität Linz Tel.: 0043-732-2468-8210, Fax: -8209 Institut für Volkswirtschaftslehre E-mail: friedrich.schneider@jku.at Altenbergerstraße 69 http://www.econ.jku.at/schneider A-4040 Linz-Auhof

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 1 / 43

Left-right party ideology and government policies: A meta-analysis

CONTENT

1. Introduction

2. Data and Methods

2.1. Selection of cases

2.2. Factors in the meta-analysis

3. Empirical Results

3.1. Overall left-right partisan effects

3.2. Mediating factors

3. Overview and Discussion

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 2 / 43

1. Introduction

(1) The question of the impact of political parties on policy has been the object of much research and debate in the past decades. The main theoretical debate concerns about the relative weight of socio-economic and political variables in determining policy outputs.

(2) Advocates of the so-called ‘convergence’ school of thought argue that industrialized societies of the twentieth century have become increasingly similar, facing the same kind of problems and applying the same kind of solutions.

Consequently, the argument goes, political, institutional and cultural differences do not matter much when it comes to explaining variations in policy outputs.

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 3 / 43

1. Introduction (cont.)

(3) In reaction to the convergence argument, policy scholars have suggested that indeed politics does matter. Without denying the importance of economic factors, advocates of the ‘politics matters’ school of thought argue that there is a correlation between partisan variables and policy outputs.

(4) Much empirical research on the policy impact of political parties has been published in the form of comparative analyses based on precise quantitative definitions and measurements within the well-established scientific framework of partisan theory (Hibbs 1977, 1992).

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 4 / 43

1. Introduction (cont.)

(5) Ceteris paribus, changes in the left-right party composition of government are hypothesized to be related to changes in policy. This is expressed by a general equation of the form:

Policy outputs = f (left-right variations in the party composition of government, variations in controlling factors).

(6) This general equation has been applied, in a large number of studies comparatively to measure partisan influence on government actions in a variety of policy domains across different countries.

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 5 / 43

1. Introduction (cont.)

(7) The quantitative literature on partisan theory displays a high level of accumulation but it has not achieved a comparable level of scientific aggregation. By accumulation one means that one has a large literature (with publications numbering in the hundreds).

(8) The goal in this lecture is to contribute to this debate by summarizing how the partisan influence literature assesses the relationship between the left-right party composition of government and changes in public policy through a meta-analysis of empirical tests of partisan theory in OECD countries.

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 6 / 43

1. Introduction (cont.)

(9) A meta-analysis is an analysis of analyses. It is the statistical analysis of a large collection of test results coming from individual studies for the purpose of integrating the findings:

i. In meta-analysis, ‘the findings of multiple studies should be regarded as a complex data set, no more comprehensible without statistical analysis than would be hundreds of data points in one study.

ii. The statistical methods used to accomplish this task are similar to the ones used in primary analyses of the original data. The major difference is that each case in a meta-analysis is an estimate obtained from the original data whereas each case in a primary analysis is obtained directly from the original data.

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 7 / 43

1. Introduction (cont.)

iii. A meta-analysis is useful not only to draw out general patterns from a wide variety of empirical sources but also because it enables researchers to resolve disputes in the literature, to determine which factors have contributed to systematic differences across studies, and to identify areas that have been neglected and warrant further investigation.

iv. The meta-analysis helps identify both substantive influences on research findings and artifacts resulting from particular research methods and data analysis techniques.

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 8 / 43

2. Data and methods

(1) Although they share a common scholarly purpose, meta-analyses (MAs) and literature reviews (LRs) are quite different from each other.

(2) LRs examine how a particular phenomenon has been studied by scholars in published works.

(3) MAs deal with specific empirical hypotheses and try to investigate, on the basis of the empirical work so far published, whether there is an impact of some independent variables upon dependent variables, and, if so, what the magnitude of this impact is.

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 9 / 43

2. Data and methods (cont.)

(4) Like any other empirical research, MAs are vulnerable to various threats to validity:

i. The main threat to validity in this MA is heterogeneity among the studies incorporated in the analysis.

ii. One tries to respond to this threat to validity; first, by establishing explicit criteria for the selection of individual studies in order to ensure that the tests within each study are comparable; and second, by coding relevant published information in the form of variables most susceptible to influencing the predictive performance of each test.

iii. The criteria for the selection of cases and the rules for constructing the variables in the MA were organized in a codebook which served as guidance for researchers to identify individual parameter estimates and their relevant attributes in each study.

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 10 / 43

2. Data and methods2.1. Selection of cases

(1) One concern in selecting individual cases for the meta-analysis was comprehensiveness and/or comparability.

To have confidence in measures, one needs to be sure that the selection process was representative of the available material. To achieve this goal, one tries to minimize discretion in selection by considering everything we could find in the published record.

(2) Over 600 studies (mainly journal articles and book chapters) were identified, containing empirical results.

(3) In this MA we are interested in reporting cross-national comparisons and include results from cross-sectional studies and times-series cross-sectional studies only.

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 11 / 43

2. Data and methods2.1. Selection of cases (cont.)

(4) The validity of the meta-analysis may also be threatened by the wide variety of measures of party ideology used by partisan theorists to explain policy outputs in individual studies (the ‘explanans’). To limit heterogeneity in party ideology variables, one considers only tests in which ideology is defined as either left or right.

(5) Another possible threat to validity in MA is the lack of sample homogeneity in the estimation methods. One of the objectives of this MA is to extract overall measures of the relationship between the party composition of government and policy outputs from single coefficients found in individual studies.

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 12 / 43

2. Data and methods2.2. Factors in the meta-analysis

(1) The dependent variable in this MA is the predictive performance of individual tests of partisan theory, measured on the basis of parameter estimates of party impact.

(2) These estimates are operationalized in terms of three attributes:

i. estimates that support the partisan theory hypothesis (the coefficient is significant and in the predicted direction) are recorded as ‘successes’;

ii. estimates that fail to confirm the hypothesis (the coefficient fails the test of statistical significance) are reported as ‘failures’;

iii. estimates that contradict the hypothesis (the coefficient is significant and in the wrong direction) are reported as ‘anomalies’.

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 13 / 43

2. Data and methods2.2. Factors in the meta-analysis (cont.)

(3) Individual tests of partisan impact on welfare policies typically postulate that, ceteris paribus, welfare intervention by the state is positively correlated with the presence of leftist parties in government and negatively correlated with the presence of rightist parties in government (denoted L+, R−).

(4) The approach postulates that spending on development assistance is positively correlated with the presence of the left in government and negatively correlated with the presence of the right in government (L+, R−).

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 14 / 43

2. Data and methods2.2. Factors in the meta-analysis (cont.)

(5) The hypothesis is reversed for military spending and military alliance membership: military spending and state involvement in military alliances correlates positively with the presence of the right in government and negatively with the presence of the left in government (L−, R+).

(6) One important variant of this approach is the political business-cycle hypothesis, which states that the standard relationship (L+, R−) holds for state intervention in macro-economic policy, public debt as a percentage of GDP, and progressivity of taxes. The reversed relationship (L−, R+) holds for privatization, and budget balance as a percentage of GDP.

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 15 / 43

2. Data and methods2.2. Factors in the meta-analysis (cont.)

(7) A final approach examines how partisan factors affect the overall size of government. The approach postulates that, ceteris paribus, government size correlates positively with the presence of the left in government and negatively with the presence of the right in government (L+, R−).

(8) Individual tests of partisan theory also vary in terms of how they operationalize observed government activity. Most studies operationalize government activity in terms of public spending or revenues from taxes (often expressed as a percentage of GDP).

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 16 / 43

2.2. Factors in the meta-analysis (cont.)

(9) In order to assess the impact of variation in the definition of policy domains on our conclusions, we record whether individual parameter estimates apply:

i. to social welfare policies,

ii. the size of the state,

iii. economic policies, and

iv. for foreign and defense policies.

We also record whether the measure of government activity in each test is based on:

a) financial data (e.g., public spending), or

b) non-financial data (e.g. regulation or administrative action).

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 17 / 43

2.2. Factors in the meta-analysis (cont.)

(10) Individual authors often calculate the presence of the left or the right in government in the form of sophisticated indices. However, so far we do not have a standardized normalized measure.

(11) There is some disagreement about the range of parties to be included in the measure. Most studies examined in this MA report the presence of the left and/or the right in government on the basis of either leftist parties, a left-right scale, conservative parties, or major parties of the right.

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 18 / 43

2.2. Factors in the meta-analysis (cont.)

(12) Individual measures of the party composition of government also differ in their definitions of relative party strength. The most frequently used measure is the party composition of the cabinet.

(13) This is justified on the grounds that in advanced liberal democracies, political power is basically exercised through the cabinet and that governmental policy is directly related to the allocation of ministerial portfolios among different parties.

´WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 19 / 43

2. Data and methods2.2. Factors in the meta-analysis (cont.)

(14) Another factor that may affect the validity of our meta-analysis is the variation in the specification of the model in each individual study, i.e. the number and nature of explanatory or control variables that partisan theorists explicitly include in their datasets.

(15) There is evidence that the statistical significance and the direction of estimates of partisan impact may be affected by institutional factors such as:

the division of power between parties in and out of government,

the frequency of electoral competition, or the time a government has been in power.

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 20 / 43

3. Empirical Results

(1) Out of over 600 studies identified in the bibliographic search, 43 proved to correspond to our selection criteria.

(2) The studies are listed in chronological order in table 3 along with information:

about the period, the number of countries over which the tests were run, the relevant policy domain, and the left-right attribute used to define the party composition of

government variable.

(3) Table 1 reports only the summary information for each study. Individual tests within each study may involve different time periods, countries, policy domains and different measures of party ideology.

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 21 / 43

3. Empirical Results (cont.)

Table 1: Characteristics of quantitative studies on the ideology-policy relationship – Part 1

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 22 / 43

Table 1: Characteristics of quantitative studies on the ideology-policy relationship – Part 2

WS 2012/13 23 / 43

3. Empirical Results3.1. Overall left-right partisan effects

(1) Table 1 also presents the number of relevant tests in each study and the frequencies with which these tests are reported as …

i. ‘successes’ (a coefficient associated with a right or a left partisan variable is significant (p < 0.10) and in the correct direction), or

ii. ‘anomalies’ (a coefficient associated with the right or the left is significant and in the opposite direction to the one hypothesized), or

iii. ‘failures’ (a non-significant coefficient)

… in this study.

(2) Once the results of each individual test have been sorted into successes, failures and anomalies, we can use the simple ‘vote-counting’ meta-analytic procedure to assess the overall relationship between the left-right party composition of government (the ‘explanans’) and policy outputs (the ‘explanandum’).

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 24 / 43

3. Empirical Results3.1. Overall left-right partisan effects (cont.)

(3) Looking at the overall distribution of tests (bottom row of table 1), we see that out of the 693 tests:

only 154 (22%) are reported as successes, and 48 tests (7%) are reported as anomalies; the remaining 491 tests (71%) fail the statistical significance test

(at 10 percent) and are reported as failures.

(4) The success rates appear remarkably stable over time: 86 out of 385 tests published in or before 1986 are reported as successes

(22.3%); 68 out of 308 tests published after 1987 are reported as successes

(22.1%).

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 25 / 43

3. Empirical Results3.2. Mediating factors

(1) Clearly, these numbers cannot be interpreted as evidence in support of the left-right party impact hypothesis. Applying the vote-counting technique to the data suggests that, since the ‘failure’ category is the modal category, we cannot reject the null (convergence) hypothesis of no party impact based on the average predictive power of individual tests.

(2) Table 2 presents a breakdown of individual tests by policy domain. We see from the table that 69 (24%) out of 288 tests in the welfare policy domain are reported as successes.

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 26 / 43

Table 2: Success rate of individual tests by policy domain.

WS 2012/13 27 / 43

3. Empirical Results3.2. Mediating factors

(3) The difference in predictive performance across specific areas within the welfare domain (from a low of 19% to a high of 27%) is not statistically significant. Next we see that 59 (37%) out of 161 tests of the size of the state are reported as successes.

(4) As in the welfare domain, the deviations from the mean in the size of the state domain are not statistically significant.

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 28 / 43

3. Empirical Results3.2. Mediating factors (cont.)

(5) The overall success rates are much lower in the two remaining policy domains. There are 20 successes (13%) out of 148 tests in the foreign affairs and defense policy domain and only 6 successes (6%) out of 96 tests in the economic policy domain. The deviations from the mean success rate are statistically significant in both the foreign affairs and the economic policy domains.

(6) Tests of left-right party impact on the level of state intervention in the economy have a (relatively) very high degree of success (39%). On the other hand, the success rate in the public budget and taxation areas is virtually zero.

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 29 / 43

Table 3: Success rate of individual tests across other variables.

WS 2012/13 30 / 43

3. Empirical Results3.2. Mediating factors (cont.)

(7) We see from table 3 that there are twice as many tests based on the presence of the left (472) as compared with tests based on the presence of the right (221) in our sample.

(8) Tests that report the presence of the right generate a higher rate of success on average (25%) than tests that report the presence of the left (21%).

(9) Looking at the party strength variable of table 3, we see that using the percentage of cabinet portfolios, although the preferred method of calculating party strength, has no statistical impact on success when compared with alternative methods.

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 31 / 43

3. Empirical Results3.2. Mediating factors (cont.)

(10) Next, we see that 79 (28%) out of 279 multivariate tests are reported as successes against 75 (18%) out of 414 bivariate tests.

(11) The paired difference for the model specification variable is statistically significant, as is the paired difference for sample size (the proportion of success is 20% for smaller samples against 28% for larger samples) and for the measure of government activity (the rate of success is 23% for financial data compared with 12% for non-financial data).

(12) The paired difference for the historical period variable is also statistically significant: 92 (27%) out of 339 tests based on the post-1973 period are reported as successes against 64 (18%) out of 354 tests based on the pre-1973 period.

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 32 / 43

3. Empirical Results3.2. Mediating factors (cont.)

Table 4: Success rate of individual tests by historical period and by policy domain.

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 33 / 43

3. Empirical Results3.2. Mediating factors (cont.)

(13) An interesting pattern emerges when we cross-tabulate the historical period variable with policy domains (table 4).

(14) Post-1973 tests of party impact on the size of the state, foreign affairs and economic policy have significantly higher average success rates than pre-1973 tests of party impact in these policy domains.

(15) However, individual tests of party impact on welfare policy follow the opposite direction.

The average success rate of estimates of party impact on welfare is smaller for the post-1973 period than for the pre-1973 period.

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 34 / 43

3. Empirical Results3.2. Mediating factors (cont.)

(16) Table 5 presents a multivariate logistic regression model in which the dependent variable takes the value of 1 when a test is reported as a success and 0 otherwise. There were too few anomaly cases to include them as a distinct variable in this analysis.

(17) Three policy domain variables are included in the model to assess the predictive performance of tests of party impact on the size of the state (total public spending and taxation), foreign affairs, and the economy as compared with the predictive performance of tests of party impact on welfare (the reference policy domain).

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 35 / 43

Table 5: Multivariate Logit analysis of individual tests’ predictive performance.

WS 2012/13 36 / 43

3. Empirical Results3.2. Mediating factors (cont.)

(18) The next two variables in the model are dichotomous variables intended to assess whether the predictive performance of individual tests is affected by the use of non-financial policy data (as opposed to financial policy data) and by the inclusion of a left (instead of right) partisan variable.

(19) From Table 5, we see that, ceteris paribus, the success rate of individual tests is not significantly affected by how partisan impact is operationalized.

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 37 / 43

3. Empirical Results3.2. Mediating factors (cont.)

(20) Whether individual tests use the left or the right to measure partisan impacts does not affect predictive performance, and neither does the definition of party strength (although measures of party strength based on the percent of popular votes perform better than measures based on cabinet portfolio allocation).

(21) The data also rule out sample size and the use of financial (as opposed to non-financial) data as causes of variation in predictive performance.

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 38 / 43

3. Empirical Results3.2. Mediating factors (cont.)

(22) Thus, keeping the other variables in the model constant, the predicted Logit of success for a multivariate, post-1973 test of the left-right division of the popular vote on the size of the state is:

L = −1.17 + 0.56 + 0.67 + 0.50 + 0.67 = 1.23.

(23) The predicted probability of success for such a test is therefore P = 1/[1 + e−(1.23)] = 0.77.

By comparison, the probability of success of a bivariate, pre-1973 test of partisan impact on welfare using a mixed measure of party strength is P = 1/[1+e−(−1.17)] = 0.24.

(24) This demonstrates that there are clearly identifiable conditions under which the probability of support for the left-right party influence hypothesis can be substantially increased.

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 39 / 43

4. Overview and discussion

(1) We started this review of 693 published cross-section estimates of left-right party impact on policy with a series of questions that have not been addressed in a meta-analytic format before:

i. how often do estimates support the left-right party impact hypothesis?

ii. What is the average magnitude of the effect size of left-right party impact?

iii. What (if any) are the determinants of variations in statistical support?

iv. Are partisan cycle explanations more sensitive to substantive factors (e.g., the policy domain of government intervention; the historical period of analysis) or to issues of measurement and methodology?

v. How do our results contribute to the advancement of knowledge in the field?

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 40 / 43

4. Overview and discussion (cont.)

(2) In response to the first two questions:

154 (22%) estimates support the left-right party impact hypothesis,

48 (7%) estimates contradict the hypothesis, and

491 (71%) estimates fail to support the hypothesis.

Applying the ‘vote-counting’ meta-analytic technique to these numbers leads us to conclude that the null hypothesis of no left-right party impact cannot be rejected.

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 41 / 43

4. Overview and discussion (cont.)

(3) We reach a similar diagnosis when we use a revised ‘combined tests’ technique to generate an ‘average effect size’ of left-right partisan impact on policy.

The 0.28 average correlational effect between party and policy indicates that approximately 8 percent of the variation in policy output is accounted for by the variation in the left-right party composition of Government.

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 42 / 43

Source

Louis M. Imbeau, Francois Pétry and Moktar Lamari (2001),

Left-Right Party Ideology and Government Policies: A-Meta-Analysis,

European Journal of Political Research, 40/1, 2001, pp. 1-29.

WS 2012/13 © Prof. Dr. Friedrich Schneider, University of Linz, Austria 43 / 43