Download - Factors influencing[1]

Factors influencing theeffectiveness of performance

measurement systemsAmy Tung, Kevin Baird and Herbert P. SchochDepartment of Accounting and Corporate Governance,

Macquarie University, Sydney, Australia

Abstract

Purpose – The purpose of this paper is to examine the association between the use ofmultidimensional performance measures and four organizational factors with the effectiveness ofperformance measurement systems (PMSs).

Design/methodology/approach – Data were collected by mail survey questionnaire from arandom sample of 455 senior financial officers in Australian manufacturing organizations.

Findings – The results reveal that the use of multidimensional performance measures is associatedwith two dimensions of the effectiveness of PMSs (performance and staff related outcomes). Theresults also reveal that organizational factors were associated with the effectiveness of PMSs.Specifically, top management support was found to be associated with the effectiveness of PMSs inrespect to the performance related outcomes, and training was associated with the staff relatedoutcomes.

Practical implications – The findings provide managers with an insight into the desirable PMScharacteristics and the specific organizational factors that they can focus on in order to enhance theeffectiveness of their performance measurement system.

Originality/value – This study contributes to the limited empirical research examining theeffectiveness of PMSs regarding the extent to which organizational processes are achieved. In addition,the study provides an empirical analysis of the association between the five perspective (financial,customer, internal business process, learning and growth, and sustainability) BSC model and fourorganizational factors with the effectiveness of PMSs.

Keywords Australia, Manufacturing industries, Performance measures,Performance measurement system, Multidimensional performance measures, Top management support,Training, Employee participation, Link of performance to rewards

Paper type Research paper

1. IntroductionTo survive in today’s rapidly changing environment, organizations must identify theirexisting positions, clarify their goals, and operate more effectively and efficiently.Performance measurement systems (PMSs) assist organizations in achieving suchobjectives. Neely et al. (1995, p. 81) defines a PMS as “a set of metrics used toquantify both the efficiency and effectiveness of actions”. An effective PMS enablesan organization to assess whether goals are being achieved, and facilitates theimprovement of the organization as a whole (Lebas, 1995) by identifying their position,clarifying goals, highlighting areas requiring improvement, and facilitating reliableforecasts (Neely et al., 1996). Hence, an effective PMS enables an organization tomeasure and control its performance in line with the defined strategy.

While the recent PMS literature has focused on the shift from traditional PMSs, whichfocus on financial measures, to multidimensional PMSs such as the performance

The current issue and full text archive of this journal is available at

www.emeraldinsight.com/0144-3577.htm

The effectivenessof PMSs

1287

Received June 2010Revised November 2010

Accepted April 2011

International Journal of Operations& Production Management

Vol. 31 No. 12, 2011pp. 1287-1310

q Emerald Group Publishing Limited0144-3577

DOI 10.1108/01443571111187457

pyramid (Lynch and Cross, 1991), the balanced scorecard (BSC) (Kaplan and Norton,1992), and the performance prism system (Neely and Adams, 2000), there is limitedempirical evidence examining the effectiveness of such PMSs. Furthermore, the majorityof these studies assess PMS effectiveness in relation to overall organizationalperformance (Crabtree and DeBusk, 2008; Braam and Nijssen, 2004; Davis and Albright,2004; Ittner et al., 2003; Hoque and James, 2000), thereby assuming a direct associationbetween the PMS and performance. This approach is inconsistent with Hamilton andChervany’s (1981) claim that the impact of the PMS on performance is indirectlyinfluenced by the effect on improvements in organizational processes. In other words,organizational objectives such as sales revenue, profit contribution and customersatisfaction will not be realized unless specific organizational objectives (e.g. motivatingperformance, developing individual’s skills and knowledge, providing useful feedbackto employees, and providing an accurate assessment of business unit performance) areachieved. Accordingly, the first objective of this study is to contribute to the limitedempirical research (Malina and Selto, 2001; Whorter, 2003) examining the effectivenessof PMSs based on the extent to which organizational processes are achieved.

The measurement of performance is an on-going task, hence, in order to achievesystem effectiveness, organizations need to devote time and effort to managing thesystem (Neely et al., 2000). Hence, in an attempt to provide practitioners with an insightinto how to achieve and maintain effectiveness, the second objective of the study is tocontribute to the contingency literature by examining the factors associated withthe effectiveness of PMSs. The first factor examined, the use of multidimensionalperformance measures, has been advocated by both academics and practitioners inorder to complement the limitations of traditional financial PMSs and to increase theeffectiveness of PMSs (Van der Stede et al., 2006; Kaplan and Norton, 2001, 1996, 1992).While many multidimensional frameworks have been advocated, and the benefits ofusing multidimensional performance measures have received wide publicity in theliterature (Van der Stede et al., 2006; Bryant et al., 2004), there is considerable variation inthe adoption rates reported for the most common multidimensional approach, the BSC(Rigby and Bilodeau, 2009 (53 percent); Chung et al., 2006 (31 percent); Ittner et al., 2003(20 percent); Speckbacher et al., 2003 (26 percent)). The variation in the adoption ofmultidimensional performance measures raises concerns regarding the contributionof such measures towards the effectiveness of PMSs. Accordingly, this study aimsto contribute to the literature by examining the association between the use ofmultidimensional performance measures and the effectiveness of PMSs.

The study also aims to provide an empirical analysis of the association betweenspecific organizational factors (top management support, training, employeeparticipation and the link of performance to rewards) with the effectiveness of PMSs.

While these organizational factors do not represent a comprehensive list of allrelevant factors, they were chosen for two reasons. First, they have been widely citedas factors contributing to the success of various management accounting practices suchas activity-based costing (ABC) (Baird et al., 2007; Shields, 1995), enterprise resourceplanning (Motwani et al., 2002; Rao, 2000), and management information system (MIS)(Raghunathan et al., 1999; Doll, 1985; Schultz and Ginzberg, 1984). Second, while theyhave been identified in previous studies as the main contingency factors associated withthe effectiveness of PMSs (Burney et al., 2009; Hoque and Adams, 2008; Cheng et al.,2007; Kleingeld et al., 2004; Chan, 2004), this was in isolation, and no study has analysed

IJOPM31,12

1288

all four factors together. Hence, this study is motivated to fill this gap in the literature byexamining the link between all four organizational factors and the effectiveness of PMSswithin Australian manufacturing organizations.

In addition, given the majority of previous studies examining the influence oforganizational factors on PMS effectiveness have used the case study approach(Kleingeld et al., 2004; Bourne et al., 2002; Emerson, 2002; Kennerley and Neely, 2002;Kaplan, 2001), there is a gap in the literature empirically examining this association.Hence, the current study is motivated to fill this gap by using the survey method in anattempt to enhance the generalizability of the findings.

The remainder of this paper is structured as follows. Section 2 provides the literaturereview and develops the relevant hypotheses. Sections 3 and 4 then discuss the methodand results. Finally, Section 5 provides the conclusion, limitations, and future directionsfor research.

2. Literature review2.1 Performance measurement systemsPMSs have become a field of interest over the last two decades with many studiesdiscussing various aspects of performance measurement such as: the purpose andusage (Marchand and Raymond, 2008; Horngren et al., 2005; Simons, 2000), design(Bhasin, 2008; Kennerley and Neely, 2002; Neely and Adams, 2000; Kaplan and Norton,1996, 1992; Lynch and Cross, 1991), and implementation (Ratnasingam, 2009; Othman,2008; Speckbacher et al., 2003; Kaplan, 2001).

An effective PMS, which is defined as the achievement of the objectives set fora task (Clinquini and Mitchell, 2005), is important for a number of reasons. First,it can encourage goal congruence. For example, an appropriate PMS can be used tocommunicate the strategy and goals of an organization and align employees’ goals withorganizational goals. Second, an effective PMS can provide accurate information toenable managers to track their own performance and evaluate employees’ performancein an effective and efficient manner. Finally, an effective PMS can provide organizationswith an indication of their current market position and assist them in developing futurestrategies and operations (Langfield-Smith et al., 2009). This study operationalises aneffective PMS as the extent to which 16 desired PMS outcomes are achieved.

Traditionally, PMSs have focused mainly on financial measures such as profit, cashflow and return on investment to evaluate the performance of employees (Chan, 2004).This focus has a number of shortcomings. First, these outcome-oriented measures donot allow managers to assess how well employees perform across the full range ofstrategically important areas, such as quality and service delivery. Second, traditionalfinancial measures describe consequences rather than causes, hence they are notactionable. Such measures provide limited guidance for future actions since they do nottell managers what needs to be fixed (Langfield-Smith et al., 2009). Third, the focus onaggregate financial outcomes may encourage managers to engage in “gaming” behaviorto maximize short-term results at the expense of long-term effectiveness (Chow andVan der Stede, 2006). Finally, traditional financial measures can conflict with strategy andthey are not externally focused (Chow and Van der Stede, 2006; Kaplan and Norton, 1996).

The limitations of traditional PMSs, together with intense competitive pressuresand changing external demands, have led to the increased advocacy of non-financialmeasures (Neely, 1999). Such contemporary PMSs have been espoused by both


1289

academics and practitioners in order to address the limitations of traditional financialperformance measures and to assist organizations to build competitive advantageunder changing economic conditions (Kaplan and Norton, 2006, 2004, 2001, 1996, 1992).The common characteristics of contemporary systems include the linking of strategies,objectives and measures, and the incorporation of both financial and non-financialmeasures that cover a range of perspectives (Langfield-Smith et al., 2009). Since theBSC is the most recognized and utilized contemporary PMS (Rigby and Bilodeau, 2009;Chang et al., 2008; Jusoh et al., 2008; Bedford et al., 2006; Pike and Roos, 2004;Atkinson et al., 1997), it is used in this study to exemplify the use of multidimensionalperformance measures.

2.2 The BSCThe first-generation BSC was mainly a PMS which proposed a specific structure tomeasure tangibles and intangibles (Speckbacher et al., 2003; Kaplan and Norton, 1992).The framework complemented the financial perspective measures with non-financialoperational measures emphasizing three other perspectives: customer satisfaction,internal processes and learning and growth. It provided a more balanced view oforganizational performance by capturing both leading (e.g. customer satisfaction,on-time delivery, employee training, etc.) and lagging (e.g. sales revenue, ROI, cashflows, etc.) performance measures (Kaplan and Norton, 1996, 1992).

In 1996, Kaplan and Norton advocated the causal links between the perspectivesincluded within the BSC. The refined model communicated the organization’s desiredoutcomes and hypothesized the means by which the desired outcomes could be achieved.For instance, if organizations trained their employees well, then the quality of servicewould be improved as well as customer satisfaction; if customer satisfaction improved,then customers would purchase more, thereby improving the overall profitability of theorganization. Hence, the second-generation BSC was proposed as a multidimensionalPMS which describes strategy through cause and effect relationships (Speckbacher et al.,2003; Kaplan and Norton, 1996). It enabled organizational units and employees tounderstand the strategy and identify how they can contribute to its achievement bybecoming aligned with the strategy. Consequently, today’s BSC has become a strategicmanagement system that implements strategy through communication, action plansand incentives (Speckbacher et al., 2003; Kaplan and Norton, 2001).

As a further development, the BSC included additional perspectives (Kaplan andWisner, 2009; Kaplan and Norton, 2006, 2004, 2001). With sustainability becoming amajor concern for various stakeholders (e.g. customers, investors, and the government)and affecting the organizational “bottom line”, a sustainability BSC was subsequentlyadvocated (Langfield-Smith et al., 2009; Epstein, 2008; Figge et al., 2002). Epstein (2008)suggested that the inclusion of the sustainability perspective is appropriate wheresustainability is considered a part of the business core strategy and important tocreating competitive advantage. To provide a more comprehensive account of the use ofmultidimensional performance measures, this study adopts the five perspective(financial, customer, internal business process, learning and growth, and sustainability)BSC model.

2.2.1 Adoption and use of the BSC. Silk (1998) estimated that 60 percent of theFortune 1000 companies in the USA have had experience with a BSC. In the UK,57 percent of businesses were reported to use a BSC and 53 percent of non-users

IJOPM31,12

1290

were discussing possible implementation. In contrast, Speckbacher et al. (2003) reportedthat more than 60 percent of the companies in their study had not considered the BSC.Similarly, Ittner et al. (2003) indicated that only 20 percent of the firms in their study useda BSC, while 50 percent of the firms had not even considered implementing it.

Use of the BSC however does not guarantee satisfaction with De Geuser et al. (2009)referring to the literature highlighting the gap between the use of the BSC and evidenceof its effectiveness (Davis and Albright, 2004; Norreklit, 2003; Speckbacher et al., 2003;Otley, 1999). Thus, while the Management Tools and Trends Survey (Rigby andBilodeau, 2009) showed that in 2008, 53 percent of organizations globally used the BSCand by the end of 2009, the usage rate was expected to reach 69 percent, it was found that51 percent of user organizations were not satisfied with their BSC. Similarly, Ittner et al.(2003) revealed that organizations were only moderately satisfied with the measurementsystem with 37.2 percent of respondents rating it as not meeting expectations.Bedford et al. (2006) also concluded that while respondents agreed that the BSC hadhelped in achieving some objectives, the extent to which the proclaimed benefits of theBSC were achieved was still fairly low. Given the mixed findings with respect to thesuccess of the BSC, this study investigates the association between the use ofmultidimensional performance measures and the effectiveness of PMSs.

2.3 The association between the use of multidimensional performance measures and theeffectiveness of PMSsMultidimensional PMSs assist organizations by enhancing the likelihood that allrelevant performance dimensions are considered (Ittner et al., 2003). Furthermore, suchsystems allow managers to focus on the “means to the end”, while also enabling themto demonstrate strong performance in a variety of areas (Baird, 2010). Hoque andAdams (2008) suggest that multidimensional PMSs are capable of providing signalsand motivating improvement in crucial activities. Similarly, Van der Stede et al. (2006)found that regardless of strategy, organizations with more extensive PMSs, especiallythose that included objective and subjective non-financial measures, have betteroverall performance. Van der Stede et al. (2006) also demonstrated that non-financialperformance measures are better than financial measures in helping organizationsimplement and manage new initiatives.

A growing stream of literature provides evidence that the use of multidimensionalperformance measures contributes to the effectiveness of PMSs (Crabtree and DeBusk,2008; Braam and Nijssen, 2004; Davis and Albright, 2004; Ittner et al., 2003; Whorter,2003; Malina and Selto, 2001; Hoque and James, 2000). Most of these studies examinedthe effectiveness of PMSs from the perspective of their contribution to the company’sfinancial performance. For example, Davis and Albright (2004) applied aquasi-experimental study in a US banking organization to investigate the relationshipbetween BSC implementation and the financial performance of bank branches.The study supports the theory that the BSC can be used to improve financialperformance, with bank branches that implemented the BSC outperforming otherbranches on key financial measures. Similarly, Braam and Nijssen (2004) suggest thatBSC usage, which is aligned to company strategy, positively influences overall companyperformance.

Ittner et al. (2003) found that while BSC usage was associated with highermeasurement system satisfaction, there was no evidence that BSC usage was related


1291

to stock returns. However, Crabtree and DeBusk (2008) extended this study toinvestigate the contribution of the BSC to shareholder returns in different public sectorcompanies, and found that BSC usage was associated with higher stock returns.

Malina and Selto (2001) and Whorter (2003) assessed the effectiveness of PMSs basedon organizational processes (e.g. communicating strategic objectives, creating strategicalignment, motivating employees and serving as a management control device)as opposed to financial performance. Malina and Selto (2001) found that the BSC was aneffective device for evaluating corporate strategy. Their results also show evidence ofcasual relations between motivation, strategic alignment and effective managementcontrol with the BSC. Similarly, Whorter (2003) showed that BSC users consistentlyreported higher agreement about having the information needed for making the bestwork-related decisions. Whorter (2003) also concluded that the BSC not only providesuseful performance feedback to employees but is also an aid in the accurate assessmentof employee performance:

H1. The extent of use of multidimensional performance measures is associatedwith the effectiveness of the PMS.

2.4 The association between organizational factors and the effectiveness of PMSsPrior studies have identified top management support (Hoque and Adams, 2008;Johanson et al., 2006; Bourne, 2005; Chan, 2004; Bourne et al., 2002; Kennerley and Neely,2002; Kaplan, 2001), training (Chan, 2004; Emerson, 2002), employee participation(Hoque and Adams, 2008; Kleingeld et al., 2004), and the link of performance to rewards(Burney et al., 2009; Chan, 2004) as key organizational factors associated with theeffectiveness of PMSs.

2.4.1 Top management support. Top management support has been highlighted asan important contingency factor in supporting various management accounting practicessuch as ABC (Baird et al., 2007; Shields, 1995) and MISs (Doll, 1985). The impact of topmanagement support on PMS effectiveness has been referred to in a number of studies(Bourne, 2005; Chan, 2004; Bourne et al., 2002; Emerson, 2002; Kennerley and Neely, 2002).For example, Bourne et al. (2002) investigated the success of the redesign of PMSs. Theyfound that top management support was influential in the successful implementationand on-going usage of the new PMS. This study also indicated that the continuousinvolvement by top management was invaluable in resolving problems when crises andconflicts arose. Chan (2004) and Emerson (2002) also reported that top managementcommitment and leadership buy in are key factors in enhancing PMS effectiveness.Similarly, Kennerley and Neely (2002) found that top-level management support wascritical for PMS design and implementation, while the availability of management time toreflect on measures was a major contributor to the effectiveness of PMSs:

H2. The extent of top management support is associated with the effectiveness ofthe PMS.

2.4.2 Training. Training is defined as “a planned effort by an organization to facilitatethe learning of job-related behavior” (Wexley, 1984, p. 13). The importance of trainingin relation to the development and implementation of a successful PMS is highlightedin a number of studies. Cavaluzzo and Ittner (2004, p. 249), for example, found thatperformance measurement development and outcomes are positively associated withthe extent of related training provided to the manager. The provision of training

IJOPM31,12

1292

resources indicates that an organization is willing to provide sufficient resources tosupport the development and implementation of PMSs.

Chan (2004) cites training as a crucial factor for PMSs to be effective. All performancemeasures need to have a clearly communicated purpose and be perceived as bothrelevant and reliable so that managers can access useful information for decisionmaking. Without training, managers may perceive the PMS measures as less useful andignore them when making decisions. Similarly, Emerson (2002) concluded that trainingis the key to maintaining the usefulness and the effectiveness of PMSs. Training not onlyallows users to understand performance measurement concepts and principles, but alsoprovides both employees and managers with an opportunity to operate the system.Hence, the better that users understand the purposes of the system and how tooperationalise it, the more likely they will commit to it, thereby enhancing the likelihoodthat the desired results will be achieved:

H3. The extent of PMS-related training provided is associated with theeffectiveness of the PMS.

2.4.3 Employee participation. Many studies have referred to the benefits of employeeempowerment (Morrell and Wilkinson, 2002; Koberg et al., 1999; Chiles and Zorn,1995) and employee involvement and participation (Cox et al., 2007, 2006; Pun et al.,2001; Wimalasiri and Kouzmin, 2000). These studies tend to operationalise theseconcepts in terms of employees’ involvement in decision making. Similarly, employeeparticipation refers to the “involvement of managers and their subordinates ininformation processing, decision making, or problem solving endeavors” (Wagner,1994, p. 312). This study operationalises employee participation in terms of the extentto which lower level employees participate in designing the PMS.

The association between employee participation and the effectiveness of PMSs hassupport from prior studies (Chan, 2004; Kleingeld et al., 2004; Kaplan and Norton,2001). These studies report that a higher level of employee participation contributed tothe effectiveness of PMSs. For instance, Kleingeld et al. (2004) found that on averagethe improvement in performance was significantly greater for those employees in ahigh participation situation as opposed to those in a low participation situation. Thisperformance improvement was attributed to both cognitive mechanisms (includingincreased communication, better utilization of knowledge, increased understanding ofthe job) and motivational mechanisms (less resistance to change, commitment to thesystem, acceptance of feedback and goals).

Similarly, Kaplan and Norton (2001) maintained that in order to achieve an effectiveBSC, employees at lower levels in the organizational hierarchy should be involved inthe establishment of performance measures. This bottom-up participation approachallows employees to take the initiative in defining their responsibilities as well as theassociated performance indicators. Therefore, employees will commit to the systemand desired outcomes can be achieved to a greater extent:

H4. The extent of employee participation in designing the PMS is associated withthe effectiveness of the PMS.

2.4.4 The link of performance to rewards. The link of performance to rewards is avital contingency factor in motivating employees (Rynes et al., 2005; McShane andTravaglione, 2003; Bonner and Sprinkle, 2002; PA Consulting Group, 1998).


1293

A survey of 500 companies reported that companies that link performance to payshowed twice the shareholder returns as those who did not (PA Consulting Group, 1998).McShane and Travaglione (2003) suggested that companies need to align rewards withperformance that is within the employee’s control. Hence, the more employees see a“line of sight” between their daily actions and the reward, the more motivated they willbe to improve performance.

Linking performance to rewards has also been identified as a crucial factorinfluencing the effectiveness of PMSs (Burney et al., 2009; Johanson et al., 2006; Chan,2004). For instance, in Chan’s (2004) study of municipal governments in the USA andCanada, it was found that the linkage of the PMS to compensation was uncommon,and “the lack of linkage of the BSC to rewards” was considered to be a barrier to thesystems’ effectiveness.

While there is a lack of empirical evidence examining the link of performance torewards on the effectiveness of PMSs, given the importance of the link of performance torewards and the increasing number of large businesses rewarding both employees andmanagers based on BSC performance (Epstein and Manzoni, 1998),H5 is stated as follows:

H5. The extent of the link of performance to rewards is associated with theeffectiveness of the PMS.

3. MethodA survey questionnaire was mailed to the senior financial officer of a random sample of445[1] Australian manufacturing business units identified from the Kompass Australia(2009) directory[2]. The manufacturing industry was selected as a number of priorstudies on PMSs suggest that manufacturing organizations are more likely to have amature and comprehensive PMS in place (Malina and Selto, 2001; Simons, 2000; Kaplanand Norton, 1996, 1992). Business units were chosen as the unit of analysis because PMScharacteristics may differ across business units within an organization. Senior financialofficers were chosen as they were expected to have a sound understanding of theirbusiness unit’s PMS. The Dillman (2007) tailored design method was employed toadminister the survey[3]. In total, 141 responses were received for a response rate of30.9 percent. In total, 23 of the questionnaires were incomplete, hence 118 questionnaireswere used for the data analysis. As was the case in Robert (1999), non-response bias wasassessed by comparing the independent and dependent variable values across early andlate respondents. No significant differences were detected.

3.1 Variable measurement3.1.1 The effectiveness of the PMS. The effectiveness of PMSs is measured byassessing the extent to which 16 desired outcomes of PMSs have been achieved. The16 measures (the Appendix) were developed based on a review of the literature relatingto the effectiveness of PMS (Lawler, 2003) with minor modifications made to fit thecontext of the study. Respondents were required to indicate the extent to which theirPMS had achieved each of the 16 perceived outcomes using a five-point Likert scalewith anchors of 1 “not at all” and 5 “to a great extent”.

Factor analysis (principal components with varimax rotation) using a cutoff point of0.60 revealed that the 16 outcomes loaded onto two dimensions, with the factor structureconsistent with Baird (2010). The first dimension included nine items which all referto the achievement of organizational goals and objectives, hence, this dimension

IJOPM31,12

1294

was labeled “performance-related outcomes”. The second dimension included sevenitems which are more concerned with employees, hence this dimension was labeled“staff-related outcomes”. These two dimensions were subsequently scored as theaverage score of the items loading on to each dimension with higher (lower) scoresrepresenting a more (less) effective PMS.

3.1.2 The usage of multidimensional performance measures. The extent to whichrespondents were using multidimensional performance measures was measured usingtwo approaches. The first approach required respondents to simply indicate if they wereusing a BSC (“yes” or “no”). Since this approach is reliant on respondents understandingof the nature of a BSC, a more comprehensive approach which focuses on the performancemeasures employed within organizations, was also adopted. This approach requiredrespondents to indicate the extent to which they were using 26 different performancemeasures (the Appendix) to assess their business units’ performance, on a five-pointLikert scale with anchors of 1 “not at all” to 5 “to a great extent”. These measures werederived primarily from the BSC literature and were mainly designed for manufacturingorganizations (Epstein, 2008; Jusoh et al., 2008; Van der Stede et al., 2006; Bryant et al.,2004; Ittner et al., 2003; Kaplan and Norton, 2001, 1996).

Factor analysis (principal components with varimax rotation) using a cutoff point of0.6 revealed that the 26 items loaded onto six specific dimensions covering the followingperspectives: financial, customer, internal business, learning, growth and sustainability.These findings are in line with Figge et al. (2002), except that the learning and growthperspectives were separated. These two perspectives were subsequently combined inaccordance with the five perspectives BSC model.

Each of the five perspectives were scored as the sum of the items loading onto eachperspective with higher (lower) scores indicating the PMS focused on each perspectiveto a greater (lesser) extent. Since a different number of items loaded onto each ofthe perspectives, average scores were calculated with the use of multidimensionalperformance measure scored as the sum of the averages across the five perspectiveswith higher (lower) scores indicating that multidimensional performance measureswere used to a greater (less) extent.

3.1.3 Organizational factors. Each of the four organizational factors was measuredusing a summated five-point Likert scale with anchors of 1 “strongly disagree” and5 “strongly agree”.

Top management support was measured using a three-item summatedscale (the Appendix) with respondents required to indicate the extent to which topmanagement provided adequate resources (Krumwiede, 1998), communicated effectively(Grover, 1993) and exercised its authority in support of the PMS. Top managementsupport was measured as the average score for the three items, with higher (lower) scoresindicating a higher (lower) level of top management support.

The level of related training was measured using three items (the Appendix) drawnfrom Baird et al. (2007), with minor adjustments made to fit the context of the currentstudy. Specifically, respondents were required to indicate if adequate training had beenprovided to develop, to implement and to ensure employees understood the PMS.Training was measured as the average score for the three items, with higher (lower)scores indicating a higher (lower) level of related training provided by the organization.

In the absence of specific measures in the literature on employee participation in aPMS context, two self-developed items (the Appendix) were adopted following a review


1295

of the employee participation/involvement literature (Sinclair et al., 2005; Harel andTzafrir, 1999; Huselid, 1995; Wagner, 1994). Specifically, respondents were required toindicate the extent to which lower level employees participated in designing the PMSand were involved in selecting performance measures. The perceived level of employeeparticipation was subsequently scored as the average score for the two items withhigher (lower) scores indicating a higher (lower) level of employee participation.

The link of performance to rewards was assessed using two items (the Appendix)based on the literature on performance and rewards (Rynes et al., 2005; Lawler, 2003;Huselid, 1995). Respondents were required to indicate the extent to which performanceis linked to financial rewards such as pay or bonus, and non-financial rewards such asrecognition or service awards in their organization. The analysis revealed that the twoquestions were measuring different factors: the extent to which performance is linkedto financial rewards and to non-financial rewards. These measures are analyzed asseparate independent variables, with higher (lower) scores indicating a stronger(weaker) link of performance to rewards.

4. ResultsTable I shows summary statistics for the dependent and independent variables. For themulti-item scales, the actual range was comparable with the theoretical range, and theCronbach’s a coefficients met or exceeded the 0.70 threshold generally consideredacceptable in regard to scale reliability (Nunnally, 1978, p. 245).

The mean scores of the effectiveness of PMSs for both the performance-relatedoutcomes (3.50) and the staff-related outcomes (3.26) are slightly higher than themid-point of the range, indicating that on average the respondents assessed theirPMS to be moderately effective. The performance-related outcomes were achievedto a greater extent, with the mean scores of all nine items equal to or greater than theseven staff-related outcomes. The performance-related outcomes that were achieved

Variables n a Mean SDMinimum

(theoretical)Maximum

(theoretical) Cronbach’s a

Independent variablesUse of multidimensionalperformance measures 118 2.94 0.70 1.17 (1) 4.67 (5)Top management support 117 3.51 1.02 1 (1) 5 (5) 0.915Training 117 3.11 1.07 1 (1) 5 (5) 0.963Employee participation 117 2.41 1.02 1 (1) 5 (5) 0.761Link of performance to financialrewards 117 3.50 1.16 1.00 (1) 5.00 (5)Link of performance to non-financial rewards 117 2.93 1.13 1.00 (1) 5.00 (5)Dependent variablesEffectiveness of PMS(performance-related outcomes) 117 3.50 0.81 1 (1) 5 (5) 0.932Effectiveness of PMS (staff-relatedoutcomes) 117 3.26 0.93 1 (1) 5 (5) 0.924

Note: aThe number of responses (n) varies due to the fact that not all survey items were completed byrespondents

Table I.Descriptive statistics

IJOPM31,12

1296

to the greatest extent included: assisting in achieving the goals (mean score of 3.68);providing useful performance feedback to employees (mean score of 3.64); developinga performance-oriented culture (mean score of 3.59); and providing an accurateassessment of business unit performance (mean score of 3.59). The staff-relatedoutcomes that were achieved to the greatest extent included: developing individual’sskill and knowledge (mean score of 3.38), identifying talented employees (mean scoreof 3.36), and rewarding talented employees (mean score of 3.31).

In regard to the four organizational factors, while the mean score of most of thefactors lie on the higher end of the scale, the mean value of the link of performance tonon-financial rewards (2.93) was slightly below the mid-point of the range indicating arelative weak link between performance and non-financial rewards.

As discussed in the method section, two approaches were used to assessthe use of multidimensional performance measures. Table II reveals that 39 respondents(33.1 percent) indicated that they were using a BSC in their business unit.The more comprehensive approach to measuring the use of multidimensionalperformance measures focused on the extent to which business units were employing26 performance measures covering the five perspectives of the BSC. Table I reveals thatthe mean score for the use of multidimensional performance measures (2.94) was slightlylower than the mid-point of the range, indicating a moderate use of multidimensionalperformance measures in Australian manufacturing organizations.

Table III provides a more detailed analysis of the extent to which measures relatingto each of the five perspectives were employed. The greatest emphasis was placed onthe financial perspective (3.59) followed by the customer (3.43), learning and growth(3.11), and internal business process (3.06) perspectives. The mean score of thesustainability perspective (2.19) was below the mid-point of the range indicating arelatively low level of usage of this perspective.

4.1 Analysis of the association between the use of multidimensional performancemeasures and organizational factors with the effectiveness of PMSsTable IV presents the results of the one-way analysis of variance (ANOVA) used toexamine the difference in the level of PMS effectiveness based on whether respondentswere using a BSC. Respondents using a BSC reported a significantly higher level ofPMS effectiveness with respect to both performance- and staff-related outcomes.

BSC usage Frequency Adjusted percentage

Yes 39 33.1No 79 66.9

Table II.BSC usage

BSC perspectives n Minimum Maximum Mean Rank

Financial 118 1.00 (1) 5.00 (5) 3.59 1Customer 118 1.00 (1) 5.00 (5) 3.43 2Internal business process 118 1.00 (1) 5.00 (5) 3.06 4Learning and growth 118 1.17 (1) 5.00 (5) 3.11 3Sustainability 118 1.00 (1) 5.00 (5) 2.19 5

Table III.Use of multidimensional

performance measures


1297

These results provide preliminary evidence that the use of multidimensionalperformance measures is associated with the effectiveness of PMSs, thereby providingsupport for H1.

The association between the use of multidimensional performance measures andPMS effectiveness was also analyzed using a more comprehensive approach based onthe extent of use of multidimensional performance measures. Stepwise regression wasused to examine the association between both the use of multidimensional performancemeasures and organizational factors with PMS effectiveness, with the results presentedin Table V. For the effect on performance-related outcomes, the model was statisticallysignificant (F ¼ 63.812, p ¼ 0.000) with an R 2 of 0.530 indicating that 53 percent ofthe variance in the achievement of performance-related outcomes can be explained by theexplanatory factors. The model reveals that the use of multidimensional performancemeasures ( p ¼ 0.000) was significantly associated with the effectiveness of PMSs.In addition, top management support ( p ¼ 0.000) was significantly associated with theperformance-related outcomes.

Table V also provides the findings for staff-related outcomes, with the model foundto be statistically significant (F ¼ 38.535, p ¼ 0.000) with an R 2 of 0.405 indicatingthat 40.5 percent of the variance in the achievement of staff-related outcomes can beexplained by the explanatory factors. The model reveals that the use ofmultidimensional performance measures was found to be significantly associatedwith the achievement of staff-related outcomes ( p ¼ 0.000). The level of training( p ¼ 0.000) was also significantly associated with PMS effectiveness.

The findings provide further support for H1 and partially support H2 and H3.The importance of the use of multidimensional performance measures in explainingthe level of PMS effectiveness prompted further exploratory analysis to investigate theassociation between each of the five perspectives of the BSC with the effectiveness ofPMSs. These findings are presented in Section 4.2.

Performance-related outcomes Staff-related outcomesBSC usage n Mean F-statistic Significance Mean F-statistic Significance

BSC user 39 3.88 14.297 0.000 3.71 15.869 0.000Non-BSC user 78 3.31 3.03

Table IV.Results of the one-wayANOVA comparing thelevel of PMS effectivenessbased on BSC usage

Performance-related outcomes Staff-related outcomesVariables Coefficient t-statistics Significance Coefficient t-statistics Significance

Multidimensional PMS 0.343 4.512 0.000 0.374 4.465 0.000Top managementsupport 0.487 6.411 0.000Training 0.362 4.325 0.000F-value 63.812 38.535p-value 0.000 0.000R 2 0.530 0.405Adjusted R 2 0.522 0.395n 115 115

Table V.Results of stepwiseregression analysisof the associationbetween the useof the multidimensionalperformance measuresand organizationalfactors with theeffectiveness of PMSs

IJOPM31,12

1298

4.2 Analysis of the association between the five perspectives of the BSC with theeffectiveness of PMSsTable VI reveals the stepwise regression analysis findings. The performance-relatedoutcomes model was statistically significant (F ¼ 63.847, p ¼ 0.000) with an R 2 of0.528 indicating that 52.8 percent of the variance in the achievement of theperformance-related outcomes can be explained by the two perspectives of the BSCfound to be significantly associated with the performance-related outcomes: the internalbusiness process ( p ¼ 0.000) and learning and growth ( p ¼ 0.000) perspectives.

The staff-related outcomes model was also statistically significant (F ¼ 56.768,p ¼ 0.000) with an R 2 value of 0.499 indicating that 49.9 percent of the variance in theachievement of the staff-related outcomes can be explained by the two perspectives ofthe BSC found to be significantly associated with the staff-related outcomes: thelearning and growth ( p ¼ 0.000) and sustainability ( p ¼ 0.000) perspectives.

5. Conclusion5.1 DiscussionThe first objective of the study was to examine the effectiveness of PMSs in respect totheir impact on organizational processes. The study evaluated the effectiveness of PMSsbased on the extent to which 16 desired outcomes were achieved. By focusing on theoutcomes achieved, the study contributes to the empirical body of knowledge on PMSssince the majority of previous studies have only assessed PMS effectiveness based onoverall organizational performance. This approach provides managers with a moredetailed insight into the ability of the PMS to assist their organization in achievingspecified desired outcomes. Factor analysis revealed that these items reflected twodimensions of PMS effectiveness: performance- and staff-related outcomes. The resultsrevealed that the mean score for the effectiveness of PMSs for both dimensions wasslightly above the mid-point of the range, indicating that the PMSs of Australianmanufacturing organizations were only moderately effective. This finding highlightsthe significance of the study’s investigation of the contingency factors associated withthe effectiveness of PMSs.

The results also showed that organizations were more successful in achievingthe performance-related outcomes than the staff-related outcomes. This suggeststhat PMSs have mainly been used as a managerial tool to assist the organization inmotivating performance, implementing the organizational strategy and achieving goals.

Performance-related outcomes Staff-related outcomesVariables Coefficient t-statistics Significance Coefficient t-statistics Significance

Internal businessprocess 0.277 3.830 0.000Learning and growth 0.558 7.730 0.000 0.539 7.445 0.000Sustainability 0.289 4.001 0.000F-value 63.847 56.768p-value 0.000 0.000R 2 0.528 0.499Adjusted R 2 0.520 0.490n 116 116

Table VI.Results of stepwiseregression analysis

of the associationbetween each of the fiveperspectives of the BSC

with the effectivenessof PMSs


1299

Less emphasis is being placed on achieving staff-related outcomes such as addressingthe concerns of staff, ensuring staff time is used efficiently, and managing poorlyperforming staff.

The latter finding is of concern given that survival in today’s rapidly changing worldis dependent on the achievement of both staff- and performance-related outcomes.Harel and Tzafrir (1999, p. 185) highlighted the importance of focusing on employees,suggesting that an organization’s staff are its strategic assets which “form a system ofresources and rare abilities that cannot easily be copied, and provide the company withits competitive edge”. Hence, organizations which view staff as potential partners andimportant assets enhance the likelihood of achieving better organizational performance.

There is also evidence that the achievement of staff-related outcomes can assist in theachievement of performance-related outcomes. If organizations adequately addressthe concerns of their employees, they are more likely to be emotionally attachedto a particular organization, and hence more willing to assist in the achievement oforganizational goals (Myer and Allen, 1991). Accordingly, we suggest that managersplace greater emphasis on the achievement of staff-related outcomes. This should beembodied in the design of the PMS so as to incorporate both contributions fromemployees as well as reflecting their personal needs.

The second objective of the study was to examine the association between the use ofmultidimensional performance measures and four organizational factors with theeffectiveness of the PMS. The initial analysis focused on ascertaining the extent towhich organizations were using multidimensional performance measures. Resultsrevealed that only 33.1 percent of organizations were using the BSC, which isconsistent with previous findings (Crabtree and DeBusk, 2008 (35 percent); Chung et al.,2006 (31 percent); Speckbacher et al., 2003 (26 percent); Whorter, 2003 (35 percent)).

A more comprehensive analysis of the use of multidimensional performancemeasures revealed that Australian manufacturing organizations placed the greatestemphasis on measures relating to the financial perspective of the BSC, followed bythe customer, learning and growth internal business process, and sustainabilityperspectives. This finding is consistent with the majority of the BSC literature whichsuggests that financial measures are still used to the greatest extent (Crabtree andDeBusk, 2008; Hoque and Adams, 2008; Davis and Albright, 2004; Braam and Nijssen,2004; Ittner et al., 2003; Hoque and James, 2000; Lipe and Salterio, 2000; Ittner andLarcker, 1998). The findings indicate that while organizations may be enticed to use aBSC, and even claim to use the BSC, the reality is that the greatest emphasis is still placedon the traditional financial-based perspective. Therefore, if organizations are to reap thebenefits of using multidimensional PMSs such as the BSC, it is crucial that they do notjust pay lip service to the inclusion of measures covering the other perspectives. Ratherthey need to acknowledge the importance of the other perspectives of the BSC and placeincreasing emphasis on using measures relating to each of the perspectives.

Analysis of the association between the use of multidimensional performancemeasures and organizational factors with the effectiveness of PMSs revealed that theuse of multidimensional performance measures, as operationalized by the BSC, and twoorganizational factors (top management support, and training) exhibited a significantassociation with the effectiveness of PMSs.

The use of multidimensional performance measures was positively associated withboth the performance- and staff-related outcomes. This finding is in line with previous

IJOPM31,12

1300

studies (Chow and Van der Stede, 2006; Van der Stede et al., 2006; Bryant et al., 2004)which have advocated that organizations should incorporate both financial andnon-financial measures in the PMS. Similarly, the findings reinforce the literatureadvocating the benefits of the BSC (Langfield-Smith et al., 2009; Epstein, 2008; Kaplanand Norton, 2006; Speckbacher et al., 2003). The findings highlight the need formanagers to evaluate the inherent characteristics of their PMS and their impact on theachievement of such outcomes. In particular managers need to focus on the extent towhich diversified performance measures reflecting the five perspectives of the BSC areincorporated in their PMS.

Additional exploratory analysis revealed that the internal business and learning andgrowth perspectives were associated with the effectiveness of PMSs regarding theperformance-related outcomes, while the learning and growth and sustainabilityperspectives were significantly associated with the staff-related outcomes. While thisfinding highlights the importance of adopting a BSC, it also provides managers withinsight into the specific BSC dimensions which warrant their attention in order toenhance PMS effectiveness. Managers are therefore encouraged to ensure that their PMSemphasises the use of performance measures in relation to the internal business process(e.g. productivity, usage of resources, cycle time and number of product returns),learning and growth (e.g. hours of training provided, improvements made to employeefacility, number of new product produced and percentage of revenue from newapplications) and sustainability (e.g. investment in environmental management,promotion of environmental causes and investment in community services) perspectivesin order to enhance the effectiveness of their PMS.

Analysis of the association between the organizational factors and the effectivenessof the PMS provides an insight into the prevailing organizational conditions thatcould enhance/prohibit PMS effectiveness. Top management support was found to beassociated with the performance-related outcomes, and the level of training wasassociated with the staff-related outcomes. While top management support has beenfound to be a critical success factor for PMS implementation (Bourne, 2005; Chan, 2004;Bourne et al., 2002; Emerson, 2002; Kennerley and Neely, 2002), the findings highlight theimportance of the continued involvement and support from top management. Hence, inorder to achieve the desired performance-related outcomes, a concentrated effort by topmanagement aimed at continuous improvement, open communication and consistentsupport is required (Kaynak, 2003). Top management is therefore encouraged topersonally commit to the PMS and ensure that enough time and resources are dedicatedon an on-going basis to properly develop and manage the existing PMS. In addition,organizations which provide more related training to their staff are able to achieve thedesired staff-related outcomes. This supports Harel and Tzafrir’s (1999) suggestion thatmoving knowledge information and power to lower levels of the organization is a way tosustain competitive advantage. Organizations could therefore employ appropriatetraining with respect to the use of PMSs across different business levels to enhance theknowledge and skill of employees in developing and implementing the systems.

The study contributes to the literature by examining PMS effectiveness in terms ofthe effect on organizational processes. The two dimensions of PMS effectiveness,performance- and staff-related outcomes, serve to make management more aware of theneed to focus on different aspects of PMS effectiveness as well as providing researcherswith a new measure which can be used to evaluate its effectiveness. In addition,


1301

the association between the use of multidimensional performance measuresand organizational factors with the effectiveness of the PMS provides managers oforganizations with an insight into the desirable characteristics of an effective PMS andthe prevailing organizational conditions which can support the PMS. Hence, managersneed to focus on using multidimensional performance measures, and increase the level oftop management support and related training in relation to their PMS.

5.2 Limitations and future researchThe study is subject to the usual limitations of the survey method. While the surveymethod is useful in ascertaining associations rather than causal relationships betweenvariables (Singleton and Straits, 2005), this approach generates potential threats asrespondents may answer questions in accordance with social desirability bias. Futurestudies may incorporate face-to-face interviews in order to provide richer descriptionsinto the hypothesised associations. Future studies could also collect data from multiplerespondents across different management levels. This may assist in overcoming thecommon method bias associated with the single respondent approach.

The study also used a number of self-developed measures. For instance, the measuresof PMS effectiveness, the usage of multidimensional performance measures, and twoorganisational factors, employee participation and the link of performance to rewards,were self-developed. The face validity of these measures was enhanced through a pilotstudy of ten academics with relevant expertise, and their content validity was enhancedby developing the measures based on an extensive review of the relevant literature.However, while factor analysis provides evidence of the construct validity of the firstthree measures, the validity of these measures still needs to be confirmed in futurestudies, especially given the sample size is considered small for performing factoranalysis. In addition while the Cronbach’s a scores confirm the reliability of the firstthree of these measures, the two items used to measure link of performance to rewardswere found to be measuring separate constructs. Hence, there is concern as to thereliability of this measure and future studies may explore alternative ways of measuringthis factor.

In addition, the current study only provides empirical evidence in relation to theassociation between four organizational factors (top management support, training,employee participation and link of performance to rewards) and the effectiveness ofPMS. Future studies may consider the association between other organizational factorssuch as organizational structure, and management style, with PMS effectiveness.To enhance the generalizability of the findings, future studies could be conductedusing similar parameters in other industries such as service and the non-profit sector.

Notes

1. Cohen’s (1988) formulae which considers the number of independent variables, statisticalsignificance and power, and effect size, was used to determine the required number of validresponses (91). Assuming a conservative response rate of 20 percent, a sample size of 445 wasdetermined.

2. The Kompass Australia business directory provides details of manufacturing businesses inAustralia. It is assumed that a random sample taken from this directory is representative ofthe Australian manufacturing industry.

IJOPM31,12

1302

3. The Dillman (2007) Tailored Design Method provides guidelines in respect to the format andstyle of questions, personalisation, and distribution procedures. There is evidence that thisapproach leads to improved response rates.

References

Atkinson, A., Balakrishnan, A.R., Booth, P., Cote, J.M., Groot, T., Malmi, T., Roberts, H., Uliana, E.and Wu, A. (1997), “New directions in management accounting research”, Journal ofManagement Accounting Research, Vol. 9, pp. 79-108.

Baird, K. (2010), “The effectiveness of strategic performance measurement systems”,unpublished working paper, Macquarie University, Sydney.

Baird, K., Harrison, G. and Reeve, R. (2007), “Success of activity management practices: the influenceof organizational and cultural factors”, Accounting and Finance, Vol. 47 No. 1, pp. 47-67.

Bedford, D.S., Brown, D.A. and Malmi, T. (2006), “Balanced scorecard content, use,and performance impacts: some Australian evidence”, paper presented at the EuropeanAccounting Association, 29th Annual Conference, Dublin.

Bhasin, S. (2008), “Lean and performance measurement”, Journal of Manufacturing TechnologyManagement, Vol. 19 No. 5, pp. 670-84.

Bonner, S.E. and Sprinkle, G.B. (2002), “The effects of monetary incentives on effort and taskperformance: theories, evidence, and a framework for research”, Accounting,Organizations and Society, Vol. 27 Nos 4/5, pp. 303-45.

Bourne, M.C. (2005), “Researching performance measurement system implementation: thedynamics of success and failure”,Production Planning andControl, Vol. 16 No. 2, pp. 101-13.

Bourne, M.C., Neely, A.D., Platts, K.W. and Mills, J.F. (2002), “The success and failure ofperformance measurement initiatives: the perceptions of participating managers”,International Journal of Operations and Production Management, Vol. 22 No. 11,pp. 1288-310.

Braam, G.J.M. and Nijssen, E.J. (2004), “Performance effects of using the balanced scorecard:a note on the Dutch experience”, Long Range Planning, Vol. 37 No. 4, pp. 335-49.

Bryant, L., Jones, D.A. and Widener, S.K. (2004), “Managing value creation within the firm:an examination of multiple performance measures”, Journal of Management AccountingResearch, Vol. 16 No. 1, pp. 107-31.

Burney, L.L., Henle, C.A. and Widener, S.K. (2009), “A path model examining the relations amongstrategic performance measurement system characteristics, organizational justice,and extra- and in-role performance”, Accounting, Organizations and Society, Vol. 34Nos 3/4, pp. 305-21.

Cavaluzzo, S.K. and Ittner, C.D. (2004), “Implementing performance measurement innovations:evidence from government”, Accounting, Organisations and Society, Vol. 29 No. 1,pp. 243-67.

Chan, Y.C.L. (2004), “Performance measurement and adoption of balanced scorecards: a survey ofmunicipal governments in the USA and Canada”, The International Journal of PublicSector Management, Vol. 17 No. 3, pp. 204-21.

Chang, C.J., Lee, H.I. and Chen, W.C. (2008), “A fuzzy AHP and BSC approach for evaluatingperformance of IT department in the manufacturing industry in Taiwan”, Expert Systemswith Applications, Vol. 34 No. 1, pp. 96-107.

Cheng, M.I., Dainty, A. and Moore, D. (2007), “Implementing a new performance managementsystem within a project-based organisation: a case study”, International Journal ofProductivity and Performance Management, Vol. 56 No. 1, pp. 60-75.


1303

Chiles, A.M. and Zorn, T.E. (1995), “Empowerment in organizations: employees’ perceptions ofthe influences on empowerment”, Journal of Applied Communication Research, Vol. 23No. 1, pp. 1-25.

Chow, C.W. and Van der Stede, W.A. (2006), “The use and usefulness of nonfinancialperformance measures”, Management Accounting Quarterly, Vol. 7 No. 3, pp. 1-8.

Chung, L.H., Gibbons, P.T. and Schoch, H.P. (2006), “The management of information andmanagers in subsidiaries of multinational corporations”, British Journal of Management,Vol. 17 No. 2, pp. 153-65.

Clinquini, L. and Mitchell, F. (2005), “Success in management accounting: lessons from theactivity-based costing/management experience”, Journal of Accounting and OrganizationalChange, Vol. 1 No. 1, pp. 63-78.

Cohen, J. (1988), Statistical Power Analysis for the Behavioral Sciences, 2nd ed.,Lawrence-Erlbaum, Mahwah, NJ.

Cox, A., Marchington, M. and Sutter, J. (2007), Embedding the Provision of Information andConsultation in the Workplace: A Longitudinal Analysis of Employee Outcomes in 1998and 2004, DTI Employee Relations Research Series, No. 72, Department of Trade andIndustry, London.

Cox, A., Zagelmeyer, S. and Marchington, M. (2006), “Embedding employee involvement andparticipation at work”, Human Resource Management Journal, Vol. 16 No. 3, pp. 250-67.

Crabtree, A.D. and DeBusk, G.K. (2008), “The effects of adopting the balanced scorecard onshareholder returns”, Advances in Accounting, Vol. 24 No. 1, pp. 8-15.

Davis, S. and Albright, T. (2004), “An investigation of the effect of balanced scorecardimplementation on financial performance”, Management Accounting Research, Vol. 15No. 2, pp. 135-53.

De Geuser, F., Mooraj, S. and Oyon, D. (2009), “Does the balanced scorecard add value? Empiricalevidence on its effect on performance”,EuropeanAccountingReview, Vol. 18 No. 1, pp. 93-122.

Dillman, D.A. (2007), Mail and Internet Surveys: The Tailored Design Method, 2nd ed., Wiley,New York, NY.

Doll, W.J. (1985), “Avenues for top management involvement in successful MIS development”,MIS Quarterly, Vol. 9 No. 1, pp. 17-35.

Emerson, B. (2002), “Training for performance measurement success: an effective trainingprogram can help get performance measurement off the ground and sustain the systemas it matures into a catalyst for government accountability and improvement”,Government Finance Review, April, available at: www.thefreelibrary.com/Trainingþforþperformanceþmeasurementþsuccess%3aþAnþeffectiveþtraining-a085048611(accessed 1 May 2009).

Epstein, M. (2008), Making Sustainability Work: Best Practices in Managing and MeasuringSocial and Environmental Impacts, Greenleaf, Sheffield.

Epstein, M. and Manzoni, J.F. (1998), “Implementing corporate strategy: from tableaux de bord tobalanced scorecard”, European Management Journal, Vol. 16 No. 2, pp. 190-203.

Figge, F., Hahn, T., Schaltegger, S. and Wagner, M. (2002), “The sustainability balancedscorecard – linking sustainability management to business strategy”, Business Strategyand the Environment, Vol. 11 No. 5, pp. 269-84.

Grover, V. (1993), “An empirical derived model for the adoption of customer-basedinter-organizational systems”, Decision Sciences, Vol. 24 No. 3, pp. 603-39.

Hamilton, S. and Chervany, N.L. (1981), “Evaluating information system effectiveness – part I:comparing evaluation approaches”, MIS Quarterly, Vol. 5 No. 3, pp. 55-69.

IJOPM31,12

1304

Harel, H.G. and Tzafrir, S.S. (1999), “The effect of human resource management practices on theperceptions of organizational and market performance of the firm”, Human ResourceManagement, Vol. 38 No. 3, pp. 185-200.

Hoque, Z. and Adams, C. (2008), Measuring Public Sector Performance: A Study of GovernmentDepartments in Australia, CPA Australia, Melbourne.

Hoque, Z. and James, W. (2000), “Linking balanced scorecard measures to size and market factors:impact on organizational performance”, Journal of Management Accounting Research,Vol. 12 No. 1, pp. 1-17.

Horngren, C.T., Bhimani, A., Datar, S.M. and Foster, G. (2005), Management and CostAccounting, 3rd ed., Pearson Education, Harlow.

Huselid, M.A. (1995), “The impact of human resource management practices on turnover,productivity, and corporate financial performance”, Academy of Management Journal,Vol. 38 No. 3, pp. 535-672.

Ittner, C.D. and Larcker, D.F. (1998), “Innovations in performance measurement: trends andresearch implications”, Journal of Management Accounting Research, Vol. 10, pp. 205-38.

Ittner, C.D., Larcker, D.F. and Randall, T. (2003), “Performance implications of strategicperformance measurement in financial services firms”, Accounting, Organizations andSociety, Vol. 28 Nos 7/8, pp. 715-41.

Johanson, U., Skoog, M., Backlund, A. and Almqvist, R. (2006), “Balancing dilemmas of the balancedscorecard”, Accounting, Auditing & Accountability Journal, Vol. 19 No. 6, pp. 842-57.

Jusoh, R., Ibrahim, D.N. and Zainuddin, Y. (2008), “The performance consequence of multipleperformance measures usage: evidence from the Malaysian manufacturers”, InternationalJournal of Productivity and Performance Management, Vol. 57 No. 2, pp. 119-36.

Kaplan, R.S. (2001), “Strategic performance measurement and management in nonprofitorganizations”, Nonprofit Management and Leadership, Vol. 11 No. 3, pp. 353-70.

Kaplan, R.S. and Norton, D.P. (1992), “The balanced scorecard-measures that drive performance”,Harvard Business Review, Vol. 70, January/February, pp. 71-9.

Kaplan, R.S. and Norton, D.P. (1996), “Linking the balanced scorecard to strategy”, CaliforniaManagement Review, Vol. 39 No. 1, pp. 53-79.

Kaplan, R.S. and Norton, D.P. (2001), The Strategy-focused Organization: How BalancedScorecard Companies Thrive in the New Business Environment, Harvard Business SchoolPress, Boston, MA.

Kaplan, R.S. and Norton, D.P. (2004), “Measuring the strategic readiness of intangible assets”,Harvard Business Review, February, pp. 52-63.

Kaplan, R.S. and Norton, D.P. (2006), Alignment: Using the Balanced Scorecard to CreateCorporate Synergies, Harvard Business School Press, Boston, MA.

Kaplan, S.E. and Wisner, P.S. (2009), “The judgmental effects of management communicationsand fifth balanced scorecard category on performance evaluation”, Behavioral Research inAccounting, Vol. 21 No. 2, pp. 37-56.

Kaynak, H. (2003), “The relationship between total quality management practices and theireffects on firm performance”, Journal of Operations Management, Vol. 21 No. 4, pp. 405-35.

Kennerley, M. and Neely, A. (2002), “A framework of the factors affecting the evolution ofperformance measurement systems”, International Journal of Operations and ProductionManagement, Vol. 22 No. 11, pp. 1222-45.

Kleingeld, A.D., Tuijl, H.V. and Algera, J. (2004), “A participation in the design of performancemanagement systems: a quasi-experimental field study”, Journal of OrganizationalBehavior, Vol. 25 No. 7, pp. 831-51.


1305

Koberg, C.S., Boss, R.W., Senjem, J.C. and Goodman, E.A. (1999), “Antecedents and outcomes ofempowerment: empirical evidence from the health care industry”, Group and OrgansiationManagement, Vol. 24 No. 1, pp. 71-91.

Kompass Australia (2009), Kompass Australia, Peter Isaacson Publications, Elsternwick.

Krumwiede, K.R. (1998), “The implementation stages of activity-based costing and the impact ofcontextual and organizational factors”, Journal of Management Accounting Research,Vol. 10, pp. 239-77.

Langfield-Smith, K., Thorne, H. and Hilton, R. (2009), Management Accounting: An AustralianPerspective, 6th ed., McGraw-Hill, Sydney.

Lawler, E.E. (2003), “Reward practices and performance management system effectiveness”,Organizational Dynamics, Vol. 32 No. 4, pp. 396-404.

Lebas, M.J. (1995), “Performance measurement and performance management”, InternationalJournal of Production Economics, Vol. 41 Nos 1-3, pp. 23-35.

Lipe, M.G. and Salterio, S.E. (2000), “The balanced scorecard: judgmental effects of common andunique performance measures”, Accounting Review, Vol. 75 No. 3, pp. 283-9.

Lynch, R.L. and Cross, K.F. (1991), Measure Up! Yardsticks for Continuous Improvement,Blackwell, Cambridge.

McShane, S. and Travaglione, T. (2003), Organizational Behaviour on the Pacific Rim,McGraw-Hill, Sydney.

Malina, M.A. and Selto, F.H. (2001), “Communicating and controlling strategy: an empiricalstudy of the effectiveness of the balanced scorecard”, Journal of Management AccountingResearch, Vol. 13, pp. 47-90.

Marchand, M. and Raymond, L. (2008), “Researching performance measurement systems:a information systems perspective”, International Journal of Operations and ProductionManagement, Vol. 28 No. 7, pp. 663-86.

Morrell, K. and Wilkinson, A. (2002), “Empowerment: though the smoke and past the mirrors?”,Human Resource Development International, Vol. 5 No. 1, pp. 119-30.

Motwani, J., Mirchandani, D., Madan, M. and Gunasekaran, A. (2002), “Successfulimplementation of ERP projects: evidence from two case studies”, International Journalof Production Economics, Vol. 75 Nos 1/2, pp. 83-96.

Myer, J.P. and Allen, N.J. (1991), “A three-component conceptualization of organizationalcommitment”, Human Resource Management Review, Vol. 1, pp. 61-89.

Neely, A. (1999), “The performance measurement revolution: why now and what next?”,International Journal of Operations and Production Management, Vol. 19 No. 2, pp. 205-28.

Neely, A. and Adams, C. (2000), Perspectives on Performance: The Performance Prism,Gee Publishing, London.

Neely, A., Gregory, M. and Platts, K. (1995), “Performance measurement system design:a literature review and research agenda”, International Journal of Operations andProduction Management, Vol. 25 No. 12, pp. 1228-63.

Neely, A., Mills, J., Platts, K., Gregory, M. and Richards, H. (1996), “Performance measurementsystem design: should process based approaches be adopted?”, International Journal ofProduction Economics, Vol. 46, pp. 423-31.

Neely, A., Mills, J., Platts, K., Richards, H., Gregory, M., Bourne, M. and Kennerley, M. (2000),“Performance measurement system design: developing and testing a process-basedapproach”, International Journal of Operations and Production Management, Vol. 20No. 10, pp. 1119-45.

IJOPM31,12

1306

Norreklit, H. (2003), “The balanced scorecard: what is the score? A rhetorical analysis of thebalanced scorecard”, Accounting, Organizations and Society, Vol. 28 No. 6, pp. 591-619.

Nunnally, J.C. (1978), Psychometric Theory, McGraw-Hill, New York, NY.

Othman, R. (2008), “Reflective practice: enhancing the effectiveness of the balanced scorecardwith scenario planning”, International Journal of Productivity and PerformanceManagement, Vol. 57 No. 3, pp. 259-66.

Otley, D. (1999), “Performance management: a framework for management control systemsresearch”, Management Accounting Research, Vol. 10 No. 4, pp. 282-362.

PA Consulting Group (1998), “An incentive scheme that works: pay purview, how oneknowledge-intensive company has overhauled its incentives”, The Economist, August,available at: www.paconsulting.com/introducing-pas-media-site/archive/an-incentive-scheme-that-works-pay-purview-how-one-knowledge-intensive-company-has-overhauled-its-incentives/ (accessed 10 April 2009).

Pike, S. and Roos, G. (2004), “Mathematics and modern business management”, Journal ofIntellectual Capital, Vol. 5 No. 2, pp. 243-56.

Pun, K., Chin, K. and Gill, R. (2001), “Determinants of employee involvement practices inmanufacturing enterprises”, Total Quality Management, Vol. 12 No. 1, pp. 95-109.

Raghunathan, B., Raghunathan, T. and Tu, Q. (1999), “Dimensionality of the strategic gridframework: the construct and its measurement”, Information Systems Research, Vol. 10No. 4, pp. 343-55.

Rao, S.S. (2000), “Enterprise resource planning: business needs and technologies”, IndustrialManagement and Data Systems, Vol. 100 No. 1, pp. 81-8.

Ratnasingam, P. (2009), “Service quality management applying the balanced scorecard:and exploratory study”, International Journal of Commerce and Management, Vol. 19No. 2, pp. 127-36.

Rigby, D. and Bilodeau, B. (2009), Management Tools and Trends 2009, Bain & Company,Boston, MA.

Robert, E.S. (1999), “In defence of the survey method: an illustration from a study of userinformation satisfaction”, Accounting and Finance, Vol. 39 No. 1, pp. 53-77.

Rynes, S.L., Gerhart, B. and Parks, L. (2005), “Personnel psychology: performance evaluation andpay for performance”, Annual Review of Psychology, Vol. 56, pp. 571-600.

Schultz, R. and Ginzberg, M.J. (1984), “Implementation research-third generation”, Applications ofManagement Science, Vol. 4, pp. 1-83.

Shields, M.D. (1995), “An empirical analysis of firms’ implementation experiences withactivity-based costing”, Journal of Management Accounting Research, Vol. 7, pp. 148-66.

Silk, S. (1998), “Automating the balanced scorecard”, Management Accounting, Vol. 79 No. 11,pp. 38-44.

Simons, R. (2000), Performance Measurement and Control Systems for Implementing Strategy,Prentice-Hall, Englewood Cliffs, NJ.

Sinclair, R.R., Leo, M.C. and Wright, C. (2005), “Benefit system effects on employees’ benefitknowledge, use, and organizational commitment”, Journal of Business and Psychology,Vol. 20 No. 1, pp. 3-29.

Singleton, R.A. and Straits, B.C. (2005), Approaches to Social Research, Oxford University Press,New York, NY.


1307

Speckbacher, G., Bischof, J. and Pfeiffer, T. (2003), “A descriptive analysis on the implementationof balanced scorecard in German speaking countries”, Management Accounting Research,Vol. 14 No. 4, pp. 361-88.

Van der Stede, W.A., Chow, C.W. and Lin, T.W. (2006), “Strategy, choice of performancemeasures, and performance”, Behavioral Research in Accounting, Vol. 18, pp. 185-205.

Wagner, A.J. (1994), “Participation’s effects on performance and satisfaction: a reconsideration ofresearch evidence”, Academy of Management Review, Vol. 19, pp. 312-30.

Wexley, K.N. (1984), “Personnel training”, Annual Reviews, Vol. 35, pp. 519-51.

Whorter, L.B.M. (2003), “Does the balanced scorecard reduce information overload?”,Management Accounting Quarterly, Vol. 4 No. 4, pp. 23-7.

Wimalasiri, J. and Kouzmin, A. (2000), “A comparative study of employee involvementinitiatives in Hong Kong and the USA”, International Journal of Manpower, Vol. 21 No. 8,pp. 614-34.

Appendix. Variable measurementEffectiveness of PMS (adapted from Lawler (2003), Ittner et al. (2003) and Kaplan and Norton(2001, 1996))Please indicate the extent to which your business unit’s PMS assists your business unit inachieving each of these.Performance-related outcomes:

Motivating performance.

Assisting in the achievement of goals.

Developing a performance-oriented culture.

Supporting change efforts.

Providing useful performance feedback to employees.

Implementing the organizational strategy.

Providing an accurate assessment of business.

Ensuring staff commitment to organizational objectives.

Linking individual performance to business unit performance.

Staff-related outcomes:

Developing individual’s skill and knowledge.

Addressing the concerns of staff.

Ensuring the concerns of staff.

Identifying talented employees.

Rewarding talented employees.

Identifying poor performing staff.

Managing poor performing staff.

The use of multidimensional performance measuresFinancial perspective:

Sales revenue.

Return on investment.

Improvement in net assets/liabilities.

IJOPM31,12

1308

Customer perspective:

On-time product delivery.

Number of new customers.

Quality of products.

Number of product returns.

Internal business process perspective:

Usage/wastage of resources.

Productivity.

Cycle time.

Expenditure on warranty claims.

Learning and growth perspective:

Hours of training provided.

Improvements made to employee facilities.

Number of employee suggestions implemented.

Number of new products produced.

Time to market for new products.

Percentage of revenue from new products/new applications.

Sustainability:

Investment in environmental management.

Promotion of environmental causes.

Investment in community services.

Community connectedness services.

Promotion of community causes.

Organizational factorsTop management support:

Top management has provided adequate resources to support the PMS.

Top management has effectively communicated its support for the PMS.

Top management exercises its authority in support of the PMS.

Training:

Adequate training has been provided to ensure employees understand the PMS.

Adequate training has been provided to develop the PMS.

Adequate training has been provided to implement the PMS.

Employee participation:

Lower level employees participated in designing the PMS.

Lower level employees were involved in selecting performance measures.

Link of performance to rewards:

Performance is linked to financial rewards (pay, bonuses, etc.) in your business unit.

Performance is linked to non-financial rewards (recognition, service awards, etc.) in yourbusiness unit.


1309

About the authorsAmy Tung has taught both undergraduate and postgraduate subjects in the managementaccounting area. Her research interests include performance measurement systems,environmental management and employee organizational commitment. She is undertaking herPhD in sustainability, with focus on environmental management systems and environmentalperformance. Amy Tung is the corresponding author and can be contacted at: [email protected]

Kevin Baird has taught both undergraduate and postgraduate subjects in the managementaccounting area for 16 years. He has also supervised Honours and PhD students across manydifferent topic areas within the management accounting discipline including activity-basedmanagement practices, total quality management, performance measurement systems,management control systems, outsourcing, employee organizational commitment andemployee empowerment.

Herbert P. Schoch has taught both undergraduate and post-graduate courses, primarily inManagement Accounting and he has supervised PhD and Honours students. He has also taughtFinancial Accounting, Business Strategy and Entrepreneurship and EntrepreneurialManagement. He has taught in Australia, Singapore, Hong Kong, Canada and the USA. Hisresearch interests include management control systems, management accounting, outsourcing,accounting education and entrepreneurship. He has published numerous journal articles, bookchapters and monographs. He also has experience in working in manufacturing, publicaccounting and has managed and operated his own business.

IJOPM31,12

1310

To purchase reprints of this article please e-mail: [email protected] visit our web site for further details: www.emeraldinsight.com/reprints