ACCOUNTABILITY PERFORMANCE REPORTING …Accountability, performance reporting, comprehensive audit -...

ACCOUNTABILIT Y,PERFORMANCE

REPORTING,COMPREHENSIVE

AUDIT -AN INTE G RATED PERSPECTIVE

b y

G U Y L E C L E R C

W . D A V I D M O Y N A G H

J E A N - P I E R R E B O I S C L A I R

H U G H R . H A N S O N

Accountability, performance reporting, comprehensive audit - an integrated perspective

Copyright © 1996 CCAF-FCVI Inc.

All rights reserved. No part of this publication may be repro-duced, stored in a data base or retrieval system, or transmitted, inany form or by any means, electronic, mechanical, photocopying,recording, or otherwise, without the prior written permission ofthe publisher, CCAF-FCVI Inc.

Published byCCAF-FCVI Inc.55 Murray St., Suite 210Ottawa, CANADAK1N 5M3(613) 241-6713 Fax (613) 241-6900

ISBN 0-919557-47-3

Printed and bound in Canada.Design and layout by Paul Edwards Design. Print coordination by Poirier Litho.

This book is available in French under the title: Reddition de comptes, rapports sur la performance et vérification intégrée - une vue d’ensemble.Translation by Traduction Nicole Plamondon.

Canadian Cataloguing in Publication Data:

Main entry under title:Accountability, performance reporting, comprehensive

audit - an integrated perspective

Issued also in French under title: Reddition de comptes, rapports sur la performance et vérification intégrée - une vue d’ensembleIncludes bibliographical references and index.ISBN 0-919557-47-3

1. Auditing. 2. Organizational Effectiveness.I. CCAF-FCVI Inc.

HF5667.A26 1996 657’.45 C96-900071-S

TABLE OF CONTENTS

PART I. ACCOUNTABILITY.................................................................................1INTRODUCTION .............................................................................................................................3

SECTION 1. ACCOUNTABILITY’S CONTEXT—GOVERNANCE ............................................7

CHAPTER 1. GOVERNANCE—DEFINITIONS AND ISSUES ...............................................................8GOVERNANCE VERSUS MANAGEMENT ........................................................................9GOVERNANCE STRUCTURES......................................................................................10SOME RELATED CONCEPTS........................................................................................10DEMOCRACY .............................................................................................................12THE NATURE OF A CONSTITUTION...........................................................................13ESSENTIAL CHARACTERISTICS OF GOOD GOVERNANCE............................................14

CHAPTER 2. GOVERNANCE IN CANADA.......................................................................................15THE FORM OF GOVERNMENT IN CANADA................................................................15THE ROLE OF ELECTED REPRESENTATIVES................................................................21INFLUENCES ON POLICY IN A DEMOCRACY...............................................................22SOME IMPLICATIONS OF FEDERALISM .......................................................................25GOVERNMENT IN OTHER COUNTRIES ......................................................................26PRIVATE SECTOR GOVERNANCE ................................................................................27GOOD GOVERNANCE — A COMMON GOAL..............................................................31

CHAPTER 3. THE LANGUAGE OF GOVERNANCE AND ACCOUNTABILITY.....................................33VOCABULARY AND TERMINOLOGY ............................................................................33RULES AND DISCRETION ...........................................................................................33DISCRETIONARY POWERS ..........................................................................................35BUREAUCRATIZATION................................................................................................35INDEPENDENCE.........................................................................................................36NEUTRALITY..............................................................................................................38AUTONOMY...............................................................................................................39SUBSIDIARITY ............................................................................................................40STEWARDSHIP............................................................................................................40VICARIOUS RESPONSIBILITY ......................................................................................41MANAGEMENT AND ADMINISTRATION .....................................................................41

SECTION 2. ACCOUNTABILITY—ISSUES & PRACTICE.....................................................43

CHAPTER 4. THE MEANING OF ACCOUNTABILITY ......................................................................44ACCOUNTABILITY DEFINED.......................................................................................44ACCOUNTABILITY—MANY CONTEXTS, MANY FORMULATIONS .................................44THE ROOT OF ACCOUNTABILITY...............................................................................47DEMONSTRATING ACCOUNTABILITY.........................................................................50EXTERNAL INDUCEMENT FOR ACCOUNTABILITY ......................................................50THE ENVIRONMENT FOR ACCOUNTABILITY..............................................................54AUTONOMY AND ACCOUNTABILITY ..........................................................................56DIMENSIONS OF ACCOUNTABILITY ...........................................................................57IN SUMMARY .............................................................................................................59

A C C O U N T A B I L I T Y , P E R F O R M A N C E R E P O R T I N G , C O M P R E H E N S I V E A U D I T - A N I N T E G R A T E D P E R S P E C T I V E

CHAPTER 5. ACCOUNTABILITY OBLIGATIONS..............................................................................61THE TRADITIONAL VIEW...........................................................................................62THE DEBATE..............................................................................................................62ACCOUNTABILITY OF PUBLIC SERVANTS....................................................................66

CHAPTER 6. ACCOUNTABILITY IN CERTAIN OTHER CIRCUMSTANCES ........................................71ACCOUNTABILITY IN THE PRIVATE SECTOR...............................................................71DIFFERENCE BETWEEN PRIVATE- AND PUBLIC-SECTOR ACCOUNTABILITY ................71GOVERNMENT IN AND OUT OF BUSINESS.................................................................71ACCOUNTABILITY IN A MONOPOLY...........................................................................73THE PRIVATE NONPROFIT SECTOR............................................................................73

CHAPTER 7. CONTROL AND CONTROLS......................................................................................75CONTROL..................................................................................................................75CONTROL AND THE CHANGE IN ORGANIZATIONAL CULTURE..................................75TWO MODELS OF MANAGEMENT..............................................................................76IMPROPERLY CONTROLLED DELEGATION—STREET-LEVEL BUREAUCRATS.................77DELEGATION AND THE DIFFICULTY OF “LETTING GO” ............................................79

CONCLUSION ...............................................................................................................................80KNOW THE BUSINESS ................................................................................................80ORGANIZATIONAL KNOWLEDGE CHECKLIST.............................................................81

APPENDIX. ..................................................................................................................................84A CASE OF CHANGE IN MANAGERIAL CULTURE ......................................................84INCREASED MINISTERIAL AUTHORITY AND ACCOUNTABILITY.................................84PUBLIC SERVICE 2000 ...............................................................................................88SHARED MANAGEMENT AGENDA (SMA)..................................................................89WAS CONTROL LOST?................................................................................................91THE TRANSITION CONTINUES ..................................................................................92THE MANAGERIAL REVOLUTION: CULTURAL AND PROCEDURAL ..............................92

PART II—PERFORMANCE REPORTING ............................................................95

INTRODUCTION ...........................................................................................................................97

SECTION 1. THE THEORETICAL BASIS FOR PERFORMANCE REPORTING......................103

CHAPTER 8. EXISTING REPORTING PRACTICES .........................................................................104PUBLIC-SECTOR REPORTING MECHANISMS .............................................................104PRIVATE-SECTOR PERFORMANCE REPORTING .........................................................109SOME RECENT INITIATIVES IN REPORTING ON PERFORMANCE...............................110

CHAPTER 9. CONCEPTUAL ISSUES IN PERFORMANCE REPORTING............................................116THE CENTRAL ISSUES ..............................................................................................116PERFORMANCE AND EFFECTIVENESS.......................................................................116VALUE FOR MONEY .................................................................................................116THE EFFECTIVENESS DILEMMA ...............................................................................120EFFECTIVENESS AND THE ACHIEVEMENT OF GOALS...............................................121SOME APPROACHES TO EFFECTIVENESS...................................................................124LISTS OF ATTRIBUTES OF EFFECTIVENESS ...............................................................125SENSITIVITY OF THE NOTION OF EFFECTIVENESS...................................................129CHARACTERISTICS OF GOOD REPORTING...............................................................130ABOUT REPRESENTATIONS ......................................................................................132


SECTION 2. PRINCIPLES AND GUIDELINES FOR PERFORMANCE REPORTING................135

CHAPTER 10. AN EFFECTIVENESS-REPORTING FRAMEWORK ....................................................136BACKGROUND .........................................................................................................136MANAGEMENT REPRESENTATIONS ..........................................................................137ATTRIBUTES OF EFFECTIVENESS..............................................................................138GUIDELINES FOR MAKING REPRESENTATIONS ON THE ATTRIBUTES.......................145NEED FOR EXTERNAL REVIEW.................................................................................148

CHAPTER 11. PRELIMINARY CONSIDERATIONS IN REPORTING EFFECTIVENESS........................150OVERALL PERSPECTIVE............................................................................................150DETERMINING IF THE ORGANIZATION IS READY TO PROCEED...............................151SETTING REALISTIC EXPECTATIONS.........................................................................153DECIDING WHERE TO FOCUS INITIAL EFFORTS ......................................................153DECIDING WHO OUGHT TO PARTICIPATE...............................................................154MANAGING THE OVERALL PROCESS ........................................................................158DEVELOPING MANAGEMENT REPRESENTATIONS.....................................................159DETERMINING WHERE AND WHEN TO REPORT REPRESENTATIONS........................160

CHAPTER 12. IMPLEMENTING AN EFFECTIVENESS-REPORTING PROJECT .................................162OVERVIEW OF THE PROCESS ...................................................................................162PHASE I—INITIAL ENGAGEMENT OF THE CEO/SENIOR MANAGEMENT.................163PHASE II—DECIDING WHERE TO FOCUS AND HOW TO PROCEED .........................172PHASE III—PREPARING FOR AND BEGINNING IMPLEMENTATION ..........................175PHASE IV—REFINING THE INITIAL MANAGEMENT REPRESENTATIONS ..................187PHASE V—WRITING THE MANAGEMENT-REPRESENTATIONS REPORT....................190PHASE VI—TABLING THE REPORT WITH THE GOVERNING BODY..........................191PHASE VII—AUDIT OF REPRESENTATIONS ON EFFECTIVENESS..............................192

CONCLUSION .............................................................................................................................193ELEMENTS OF A GOOD PERFORMANCE-MANAGEMENT SYSTEM..............................193ELEMENTS OF GOOD PERFORMANCE MEASURES.....................................................195

APPENDIX — EXHIBITS ..............................................................................................................196EXHIBIT A ...............................................................................................................196EXHIBIT B ...............................................................................................................197EXHIBIT C...............................................................................................................200EXHIBIT D ..............................................................................................................203EXHIBIT E ...............................................................................................................209EXHIBIT F ...............................................................................................................210EXHIBIT G...............................................................................................................213

PART III COMPREHENSIVE AUDIT .................................................................217

INTRODUCTION .........................................................................................................................219

SECTION 1. BACKGROUND............................................................................................221

CHAPTER 13—UNDERSTANDING AUDIT...................................................................................222RELATIONSHIP TO ACCOUNTING ............................................................................222DEFINITIONS OF AUDIT ..........................................................................................224CHARACTERISTICS OF AUDIT ..................................................................................224


CHAPTER 14—UNDERSTANDING COMPREHENSIVE AUDIT ......................................................226HISTORICAL DEVELOPMENT ...................................................................................226THE COMPREHENSIVE AUDIT CONCEPT .................................................................228NON-AUDIT FORMS OF PRACTICE...........................................................................232

CHAPTER 15—THREE APPROACHES TO COMPREHENSIVE AUDIT ............................................234AN EVOLVING CONCEPT .........................................................................................234AUDIT REPORTING ON MANAGEMENT SYSTEMS AND PRACTICES ...........................236AUDIT ATTESTATION TO MANAGEMENT REPRESENTATIONS ON PERFORMANCE ....237AUDIT REPORTING ON PERFORMANCE ...................................................................241

SECTION 2. GENERAL CONSIDERATIONS ......................................................................243

CHAPTER 16—COMPREHENSIVE AUDIT—GENERAL CONSIDERATIONS ...................................244THREE KEY VARIABLES.............................................................................................245FOCUS OF EXAMINATIONS.......................................................................................247DECIDING THE SUBJECT FOR AUDIT.......................................................................248

SECTION 3. COMPREHENSIVE AUDIT PRACTICE...........................................................251

CHAPTER 17—CONDUCTING COMPREHENSIVE AUDITS ..........................................................252PLANNING THE AUDIT ............................................................................................252THE CONDUCT PHASE ............................................................................................262THE COMPREHENSIVE AUDIT REPORT ....................................................................267

CHAPTER 18—COMPREHENSIVE AUDIT—REPORTING ISSUES..................................................271INTRODUCTION ......................................................................................................271SCOPE......................................................................................................................271ASSURANCE .............................................................................................................271SIGNIFICANCE .........................................................................................................276AUDITORS’ RECOMMENDATIONS ............................................................................282FAIR AND BALANCED REPORTING ...........................................................................284

CHAPTER 19—AUDIT CRITERIA ................................................................................................286WHAT ARE AUDIT CRITERIA?...................................................................................286SUITABILITY OF CRITERIA........................................................................................287USING AUDIT CRITERIA...........................................................................................288SOURCES OF AUDIT CRITERIA .................................................................................289

CHAPTER 20—EVIDENCE ..........................................................................................................291WHAT IS EVIDENCE? ...............................................................................................291STANDARDS OF VFM-RELATED AUDIT EVIDENCE...................................................291EVIDENCE FACTORS TO CONSIDER IN PLANNING ...................................................293RELIANCE AS A FORM OF EVIDENCE .......................................................................293METHODS OF GATHERING AUDIT EVIDENCE..........................................................295AUDIT PROCESS AND EVIDENCE..............................................................................295


SECTION 4. OTHER KEY CONSIDERATIONS..........................................................297

CHAPTER 21—INTERNAL AUDIT ...............................................................................................298WHAT IS INTERNAL AUDITING? ..............................................................................298CLIENT FOR INTERNAL AUDITING ..........................................................................299SENIOR MANAGEMENT SUPPORT.............................................................................300LEADERSHIP AND STAFFING ....................................................................................301ORGANIZATIONAL ARRANGEMENTS FOR INTERNAL AUDITING .............................302INTERNAL AUDIT REPORTS......................................................................................310

CHAPTER 22—OTHER REVIEW PROCESSES—COORDINATION & RELIANCE............................312COORDINATION AND COOPERATION......................................................................312RELIANCE ................................................................................................................313PLANNING FOR RELIANCE .......................................................................................313DEVELOPING A TENTATIVE STRATEGY FOR RELIANCE.............................................319OPTIMIZING PLANNED RELIANCE POTENTIAL ........................................................320TIMING ...................................................................................................................321CHANGES IN TENTATIVE RELIANCE STRATEGIES .....................................................322PLACING RELIANCE .................................................................................................322REPORTING ON RELIANCE.......................................................................................324OTHER FORMS OF REVIEW......................................................................................325

CHAPTER 23—STANDARDS AND QUALITY ASSURANCE.............................................................332VALUE-FOR-MONEY AUDIT STANDARDS ..................................................................332QUALITY ASSURANCE ..............................................................................................340AUDIT MANUALS .....................................................................................................340REVIEW PROCESSES .................................................................................................340STAFFING.................................................................................................................342

CHAPTER 24—PROFILE OF THE COMPREHENSIVE AUDITOR ....................................................343COMPREHENSIVE AUDIT LEADERS...........................................................................344MANAGEMENT FUNCTIONS AND SYSTEMS..............................................................345THE AUDIT TEAM....................................................................................................347

CONCLUSION .............................................................................................................................349INDEX.........................................................................................................................................352


HARRY ALLEN, GEORGE ANDERSON, ROBERT ANDERSON, RODNEY ANDERSON, ALAN ANDREWS, DOUGLAS ARCHER, JIM ARMSTRONG,PETER ARMSTRONG, MARTA ARNALDO, ROLAND ARPIN, PHILLIP ASPINALL, JOHN ASTLE, JOHN ATKINSON, JOYCE BAILEY, NATALIE BALKO,ALAN BARNARD, DAVID BARNES, PETER BARNES, MARIA BARRADOS, DOUGLAS BARRINGTON, JOHN BART, WILLIAM BARTON, GILLESBÉDARD, CLAUDE BÉGIN, KENNETH BELBECK, CAROL BELLRINGER, BRUCE BENNETT, JEAN BERNARD, LIBA BERRY, HUGUETTEBERTRAND, DAVID BIGNEY, BOB BLACK, RALPH BLACK, ANDY BOEHM, JEAN-PIERRE BOISCLAIR, SHIRLEY BOISCLAIR, ANDRÉ BOLDUC,HUGH BOLTON, DAVID BONHAM, WILLIAM BRADSHAW, JOHN BRADT, FRANCINE BRAZEAU, GUY BRETON, ROBERT BROMLEY, JAMESBROPHY, DONALD BROWN, NEIL BROWN, JUDY BURNS, RICHARD BUSKI, BRIAN CAINE, JAMES CAIRNCROSS, DOUG CAMERON, FRANCESCAMERON, HEATHER CAMPBELL, DENIS CARO, MARCEL CARON, CHARLES CARRIERE, JOHN CARSON, CLAUDE CARTER, EDWARD CASE, MARLENE CATTERALL, HOWARDC H A N , M I K E C H A R L T O N , R H É A L C H A T E L A I N , WA R R E NCH I P PI N D A L E, GA I L CH R I S T I E ,IAN CLARK, TOM CLOSSON, JIMC LU F F, C A RO L CO D O R I , J O H NC O L E , B A R R Y C O O P E R S M I T H ,PE N I N A CO O PE R S M I T H, BR I A NC O R B I S H L E Y, PAU L C O R M I E R ,KAREN CORNELIUS, CATHERINECORNELL, JEAN COWPERTHWAITE,R O B E R T C U M M I N G S , J A M E SC U R R I E , J O H N C U R R I E , J A M E SC U T T , K E I T H D A L G L I S H ,FRANCINE DALPHOND, THOMASDALTON, ALEXANDER DAVIDSON,CATHIE DAVIS, COLIN DEACON,DENIS DESAUTELS, PETER DEY,G A R Y D I C K I N S O N , E L W Y ND I C K S O N , A L A N D I LW O R T H ,M A R I L Y N D O L E N K O , E N D R ED O L H A I , E L I Z A B E T HD O M B R O S K I E , B I L L D R O V E R ,R AY M O N D D U B O I S , A L I S TA I RDUFF, VIVIANE DUNN, GORDONDUNNET, KENNETH DYE, WALLYEAMER, PAUL EDWARDS, ROBERTAELLIS, ROBERT ELTON, BRENDAE P R I L E , W I L L I A M FA R L I N G E R ,ROBIN FARQUHAR, TOM FARRELL,LYNN FAUCHER, JUDY FERGUSON,MARY FERGUSON-PARÉ, PENELOPEF I L I A T R A U LT- M A C D O N A L D ,

E L I Z A B E T H F L E E T , C E C I LFLEMING, JEAN-PIERRE FORTIN,PAUL FORTUGNO, C.E.S. FRANKS,ALISTAIR FRASER, RONALD GAGE,RICHARD GAGNÉ, IDA GARRETT,DONALD GASS, YVAN GAUDETTE,Y V E S G AU T H I E R , H E L E N G AY,F R A N K G E L I N , M U R R AY G I L L ,ALAN GILLMORE, FRED GINGELL,R O B E R T G I R O U X , J E A N - PA U LGOBEIL, ROBERT GOBEIL, BERNIEG O R M A N , L O U I S G O S S E L I N ,MA RC E L GO U L E T, TR E N T GOW,D O N A L D G R A C E Y, K AT H E R I N EG R A H A M , A L A N G R A T I A S ,C H R I S T O P H E R G R E E N , J O A NGREEN, PETER GREGORY, RONALDG R E Y, R O D G E R G U I N N , R O YG U N N , K E N N E T H G U N N I N G ,ERNEST HALL, LYLE HANDFIELD,J A C K H A N N A , J O H N H A N N A ,H U G H H A N S O N , W I L L I A MH A R K E R , M A RY- J A N E H A R R I S ,RICHARD HARRIS, JIM HAWKES,JOHN HAYES, ROBERT HAYWARD,B R Y A N H E L D , P E T E R H E L D ,JACQUES HENRICHON, STEPHENHE R B E RT, RAY HE S S I O N, JO H NHICKMAN, JANET HOFSTETTER,J O H N H O L D S T O C K , G O R D O NHUDSON, JOE HUDSON, FRANKH U G H E S , RO N H U N T I N G T O N ,CA RO L I N E HU P É, JE A N HU R S T,

OLE INGSTRUP, FRED JAAKSON, JOHN JACKSON, CRAIG JAMES, JOCK JARDINE, KATHERINE JAST, WARREN JOHNSON, DIANE JOLY,FRANÇOIS JOLY, MALCOLM JORDAN, GUYLAINE JUTEAU, TIM KAPTEIN, JOHN KELLY, RONALD KIGGINS, JOSEPH KIRCHNER, ANNEKIRKALDY, PAUL LABBÉ, HUGUETTE LABELLE, FRANÇOIS LACASSE, ROGER LACHANCE, PATRICK LAFFERTY, JEAN-GUY LALIBERTÉ, ROBERTLALONDE, JACQUES LAMONDE, CLAUDE LAMOUREUX, MARVIN LAMOUREUX, JEAN-MARIE LAMPRON, ERIC LANDE, WILLIAM LANGDON,SPENCER LANTHIER, GÉRARD LAROSE, GASTON LATULIPPE, VICTOIN LAURIN, LUC LAVOIE, GUY LECLERC, GUY LEFEBVRE, KENLEISHMAN, JULIAN LEMIEUX, DAVID LEVINE, DOUGLAS LEWIS, JAMES LIBBEY, FULVIO LIMONGELLI, ALWYN LLOYD, ROBERT LORD, CAROLASHFIELD LOUGHREY, JOHN LYNCH, NANCY LYNCH, HOWARD LYONS, ANDREW MACDONALD, DUNBAR MACDONALD, LYMAN MACINNIS,MICHAEL MACKENZIE, DAVID MACKINNON, JANICE MACKINNON, R.D. MACLEAN, HARVEY MACLEOD, ELIZABETH MACRAE, PAUL-ANDRÉ MALO, CLAYTON MANNESS, LORNA MARSDEN, ELIZABETH MARSHALL, BRIAN MARSON, ALAN MARTIN, GUY MARTIN, RITAMATHERS, SUNNY MATHIESON, LES MCADAMS, MARY MCBRIDE, HENRY MCCANDLESS, TOM MCCANN, JAMES MCCRINDELL, NEVINMCDIARMID, LEONARD MCGIMPSEY, JOSEPH MCGRATH, MICHAEL MCLAUGHLIN, DORIS MCMULLAN, EDWARD MCNAMARA, HUGHMCROBERTS, GILES MEIKLE, LARRY MEYERS, JIM MILES, RICHARD MINEAU, NICK MISHCHENKO, MARCEL MIVILLE-DÉCHÊNE,ROLLANDE MONTSION, DONALD MOORS, ELAINE MORASH, RAYMOND MORCEL, GEORGE MORFITT, ERMA MORRISON, DAVID MOYNAGH,DAVID MUIR, TERESA MURPHY, WAYNE MURPHY, BEV ANN MURRAY, MELISSA NAPIER-ANDREWS, NIGEL NAPIER-ANDREWS, KARENNELSON, EUGENE NESMITH, EDWARD NETTEN, DAVID NEUMANN, PETER NEWDICK, CLIFFORD NORDAL, PAUL NORTHOVER, ELAINE ORR,JAMES OTTERMAN, JOHN PALMER, RICHARD PALMER, PIERRE-ANDRÉ PARÉ, GARY PEALL, LEWIS PERINBAM, ERIK PETERS, JEAN PICARD,LAURENT PICARD, LISE PISTONO, NICOLE PLAMONDON, BERNIE POIRIER, JOYCE POTTER, COLIN POTTS, ED POWELL, ERROL PRICE,DOROTHY PRINGLE, ROBERT PROSSER, ROSS QUANE, MICHEL RACINE, WILLIAM RADBURN, PAUL RAMSEY, ANNE RANDELL, DAVIDRATTRAY, MICHAEL RAYNER, ANGUS REE, PATRICK REID, PAUL REINHART, BRIAN REINKE, STANLEY REMPLE, ROBERT RENNIE, ARTHURROBERGE, PAUL ROBINSON, RONALD ROBINSON, LUCIE ROCHETTE, DOUGLAS ROGERS, HARRY ROGERS, LAWRENCE ROSEN, EDWARDROWE, TERRY RUSSELL, LEONARD RUTMAN, VINOD SAGHAL, ALINE SAINT-AMAND, DONALD SALMON, ROY SALMON, KARN SANDY,DONALD SCOTT, NORMAN SCOTT, RONA SHAFFRAN, JOHN SHARPE, PETER SIMEONI, SONJA SINCLAIR, HAR SINGH, JON SINGLETON,ALISTAIR SKINNER, WILLIAM SLOAN, JACK SMITH, STUART SMITH, KIMBERLEY SPEEK, GEORGE STEPHENSON, ESTHER STERN, KENNETHSTEVENSON, MARGARET STOCKTON, WAYNE STREILOFF, CLAUDE TAYLOR, GAIL TAYLOR, ROBERT TAYLOR, JOHN THOMPSON, HUGH TIDBY,MARTHA TORY, SEYMOUR TRACHIMOVSKY, ALEC TRAFFORD, GUY TRUDEL, PETER TRUEMAN, ROBERT TURNBULL, NATALIE UMIASTOWSKI,PETER VALENTINE, PETER VAUGHAN, CHARLES VINCENT, DIANA VOSSELER, EDWARD WAITZER, ROSS WALKER, JOSIE WALSH, RONALDWARME, JOHN WATSON, SUSAN WATSON, JAMES WAUGH, BARRIE WEBB, LINDA WEEKS, MICHAEL WEIR, SUZANNE WERHAR-SEEBACH,DAVID WHITE , STAN WHITELEY, NICOLE WIECZOREK, PETER WILEY, ALAN WILLIS, GEOFFREY WILSON, JOHN WILSON, PETER WILSON,ALAN WINBERG, ANDREW WINGATE, DOUGLAS WOOD, BRENT WORTMAN, JAMES WRIGHT, GEORGINA WYMAN, DONALD YEOMANS,CARMAN YOUNG, DON YOUNG, DONALD M. YOUNG, DAVID ZUSSMAN, AND THE 119 CCAF INTERNATIONAL GRADUATE FELLOWS.

THIS BOOK IS DEDICATED TO:

THE GENIUS AND MEMORY OF THE LATE

J A M E S J . M A C D O N E L L

CCAF’S FOUNDING CHAIRMAN (1980-83),

WHOSE INTELLECT AND ENERGY CREATED THE VISION

AND POINTED THE WAY,

a n d

G O R D O N H . C O W P E R T H W A I T E

CCAF CHAIRMAN (1983-93),

WHOSE DISTINGUISHED LEADERSHIP SUSTAINED THE

VISION AND ADDED TO IT NEW DIMENSION AND

SIGNIFICANCE,

a n d

ALL THOSE WHO HAVE WORKED TO CREATE

THE BODY OF KNOWLEDGE AND EXPERIENCE

WHICH THIS BOOK REFLECTS.

INTRODUCTION

Students of comprehensive auditing and prac-titioners or clients newly engaged in this area ofaudit practice are faced with a formidable chal-lenge.

Existing literature on comprehensive audit-ing is both diverse and fragmented, oftenapproaching the subject in terms of a specificaspect of practice—or in relation to a specifictime period—or through the perspective, policiesand procedures of a specific audit organization.Rarely has the literature ventured into an exami-nation of the accountability context within whichcomprehensive auditors operate and the perspec-tives and roles of other key stakeholders to theaccountability process—that is, those who governand manage our institutions. Indeed, many gener-al texts on auditing start from the assumptionthat the practitioner already has a knowledge ofthis accountability environment.

Thus, those who are new to comprehensiveauditing face the difficult task of having tosearch far, wide and long for basic informationand discussion on such matters. The onus hasbeen on them to identify and envelop all thesesources of knowledge and practice, sort out thecontemporary from the passé, fill in the blanks,and somehow pull it all together and make theright connections.

This book seeks to remedy this problem. Itmakes no assumption that readers already possess athorough grasp of the subject of accountability andrelated reporting principles. Instead, it deals withthese matters extensively. It provides readers withthe basic concepts, frameworks, tools and practiceguidelines they need to begin their learning orcareers in the area of comprehensive auditing. It is,however, more than a collection of pieces of

knowledge and experience. The structure and con-tent of this book reflect an important underlyingphilosophy and perspective. Simply stated, the phi-losophy is this: the defining role of audit is to servean accountability relationship, and thus, to trulyunderstand comprehensive auditing one must lookbeyond the borders of technical practice. It isessential to be knowledgeable about the broaderaccountability context and the roles and perspec-tives of those who operate within it. Having suchknowledge permits practitioners to identify theaudit approaches that will add the greatest value inthe circumstances, assess the implications of suchapproaches for the actions of other key stakehold-ers in the accountability process and explain theconsequences.

In this respect, this book is an extension ofCCAF itself. The foundation’s perspective is that ifaccountability is going to work properly, both par-ties to the accountability relationship—governingbodies and management—and the auditor whoserves this accountability relationship, play impor-tant and interdependent roles. The research andeducation work of CCAF—supported by theefforts of its members—is designed to help allthree parties achieve their mutual interest inimproving performance and accountability.

The book is divided into three parts. Part Ideals with the subject of accountability. It takes thereader through a wide-ranging and thoughtfulexamination of underlying theory, concepts andprinciples, and it connects these matters to con-temporary thinking and practice in the areas ofgovernance and management. We felt compelled todevelop this material since, to be well-positioned topractise comprehensive auditing, the practitionermust first acquire a thorough knowledge of thebusiness, an essential aspect of which is theaccountability environment. The discussion thatPart I gives to these issues is, we believe, unique inscope and character.

A C C O U N T A B I L I T Y , P E R F O R M A N C E R E P O R T I N G , C O M P R E H E N S I V E A U D I T - A N I N T E G R A T E D P E R S P E C T I V E I

Part II focuses on performance reporting. Itprovides an overview of existing reporting practicesin the public and private sectors, noting a numberof recent initiatives in Canada and globally. Itmakes the case that concepts like performance andeffectiveness should be viewed as multidimension-al, arguing that this is necessary in order torespond to the reality of today’s accountability anddecision-making environment. Ideally, it is man-agement, not the auditor, that is in the best posi-tion to report (make representations) on the per-formance or effectiveness of the organization. Andif management does take on this responsibility,then the auditor plays a key role in providingassurance to the governing body (those to whommanagement reports) about the fairness and com-pleteness of the information management has pro-vided. Part II describes a framework of twelveattributes that is useful as a basis for such perfor-mance reporting. Drawing on actual practice inapplying this framework, key considerations arediscussed and a strategy for implementation is sug-gested. Audit practitioners need to have a thoroughappreciation of such performance reporting con-cepts and strategies if they are to conduct theirassurance role or, possibly, if they are called uponto provide such information to the governing bodydirectly, in the absence of management taking onthis reporting responsibility.

Part III provides an extensive review of com-prehensive audit theory and practice. It examinesseveral basic factors that influence the nature of theaudit process and product. Three approaches ormodels of comprehensive auditing, and relatedconsiderations and implications, are also discussed.Most important for readers to know is that com-prehensive audit is not a one-size-fits-all proposi-tion. Moreover, what these different approachesentail, and how they are explained and implement-ed, is very much linked to matters discussed in ear-lier parts of the book. Part III also provides an

overview of the issues and factors practitionersneed to keep in mind, and the methods theyemploy, in conducting a comprehensive audit.Several key professional practice issues are given in-depth treatment, among them, reporting, audit cri-teria and evidence, existing standards of practiceand quality assurance. The links between compre-hensive audit and internal audit are also examined.

All three parts of the book emphasize thatthere is no universal template or panacea. There arechoices. What the book seeks to do is provide read-ers with a basic understanding of what these choic-es are and how they are interconnected. It gives thereader a starting point and set of considerationsfrom which to assess the relative merits of thesechoices in the circumstances concerned. And itsuggests frameworks and strategies that can behelpful to the practitioner in engaging those whogovern and manage the enterprise and, ultimately,in implementing the decisions that result from thisprocess.

Educators in the areas of management, publicadministration and audit will find this book help-ful as a basic reference text around which to devel-op a program of study for their students. Moreexperienced comprehensive auditors will also findaspects of it useful—in reinforcing existingapproaches, and perhaps in other ways, by addingnew ideas and perspectives to their work.

This textbook is based on almost two decadesof practice in the field of comprehensive auditing.It flows from the thinking and experience of sever-al dozens of leading practitioners and researchers.It is the product of four individuals. First to men-tion is Guy Leclerc, former Deputy ComptrollerGeneral of Canada and for two years a researchassociate with CCAF. His efforts were joined bythose of W. David Moynagh, CCAF’s director ofresearch and Jean-Pierre Boisclair, the foundation’spresident. Hugh R. Hanson played a key role aswriter and as editor-in-chief ensuring that all the

A C C O U N T A B I L I T Y , P E R F O R M A N C E R E P O R T I N G , C O M P R E H E N S I V E A U D I T - A N I N T E G R A T E D P E R S P E C T I V EI I

pieces fit together. Liba Berry copyedited the book,Nicole Plamondon translated it into French, andSuzanne Seebach, Director, Operations &Communications, coordinated production andpublication arrangements. All the members ofCCAF’s secretariat played an important supportingrole in the development of this textbook. Withoutthe efforts of all these people, this project wouldnot have been possible.

A senior advisory panel was also establishedto provide strategic advice on a number of impor-tant policy issues relating to the focus of the bookand the development of key positions taken withinit. Comprising this panel were: Guy Breton,Auditor General of Quebec; Carol Bellringer,Provincial Auditor of Manitoba; Alexander M.Davidson, managing partner of Accounting andAuditing, Coopers & Lybrand; Denis Desautels,Auditor General of Canada; J. Colin Potts, partner,Deloitte & Touche; and Carman L. Young,Auditor of the Bank of Canada and former chair-man of the board of the Institute of InternalAuditors. The perspective and wisdom that thesedistinguished individuals brought to their rolemade a major contribution to this initiative.

The foundation’s board of governors andexecutive relied on CCAF’s Research Committee toprovide a rigourous professional challenge of con-tent. In exercising this role, all members of thecommittee played a critical part in the develop-ment of this volume. It has benefitted greatly fromtheir insights and suggestions.

A C C O U N T A B I L I T Y , P E R F O R M A N C E R E P O R T I N G , C O M P R E H E N S I V E A U D I T - A N I N T E G R A T E D P E R S P E C T I V E I I I

P A R T I

ACCOUNTABILITY

A C C O U N T A B I L I T Y , P E R F O R M A N C E R E P O R T I N G , C O M P R E H E N S I V E A U D I T - A N I N T E G R A T E D P E R S P E C T I V E 1

A C C O U N T A B I L I T Y , P E R F O R M A N C E R E P O R T I N G , C O M P R E H E N S I V E A U D I T - A N I N T E G R A T E D P E R S P E C T I V E2

TO SERVE… AND PRESENT MY TRUE ACCOUNT…

J O H N M I L T O N , S O N N E T X V I ,

O N H I S B L I N D N E S S

P A R T 1

ACCOUNTABILITY

I N T R O D U C T I O N

Effective audit practice starts with a thoroughunderstanding of the context within which theclient and other key stakeholders operate. Thiscontext involves both the governance and manage-ment process, and the accountability arrangementsthat bind them together.

There has been much discussion aboutaccountability. Accountability implies responsibilityand public trust. The contemporary emphasis is oneverybody’s assuming responsibility and beingaccountable. Saying it, however, does not necessari-ly make it so. So many situations and circum-stances demand that public officials and people ingeneral be inspired by a sense of demonstrableresponsibility, and yet it is the lack of accountabili-ty or its inadequacy that drives the current dis-course on the subject. What characterizes the dis-cussion is that beyond the utterance of the word,there is so often little explanation as to what ismeant by it.

The word accountability evokes, to some, a setof lofty ideals intuitively and eminently sensible.To others, it is a normal expectation from anyoneentrusted with a responsibility. Still others see inthe word an element of confrontation.

The concept of accountability draws itsmeaning from a diverse body of literature in theareas of political science, religion, philosophy, soci-ology, management science, and public administra-tion. Each of these disciplines has somethingimportant to say on the subject. What is said, how-ever, often has a one-dimensional quality and is

A C C O U N T A B I L I T Y , P E R F O R M A N C E R E P O R T I N G , C O M P R E H E N S I V E A U D I T - A N I N T E G R A T E D P E R S P E C T I V E 3

couched in the language of that discipline. Addedtogether, an almost surrealistic picture of the sub-ject of accountability emerges.

Trying to apply simple logic to bring thepieces into focus often leads to oversimplification.In much of the management literature, account-ability is assumed, relegated to the status of a tech-nical, bureaucratic process, and then quickly dis-patched in favour of other topics. Indeed, account-ability does include process elements, but it alsoinvolves a wide range of values, beliefs, attitudes,and behaviours, which are important determinantsof the nature and endurance of accountabilityarrangements.

Part I of this book focuses on the subject ofaccountability and related issues of governance andcontrol. While it identifies the key elements of theliterature, it also attempts to go deeper, explainingthe varied interpretations those concepts are givenand how they can be connected. For the practition-er to provide professional advice and products inthe service of improving accountability, it is impor-tant to understand the subtleties, and sometimesthe contradictions, associated with the subject andwith the perspectives and strategies that governors,management and other stakeholders bring to bearin their respective accountability relationships.

The context of accountability is governance,another term that has recently gained currency.Public sector—and corporate—governance isunder stress. Ineffective governance processes arebarriers to an organization’s effective performance.An efficient and accountable management cannotensure good performance if those responsible forthe direction of the organization cannot or will notperform their duties appropriately.

In dealing with the subject of governance inthis first section of the book the purpose is not toprovide a definitive and exhaustive explanation ofthe concept; rather, it is to impart a sense of theculture and characteristics of governance. It

attempts to discover basic principles that will helpthe reader understand the concept in various situa-tions.

Chapter 1 provides a general context, explor-ing conceptual issues, offering examples of philo-sophical approaches to the subject of governance.The second chapter deals with governance in boththe public and private sectors. It describes the gov-ernment structures in Canada to provide gover-nance and some of the constraints under whichgovernment operates. Reference is also made to theform of government in other countries. Chapter 2also explains the responsibilities of directors of cor-porations whose securities are issued to the public.

Chapter 3 examines some of the conceptssurrounding both governance and accountability,and explains various terms that are used in thatcontext.

Chapter 4 discusses the concept of account-ability. This general discussion tries to give thereader an understanding of the theory and some ofthe issues involved in establishing an effectiveaccountability regime.

Chapter 5 addresses the concept of ministeri-al responsibility and its diverse interpretations.These differing views introduce the reader to thesubtleties involved in issues of this nature.

Chapter 6 describes accountability in suchcontexts as the private sector, monopolies, andthe nonprofit sector. The differences among thevarious accountability regimes highlight thecomplexity of the application of the concept ofaccountability.

Control and controls are the subject ofChapter 7. Two polar approaches to control in anorganization are discussed, as well as the operationof controls in certain circumstances.

The conclusion of Part I stresses the impor-tance of knowledge of governance and accountabil-ity issues to audit practitioners and those involvedin performance reporting. It contains a checklist of


questions that the leader of a comprehensive oughtto be able to answer in respect of the client’s gover-nance and accountability arrangements.

The Appendix uses the government ofCanada to illustrate the complex issues surround-ing control and controls, and the major attemptsto improve its systems. This emphasis on theCanadian government is not meant to exalt it overother institutions; it is used as an illustration ofcontrol and controls because it is big, complex,well-documented, and is observed and commentedupon considerably more than others. And it con-tains all the dilemmas in accountability facingother institutions.

P A R T 1 – A C C O U N T A B I L I T Y 5

S E C T I O N 1

ACCOUNTABILITY’SCONTEXT–

GOVERNANCE

P A R T I . A C C O U N T A B I L I T Y 7

THE WORD GOVERNANCE COMES FROM THE GREEK WORD

KYBERNAN, TO DIRECT THE COURSE OF A SHIP, OR TO STEER

THE SHIP. THE ROMANS BORROWED THE WORD AS GUBERNARE

AND IT EVENTUALLY CROSSED THE ENGLISH CHANNEL AS

GOVERNOR, A STEERMAN OR A PILOT. IN FRENCH, THE

RUDDER OF A SHIP IS CALLED A GOUVERNAIL. IN THE

FAMILIAR LANGUAGE OF POLITICS IN THE UNITED STATES, ONE

HEARS OCCASIONAL REFERENCE TO “STEERING THE SHIP OF

STATE,” A METAPHOR FOR GOVERNANCE.1

C H A P T E R 1

GOVERNANCE—DEFINITIONS ANDISSUES

Governance is an elegant word summarizingthe all-embracing concept of authority and control,of governing. Recently, the term has appeared withincreasing frequency in the literature of manage-ment and public administration. Recognition of itscrucial importance to effective public administra-tion and prosperous private enterprises has givengovernance its current prominence as an issue. Itsrecognition has even spawned research institutes2

and caused existing organizations to gain a betterunderstanding of the concept and to review theirgovernance practices.

Depending on the context, the word gover-nance may be used to describe a variety of notions:

• the art of governing: the concepts and methodsinvolved in governance, (for example, parlia-mentary or presidential, unitary or federated,military or civilian, authoritarian versusdemocratic, in the case of governments);

• the exercise of authority: the use of power, theprocess of governance;

• the structure of authority: the arrangement—hierarchical, bureaucratic—for governance totake place; and

• the jurisdiction: the area in which the govern-ing body has authority.

The highly generic and detailed definition ofgovernance developed by Dr. Duncan Sinclair toguide an academic medical centre is applicable toany governing body:


Governance is the exercise of authority,direction and control. It can be thought of asthe right and responsibility to determine thepurposes and principles by which an organi-zation will function and then to arrange forits management accordingly. The purposes arewhat the organization seeks to accomplish;the principles are the context, the value sys-tem, within which it operates. Governancedeals with what an organization is to do andis, therefore, highly focused on planning, set-ting goals and objectives, and on the develop-ment of policies to guide the organizationand monitor its progress toward implementa-tion of its plans. Provided that the governingbody has confidently arranged for effectivemanagement of the organization, the primaryfocus of governance should be on the long-term—the organization’s mission, values,policies, goals, objectives and, for public sec-tor institutions… its accountability under theterms of its implicit social contract.3

A DEFINITION OF GOVERNANCE

(IN THE NONPROFIT CONTEXT)

GOVERNANCE IS THE FULFILLMENT OF RESPONSIBLE OWNER-

SHIP ON BEHALF OF THE COMMUNITY.4

DEFINITION OF GOVERNANCE

(USED IN THE WORLD BANK)

FROM A GENERAL DEFINITION OF GOVERNANCE AS “THE EXER-

CISE OF AUTHORITY, CONTROL, MANAGEMENT, POWER OF

GOVERNMENT,” THE WORLD BANK HAS FORMULATED A MORE

RELEVANT DEFINITION FOR ITS PURPOSES: “THE MANNER IN

WHICH POWER IS EXERCISED IN THE MANAGEMENT OF A

COUNTRY’S ECONOMIC AND SOCIAL RESOURCES FOR

DEVELOPMENT.”5

G O V E R N A N C E V E R S U S

M A N A G E M E N T

Elaborating on his definition, DuncanSinclair contrasted governance with management:

Governance is where the buck stops. Butgovernance is something quite different frommanagement. Governors cannot and shouldnot attempt to manage organizations whosepolicies they control.

Management is the act, art or manner ofcontrolling or conducting affairs, the skillfuluse of means to accomplish a defined pur-pose. If governance has to do with what anorganization is to do, management deals withhow it does it. Management, in our complicat-ed world with all its rules and regulations,requires expertise, experience and highlydeveloped sophisticated skills. It is (or shouldbe) a very professional activity that governorshave to ensure is firmly in place to serve theneeds and execute the plans of their organiza-tion. Just as governors should not try to man-age their organizations, so should managersnot try to provide them with governance.Managers are accountable to governors.6

In Canada, the exercise of authority over pub-lic hospitals has given rise to public debate.Clarification of this issue came from Ontariowhere a ministerial committee suggested that newlegislation define governance and specify the dis-tinction between governance and management.The committee suggested that governance, in thePublic Hospitals Act, be defined as :

The exercise, by the hospital’s board ofdirectors, of authority, direction and controlover the hospital.

The fundamental responsibilities of thehospital board are to ensure that the hospitalfulfills its purposes and principles, its social

P A R T I . C H A P T E R 1 . G O V E R N A N C E – D E F I N I T I O N S A N D I S S U E S 9

contract and its objectives for patient caremanagement, quality of programs and ser-vices, fiscal integrity and long-term viability.

Management’s responsibility is to devel-op and implement the strategies and pro-grams to achieve the principles, purposes,goals and objectives set by the board.

There are shades of gray separating gov-ernance from management. The differencebetween them, however, is that the board’sauthority derives from both the hospital cor-poration and the community, whereas man-agement’s authority derives from the board towhich it is accountable. Procedures should bedeveloped to enable the hospital corporationand the community to assess the effectivenessof the hospital’s governance, and to provide abasis for public scrutiny of the hospital’s ful-fillment of its social contract.7

G O V E R N A N C E S T R U C T U R E S

Institutionalized governance is exercisedthrough a governing body that has the power ofscrutiny or direction, such as a board of directorsor governors, a regulatory body, a cabinet in itsexecutive role, a city council, or a legislative assem-bly. The form that a governance structure takesdepends on a number of factors. It may, for exam-ple, be established by legislation or honoured tradi-tion. Different structures suit different organiza-tions. In any event, how things happen within aformal governance structure will be influenced bythe human factor—the personalities, talents, anddesires of the people involved.

Michael Atkinson defines governance struc-tures as:

The informal and patterned ways inwhich different institutions and actors inter-act within particular political and administra-tive settings to develop policy goals, select

among means, cope with uncertainty andcontroversy, and foster legitimacy and supportfor policies.8

S O M E R E L A T E D C O N C E P T S

To better understand the general nature ofgovernance and the structures through which it ispractised, it is useful to touch on some key philo-sophical underpinnings.

C I V I L G OV E R N A N C E A N D L I B E R T Y

Much of the concept of civil governancederives from the theory of the manner in whichthe state relates to its population. Civil governanceis related to civil liberty.9 The contrast is naturalliberty, which implies absolute freedom to do whatone wants to do. Civil liberty also means freedomof action, but only as long as it does not harm the“common good” and does not infringe on someoneelse’s liberty. Political liberty is the freedom to par-ticipate in civil governance by voting, holding pub-lic office, and expressing one’s political opinions inpublic.

The translation of the concept of civil gover-nance into social and political arrangements pro-vides the basis of a constitution from which thelaws of the land can be promulgated. It has alsoengendered social contracts, compacts, covenants,and citizen’s charters. Let’s explore these notions,starting with the social contract.

S O C I A L C O N T R AC T

The concept of a social contract has beenarticulated by philosophers like Thomas Hobbes,John Locke, and others. The term is most oftenassociated with the eighteenth-century Frenchthinker Jean-Jacques Rousseau. He imagined astate in which free citizens, acting freely, wouldrelinquish part of their freedom to the state.

A C C O U N T A B I L I T Y , P E R F O R M A N C E R E P O R T I N G , C O M P R E H E N S I V E A U D I T - A N I N T E G R A T E D P E R S P E C T I V E1 0

Participatory democracy would characterize theprocess. In our present context, however, the termdoes not apply to a document binding two partiesin a set of mutual obligations, but rather to a polit-ical arrangement describing the relations between agovernment or a major public institution and thecommunity it serves. The social contract becomes apromise, an undertaking, a declaration, in effect an“unwritten agreement between society and thosewho seek to serve it.”10

S O C I A L C O M PAC T

People in the labour movement may relate tothe expression “social compact” as a variant of asocial contract. A compact is an agreement betweenindividuals or groups; contract essentially means thesame thing, except the term is used to describe theformal document reflecting the compact.

In the word compact was the connotation ofan agreement between factions that choose to lookat each other as equals. The term has somewhatmodified its meaning through time. The domina-tion of the government of Upper Canada by aclique of like-minded people at the turn of thenineteenth century was called the Family Compact;so was the expression describing the alliance in theearly 1700s between the Bourbon rulers of Franceand those of Spain. More recently, in 1974, theBritish Labour government promised price subsi-dies, and price and dividend controls to tradeunions, in exchange for restrained wage demands.Social compact was the name given to theproposal.

THE MAYFLOWER COMPACT, CONTRARY TO WHAT IS SO

OFTEN THOUGHT TODAY, DOES NOT REALLY SUGGEST A

SOCIAL CONTRACT OF INDEPENDENT AND EQUAL PEO-

PLE CONSTITUTING BY CONSENT THEIR OWN SOVEREIGN

AND REPRESENTATIVE GOVERNMENT FOR THE PURPOSE

OF THE PROTECTION OF THEIR OWN LIBERTIES AND

PROPERTY. ON THE CONTRARY: THE COMPACT IS ONE

FORMED AMONG PEOPLE WHO CHARACTERIZE THEIR

STATUS AS THAT OF “LOYALL SUBJECTS”, OF “OUR

DREAD SOVERAIGNE LORD, KING JAMES.” THEIR GUID-

ING PURPOSE, THEY DECLARED, WAS TWOFOLD: “THE

GLORIE OF GOD,” I.E. THE “ADVENCEMENTE OF THE

CHRISTIAN FAITH,” AND “THE HONOR OF OUR KING &

COUNTRIE.”11

COV E N A N T

In the United States, the Constitution isoften regarded as a covenant, a term borrowedfrom the Bible and sometimes deemed to be justanother word for social contract. However, notonly is the Constitution viewed as having intellec-tual origins (in the sense of getting agreement onthe most appropriate political system), but it is alsoseen as spiritually inspired and grounded in reli-gious ethics. In this sense, the framers of theConstitution are said to have brought forward acovenant, not merely a social contract.12

C I T I Z E N ’ S C H A R T E R

In the United Kingdom, a citizen’s charterwas tabled in Parliament in 1991. It focused on thepromise of raising the quality of public services andmaking them responsive to the needs of citizens. Inintroducing the charter, Prime Minister JohnMajor was specific about the intent:

How we will, for example, be introduc-ing guaranteed minimum waiting times for

P A R T I . C H A P T E R 1 . G O V E R N A N C E – D E F I N I T I O N S A N D I S S U E S 1 1

hospital operations. How we will require allschools to provide parents with reports. HowBritish Rail will be introducing new compen-sation schemes for poor service. How thosewho regulate electricity, water, gas andtelecommunications will be given the samestrong powers to insist on good service stan-dards for the customer. How we will toughenup inspection and audit, relate pay moreclosely to performance, and provide the citi-zen with more and better information.13

In the mind of the prime minister, the charteris not limited to improving the quality of service:“The citizen’s charter is about giving more powerto the Citizen,” insisting, however, that citizenshave responsibilities—as parents, as taxpayers—aswell as entitlements. There are four themes in thecharter: quality, choice, standards, and value. Thespirit of the citizen’s charter, which covers thewhole of the public service, is present in the char-ter drawn by a large number of government agen-cies. Those public declarations usually state thepromise, the commitments, the rights of the citi-zens, but some of them include the role and oblig-ations of citizens. For instance, the Job SeekersCharter insists that people applying for a job keepappointments on time.

Closing the loop in the accountability undera social contract, or an arrangement under anyother name, is not an easy step, as it generally con-sists of an agreement between the government andthe governed, which is the population at large. Thelatter cannot effectively organize to represent them-selves and negotiate with equal cohesion and con-sistency. It may well be, however, that the terms ofthe U.K. Citizen’s Charter are such that redress isavailable for mal- and nonfeasance by the govern-ment authority concerned.

The promises made by a political party that issuccessful in an election is an informal charter—a

commitment to do certain things while in office. Ifcitizens think that the government has not lived upto its implied commitment, they have to wait forthe next election to show their displeasure.

D E M O C R A C Y

Democracy is a form of government that rec-ognizes the right of all members of society to influ-ence political decisions, either directly or indirectly.

In a direct democracy, power is exerciseddirectly by the people: clearly, this is possible onlywhere the population is small. In representativedemocracies—which is what modern democraciesare—political decisions are taken by citizens elect-ed by the people to be their representatives.

The central institution of a modern democra-cy is a representative legislature in which decisions


THE LANGUAGE OF DEMOCRACY

WHAT, SIR, IS THE GENIUS OF DEMOCRACY? LET ME READ

THAT CLAUSE OF THE BILL OF RIGHTS OF VIRGINIA, WHICH

RELATES TO THIS: 3D CL. “THAT GOVERNMENT IS OR OUGHT

TO BE INSTITUTED FOR THE COMMON BENEFIT, PROTECTION,

AND SECURITY OF THE PEOPLE, NATION, OR COMMUNITY:

OF ALL THE VARIOUS MODES AND FORMS OF GOVERNMENT,

THAT IS BEST WHICH IS CAPABLE OF PRODUCING THE

GREATEST DEGREE OF HAPPINESS AND SAFETY, AND IS MOST

EFFECTUALLY SECURED AGAINST THE DANGER OF MAL-

ADMINISTRATION, AND THAT WHENEVER ANY GOVERNMENT

SHALL BE FOUND INADEQUATE, OR CONTRARY TO THESE

PURPOSES, A MAJORITY OF THE COMMUNITY HATH AN

UNDUBITABLE, UNALIENABLE AND INDEFEASIBLE RIGHT TO

REFORM, ALTER, ABOLISH IT, IN SUCH MANNER AS SHALL BE

JUDGED MOST CONDUCIVE TO THE PUBLIC WEAL.” THIS, SIR,

IS THE LANGUAGE OF DEMOCRACY: THAT A MAJORITY OF THE

COMMUNITY HAVE A RIGHT TO ALTER THEIR GOVERNMENT

WHEN FOUND TO BE OPPRESSIVE.14

P A T R I C K H E N R Y , 1 7 8 8

are taken by majority vote. The characteristics ofsuch a democracy are regular elections, free choiceof candidates, universal suffrage, freedom to orga-nize rival political parties, independence of thejudiciary, freedom of speech, freedom of the press,and respect for civil liberties and minority rights.

T H E N A T U R E O F A C O N S T I T U T I O N

A constitution defines the fundamental valuesand rules of a society.

THE ESSENCE OF A CONSTITUTION

[A CONSTITUTION] IS CONCERNED WITH WHAT IS DONE TO MAKE

SOCIETY INTO A PROPERLY STRUCTURED, CONTINUOUS LIVING BODY,

SO THAT THE POLITICAL ACTION OF WHICH THAT SOCIETY IS

CAPABLE CAN BE EFFICIENTLY AND EFFECTIVELY CONDUCTED.

MACHINERY, YES. BUT ALSO THOUGHT, THE DOCTRINE, THE

TEACHING, THE CONVENTIONAL NOTIONS. WHAT DOES THE SOCIETY

THINK ITS GOVERNMENT IS, HOW DOES IT TREAT IT, WHAT DOES IT

DO TO AMEND IT? WHAT FORMS OF CHANGE ARE POSSIBLE, WHAT

REFORMS...?15

[T]HE CONSTITUTION OF AN ORGANIZATION IS ITS FUNDAMENTAL

NORMATIVE STRUCTURE… A SET OF AGREEMENTS AND UNDER-

STANDINGS WHICH DEFINE THE LIMITS AND GOALS OF THE GROUP

(COLLECTIVITY) AS WELL AS THE RESPONSIBILITIES AND RIGHTS OF

THE PARTICIPANTS STANDING IN DIFFERENT RELATIONS TO IT.16

A constitution is often conceived of as a char-ter, a declaration or text outlining the nature of agovernment or other organization. A country’s con-stitution specifies how power will be shared amongthe people, the legislative and executive bodies, andthe judiciary. It has legal precedence over all otherlaws of the land. It is the basic law used to inter-pret all other laws.

Not all countries have written constitutions.Great Britain is the leading example of a countrywith an unwritten constitution. But it is unwrittenonly in the sense that there is no single documentreferred to as the constitution:

There are in fact various laws of consti-tutional significance, and there is a great cor-pus of authoritative constitutional writing inwhich scholars and lawyers discuss the consti-tution as it is, and as they think it ought tobe. It remains true, however, that a numberof important constitutional practices are fol-lowed with rigidity although they are nothingmore than conventions. The fact that Britaindoes not have a formally written constitutionis thus not of a great significance for the prac-tice of government and politics. A bindingauthoritative constitution does exist andpoliticians and public administrators are noless constitutional in their behavior than theircounterparts in those countries which dohave a formally designated document.17

ON THE NATURE OF A CONSTITUTION

THE CONSTITUTION APPEARS TO BE A RESTRAINT WHEN IN

FACT IT IS NONE AT ALL… SIR, I WILL NOT DECLAIM, AND SAY

ALL MEN ARE DISHONEST; BUT I THINK THAT, IN FORMING A

CONSTITUTION, IF WE PRESUME THIS, WE SHALL BE ON THE

SAFEST SIDE… MANY MILLIONS OF MONEY HAVE BEEN PUT

INTO THE HANDS OF GOVERNMENT, WHICH HAVE NEVER BEEN

ACCOUNTED FOR, THE ACCOUNTS ARE NOT SETTLED YET, AND

HEAVEN ONLY KNOWS WHEN THEY WILL BE.18

M E L A N C T O N S M I T H

A N T I - F E D E R A L I S T , 1 7 8 8

Concepts such as rights and responsibilities,power sharing, representation, participation andinfluence, which, as noted above, are rooted in

P A R T I . C H A P T E R 1 . G O V E R N A N C E – D E F I N I T I O N S A N D I S S U E S 1 3

such notions as social contract, democracy, andconstitution, are very much a part of the gover-nance equation in relation to our public- and pri-vate-sector institutions. Earlier, key definitionaland structural aspects were highlighted. Let us nowexamine a set of characteristics that, taken together,provide a framework for looking at the quality ofgovernance. The discussion will return to theseissues as it delves further into the subjects of gover-nance and accountability and the arrangementsthat bind them.

E S S E N T I A L C H A R A C T E R I S -

T I C S O F G O O D G O V E R N A N C E

Governance can be either good or bad, assidu-ous or negligent. Good governance displays a desireto move away from exercise of authority throughcontrols which may be effective but ephemeral,towards exercise of leadership which is at onceeffective, inspiring, continuous, and lasting.

To govern well implies the application offoresight, knowledge, understanding, and judg-ment, as well as considerable trust. Affirmation of

power and imposition of rigorous controls are leastlikely to be used in enlightened governance. As acorollary, good governance is very demanding ofaccountability.

But what is good governance and how will weknow we have it? The following are key characteris-tics of good governance.19 We know we have goodgovernance when governing bodies:

• comprise people with necessary knowledge,ability, and commitment to fulfill theirresponsibilities;

• understand their purposes and whose intereststhey represent;

• understand the objectives and strategies of theorganizations they govern;

• understand what constitutes reasonable infor-mation for good governance and obtain it;

• once informed, are prepared to ensure thatthe organization’s objectives are met and thatperformance is satisfactory; and

• fulfill their accountability obligations to thosewhose interests they represent by reporting ontheir organization’s performance.


1 SEE RICHARD LEDERER, “LANGUAGE-HATS IN THE RING, (THE PARLANCE OF POLITICS),” DELTA AIRLINES SKY MAGAZINE, NOVEMBER 1992, 128.2 THE INSTITUTE ON GOVERNANCE IN OTTAWA AND THE INSTITUTE OF CORPORATE DIRECTORS IN TORONTO, FOR INSTANCE.3 DUNCAN SINCLAIR, DEAN OF MEDICINE, QUEEN’S UNIVERSITY, “GOVERNANCE OF THE ACADEMIC MEDICAL CENTRE,” IN ACMC FORUM, VOL. XX, NO. 4, 12.4 R. J. UMBDENSTOCK, W. M. HAGEMAN AND B. AMNDSON, “THE FIVE CRITICAL AREAS FOR EFFECTIVE GOVERNANCE OF NOT-FOR-PROFIT HOSPITALS,” HOSPITALS AND HEALTH SERVICES

ADMINISTRATION, NO. 35, 1990, 481-92.5 THE WORLD BANK, GOVERNANCE AND DEVELOPMENT (WASHINGTON, D.C.: THE WORLD BANK, 1992), 3.6 SINCLAIR, “GOVERNANCE OF THE ACADEMIC MEDICAL CENTRE,” 12.7 ONTARIO MINISTRY OF HEALTH, INTO THE 21ST CENTURY, ONTARIO PUBLIC HOSPITALS, REPORT OF THE STEERING COMMITTEE, PUBLIC HOSPITALS ACT REVIEW (TORONTO: ONTARIO

MINISTRY OF HEALTH, FEBRUARY 1992), ES-3.8 MICHAEL M. ATKINSON, GOVERNING CANADA, INSTITUTIONS AND PUBLIC POLICY (TORONTO: HARCOURT BRACE JOVANOVICH CANADA INC., 1993), GLOSSARY.9 SEE MICHAEL G. KAMMEN, THE ORIGINS OF THE AMERICAN CONSTITUTION (TORONTO: PENGUIN BOOKS, 1986), IX.10 THIS CHARACTERIZATION OF THE SOCIAL CONTRACT IS FROM WHITE, KERR AND J. E. CONNELLY, “THE MEDICAL SCHOOL’S MISSION AND THE POPULATION’S HEALTH,” ANNALS OF

INTERNAL MEDICINE, DECEMBER 1991, VOL. 115, NO. 12, AS CITED IN UNIVERSITY OF OTTAWA HEALTH SCIENCES COMPLEX STRATEGIC PLANNING PROCESS DISCUSSION PAPER, HEALTHY

DIRECTIONS (OTTAWA: UNIVERSITY OF OTTAWA HEALTH SCIENCES COMPLEX, 1992), 53.11 THOMAS L. PANGLE, “THE CONSTITUTION’S HUMAN VISION,” THE PUBLIC INTEREST, NO. 86, WINTER 1987, 79.12 IRVING KRISTOL, “THE SPIRIT OF ‘87,” THE PUBLIC INTEREST, IBID., 5.13 TREASURY, THE CITIZEN’S CHARTER: RAISING THE STANDARD (LONDON: HMSO, CM 1599, JULY 1991), 1.14 SPEECH OF PATRICK HENRY IN THE VIRGINIA RATIFYING CONVENTION, JUNE 5, 1788, AS CITED IN THE AMERICAN CONSTITUTION, FOR AND AGAINST, THE FEDERALIST AND ANTI-

FEDERALIST PAPERS, J. R. POLE, ED. (NEW YORK: HILL AND WANG, 1987), 119.15 G. R. ELTON, THE FUTURE OF THE PAST (CAMBRIDGE: CAMBRIDGE UNIVERSITY PRESS, 1968), 24-25, AS CITED IN PETER HENNESSY, WHITEHALL (NEW YORK: THE FREE PRESS, 1989), 11-

12.16 M. N. ZALD, POWER IN ORGANIZATIONS (NASHVILLE: VANDERBILT UNIVERSITY PRESS, 1970), 225.17 J. E. KINGDOM, ED., THE CIVIL SERVICE IN LIBERAL DEMOCRACIES, AN INTRODUCTORY SURVEY, (LONDON: ROUTLEDGE, 1990), 11. 18 THE AMERICAN CONSTITUTION, BACK PAGE.19 CCAF, IN SEARCH OF EFFECTIVE GOVERNANCE, VIDEO SCRIPT, 1994, 15.

FEDERATION VERSUS CONFEDERATION

THE SEMANTICS OF CONSTITUTIONAL ARRANGEMENTS CAN BE

TRICKY. MANY AUTHORITIES MAKE A DISTINCTION BETWEEN A

FEDERATION AND A CONFEDERATION. IN A FEDERATION, IN

THEIR VIEW, THE COMMON GOVERNMENT HAS ASCENDENCY

OVER THE GOVERNMENTS OF THE STATES COMPRISING IT; IT IS

SUPREME. IN A CONFEDERATION, THE EMPHASIS IS ON THE

SOVEREIGNTY AND AUTONOMY OF EACH CONSTITUENT STATE

AND IS USUALLY FORMED TO ACHIEVE EFFICIENCY IN EXTER-

NAL PURPOSES—DEFENSE, INTERNATIONAL TRADE, EXTERNAL

AFFAIRS. THE FEDERAL SYSTEM IN CANADA HAS ENOUGH OF

BOTH TO SATISFY LARGE ELEMENTS OF THE TWO DEFINITIONS.

THE FATHERS OF CONFEDERATION HAVE GIVEN US A FEDERAL

GOVERNMENT, AND THE PROVINCES, OVER 125 YEARS, HAVE

DISPLAYED DIFFERENT POINTS OF VIEW ABOUT THEIR UNDER-

STANDING OF WHAT EXACTLY CANADA IS.

NATIONAL VERSUS FEDERAL

WE MIGHT ADD A FATHER OF THE U.S. CONSTITUTION AND

THE FOURTH PRESIDENT OF THE UNITED STATES, JAMES

MADISON’S DISTINCTION 20 BETWEEN NATIONAL AND FEDERAL

WHICH, IN A WAY, EXPLAINS WHY THE MEECH LAKE AGREE-

MENT HAD TO BE RATIFIED BY EVERY PROVINCE REGARDLESS

OF ITS POPULATION INSTEAD OF BY THE MAJORITY OF THE

POPULATION IN THE COUNTRY. IF THE CONSTITUTION IS

NATIONAL IN CHARACTER, THE SUPREME AND ULTIMATE

AUTHORITY WOULD RESIDE IN THE MAJORITY OF THE PEOPLE

OF CANADA; IF IT IS FEDERAL, THE CONCURRENCE OF EACH

PROVINCE WOULD BE ESSENTIAL TO EVERY ALTERATION OF THE

CONSTITUTION THAT WOULD BE BINDING ON ALL. THIS

IMPLIES A DIFFERENT UNDERSTANDING OF FEDERAL THAN IS

GIVEN ABOVE.

C H A P T E R 2

GOVERNANCE INCANADA

Canada’s constitution contains both writtenand unwritten elements. We inherited from GreatBritain an accumulation of constitutional decisions,precedents, and practices defining governmentauthority. The British North America Act of 1867(BNA Act) granted Canada its independence and isthe basis for the written part of the constitution. Ithas been formally renamed the Constitution Act,1867, and has been amended several times, mostrecently by the Constitution Act, 1982, which con-tains the Canadian Charter of Rights and Freedoms.An important part of the Canadian constitutiondivides the totality of governmental powers betweenthe provinces and the federal government, a pointof controversy over the decades. It also containsprovisions for such matters as periodic elections (atleast every five years) that are essential to ensure acontinuing democracy.

The Canadian constitution sets out broadlythe nature of our government. It draws heavily onthe Westminster (British) model, and the princi-ples of parliamentary government apply equally tothe federal government and the provinces. Thepositions of prime minister and premier do notappear in the written constitution, nor does theterm cabinet. They are elements of the unwrittenpart of the constitution, but no less important tothe essential working of Canadian governments.

T H E F O R M O F G O V E R N M E N T

I N C A N A D A

The Crown is the sovereign authority—thatis, head of state—in Canada’s system of govern-

P A R T I . S E C T I O N 1 . A C C O U N T A B I L I T Y ’ S C O N T E X T — G O V E R N A N C E 1 5

ment. The Crown is represented at the federal levelby a governor-general, who as a public figure isvery active, but within the system of governmentplays a largely ceremonial role. The governor-gen-eral takes advice from the Privy Council ofCanada, more precisely from its operational part,the cabinet headed by the prime minister. At theprovincial level, a lieutenant-governor representsthe Crown and takes advice from the provincialexecutive council.

CANADA HAS ADOPTED FROM GREAT BRITAIN THE

WESTMINSTER MODEL OF PARLIAMENTARY GOVERN-

MENT. IT HAS A PARLIAMENT IN THE FORM OF A BICAM-

ERAL ASSEMBLY, CONSISTING OF AN APPOINTED

SENATE,21 AND AN ELECTED HOUSE OF COMMONS. THE

FORMAL ROLE OF PARLIAMENT IS THAT OF LEGISLATION,

BUT MOST LEGISLATION IS FRAMED BY THE CABINET,

WHICH IS ABLE TO RELY ON THE SUPPORT OF ITS PARTY

MAJORITY IN THE HOUSE OF COMMONS, SO THAT THE

EFFECTIVE ROLE OF THE HOUSE IS ONE OF SCRUTINIZ-

ING THE EXECUTIVE, AND PROVIDING A FORUM FOR

POLITICAL DEBATE.22

THE MOST IMPORTANT DETERMINANTS OF CONTROL

AND USE OF POWER IN THE CANADIAN PARLIAMENTARY

SYSTEM ARE THE POLITICAL PARTIES. ELECTIONS ARE

MORE A MATTER OF VOTERS CHOOSING BETWEEN PAR-

TIES AND PARTY LEADERS THAN BETWEEN INDIVIDUAL

CANDIDATES. THE WINNING PARTY BECOMES THE GOV-

ERNMENT, WITH A MONOPOLY OVER EXECUTIVE POWER

AND DOMINATION OF PARLIAMENT. WITHIN THE HOUSE

OF COMMONS, THE BASIC STRUCTURE OF PROCEEDINGS

IS THE ADVERSARIAL FORMAT OF CONTEST AND DEBATE

BETWEEN THE GOVERNMENT AND OPPOSITION PARTIES.23

T H E P R I M E M I N I S T E R A N D T H E C A B I N E T

The prime minister and the cabinet of minis-ters are not directly elected to those positions bythe people, though they are usually popularly elect-ed as members of the House of Commons. Theprime minister is usually the leader of the politicalparty that won the majority of seats in the last elec-tion. In turn, the cabinet is chosen by the primeminister from among his or her supporters in theHouse of Commons and the Senate.24

Executive power is really in the hands of cabi-net led by the prime minister. Interestingly, “nei-ther of these are actually mentioned in the BNAAct. The origins of the Canadian cabinet lie in thePrivy Council, a body formally charged with thefunction of advising the Governor-General. Oncechosen, cabinet members formally gain theirauthority by being sworn in as members of thisCouncil. Though remaining privy councillors forlife, but they lose their executive authority oncethey cease to be members of the cabinet.”25

THE WORD EXECUTIVE

THE WORD EXECUTIVE IS DERIVED FROM THE LATIN VERB

EXSEQUI, TO FOLLOW THROUGH OR TO CARRY OUT. IN THE

PRIVATE SECTOR, CORPORATE EXECUTIVES HEADED BY A CHIEF

EXECUTIVE OFFICER (CEO) CARRY OUT THE WILL OF THE

BOARD OF DIRECTORS. IN PRIVATE FAMILY MATTERS, EXECU-

TORS OF ESTATES FOLLOW THE WISHES OF DECEASED TESTA-

TORS. IN OUR GOVERNMENT, THE WORD EXECUTIVE REFERS

TO THE CABINET AND BUREAUCRACY: THE EXECUTIVE ARM OF

GOVERNMENT. THE MOST SENIOR NONELECTED OFFICIAL IN

THE PRIVY COUNCIL OFFICE IS THE CLERK OF THE PRIVY

COUNCIL AND HOLDS THE SIMULTANEOUS TITLE SECRETARY

TO THE CABINET. FARTHER DOWN THE HIERARCHY ARE OFFI-

CIALS WITH THE TITLE EXECUTIVE DIRECTOR.


By convention, the great majority of cabinetmembers are chosen from the House of Commons.The reason is that the ministers can then beaccountable to the elected chamber; the Senate isconsidered an inferior forum in which to holdministers to account.

COLLECTIVE RESPONSIBILITY

IN CANADA’S SYSTEM OF GOVERNMENT, CABINET MINISTERS

ARE NOT ONLY ACCOUNTABLE FOR THEIR OWN PORTFOLIOS,

THEY ALSO HAVE A COLLECTIVE RESPONSIBILITY AS MEMBERS

OF THE GOVERNMENT. THE CABINET ASSUMES RESPONSIBILITY

FOR THE POLICIES AND PERFORMANCE OF ITS GOVERNMENT.

INDIVIDUAL MINISTERS HAVE THE TASK OF MESHING THEIR

INDIVIDUAL ACCOUNTABILITY WITH RESPECT TO THEIR OWN

DEPARTMENTAL POLICIES AND PROGRAMS WITH THEIR COL-

LECTIVE RESPONSIBILITIES AS MINISTERS OF THE CROWN IN

SUPPORT OF THEIR OWN GOVERNMENT.

The executive head, the prime minister, hasconsiderable power within the system. The formalconstraints inherent in the principle of the separationof powers that can limit the actions of the presidentof the United States are absent, since the prime min-ister remains a member of the House of Commons,wherein his party remains in control of the majority.On all important issues, party discipline is strict;members vote as their leaders tell them to.

THE PRIME MINISTER IS GIVEN THE HONORIFIC “RIGHT

HONOURABLE”; OTHER FEDERAL CABINET MINISTERS ARE

“HONOURABLE.” THEY KEEP THESE TITLES FOR LIFE.

MEMBERS OF PROVINCIAL CABINETS ARE MEMBERS OF

THE EXECUTIVE COUNCIL AND KEEP THE TITLE

“HONOURABLE” ONLY WHILE THEY HOLD OFFICE. ON

OCCASION, PROVINCIAL POLITICIANS ARE INDUCTED INTO

THE PRIVY COUNCIL OF CANADA AND ARE

“HONOURABLE” FOR AS LONG AS THEY LIVE THEREAFTER.

Although tradition has it that the premier isprimes inter pares—first among equals—within thecabinet, the prime minister’s authority is absolute.Moreover, the authority to make top public serviceappointments (deputy ministers and other order-in-council appointments) represents a power akin

P A R T I . C H A P T E R 2 . G O V E R N A N C E I N C A N A D A 1 7

MINISTERIAL RESPONSIBILITY: THE DOCTRINE

MINISTERIAL RESPONSIBILITY, ALONG WITH THE FUSION OF

THE EXECUTIVE AND LEGISLATIVE BRANCHES, ARE DISTIN-

GUISHING FEATURES OF RESPONSIBLE GOVERNMENT. THE

RULES RELATING TO THESE FEATURES ARE NOT SET DOWN IN

THE CONSTITUTION. THEY ARE GOVERNED BY CONVENTION,

PRECEDENT AND COMMON SENSE. THERE IS NO SINGLE DEFIN-

ITION OF MINISTERIAL RESPONSIBILITY; THERE ARE, IN FACT,

THREE PARTS TO THE DOCTRINE.

FIRST, THERE IS THE RESPONSIBILITY OF A MINISTER TO THE

QUEEN OR THE GOVERNOR GENERAL; THIS IS OFTEN OVER-

LOOKED, BUT IT IS BASIC TO OUR CONSTITUTIONAL ORDER.

GOVERNMENTS ARE NOT ELECTED BUT APPOINTED, AND MIN-

ISTERS SERVE NOT FOR A TERM, BUT UNTIL THEY DIE, RESIGN,

OR ARE DISMISSED.

SECOND, THERE IS THE INDIVIDUAL RESPONSIBILITY OF A MIN-

ISTER TO THE HOUSE. THIS REVOLVES AROUND THE QUESTION

OF WHEN A MINISTER SHOULD OFFER HIS OR HER RESIGNA-

TION, AND WHEN IT SHOULD BE ACCEPTED OR ASKED FOR.

THE ANSWER SEEMS TO TURN ON THE PERSONAL RELATION-

SHIP BETWEEN THE MINISTER AND THE PRIME MINISTER. THE

PRINCIPLE IS ACCEPTED, HOWEVER, THAT WHERE THERE IS A

PERSONAL CULPABILITY ON THE PART OF A MINISTER, IN THE

FORM OF PRIVATE OR PUBLIC CONDUCT REGARDED AS UNBE-

COMING AND UNWORTHY OF A MINISTER OF THE CROWN, THE

MINISTER SHOULD RESIGN.

THE THIRD RESPONSIBILITY IS THAT OF THE MINISTRY COL-

LECTIVELY TO THE HOUSE. IF THE CONFIDENCE OF THE

HOUSE IS LOST, IT SPELLS THE END FOR THE MINISTRY UNLESS

THE GOVERNMENT IS GRANTED A DISSOLUTION AND IS SUS-

TAINED BY THE ELECTORATE.26

to that of the president of the United States, butfree of the need to obtain legislative approval ofappointments. “The main constraints are probablypolitical rather than constitutional, deriving fromthe balance of support within his party, from pub-lic opinion and from the activities of interestgroups.”27

Much of the working of the cabinet isthrough a number of committees. As of 1994,there were four such committees: EconomicDevelopment Policy, Social Development Policy,Special Committee of Council, and TreasuryBoard.28 Previously, there had been a much largernumber of committees, coordinated by other, over-arching committees.

Ministers, their staffs, and many senior publicservants are constrained by conflict-of-interestguidelines designed to ensure probity in the con-duct of public business. A formal structure exists tomonitor compliance with these standards.

C E N T R A L AG E N C I E S

Both the provincial and federal governmentsuse central agencies to oversee and bring cohesionto their bureaucracies. These agencies exercise thewill of the government as a whole, rather than theinterests of particular departments. They go by var-ious names. Probably the most central of thesebodies is the one that serves the cabinet. Others,such as treasury or management boards are con-cerned, among other things, with financial alloca-tions and procedures. A civil or public service com-mission deals with personnel matters and is typical-ly charged with ensuring the integrity of the meritsystem of hiring and promotion within the bureau-cracy. The degree to which authority in the fieldsof interest of these central agencies is delegated tothe operating departments and agencies variesgreatly from jurisdiction to jurisdiction.


Monarchy

Governor-General

EXECUTIVE

Prime Minister

Cabinet

Departments

JUDICIARY

Supreme Court of Canada

LEGISLATURE

House of CommonsSenate

Privy Council Office

Federal Courtof Canada

Treasury Board

Tax Court of Canada

Provincial/TerritorialCourt of Appeal

Provincial/TerritorialDistrict Supreme

Courts

Monarchy

Governor-General

EXECUTIVE

Prime Minister

Cabinet

Departments

JUDICIARY

Supreme Court of Canada

LEGISLATURE

House of CommonsSenate

Privy Council Office

Federal Courtof Canada

Treasury Board

Tax Court of Canada

Provincial/TerritorialCourt of Appeal

Provincial/TerritorialDistrict Supreme

Courts

F I G U R E 2 . 1 :

G O V E R N M E N T O F C A N A D A

SOURCE: THE 1996 CANADIAN GLOBAL ALMANAC (TORONTO: MACMILLAN CANADA, 1996).

THE CABINET IS SERVICED BY A SECRETARIAT KNOWN AS

THE PRIVY COUNCIL OFFICE. THIS SERVES TO LINK THE

EXECUTIVE HEAD WITH THE BUREAUCRACY THROUGH A

TWO-WAY TRANSMISSION OF INFORMATION, AND THE

COMMUNICATION TO THE APPROPRIATE DEPARTMENT OF

CABINET DECISIONS REQUIRING BUREAUCRATIC ACTION.29

TO SOME EXTENT, THE PRIVY COUNCIL OFFICE CAN BE

SEEN AS THE INSTRUMENT OF THE PRIME MINISTER, IN

THAT IT SHARES HIS OVERARCHING, COORDINATING

FUNCTION. THROUGH THE OFFICE HE IS ALSO ABLE TO

MANAGE THE CABINET’S BUSINESS AND DECIDE THE

ORDER AND THE CONTENT OF THE AGENDA.30

All jurisdictions also have a number of spe-cialized offices that enjoy government-wide juris-diction, but are not themselves arms of the govern-ment. These include the auditors, privacy commis-sioners, and ombudsmen. These officeholders typi-cally report to the legislature rather than to a min-ister and represent means through which the legis-lators hold the government accountable for itsadministration.

G OV E R N M E N T D E PA R T M E N T S

The federal and provincial public servicescomprise departments and agencies that togethershare the totality of the bureaucratic responsibility.The allocation of responsibilities among depart-ments is largely functional (for example, transport,communications, municipal affairs).

The essential feature of government depart-ments is that they are answerable to the legislaturethrough the minister. The organization chart of thedepartment is hierarchical, with each level ofresponsibility reporting to the one above in a chainof command and responsibility rising through tothe deputy minister and culminating in the minis-ter’s relationship with the House.31

Entry and promotion within the bureaucracy(except at the highest levels) is governed by themerit system, featuring competition for positionswith provision for appeal. A large percentage ofpublic servants belong to unions, through whichthey bargain with the government on salary issuesand other conditions of service.

Senior public servants have substantial influ-ence. They are responsible for overseeing the deliv-ery of government services and controlling thebureaucracy. As senior advisers, they are responsiblefor developing policy for recommendation to theirministers. Since most of these people are careercivil servants, they often have more expertise con-cerning the issues facing their departments, and theconstituencies they serve, than do the ministers,who typically serve only a few years in any oneportfolio. Some observers argue that the influenceof top bureaucrats increases when cabinets arereduced in size, as has been the trend. This is


POLICY-MAKING IN CANADA CONSISTS OF NOT ONE BUT

MANY PROCESSES, EACH WITH ITS PARTICULAR TRAITS.

BUT THE MOST IMPORTANT PROCESSES, INCLUDING

RATIONAL POLICY-MAKING, BUREAUCRATIC PLURALISM,

AND EXECUTIVE FEDERALISM, ARE EXECUTIVE-CEN-

TERED. EXECUTIVE-CENTREDNESS LEADS TO DIFFICUL-

TIES IN BUILDING CONSENT. POLICY-MAKING IS PRI-

VATE, BELOW THE LEVEL OF PUBLIC VISIBILITY AND

OFTEN POLICIES, WHEN THEY EMERGE, ARE SPRUNG

FULL-BLOWN ON A SURPRISED, UNSUSPECTING AND

SOMETIMES NON-TOO-PLEASED PUBLIC. IT IS HARD TO

LINK THE INSIDE, PRIVATE SPHERE OF GOVERNMENT

WITH THE OUTSIDE PUBLIC WORLD OF POLITICS.

EXECUTIVE-CENTRED POLICY-MAKING DOES NOT LEAD

TO THE MOBILIZATION OF CONSENT WHILE POLICIES

ARE BEING DEVELOPED. PARLIAMENT IS UNIMPORTANT.

IT RATIFIES AND AUTHORIZES DECISIONS WORKED OUT

ELSEWHERE.32

because ministers have wider responsibilities thanbefore and must rely more heavily on the advice oftheir senior officials. The issue of how the publicservice should be held accountable is one arousingcontroversy, as is discussed in the next section.

G OV E R N M E N T AG E N C I E S

Outside the departmental structure, in boththe provinces and federal government, there is a sub-stantial group of agencies referred to as Crown cor-porations serving at once a commercial and publicpolicy purpose (for example, Export DevelopmentCorporation; Ontario Hydro). They ultimatelyreport to a minister, but they enjoy varying degreesof freedom from ministerial and legislative control.Typically, Crown corporations have a governingbody in the form of a board of directors drawn fromthe public and, where appropriate, includes a seniorofficial from the department which has a key policyinterest in the business of the agency.

Thus it can be seen that the Canadianpolitical culture permits a high degree ofautonomy to the agencies of public adminis-tration, resulting in a complex and variedstructure. However, the civil service, with itstight level of control and accountability,remains central to the whole system, playing adominant role in key policy areas and remain-ing close to the processes of central resourceallocation and policy-making. Clearly the highdegree of political control afforded by the gov-ernment department remains valued, thoughthe presence of a large quasi-autonomous sec-tor helps to create an environment in whichsuch control is constantly questioned.33

Moreover, numerous regulatory or quasi-judi-cial agencies are given substantial autonomy toadminister the law. These bodies regulate trans-portation and communications, settle disputes con-

cerning municipal bylaws, determine the status ofrefugee claimants, decide on compensation for vic-tims of violent crime, and perform a number ofother functions. They enjoy freedom from directpolitical oversight because of the nature of theirresponsibilities. They are specialized in a way thatcourts could not be and rely on the expertise oftheir members and staffs to carry out the broadintent of the legislation that gave them life.Usually, appeals can be made to the courts fromdecisions of these bodies.

T H E J U D I C I A RY

Disputes involving the interpretation ofParliament’s laws are handled by the Federal Courtof Canada, which has both a trial and appeals divi-sion. Criminal law and laws under provincial juris-diction are dealt with by the provincial courts.Each province determines the structure of its judi-ciary, although many provincial court judges areappointed by the federal cabinet. The final court ofappeal for all Canadian legal disputes is theSupreme Court of Canada.

Although appointed by governments, judgesbecome independent of political influence onappointment. Typically, recommendations for judi-cial appointments are made by independent com-mittees that include members of both the benchand the bar; they often include lay membership.Judges may serve until retirement age. Althoughthere are provisions for removing judges fromoffice, the process is difficult and, if instituted,invariably attracts widespread public attention.

LO C A L G OV E R N M E N T

Under the Canadian constitution, eachprovince has authority to determine the organiza-tion of local governments within its borders.Practice varies, and provinces have devised a wide


range of local bodies to fulfill responsibilities thattheir legislatures have delegated to them. Regionalgovernments, municipalities (usually distinguishingamong cities, towns, villages, improvement dis-tricts, and so on), school boards, hydro and otherutility commissions, and health-planning areas aresome of the more common instruments thatprovinces have chosen to implement policy anddeliver services. All these bodies are creatures ofprovincial legislatures, which have the legal power(though often not the political support) to change,abolish, amalgamate, or rename them.

Some of these local bodies, such as municipalcouncils and most school boards, are elected; oth-ers are appointed. Some have the power to levytaxes; others do not. A few positions on these gov-erning bodies are considered to be full-time andare paid accordingly; others receive no remunera-tion. Whatever their nature, taken together, theselocal bodies comprise tens of thousands ofCanadians who, in one way or another, feel calledupon to serve their communities through localgovernment.

N O N G OV E R N M E N TA L O R G A N I Z AT I O N S

In addition to this, but at some remove fromgovernment, are the myriad agencies, funded inwhole or part by public money, that provide ser-vices in such fields as recreation, health, social ser-vices, and the like. These organizations are guidedby citizens who provide the governance essential totheir effective and continuing existence.

Typically, these entities are the creations ofprivate citizens interested in a particular policy orservice. They often represent examples of powersharing within society, with organizations like theRed Cross blood service and children’s aid societiescarrying out mandates established in legislationand supervised by government departments.Governments sometimes use these organizations to

deliver services in the belief that the involvementof local citizens will provide a more relevant service.

Recently, governments have assumed increas-ing control over organizations such as hospitals forwhich they are footing the major part of—or allof—the bill. Governments with differing politicalphilosophies may take different approaches to theseorganizations, so that there may be variationsbetween jurisdictions and transitions over timewithin jurisdictions.

T H E R O L E O F E L E C T E D

R E P R E S E N T A T I V E S

Acting as the elected representative of a con-stituency is not a simple task. The following issuehas never been entirely resolved with respect torepresentatives and their electorates.

T H E B U R K E - J E F F E R S O N Q U E S T I O N

There are on the one hand the“Burkeans” adhering to the views of theBritish Parliamentarian Edmund Burke whoclaimed not to be a mere delegate of his elec-tors but a free-thinking member ofParliament who owed them the independentexercise of his judgment.

On the other hand are the“Jeffersonians.” The American ThomasJefferson asserted, in his thorough-goingadvocacy of democracy, that elected represen-tatives must remain directly accountable tothose who elected them.34

If Canadians have traditionally beenBurkeans, inclined to invest power andresponsibility in their elected representatives,they appear now to be becomingJeffersonians, constitutionally distrustful ofgovernment and insistent that their represen-tatives respond more sensitively and directlyto the voice of the people.35


In recent years, Canadians have increasinglyused informal meetings to discuss public issues.The public consultations that preceded the nation-al referendum of 1992 and the Ontario treasurer’s1994 pre-budget public meetings are but two ofmany examples. These town-hall meetings, as theyare often called, reflect citizens’ desire to influencethe public debate directly, instead of relying solelyon their elected representatives. Some observers havedubbed this phenomenon participatory democracy.

The trend toward achieving a louder publicvoice has strengthened the position of those whoargue for greater independence in the voting powerof individual members of legislatures; they argue fora weakening of the party discipline that many haveconsidered essential to the operation of parliamen-tary democracy. Proponents of this change want tosee fewer votes being considered challenges to con-fidence in the government and to allow members tovote according to the views of their constituents orthe dictates of their individual consciences.

JEFFERSONIANS

WESTERN CANADIANS WERE VERY MUCH JEFFERSONIANS IN

THE EARLY PART OF THE CENTURY. MEMBERS ELECTED TO

PROVINCIAL LEGISLATURES SIGNED PLEDGES TO VOTE A PAR-

TICULAR WAY ON SPECIFIC ISSUES AS INSTRUCTED BY CON-

STITUENTS. SIMILAR PLEDGES WERE EXTRACTED FROM

PROGRESSIVE PARTY MEMBERS ELECTED FROM WESTERN

CANADA TO THE HOUSE OF COMMONS IN 1921.36

Elected members occasionally face a dilemmawhen they want to vote according to their con-science or according to their constituents’ wishes,against their party’s position. Reelection dependsheavily on staying in touch and favour with the vot-ers back home; advancement, however, depends ongood standing with the party leadership. To whom,then, is the elected member primarily accountable?

I N F L U E N C E S O N P O L I C Y I N A

D E M O C R A C Y

As explained above, Canada is a federal, con-stitutional democracy. It is also a pluralistic society,encompassing many interests and forces that havean effect on the way our governments function.

S P E C I A L I N T E R E S T G R O U P S

Not only do elected members—and govern-ments—have to keep a finger on the pulse of theirconstituents, they must also deal with specialinterest groups. These groups—sometimes calledsingle-issue or pressure groups—are organizationsof citizens devoted to a specific area of public poli-cy, promoting a specific view. The Church inmedieval (and more recent) times, the antislaverymovement of English abolitionist WilliamWilberforce and his associates, the suffragette andtemperance movements in the early part of thetwentieth century, and the environmental, pro-choice, and antiabortion groups of today are allexamples.

These groups share a common method ofoperation: they try to attract the greatest possibleattention to their cause in an attempt to shapepublic opinion and influence government policy.Many of these groups are adept at manipulatingthe media, and that part of the media that thriveson controversy is often all too happy to give themthe publicity they crave. Because of the single-mindedness of their advocates and the media atten-tion they attract, issues promoted by these groupsfrequently acquire a visibility, and possibly have animpact, out of proportion with either the basicvalue represented or the number of people sympa-thetic to these causes.

Since some issues spawn diametricallyopposed interest groups (for example, pro-choiceand pro-life groups), elected representatives fre-quently find themselves on the horns of a dilemma.


Jeffersonians would do the political calculus andvote with the majority of their constituents regard-less of the validity of their thinking; Burkeanswould vote according to their conscience and riskoffending their electorates. Others would abstainfrom voting in the belief that inaction would bethe least disturbing course to follow.

LO B B Y I S T S

Lobbyists are people who influence legisla-tors and governments to act in the interests oftheir clients. They do this by seeking interviews,writing letters, bringing external pressures tobear, and so on. In this respect, they act muchlike special interest groups (who also engage inlobbying), with the difference that lobbyists arenot necessarily personally committed to the causethey are promoting.

Lobbying37 is not limited to getting the earof legislators on the government side; it is alsodirected to the opposition parties. Furthermore,recognizing the importance of the nonelectedpart of the executive arm, lobbying activities arefrequently and intensely aimed at civil servantswho enjoy influence in policy development andimplementation.

Lobbying is sometimes viewed as good, some-times as bad. Lobbyists can often help bring issuesto a satisfactory expeditious resolution. They maycontribute to the clarification of an issue or suggestcompromises such that the legislation or interven-tion accommodates the general welfare even if itappears to have sided with a particular interest.Lobbyists often sponsor coalitions to promote pub-lic knowledge, debate, and support. They also rel-ish an opportunity to get their client’s point ofview expressed in the media.

IT HAS BEEN REPORTED THAT THE NUMBER OF LOBBY-

ISTS IN WASHINGTON ROSE FROM APPROXIMATELY 400

IN 1960 TO 7,000 IN 1990. MOREOVER, THERE WERE

AN ESTIMATED 27,000 LAWYERS WHO WERE READY TO

MAKE REPRESENTATIONS OR LOBBY ON BEHALF OF

THEIR CLIENTS. IN OTTAWA, ACTIVITIES RESEMBLING

LOBBYING HAVE BEEN GOING ON FOR AS LONG AS THE

GOVERNMENT HAS EXISTED. IN 1992, SOME 300 FIRMS

ACKNOWLEDGING ACTIVITIES OF THE TYPE REGULATED

BY THE FEDERAL LOBBYISTS REGISTRATION ACT.

Lobbying is often seen as undemocraticbecause so much of it occurs behind closed doors.Skeptics suspect, rightly or wrongly, that shadydeals are made. Even the satisfactory resolution ofan issue may not allay the suspicions of individualsdevoted to transparency and accessibility in publicaffairs. No one has yet devised a way of makinglobbyists publicly accountable; this does not mean,however, that they are unaccountable in the senseof being irresponsible.

Whatever one may think of them, lobbyistsplay a significant role in public policy develop-ment, and are here to stay.

O P I N I O N P O L L S

Public opinion polling is another source ofinfluence brought to bear on elected representativesand governments. Often acting independently,pollsters pick the issues, formulate the questions,assemble the replies, analyze the answers, andrelease the results to the media. The issues selectedare based on their topicality. Sometimes topicalvalue is raised simply by taking a poll and publiciz-ing the results.

Alternatively, pollsters undertake surveys onbehalf of a sponsor, such as a political party, thepremier’s office, other parts of the executive branch


of government, corporations, or pressure groups. Inthese cases, the issues are those of the sponsor, pos-sibly the questions, as well. Basic data, for example,the actual responses, often remain confidential, inthe hands of the polling organization or the spon-sor, and thereby the pollsters themselves becomepart of the circle of private advice to the govern-ment or organization and are no doubt contribut-ing to the agenda of the day in a manner largelybeyond official access-to-information mechanisms.

The public and the media pay considerableattention to poll results as they are announced withthe seeming regularity of weather reports.Responsible pollsters announce the sample size(usually a minuscule portion of the population)and the degree of statistical confidence in theresults. The major pollsters in Canada are techni-cally expert and should be viewed as objective.Subjectivity may arise in the formulation and thesequence of the questions asked. Pollsters willsometimes publish the questions along with theresults to help readers, commentators, and analystsform their own opinions.

David Flaherty, a prominent Canadian acade-mic remarked: “Although such companies claimthat they are simply measuring opinion in anobjective fashion and giving politicians a road mapof voters’ attitudes, one suspects that the publishedresults of their polling shape reality as much asreflect it. Polls at least influence everyone’s percep-tion of the present.”38

In favor of such polls is the notion of consult-ing the population directly, which allows citizens tohave direct input into policy-making. Assumingthe polls are accurate—and there is evidence tosuggest that they often are—the benefit to thedemocracy is a punctual awareness of the mood ofthe population on many policy issues.Nevertheless, as an expression of participatorydemocracy, polls cannot replace a good public dis-cussion. That the results of polls are often kept

confidential by the government leads some peopleto think that polls contribute to an erosion of par-liamentary control.

T H E M E D I A

A free press—print and broadcast—is essen-tial for a democracy; it is impossible to imagine atruly functional democracy without one.

The press plays several roles. In one role itsupplements the work of the political oppositionby acting as a watchdog: probing, publicizing,commenting, sometimes exposing. It provides avehicle for information sharing and a forum forpublic debate. It can inform and influence boththe public and the policymakers.

Much of reporting, editorial and commen-tary, is unbiased, serving nothing but the truth.Some reporting, however, advocates a particularviewpoint or philosophy, reflecting the inclinationof the reporters or owners of the media outlet. Andnearly all media are subject to market forces ofcompetition—they need readers, listeners, viewers,advertisers. Foreign competition must also be met.In such an environment, the temptation to indulgein sensationalism in order to foster sales isomnipresent. To prevent excesses and to promoteresponsible journalism in Canada, the laws of libelare buttressed by a press council to which appealcan be made.

Understandably, very few decision makersignore the media. An overreliance on the press,however, especially at the expense of keeping intouch with the views of ordinary citizens, can leadto a skewing of public policy in favour of somespecial interest that has managed to obtain broadmedia exposure. In such cases, citizens may have towait for the next election to enjoy an occasion toimpose accountability.


E X T E R N A L F O R C E S

External factors present certain constraints onour ability to act with full independence. These fac-tors include the growth of continentalism, globaliza-tion of industry, trade and commerce, militaryalliances, and various agreements that Canada hasentered into with other countries and with interna-tional agencies. These arrangements are often com-plex and therefore entrusted to expert public servants;the implications of these commitments are often notclearly understood by members of Parliament.

The Honourable Lloyd Axworthy, M.P.,wrote the following observation about the impactof globalization on our democratic institutions:

Today, the world economy conspires toerode the capacity and competence of nationstates. Money moves across national bordersin seconds. Economic decisions are made inboardrooms far removed from national con-trol or accountability. Great power blocs jostlefor military, economic, and political control.The global village is very much upon us, andCanada, for reasons of history and proximity,is on the front line of this phenomenon…

The sensitive matter of public regulationof business, especially the need to holdaccountable the increasing concentration ofprivate wealth and economic power operatingacross national borders, is becoming a severetest for the Canadian political system. It mustfind a way to maintain the nation’s sovereign-ty and integrity against an age of continentaland international pressures.39

S O M E I M P L I C A T I O N S O F

F E D E R A L I S M

Federalism in Canada involves two levels ofgovernment—one national, one provincial—exer-cising powers independently. Those powers derive

from the same people, as citizens of the countryand of the provinces in which they reside.

The term fiscal federalism describes the web ofarrangements—rental of tax powers, financialtransactions, transfer payments, protocols, rulesand regulations governing them—that has becomea central feature of federal-provincial relations overthe last several decades. Many similar understand-ings have been implemented by the provinces andtheir creations—the municipalities and other formsof local governments. There are also numerousagreements on other subjects of interest to bothlevels of government.

Federal-provincial arrangements and theircontinuing administration and adaptation give riseto often delicate and protracted negotiations.Sometimes, these take place in private, away fromthe seat of government. Other meetings are heldunder the glare of public television, a concession tovisible accountability. The prime minister and theprovincial premiers sit as equals and engage in bar-gaining with little, if any, reference to Parliamentand the provincial legislatures. There are alsonumerous meetings between federal ministers andtheir provincial counterparts with responsibilitiessuch as finance or health.

So important are these federal-provincialrelationships that governments maintain depart-ments or secretariats of intergovernmental affairsto deal with their counterparts. Ministers, sup-ported by their senior civil servants, become thearchitects of federal-provincial relationships,determining the practical effect of the division ofpowers set out in the constitution. Thus impor-tant political decisions are not taken by legisla-tures, or the political parties comprising them,but by levels of government, hence the expressionexecutive federalism. Nevertheless, since many ofthese agreements require legislation to makethem effective, an opportunity to debate themarises when governments place them before their


legislatures. By that time, however, the die is usu-ally cast.

Agreements reached by heads of governmentat federal-provincial meetings are seldom rejected,because the government leader who negotiated theagreement usually controls the majority in the leg-islature. Nevertheless, despite the strong supportgiven to them by leaders at the time they weredrawn up, both the Meech Lake (1987) andCharlottetown (1992) Accords failed to gain accep-tance—of all provinces in the former, and of thepeople in a national referendum in the latter.

G O V E R N M E N T I N O T H E R

C O U N T R I E S

Not all countries follow the British parliamen-tary tradition. Even within the Commonwealth,there are republics that vest the ultimate power inthe people rather than the Crown. Most of thosecountries, however, even those with a president,follow the parliamentary model of responsible gov-ernment, with the executive chosen from among,and accountable to, the elected members of thelegislature. Some non-Commonwealth countries,such as Israel, also adopt this form of governance.

The form of government in the United Statesis different than this. The United States has a presi-dent for whom, through the proxy of the electoralcollege, the entire electorate may vote. This givesthe president enormous political power. To counterthat power, the U.S. Constitution provides for adivision of powers among the executive, the legisla-ture and the judiciary. Unlike the parliamentarysystem, neither the president nor any of his cabinetare members of the legislature. The president canpropose legislation, can sign into law or exercise alimited power of veto over bills passed byCongress, but he cannot on his own enact statutes.

The U.S. Congress has two houses: theHouse of Representatives and the Senate. Among

its other responsibilities, the Senate, comprisingtwo senators from each state elected for staggeredsix-year terms, has the power to approve or turndown senior presidential appointments, and bothHouses can launch investigations into what theexecutive branch has done. The president does not,however, testify before congressional committees;that is left to appointed officials. There is no dailyquestion period for the president and his cabinet,as there is in the parliamentary tradition.

B A S I C I N S T I T U T I O N S O F

A M E R I C A N G O V E R N M E N T


proposes legislation

appoints

reviews and may void legislation

conf

irmsCongress

JudiciaryPresident House ofRepresentatives Senate

Departments

Administra-tion

People

CongressJudiciaryPresident House of

Representatives Senate

Departments

Administra-tion

People

elect elect decide cases

administer

appoints cabinet

elect

F I G U R E 2 . 2

SOURCE: PEOPLE, POLITICS AND GOVERNMENT (SCARBOROUGH: PRENTICE-HALL

CANADA INC., 1995).

Like Canada, the United States is a federa-tion, with certain responsibilities and powers givento the states. Although the mechanisms are quitedifferent, both countries display some overlap ofprograms and policies at the two senior levels ofgovernment.

Much more so than Canadians, Americansvote directly for a variety of public officials. Somejudges, state cabinet officials, and many local offi-cials are elected. Views are strongly held on bothsides of the question of whether this makes for bet-ter accountability.

DEMOCRACY IS THE WORST FORM OF

GOVERNMENT —EXCEPT FOR ALL THE REST.

S I R W I N S T O N C H U R C H I L L

Elsewhere in the world, other forms of gov-ernment reflect the history and nature of countriesand their citizens. Some presidents are strong; oth-ers are figureheads. Some governments are fullyaccountable; others avoid scrutiny and criticism.Democracy takes many forms, as do dictatorial andtotalitarian regimes. Names do not always corre-spond to reality, as has been seen in certain “peo-ple’s democracies” and “democratic republics.” Thekey to determining whether there is a good systemof governance in a country is to see if the govern-ment reflects the will of the people and acts intheir interest; and should that government fail inthis regard, that peaceful mechanisms are availableto replace it.

P R I V A T E S E C T O R

G O V E R N A N C E

The term governance, when used in the con-text of the private sector, usually refers to for-prof-it, incorporated entities (there is scant literature on

nonprofit private organizations). The followingdeals with this interpretation and serves only as anintroduction to the subject. Indeed, comprehensivetreatment of corporate governance would require aseparate book.

“CORPORATE GOVERNANCE” MEANS THE PROCESS AND STRUC-

TURE USED TO DIRECT AND MANAGE THE BUSINESS AND AFFAIRS

OF THE CORPORATION WITH THE OBJECTIVE OF ENHANCING

SHAREHOLDER VALUE, WHICH INCLUDES ENSURING THE FINAN-

CIAL VIABILITY OF THE BUSINESS. THE PROCESS AND STRUC-

TURE DEFINE THE DIVISION OF POWER AND ESTABLISH MECHA-

NISMS FOR ACHIEVING ACCOUNTABILITY AMONG SHAREHOLD-

ERS, THE BOARD OF DIRECTORS AND MANAGEMENT. THE DIREC-

TION AND MANAGEMENT OF THE BUSINESS SHOULD TAKE INTO

ACCOUNT THE IMPACT ON OTHER STAKEHOLDERS SUCH AS

EMPLOYEES, CUSTOMERS, SUPPLIERS AND COMMUNITIES.40

Corporate governance became an issue in thelate 1970s, largely as a result of the failure of com-pany officers of some of the largest corporations inNorth America to provide important informationnot only to shareholders but to their own directors,as well. That in itself was not unusual at the time,except that in these cases the officers were in viola-tion of the law. The most famous cases prosecutedwere for illegal political contributions, bribery, andinterference in the affairs of a foreign country. Inseveral instances, the governing body—the boardof directors, and the shareholders—had been keptin the dark about these felonious activities.

In the 1980s, concerns about governancefocused not so much on deliberate violations of thelaw (although there were some serious unethical,even illegal, activities), but on the quality of deci-sions that had serious consequences for the corpo-ration. Greed had prevailed over competence andprudence, and in many cases corporate perfor-mance suffered considerably. Frequently, with the


approval of boards of directors—and sometimesunknown to them—corporate officials engaged ina frenzy of transactions of dubious value (for exam-ple, leveraged buy-outs, mergers, stripping of com-pany assets). In many financial institutions, com-petition for funds, market share, and the quest forshort-term profits led to risky lending practices.Many corporate officers had to engage in activitiesbordering on illegality, when not clearly illegal,simply to salvage what they could from bad deals.Simultaneously, recognition of conflicts of intereston corporate boards started to affect the credibilityof many governing bodies.

While it is possible to hold corporate officersresponsible for the actions that led to some spec-tacular debacles, the public—and indeed the share-holders—have been raising questions about corpo-rate governance, as have financial analysts and thefinancial press: What if the board of directors hadintervened or demanded a regular rendering ofaccount? Did the board have sufficient indepen-dence and competence to intervene, or at least tochallenge the officials? How would the latter havebehaved had they known they were to be held toaccount by the board?

CORPORATE GOVERNANCE

DERIVED FROM THE LAW

THE POWER AND LEGAL RESPONSIBILITIES OF CORPORATE

DIRECTORS AND OFFICERS DERIVE FROM CORPORATE LAW. TO

BE SECURE, POWER HAS TO BE LEGITIMATE, AND FOR POWER

TO BE LEGITIMATE, WHETHER PUBLIC OR PRIVATE, IT HAS TO

BE ACCOUNTABLE. THE MODERN BUSINESS CORPORATION IS

ACCOUNTABLE ACCORDING TO ITS CONSTITUTIONAL LAW, JUST

AS THE GOVERNMENT IS ACCOUNTABLE ACCORDING TO CON-

STITUTIONAL LAW. THE CONSTITUTIONAL LAW OF THE BUSI-

NESS CORPORATION IS CALLED CORPORATE GOVERNANCE,

WHICH DEFINES THE ACCOUNTABILITY OF DIRECTORS TO

SHAREHOLDERS AND OF OFFICERS TO DIRECTORS.41

Changes will come when evolving societalexpectations are reflected in law.

B O A R D S O F D I R E C T O R S

Corporate governance in North America is aprocess by which owners of the corporation exer-cise influence over its management. The corpora-tion is governed by a board of directors whosemembers are elected by the company’s shareholdersat annual meetings.

Laws and traditions lead to differences incorporate governance from country to country.Ownership representation and insider participa-tion are usually the major areas of difference. Insome countries, owners, as distinct from manage-ment, are not represented on the governing body;that does not mean that they do not have anyinfluence. In Japan, shareholders elect membersof the board, but the latter have to be insiders.Major institutional share owners, however, exer-cise considerable influence on senior corporateofficials through informal channels. In Canada,both corporate officials (or insiders) and outsidedirectors are found on the boards of most largecorporations.

INSIDE DIRECTORS ARE THE COMPANY’S CHIEF EXECU-

TIVE OFFICER (CEO) OR EQUIVALENT AND THE OTHER

DIRECTORS WHO ARE FULL-TIME MANAGEMENT

EMPLOYEES. OUTSIDE DIRECTORS ARE MEANT TO PRO-

VIDE OBJECTIVE AND INDEPENDENT ADVICE AND TO

CHALLENGE THE ACTIONS OF THE CEO AND

MANAGEMENT.

The independence of outside directors issomewhat compromised when they are nomineesof the CEO, especially when the CEO also hap-pens to be chairman of the board. This is the casein many corporations. Furthermore, outside direc-


tors are not necessarily independent when theyhave been named to represent a vested interest; norare they necessarily effective when they have beenappointed for symbolic reasons (or tokenism, as itis often called), or simply to give the board an airof respectability.

Currently, there are some serious debates overthe structure of governance of publicly held corpo-rations, centered on the corporate board of direc-tors. One of the issues is the separation of the roleof the chairman from that of the chief executiveofficer. Another is the manner in which directorsare chosen. How the board will discharge itsresponsibility is another serious concern; at whatpoint does the board intervene managerially orotherwise on evidence or rising apprehensions ofnonperformance of the CEO? How to assess theperformance of the board itself has become a cen-tral issue. Finally, and not least of all, are the issuesof directors’ liability and the possibility of class-action suits responding to recent legislation andlegal interpretation.

S H A R E H O L D E R S

Shareholders elect the board of directors andentrust the governance of the corporation to them.This does not mean that they play no role in cor-porate governance. Major institutional sharehold-ers, for example, may regard themselves asinvestors, as has been the tradition, or they may seethemselves as both major investors and owners, amore recent development. The difference betweenthe two may not be immediately apparent: beinginvolved as an investor may focus the interest onshort-term issues, whereas seeing oneself as anowner involves a concern for the company’s long-term prosperity. This distinction may be recog-nized by someone like the investment manager of acorporate pension plan, who likely has a longer-term perspective. In short, shareholders who regard

themselves as owners (as distinct from those whoact solely as investors) can maximize the benefits oftheir investments if they recognize that certainresponsibilities come with ownership.

Shareholders have influenced corporatebehavior in many instances, such as in the provi-sion of additional information; sometimes share-holders’ input has affected operations that theythought were politically or socially offensive.

S O C I A L R E S P O N S I B I L I T Y

There is a growing propensity for institution-al investors not only to foster the best financialinterest of their clients, but also to respect the placethe publicly held company occupies in the life ofthe community or of society in general. Theysometimes reflect the values held collectively bytheir clients such that investment of funds is influ-enced by both the potential financial return and bya view of the political, societal, or moral validity ofthe business. The expectation is that a company besocially responsible towards the general populationand that it must deal fairly with its employees.

Since the large-scale business enterpriseis the central institution of the modern worldeconomy, the directors of these corporationshave come to have responsibilities that affectevery one’s welfare. There is, therefore, a con-nection between the responsibility of thedirectors of the corporation and the distressof American industry in today’s world econo-my. But it is essential to keep the causal con-nection straight… the boardroom is a mirrorof fundamental change in business. “Causeand effect” generally run from the economyto the boardroom, not the reverse. Change inthe economic environment of the present-dayAmerican business has expanded the nature ofdirector responsibility.42


IN A WAY, ECONOMIC LEGITIMACY (COMPETITIVENESS),

AND POLITICAL LEGITIMACY (ACCOUNTABILITY) ARE

TWO SIDES OF THE SAME THING. THE FOUNDATION OF

OUR CONCEPT OF CORPORATIONS IS OUR BELIEF THAT

BECAUSE SHAREHOLDERS CAN BE COUNTED ON TO

REQUIRE THAT THEIR OWN LONG-TERM INTERESTS BE

ACCOMMODATED, CORPORATIONS WILL BE DIRECTED

ALONG THE LINES MOST BENEFICIAL TO SOCIETY. THIS

ACCOUNTABILITY ALLOWS US TO GIVE CORPORATIONS

ENORMOUS POWER TO MAKE DECISIONS THAT AFFECT

EVERY ASPECT OF OUR LIVES.43

S TA K E H O L D E R S

Insiders, outsiders, shareholders, customers,interested parties in the community, or in thewider society, constitute an impressive variety ofpeople with interests in the corporation. Collectively,they are called stakeholders, a term that acquiredcurrency not so long ago in recognition of the need

for people charged with governance to account notonly to owners and on the financial performance,but to a multitude of institutions and individualson an array of issues, both business and societal.

A list of stakeholders would include the fol-lowing, each of whom would need to be informedon the performance of the company:

• management; • board of directors; • shareholders;• lenders; • regulators;• customers;• community.

To this list could be added the suppliers andother distinct sets of stakeholders, not least thegovernment. The challenge for private corporationsis to provide governance that is responsivelyaccountable to all these stakeholders.

TO R O N T O S T O C K E X C H A N G E CO M M I T T E E O N

CO R P O R AT E G OV E R N A N C E

In 1993, the Toronto Stock Exchange (TSE)established a blue-ribbon Committee on CorporateGovernance in Canada with a mandate to conducta comprehensive study of corporate governanceand to recommend improvements for the mannerin which Canadian corporations are governed. Thework of the committee focused on many of thematters discussed above, and more.

Five issues were prominent in the committee’sresearch:

• the state of corporate governance in Canada;• the duties of directors;• the relationship between directors and man-

agement;• the relationship between directors and share-

holders; and• enhancing board effectiveness.


STAKEHOLDERS

ACCORDING TO DON MCGILLIVRAY, A NOTED ECONOMIC JOUR-

NALIST, AND A STUDENT OF THE LANGUAGE AND THE EVOLUTION

OF ITS USAGE, THE WORD STAKEHOLDER GOES BACK ABOUT 300

YEARS IN BRITAIN. THE STAKEHOLDER HELD THE STAKE, THE

MONEY THAT TWO OTHER PEOPLE HAD WAGERED AGAINST EACH

OTHER ON SOME EVENT WITH AN UNCERTAIN OUTCOME. HE WAS

ENTRUSTED WITH THE MONEY THAT EVENTUALLY HAD TO BE

TURNED OVER TO THE WINNER.44 MORE RECENTLY, THE EXPRES-

SION ACQUIRED A NEW MEANING TO DESCRIBE A PERSON WITH A

“STAKE,” A PERSONAL INTEREST IN THE OUTCOME OF AN ACTIVI-

TY OR IN AN INSTITUTION. ACCORDING TO A DIFFERENT EXPLA-

NATION, THE WORD WAS BORROWED FROM A MORE RECENT

USAGE IN RELATION TO METAL-MINING EXPLORATION: TO “STAKE A

CLAIM” IS TO RECORD OFFICIALLY ONE’S INTEREST IN A PROSPEC-

TIVE MINERAL FIND.

In exploring these issues and related questions,the committee received submissions from participantsin all aspects of corporate governance in Canada.

A set of proposed guidelines for improvingcorporate governance emerged from the research.The core of this guidance is the committee’s viewthat “the board of directors of every corporationshould explicitly assume responsibility for the stew-ardship of the corporation and, as part of the over-all stewardship responsibility, should assumeresponsibility for the following matters:

• adoption of a strategic planning process• identification of the principal risks of the cor-

poration’s business and ensuring the imple-mentation of appropriate systems to managethese risks

• succession planning, including appointing,training and monitoring senior management

• a communications policy for the corporation, and• integrity of the corporation’s internal control

and management information systems.”45

To help increase effectiveness in fulfilling thesefive responsibilities, the committee made recom-mendations in relation to the board’s constitutionand to the governance-related functions carried out

by the board. Among other matters dealt with arethe relationship between the board and shareholders,with emphasis on the need for two-way communi-cation, and the importance of the quality and time-liness of the information published by corporations.

G O O D G O V E R N A N C E —

A C O M M O N G O A L

The desire to improve governance structuresand performance is clearly demonstrated in boththe public and private sectors. The appendix toPart I illustrates how the government of Canadahas attempted, over about a decade, to strengthenits governance and accountability practices. Thatexperience is echoed in many other parts of theCanadian public sector. The initiative of the TSEdescribed above shows how seriously the private sec-tor treats the subject of corporate governance.

Although the specific approaches may vary,there is substantial commonality between that ofthe public and that of the private sector. The fiveareas for which the TSE committee recommendsboards of directors assume responsibility tie inclosely with the characteristics of good governancepresented in Chapter 1. The following table illus-trates this similarity.


CHARACTERISTICS OF GOOD GOVERNANCE

1 COMPRISING PEOPLE WITH THE NECESSARY KNOWLEDGE, ABILITY, AND COMMITMENT TO FULFILL THEIRRESPONSIBILITIES

2 UNDERSTANDING THEIR PURPOSES AND WHOSE INTERESTSTHEY REPRESENT

3 UNDERSTANDING THE OBJECTIVES AND STRATEGIES OF THEORGANIZATIONS THEY GOVERN

4 KNOWING AND OBTAINING THE INFORMATION THEY REQUIRETO EXERCISE THEIR RESPONSIBILITIES

5 ONCE INFORMED, BEING PREPARED TO ACT TO ENSURE THATTHE ORGANIZATION’S OBJECTIVES ARE MET AND THATPERFORMANCE IS SATISFACTORY

6 FULFILLING THEIR ACCOUNTABILITY OBLIGATIONS TO THOSEWHOSE INTERESTS THEY REPRESENT BY REPORTING ON THEIRORGANIZATION’S PERFORMANCE

…TSE DIRECTOR’S RESPONSIBILITIES

…APPOINT, TRAIN, MONITOR SENIOR MANAGEMENT, …MANAGE RISK, AND…ADOPT STRATEGIC PLANNING PROCESSES

…MONITOR SUCCESS IN IMPLEMENTING STRATEGIC PLAN…APPOINT, TRAIN, MONITOR SENIOR MANAGEMENT…INTEGRITY OF INTERNAL CONTROL AND MIS

…COMMUNICATION POLICY WITH SHAREHOLDERS


20 THE AMERICAN CONSTITUTION, 202.21 NO PROVINCIAL LEGISLATURE HAS A SECOND CHAMBER ANY LONGER; THEY ARE ALL UNICAMERAL. WITH THAT EXCEPTION, THESE DESCRIPTIONS HOLD FOR THE PROVINCES AS WELL AS FOR

THE FEDERAL GOVERNMENT.22 KINGDOM, THE CIVIL SERVICE, 35.23 C.E.S. FRANKS, THE PARLIAMENT OF CANADA (TORONTO: UNIVERSITY OF TORONTO PRESS, 1987), 35.24 INTERESTINGLY, THE 1993 CONSERVATIVE LEADERSHIP RACE HAD A CANDIDATE WHO SUGGESTED THAT THE CABINET OUGHT TO BE CHOSEN BY THE PARTY CAUCUS.25 KINGDOM, THE CIVIL SERVICE, 41-42.26 REPORT OF THE SPECIAL COMMITTEE ON REFORM OF THE HOUSE OF COMMONS, JAMES MCGRATH, P.C., M.P., CHAIRMAN, JUNE 1985, 6.27 KINGDOM, THE CIVIL SERVICE, 41-42.28 CANADA COMMUNICATIONS GROUP—PUBLISHING, THE NEW FACE OF GOVERNMENT: A GUIDE TO THE NEW FEDERAL GOVERNMENT STRUCTURE, 2ND ED. (OTTAWA: CANADA

COMMUNICATIONS GROUP—PUBLISHING, 1994), 126.29 IBID., 126.30 IBID.31 IBID., 45.32 FRANKS, PARLIAMENT, 215.33 KINGDOM, THE CIVIL SERVICE, 46.34 PATRICK BOYER, M.P., “THE PARADOXES OF REPRESENTATION,” THE HILL TIMES, 31 OCTOBER 1991. IN THE SAME ARTICLE, HE CITES PETER DOBELL IN A DISCUSSION PAPER—SEE NEXT

PARA.35 PETER DOBELL, IBID.36 BOYER, IBID.37 LOBBYING ACQUIRED ITS NAME FROM THE ACTIVITY OF CERTAIN PERSONS IN THE LOBBY, WHICH IS THE SPACE IN A LEGISLATIVE BUILDING, JUST OUTSIDE THE CHAMBER, WHERE THE PUBLIC

HAS ACCESS TO MEET THE LEGISLATORS.38 DAVID H. FLAHERTY, “WHO RULES CANADA?” DAEDALUS, JOURNAL OF THE AMERICAN ACADEMY OF ARTS AND SCIENCES, VOL. 117, NO. 4, 1988, 105.39 HON. LLOYD AXWORTHY, “THE FEDERAL SYSTEM—AN UNCERTAIN PATH,” DAEDALUS, IBID., 150-151.40 TORONTO STOCK EXCHANGE COMMITTEE ON CORPORATE GOVERNANCE IN CANADA, WHERE WERE THE DIRECTORS: GUIDELINES FOR IMPROVED CORPORATE GOVERNANCE IN CANADA

(TORONTO: TORONTO STOCK EXCHANGE, 1994), 7.41 ARTHUR FLEISHER, JR., GEOFFREY C. HAZARD, JR., MIRIAM Z. CLIPPER, BOARD GAMES, THE CHANGING SHAPE OF CORPORATE POWER (TORONTO: LITTLE, BROWN & COMPANY (CANADA),

1988), 11.42 IBID., BOARD GAMES, 12.43 ROBERT A.G. MONKS, NEIL MINOW, POWER AND ACCOUNTABILITY (U.S.A.: HARPER COLLINS, 1991), 17.44 DON MCGILLIVRAY, THE OTTAWA CITIZEN, 12 DEC. 1992, A-1.45 TORONTO STOCK EXCHANGE COMMITTEE, WHERE WERE THE DIRECTORS?, 4.

C H A P T E R 3

THE LANGUAGEOF GOVERNANCEANDACCOUNTABILITYV O C A B U L A R Y A N D

T E R M I N O L O G Y

The development of a vocabulary to conveythe subtlety of a discipline is very much part of itsmaturing process. In the early stages of a discipline,familiar words serve as descriptors. As the disci-pline develops, particular meaning is attached tothe words chosen. Later, as the need arises to intro-duce subtler nuances, other words, often close syn-onyms, are introduced. The development of thelanguage of a discipline may be chaotic at first, butusually becomes orderly through usage. This processcan be accelerated if a deliberate effort is made bythe professional body to standardize terminology.

This chapter explores the meanings of wordsand expressions used in discussions about gover-nance and accountability. It is hoped that this willcontribute to a clearer understanding of the con-cepts involved.

R U L E S A N D D I S C R E T I O N

The discussion of the two terms rules and dis-cretion together is meant to illustrate a rarely debat-ed issue concerning the principles underlying theconduct of government at the policy level. Thequestion is whether policies ought to be promul-gated as long-standing rules or simply adjusted asconditions change. The latter approach is calleddiscretionary policy.

Rules can be used to justify action or todepoliticize difficult allocative choices.46 Standard-ization is often cited as the reason for creating rulesand enforcing them. Rules are also used to formal-ize or consolidate authority or a source of power.

Sometimes, government lacks the knowledgerequired to adhere successfully to a discretionarypolicy. This is particularly true in the managementof the economy. Not being able to predict thecourse of the economy, or aspects of it, let alone toconfidently anticipate the response to governmen-tal policies, suggests that a flexible and timelyintervention might actually be detrimental to thestability of the economy in the longer run.

The alternative is preset rules. The belief isthat rules reduce risk of chaos and in a way offer abetter climate of confidence in the future.Conformity, order—indeed predictability—bring agreater feeling of security, of a purpose pursued bya large segment of the population. To use the man-agement of the economy as an example, pegging ofthe currency at a given rate of exchange, or rulingthat the money supply will rise at a constant rateare preset rules.

Complicating the problem is a phenomenoncalled inconsistency of time—the passage of timebetween the taking of a decision and its imple-mentation. What looked appropriate and optimalat the time of decision taking no longer is appro-priate and optimal when put in practice. Thisphenomenon is often attributed to the gapbetween theory and practice; it is more that thedecision was correct in practice at the time of itstaking, but in the interim, new, unforeseen factorsmake it incorrect today.

In the absence of a solid commitment to thepursuit of the original plan, decision makers willmodify policy to suit the new circumstances.Knowing this, the people affected may well antici-pate what the authorities will do and may frustratethe policymakers’ intended outcome. However, this

P A R T I . S E C T I O N 1 . A C C O U N T A B I L I T Y ’ S C O N T E X T — G O V E R N A N C E 3 3

too can be anticipated, and further adjustmentsmade to reflect it. And on it goes, up to a pointwhere it becomes very difficult to maintain theoriginal goal or even retain credibility.

Hence the attraction of rules. The problem isthat a rule, to resist the phenomenon of inconsis-tency of time, would have to be perfect, whichobviously is not possible given the complexity anddynamics of society. The reality is that govern-ments resort to both rules and discretion in policy-making and implementation in the hope of reduc-ing inconsistency and maintaining credibility. Ifthey cannot devise rules that they believe will beeffective, they may well abandon the idea of a pub-lic intervention altogether.

From the point of view of accountability,inconsistency of time has to be well understood.47

In politics, inconsistency of time is one of its great-est afflictions. Political promises at election timemay be sincere but may later turn out to be impos-sible to honour or deemed to be against the inter-ests of the population. Opposition parties may wellunderstand the situation but choose to exploit itfor partisan purposes. Trying to hold the govern-ment accountable for its promises becomes afavoured tactic of the opposition. Rendering anaccount against an original plan without takinginto account changed circumstances, however, isnot good accountability.

Discretion is used in decision making whensituations are unclear, when goals pursued are con-tradictory, when the ends are understood but themeans not known, generally speaking when situa-tions and circumstances are so volatile that theyrequire continual adaptation.

Discretion is best understood in the contextof implementation of rules and policies alreadyestablished. Discretion is usually dictated by thecircumstances of decision making or the status ofthe person exercising discretion—ministerial discre-tion in immigration cases, for instance. Such cases

can be politically delicate or morally difficult. Theymay present some serious dilemmas, as they ofteninvolve a clash of values and interests. There maybe no precedents to help in finding a solution, norguidelines to follow. There are no doubt someprinciples, but insufficient links to the situation athand. The domain may be so complex as to pre-vent any routinization of decision making andrequire that each case be judged on its own merits.

When ministerial discretion is exercised, theresulting decision is either welcome, treated withindifference, or condemned; in the latter, it will becharacterized as arbitrary, whether or not the min-ister agonized over the decision. There are indeeddecisions that will appear arbitrary in character, butnot necessarily whimsical; whoever has taken thedecision assumes the responsibility of the decision.The question is then, does that person accept theresponsibility for the consequences, personally? Asa custom, and in logic, probably yes. Should thatindividual be sanctioned if he or she erred?Certainly not. He or she is only acting in accordwith the responsibility delegated by the legislationand should be respected accordingly.

At the other extreme, some decisions defyroutinization but are so inconsequential that theyare left to the discretion of a manager. Thus, dis-cretion can be exercised on the most intractabledilemmas or on the most trivial issues.

There are situations where a person is knownto be so trustworthy that imposing external ruleson his or her conduct is eschewed. Some churchleaders are accorded such confidence. A differentcase is that of the referee, the arbiter, who for rea-sons of neutrality or independence has to exercisediscretion in the application of the rules, some-times with no recourse. If there is no recourse, it isby design, as the chain of events that may occur asa result of a successful recourse is deemed to beworse in consequence than whatever gave rise tothe erroneous decision in the first place.


D I S C R E T I O N A R Y P O W E R S

A good example of discretionary powers isone that describes certain relationships betweenone level of government and the junior level. Oneprovince—and this is likely to be the case in sever-al others—has municipal affairs legislation thatcontains an article giving the minister “discre-tionary power” over the actions initiated by themunicipalities, no matter if such actions have beendemocratically endorsed by the population imme-diately affected or not.48

A case in point was the fusion of police andfire-protection services of two neighbouringmunicipalities. The court ruled that the minister ofmunicipal affairs could exercise discretionary powerand turn down the merger proposal. The ministercould have been concerned about the legality ofthe initiative, the integrity of the process, or forthat matter, his view of the general interest of thepopulation. Indeed, the minister may well deemthat the interest of the population immediatelyaffected is narrower than the general interest he orshe feels mandated to protect. Thus, the authorityand the responsibility are reasonably clear.Accountability is not so clear, however.

B U R E A U C R A T I Z A T I O N

Bureaucratization replaces discretion; it setsup rules, regulations, and procedures to providea means of controlling behaviour without havingto be on the premises to give instructions.Bureaucratization is the process of forming abureaucracy. A bureaucracy is a formal organiza-tional arrangement characterized by a division oflabour, job specialization with no functional over-lap, exercise of authority through a vertical hierar-chy or chain of command, and a system of internalrules, regulations, and record keeping.49

It is assumed that most of those working in abureaucracy in a democracy are professionals in

their specialities, and that their occupational loyal-ties rest with their organizations rather than with apolitical party or other external affiliation.50

BUREAUCRACY

BUREAUCRACY LITERALLY MEANS RULE BY OFFICE, COMING

FROM THE FRENCH WORD BUREAU MEANING OFFICE AND THE

GREEK ROOT KRATEIN, TO RULE. BUREAU DERIVES FROM THE

SAME WORD FOR DESK, ORIGINALLY A TABLE COVERED WITH A

CLOTH KNOWN AS BUREL. “RULE BY OFFICE” IS HOW CORPO-

RATIONS WORK, AND THE IMAGE OF THE BAIZE-COVERED

TABLE IN THE OLD GOVERNMENT BUREAU NOW SUMMONS UP

MOST OF ALL THE IMAGE OF THE CORPORATE BOARDROOM.51

While most large private sector corporationswould satisfy the above description, the termbureaucracy commonly connotes the administrativearm of government.

In some popular usage, bureaucracy is oftensynonymous with lethargy. However, in somecases, a bureaucracy may have been intended to bepassive, simply executing decisions taken at thepolitical level. A good example is what the framersof the U.S. Constitution intended: “They placedtheir faith in periodic elections, legislatures, and anelected Chief Executive rather than in a bureaucra-cy, however pure and efficient. There is nothing tosuggest that they believed sound administrationcould compensate for bad political decisions.Redressing grievances and bad political decisions[was] the function of the political process, ratherthan of administrative machinery.”52

The word technocracy is sometimes used todescribe a bureaucracy. This is a term most oftenheard on the European continent. Strictly speaking,technocracy is government by experts, and its par-ticular characteristics have an impact on account-ability. In such a political arrangement, government

P A R T I . C H A P T E R 3 . T H E L A N G U A G E O F G O V E R N A N C E A N D A C C O U N T A B I L I T Y 3 5

officials (nonelected) write their own rules, inspiredby internally generated principles, and oversee theirapplication. What is called a bureaucracy in theBritish parliamentary system is different: it is thebody of administration clearly under the control ofParliament and accountable to it.

The above definition is fairly descriptive ofbureaucracies in the Canadian public sector. Withrespect to certain public institutions, however, thenetwork of interdependences, interrelationships,and indeed loyalties within the bureaucracy tend tobe markedly different. Hospitals are an example,universities are another. And a military organiza-tion would be quite different again.

A hospital is an umbrella organization for thecoordinated and integrated exercise of several profes-sions complementing one another with the same pur-pose in mind: to ensure the welfare of patients. Theprofessions were there before the institutions. Theprofessionals’ first loyalty is to the patient, naturally;the second is to the profession itself; and the third isto the hospital. That should not be construed as dis-loyalty towards the last. Indeed, it is through theirprofession that the practitioners acquired their train-ing, standards, ethics, networks, and ensured theirrenewal. As a result, accountability arrangements in ahospital are more difficult to determine than in a typ-ical government department or a large corporation.

The most cherished value in a university isacademic freedom. In the minds of many, thepreservation of that essential freedom should notmean that universities ought not to demonstratetheir accountability to the public or ought notwork to strengthen their governance and manage-ment systems and practices. Universities, theythink, need to recognize that “autonomy is toaccountability what rights are to responsibilities.”53

But the accountability arrangements can be diffi-cult if they lead to making the institution account-able. To whom should one be made procedurallyaccountable is even a more delicate problem.

In short, within the context of a health careinstitution, accountability is possible but it is natu-rally diffuse. In the context of an educational insti-tution, accountability is possible, too, but naturallyweak, given the particular societal constraint. Incontrast to these two examples is a military organi-zation, which requires a principle of authoritycalled “unity of command”; that is, no one in aparticular organization receives orders from morethan one supervisor. This concept makes it possibleto create a hierarchy with clear accountability rela-tionships between units and capable of beingaggregated at the institution level.

I N D E P E N D E N C E

Independence is a term associated with suchnouns as neutrality, autonomy, objectivity, as well assuch adjectives as separate, uncommitted, unallied.Germane expressions include self-reliance, self-con-trol, non-partisanship. Within a bureaucracy or inpublic administration in general, a person who hasa status of independence is someone who isallowed to act free from authority or from the con-trol and influence of others.

INDEPENDENCE

INDEPENDENCE IS A WORD FORMED FROM THE ENGLISH PRE-

FIX IN, NOT, AND THE LATIN DEPENDERE, “TO HANG, TO BE

SUSPENDED.” AN INDEPENDENT PERSON DOES NOT HANG ON

OR FROM ANY CREED OR CLUSTER OF POLITICAL IDEAS.54

Realistically, no one can be completely inde-pendent. To achieve independence requires theconfluence of various circumstances. First, therehas to be an officially proclaimed status either forthe function within the organization or for a par-ticular position. Second, the persons in the func-


tion or occupying the position must be recognizedas being capable of independent thinking andaction by virtue of their knowledge, expertise,ethics, and professionalism.

As an illustration, the government ofCanada’s internal auditing standards begin with thenotion that internal auditors should be indepen-dent of the activities they audit.55 Internal auditorsare required to be objective in performing audits,and the organizational status of the internal audit-ing group should permit the discharge of its auditresponsibilities in that manner. Independence per-mits internal auditors to render impartial andunbiased judgments essential to the proper con-duct of audits.

To achieve this degree of independence, thehead of internal auditing should be responsible toan individual in the organization with sufficientauthority to promote independence and to ensurebroad audit coverage, adequate consideration ofaudit reports, and appropriate action on audit rec-ommendations. Direct communication with thehead of the organization is also a requirement.Organizations, in both the private and public sec-tors, are in many instances creating audit commit-tees at the highest corporate level to ensure moreeffective and independent audit activities, orientthe work of the auditors, challenge their work, andlend support to internal audit activities.

I N D E P E N D E N C E A N D AC C O U N TA B I L I T Y

Governments in Canada at both the federaland provincial levels are usually thought of asbureaucracies organized into ministries, with min-isters responsible for them. A less orderly arrange-ment typical of those governments takes the shapeof so-called government agencies, boards, commis-sions, and administrative tribunals. They may existto deliver services, often in a regulated form, but alarge number of them are regulatory agencies.

Some of them are simply advisory. For the mostpart, all of those agencies are designed to operate atarm’s length, free of political interferences. Theyare said to be independent. The question is, howare they accountable?

In the late 1980s, the government of theProvince of Ontario initiated a review under thedirection of Robert Macauley, a former conserva-tive cabinet minister, on the subject of governmentagencies, their number, their growth, and theirimportance in society. This led to the MacauleyReport.56 Macauley addressed the problem of inde-pendence as follows:

The word ‘independent’ is the wrongword to describe the relationship which infact, and in law, exists between an agency andthe Government. The proper perspective is tounderstand that agencies operate at arm’slength in their decision-making role, but areaccountable to a particular Ministry and tothe Legislature for their operations and policyfunctions.57

With a different point of view, in a report tothe Canadian Bar Association, in 1990, ProfessorEdward Ratushny reiterated the importance of anagency’s independence:

While the principle of independence ofthe judiciary is well established in Canadiansociety, the principle of independence ofadministrative tribunals is in its infancy…The difficulty is that governments want tohave it both ways. They want to establish tri-bunals to reach decisions outside government,yet at the same time they seek, in certaincases, to control those decisions, often behindthe scenes… The ‘bottom line’ is that ifParliament creates an independent tribunal oragency, then it is incumbent upon the govern-ment to respect that independence.58


After placing his emphasis on independence,Ratushny added this caution:

[T]he basic point is that the principle ofindependence does not relieve tribunals andagencies, or their members, of responsibilityand an appropriate degree of accountability.What is improper is for accountability to beto government when it impinges upon theirindependent fact-finding, policy developmentor decision-making functions.59

The above quotations, as well as those thatfollow under this rubric, are part of an article onthe debate of independence and accountabilitywith respect to Ontario’s agencies.60 Donald C.MacDonald, chairman of the Ontario Commissionon Electoral Finance, remarked on the differencebetween the view of the courts and that of thenonjudiciary: “There is a point where compromiseis not possible if agencies are to remain the cre-ation of the legislature, established for implementa-tion of legislatively authorized policy and thereforedependent on the legislature.” The HonourableFlora MacDonald had this to say regarding legisla-tion on the Canadian Radio-television andTelecommunications Commission (CRTC):

While conceding that regulatory agen-cies including the CRTC require considerableindependence, reluctantly it is the govern-ment which must bear the responsibility forthe actions of the CRTC. The CRTC’s inde-pendence cannot be so great that it isaccountable to no one.61

Rosalie Abella, then chair of the OntarioLabour Relations Board, commented as followsconcerning the character of the relationshipbetween agencies and the government and itsbureaucracy:

Ministers and bureaucrats find them-selves in the most awkward position. Because

we are quasi-judicials, they expect indepen-dence as the courts know it. Because we arepolicy instruments, governments too oftenexpect us to be accountable for our decisionsas either reflective or destructive of their poli-cy objectives. But it must be clearly andforcefully understood that while governmentshave the right, through law and regulations,to design the framework policy, tribunalshave the exclusive responsibility for interpret-ing the application of that policy in each casein accordance with their statutes… Thebureaucracy thus, in our policy role, sees uscloser to the government; the lawyers, in ourlegal decision-making role, see us closer to thecourts. We are spiritually closer to both but,in fact, in character we are closer to andshould be treated more like courts. The policyrole gives us flexibility and wide discretion,but it does not make us a government depart-ment. We are meant to replace bureaucraticdecision-making, not to provide a parallelroute. And the legal role, while it gives us aduty to behave with procedural fairness andwithin our jurisdiction, does not oblige us tobe procedural mimics of the courts.62

Donald C. MacDonald concludes: “The issue ofindependence and accountability is beset with tensionsbetween the courts and the agencies, and between thegovernment and their bureaucrats and the agencies.The challenge is to keep them in balance.”63

N E U T R A L I T Y

Neutrality can be seen as an aspect of inde-pendence. In day-to-day parlance, neutrality meansnot taking sides. In the context of accountability,being neutral means that lines of investigation,information gathering, reaching conclusions, andpresentation of results ought to be free of bias, per-


sonal and otherwise, and not intended to achievesome predetermined purpose or to reinforce a par-ticular orthodoxy.64 Neutrality is the ability to dothe work expertly, and to do it according to explic-it objective standards, free of personal, party orother obligations and loyalties.65

P O L I T I C A L N E U T R A L I T Y

Political neutrality describes the behaviour ofpublic servants in their support of the governmentof the day and the people who were elected to leadit. It is “a feature of bureaucracy whereby it carriesout directives of other institutions of government(such as the chief executive or the legislature),without acting as a political force in its own right;a traditional notion concerning bureaucratic behav-ior in Western governments.”66

The concept goes back a long time and is stillappropriate today. In the United Kingdom, politi-cal neutrality of the civil service became a require-ment in 1855 when the Civil Service Commissionwas established. The United Kingdom’s CivilService Pay and Conditions of Service Code con-tains the following statement about civil servantsand political activities:

Civil servants owe their allegiance to theCrown. In its executive capacity, the authorityof the Crown is exercised by the Governmentof the day. Civil servants are thereforerequired to discharge loyally the dutiesassigned to them by the Government of theday of whatever political persuasion. For theCivil Service to serve successive governmentsof different political complexions it is essen-tial that ministers and the public should haveconfidence that civil servants’ personal viewsdo not cut across the discharge of their offi-cial duties.

The intent of the rules governing politi-cal activities by civil servants is to allow them

the greatest possible freedom to participate inpublic affairs without infringing these funda-mental principles. The rules are concernedwith political activities liable to give publicexpression to political views, rather than theprivately held beliefs and opinions.67

IT IS THE MINISTER’S BUSINESS TO COMPASS AND IMAGINE

SOCIAL AMELIORATION, ECONOMIC REFORMS, AND DIPLOMAT-

IC PATTERNS. FROM THE PERMANENT STAFF WE ASK THE CRITI-

CAL MIND WHICH CAN DISTINGUISH CAUSES AND CONSE-

QUENCES, DISTANT AS WELL AS IMMEDIATE REPERCUSSIONS

AND RELATIONSHIPS, AND THE DETERMINATION OF WAYS AND

MEANS. THIS RELATIONSHIP… GIVES POINT… TO THE NEED

FOR THE ANONYMITY OF OFFICIALS. THEIR VIEWS, THEIR

ADVICE, ARE PRIVATE; THEIR ACTIONS ARE ANONYMOUS. ONLY

THE MINISTER HAS VIEWS AND TAKES ACTION. IF THIS CON-

VENTION IS NOT OBEYED, THEN CIVIL SERVANTS MAY BE PUB-

LICLY ATTACKED BY ONE PARTY AND PRAISED BY ANOTHER,

AND THAT MUST LEAD TO A WEAKENING OF THE PRINCIPLE OF

IMPARTIALITY.68

Many commentators think that unless theprinciple of impartiality, or neutrality, is main-tained, ministers would be unable to rely on civilservants for frank and full advice. This notion is animportant element in the argument for the meritprinciple in the public service.

A U T O N O M Y

Autonomy is a lesser form of independence,although it is often used to mean the same thing.It is the context that gives rise to its particularmeaning. As a country, Canada is independent andsovereign. It is not autonomous. The provinces,however, are autonomous within Canada, but notindependent. Similarly, Canada viewed as part ofthe Commonwealth is autonomous. Autonomy isthe ability to be self-governing within a larger


framework of governance. In a government setting,some boards with regulating powers (as the onesfound in transportation, broadcasting, and so on)are said to be autonomous, not independent.Within government policy and legislation, suchboards exercise their own judgment in renderingindividual decisions.

SOVEREIGNTY

“WE, THE PEOPLE.” THE PRINCIPLE IS THAT WE, THE PEOPLE,

CREATE THE GOVERNMENT AND ONLY WE, THE PEOPLE, CAN

IMPORTANTLY CHANGE IT.69

S U B S I D I A R I T Y

Not far from the notion of autonomy is sub-sidiarity. Recently, the term has been used in conti-nental Europe largely in relation to the CommonMarket, in response to the management philoso-phy given to devolution, deconcentration, decen-tralization, and delegation.

SUBSIDIARITY IS AN OLD CONCEPT, ARTICULATED BY ARISTOTLE

THROUGH HIS PHILOSOPHY REGARDING THE NEED TO PLACE

LIMITATIONS ON THE EXERCISE OF POWER, AS WELL AS IN THE

THIRTEENTH CENTURY BY THOMAS AQUINAS WHO BELIEVED IN

THE SELF-ESTEEM AND DIGNITY OF THE INDIVIDUAL. THE CON-

CEPT WAS CLARIFIED WITH THE ENTRENCHMENT OF THE BILL

OF RIGHTS IN ENGLAND IN 1689: INDIVIDUALS ARE SOVEREIGN,

AND ONLY WHEN THEY ARE INCAPABLE OF PROVIDING ESSEN-

TIALS FOR THEMSELVES SHOULD THEY RESORT TO A COLLECTIVE

ACTION AND HAVE THE COMMUNITY ASSUME THE RESPONSIBILI-

TY ON THEIR BEHALF. THE BELIEF IS THAT DECISIONS MUST BE

TAKEN AT THE LOWEST POSSIBLE LEVEL. ONE USAGE OF THE

WORD IS TO DESCRIBE THE AGREEMENT BY WHICH EACH LEVEL

OF GOVERNMENT ASSUMES RESPONSIBILITIES AND POSSIBLY IS

GIVEN THE AUTHORITY IN AREAS WHERE IT IS MORE COMPE-

TENT, OR WHERE, TO USE AN EXPRESSION FOUND IN ECONOM-

ICS, IT HAS A COMPARATIVE ADVANTAGE.

The principle of subsidiarity is largely propa-gated to reassure member countries of theEuropean Community (EC) that they ought notfear some bureaucratic monstrosity concentrated inthe headquarters in Brussels. The concept is natu-rally defended and promoted by countries with afederal system, Germany, for instance.70

The concept of federalism rests essen-tially on two principles called subsidiarity andnon-subordination. According to the princi-ple of subsidiarity, the central governmentshould be entrusted with only those activitieswhich cannot be managed fairly and effective-ly at the local level. According to the princi-ple of non-subordination, neither of the twolevels of government is subject to the authori-ty of the other in the exercise of the sovereignpowers granted by the Constitution. Hencethe necessity of a mechanism of concertedaction, harmonization, joint decision-makingto manage the problems of interface betweenthe two levels of government.71

S T E W A R D S H I P

The word steward—a person entrusted with acertain responsibility, usually material—precededstewardship. To the accounting profession, steward-ship is the obligation to report on safe custody orproper disposition of assets entrusted. The word isevocative not only of the nature of the trust butalso the necessity to render an account.

The Auditor General of Canada, in his 1992Report to the House of Commons, introduced theexpression “global stewardship”: “a regular account-ing on the entire business of government…” toreflect the full range of departmental activities. Hethen identified three types of information of vitalimportance for maintaining the confidence ofParliament:


Stewardship information: comprisesinformation on the broader duties and obliga-tions to preserve, maintain, and foster thegovernment enterprise, seen from a macroperspective. Reporting on stewardship bysenior management of an organization meansarticulating its mission in a manner thatenables it to determine its success.

Financial information: reflects all of adepartment’s financial activities, even thosethat do not require parliamentary spendingauthority.

Operational information: includes moredetailed information in a variety of areas andis clearly linked to the financial information.Both financial and operational informationflow out of stewardship information.72

RECIPROCAL RESPONSIBILITY

THIS TERM WAS USED BY PRESIDENT BILL CLINTON DURING

THE ELECTION CAMPAIGN TO DEFINE THE NEED TO REBUILD A

SENSE OF COMMUNITY AS WELL AS A SENSE OF INDIVIDUAL

RESPONSIBILITY IN RECIPIENTS OR BENEFICIARIES OF STATE

SOCIAL PROGRAMS THROUGH PUBLIC SERVICE.

V I C A R I O U S R E S P O N S I B I L I T Y

We do not hear the term vicarious responsibilityvery often in public administration, but it has becomea topic of interest in the health care system, particu-larly in hospitals. It is widely believed—and with con-siderable justification—that a hospital could be vicari-ously liable to a patient for the conduct of any staffmember employed by the hospital. This should notbe confused with the direct liability of the hospital, asthe hospital itself may not have been negligent in itsconduct, and may have exercised due care in all cir-cumstances. Nevertheless, it assumes responsibility forthe negligence of its employees. These notions willgradually find their ways into courts.

M A N A G E M E N T A N D

A D M I N I S T R A T I O N

While as a word, management has enoughcurrency to convey its meaning simply by utter-ance, when related to the word administration, inparticular public administration, it loses some of itsconvenience to summarize what people do whenthey are in charge. In other words, managementneeds the context to reveal what it actually does.For instance, public sector management is not thesame as managing the public service. Indeed, theterm public sector management is sometimes usedin lieu of public administration. To many, the for-mer is deemed more encompassing than the latter.The higher echelons of many government and cor-porate hierarchies are referred to as the manage-ment category for purposes of classifying positionsaccording to levels of responsibilities.

Manager and administrator, management andadministration, are often used interchangeably. InCanada, it is becoming accepted usage that man-agers are expected to achieve the mission of theirorganization and deliver programs, and in the caseof very senior managers, advise on policy objec-tives, while administrators, on a lower rung of thehierarchy, are to apply and execute the rules of thebureaucracy and maintain the capability of theentity to deliver the program. By the usual rules ofsemantics, public administration would logically bewhat public administrators do. In a generic conver-sation, this happens to be true. In narrower con-texts, however, administrators are to be foundunder managers. Often, administrators are attachedto managers, but in a subordinate role with a titlesuch as administrative assistant instead of assistantmanager, indicating that the incumbent is con-cerned with only one part of the manager’s role.Simultaneously, however, in the United Kingdom,the administrative class is the highest echelon ofthe civil service.


The use of the word manager in the publicsector is relatively recent. It was borrowed from theprivate sector, which used the word to give a titleto those functions that were not backed up by aprofessional distinction, as is the case for the chiefaccountant, chief engineer, chief medical officer,and so on. That usage gave rise to the productionmanager, sales manager, marketing manager, officemanager.

Administration is often used in the specificcontext of stewardship: “I administer the estate ofso and so.” Administration is also another word forgovernment, often preferred to the latter, particular-ly in the United States where executive power isexercised by “the administration,” which describesthe political executive branch of government. Theword can be used to embrace the permanentbureaucracy, as well.

M A N AG E M E N T A N D L E A D E R S H I P

The simplest meaning of leadership is “tohave a following.” This is not necessarily true ofmanagement, which to a large extent calls for obe-dience and compliance.

Management is the capacity to handle multi-ple problems, neutralize various constituencies andwork within a budget. Leadership is essentially amoral act, not—as in most management—anessentially protective act. It is the assertion of avision, not simply the exercise of style: the moralcourage to assert a vision of the institution and theintellectual energy to make that vision compelling.

A. Bartlett Giametti, the late president of YaleUniversity and commissioner of baseball, theauthor of that interesting distinction, made it inthe context of the elimination of athletic programsin his university. He concluded the above observa-tion by saying:

“Ultimately, the administration’s deci-sion to cut athletic programs reflects animpoverished view of the university, one thatconfuses management with leadership.”73


46 GUY BENVENISTE, PROFESSIONALIZING THE ORGANIZATION, REDUCING BUREAUCRACY TO ENHANCE EFFECTIVENESS (SAN FRANCISCO: JOSSEY-BASS, 1987), 10. 47 THE SUBSTANCE OF THIS SECTION HAS DRAWN ON “SCHOOLS BRIEF, RULES V. DISCRETION,” THE ECONOMIST, 2 MARCH 1991, 71.48 SEE “RYAN REMPORTE EN APPEL SA BATAILLE RÉFÉRENDAIRE CONTRE DEUX MUNICIPALITÉS,” LE DEVOIR, 3 NOVEMBER 1992, A-3.49 GEORGE J. GORDON, PUBLIC ADMINISTRATION IN AMERICA, 3D ED., (NEW YORK: ST. MARTIN’S PRESS, 1986), GLOSSARY.50 IBID., 6.51 HERSCHEL HARDIN, IN THE NEW BUREAUCRACY (TORONTO: MCCLELLAND & STEWART, 1991), 14.52 RICHARD S. PAGE, “THE IDEOLOGICAL-PHILOSOPHICAL SETTING OF AMERICAN PUBLIC ADMINISTRATION,” IN PUBLIC ADMINISTRATION IN A TIME OF TURBULENCE, DWIGHT WALDO, ED.

(SCRANTON, PA.: CHANDLER, 1971), 63. 53 JAMES CUTT, “HIGHER LEARNING IN ACCOUNTABILITY,” CAMAGAZINE, AUGUST 1993, 47-48. 54 RICHARD LEDERER, “LANGUAGE-HATS,” 128.55 TREASURY BOARD SECRETARIAT, GOVERNMENT REVIEW AND QUALITY SERVICES DIVISION, INTERNAL AUDITING STANDARDS IN THE GOVERNMENT OF CANADA (OTTAWA: TREASURY BOARD

SECRETARIAT, MAY 1994).56 ROBERT MACAULEY, DIRECTIONS—REPORT ON A REVIEW OF ONTARIO’S REGULATORY AGENCIES (TORONTO: QUEEN’S PRINTER FOR ONTARIO, 1989).57 IBID., 2-17.58 EDWARD RATUSHNY, “REPORT ON THE INDEPENDENCE OF FEDERAL ADMINISTRATIVE TRIBUNALS AND AGENCIES,” PRESENTED TO THE CANADIAN BAR ASSOCIATION, SEPTEMBER 1990, 6-7.59 IBID., 79.60 DONALD C. MACDONALD, “ONTARIO AGENCIES, BOARDS, AND COMMISSIONS COME OF AGE” IN CANADIAN PUBLIC ADMINISTRATION, VOL. 36, NO. 3, FALL 1993, 349.61 ROBERT MACAULEY, PRACTICE AND PROCEDURE BEFORE ADMINISTRATIVE TRIBUNALS, VOL. 2 (TORONTO: CARSWELL, 1991), 32-39.62 ROSALIE ABELLA, “CANADIAN ADMINISTRATIVE TRIBUNALS: TOWARDS JUDICIALIZATION OR DEJUDICIALIZATION,” ADDRESS DELIVERED TO THE FOURTH ANNUAL CONFERENCE OF

CANADIAN ADMINISTRATIVE TRIBUNALS, MAY 29-JUNE1, 1988, PUBLISHED IN CANADIAN JOURNAL OF ADMINISTRATIVE LAW AND PRACTICE NO. 2 (1988-89), 6-8.63 MACDONALD, ONTARIO AGENCIES, 359.64 ROSS M. SKINNER, ACCOUNTING STANDARDS IN EVOLUTION (TORONTO: HOLT, RINEHART, AND WINSTON OF CANADA, 1987), 643.65 HERBERT KAUFMAN, “EMERGING CONFLICTS IN THE DOCTRINES OF PUBLIC ADMINISTRATION,” IN AMERICAN POLITICAL SCIENCE REVIEW, VOL. 50, DECEMBER 1956, P. 1960.66 GORDON, PUBLIC ADMINISTRATION, 608.67 HENNESSY, WHITEHALL, 368.68 HERMAN FINER, THE BRITISH CIVIL SERVICE (LONDON: THE FABIAN SOCIETY AND GEORGE ALLEN AND UNWIN, 1937), 190.69 DWIGHT WALDO, PUBLIC ADMINISTRATION, 33. “WE THE PEOPLE . . .” ARE THE FIRST THREE WORDS OF THE CONSTITUTION OF THE UNITED STATES OF AMERICA.70 CHANTAL MILLON-DELSOL REVIEW OF L’ETAT SUBSIDIAIRE, BY PIERRE DROUIN, LE MONDE, 22 FEBRUARY 1992; SEE ALSO LE DEVOIR, 21 OCTOBER 1992.71 ANDRÉ BURELLE, FORMER CONSTITUTIONAL AIDE TO PRIME MINISTER PIERRE ELLIOTT TRUDEAU, IN AN ARTICLE IN LE DEVOIR, 3 MAY 1993. 72 REPORT OF THE AUDITOR GENERAL OF CANADA TO THE HOUSE OF COMMONS, 1992, 167-168.73 A. BARTLETT GIAMETTI, AS CITED BY JEAN EDWARD SMITH IN AN ARTICLE, “SOUND BODY, SOUND MIND,” GLOBE AND MAIL, JANUARY 22, 1993.

S E C T I O N 2

ACCOUNTABILITY–ISSUES & PRACTICE

P A R T I . A C C O U N T A B I L I T Y 4 3

C H A P T E R 4

THE MEANING OFACCOUNTABILITYA C C O U N T A B I L I T Y D E F I N E D

Accountability is the obligation to render anaccount for a responsibility conferred.74

Sometimes monitoring is associated withaccountability while the responsibility is dis-charged, as well as the notion of rewards and sanc-tions in the discharge of such responsibility.

Accountability is also frequently linked withunfortunate outcomes: when things do not go well,the person responsible who refuses to take theblame is often criticized for not being accountable.Curiously, the person who takes credit for a gooddeed is not usually described as being accountable.

Accountability is generally considered to be avery positive democratic value; it has beenviewed—in liberal societies, with a characteristicdistrust of large concentrations of power—as anantidote, a countervailing power to the bureaucra-cy of governments and big business. And since theconcept is used in a wide variety of organizations,the arrangements for achieving it vary dependingon the circumstances.

A C C O U N T A B I L I T Y —

M A N Y C O N T E X T S ,

M A N Y F O R M U L A T I O N S

Many writers have explored this subject, andit is instructive to examine some of their thoughts.The following quotes illustrate the range ofthought given to accountability:

Accountability [is] an obligation on thepart of an individual or group to reveal, toexplain, and to justify the discharge of

responsibilities whose origins may be politi-cal, constitutional, statutory, or contractual.75

Public accountability involves threeinterrelated groups: (a) the general public andparticularly the recipients of public serviceswho are interested in service providers beingaccountable to them; (b) political leaders andsupervisors of service providers to be account-able for a mixture of public policy and privateand parochial interests; and (c) the serviceproviders themselves whose objectives andinterests often differ from the first two.76

Accountability is the working principleof our parliamentary system and a processwhose effective functioning is essential to ourdemocratic government. The reality of thatsystem is expressed through universal suffrageand popular representation in Parliament.77

Accountability is the essence of ourdemocratic form of government. It is the lia-bility assumed by all those who exerciseauthority to account for the manner in whichthey have fulfilled responsibilities entrusted tothem, a liability ultimately to the Canadianpeople owed by Parliament, by theGovernment and, thus, every governmentdepartment and agency.78

Nothing could be simpler than the theo-ry of parliamentary accountability. Its essenceis ministerial responsibility, which means thateach minister is responsible to Parliament forthe conduct of his Department. The act ofevery civil servant is by convention regarded asthe act of his minister.79

Accountability involves an obligation toexplain or justify specific actions. Stanyer hasstressed that there is always a precise logicalstructure involving the form and substance ofthe account; the occasion in terms of time,place and audience; and the consequences.Such a logical structure attaches to the exer-


cise of accountability in any context.80

When systems of accountability for pub-lic expenditures are being devised, three broadissues must be considered: (a) the type ofaccountability; (b) the techniques of measure-ment; and (c) the institutions to whichaccount is rendered. This framework providesa helpful structure for organizing discussionalthough there are interactions. For example,the type of accountability may determine bothtechniques of measurement and audience.81

Accountability [is] a political principleaccording to which agencies or organizations,such as those in government, are subject tosome form of external control, causing themto give a general accounting of and for theiractions; an essential concept in democraticpublic administration.82

The basic underlying framework ofaccountability holds democracies togetherunder the most difficult times… It gives peo-ple a fundamental faith in the integrity oftheir political institutions…83

[Police officers should remember that]being at the service of citizens, they areaccountable to the people, through the inter-mediary of elected officials and politicians…this game of democracy… is essential if we donot want to run the police for the police…regenerating rapidly into a fascist state, creat-ing a police state.84

TO BE EFFECTIVE, ACCOUNTABILITY IN THE SENSE THAT IT

HAS TO DO WITH INFORMATION, THE RENDERING OF

ACCOUNT, INVOLVES THE ANSWER TO A FEW BASIC QUESTIONS:

• HOW DO YOU ORDAIN THE RENDERING OF ACCOUNT?

• THROUGH WHAT MECHANISM OR PROTOCOL?

• WHO HAS THE RIGHT TO DEMAND THE INFORMATION?

• WHEN, FROM WHOM, AND ABOUT WHAT?

[A]ccountability is a synonym forresponsibility. It is a type of relationship thatcomes to existence when an obligation istaken on by an individual (or corporate enti-ty), such as the responsibility to assume a roleor discharge a task.85

The concept of accountability is some-times taken as merely another name for stew-ardship accounting… Stewardship is theobligation to report on safe custody or properdisposition of assets entrusted… The termaccountability, in contrast, focuses attentionon identification of those parties entitled toan accounting and the purposes for whichthey are presumed to use the accounting.86

The missing link all along has beeneffective accountability for the use of authori-ties for which people have been entrusted…asPublic Service 2000 simplifies the PublicService’s administration, and more and morestress is placed on results-oriented and client-sensitive culture, the importance of effectiveaccountability is going to become corre-spondingly greater.87

Each manager will be expected to havean agreed statement of anticipated results andperformance standards.88… Each level ofmanagement will be accountable for resultsachieved…89

Accountability is the obligation of adeputy minister to answer to a person orgroup for the exercise of responsibilities con-ferred on him or her by that person orgroup… Management responsibility is therequirement for deputy ministers to respondto the concerns of individuals or groups with-in the overall context of their accountabilityobligations… Answerability is the obligationof deputy ministers to provide informationand explanations to Parliament on behalf oftheir ministers and the government.90

P A R T I . C H A P T E R 4 . T H E M E A N I N G O F A C C O U N T A B I L I T Y 4 5

INDIVIDUAL VERSUS

INSTITUTIONAL ACCOUNTABILITY

SOME PEOPLE ARGUE THAT IT IS THE INSTITUTION AS A WHOLE

THAT SHOULD BE HELD RESPONSIBLE, NOT THE INDIVIDUAL.

CONVENTIONAL WISDOM HAS IT, HOWEVER, THAT IN THE

FINAL ANALYSIS ONLY INDIVIDUALS CAN BE MADE RESPONSIBLE

AND ACCOUNTABLE BECAUSE ONLY THEY ARE ENTRUSTED WITH

SPECIFIC RESPONSIBILITIES. IF AN INSTITUTION IS NOT AS

ACCOUNTABLE AS IT SHOULD BE, IT IS BECAUSE THE INDIVID-

UAL OR INDIVIDUALS IN CHARGE PREVENT IT FROM BEING SO.

THAT DEBATE HAS NEVER BEEN ENTIRELY RESOLVED IN A CON-

VINCING MANNER BECAUSE THE ISSUE IS USUALLY POSED IN

THE FORM OF A DILEMMA.

The following definition, formulated by PaulC. Light, a prominent scholar, has enjoyed curren-cy over the years—and still does, particularly in theUnited States:

Despite experiments with performanceincentives, such as merit pay, and occasionalinvestments in civil service reforms, the defin-ition of accountability in government hasremained relatively constant over the pastfifty years: limit bureaucratic discretionthrough compliance with tightly drawn rulesand regulations.91

This definition emphasizes the constraintsimposed on people in a position of public trustand may well help promote a healthy responsibleattitude, but it does not make explicit the obliga-tion to render an account. What makes this defini-tion appear old-fashioned is its emphasis on gov-erning and managing according to rules, honesty,and with a minimum of waste. In the words ofProfessor Light, it is a command-and-controlapproach to public administration, possibly in con-flict with currently fashionable values such as ini-

tiative, creativity, innovation, and the taking ofrisks. Light also cites two other short definitions ofJames Fesler and Donald Kettl, the result of divid-ing accountability into two dimensions:

One is accountability; faithful obedienceto the law, to higher officials’ directions, andto standards of efficiency and economy.

The other is ethical behavior: adherenceto moral standards and avoidance even of theappearance of unethical actions.92

The following definition also does not makeexplicit the need to render an account and seemsto be the product of different values. It introducesthe notion of punishment, which serves as a con-stant reminder for the need to behave responsiblyand honestly. Most cultures use reward and pun-ishment as strong motivators to guide action:

[Accountability] is found where rulersreadily delegate authority, where subordinates


CURIOUSLY, SOME TIME AGO, AN INTERNATIONAL CON-

FLICT LED TO A RETALIATION OF ONE COUNTRY AGAINST

ANOTHER. THE INITIATIVE WAS CALLED “OPERATION

ACCOUNTABILITY.” IT IS NOT KNOWN WHY THE NAME

WAS CHOSEN; ONE CAN ONLY SPECULATE. THE WORD

ACCOUNTABILITY HAS A CERTAIN AURA OF LEGITIMACY,

OF DUTY. THE LAUNCHING OF THE RETALIATION—JUSTI-

FIED OR NOT—MAY HAVE COME FROM A FELT OBLIGA-

TION TO ACT OR AN ATTITUDE TO THAT EFFECT. OR IT

MAY HAVE BEEN DIRECTED AT THE ENEMY, “NOW IS THE

TIME TO ACCOUNT,” “BRING TO ACCOUNT,” OR TO “SET-

TLE AN ACCOUNT,” HENCE GIVING ACCOUNTABILITY A

PUNITIVE MEANING: “THOSE WHO TAKE UP ARMS

AGAINST US MUST PAY THE PRICE FOR THEIR IMPU-

DENCE,” AS SOMEONE SAID. ONE CAN SPECULATE THAT

OPERATION ACCOUNTABILITY WAS THE TRANSLATION OF

SOMETHING CLOSER TO “RECKONING” OR

“COMEUPPANCE.”

confidently exercise their discretion, wherethe abuse of power is given its proper name,and is properly punished under a rule of lawwhich stands above political faction.93

Still another definition links accountability toperformance directly, in the sense of results andeffectiveness; it does not mention the resort tosanctions to promote accountability, and it has aninteresting leading observation:

In the vocabulary of management, peo-ple “accountability” threatens to become oneof those buzz words that can take on whatevermeaning a speaker or writer wishes to conveywithout strictly committing himself.Nevertheless, when expressed with care andprecision, accountability is an invaluable con-cept for focusing organizations and individualsdirectly on established goals and objectives.

Accountability includes mission state-ments, responsibility assignments, resultsmeasurements, reporting, and evaluation. Itpromotes disciplined use of management byobjectives and other executive tools. It hasimpact from the top level of an organizationto the lowest managerial rung…

Accountable organizations and personsknow their purposes and responsibilities andare able to differentiate between essentials andnonessentials. Accountability means beingresponsible for and responsive to acts andresults that relate to the mission. To attainresults is to realize the mission, or objectives,of the organization.94

The above quotes define accountability and,in most cases, set out its boundaries. Let us addone more illustration, which served as the inspira-tion for the short definition that appears at thebeginning of this chapter. The Report of the

Independent Review Committee on the Office ofthe Auditor General of Canada (WilsonCommittee) defined accountability as :

[T]he obligation to answer for a respon-sibility that has been conferred. It presumesthe existence of at least two parties: one whoallocates responsibility and one who accepts itwith the undertaking to report upon themanner in which it has been discharged.95

The Wilson definition—or the meaning itgenerally conveys—has gained wide acceptance inthe last decade and is the one this book follows.

T H E R O O T O F

A C C O U N T A B I L I T Y

In the definitions given above there are threeseparate but related notions: responsibility,accountability, answerability. These words, particu-larly responsibility and accountability, are often usedinterchangeably. They nevertheless convey differentmeanings.96

AS A WORD IN OUR DAY-TO-DAY WORK, ACCOUNTABILITY HAS

GAINED CURRENCY ONLY RECENTLY IN SPITE OF THE FACT

THAT THE TERM IS OLD AND THE NOTION BIBLICAL. “WHAT

HAVE YOU DONE WITH YOUR TALENTS?”—A QUESTION,

ACCORDING TO THE APOSTLES, THAT WILL BE ASKED OF ALL

MORTALS BY ST. PETER, IS A VERY EARLY INDICATION OF THE

NEED TO PROVIDE ACCOUNTABILITY FOR ONE’S ACTIONS.

In trying to understand these different terms,it is easiest to distinguish between them on thebasis of attitudes. Feeling responsible is the primarynotion. It starts with the sense that you owe it toyourself to discharge your role according to expec-tations. Failing this, feeling responsible means thatat least you owe it to those who have assigned youyour role to act according to their expectations.


The notion of responsibility is as old as theconcepts of power, authority, and freedom, andacquires meaning only when viewed in the contextof those concepts. In the absence of power, author-ity, or freedom of action, feeling responsible is notpossible, nor even necessary. It is a counterpart tofreedom or counterweight to power. There are noparticular rules or laws to impose a sense ofresponsibility. Instead, it emerges from individualsaccording to their culture, beliefs, and values—inshort, their humanity—and in line with the poweror authority they have been given.

The sense of responsibility is not equallydeveloped in everyone. Basic urges—for love,revenge, justice and glory, food, shelter and cloth-ing, success and security—all wrestle for satisfac-tion within the human heart. Self-interest is miti-gated by a recognized need to benefit from the“common good,” the advantages that come frombeing a part of an ordered society. The aggregation,or the netting out of all these primary urges withineach of us—some noble, some less noble—definesour character. It takes a certain amount of civilizedmaturity to acquire and maintain a sense ofresponsibility. It is largely inherited or developedfrom one’s culture and upbringing. It is inevitablethat some people are more responsible than others.

Being accountable starts with an attitude.Becoming accountable is a natural extension offeeling responsible by formalizing the notion into aprocedure. This technical modification, the act ofaccountability, constitutes the fundamentalacknowledgement of responsibility. You feel thatyou have to render an account to someone in par-ticular on the responsibilities conferred; you are ina situation of accountability.

This attitude needs a conducive environmentto flourish. The person conferring the responsibili-ty has to expect accountability; the person who hasbeen given the responsibility must be prepared torender an account.

THE PHILOSOPHY [UNDERLYING ACCOUNTABILITY, MAN-

AGEMENT REPRESENTATIONS, AND AUDIT]… IS AN

IMPORTANT ONE AND BEARS REPEATING. IT PLACES UPON

MANAGEMENT THE RESPONSIBILITY, AND HENCE THE

ACCOUNTABILITY, FOR ITS ACTIONS. IT IMPLIES A LEVEL

OF TRUST BETWEEN MANAGEMENT AND THE GOVERNING

BODIES. AND ULTIMATELY, IT BOILS DOWN TO A SENSE OF

VALUES THAT REQUIRES MANAGEMENT TO INTERNALIZE A

SENSE OF ACCOUNTABILITY. WITHOUT THE INTEGRATION

OF THIS VALUE SYSTEM, ALL THE SYSTEMS AND PRACTICES

IN THE WORLD WILL NOT BE SUFFICIENT TO PROTECT

THE INVESTMENT OF PUBLIC FUNDS.97

A situation of accountability is generated eachtime a rendering of account is demanded or volun-teered. That rendering of account, even if it occursonly occasionally, establishes the accountabilityrelationship.

The central feature of a democratic govern-ment is holding elected representatives to accountfor their actions. This constitutes a highly visibleaccountability situation. Modern governments aresupported by bureaucracies staffed with nonelectedpersonnel entrusted with the substantial responsi-bility of administration. The requirement foraccountability on the part of the bureaucracy is asvital as for elected politicians, but, because of therelative anonymity of bureaucrats, its fulfillment isless visible and more difficult to ensure.

Professor C.E.S. Franks, a noted political sci-entist, makes an interesting distinction between theconcepts of objective accountability and subjectiveaccountability:

In objective accountability, someone isresponsible for something and accountable to someperson or body in a formal way, through clearlydefined rules and mechanisms.

In subjective accountability, a person feels aduty towards the profession of public service or a


sense of the public good and the nation whichdetermines and defines conduct even though thereare no formal mechanisms through which thisaccountability can be enforced.98

The use of the word answerable in the litera-ture of public administration is even more recent

than the use of accountability and is probablyfound less frequently. Often it is a synonym foraccountable. Elsewhere it describes the onus onsenior public servants to answer questions by leg-islative committees while not being accountable to them.


S O M E H I S T O R I C A L N O T E S

SOME BELIEVE THAT RESPONSIBILITY AND ITS FORMALIZATION INTO ACCOUNTABILITY ORIGINATED WITH THE CREATION

OF MODERN STATES. THE MAGNA CARTA ISSUED IN 1215 BY KING JOHN OF ENGLAND UNDER PRESSURE FROM THE FEUDAL

BARONS WAS A CHARTER OF LIBERTIES. IT WAS INTENDED TO PREVENT THE KING FROM USING HIS ROYAL PREROGATIVE ARBI-

TRARILY TO INCREASE FEUDAL DUES. EXACTING ACCOUNTABILITY FROM THE KING BY HIS SUBJECTS WAS AN ATTEMPT TO

INSTILL A SENSE OF RESPONSIBILITY. IT IS OBVIOUS THAT KING JOHN DID NOT HAVE IT IN HIM AS HE SOON REPUDIATED THE

CHARTER. HIS SUCCESSOR, HENRY III, HAD A MORE NOBLE DISPOSITION AND REISSUED THE CHARTER IN 1216. OTHER KINGS

RECONFIRMED THE CHARTER IN SUBSEQUENT REIGNS. INTERESTINGLY, THE CONCEPT OF TAXATION WITHOUT REPRESENTATION

INTRODUCED IN THE SEVENTEENTH CENTURY WAS READ AS AN INTERPRETATION OF THE MAGNA CARTA.

BEFORE THE GLORIOUS REVOLUTION OF 1688, WHICH LED TO THE BILL OF RIGHTS OF 1689, THE INTERPRETATION

OF THE MONARCHY AS A DIVINE RIGHT TOOK PRECEDENCE OVER THE AUTHORITY OF THE PARLIAMENT. THIS MEANT THAT

THE MONARCH FELT RESPONSIBLE BUT ONLY TO GOD FROM WHOM HE HAD RECEIVED HIS AUTHORITY. AS A MATTER OF

FACT, THE EARLIER CHALLENGE OF THE BARONY AND CHURCHMEN IN THE THIRTEENTH CENTURY HAD TO DO WITH REPLAC-

ING A DIVINE RIGHT WITH A CONSTITUTIONAL AND LATER A PARLIAMENTARY MONARCHY.

CLOSER TO HOME, MELANCTON SMITH, A PROMINENT NEW YORK CITY MERCHANT AND AN ANTI-FEDERALIST, SIDED

WITH THE PROPOSED U.S. CONSTITUTION, INASMUCH AS THE MONEY PUT IN THE HANDS OF GOVERNMENT HAD TO BE

ACCOUNTED FOR. THIS OCCURRED IN 1788, AT THE NEW YORK RATIFYING CONVENTION.

IN CANADA, THERE IS NOT MUCH EVIDENCE OF PARLIAMENTARIANS DEBATING THE SPECIFIC SUBJECT OF RESPONSI-

BILITY AND ACCOUNTABILITY PER SE UNTIL INTRODUCED IN A HOUSE OF COMMONS DEBATE ON MARCH 22, 1921,

REGARDING CANADIAN NATIONAL RAILWAYS (CNR). IT WAS IN THE EARLY SIXTIES AT THE TIME OF THE GLASSCO

COMMISSION THAT ACCOUNTABILITY AS A PRECISE TOPIC WAS REALLY DISCUSSED IN A DELIBERATE AND ORDERLY FASHION.

MORE HISTORICAL MATERIAL CAN BE FOUND IN THE LITERATURE OF PUBLIC ADMINISTRATION, HOWEVER SCANT IT

MAY BE. FOR INSTANCE, DWIGHT WALDO, A PROMINENT PUBLIC ADMINISTRATION SCHOLAR IN THE UNITED STATES, HAD

AN ARTICLE IN THE PROCEEDINGS99 OF A SEMINAR ORGANIZED BY L’ECOLE NATIONALE D’ADMINISTRATION PUBLIQUE. HE

EXPLORES THE HISTORICAL DEVELOPMENT OF THE NOTION OF ACCOUNTABILITY GOING BACK 6,000 YEARS! HE DOES IT BY

DEVELOPING SEVERAL “NUMBERED POINTS” WITH NO ATTEMPT, FOR LACK OF SPACE, TO CONNECT THEM. HERE IS THE GIST:

1) WHEN GOVERNMENT, THE STATE, APPEARED AS A CENTRAL PART OF THE HUMAN STORY 6,000 YEARS AGO,

ACCOUNTING AS A CONCEPT AND PROCESS BECAME AN INTEGRAL PART OF IT, FOR EXAMPLE, THE TAXING-FISCAL-ECONOMIC

MANAGEMENT APPARATUS PART OF IT. WRITTEN LANGUAGE AND MATHEMATICS ARE CLOSELY CONNECTED WITH THIS

ACCOUNTABILITY. STATISTICS HAS AS ITS ROOT THE WORD STATE.

2) THE MONARCHICAL STATES ARE INDEBTED TO THE CHRISTIAN FEUDAL MILLENNIUM (MIDDLE AGES). THEN THERE

IS THE SPELL OF ROME: “THE KING IS EMPEROR ON HIS OWN REALM.” THE ROMANS DEVELOPED ACCOUNTABILITY DEVICES

D E M O N S T R A T I N G


Those who feel accountable and answerablewill want to demonstrate their accountability. Theywill try to provide information about how theyhave been successful in discharging their responsi-bilities. Given the varied interpretations of whatperformance means and the difficulty of assem-bling credible indicators to substantiate it, such ademonstration can be very difficult.

While objectivity can be vigorously pursued,value-laden opinions will inevitably be part of theassessment of performance. This assessment willinvolve the weighting of multiple, often competingand sometimes contradictory objectives and mea-sures.100 Most people have some criteria with whichto gauge the success of an undertaking, and suchcriteria may vary over time and with changing cir-cumstances. They also vary according to the pointof view of the person assessing performance.Hence, demonstrating accountability can be a verydifficult, but not impossible, task.

E X T E R N A L I N D U C E M E N T F O R


Accountability often does not come easily;not everybody has the proper attitude, or a naturaldisposition reinforced by a strong set of personalvalues or ethics. Sometimes accountability is legis-lated, and sometimes it is promulgated as an insti-tutional value and becomes a managerial policy.Often, accountability remains merely an exhorta-tion.

Where there is an insufficient natural disposi-tion for accountability, for assumption of responsi-bility, or for a rendering of account, an externalpressure becomes necessary. It may take the formof legislation that is quite specific. Or it may be asocial pressure, quite general and ill-focused.Sometimes, the pressure takes the form of rewardsand punishment. There is a widespread belief thatsuch a regime is sound and appropriate and thatmany people humanely react to it. Nevertheless, ithas limitations.


WOVEN INTO THE ROMAN LAW. THE CONTINENTAL LEGAL SYSTEM, ABOVE ALL THE ADMINISTRATIVE LAW CODE AND

COURTS, REFLECT THIS BACKGROUND.

3) IN THE MODERN PERIOD, PRIVATE BUSINESS ENTERPRISES AND CAPITALISM EMERGE AS A SIGNIFICANT FACTOR IN

THE DEVELOPMENT OF CONCEPTS AND DEVICES OF ACCOUNTABILITY. WITHOUT CAPITALISM, THE ACCOUNTING PROFESSION

WOULD NOT HAVE RISEN AS IT DID.

4) THE HIERARCHICAL DIMENSION IS AN INTEGRAL PART OF GOVERNMENT ORGANIZATION. HISTORICALLY, GOVERN-

MENT AND RELIGION, STATE AND CHURCH, WERE ONE: AUTHORITY CAME FROM ABOVE. BUT IN MODERN TIMES, AUTHORITY

COMES FROM BELOW WITH THE RISE OF DEMOCRACY. AUTHORITY AND LEGITIMACY IN DEMOCRATIC REGIMES ARE CON-

CEIVED AS RISING FROM THE PEOPLE. RULERS, THOSE NOMINALLY AT THE TOP OF GOVERNMENT, ARE DEEMED TO BE

ACCOUNTABLE TO THE PEOPLE, AT LEAST TO THE ELECTORS. AND THE PEOPLE, ACTING ON THE THEORY OF POPULAR SOV-

EREIGNTY, MAY FEEL FREE TO ENFORCE ACCOUNTABILITY NOT ONLY INDIRECTLY VIA THE RULERS AND THE HIERARCHICAL

CHAIN BUT DIRECTLY THROUGH VARIOUS ARRANGEMENTS (FOR INSTANCE, ELECTED JUDGES AND OUTSIDE AUDITS) THAT

CREATE A DIRECT A-B-C PATTERN, OR CUTTING THROUGH THE HIERARCHICAL CHAIN.

R E WA R D S A N D P U N I S H M E N T S

“The impulses to praise what is noble and topunish what is depraved are fundamental tohuman experience.”101 This observation is readilyunderstood but conceals a long-standing debateamong philosophers and ideologists. One view hasit that punishment is needed, if only to underlinethe significance of praise. Another view looks atpunishment as retribution, or retributive justice:you get what you deserve. A third view is that asanction should not be imposed as a retributionbut as a deterrent designed to prevent harm to oth-ers or to ensure future social cooperation. Much ofthe debate focuses on the relationship betweencrime and punishment, and on that score is not sorelevant to bureaucratic behaviour that is moreafflicted by negligence, ineptitude, lack of propermotivations, misguided beliefs, or simple bungling.Nevertheless, several elements of the debate regard-ing the notion of rewards and punishment are rele-vant in a bureaucratic environment.

Depending on people’s cultural backgroundsand the values that inspire them, reactions to exter-nal inducement vary. First, for those who are natu-rally disposed to volunteer a rendering of account,an encouragement through rewards and punish-ment should theoretically be indifferent. When sit-uations are not perceived as fair, however, peoplewith a noble disposition may well modify theirattitude. Among those who are not prepared to beaccountable unless coerced, the tendency will be toassess the trade-off between reward and no reward,punishment and no punishment and, indeed,reward against punishment.

In many large institutions, particularly publicinstitutions such as government departments,rewards are difficult to arrange; often they arelargely symbolic or honorific, not tangible. Thereason is that there are usually many peopleinvolved in decision making and administering a

program, and it is difficult to pinpoint where toplace the credit for good performance.

Similarly, the creation of a system of punish-ment can be as difficult—particularly when itcomes to identifying the nature of the sanction andtargeting the individual to be punished—owing tothe diffusion of responsibility characteristic of alarge bureaucracy. The risk of attributing errors tothe wrong person is considerable. In addition,where “no one is really responsible, but everybodyis,” the individual’s complicity in collective disre-spect is less deserving of punishment than in situa-tions where each individual’s responsibilities andfailure to assume them can be clearly identified.

The process of rewarding and punishing hasto be visibly and convincingly fair and not used toachieve other purposes. In cases where there isalready an excellent climate of accountability, creat-ing a regime of rewards and punishments can beseen as insensitive and misplaced. Somehow, rewardsand punishments do not promote transparency;institutional and personal values do, and transparen-cy leads naturally to greater accountability.

Thus, while a regime of reward and punish-ment is intuitively appealing, on practical groundsit is difficult to make it an effective incentive forindividual accountability.

S A N C T I O N S W I T H O U T R E WA R D S

It is possible to envision a regime under whichpunishments overshadow rewards. This may occurwhere, although sanctions are clear, rewards are notdeemed important or appear to be unfairly granted.Rendering an account under such a regime may sufferfrom lack of comprehensiveness, pertinence, honesty,candor; moreover, it could be done to cloak ratherthan inform, even to deceive. Whatever truth remainsrisks being obfuscated. Thus, a system of rewards andpunishments will not be successful if employees con-sider that the latter overshadows the former.


PROMISE OF NO SANCTIONS

IN SOME INSTITUTIONS, VOLUNTEERING INFORMATION THAT

MAY REFLECT BADLY ON THE PERSON TAKING ACTION IS PRO-

TECTED BY A PROMISE OF NO SANCTION. THE PRINCIPLE IS

THAT NO HARM WILL ACCRUE TO THE PERSON WHO HAS VOL-

UNTEERED INFORMATION. THE BELIEF IS THAT REPORTING

HAS TO BE MADE A COMFORTABLE PROCESS TO ENSURE A CON-

STANT STREAM OF IMPORTANT INFORMATION.

THIS APPROACH IS OFTEN USED IN PROGRAM AREAS IN WHICH

SAFETY OF PEOPLE AND MATERIAL IS REGARDED AS ABSOLUTE-

LY VITAL. EXAMPLES ARE THE REQUIREMENT FOR PILOTS TO

REPORT ANY INCIDENTS AND OCCURRENCES,102 AND FOR HOS-

PITALS TO BE TOLD OF ALL PHARMACOLOGICAL ERRORS. IN

SUCH CASES, THERE IS NO REWARD, NOR IS THERE A PUNISH-

MENT, FOLLOWING THE REPORT. THIS ALLOWS FOR ONE’S

INDIVIDUAL SENSE OF RESPONSIBILITY TO BE MORE DEEPLY

FELT AND EXPRESSED. WHERE THE MISTAKE COULD HAVE

BEEN AVOIDED, THE RESULTING FEELING OF GUILT BECOMES,

IN A WAY, THE SANCTION. THE EXTERNAL CONTROL HAS BEEN

REPLACED BY A PERSONAL VALUE.

Sanctions are sometimes viewed as inappro-priate in promoting good behaviour. It is oftenseen as a juvenile approach to behaviour or rathernot quite effective with adults. That does not meanthat the opposite—rewards—is viewed in the samenegative manner.

L E G I S L AT I N G AC C O U N TA B I L I T Y

The situation is paradoxical. While account-ability starts as an attitude or a penchant, the factremains that in most instances accountability hasto be exacted, extracted, legislated; it is rarely vol-unteered.

[I]f we really want to see accountabilitydelivered on in the next decade in the publicsector, we must recognize that good account-ability will not happen by itself. We have

reached a different stage of evolution thatmakes it reasonable to shape accountabilityregimes in legislation in ways that have notbeen done in the past.103

Not surprisingly, no one… embracesaccountability voluntarily. None of us like hav-ing to account to others and it takes a certainamount of reflection about the nature of thesystems that we operate in to acknowledge thata measure of accountability is warranted.104

One argument against legislating accountabil-ity comes from the belief that most people do notreact positively to commands, which is what lawsturn out to be. They react better to inducements,benefits, privileges. Moreover, the uneven enforce-ment of a law is not much better than unconvinc-ing inducements. An excessive number of laws,rules, and regulations augments the risk of theirnot being adhered to and runs the risk of breedingcynicism towards the lawmakers.

The argument in favor of legislation is usuallythe guarantee that everybody will be treated evenly.Legislation concerning accountability is designednot just to ensure that the reluctant become com-pliant, but also to demonstrate to everyone thesolid and ethical intention of the government andlegislators, and to provide an objective and reason-able basis for dealing with mal- and nonfeasance.Legislating or establishing rules on accountabilityclarifies the accountability relationships and is anormal feature of public administration.

In the view of many people, a positive dispo-sition towards accountability must precede legisla-tion. Elected people have to be sufficiently con-vinced to demand it from administration andbecome more fully accountable to their con-stituents. Managers also need to be convinced thatwhatever they do in relation to reporting perfor-mance will be used constructively.


Assistant Auditor General of BritishColumbia, Peter Gregory, maintains:

Once the concept of accountability isaccepted, the message should be translatedinto legislation not as a means of coercion,but for the benefits that using legislation willbring. And I think there are several benefits.

One is that everybody starts at the sametime. At present, there are many that would bewilling to provide accountability, however theyare not prepared to be the first thereby violat-ing a fundamental rule in the bureaucracy.

The second benefit is that it promotes aconsistent approach. Legislation would setparameters that would ensure that all entitiesare subject to the same level of accountability.The more it is left to individual choice, themore likely that poor performers will findways to evade the accountability net.

Another benefit is endurance.Legislation is hard to change; also, it is morelikely to survive changes of political masters.

And lastly, I think public debate itself isa benefit. By involving the LegislativeAssembly in the process, a better understand-ing by other parties would result and thepublic would be well served as well.107

Regardless of legislation, however, trueaccountability requires the development of a per-sonal culture of accountability and a strongly feltdesire to have transparency in processes andactivities.


THE FRIEDRICH-FINER DISPUTE:(SUBJECTIVE ACCOUNTABILITY VERSUS OBJECTIVE ACCOUNTABILITY)

A DEBATE TOOK PLACE MORE THAN FIFTY YEARS AGO BETWEEN

TWO PROMINENT POLITICAL SCIENTISTS, CARL J. FRIEDRICH AND

HERMAN FINER.105 CARL FRIEDRICH ARGUED THAT ADMINISTRA-

TORS, IF THEY HAVE THE TECHNICAL KNOWLEDGE AND A GOOD

GRASP OF WHAT IS EXPECTED FROM THE POPULATION BEING

SERVED, WILL NATURALLY FEEL RESPONSIBLE AND ACT ACCORD-

INGLY. THIS ATTITUDE WILL BE REINFORCED BY LOYALTY TO

ONE’S OWN STANDARDS, PARTICULARLY IF HE OR SHE WAS IN THE

PROFESSIONS. HERMAN FINER DID NOT HAVE THE SAME CONFI-

DENCE IN HUMAN BEINGS. HE THOUGHT THAT SOONER OR LATER

THE ABSENCE OF EXTERNAL PUNITIVE CONTROLS WOULD LEAD TO

ABUSE OF POWER.

THE DEBATE WAS REALLY OVER WHETHER A SENSE OF RESPONSIBILI-

TY—AND BY EXTENSION, OF ACCOUNTABILITY—CAN BE ACHIEVED

BY RELYING ON THE MORAL SENSE OF THE INDIVIDUAL ADMINISTRA-

TOR (OR BY RESORTING TO INTERNAL CHECKS ONLY) OR WHETHER

IT REQUIRES SOME EXTERNAL POLITICAL CHECKS.

FRIEDRICH BELIEVED THAT PUBLIC SERVICE WAS A PROFESSION

THAT WOULD DEFINE ITS OWN STANDARDS AND ACCOUNTABILITY.

AS A MORAL PERSON, THE ADMINISTRATOR WOULD HAVE PROPER

REGARD FOR EXISTING PREFERENCES AND STANDARDS OR EXPECTA-

TIONS OF THE COMMUNITY BEING SERVED. FINER, ON THE OTHER

HAND, THOUGHT THAT RESPONSIBILITY AND ACCOUNTABILITY

OUGHT TO BE FORMAL AND DIRECT TO ELECTED OFFICIALS, TO

THE LEGISLATURE, AND ULTIMATELY TO THE ELECTORATE.

FRIEDRICH AND FINER DID NOT PUT FORWARD THEIR POINTS OF

VIEW AS ABSOLUTE; THEY RECOGNIZED THAT A COMBINATION OF

THE TWO APPROACHES WAS NEEDED. EXPERIENCE INDICATES

THAT THIS IS RIGHT; ONE APPROACH NEEDS TO BE BALANCED BY

THE OTHER.

IN OTHER WORDS, RELYING ON A SYSTEM OF EXTERNAL CHECKS,

REWARDS, AND PUNISHMENTS WOULD, TO SOME, BE REASSURING,

ALTHOUGH THAT SYSTEM WOULD NEED TO BE SO COMPLEX AS TO

BE UNWIELDY. THE ESSENTIAL POINT IS THAT WHILE MECHANISMS

AND PROCEDURES BE POSITIONED TO ENSURE THAT PUBLIC SER-

VANTS ACT RESPONSIBLY, THE ULTIMATE SAFEGUARD IS IN THE

CHARACTER AND INCLINATIONS OF BUREAUCRATS.106

U N I T E D S TAT E S G OV E R N M E N T P E R F O R M A N C E A N D

R E S U LT S AC T

A clear example of a legislated accountabilityregime is the United States Government Performanceand Results Act (GPRA), passed in 1993.

Described as “landmark legislation [that]seeks to fundamentally change the focus of federalmanagement and accountability… the GPRAestablishes a legislative framework for having agen-cies set strategic goals, measure performance, andreport on the degree to which goals were met.”108

Dozens of pilot projects were under way in 1995.By 1997, agencies are to submit a strategic plan tothe Office of Management and Budget (OMB)and to Congress. Beginning in fiscal year 1999,each agency is to submit to OMB an annual pro-gram performance plan. By March 31, 2000, eachagency is to submit an annual program perfor-mance report to the president and Congress, cover-ing the previous fiscal year. This report will discussperformance achieved against goals identified inthe annual performance plan, and actions neededto address performance shortfalls.

While discussion about the GPRA points tothe value of having statutory planning and report-ing requirements as a basis for encouraging com-mitment and promoting continuity, it is also rec-ognized that legislation alone cannot make it hap-pen. Developing and sustaining top managementcommitment, building capacity, creating incen-tives, integrating GPRA activities into ongoingoperations, and enhancing congressional oversightare all seen as key challenges.

T H E E N V I R O N M E N T F O R


A number of factors must be present foraccountability to be effective; it has to rest on a

solid psychological foundation. In proximity toaccountability reside ethics, morality, and codes ofconduct, all serving to compensate for obscureaccountability links or to reinforce them. Likeaccountability, ethics starts as an attitude, a pen-chant. Fully informed and morally fit people areable to make highly defensible decisions and natu-rally feel responsible and are accountable.

ACCOUNTABILITY DOES NOT JUST HAPPEN—APPROPRI-

ATE CONCEPTUAL FRAMEWORKS AND MECHANISMS

NEED TO BE DEVELOPED AND PUT IN PLACE. IT IS NOT

SOMETHING THAT “TRICKLES UP” TO THE BOARD—IT

NEEDS TO BE LED, INSISTED UPON AND CAREFULLY

NURTURED BY THE BOARD AND EQUALLY SUPPORTED BY

HOSPITAL MANAGEMENT AND PROFESSIONALS.109

Having a proper attitude, a healthy disposi-tion towards accountability is not sufficient. Itrequires a technical structure, one that is organiza-tionally sound.

In addition, the circumstances around anaccountability event (for instance, the considera-tion of accountability reports by a governingbody)—the timeliness, the place, the degree ofcommitment to the notion of accountability, thetime allowed—must be conducive to the effectiveexercise of an oversight role. There has to be anopportunity to challenge, to develop facts or argu-ments. Over time, the failure of governing bodiesto exact proper accountability, or appropriately toconsider fairly, if at all, accountability reports ren-dered to them, will turn a responsible attitude intoindifference. Eventually, indifferent people willcease being accountable, to the detriment of everyone.


FEW WOULD DISPUTE THAT WE NOW LIVE IN CONDI-

TIONS IN WHICH IT IS DIFFICULT TO SAY PRECISELY

WHAT WE MEAN IN PRACTICE BY ACCOUNTABILITY IN

ADMINISTRATION, AND EVEN MORE DIFFICULT TO

ASSERT WITH CONFIDENCE THAT WE KNOW HOW TO

ENFORCE IT.110

The institutional ethos is extremely impor-tant. It may come with tradition, which in turnmay be reinforced by a good set of values, particu-larly a strong sense of ethics among leaders whowill act in an accountable fashion even if notspecifically called upon to render an account. Suchpublic service values in those who possess them areoften viewed as superior to formal structures in thepromotion of accountability.

I N A P P R O P R I AT E AT T I T U D E S

Sometimes accountability is simply notdemanded by the governing bodies that are sup-posed to expect it. Even worse, in some cases, peo-ple who are publicly accountable count on theirsubordinates to ensure that as little as possible ren-dering of account is offered, to shield them fromtheir own accountability obligations.

A further example of a distorted view is thesincere belief held by some that, once entrustedwith a responsibility, they should be exemptedfrom having to render an account. They should betrusted implicitly to do the right thing, to actresponsibly; otherwise, why would they be maderesponsible? they ask. These views are not onlyheld by certain people in responsible positions whobelieve in their own trustworthiness, but also bysome who are affected by their actions and wouldconsider it inappropriate or disrespectful todemand an account from such obviously honestand capable individuals. It is not that these people

do not act as responsibly as they would if they wereaccountable, it is simply that their sense of respon-sibility does not extend to accountability.

In some cases, people entrusted with aresponsibility believe that to be in an accountabili-ty arrangement with the person or governing bodythat gave them legitimacy is an inferior position, aservant/master relationship, a state of dependence.They are not comfortable with the notion of ren-dering an account. It is unfortunate that the con-cept of accountability lends itself to that interpreta-tion. In reality, accountability is the recognition ofthe interdependence among the several agents, notof dependence in the bureaucratic or hierarchicalsense of the word.

I N A P P R O P R I AT E AC C O U N TA B I L I T Y A R R A N G E M E N T S

Something is happening to a natural disposi-tion towards accountability if in actual practice thenotion is eschewed. The problem is most likely tobe found in the leadership, in the governance.

Poor accountability may result where arrange-ments for it are so complicated and so burdensomethat they discourage the best-intentioned people.There are also cases where accountability arrange-ments and formal systems have been put in placethat are too diffuse to be truly useful. Sometimes,the structure of an organization does not provide anatural conduit for accountability; the accountabil-ity links are unclear because actual accountabilityrelationships are not overlaid on the organizationchart. For instance, there have been cases where anorganization has had a number of relatively nar-rowly defined programs, each with a strict verticalchain of command. As a result, integrated decisionmaking can only take place at the apex, and there-fore there is little overall corporate accountability;there is no provision for accountability for thecoordination of delivery of its various programs inthe field.111


Some large organizations rely on a teamworkapproach. If the team leader has little or no say inselecting and keeping team members, handing outtasks, getting appropriate resources and necessarycollaboration, he or she cannot be held account-able. Accountability in an organization or a teamcan be achieved only by clarifying and rationalizingthe responsibilities and authorities of the leaderand all subordinates so that together they candeliver on their promises.

A U T O N O M Y A N D


Certain institutional arrangements, particu-larly in government, have conferred a status of

independence or autonomy to particular agenciesthat have to be managed free of political interfer-ence, or dealt with at arm’s length, given the politi-cally sensitive nature of their activities. There aremany such agencies: the central banking authority,the state broadcasting system, grant-giving agenciesin support of culture and science, regulatory andquasi-judicial agencies, and so on. They are placedat a deliberate distance from the very body—Parliament or the government—that created them.This autonomous status should not exempt themfrom being accountable, although some wouldargue that to make them directly accountable tothe body that created them is to invite interference.


AUTONOMY VERSUS ACCOUNTABILITY: AN ILLUSTRATION

AN OUTCOME OF THE DIEFENBAKER GOVERNMENT’S CONFLICT IN 1961 WITH JAMES COYNE, THE GOVERNOR OF THE BANK OF

CANADA, OVER POLICY DIFFERENCES WAS A PROCEDURE BY WHICH THE GOVERNOR WOULD MAINTAIN HIS INDEPENDENCE IN CON-

TROLLING MONETARY POLICY. IF, HOWEVER, THE MINISTER OF FINANCE DID NOT AGREE WITH THE MANNER IN WHICH THE GOVER-

NOR WAS EXERCISING HIS AUTHORITY, HE COULD SEND HIM A LETTER OR ISSUE A POLICY DIRECTIVE. TWO GOVERNORS SINCE

THEN HAVE SAID THEY WOULD RESIGN IF THEY RECEIVED SUCH A LETTER. THE NEGATIVE CONSEQUENCES WITH RESPECT TO THE

MONEY MARKETS ARE HIGHLY PREDICTABLE. WOULD THE GOVERNMENT IGNORE SUCH CONSEQUENCES? PROBABLY NOT. AND

BECAUSE OF THAT, IT WOULD NOT SEND THE LETTER. COULD IT WAIT UNTIL THE SEVEN-YEAR TERM EXPIRES? YES, PROBABLY, PAR-

TICULARLY IF THE GOVERNOR IS NEARING THE END OF HIS TERM. BUT NOT REAPPOINTING A GOVERNOR WHO IS READY TO

UNDERTAKE A SECOND TERM WOULD LIKELY SEND THE SAME KIND OF SIGNALS IN THE MONEY MARKETS, WITH THE SAME CONSE-

QUENCES, SO THE CONVENTIONAL WISDOM GOES.

IRRESPECTIVE OF THE INDIVIDUALS CONCERNED, THE TECHNICAL STRUCTURE FOR ACCOUNTABILITY IS SOMEWHAT CUMBERSOME,

TO SAY THE LEAST. THE GOVERNOR IS INDEPENDENT, AS HE SHOULD BE. BUT THAT MAY BE INTERPRETED AS MAKING HIM UNAC-

COUNTABLE TO ANYONE, SAVE TO HIMSELF. HE WILL SAY THAT HE IS ACTING RESPONSIBLY AS HE HAS A MORAL OBLIGATION TO ACT

ACCORDING TO THE DICTATES OF HIS CONSCIENCE. HE MAY ADD THAT HE HAS A BOARD TO WHICH HE IS ACCOUNTABLE. THUS,

ONE COULD SAY IT IS THE BANK OF CANADA AS A WHOLE THAT WIELDS ALL THIS POWER. THIS IS PROBABLY TRUE BUT NOT USE-

FUL, AS IT IGNORES THE IMPACT OF THE TRADITIONAL MANNER IN WHICH THE CENTRAL BANK GOVERNOR PERSONIFIES NOT ONLY

THE INSTITUTION BUT HIS POLICIES, AS WELL. WHATEVER IS DONE TO THE INCUMBENT IS DONE TO HIS POLICIES, HENCE HIS

TREMENDOUS POWER. AT TIMES WHEN MONETARY POLICY IS THE ONLY INSTRUMENT OF ECONOMIC POLICY BEING PURSUED, AS

HAS BEEN THE CASE AT TIMES, THE MOST IMPORTANT PERSON IN CANADA AFFECTING THE DAILY LIFE AND FUTURE OF ALL

CANADIANS IN SOME UNMISTAKABLE FASHIONS, AS IT TURNS OUT, IS NOT AN ELECTED REPRESENTATIVE. ACCORDING TO SOME

CRITICS, SINCE HE HAS TO REMAIN INDEPENDENT OF THOSE WHO ARE THE ELECTED REPRESENTATIVES, THAT MAKES HIM UNAC-

COUNTABLE, PERIOD. HIS POWER IS ABSOLUTE. THE GOVERNOR, HOWEVER, MIGHT SEE IT DIFFERENTLY.

D I M E N S I O N S O F


In the literature of public administration, onefinds many attempts by different authors to recog-nize a variety of accountability relationships orsimply to pinpoint the locus of responsibilities.The following classification, more or less modifiedto suit the particular needs of these authors, hasappeared recently in numerous articles or studies.Some of the issues surrounding certain types ofaccountability receive more elaborate treatment insubsequent chapters. Meanwhile, the followingacquaints the reader with the essence of theseaccountability distinctions.

I N T E R N A L A N D E X T E R N A L AC C O U N TA B I L I T Y

Internal: a rendering of account fromthe lowest echelons to the top, in a hierarchy.Objectives are defined at the top and trans-mitted to lower levels for execution.Authority is delegated accordingly, followedlater by the rendering of account and possiblythe application of a reward system. Within agovernment structure, for instance, the ren-dering of account would take place at succes-sive echelons up to the deputy minister. Inturn, the latter would be accountable to theminister responsible for that particulardepartment. Generally, this internal account-ability is not public; it remains within man-agement.

External: a rendering of account bymanagement to their governing bodies. Thisrendering of account is public when it takesplace, for instance, at the assembly of thepeople’s representatives, the elected body, orwhen it is directed at stakeholders.

P O L I T I C A L AC C O U N TA B I L I T Y

Constitutional: the accountability of thegovernment to Parliament, the so-called min-isterial responsibility, is said to be a constitu-tional requirement.

Decentralized: the establishment of localauthorities, regional boards as a response tothe overload in a central or a provincial gov-ernment engenders a dispersion of account-ability and possible conflicts between the cen-tre and the locality.

Consultative: representative democracy issupplemented by participatory democracy.Elected representatives feel obligated to con-sult the population; they have a close rapportwith special interest groups and even feel acertain accountability to them. Such groupsoperate outside the electoral process and arenot necessarily representative of the broadconstituency they claim to represent. Theaccountability relationships are not very clearin such circumstances.

M A N AG E R I A L AC C O U N TA B I L I T Y

Commercial: when government servicesare financed by user fees rather than by bud-get appropriation, they may be judged asmuch, if not more, on their commercial per-formance as on the attainment of their publicpolicy purposes. The framework of account-ability of many Crown corporations or stateenterprises would assume this character.

Resource: accountability for resources istypically indicated for nonmarket provision ofservices. Budget-control frameworks mustensure efficiency and be capable of evaluatingmanagement performance. Resource account-ability can be divided into:


• financial-management accountabilityframework;

• human-resources accountability frame-work; and

• assets-management accountabilityframework.The human-resources component with-

in the context of an administration at the ser-vice of a representative government takes on aspecial dimension. Merit is the principle ofcompetence, and the so-called merit systemcharacteristic of our public service has somedefinite implications on accountability forhuman resources.

Professional: the allocation of resourcesin a public institution is often largely influ-enced, when not decided, by professionalswho owe their standards to a self-regulatingbody. In the name of professional freedom,they appear to operate largely outside thedemocratic control, although they do owe tothe legislator the right to exercise their profession.

L E G A L AC C O U N TA B I L I T Y

Judicial: the government allows reviewsof public servants’ actions through judicialreview of cases brought by aggrieved citizens(in the Federal Court of Canada, forinstance).

Quasi-judicial: largely in the form ofrecourse with respect to application of the lawwhere a great deal of administrative discretionis prevalent because of the necessity to oper-ate at arm’s length from politics. A specializedtribunal like the Tax Court of Canada is anexample of an entity operating within aquasi-judicial framework of accountability.

Regulatory: some regulatory agencies (theCRTC, for example) operate with a largedegree of independence, applying broad leg-islative mandates affecting the individualinterests of citizens by rendering administra-tive decisions free of political interference.This is possibly one of the most complexaccountability situations.

P R O C E D U R A L A N D C O N S E Q U E N T I A L

AC C O U N TA B I L I T Y 112

Procedural: in the sense that if all therequirements with regards to inputs are satis-fied, the output, or the intended outcomes orresults are deemed assured. Emphasis is onthe management procedures, practices, andsystems, as well as on compliance to rules andregulations.

Consequential: the most significant sig-nals emanate from the monitoring of the out-put to determine if intended goals have beenattained, presumably as a result of the effortsthat went into the initiative. Outputs maynot all lend themselves to retracing the corre-sponding inputs. The emphasis is on results,eventual outcomes, impacts, and constitutesan enlargement of the scope of accountabilityinto what is called effectiveness.

Accountability depends on a variety of mech-anisms that reflect the diversity and complexity ofthe machinery of government and other publicinstitutions. In such a situation, accountabilityrelationships are not always clear. It is because ofthis fragmentation that it is important for publicbodies to assume their responsibilities and be in aposition to explain and justify the manner inwhich they have discharged their responsibilities.This is essential to the functioning of the democra-tic political system.


I N S U M M A R Y

Accountability, as a personal trait and as alogical response to expectations and pressures, willnot naturally lead to the institutionalization of for-mal accountability regimes. This is in spite of thegrowing number of officials, managers, and mem-bers of governing bodies who do adhere to theappropriate values and are imbued by a sense ofresponsibility and accountability. Good faith and anoble disposition are not sufficient.

Ideally, what is needed is a focus on issues ofaccountability by governance: the elite, parliamen-tarians, members of governing bodies, senior man-agement, all those who are well placed and, giventhe ties to their respective constituencies, have agood understanding of public expectations.

Effective procedures of governance to guaran-tee accountability are needed, including a challengeprocess, appropriate arrangements for rendering anaccount, a broad role for program and policy mon-itoring and auditing bodies, as well as appropriateprotocols for all participants and stakeholders inthis process. It follows, too, that selection standardsused in recruiting and promoting senior managershave to ensure a stream of managers naturally dis-posed to serve in a responsible and accountablefashion.

Perhaps the easiest way of summarizing whathas been discussed is to describe truly accountablemanagers. They are people who have:

• said what they were going to do, how theywere going to do it, and to what ends;

• believed in what they said;• done what they have said they would do; and• showed what they had done.



74 REPORT OF THE INDEPENDENT REVIEW COMMITTEE ON THE OFFICE OF THE AUDITOR GENERAL OF CANADA [WILSON COMMITTEE] (OTTAWA: INFORMATION CANADA, 1975) 9.75 ATKINSON, GOVERNING CANADA, GLOSSARY.76 THE WORLD BANK, GOVERNANCE AND DEVELOPMENT, 14.77 ROYAL COMMISSION ON FINANCIAL MANAGEMENT AND ACCOUNTABILITY [LAMBERT COMMISSION], FINAL REPORT (OTTAWA: MINISTER OF SUPPLY AND SERVICES CANADA, 1979), 369.78 IBID., 21.79 C. E. S. FRANKS, THE PARLIAMENT, 227.80 DAVID HEALD, PUBLIC EXPENDITURE (OXFORD: MARTIN ROBERTSON, 1984), 154.81 IBID., 155.82 GORDON, PUBLIC ADMINISTRATION, 607.83 PAUL LOHNEIS, “ACCOUNTABILITY, COOPERATION, RECEPTIVITY,” IN ACCOUNTABILITY, NEWSLETTER OF FINANCIAL MANAGEMENT IMPROVEMENT LATIN AMERICA AND THE CARIBBEAN,

APRIL 1992, 3.84 HENRI-PAUL VIGNOLA, FORMER HEAD OF THE MONTREAL URBAN COMMUNITY POLICE, IN HIS 1982 BOOK, POLICE, THE CHALLENGE OF 21ST CENTURY, AS CITED IN AN EDITORIAL OF

OTTAWA’S LE DROIT, 21 JULY 1993, AND REPRODUCED, TRANSLATED, IN THE OTTAWA CITIZEN EDITORIAL PAGE, 26 JULY 1993.85 TIMOTHY W. PLUMPTRE, BEYOND THE BOTTOM LINE: MANAGEMENT IN GOVERNMENT (SOUTH HALIFAX: THE INSTITUTE FOR RESEARCH ON PUBLIC POLICY, 1988), 182.86 SKINNER, ACCOUNTING STANDARDS, 636.87 PUBLIC SERVICE 2000, THE RENEWAL OF THE PUBLIC SERVICE OF CANADA (OTTAWA: MINISTER OF SUPPLY AND SERVICES CANADA, 1990), 89.88 IBID., 90.89 IBID., 91.90 GORDON F. OSBALDESTON, KEEPING DEPUTY MINISTERS ACCOUNTABLE (TORONTO: MCGRAW-HILL RYERSON, 1989), 5-6.91 PAUL C. LIGHT, MONITORING GOVERNMENT, INSPECTORS GENERAL AND THE SEARCH FOR ACCOUNTABILITY (WASHINGTON, D.C.: THE BROOKINGS INSTITUTION/THE GOVERNANCE

INSTITUTE, 1993), 12. 92 JAMES FESLER AND DONALD KETTL, IN THE POLITICS OF THE ADMINISTRATIVE PROCESS (CHATHAM, N.J.: CHATHAM HOUSE 1991), 317.93 JOHN LONSDALE, “POLITICAL ACCOUNTABILITY IN AFRICAN HISTORY” IN POLITICAL DOMINATION IN AFRICA, PATRICK CHABAL, ED., (NEW YORK: CAMBRIDGE UNIVERSITY PRESS, 1986), AS

CITED IN THE WORLD BANK, GOVERNANCE AND DEVELOPMENT, 13.94 BARRY M. MUNDT, RAYMOND T. OLSEN, HAROLD I. STEINBERG, MANAGING PUBLIC RESOURCES (PEAT MARWICK INTERNATIONAL, 1982), 3-4.95 WILSON COMMITTEE, 9.96 A FOURTH WORD, LIABILITY, WAS CENTRAL TO LAMBERT’S DEFINITION CITED EARLIER. AT THAT TIME AND IN THAT CONTEXT, IT MAY HAVE BEEN PREFERRED TO THE WORD RESPONSIBILITY,

THUS MAKING THE NOTION LOOK MORE FORMAL WHILE CONVEYING THE SAME INTENT. TODAY, IN THE LANGUAGE OF ACCOUNTABILITY, IT IS NECESSARY TO SEPARATE RESPONSIBILITY

FROM LIABILITY BECAUSE THE LATTER HAS REACQUIRED ITS STRONG LEGAL CONNOTATION, PARTICULARLY IN CORPORATE GOVERNANCE—AS IN DIRECTOR’S LIABILITY—AND IN THE ENVI-RONMENTAL FIELD.

97 GEORGE ANDERSON, PRESIDENT OF CMHC, IN REPORTING AND AUDITING EFFECTIVENESS: THE CMHC EXPERIENCE, CCAF APPLIED RESEARCH SERIES [HEREINAFTER CMHC] (OTTAWA:CCAF, 1990), II.

98 FRANKS, PARLIAMENT OF CANADA, 229.99 DWIGHT WALDO IN CEPAQ-ENAP, “DÉBATS SUR L’IMPUTABILITÉ”, ACTES DU COLLOQUE SUR L’IMPUTABILITÉ, CHÂTEAU MONT STE-ANNE, QUE., 9 ET 10 JUIN, 1983.100 CCAF, EFFECTIVENESS REPORTING AND AUDITING IN THE PUBLIC SECTOR [HEREINAFTER EFFECTIVENESS REPORTING] (OTTAWA: CCAF, 1987), 64.101 STANLEY C. BRUBAKER, “REJOINDER TO RAWLS,” THE PUBLIC INTEREST, NO. 99, SPRING 1990, 133. 102 SEE COMMISSION OF INQUIRY INTO THE AIR ONTARIO CRASH AT DRYDEN, ONTARIO (CANADA) FINAL REPORT, VOL. III, (OTTAWA: MINISTER OF SUPPLY AND SERVICES CANADA, 1992), 1165

FOR AN ELABORATION OF THAT PRINCIPLE AND A CONTROVERSY SURROUNDING ITS APPLICATION.103 KEYNOTE ADDRESS OF ROSS WALKER TO CCAF’S 13TH ANNUAL CONFERENCE, IN DELIVERING ON ACCOUNTABILITY: THE NEED FOR LEGISLATIVE AND REGULATORY REFORM, (OTTAWA:

CCAF, 1993), 11.104 EDWARD WAITZER, CHAIRMAN OF THE ONTARIO SECURITIES COMMISSION, IN AN INTERVIEW ON THE THEME OF “IN SEARCH OF EFFECTIVE GOVERNANCE” REPRODUCED IN A CCAF

VIDEO SCRIPT, 1994, 9.105 CARL J. FRIEDRICH, “PUBLIC POLICY AND AND THE NATURE OF ADMINISTRATIVE RESPONSIBILITY,” IN PUBLIC POLICY, 1940, NO. 1, PP. 3-24, AND HERMAN FINER, “ADMINISTRATIVE

RESPONSIBILITY AND DEMOCRATIC GOVERNMENT,” PUBLIC ADMINISTRATION REVIEW, SUMMER 1941, 335-350.106 GORDON, PUBLIC ADMINISTRATION, 564.107 PETER GREGORY SPEAKING IN THE SUBJECT FORUM ON “CHANGING THE FACE OF ACCOUNTABILITY LEGISLATION “ AT THE CCAF 14TH ANNUAL CONFERENCE IN OTTAWA, NOVEMBER

1993.108 TESTIMONY OF JOHNNY C. FINCH, ASSISTANT COMPTROLLER GENERAL OF THE UNITED STATES GENERAL ACCOUNTING OFFICE BEFORE THE SUBCOMMITTEE ON GOVERNMENT

MANAGEMENT, INFORMATION AND TECHNOLOGY, COMMITTEE ON GOVERNMENT REFORM AND OVERSIGHT, HOUSE OF REPRESENTATIVES, MANAGING FOR RESULTS: STATUS OF UNITED

STATES GOVERNMENT PERFORMANCE AND RESULTS ACT (GAO/T-GGD-95-193, JUNE 27, 1995).109 MICHAEL DECTER, ONTARIO DEPUTY MINISTER OF HEALTH, IN REPORTING ON EFFECTIVENESS, THE EXPERIENCE OF THE QUEEN ELIZABETH HOSPITAL, CCAF APPLIED RESEARCH SERIES,

[HEREINAFTER QEH] (OTTAWA: CCAF, 1992), 3.110 N. JOHNSON, “DEFINING ACCOUNTABILITY,” PUBLIC ADMINISTRATION BULLETIN, NO. 17, DECEMBER 1974, 3, AS CITED IN P. M. JACKSON, THE POLITICAL ECONOMY OF BUREAUCRACY

(OXFORD: PHILIP ALLEN, 1982), 212.111 SEE COMMISSION OF INQUIRY INTO THE AIR ONTARIO CRASH AT DRYDEN, P. 1035 FOR A DISCUSSION ON PROBLEMS OF ACCOUNTABILITY ARISING FROM THE ORGANIZATION AND THE STRUC-

TURE OF AUTHORITY.112 SEE JAMES CUTT, COMPREHENSIVE AUDITING IN CANADA, THEORY AND PRACTICE (NEW YORK: PRAEGER, 1988), 53-54. PROFESSOR CUTT HAS USED THE DISTINCTION FREQUENTLY AS A

BASIS OF EXPOSITION.

KERNAGHAN ON PARLIAMENTARY OVERSIGHT

A COMPREHENSIVE AND COHERENT SYSTEM OF RESPONSIBILITY IN

GOVERNMENT REQUIRES THE MAINTENANCE AND THE MESHING OF

POLITICAL AND ADMINISTRATIVE RESPONSIBILITY. IN CANADA’S

PARLIAMENTARY SYSTEM, WHERE THE CABINET DOMINATES

PARLIAMENT, CONCERN OVER THE STATE OF GOVERNMENT RESPON-

SIBILITY CENTRES ON THE ACTIONS OF MINISTERS AND PUBLIC SER-

VANTS. PARLIAMENT’S PERFORMANCE IN CHECKING THE EXERCISE

OF MINISTERIAL AND BUREAUCRATIC POWER DEPENDS LARGELY ON

THE CAPACITY AND INCLINATION OF OPPOSITION MEMBERS IN THE

HOUSE OF COMMONS TO CONTROL OR INFLUENCE MINISTERS.

THE POWER OF PARLIAMENT OVER THE PUBLIC SERVICE IS WIELD-

ED PRIMARILY IN AN INDIRECT FASHION THROUGH QUESTIONING

AND CRITICISM OF MINISTERS RESPONSIBLE TO PARLIAMENT FOR

THE ADMINISTRATION OF THEIR DEPARTMENTS. SINCE MINISTERS

ARE THE FORMAL CONSTITUTIONAL INTERMEDIARIES BETWEEN

PARLIAMENTARIANS AND PUBLIC SERVANTS, PARLIAMENT’S ABILITY

TO AFFECT THE RECOMMENDATIONS AND DECISIONS OF PUBLIC

SERVANTS RESTS HEAVILY ON THE INTERPRETATION AND APPLICA-

TION OF THE CONSTITUTIONAL CONVENTION OF MINISTERIAL

RESPONSIBILITY. THUS, PARLIAMENT’S SUCCESS IN PRESERVING

AND PROMOTING ADMINISTRATIVE RESPONSIBILITY DEPENDS

DIRECTLY ON THE INTERPRETATION OF MINISTERIAL RESPONSIBILI-

TY. MINISTERIAL RESPONSIBILITY IS IN TURN TIGHTLY BOUND UP

WITH THE CONVENTIONS OF POLITICAL NEUTRALITY AND PUBLIC

SERVICE ANONYMITY. THESE THREE CONSTITUTIONAL CONVEN-

TIONS (OR DOCTRINES) ARE CENTRAL TO THE ACHIEVEMENT OF

ADMINISTRATIVE RESPONSIBILITY IN THAT THEIR DEFINITIONS AND

USAGE SHAPE TO A LARGE EXTENT THE PATTERN OF INTERACTIONS

BETWEEN PUBLIC SERVANTS ON THE ONE HAND AND PARLIAMEN-

TARIANS AND MINISTERS ON THE OTHER.

C H A P T E R 5

ACCOUNTABILITYOBLIGATIONS

The development of responsible governmentcan be seen as a struggle by the people to control,through their elected representatives, what had pre-viously been the absolute power of the monarchy.That struggle is now virtually complete, with theCrown now performing in little more than a cere-monial capacity. But modern governments haveassumed enormous power. That power is exercisedby ministers (typically elected representatives)through substantial bureaucracies. The issue, there-fore, is not how to control the monarch, but how tohold ministers and their civil servants accountable.

In a parliamentary democracy like Canada,the electorate has only infrequent opportunities tovoice its opinion of the government in a generalelection. The day-to-day oversight falls to the freepress and, importantly, to the legislature. Politicalscientist Kenneth Kernaghan emphasized this rolein a paper published in 1979.113

There seems to be no dispute that ministershave a responsibility to account to Parliament fortheir actions and policies. As a Privy CouncilOffice submission to the Lambert Commission onthe constitutional responsibility of ministers stated:“Ministerial responsibility is a fundamental princi-ple of the constitution. It requires that a ministerbe personally answerable to the House of Commonsfor the exercise of power.”114

What is not so clear is the extent to whichministers should be held responsible for the actionsand decisions of their public servants. Also in dis-pute is the extent to which public servants areanswerable and must provide information to leg-islative committees.

P A R T I . S E C T I O N 2 . A C C O U N T A B I L I T Y — I S S U E S & P R A C T I C E 6 1

T H E T R A D I T I O N A L V I E W

The traditional or purist view is the oneinherited from British convention. It holds thatministers are personally responsible and account-able for the actions of their public servants. Inreturn for this protection, public servants are to bepolitically neutral, and the advice they give theirministers is kept confidential, assuring theanonymity of bureaucrats. Simply stated, the tradi-tional view holds that public servants are account-able to ministers who, in turn, are accountable toParliament.

Thus, ministers are really responsible andaccountable to Parliament for the conduct of theirdepartments. This responsibility includes:

• political accountability for policies and otherpolitical acts and decisions;

• administrative accountability for managementand administration; and

• financial accountability for the use offunds.115

Sympathetic to this view is Jim Mitchell, aformer senior civil servant, who adds: “I see noneed… to modify in any way the fundamentalrelationships of accountability that have existedover at least two hundred years of parliamentarytradition among parliament, Ministers and theirofficials.”116 Mitchell is particularly concerned withthe possible loss of neutrality of the public serviceif officials were directly accountable to Parliament.He argues that the public profile given to seniorcivil servants and the potential controversy generat-ed by such an accountability arrangement wouldcertainly have some unpredictable consequences onrelationships between ministers and their offi-cials.117

PRESIDENT DWIGHT D. EISENHOWER

ON THE ISSUE

[T]HE GOVERNMENT OF THE UNITED STATES HAS BECOME

TOO BIG, TOO COMPLEX, AND TOO PERVASIVE IN ITS INFLU-

ENCE ON ALL OUR LIVES FOR ONE INDIVIDUAL TO PRETEND TO

DIRECT THE DETAILS OF ITS IMPORTANT AND CRITICAL PRO-

GRAMMING. COMPETENT ASSISTANTS ARE MANDATORY; WITH-

OUT THEM THE EXECUTIVE BRANCH WOULD BOG DOWN.

TO COMMAND THE LOYALTIES AND DEDICATION AND BEST

EFFORTS OF CAPABLE AND OUTSTANDING INDIVIDUALS

REQUIRES PATIENCE, UNDERSTANDING, A READINESS TO DELE-

GATE, AND AN ACCEPTANCE OF RESPONSIBILITY FOR ANY HON-

EST ERRORS—REAL OR APPARENT—THOSE ASSOCIATES AND

SUBORDINATES MIGHT MAKE. SUCH LOYALTY FROM SUCH PEO-

PLE CANNOT BE WON BY SHIFTING RESPONSIBILITY, WHINING,

SCOLDING, OR DEMAGOGUERY. PRINCIPAL SUBORDINATES

MUST HAVE CONFIDENCE THAT THEY AND THEIR POSITIONS

ARE WIDELY RESPECTED, AND THE CHIEF MUST DO HIS PART

IN ASSURING THAT THIS IS SO.118

The traditional view argues that for ministeri-al accountability to be effective it has to be clearand unambiguous, in spite of the immense com-plexity of modern public administration. The clari-ty comes from the fact that in this principle theminister is personally accountable to Parliament ina formal and visible manner.

T H E D E B A T E

There are other views. Some observers arguethat in the early years of the Canadian Confederationthe concept of ministerial accountability wasappropriate as life, governing, and administeringwere so simple as to be seamless. Ministers wereable to stay current with events in their depart-ments. Obviously, this is no longer the case, theyinsist, and it is now unrealistic to expect a minister


to be aware of, and hence accountable for, all theactions of the sometimes thousands of public ser-vants in today’s governmental departments.

In addition, given the complexity of moderngovernment, it is argued that senior public servantsare deeply involved in policy development andshould be held accountable for the advice they giveministers. With the recent streamlining of the fed-eral cabinet and similar initiatives in certainprovinces resulting in considerably fewer (but oftenlarger) ministries, senior bureaucrats will likelyhave more to do with shaping the policy agendathan ever before. Not only does this erode theactual responsibilities of ministers, it also argues forappropriate public mechanisms for holding publicservants accountable.

A thoughtful elected representative observedthat “the current system is inadequate and we mustfind ways to ensure that senior public servants whowill be given greater responsibility are held fullyaccountable, so that we politicians, in turn, can beanswerable to our constituents.”119 The LambertCommission thought that the minister should berelieved of responsibilities specifically assigned todeputy ministers. The implication is a profoundchange in ministerial responsibilities.

“PARLIAMENT SHOULD HOLD DEPUTY MINISTERS

ACCOUNTABLE FOR ADMINISTRATIVE MATTERS WHILE

HOLDING POLITICIANS ACCOUNTABLE FOR POLITICAL

DECISIONS.” ACCORDING TO C.E.S. FRANKS, THE RATIO-

NALE FOR THIS IS TO ALIGN THE DOCTRINE WITH THE

PRACTICE.120

The concern over this view is that a regime ofaccountability within the government wouldbecome part of the accountability regime betweengovernment and Parliament. The continuingaccountability of government to Parliament is a

constitutional convention and is satisfied by theconcept of ministerial accountability to Parliament.If the minister were to share his departmentalresponsibility with the deputy minister inParliament, it would mean that internal account-ability would become part of external or publicaccountability. In this arrangement, some peoplesee a dilution of ministerial accountability on theone hand and the erosion of public servantanonymity and neutrality on the other.

The debate still rages.

T H E O S B A L D E S T O N V I E W

The 1988 report Keeping Deputy MinistersResponsible, by former Clerk of the Privy Counciland secretary to the Cabinet Gordon Osbaldeston,argues that “ministerial responsibility was funda-mental to responsible government, and that to sub-tract whole areas of public administration fromministerial responsibility would be to break thechain of responsibility from the public to the min-ister of the Crown and back again.”121

Osbaldeston’s solution to the debate was todifferentiate between answerability and account-ability to Parliament. Deputy ministers, he said,have an “obligation to provide accurate and com-plete information to parliamentary committees andto help… explain departmental policies and pro-grams.” This essentially agrees with Privy Councilsuggestions to the Lambert Commission:

Officials are accountable to theirministers, who must answer to the House fortheir use of the authority conferred uponthem in law and by virtue of their responsi-bility to the House of Commons. It is, how-ever, possible to distinguish between adeputy’s accountability to his minister for allthat occurs under the minister’s responsibility,and his accountability before parliamentarycommittees for administrative matters so long

P A R T I . C H A P T E R 5 . A C C O U N T A B I L I T Y O B L I G A T I O N S 6 3

as they do not call directly into question theexercise of the minister’s responsibility. Theaccountability of officials before parliamen-tary committees for administrative matterscannot be said to alter the formal and directresponsibility of the minister personally toParliament for any matter within his disci-pline for which the House chooses to holdhim answerable.122

Osbaldeston said more simply that deputyministers are answerable to Parliament, while min-isters are accountable to it. Such a clear distinctionis instinctively appealing. Experience shows, how-ever, that when an official appears before House ofCommons committees and provides informationabout something the opposition sees as govern-ment extravagance, the reports in the media leadone to believe that the official who was answerableand who answered was the person directly respon-sible, leaving the minister off the hook. This mayexplain, in part, the frequent absence of ministersfrom committee meetings.

T H E V I E W O F T H E M CG R AT H CO M M I T T E E

The Special Committee on Reform of theHouse of Commons held that:

The idea of a minister being responsiblefor everything that goes on in a departmentmay once have been realistic, but it has longsince ceased to be so. A minister cannot possi-bly know everything that is going on in adepartment. The doctrine of ministerialresponsibility undermines the potential forgenuine accountability on the part of the per-son that ought to be accountable —the seniorofficer of the department…

We have heard many arguments that anew doctrine of deputy ministerial responsi-bility relating exclusively to matters of admin-

istration should be established. In this contextadministration includes policy implementa-tion. Such a doctrine would set out the oblig-ations of senior public servants and includethe obligation to testify before parliamentarycommittees on matters of administration.Under this system, the testimony of deputyministers before committees would be aneveryday occurrence. Furthermore, regularopen contact between the senior public ser-vice and Members of Parliament should leadto a more realistic understanding of adminis-trative practices and more precise pinpointingof accountability.123

CANADA AND THE BRITISH TRADITION

THE CANADIAN DOCTRINE OF MINISTERIAL RESPONSIBILITY IS

LARGELY INSPIRED FROM BRITISH TRADITIONS. IT SHOULD BE

REMEMBERED, HOWEVER, THAT CANADA DIFFERS FROM

BRITAIN. FOR INSTANCE, IN BRITAIN THE DEPUTY MINISTER,

WHO IS CALLED THE “PERMANENT SECRETARY,” IS ALSO THE

“ACCOUNTING OFFICER”: THIS IS THE PERSON WHO SIGNS

THE ACCOUNTS OF THE DEPARTMENT, VOUCHES FOR THEIR

VERACITY, DEFENDS THEM BEFORE THE PUBLIC ACCOUNTS

COMMITTEE, AND ASSUMES THE RESPONSIBILITY FOR ERRORS

IN THEM. THE INTENT IS TO KEEP FINANCIAL ADMINISTRATION

AWAY FROM MINISTERIAL OR POLITICAL CONTROL. THIS IS

ACCORDING TO BOTH LAW AND PRACTICE. CANADIAN PRAC-

TICE HOLDS THE MINISTER RESPONSIBLE AND ACCOUNTABLE

TO PARLIAMENT FOR ALL ASPECTS OF POLICY AND ADMINIS-

TRATION.124

A V I E W A B O U T M I N I S T E R S A N D P U B L I C AC C O U N T S

CO M M I T T E E S

In 1980, the Canadian ComprehensiveAuditing Foundation’s (CCAF) board of governorspublished the results of its first research project in


the form of a book titled Improving Accountability—Canadian Public Accounts Committees and LegislativeAuditors. The study produced a number of recom-mendations including the following: “Ministers becalled as witnesses before Public AccountsCommittees only when they have been personallyinvolved in decisions under examination.”126

This recommendation was largely shaped byprovincial and federal deputy ministers interviewedduring the research project. Those officials felt thatit was both fair and proper that they themselvesshould answer for actions and decisions they take ontheir own authority. Two arguments were put for-ward in favour of that position: first, the presence ofministers at committee meetings is an invitation topartisan political discussion, something that thosecommittees should definitely eschew; second, theavailability of a minister should not be used toshield public servants from answering for actionsthey have taken in the administration of a program.

Interestingly, when ministers were inter-

viewed, opinions were divided. One view was thestrict interpretation of the doctrine of ministerialresponsibility—the minister assumes responsibilityfor everything in his or her department. Anotherview proposed that the Public Accounts Committee’sinterest is in policy issues (a possible misinterpreta-tion of the role of the committee), and for that theminister could not leave the answering of questionsto his public servants. A third view argued thatministers are responsible for determining policy,not their execution or administration, and there-fore could not be held accountable for decisionstaken in the administering of departmental programs.

T H E P R I V Y CO U N C I L V I E W

The government of Canada seemed to haveretained the classical view of ministerial account-ability on the basis of the following statement inthe 1990 White Paper on the renewal of the publicservice, Public Service 2000:

Ministers are individually and collective-ly responsible to the House of Commons forthe activities of government including themanagement and conduct of the PublicService. Individually, Ministers are responsiblefor administration of the decisions of theGovernment and the activities of PublicServants falling under their jurisdiction.Collectively, they are responsible for the deci-sions of the Government as a whole and theactivities of all of their colleagues.127

In the subsequent report issued on June 30,1992, on implementation of Public Service 2000initiatives, the Privy Council Office inserted a clar-ification on the issue of ministerial responsibility.The report came on the heels of a debate about theexpedited entry into Canada of the former Iraqiambassador to the United States. The clarificationreads as follows:


T H E D E P U T Y M I N I S T E R

THE RELATIONSHIP OF DEPUTY MINISTERS AND SENIOR

OFFICIALS TO ELECTED GOVERNMENTS IS COMPLICATED

AND AMBIGUOUS—RATHER LIKE THE PROCESS OF GOV-

ERNANCE ITSELF. YET IF THERE ARE FEW ABSOLUTES IN

THE RELATIONSHIP, THE MAIN ELEMENTS BECOME

CLEAR ENOUGH IF ONE STEPS BACK A BIT. OFFICIALS

SERVE ELECTED GOVERNMENTS, BUT ARE NOT OF THEM.

WE BRING TO OUR JOBS A GOOD DEAL OF KNOWLEDGE,

A FAIR CAPACITY FOR HARD WORK, AND AS MUCH

POLITICAL NEUTRALITY AS COULD REASONABLY BE

LOOKED FOR IN HUMANS. IF WE CAN NO LONGER BE AS

ANONYMOUS AS WE ONCE WERE, WE ALSO KNOW THAT

WE BELONG ONLY AT THE EDGE OF THE PUBLIC STAGE;

THE CENTRE IS RESERVED FOR THOSE WHO EARNED A

PLACE THERE BY GETTING THEMSELVES ELECTED.125

The most important point to be clearon is that the constitutional principle of min-isterial responsibility requires that Ministers,and only Ministers answer to the House ofCommons for the activities carried out intheir names. Ministerial responsibility doesnot signify the Ministers must resign when anofficial makes a mistake or acts inappropriate-ly. In such circumstances, Ministers exercisetheir responsibility by taking any necessaryremedial measures. In point of fact, there isnot a single case since Confederation of aMinister resigning because of an error orother action by an official.128

This statement is not inconsistent with theposition outlined in the 1990 report. It does, how-ever, vary from the classical position, the oneobserved even in recent years in the BritishParliament.

The Privy Council Office report went on todeal with the fundamentals of that position,described as reflecting current practice, the issuesof 1992:

• Ministers… are constitutionally responsibleto the House of Commons for their actionsand the actions of their officials.

• … the power of the state is to be exercisedunder the authority of elected officialsaccountable to the representatives of the elec-torate.

• Public servants are responsible toMinisters…Officials do, of course, answer onbehalf of their Ministers to committees of theHouse provided this does not draw officialsinto partisan controversy.

• Ministers fulfil their responsibilities and arecalled to account… in accordance with prac-tice and convention… Ministers cannot knoweverything that is done under theirauthority…

• The way in which Ministers exercise theirresponsibility for the actions of their officialswill depend upon the circumstances andwhether an official has acted, for example, ina clearly unacceptable manner of which theMinister had no knowledge.

The report cites certain eminent authoritiesin support of its position. On the question ofassumption of responsibility, one authority says:“When action is taken of which the Minister dis-approves and which he has no prior knowledge,there is no obligation on his part to endorse it, todefend the errors of his officers, or to resign.”129

On the remedial action, another authoritysays: “It is now universally accepted that it isunreasonable to hold a minister personally respon-sible in the form of resignation for the administra-tive failings of subordinates… The usual practice isthat the minister informs Parliament that the faultlies with his officials and he promises that theoffenders will be disciplined and their mistakescorrected.”130

A C C O U N T A B I L I T Y O F P U B L I C

S E R V A N T S

On the accountability of public servants thePublic Service 2000 report said:

The doctrine of ministerial responsibili-ty imposes obligations on public servants aswell as on Ministers. Ministers depend on thequality and continuity of advice and informa-tion from their officials in order to fully exer-cise their responsibility to the House. Publicservants must therefore ensure that their con-duct will not get their minister into difficulty,and they must ensure that the Minister isfully aware of any significant matter thatcould give rise to public comment. “Why wasI not told?” is one of the most serious indict-


ments of an official that a Minister canmake…

In short, officials, particularly seniorofficials, are paid to exercise their judgement,and first and foremost this means decidingwhat the Minister needs to know.

Public servants are accountable for theirmistakes and errors of judgement to theirMinister, who in turn must answer toParliament. The accountability of officials isrendered through the chain of command tothe Deputy Minister who has overall respon-sibility for the conduct of the members of hisor her department. Officials are subject to astatutory disciplinary code prescribed byParliament that encompasses a spectrum ofsanctions, of which the most serious is dis-missal…

And when things go wrong, Ministersmust take their responsibility, cause enquiryinto the circumstances and ensure that theDeputy takes whatever remedial action maybe necessary to correct procedures and exer-cise disciplinary action as appropriate.131

On the anonymity of public servants, thereport has this to say:

It is obvious that Ministers cannot pos-sibly know everything about all the activitiesof their departments. Moreover, Ministers arerequired to provide extensive amounts ofinformation to Parliament through a varietyof mechanisms such as written and oral ques-tions, motions for papers, study of the esti-mates, scrutiny of government bills, andreview of the Public Accounts and the reportsof the Auditor General. Public servants are,therefore, called upon to support ministers intheir answerability to Parliament.

Without prejudicing its right to holdministers responsible, Parliament has increas-

ingly accepted that officials answer (always onbehalf of their Ministers) for matters that areunlikely to involve the House’s confidence inMinisters…

[T]he practice of asking officials toanswer to Parliamentary committees onbehalf of their ministers has affected the tra-ditional anonymity of the Public Service. Inthis era of constant parliamentary and mediascrutiny, officials are far more likely than inthe past to be known publicly. This has not,however, altered the fundamental principlethat officials are accountable to Ministers andMinisters are responsible to the House ofCommons.132

T H E J U R I S P R U D E N C E

The subject of ministerial accountability is notone on which the courts have had frequent occasionto rule. The Bhatnager case is an exception.133 Theissue before the Supreme Court of Canada waswhether two ministers could be held in contemptof court because their officials had failed to complywith the terms of a court order issued to the minis-ters. The ministers had had no personal knowledgeof the court order. The Federal Court, trial division,had ruled there was no contempt; the appeal divi-sion had ruled that there was.

In his reasons for the Supreme Court judg-ment, The Hon. Mr. Justice John Sopinka wrote:

In the case of Ministers of the Crownwho administer large departments and areinvolved in a multiplicity of proceedings, itwould be extraordinary if orders werebrought, routinely, to their attention. In orderto infer knowledge in such a case, there mustbe circumstances which reveal a special reasonfor bringing the order to the attention of theMinister…

This does not mean that Ministers will


be able to hide behind their lawyers so as toflout orders of the court. Any instructions tothe effect that the Minister is to be kept igno-rant may attract liability on the basis of thedoctrine of willful blindness. Furthermore,the fact that a Minister cannot be confidentin any given case that the inference will notbe drawn will serve as a sufficient incentive tosee to it that officials are impressed with theimportance of complying with court orders.

The unanimous decision of the court wasthat the two ministers were not in contempt.

While this decision may clarify ministers’legal position in respect of the activities of theirofficials, it does not necessarily resolve the issue ofministerial accountability to Parliament for offi-cials’ actions, which is a matter of convention,not law.

T H E W E S T L A N D C A S E

Parliamentary crises, like revolutions, bringout the best and the worse in people. The Westlandepisode in Great Britain in late 1985 and early1986 gave rise to inquiries by House of Commonscommittees.134 One such inquiry focused on theduties and responsibilities of civil servants to minis-ters (the Treasury and Civil Service SelectCommittee). Another one (the Defence SelectCommittee) produced a report critical of improperministerial and official behaviour in the affair.Subsequently, the government published a reply tothe Treasury Committee in which it restated thetraditional constitutional view of accountability:

The Government endorses the commit-tee’s [the Defence Committee’s] two basicpropositions on accountability: that ministersand not officials are responsible and account-able for policy; and that officials’ advice toMinisters is and should remain confidential.

Constitutionally, Ministers are responsibleand accountable for all actions carried byCivil Servants of their departments in pursuitof Government policies or the discharge ofresponsibilities laid on them by Parliament.135

This statement of ministerial accountabilitydoctrine is extremely simple, but even a simpleaccountability arrangement can generate seriousproblems. The minister involved in the allegedimproper behaviour resigned. Another ministeraccepted full responsibility for the actions of thefive civil servants who were alleged to have behavedimproperly by leaking a document, a most damag-ing act, under the circumstances. He resigned. Butthe prime minister, who publicly identified theofficials instead of protecting their anonymity, didnot assume any responsibility for their actions. Tocomplicate matters, the five named officials wereprevented by their minister from defending them-selves.

While the restatement may sound reassuringfor those who have particular affection for the tra-ditional doctrine, it did not go down well with thetop civil servants’ union as a way to fill the breachcaused by the Westland affair:

It… seems that the conventions regard-ing accountability are no longer accepted.Individual civil servants have been placed inan intolerable position, both in relation totheir own conduct and by being asked toaccount for the veracity of minister(s), andthere is no reason to believe this may nothappen again. One should not underestimatethe effect this has on general as well as indi-vidual morale. Senior civil servants are notshrinking violets, they are perfectly capable ofgiving a robust account of themselves andrebutting criticism if allowed to do so. At pre-sent they are not.136


A D V I C E A N D A N O N Y M I T Y

Members of House of Commons committeesoften question officials on particular items ofadministration, policy, and the nature of theiradvice to ministers. That does not mean that pub-lic servants will reveal the nature of the advice, butmay feel under pressure to do so. In addition, thetraditional anonymity of public servants, which ispart of this simple accountability doctrine, has alsobeen broken on certain occasions and public ser-vants have been named in political declarations.137

[T]he requirement to explain complicat-ed programs, and to consult the public aboutproposed courses of action, is far in excess ofwhat any minister could possibly manage. Forofficials to take on these kinds of functions insupport of their ministers makes practicalsense, always provided that we don’t forgetwhat our place is in the scheme of things.Deputy ministers’ names should not behousehold words, now or in the future.

It follows from the description I’ve justgiven that the text book formula of an electedgovernment that makes policy decisions and apublic service that carries them out is a con-siderable over-simplification. The knowledgeof officials is not limited to the managementand technical aspects of the department. Theyalso know the issues that go with the portfo-lio; they have lived through past attempts,successful and otherwise, to deal with theseissues; and they can have a fairly extensiveknowledge of the department’s client popula-tion.

For all these reasons, the advice provid-ed by deputy ministers and senior officialsincludes advice that is ‘political’—not partisanin the sense of advising the government howto do in the opposition, but political in thesense of giving ministers assessments of likely

reactions, particularly by the department’sclientele, to a course of action that is underdiscussion. There is nothing improper aboutthis.

… The suggestion is sometimes madethat the principle of keeping confidential theadvice we give the ministers exists for thepurpose of protecting officials. The truth isthe exact opposite: it is to give ministers themaximum scope for judgement in arriving attheir decisions. Minister’s jobs are quite toughenough already; they should not have to con-tend as well with being second-guessed inpublic by their officials. More fundamentally,a minister is entitled to assume that theadvice he or she gets is the best that officialscan come up with, and is not influenced by any consideration of playing to the galleries.138

A C A S E I N P O I N T

In Canada, an interesting event took placewhen the deputy minister of finance resigned onthe heels of the November 1981 budget, whichhad been widely and publicly criticized. Hisdepartment and his officials had been the object ofsevere and sometimes personal criticisms in thepress. His letter of resignation made the followingpoints: first, he assumed full responsibility for thekind of advice he had given to his minister and thecabinet; second, he assumed full responsibility forthe work of his subordinates, as a leader should.He was reaffirming the tradition of neutrality,objectivity, and anonymity of the public service.And he deplored the fact that, being possessed bythese very noble attributes, his subordinates couldnot respond publicly to criticism.

Curiously, some of the most serious elementsin the media ignored the restatement of essentialconstitutional principles and interpreted the


deputy minister’s resignation and indeed the letteras accepting responsibility for the budget, leavingthe elected officials off the hook. In accepting theresignation, the prime minister reiterated the fun-damental principles and agreed that civil servantsare responsible for the advice, not the decision.This reassurance did not produce lasting effects,however, as over the years there have been severalsenior civil servants who took the brunt for theirministers while the latter were left relativelyuntouched. Thus, over the years, the traditionaldoctrine of ministerial responsibility has beenseverely shaken.


113 KENNETH KERNAGHAN, “POWER, PARLIAMENT AND PUBLIC SERVANTS IN CANADA: MINISTERIAL RESPONSIBILITY REEXAMINED,” CANADIAN PUBLIC POLICY-ANALYSE DE POLITIQUES, VOL. 3,1979, 384.

114 SUBMISSIONS TO THE ROYAL COMMISSION ON FINANCIAL MANAGEMENT AND ACCOUNTABILITY (OTTAWA: PRIVY COUNCIL OFFICE, 1979), 1-3.115 FRANKS, PARLIAMENT, 227.116 JIM MITCHELL, “ACCOUNTABILITY,” OPTIMUM, SUMMER 1992, VOL. 23, NO. 1, 13.117 C. LLOYD BROWN-JOHN, A PROFESSOR OF PUBLIC ADMINISTRATION IN THE DEPARTMENT OF POLITICAL SCIENCE AT UNIVERSITY OF WINDSOR, CRITICIZED JIM MITCHELL’S EARLIER ARTI-

CLE, IBID. SEE THE SPRING 1993 ISSUE, VOL. 24, “ACCOUNTABILITY RE-REVISITED.”118 GORDON, PUBLIC ADMINISTRATION, 104.119 JEAN-ROBERT GAUTHIER, CHAIRMAN OF THE HOUSE OF COMMONS PUBLIC ACCOUNTS COMMITTEE, AT THE 13TH ANNUAL CCAF CONFERENCE, 23 NOVEMBER 1992.120 SIMON MCINNES AND PERRY BILLINGSLEY, “CANADA’S INDIANS; NORMS OF RESPONSIBLE GOVERNMENT UNDER FEDERALISM,” CANADIAN PUBLIC ADMINISTRATION, SUMMER 1992, VOL. 35,

NO. 2, 219.121 GORDON OSBALDESTON, KEEPING DEPUTY MINISTERS RESPONSIBLE, (LONDON, ONT.: NATIONAL CENTRE FOR MANAGEMENT RESEARCH AND DEVELOPMENT, 1988), 159.122 PRIVY COUNCIL OFFICE, SUBMISSIONS TO THE ROYAL COMMISSION ON FINANCIAL MANAGEMENT AND ACCOUNTABILITY, 1-51. 123 REPORT OF THE SPECIAL COMMITTEE ON REFORM OF THE HOUSE OF COMMONS, 21.124 SEE FRANKS, PARLIAMENT, P. 244, FOR AN ELABORATION ON THE DOCTRINE.125 ARTHUR KROEGER, “ON BEING A DEPUTY MINISTER,” POLICY OPTIONS, VOL. 13, NO. 4, MAY 1992, 5.126 IMPROVING ACCOUNTABILITY—CANADIAN PUBLIC ACCOUNTS COMMITTEES AND LEGISLATIVE AUDITORS (CCAF: 1981) 20.127 PUBLIC SERVICE 2000: THE RENEWAL OF THE PUBLIC SERVICE OF CANADA (OTTAWA: PRIVY COUNCIL OFFICE, 1990), 8.128 PUBLIC SERVICE 2000: A REPORT ON PROGRESS (OTTAWA: PRIVY COUNCIL OFFICE, 1992), 94.129 GEOFFREY MARSHALL, CONSTITUTIONAL CONVENTIONS, 2ND ED. (OXFORD, 1986), 65, AS CITED IN IBID., 96.130 KENNETH KERNAGHAN, “POWER, PARLIAMENT AND PUBLIC SERVANTS IN CANADA: MINISTERIAL RESPONSIBILITY REEXAMINED,” CANADIAN PUBLIC POLICY-ANALYSE DE POLITIQUES, VOL. 3,

1979, 386-387.131 PUBLIC SERVICE 2000, A REPORT, 96-98.132 IBID., 98-99.133 M.E.I. ET AL V. BHATNAGER, [1990], S.C.J. NO. 20771, SOPINKA, DICKSON, LAMER, WILSON, LA FOREST, L’HEUREUX-DUBÉ, GONTHIER, CORY, MCLACHLIN, 21 JUNE 1990.134 WESTLAND REFERS TO THE BRITISH HELICOPTER FIRM OF THAT NAME. THE ISSUE WAS OVER THE FINANCIAL RESCUE OF THE FIRM EITHER BY A EUROPEAN SAVIOUR AS THE SECRETARY OF

STATE FOR DEFENCE WISHED OR BY AN AMERICAN FIRM AS THE PRIME MINISTER AND THE SECRETARY OF STATE FOR TRADE AND INDUSTRY PREFERRED. THE SECRETARY OF DEFENCE TRIED

UNSUCCESSFULLY TO PERSUADE THE CABINET AND THEN WENT PUBLIC IN A MANNER THAT BROUGHT HIM WIDE SUPPORT. THIS EPISODE CREATED SUBSTANTIVE CONFUSION IN THE

MACHINERY OF GOVERNMENT AND RAISED SOME INTERESTING CONSTITUTIONAL ISSUES.135 HENNESSY, WHITEHALL, 305.136 IBID., 306.137 FOR EXAMPLE, THE AL-MASHAT AFFAIR IN OTTAWA, AND THE WESTLAND AFFAIR IN THE UNITED KINGDOM.138 KROEGER, ON BEING A DEPUTY, 4.

C H A P T E R 6

ACCOUNTABILITYIN CERTAINOTHERCIRCUMSTANCES

A C C O U N T A B I L I T Y I N T H E


For a long time, a company’s performance hasbeen assessed through traditional indicators of suc-cess: profit, share prices, price-earnings ratios, andso on. In recent years, there has been a more fre-quent use in annual reports of management discus-sion and analysis of operating results and financialcondition—MD&A. This material helps explainvariations in financial results and the causes behindthem. It does not, however, render a full accountof the effectiveness of all aspects of the enterprise.

The private sector generates a great deal ofrelevant information that can be used to assess per-formance. Stock prices and volume, product price,quarterly reports, assessment by bond-rating ser-vices, and similar indicators are all publicly avail-able.

The conventional wisdom is that the privatesector, because of its “bottom line” as the ultimateindication of effectiveness, carries its own account-ability. Hence, by contrast, the absence of such aquantification in the public sector forces it to cre-ate a substitute for the purpose of demonstratingeffectiveness. Nevertheless, the sought-after intro-duction of more businesslike methods in publicadministration, would not overcome the lack ofsuch a simple, single indicator of effectiveness.

D I F F E R E N C E B E T W E E N

P R I V A T E - A N D P U B L I C

S E C T O R A C C O U N T A B I L I T Y

The automatic accountability provided bymarket data for private corporations is useful butnotably narrow. It deals only with the parties to themarket transactions. The concern in the public sec-tor is much wider. For instance, it is possible tokeep track of contributions of workers andemployers to the unemployment insurance fund aswell as of payments of benefits to unemployedworkers. The concern of elected officials and tax-payers, however, extends beyond the efficiency oftransactions between beneficiaries and the unem-ployment insurance commission. The effectivenessof the public intervention as a whole is a majorconcern and is not readily captured by the work-ings of the administration.

In a democracy, the opportunity for taxpayers(consumers) to shift allegiance (preference) to adifferent government through elections every fouror five years constitutes accountability. It doesindeed, but it is not punctual, nor are the signalsvery rich in information to guide other than globaldecisions. To use the business vocabulary, electionstell more about consumer preference with regardto the supplier than with regard to the goodsthemselves.

G O V E R N M E N T I N A N D O U T O F

B U S I N E S S

It can be argued that as a result of the nation-alization of an enterprise, when hitherto privatelyproduced/distributed goods have been moved tothe public sector, accountability with regard to thesupplier will have been transformed but not withregard to the goods themselves. Theoretically, theconsumer can still alter his or her preferences withrespect to the goods and services. Thus, in a


nationalized context, some built-in accountabilityremains, particularly in reference to basically com-mercial activities. In reality, however, nationaliza-tion has usually taken the form of a monopoly, andthe range of options available to consumers isseverely restricted. As for the public policy aspectof a nationalized company, accountability does notarise mechanically and may indeed becomeunclear.

Conversely, in a privatization, a built-inaccountability regarding price is being acquired,but any accountability for the public policy aspects

will have been lost unless it is artificially main-tained. But it may not be necessary to maintain it.For example, if a provincial government decidedthat its only interest in the distribution of alcoholicbeverages was the revenue it provides, it mightabandon its liquor stores and get the revenue fromprivate distributors. It would not feel the need toaccount for what it considered to be nonexistentpublic policy issues.

A different example is the initiative in theUnited States to privatize some penitentiaries.Such privatizations had at least a partial built-in


ACCOUNTABILITY IN THE MARKET SYSTEM: THE MECHANICS

ALTHOUGH TODAY, HARDLY ANYONE BELIEVES THAT THE BOTTOM LINE ADDRESSES THE COMPLETE RANGE OF EFFECTIVENESS

ISSUES FACING A CORPORATION, THE WORKINGS OF THE MARKET NEVERTHELESS PRODUCE A LOT OF USEFUL INFORMATION.

A SIMPLE MODEL OF ACTIVITIES IN THE PRIVATE SECTOR COULD EXPLAIN HOW IT DOES THAT.

PRIVATE SECTOR ACCOUNTABILITY OPERATES AT TWO RELATED LEVELS. PERCEIVED DISSATISFACTION AT ONE LEVEL (TOWARDS THE

GOODS) WILL, IN TIME, CAUSE DISSATISFACTION AT THE OTHER LEVEL (THE FIRM).

IN THE PRIVATE SECTOR, THE MARKET ECONOMY, THE INTERACTIONS OF THE MARKET ITSELF, CONSTITUTE A FORM OF ACCOUNTABIL-

ITY. IT IS THE COMPETITIVE NATURE OF THE MARKET, TO THE EXTENT THAT IT IS INDEED COMPETITIVE, THAT ALLOWS CONSUMERS

TO SHIFT THEIR PREFERENCES TO ANOTHER SUPPLIER—OR TO ALTERNATIVE GOODS OR SERVICES FROM THE SAME SUPPLIER—IN

CASES OF DISSATISFACTION OR CHANGE OF TASTE. ALTERNATIVELY, CONSUMERS MAY WELL ABANDON THEIR INTENTION TO BUY ALTO-

GETHER. THUS THE SUPPLIER/PRODUCER MUST RESPOND TO SHIFTS IN CONSUMER PREFERENCES. THIS RESPONSIVENESS IS NECESSARY

TO MAINTAIN PROFITABILITY.

A PUBLIC COMPANY’S PROFIT, EXPRESSED NUMERICALLY IN ITS FINANCIAL STATEMENTS, REPRESENTS A SIMPLE FORM OF ACCOUNT-

ABILITY FROM THE FIRM TO ITS OWNERS, THE SHAREHOLDERS. IN THE EVENT THE SHAREHOLDERS ARE UNHAPPY WITH THEIR FIRM’S

PERFORMANCE, THEY CAN DISPOSE OF THEIR SHARES, SINCE OWNERSHIP CAN BE TRADED LIKE ANY OTHER GOOD OR ASSET.

IN THIS SENSE THE FREE MARKET SYSTEM FORCES ITS AGENTS TO BE RESPONSIVE, HENCE ACCOUNTABLE. FOR THIS REASON, THE

PRIVATE SECTOR IS SAID TO HAVE A BUILT-IN ACCOUNTABILITY MECHANISM, AND ITS PRICE SIGNALS ARE A PERMANENT REFEREN-

DUM ON PERFORMANCE. THE ACCOUNTABILITY IS PUNCTUAL AND INDEED CONTINUOUS; THE SYSTEM PRODUCES A VAST ARRAY OF

SIGNALS OF USE NOT ONLY TO SHAREOWNERS AND CONSUMERS, BUT AS A FEEDBACK TO THE SUPPLIER, AS WELL. IN TURN, BECAUSE

OF THE SIMPLICITY OF THE ARRANGEMENTS, THE COMPANY CAN REACT OR RESPOND QUICKLY.

THUS, IN THE PRIVATE SECTOR, ACCOUNTABILITY CAN BE SEEN AS THE SYSTEM ITSELF. IT IS, IN EFFECT, THE ALLOCATOR OF

RESOURCES. THIS ATTRIBUTE CONFERS, OSTENSIBLY AT LEAST, A SUPERIORITY OVER ACCOUNTABILITY IN THE PUBLIC SECTOR,

WHICH HAS TO BE CREATED AND CONSTANTLY NURTURED.

NEVERTHELESS, THERE ARE SEVERAL ASPECTS OF EFFECTIVENESS THAT THE DATA AUTOMATICALLY GENERATED BY THE MARKET

DOES NOT READILY REFLECT, AT LEAST NOT IN THE SHORT OR MEDIUM TERM.

accountability system: costs of operating the facil-ities were easy to calculate, and the inmates werekept inside as they should be. But obviously thecosting system could not shed light on the effica-cy of the rehabilitation of prisoners, one of theobjectives of the institutions. This points to theneed to maintain a public sector approach toaccountability at least for the public policy part ofan otherwise commercial operation that has beenprivatized.

A C C O U N T A B I L I T Y I N A

M O N O P O L Y

For companies—public or private—that aremonopolies, being accountable implies achievingpositive results within the intent of the policy thatgave rise to the granting of the monopoly. Clearattribution of costs is needed to provide the rightinformation for management to achieve cost-effec-tive performance. Ignoring the discipline of marketprices—or the markets, in general—is tantamountto being nonaccountable. Structuring the organiza-tion along business lines will usually help to clarifyaccountability by identifying the costs of specificaspects of the enterprise.

In certain monopolies, electric utilities, forinstance, years of adjustment to market situationsand responses to public policy have led to perma-nent structures of cross-subsidization between thetypes of services (production, transportation, distri-bution) and between types of client, (government,commercial, industrial, residential). The result isthat, while the nature of the business would lenditself easily to the discipline of the market and itsability to reveal performance, this cross-subsidiza-tion makes it difficult, if not impossible, to identi-fy the costs associated with different functions.Furthermore, because prices charged for certainservices do not directly reflect costs, little incentiveexists for customers to be particularly concerned

with consumption. In such circumstances the so-called discipline of the marketplace is somewhat ofan illusion.

With the current pressure to reduce costs,many analysts think that such organizations oughtto be restructured according to lines of business soas to benefit from the market orientation and gen-erate, both internally and externally, more account-ability information. It is thought that the disciplinethis would bring ought to allow decision makingleading to increased performance and better cost-effectiveness; here again the link of accountabilityto performance would be clearer.

T H E P R I V A T E N O N P R O F I T

S E C T O R

The private nonprofit sector comprises a vari-ety of institutions devoted to public causes, organi-zations often managed and governed in much lessformal ways than government or private corpora-tions. Nonprofit institutions, many of them classi-fied as charitable, often have such nobility of pur-pose, they escape both the discipline of the marketand the kind of scrutiny to which governments aretypically subjected. There are relatively few regula-tions constraining the operations of nonprofitorganizations. Models of governance of these bod-ies are not usually explicit with regard to account-ability arrangements.

P A R T I . C H A P T E R 6 . A C C O U N T A B I L I T Y I N C E R T A I N O T H E R C I R C U M S T A N C E S 7 3


139 ROGER CANS, LE MONDE, PARIS, FRANCE, AS REPRODUCED IN LE DEVOIR, MONTREAL, 14 JANUARY 1993.

GOVERNANCE OF A NOT-FOR-PROFIT

INTEREST GROUP: A CASE

ONE NOT-FOR-PROFIT ADVOCACY ORGANIZATION IS SUBSTAN-

TIAL IN TERMS OF ANNUAL REVENUES, PHYSICAL ASSETS, NUM-

BER OF EMPLOYEES, AND THE NUMBER OF COUNTRIES IN

WHICH IT OPERATES. IN MANY RESPECTS, IT LOOKS LIKE A

MULTINATIONAL CORPORATION. IT PLANS ITS ACTIVITIES

CAREFULLY TO ENSURE A COMFORTABLE STREAM OF DONA-

TIONS. THE HEAD OF THE NATIONAL BRANCH IN ONE

EUROPEAN COUNTRY WAS QUOTED AS SAYING:

“I MANAGE THIS ORGANIZATION LIKE ANY CHIEF EXECUTIVE

IN THE BUSINESS SECTOR. I MAKE PLANS, I PREPARE BUDGETS,

AND I REALLY RUN THE SHOW.” THE OBSERVER SUMMARIZED

THE PRIVILEGED SITUATION OF THE ORGANIZATION: “THE

HEAD OF THE ORGANIZATION IS A VERY FORTUNATE MANAGER;

HE HAS NO SHAREHOLDER TO REWARD, NO SUPPORTERS TO

WHOM HE OWES A RENDERING OF ACCOUNT. THE ORGANIZA-

TION IS A FOUNDATION MAINTAINED ENTIRELY AND SOLELY BY

DONORS; THERE IS NO NEED FOR AN ANNUAL MEETING. THE

DREAM OF ANY CEO!”139

C H A P T E R 7

CONTROL ANDCONTROLS

C O N T R O L

There is no accountability without responsi-bility. A responsibility cannot be fulfilled withoutcommensurate authority. The exercise of authorityrequires a structure, a procedure—control—andchosen instruments to effect it—controls.Accountability will thrive under a suitable manage-ment philosophy, an appropriate approach to con-trol, and well-chosen controls.

One has to distinguish between control,which is the exercise of power or authority, andcontrols, which are the chosen mechanisms orinstruments to effect control. Control starts withthe will to exercise authority, to manage, to admin-ister, to intervene.

To be “in control” is:• knowing where one wants to go in relation to

available resources;• being well-organized to get there;• being as cost-effective as possible;• understanding what results are to be accom-

plished; • recognizing the actual results achieved; and • demonstrating what has been achieved.

To be in control implies having agreed-uponperformance expectations: program objectives andoperational objectives, as well as service standards.Moreover, to be in control implies being able todemonstrate that you are indeed in control.

The design of controls is largely influenced bythe societal culture and the management philoso-

phy of the organization. The nature of the organi-zation has a strong influence, too: controls appro-priate for a bank may be out of place in a doctor’soffice.

A broad view of the concept of control—onethat describes control in terms of people, structure,and processes and that sees control processes asdynamic, constantly interacting with and adaptingto the organizational environment—is reflected inguidance put forward by the Canadian Institute ofChartered Accountants (CICA).140 This guidancesets out a control framework, including a set ofrelated criteria, to assist people throughout anorganization in thinking about control as it relatesto their particular circumstance and to develop,assess, and change control.

C O N T R O L A N D T H E C H A N G E

I N O R G A N I Z A T I O N A L

C U L T U R E

Canadians enjoy a great number of individualrights vis-à-vis their governments: rights to expressopinions, to choose, to be consulted, to participate,to protest, and so on. Moreover, the state has grad-ually made available means for citizens to exercisetheir rights. It was just a matter of time before citi-zens laid claim to similar rights in the privateplace—the workplace—by asking how much of theliberty enjoyed in public can be carried into thejob. Should not the rights of an employee be thesame as those of a citizen? is a different way ofdescribing this.


BUT WHAT IS GOVERNMENT ITSELF BUT THE GREATEST OF ALL

REFLECTIONS ON HUMAN NATURE? IF MEN WERE ANGELS, NO

GOVERNMENT WOULD BE NECESSARY. IF ANGELS WERE TO

GOVERN MEN, NEITHER EXTERNAL NOR INTERNAL CONTROULS

ON GOVERNMENT WOULD BE NECESSARY. IN FRAMING A GOV-

ERNMENT WHICH IS TO BE ADMINISTERED BY MEN OVER MEN,

THE GREAT DIFFICULTIES IS THIS: YOU MUST FIRST ENABLE

THE GOVERNMENT TO CONTROUL THE GOVERNED: AND IN

THE NEXT PLACE OBLIGE IT TO CONTROUL ITSELF. A DEPEN-

DENCE ON THE PEOPLE IS, NO DOUBT, THE PRIMARY CON-

TROUL ON THE GOVERNMENT; BUT EXPERIENCE HAS TAUGHT

MANKIND THE NECESSITY OF AUXILIARY PRECAUTIONS.141

J A M E S M A D I S O N

( F E D E R A L I S T 5 1 , 1 7 8 8 )

In the last decade or so, many large organiza-tions—governments, public institutions, privatesector corporations—have undergone fundamentalchanges in their approach to management, a defi-nite shift in their internal conduct. The so-calledmanagerial revolution not only seeks to introducemore efficient methods, but as a preconditionrequires a change in the culture and attitude ofemployees to guide their work-related actions.

Organizational culture became a centraltheme of the transformation. The positive attitudeor disposition of the employee, either natural orinduced, was to replace conformity to preset rulesand procedures as the driving force behind theattainment of corporate goals. To some extent,matching the personal goals of the employees withthose of the organization became a central purposeand is responsible for the widespread practice ofinvolving employees in the drafting of missionstatements for the organization and value state-ments for the employees.

T W O M O D E L S O F M A N A G E M E N T

DELEGATION

DELEGATION HAS ALWAYS BEEN A TRAIT OF LARGE BUREAUCRA-

CIES; THEY CANNOT OPERATE WITHOUT IT. SOME SENIOR MAN-

AGERS HAVE FOR A LONG TIME BEEN DELEGATING OUT OF THE

SAME BELIEFS THAT INSPIRE THE MOST RECENT MANAGERIAL

REVOLUTION. NEVERTHELESS, THERE ARE INSTANCES IN WHICH

DELEGATING HAS BEEN A PRETEXT FOR ABDICATING RESPONSI-

BILITY, OR ASSERTING POWER OVER SUBORDINATES WITHOUT

ACCEPTING THE RESPONSIBILITY: DELEGATING THE BLAME, TO

PUT IT SIMPLY. FINALLY, DELEGATING CAN BE A COVER FOR

INEPTITUDE. NEVERTHELESS, DELEGATING IS ESSENTIAL TO

SOUND MANAGEMENT.

One can contrast two models of manage-ment: the traditional model and the new approach.Interestingly, much of the literature condemns thetraditional one by calling it an “excessively bureau-cratized organization” and promotes the secondone as the “effective organization.” Of course, theold approach, where and when the sociology of thetimes and the workplace required it, was just aseffective as the new one, in theory at least. Be thatas it may, they can be described as follows:

The “excessively-bureaucratized organiza-tion”—the traditional model—has the followingcharacteristics:

• centralized, top-down decision making, verti-cal organization;

• layered hierarchy (many levels of management,some without or with very little value-added—more people in head office, fewer people in thefield, where the action happens to be);

• tight links between management, supervisorylevels, and the executants—decisions orinstructions from the top are transmitteddownward intact;

• stated goals, targets often quantified—such


objectives inspire the nature of the rules to effectcontrols;

• such controls are tied to a system of rewardand punishment; and

• the rhythm of business, the pace, is set by thetop; the organization cannot move faster thanits senior managers.

The “effective” (sometimes called “empow-ered”) organization, in contrast, displays the fol-lowing traits:

• decentralized decision making, (decisionsmade in the field or close to it—employeesare empowered to make decisions, less trafficof information upward and downward);

• horizontal organization (fewer layers of man-agement, consultative approach to decisionmaking, employee participation, client input);

• loose links and room for maneuver, innova-tion and risk taking; and

• influence of organizational values and profes-sional standards greater than reliance on in-house rules and procedures.

DELEGATION AND EMPOWERMENT

IN A COMMAND AND CONTROL MANAGEMENT CULTURE,

DELEGATION IS USUALLY UNDERSTOOD TO INVOLVE

HANDING OVER TASKS TO EMPLOYEES WHO FOLLOW

GUIDELINES, AVOID TAKING RISKS AND WHO CARRY OUT

DUTIES IN TRADITIONAL AND SANCTIONED WAYS.

EMPOWERMENT, BY CONTRAST, ENCOURAGES MAN-

AGERS, SUPERVISORS AND EMPLOYEES TO TRY NEW WAYS

OF ACHIEVING GOALS, MOTIVATING THEM TO BE CRE-

ATIVE AND INNOVATIVE IN IMPROVING THE SERVICE

THEY DELIVER. EMPOWERMENT ASKS EMPLOYEES TO

ASSUME RESPONSIBILITY FOR CHANGE AND TO BE

ACCOUNTABLE FOR THEIR ACTIONS WITHIN AN ENVI-

RONMENT WHICH ACCEPTS A DEGREE OF RISK-TAKING

AND ACKNOWLEDGES INTENT AS WELL AS RESULTS.142

In effect, the second model of management: • is more concerned with effectiveness than

with controls;• defines effectiveness both in organizational

and professional terms;• develops confidence in the professional and

support staff;• monitors performance, but allows consider-

able discretion and authority in the field; and• knows that institutional values in the “flat”

organization are propagated faster and with aminimum of distortion.

To change from the traditional style of man-agement to the new approach, large organizationshave to take highly visible actions accompanied bywell-orchestrated efforts designed to propagate thenew beliefs, sometimes using slogans and acronymsas rallying cries for the forces of revolution.

I M P R O P E R L Y C O N T R O L L E D

D E L E G A T I O N — S T R E E T - L E V E L

B U R E A U C R A T S

Too much local interpretation of overall poli-cy and procedures could result in the delivery ofsomething not originally intended. It could alsolead to a dilution of national or corporate stan-dards, thus compromising the program and possi-bly notions of fair and equitable treatment. Publicadministration literature in the United States hasreferred to this excess of delegation and empower-ment as “street-level policy-making.” It is anexpression derived from the action of so-calledstreet-level bureaucrats—for example, public healthnurses, social workers, teachers, and police—whonaturally have a lot of discretion in dischargingtheir responsibilities and making decisions on theirown. Often, demands made on their services aregreater than the resources at their disposal to satisfy

P A R T I . C H A P T E R 7 . C O N T R O L A N D C O N T R O L S 7 7

those demands; choices must be made to balancepolicy directives and standards with publicdemand.143

When faced with a situation in whichthey need to reach an accommodation betweentheir seemingly limitless responsibilities formaking decisions about clients’ problems andtheir limited capacity to effect resolutions (dueto the lack of authority, expertise, support, ortime), street-level bureaucrats remake policy bysetting new or altered priorities, limiting or

controlling clientele’s demands, rationing ser-vices, and modifying goals. These accommoda-tions and coping mechanisms form patterns ofbehavior that become the government programthat is actually delivered or implemented.Thus, street-level bureaucrats become policy-makers.

Controlling the street-level bureaucrat’sdiscretion is an enduring problem con-fronting human services agencies. Limitingdiscretion is necessary to ensure a fairer distri-bution of services to clients and to enable theevaluation of a program to meet the publicdemand for accountability.

Management control systems usually failto oversee discretion because managers nei-ther impose a routine on the unpredictablenature of street-level interactions betweenbureaucrats and client, nor do they monitorthe large number of encounters that takeplace beyond the surveillance of programsupervisors.

Thus, street-level bureaucrats remake policyas they implement it. They make decisions in theirrespective work areas to fit the realities of theirday-to-day tasks. They may not even be consciousof it. Sometimes the reality of the situation facingthem is not what the policymakers had expectedwhile preparing directives. Sometimes the execu-tion deviates from the policy because of a differ-ence in value systems between those who conceivedthe policy at head office and the people who inter-pret it at the local level. Sometimes the policy isclear, but the accompanying directives are not.

The discretion exercised by street-levelbureaucrats is consistent with the contemporaryemphasis on clients, service, and results.Empowerment acknowledges and sanctions what isbeing done already, and in light of frontline reali-ties, it is possible for a policy to be modified to be


ON HIERARCHIES

HIERARCHIES HAVE BEEN AROUND FOR A LONG TIME. THE

CONCEPT IS STRONGLY IMBEDDED IN OUR CULTURE. IT STARTS

WITH THE FAMILY ARRANGEMENT FOR THE EXERCISE OF

AUTHORITY IN WHICH PARENTS ARE RESPONSIBLE, INDEED

ACCOUNTABLE, FOR THEIR CHILDREN.

ORGANIZATIONS ACHIEVE THEIR PURPOSES BY MEANS OF A

MANAGEMENT STRUCTURE. A MANAGER GETS THE WORK DONE

THROUGH SUBORDINATES, AND WHILE EACH OF THESE IS

RESPONSIBLE AND ACCOUNTABLE FOR HIS OR HER ASSIGNED

TASKS OR ROLES, THE MANAGER IS ACCOUNTABLE TO THE

HIGHER ECHELON FOR THE WORK PERFORMED AT THE LOWER

ECHELON BY SUBORDINATES. THE LEVEL OF THE MANAGER,

HIS OR HER PLACE IN THE LADDER, MARKS THE LEVEL OF

RESPONSIBILITY.

MANAGERS TRANSMIT THE MISSION, THE OBJECTIVES, THE

GOALS OF THE ORGANIZATION TO THE WORK FORCE. THEY

PLAY TWO MAIN ROLES: TO EXERCISE CONTROL AND TO PRO-

VIDE LEADERSHIP. REGARDLESS OF HOW PROFESSIONAL AND

PROFICIENT EMPLOYEES ARE, THEY WILL EXPECT LEADERSHIP.

IN THE ABSENCE OF LEADERSHIP, THEY MIGHT EVEN CREATE

IT. THIS IS NOT TO SAY THAT A HIERARCHY IS THE ANSWER TO

EVERYTHING. SOMETIMES IT CAN BE A HINDRANCE AND THE

SHIFT TO WORKING GROUPS, PROJECT MANAGEMENT, AND

TASK FORCES, IS THE WAY TO REGAIN MOMENTUM OR ACCEL-

ERATE IT. SUCH ARRANGEMENTS, HOWEVER, ARE STILL

ATTACHED TO A HIERARCHICAL SYSTEM.

more respectful of local conditions without com-promising its basic thrust. It requires a process ofadaptation.

It should be recognized, however, that whenthe policy is for a deliberate public intervention tomodify citizens’ behaviour in particular situations(sometimes called social engineering), street-levelpolicy-making may be counterproductive. Serviceproviders sometimes get so close to the communitythey serve, they unconsciously adopt some of itsvalues. It could be that these are the very ones theprogram is designed, at least in part, to alter. Thiscan happen when there is little contact betweentop management and the field, or when proceduresallow maximum discretion and little feedback—orno accountability. To the extent that this happens,it would constitute an obvious loss of control,albeit probably insufficient to condemn empower-ment as an approach. What is needed is bettercommunication and a more effective balancebetween the amount of discretion one is preparedto allow and the amount of control one must have.As the model changes from rule-book interpreta-tion to empowerment, it puts a premium onaccountability and reporting.

D E L E G A T I O N A N D

T H E D I F F I C U L T Y O F

“ L E T T I N G G O ”

Embracing the new managerial culture doesnot come naturally to everybody in the system.First, when confronted with the necessity to dele-gate, those who are autocratic by nature will not letgo easily. They are the silent apologists for the sta-tus quo. Second, those very senior managers whoattained their positions by learning the system andhoning their skills at running things from the topare not likely to give up controls readily. They areoften replaced by people of the same persuasionpending the emergence of new standards of selec-tion. Third, many employees find it comfortableand secure to work in an environment of controlwith minimal risk; they are not ready or eager toaccept new responsibilities. Frequently, they, too,are replaced by people with the same disposition.

P A R T I . C H A P T E R 7 . C O N T R O L A N D C O N T R O L S 7 9

140 THE CANADIAN INSTITUTE OF CHARTERED ACCOUNTANTS, GUIDANCE ON CONTROL (TORONTO: CICA, 1995).141 AMERICAN CONSTITUTION, 235.142 SERVICE TO THE PUBLIC TASK FORCE REPORT (OTTAWA: PRIVY COUNCIL OFFICE, 1990), 51-52.143 THE FOLLOWING PARAGRAPHS BORROW LARGELY FROM A PAPER PRESENTED BY JUANITA CEBE AT AN AMERICAN EVALUATION SOCIETY MEETING IN WASHINGTON IN 1990 (PP. 15-16). THIS

PARTICULAR TOPIC WAS GIVEN VERY ELABORATE TREATMENT BY MICHAEL LIPSKY IN “STREET-LEVEL BUREAUCRACY,” DILEMMAS OF THE INDIVIDUAL IN THE PUBLIC SERVICES (NEW YORK:RUSSELL SAGE FOUNDATION, 1980).

CONCLUSIONK N O W T H E B U S I N E S S

The introduction to Part I includes a state-ment that effective audit practice starts with a thor-ough understanding of the context within whichthe client and other key stakeholders operate. Thiscontext involves both the governance and manage-ment processes and the accountability arrange-ments that bind them together.

Part I opens up notions of accountability andgovernance in an attempt to help those who maybe in the future or are now involved with compre-hensive auditing and/or performance reporting.The material relates primarily, but not exclusively,to the public sector.

In effect, Part I addresses a vitally importantunderpinning not only of audit but of any disci-plined approach to designing or developing man-agement and information systems; the principle issimply that those who do so must have a sufficientknowledge of the business of the organization inquestion. Failure to sufficiently understand thebusiness is tantamount to driving a car whileblindfolded—it introduces the risk of almost cer-tain failure.

As J. R. M. Wilson wrote in 1953:“[A]ccounting is more properly described as an artthan a science, because accounting, unlike science,has no rules which are in the nature of fundamen-tal truths… this is a field in which judgment isnecessary at every step… ”144 The problem thatWilson talked about in 1953 becomes exacerbatedwhen auditors are asked to give assurance on abroader span of information. Compounding thechallenge is that the information that they seek toprovide some level of assurance on will often bedeveloped by people from disciplines other thanfinancial accounting.

Having a sufficient knowledge of the businessin an audit context will help practitioners to makethe sound judgments in relation to assessing risk(both audit and the client’s), determining scope andrelated intended degrees of audit assurance andmateriality (significance) thresholds, the most appro-priate manner of reporting, the nature of controlsthat may be most effective and those that might beleast effective, the nature and reliability of differentforms of audit evidence, the nature of the perfor-mance reporting principles that may be most appro-priate, and so on. In other words, all key aspects ofthe audit process and judgments are affected by thequality of the practitioner’s knowledge.

Several dimensions to the knowledge of theorganization are required by those who undertaketo audit or design management systems and prac-tices including performance information systems.

The first is at the level with which Part Ideals. That is, with respect to the overall environ-ment within which the organization operates, andin particular with reference to understanding thegovernance and accountability precepts, principles,approaches, and arrangements that give rise to theexistence of the organization and that determinethe manner in which it is mandated and givenauthority and resources to accomplish its mandate.Included in this understanding are the principles,types of arrangements, and behavioural issues asso-ciated with the broad mechanisms that have beenadopted to control, exercise stewardship for, andaccount for the performance of institutions at alllevels. At this level, the principles apply to virtuallyall organizations in the public sector, and to manyin the private sector.

Accountability does not exist in a vacuum. Itneeds some form of a doctrinal basis to determinereference points—a value framework. To guide theiractions, people have their culture, their ethics, theirvalue system, and they usually bring these into theworkplace. The organization develops its own value


system. That system is largely influenced by the oneespoused by the organization’s leaders, by the onethat employees bring with them or share with theirleaders, and by the one that is dictated by the cor-porate purpose, legislation, rules, regulations,indeed the tradition, and the need to maintain agood reputation. Some of these values are reflectedin established procedures; some are not.

An audit has to be conducted with an under-standing of the value framework from which therelevant accountability regime is derived.

This constitutes possibly the greatest profes-sional challenge to practitioners of audit. Audit islargely a normative discipline. It tends to observedeviations from the norms or the failure to respectthem. Auditors would naturally revert to their ownpersonal value systems—their own standards andtheir own perceptions—in the absence of a gener-ally accepted framework of accountability. But theymust not. Nevertheless, given the conflictualnature of our social arrangements, such frame-works, when they exist, are not always entirely rele-vant, nor widely or generally accepted. Audit, eventhough it has given itself a set of principlesabsolutely pertinent to its calling, must be prac-ticed in a very imperfect world. But running intodilemmas is a normal feature of audit, and muchof an auditor’s effectiveness lies in the ability tohandle difficult choices.

Familiarity with parliamentary frameworks ofaccountability, contemporary management culturesand frameworks, generally accepted standards, andaccountability regimes, is not only relevant on itsown, but also brings consistency in the practice ofauditing. Some regimes or protocols are mandated,some are not. Different hierarchical arrangementspose their particular challenges with respect toaccountability structures and it is necessary todevelop a good understanding of their peculiarities.

There are situations where the accountabilityframework itself has undergone profound transfor-

mations, over a period of years, involving changesin both institutional and managerial cultures,accompanied by corresponding procedural ramifi-cations. Auditors, in such cases, within the shiftingparadigm, must develop a plausible and realistic setof norms to guide their work.

The second area of knowledge auditorsrequire involves the nature and typical operatingand managerial processes and approaches that areassociated with, and that are derived from, the spe-cific nature of the services, products, and activitiesundertaken or provided by an organization. Part Idoes not attempt to deal with this level of knowl-edge; to do so would be a task beyond the scope ofone publication. This level of knowledge is bestacquired as the practitioner or student becomesengaged in a specific assignment.

Effectively leading and ensuring the qualityand efficacy of comprehensive audits (and/or design-ing and developing performance reportingapproaches in support of governing bodies andmanagement) requires an ongoing knowledge-build-ing process about the matters introduced in Part I.Real understanding is largely contingent on in-depthexperience gained over time. Part I is also intendedto be only a starting point to the knowledge thatauditors require of the organization(s) they examine.

O R G A N I Z A T I O N A L

K N O W L E D G E C H E C K L I S T

The preceding section underlines the impor-tance of understanding by auditors of the gover-nance and accountability regimes and backgroundof clients of comprehensive audits. The following isa list of areas for which a good understanding isrequired. The range of these issues is substantial.Even so, in certain circumstances, there may beother areas where a good understanding is neces-sary for a satisfactory audit. In preparing for anaudit, auditors should ask themselves:

P A R T I . C O N C L U S I O N 8 1

Do I (or we) understand:• the informal and formal ways in which this

institution and those from whom it derives itsmandate and authority interact, including thelegal governance and accountability frame-works that are in place?

• the ways in which the principal decision mak-ers within the organization and those whoinfluence it interact to develop policy, choosedelivery mechanisms, establish performancegoals, and manage risk?

• the fundamental purposes and responsibilitiesof the governing body and have reasonableexpectations of them?

• the differences between the purposes andresponsibilities of the governing body andthose whom the governing body has empow-ered to manage the resources and administerthe policies established by it?

• the areas where governing body members andmanagement share responsibilities and theinterdependencies of both groups (especiallyas these pertain to policy setting, strategicplanning, stakeholder communications, con-trol, and stewardship)?

• what constitutes an appropriate mix of peoplewith the requisite knowledge, ability, andcommitment to exercise stewardship andensure the long-term effectiveness of the orga-nization in question?

• how the governing body interprets its purpos-es and distinguishes itself as a body politicfrom the interests of individual governingbody members or the particular interests ofthose that specific governing body membersrepresent?

• how the governing body acts to ensure thatthe organization it is accountable for meets itsobjectives?

• how the governing body fulfills its account-ability obligations to those whose interests

they represent and who have empoweredthem?

• how the governing body addresses the needfor its own efficacy and manages the qualityof governance itself?

• the meaning of accountability in both its for-mal and discretionary dimensions and theevolution of the concept in respect of theorganization in question?

• how concepts of independence, neutrality,answerability, autonomy, stewardship, andleadership are exercised in both their formaland informal senses?

• the impact of historical and current value sys-tems and ethics, mind-sets, and cultures onthe governance and accountability arrange-ments in place?

• the extent to which and the manner in whichgoverning bodies and management sharepower both formally and informally?

• the status of and major factors that haveinfluenced the quality of trust between thegoverning body and management?

• the major formal and informal inducements,rewards, and punishments that have beenprovided to encourage accountability, rela-tionships based on trust and respect, “reason-able man” or rational behaviour, and the ful-fillment of fundamental obligations ofaccountability?

• the formal and informal inducements,rewards, and sanctions, for governing bodiesto undertake their responsibilities with duediligence, including such jurisprudence asmay exist?

• the delineation between internal accountabili-ties and political and managerial accountabili-ties and how these delineations are dealt within the organization?

• the specific role, and the manner in which itis formally and informally executed, of the


governing body as a whole and segments ofthe governing body in respect of accountabili-ty and stewardship and any trends or direc-tions which relate to it? The efficacy of thesegroups?

• comparable approaches to accountability andgovernance between the public and privatesectors, the strengths and weakness of each,and the manner in which each may con-tribute to the other?

• the exigencies that affect the operation ofaccountability and governance mechanismsand approaches in different types of public-or private sector institutions—for example,commercial or quasi-commercial monopoliesas compared to competitive or quasi-competi-tive government enterprises, mixed enterprisesas compared to wholly owned enterprises,enterprises that operate at arm’s length ascompared to those that do not?

• the formal and informal linkages and delin-eation-of-control concepts versus accountabil-ity concepts and how these delineations andlinkages are recognized and provided for inthe organization?

• the bases on which the governing body andmanagement arrive at agreement on accept-able risk levels and at a balance between riskand control?

Answers to these questions, or to those thatare relevant in the circumstances, should besought, discussed, and digested before detailedplanning of an audit begins.

P A R T I . C O N C L U S I O N 8 3

144 H. C. DELL & J. R. M. WILSON, AUDITING PROCEDURES, 3D ED. (TORONTO: THE CANADIAN INSTITUTE OF CHARTERED ACCOUNTANTS, 1951), 21.

A P P E N D I X

A CASE OFCHANGE INMANAGERIALCULTURE

Numerous large organizations have success-fully made the shift from the one style of manage-ment to another.145 Out of familiarity, and becauseof the considerable documentation on the subject,the government of Canada is chosen as an illustra-tion. As one would expect in any organizationinteracting with a changing environment andlearning from its own experience, the specifics ofthe federal government system continue to evolve.This illustration, which focuses on a number ofkey developments over a decade, provides aglimpse into the thinking, considerations, andprocesses that have marked this transition in man-agement culture and style. The forces in operationand the principles articulated likely apply to alllarge organizations. After all, governments—bethey federal, provincial, or local—and large privatesector corporations share common managementand control issues, and they all use a bureaucracyto conduct their business. Governments have cen-tral agencies (such as treasury or managementboards) and departments; corporations have headoffices and branches. In many respects, it is all the same.

A substantial portion of new managementtheory has been promoted or put into practice inthe federal government. Some of the proceduresdescribed below may not have survived, as is often

the case in large bureaucracies; some may havebeen modified, while others have been ignored orabandoned. The culture may have remained, how-ever. The following describes the original plan,where the links between the theory, concepts, andprocedure are the clearest.

The first step the federal government tookwas to modify the relationship of the TreasuryBoard (the agency responsible for overseeing thefinancial and administrative practices of the gov-ernment) with the departments and agencies. In1986, the board undertook a program in whichdepartments, in full consultation with it, assumegreater responsibility in allocating resources andplanning expenditure levels and ceilings, for a peri-od of three or four years. The second step was thedevelopment of a new managerial culture withinthe departments, largely resulting in delegation toand empowerment of frontline civil servants. Athird step focused on those relations between theTreasury Board and departments that did not lendthemselves to delegation of authorities.

THE TREASURY BOARD

THE TREASURY BOARD OF CANADA IS A CABINET COMMITTEE

OF THE EXECUTIVE. IT ESTABLISHES AND COMMUNICATES TO

DEPARTMENTS GOVERNMENT-WIDE MANAGEMENT AND ADMIN-

ISTRATIVE POLICIES, ENSURES THAT MANAGEMENT POLICIES

AND ACCOUNTABILITY FRAMEWORKS ARE IN PLACE IN DEPART-

MENTS, AND REVIEWS THEIR EFFECTIVENESS.

I N C R E A S E D M I N I S T E R I A L

A U T H O R I T Y A N D


The first step was referred to as IMAA, anacronym for Increased Ministerial Authority andAccountability. It was based on the conviction that


by delegating decision making to departments andthen holding them accountable for their actions,the government would be removing some of theobstacles to cost-effective decision making. Theaim was to create an environment where decisionscan be made by the managers responsible for pro-gram delivery.

The characterization of the IMAA regime asexpecting accountability reporting as a quid proquo for administrative delegations is more mun-dane than the stated intent, which was to considerboth delegation and accountability reporting aspart of a mature and productive relationshipbetween line departments and government-widemanagement. One desired outcome was to relieveTreasury Board ministers from the tedium ofexamining thousands of departmental submissionsannually. The board, in time, would become moreeffective by moving away from a case-by-caseapproach towards governance at a higher level.IMAA was a deliberate attempt to introduce thenew approach to management and control in thegovernment.

IMAA ANTECEDENT

THE ACCOUNTABILITY PROCESS PROPOSED BY IMAA

CLOSELY PARALLELS THE PROCESS THAT THE ONTARIO

GOVERNMENT ALREADY HAD IN PLACE FOR SOME TIME.146

IMAA D E L E G AT I O N S

IMAA delegations had both a universal and adepartmental-specific component. The universaldelegations represented significantly increasedauthority limits—or, in some instances, completedelegation—in designated administrative policyareas. Compliance was to be monitored by cyclicalaudits and reported in annual management reports.

The particular delegations represented specificauthorities sought by the minister and the depart-ment. For the most part, they were negotiatedbetween officials from the secretariat of theTreasury Board and from the department. Onequid pro quo for these further delegations was anagreed-upon reporting of results-based program-performance information (with targets, as appro-priate) to replace operational information, particu-larly that which uses inputs as a proxy for the mea-surement of effort.

EX-ANTE CONTROLS

EX-ANTE CONTROLS ARE GENERAL RULES IN ADVANCE OF

DECISION MAKING AND/OR SPECIFIC RULES AT TIMES OF DECI-

SION MAKING. WITHIN THE GOVERNMENT OF CANADA, GEN-

ERAL RULES (OR INTERPRETATIONS OF THOSE RULES) ARE

LIKELY TO BE SERVICE-WIDE, ORIGINATING FROM:

• LEGISLATION: SUCH AS THE FINANCIAL

ADMINISTRATION ACT, THE PUBLIC SERVICE

EMPLOYMENT ACT; AND

• TREASURY BOARD: IN THE FORM OF ADMINISTRATIVE

AND PERSONNEL REGULATIONS, POLICIES, AND

GUIDELINES.

THERE ARE ALSO DEPARTMENTAL RULES OR INTERPRETATIONS OF

LEGISLATION AND REGULATIONS OR OF TREASURY BOARD POLI-

CIES AND GUIDELINES TO SUIT THE PARTICULAR CIRCUMSTANCES

AND DISTINCTIVENESS OF THE DEPARTMENT IN QUESTION.

This approach implied a reward, and early“memoranda of understanding” between the presi-dent of the Treasury Board and the departmentalminister specifically identified the areas of controlbeing delegated, as well as the nature of account-ability sought in exchange. The expectation of anannual accountability report referred to as anAnnual Management Report, and subsequently anin-depth review report in the last year of the period

P A R T I . A P P E N D I X . A C A S E O F C H A N G E I N M A N A G E R I A L C U L T U R E 8 5

covered by the agreement, summarized the natureof the quid pro quo. It was also understood that theaccountability reports would cover more than justthe specifics of the new delegations. In retrospect,deciding on what powers ought to be delegatedwas comparatively easier (except for a natural resis-tance to “letting go”) than the actual formulationof the accountability arrangements.

M E M O R A N D A O F U N D E R S TA N D I N G

A memorandum of understanding like theone developed under the federal initiative (for thepurpose of the IMAA procedure) can help sort outsome complex accountability relationships. Inorganizations like the federal, provincial, and localgovernments, there are multiple relationships onehas to recognize involve the substance of account-ability, as well as the identification of the parties tothe accountability relationship. The following is acomprehensive, but by no means exhaustive, check-list147 of issues to resolve in the development of agood memorandum of understanding between suchparties. It is intended as a guide to questions thatmay arise in the negotiation of an understanding.

As an accountability agreement, a good mem-orandum of understanding would dwell on thefollowing:

• clarifying the task or the responsibility;• the extent of the delegation;• clear identification of the parties to the agree-

ment;• the nature and timing of reporting back; • the criteria of performance and selection of

indicators; and • the accountability contract.

And it would be necessary to clarify what thedepartments would be accountable for:

• resource utilization;• service efforts;

• accomplishments;• results, effects, outcomes;• compliance with Treasury Board policy/guide-

lines;• administrative, management; and/or program

performance.However, it is difficult to formalize a memo-

randum of understanding unless the parties under-stand to whom the departmental minister isaccountable for the delegated responsibilities:

• the president of the Treasury Board;• the prime minister; • Cabinet;• Parliament (through standing committees of

the House of Commons).

It would be necessary to distinguish between:• the collective responsibility of the minister,

and• the departmental responsibility.

Similarly, it becomes necessary to agree towhom the deputy minister is accountable:

• the prime minister;• the minister;• the Clerk of the Privy Council;• the Committee of Senior Officials under the

Privy Council Office (COSO);the president of the Treasury Board, the min-isters of the board;

• the secretary of the Treasury Board; and incertain matters

• the chairman of the Public ServiceCommission.

To complicate the question, assuming thatthe answer is that the deputy minister is account-able to any of the above persons, it is necessary tospecify just what he or she is accountable for:

• departmental performance;• own personal performance;


• his or her relationship with the minister;• only resource utilization.

In any event, what performance do we havein mind?

• administrative;• managerial;• policy;• leadership.

And in respect of what values, what criteria?• prudence, due diligence, probity, integrity,

equity;• economy, efficiency, effectiveness (which can

include the above);• attributes of effectiveness (for example, the

CCAF approach which includes all of theabove).

Pursuing what purpose?• departmental; • superior objectives of the government (equity,

regional concern, bilingualism and multicul-turalism, and so on).

How often?• annually;• three or four years from now with an in-

depth review;• ongoing monitoring.

How detailed?• highly summarized;• comprehensive; • on significant matters only;• by exception;• tactically; • strategically.

Precise answers to every one of the above andsimilar questions are not likely to be found. Some

experienced people think that such answers maynot even be necessary. They would like to seeaccountability relationships established at a higherlevel of generality. Although they recognize thatwhile the relationships ought to be relevant withthe respective responsibilities of the parties to theagreement, they would like to see the latter lessspecific and sufficiently aerated to allow for aneffective stream of comfortable reporting and notlost in the tedium of a contractual agreement.Parties to an accountability agreement or under-standing ought to feel good about each other.Otherwise, the accountability relationship willremain largely procedural, and in a way, sterile.

R E P O R T I N G O N P R O G R A M D E L I V E RY

One serious challenge raised by IMAA whosesolution remained elusive is the specification ofprogram results and reporting on program delivery.There are significant difficulties with aggregatingmultifaceted departmental activities, the costing ofthese products or services, and their monitoringand control against predetermined benchmarks,often because of an absence of service-quality stan-dards.

Measuring immediate or ultimate outcomesof government activity is difficult. Governmentprograms tend to produce many heterogeneousand highly disaggregated outputs. It may be possi-ble to measure the output and some of the out-comes with respect to one specific product or ser-vice. It’s more difficult to find a set of measuresthat capture the essence of a complex program.The more one aggregates, the more outside factorsenter the equation to affect the outcomes, makingaccountability more arduous and less clear.Compounding the problem sometimes is theunjustified belief in “simple” indicators of perfor-mance of a complex process, a view that oftenplaces expediency ahead of common sense.


Another factor that is difficult to assess is theextent to which using a known criterion to mea-sure performance will actually result in a concen-tration by program managers on that aspect of per-formance, perhaps at the expense of broader con-siderations.

D E PA R T M E N TA L V E R S U S G OV E R N M E N T- W I D E

R E S P O N S I B I L I T Y

Another issue raised by IMAA is that ofdepartmental freedom versus government-widecorporate responsibility—the bureaucratic versionof the issue of individual versus the collectiveresponsibilities of a minister. Briefly stated, themore one delegates to departments the power tomake decisions unfettered by central-agency over-sight, the less one can confidently undertake gov-ernment-wide initiatives to respond to governmen-tal priorities.

P U B L I C S E R V I C E 2 0 0 0

The term Public Service 2000 (commonlyreferred to as PS2000) was chosen to indicate thefederal government resolve in the late 1980s tomodernize its civil service by embracing the latesttenets of the managerial revolution that was takingplace in Canada and abroad in both the public andthe private sectors. The reform was intended notonly to modify the system but also to alter the cul-ture of the bureaucracy. It was largely undertakenin response to citizens’ growing dissatisfaction withthe government. One of the major purposes of thisvast undertaking was to establish the primacy ofthe citizen—the taxpayer, the client—in the day-to-day provision of government services.

PS2000 had four main themes:148

• improved service to the public through focuson results and client-oriented delivery;

• empowerment of public servants to use

resources in the most creative and responsiblemanner;

• dealing with public servants in a way thatreflects the critical importance of people tosuccessful government operations; and

• emphasis on accountability.

DELIVERING QUALITY SERVICE TO CLIENTS

AIMING TO IMPROVE QUALITY OF SERVICE TO CLIENTS RAISES

A CONCEPTUAL PROBLEM BECAUSE QUALITY MEANS DIFFERENT

THINGS TO DIFFERENT PEOPLE. FOR SOME, QUALITY IS SIMPLY

VALUE FOR MONEY. FOR OTHERS, QUALITY IS WHEN THE

GOODS ARE DELIVERED ON TIME, ERROR-FREE, AND HASSLE-

FREE. TO SOMEONE ELSE, QUALITY IS FOUND IN EQUITY AND

FAIRNESS, AND EQUALITY OF OPPORTUNITY. FINALLY, TO

ACTIVISTS, QUALITY IS LARGELY A FUNCTION OF THE EXTENT

TO WHICH THEY HAVE HAD AN OPPORTUNITY TO PARTICIPATE

IN THE DECISION MAKING CONCERNING THE PROGRAM IN

QUESTION. THERE IS VALIDITY IN EACH OF THESE VIEWS.

IMAA and PS2000 go together; one followedthe other. PS2000 would have been possible with-out IMAA, but probably ineffective. PS2000 wasactually an extension of IMAA within the depart-ments, pushing the delegation further towardsfrontline civil servants. Delegation, empowerment,and accountability form the basic vocabulary of thenew culture. Controls were to be relaxed, andfewer; those that remained were meant to be morestrategic and penetrating. Rules and procedureswere to be reduced and simplified. The applicationof what remains was to be delegated, to the extentpossible. Frontline civil servants were to acquire anew authority: they would interpret the rules andbe encouraged to exercise judgment—even accept-ing reasonable risks. In short, they were to beempowered. Thus, programming or service deci-sions would increasingly be inspired more by the


reality of the situation at hand than by rules andprocedures emanating from departmental head-quarters or the Treasury Board.

In this evolving environment, achievingresults in a cost-effective manner and the necessityof demonstrating their effectiveness become theessence of the role of frontline civil servants andtheir supervisors. Accountability is expected. In theabsence of feedback from the field to headquartersor Treasury Board, the government could not pos-sibly gauge the effectiveness of the public interven-tion and the continued need for it.151 The same istrue for large private-sector corporations.

EMPOWERMENT IS NOT ALWAYS LIMITED TO THE MAN-

AGERS AND EMPLOYEES OF AN ORGANIZATION. THERE

ARE MANY PUBLIC DECLARATIONS INTENDED TO EMPOW-

ER THE CITIZEN OR THE CUSTOMER, SPELLING OUT

THEIR RIGHTS AND MANNER OF REDRESS AND OFFERING

A SAY IN DECISION MAKING. SOME HOSPITALS IN CANADA

HAVE A BILL OF RIGHTS OF THE PATIENTS. IN THE

UNITED KINGDOM THERE IS A TENANTS CHARTER, A

BRITISH RAIL PASSENGER CHARTER, A NATIONAL

HEALTH SERVICE PATIENTS CHARTER, AND OTHERS.

In the first move, Treasury Board, which wasthe custodian of the “rule book” and indeed itsinterpreter, passed it to the department under anIMAA agreement. In the second move, the depart-mental headquarters gave the rule book to thefrontline civil servant or supervisor. This was aPS2000 procedural change.

S H A R E D M A N A G E M E N T

A G E N D A ( S M A )

In 1991, the introduction of SMA followedPS2000 and IMAA. It was derived from the notionof partnership, an administrative doctrine that


EMPOWERMENT

EMPOWERMENT IS NOT THE ABDICATION OF RESPONSIBILI-

TIES. NOR IS IT SIMPLY THE DELEGATION OF AUTHORITY.

TRUE EMPOWERMENT MEANS THAT MANAGERS WITH DELEGAT-

ED AUTHORITY ARE PROVIDED WITH THE INFORMATION, THE

TOOLS, THE TRAINING AND THE SUPPORT THEY NEED TO

ACHIEVE DESIRED RESULTS.149

THE FOLLOWING IS BORROWED FROM THE TREASURY BOARD

OF CANADA:150

THE OBJECTIVE OF EMPOWERMENT:

LEADERSHIP AND MANAGEMENT PRACTICES AND BEHAVIORS

THAT EMPOWER EMPLOYEES BY:

• ENABLING THEM TO EXERCISE AUTHORITY, WITH DISCRE-

TION AND GOOD JUDGEMENT, OVER THEIR AREAS OF RESPON-

SIBILITY;

• PLACING AUTHORITY AS CLOSE AS POSSIBLE TO THE POINT

OF SERVICE DELIVERY, REMOVING OBSTACLES; AND

• PROVIDING THE FRAMEWORK AND SUPPORT NEEDED FOR

CLIENT-FOCUSED RESPONSIVENESS AND INNOVATION.

THE INDICATORS OF EMPOWERMENT:

• A BETTER WORKING UNDERSTANDING OF EMPOWERMENT,

INCLUDING THE CONDITIONS UNDER WHICH CREATIVITY,

INITIATIVE, TRUST, TEAMWORK, AND EXCELLENCE IN THE

DELIVERY OF GOVERNMENT SERVICES CAN FLOURISH;

• AVAILABILITY TO MANAGERS OF INFORMATION, CONCRETE

EXAMPLES, EXEMPLARY PRACTICES, AND SPECIALIST SUPPORT

THAT FURTHER APPROPRIATE EMPOWERMENT OF EMPLOYEES,

PARTICULARLY THOSE PROVIDING FRONTLINE SERVICE;

• AN INCREASE IN DELEGATION THAT PUTS AUTHORITY

WHERE IT HAS THE MOST IMPACT ON SERVICE AND SUPPORTS

SERVICE-RELATED EMPOWERMENT OF EMPLOYEES; AND

• IDENTIFICATION OF HUMAN RESOURCES IMPLICATIONS OF

OPERATING BUDGETS AND DEVELOPMENT OF MODELS AND

CONCRETE APPROACHES THAT MAKE IT EASIER TO USE

OPERATING BUDGETS EFFECTIVELY FROM A HUMAN

RESOURCES PERSPECTIVE.

acquired currency in the late 1980s and begot theexpression “shared agenda.” SMA is simply the per-ception of management, internal to an institution,or a government department or agency, as a part-nership. It is construed as a forum of discussion,largely paperless and highly personal. The processwas intended to have Treasury Board and thedepartment agree to share a limited number ofmanagerial and administrative issues at a highlevel—that is, between the Treasury Board Secretary(TBS) and the departmental deputy minister.

SHARED MANAGEMENT AGENDA

SMA IS THE MOST RECENT IN THE LONG LINE OF

ACRONYMIC MANAGERIAL FRAMEWORKS THAT TREASURY

BOARD HAS FOSTERED OVER THE LAST THREE DECADES:

PPBS, OPMS, MBO, OPFS (OPERATION PLANNING

FRAMEWORK SYSTEM), IMPAC (IMPROVED

MANAGEMENT PRACTICES AND CONTROL), PEMS

(POLICY AND EXPENDITURE MANAGEMENT SYSTEM),

IMAA, PS2000, TQM, AND THIS ONE, SMA. EACH OF

THESE INITIATIVES RESPONDED TO THE EVOLVING NEEDS

OF THE TIME AND CONTRIBUTED TO THE NEXT ONE.

Parallel and complementary priority settingon managerial issues within TBS and the depart-ment was the main focus of SMA. Areas in need ofmutual support were identified, and agreement wasreached on departmental initiatives and resultingsubmissions to the Treasury Board.

SMA also served the annual TBS departmentmanagement assessment as an input for the COSO(Committee of Senior Officials) performance eval-uation of deputy ministers.

The SMA differed from IMAA in severalrespects: IMAA operated at the political level, theTreasury Board ministers and the departmental

minister. SMA was at the managerial/administra-tive level, the Secretary of Treasury Board and thedeputy minister. The main focus of IMAA wascomprehensive and largely on delegation andaccountability; that of SMA had to do with select-ed management issues, which can be construed asan aspect of managerial accountability.

While SMA as a TBS initiative was createdchronologically later than IMAA, it is intended tostand alone.

From the point of view of accountability,IMAA provided the vehicle for ministers to beaccountable with respect to the authorities delegat-ed by the Treasury Board. Only indirectly canIMAA be seen as providing an assessment of thepersonal performance of the deputy minister. TheSMA process, with the emphasis it placed ondepartmental management and its adherence togovernment-wide administrative policies, provideda good mechanism to assess the actual performanceof the deputy minister in areas of relevance to theTreasury Board. This assessment was then fed toCOSO which is responsible for the overall personalperformance evaluation.

SMA outlined the global criteria to guide theassessment of departmental management. It used asa model a contemporary framework that includesattention to results, the client, leadership, employeeparticipation in decision making, accountability,and so on: in short, all of the notions propagatedby the PS2000 initiative. As such, SMA was therealization of PS2000, assessing departmental man-agement and the deputy minister using its preceptsas criteria. It also encouraged the response ofdepartments to such government-wide obligationsas downsizing, employment equity, official lan-guages, and in extending cooperation to otherdepartments, sharing of good practices, andtransparency in the department’s dealings with thecentral agencies.


W A S C O N T R O L L O S T ?

Taken together, IMAA and PS2000 wereredefining control and controls in the federal gov-ernment, a fact that generated skepticism in certainquarters. There have been serious debates amongsenior management, financial managers, and audi-tors alike over the apparent reduction in control ata time when some concerned Canadians and criticsare calling for more or at least better.

Nothing in the description above suggeststhat control had been lost. The Treasury Board wasstill very much in control. In a way, IMAA was thecontrol; so was PS2000, its extension. Controls,however, changed. There was no abdication of con-trol because of delegation, just a change in themechanisms to exercise it, the controls themselves.The rules were still there. The rule book, instead ofstaying in the Treasury Board, was shared andplaced in the hands of frontline civil servants forinterpretation and application. They have to followthe rules. But not every situation that can arise inthe field will have been foreseen by the authors ofthe rule book. This well-understood possibility ledthe framers of PS2000 to empower frontlineemployees and exhorted them to use their judg-ment in such cases. Thus, the interpretation of therules was to be made in the light of the local situa-tion instead of “checking with head office” for aruling by someone who cannot have a feel for thesituation and who has only the possible advantageof having heard of similar problems previously.

Some people in senior management do notfeel comfortable with this type of delegation asthey are used to formal procedures and top-downcontrols. Some employees who have received dele-gation are not used to handling such responsibili-ties, being accustomed to acting on instructions,doing what they are told, and deriving a certainlevel of comfort from that approach.

SOME HAVE RESERVATIONS…

THERE HAS BEEN SOME CYNICISM ABOUT THESE NEW

APPROACHES. MANY PEOPLE LOOK AT THEM AS FADS. USE OF

SUCH TERMS AS EMPOWERMENT AND TOTAL-QUALITY MANAGE-

MENT AND THEIR VACUOUS REPETITION WITHOUT TRANSLAT-

ING THEM INTO THEIR BUREAUCRATIC IMPLICATIONS DOES

NOT HELP PUT THEORY INTO PRACTICE. TO BE SUCCESSFUL,

EMPOWERMENT HAS TO BE INCORPORATED IN THE ACTUAL

PROCEDURAL WORKINGS OF THE ORGANIZATION, AND TO

DATE THERE HAS BEEN MUCH MORE TALK THAN ACTION. IN

ADDITION, LOW QUALITY AND LACK OF RESPONSIVENESS TO

THE CUSTOMER MAY HAVE FAR MORE TO DO WITH THE STRUC-

TURE OF THE ORGANIZATION, ITS BASIC PROCESSES AND PRO-

CEDURES, OR ITS LACK OF VISION AND PURPOSES, THAN WITH

THE EMPLOYEES THEMSELVES.

Senior managers who are concerned aboutdelegation ought to feel equally worried aboutempowerment. They would be right if there is rea-son to believe that frontline employees have insuf-ficient knowledge and understanding to be able toreconcile the intent of the program with the natureof an unforeseen demand being made on it. Inprinciple, however, if the employee is familiar withthe program, is properly trained, knows the rules,knows the clientele, gives indications of beingresponsible, and is held accountable, there is noreason for senior managers to apprehend a loss ofcontrol.

Many of those who feared a loss of controlunder the new approach believe that the old, moredetailed and centralized system guaranteed compli-ance. However, substantial evidence indicates that,under the previous managerial culture, central con-trol and centralized controls through a hierarchicaltop-down management structure were not a guar-antee of compliance. The existence of many levelsof control or supervision—indeed, the excessivelayering—made the process so heavy and paper-


burdened that in some cases frontline civil servantsbecame adept at exploiting loopholes in the rules.They effectively ignored directions cascading downfrom headquarters and ended up doing what theyjudged was best, presumably in favour of theclient. They might as well be empowered, with theadded safeguard of being held accountable; theyalready felt responsible.

T H E T R A N S I T I O N C O N T I N U E S

In February 1995, the federal governmentannounced the establishment of a new expendituremanagement system.152 One of its objectives is to“enhance accountability through a focus on perfor-mance and better public information.”

Two key features of this system are depart-mental Business Plans and departmental Outlooks.Their introduction built on the philosophy of pastinitiatives, such as the IMMA regime and SMAprocess, but replaced or modified their proceduralaspects. These Business Plans and Outlooks areintended to provide departments, Treasury Board,and Parliament with the means to understand,communicate, and review where the department oragency is going, how it’s going to get there, andrelated resource and service implications.

Departmental Business Plans are based on theservice lines or major businesses of departments.They are intended to draw departmental ministersand deputy ministers into direct dialogue with theTreasury Board on issues of strategic import,including the impact of planned adjustments in ser-vice, potential funding pressures, and program risks.

Business Plans are change documents thatthey place emphasis on the incremental nature ofactivities. They integrate the key program, resourceand policy elements that are critical to a depart-ment in making business adjustments, and theyprovide a basis for departments and the TreasuryBoard to work together to address related chal-

lenges. Moreover, they provide the means by whichdepartments can propose specific authorities orflexibility that they need in order to implementchange strategies.

Departmental Outlooks expand the scrutinyrole of parliamentarians. They assist parliamentarycommittees in carrying out their new responsibili-ties for reviewing future expenditure trends andpriorities, and provide a context for their examina-tion of Estimates.

These Outlooks are documents that depart-ments, through their ministers, provide to theirrespective standing committees. They draw on theresults of the business-plan process and, based onresources allocated in the budget, set out depart-mental strategies to adapt to the fiscal and policyenvironment and to meet specific service targets.As a starting point for discussion with standingcommittees, these Outlooks explain the back-ground context for departmental strategies andchanges, departmental priorities, new directionsand their implications, key initiatives being takento adjust to significant change, the issues and chal-lenges related to important stakeholders such asother departments, and performance indicators togauge the impact of adjustments on the qualityand level of service.

T H E M A N A G E R I A L

R E V O L U T I O N : C U L T U R A L A N D

P R O C E D U R A L

PS2000 and similar initiatives in both thepublic and private sectors have been labelled a cul-tural revolution. The original belief was that thechange in culture would be slow—some were say-ing one generation—while the systems could bechanged quite quickly. Some observers think thatthe culture was already largely there: frontlineemployees were ready; middle management, to alarge extent, was ready; it was only a matter of time


before senior management would notice and takeappropriate action. Those observers would alsonote that it is taking a long time for the philoso-phy to be translated into operational procedures.

The managerial revolution may well be theonly appropriate framework for the emerging gen-eration of managers and non-managers alike. Themanagerial class of today was largely brought up toenjoy greater freedom than was allowed to previousgenerations; culturally, the family setting precededthe workplace.


145 EXAMPLES OF EXPERIENCES IN BRITISH COLUMBIA AND ONTARIO ARE GIVEN IN CANADIAN PUBLIC ADMINISTRATION/ADMINISTRATION PUBLIQUE DU CANADA, SUMMER 1992 AND SPRING

1991, BY KENNETH KERNAGHAN AND BRIAN MARSON, RESPECTIVELY.146 FOR A FULL DESCRIPTION OF THE APPROACH OF THE ONTARIO GOVERNMENT, SEE ANNUAL REPORT OF THE PROVINCIAL AUDITOR OF ONTARIO FOR THE FISCAL YEAR ENDED MARCH 31, 1986. 147 SEVERAL ELEMENTS OF THE CHECKLIST MAY HAVE BEEN USED UNDER IMAA. HOWEVER, THIS PARTICULAR CHECK-LIST WAS DEVELOPED BY ONE OF THE AUTHORS OF THIS BOOK USING

SOME MATERIAL FROM IMAA PROCEDURES BUT LARGELY RELYING ON THE LOGIC AND UNDERSTANDING OF THE MACHINERY OF GOVERNMENT.148 IAN CLARK’S PRESENTATION AT A CCMD (CANADIAN CENTRE FOR MANAGEMENT DEVELOPMENT) ARMCHAIR DISCUSSION ON 5 MARCH 1992, “SHARED MANAGEMENT AGENDA: IMPROV-

ING MANAGEMENT THROUGH PARTNERSHIPS”.149 REPORT OF THE AUDITOR GENERAL OF CANADA TO THE HOUSE OF COMMONS, 1992, 204.150 HUMAN RESOURCES DEVELOPMENT BRANCH OF THE TREASURY BOARD SECRETARIAT ON BEHALF OF THE HUMAN RESOURCES DEVELOPMENT COUNCIL, “STRATEGIES FOR PEOPLE, AN

INTEGRATED APPROACH TO CHANGING THE PUBLIC SERVICE CULTURE,” JULY 1992.151 THIS, OF COURSE, IS NOT TRUE OF ALL SERVICES, OR ALL LEVELS OF GOVERNMENT. THE POPULATION AND ITS REPRESENTATIVES DO NOT NEED THE SAME KIND OF ACCOUNTABILITY OF

FRONT-LINE CIVIL SERVANTS TO FIND OUT IF SOME SERVICES ARE EFFECTIVE OR NOT, FOR EXAMPLE IF THE WATER IS CUT OFF, IF THE SNOW IS NOT REMOVED, OR IF THE HYDRO BLACKS OUT.152 SEE PLANNING AND COMMUNICATIONS DIRECTORATE, TREASURY BOARD OF CANADA, THE EXPENDITURE MANAGEMENT SYSTEM OF THE GOVERNMENT OF CANADA (OTTAWA: TREASURY

BOARD, 1995).

P A R T I I

PERFORMANCEREPORTING

A C C O U N T A B I L I T Y , P E R F O R M A N C E R E P O R T I N G , C O M P R E H E N S I V E A U D I T - A N I N T E G R A T E D P E R S P E C T I V E 9 5

REPORT ME AND MY CAUSE ARIGHT.

W I L L I A M S H A K E S P E A R E , H A M L E T , V . I I . 3 5 3

P A R T I I



Accountability, as we have seen, is the obliga-tion to render an account for a responsibility con-ferred. Part II focuses on the manner in which thatresponsibility is discharged.

The ways in which people discharge theiraccountability obligations has been a matter ofmuch public attention. Over the years, theCanadian public sector has been subjected to stud-ies by royal commissions, parliamentary anddepartmental committees, special inquiries, andacademic and consultant assessments. A number ofthese relate directly to how the organization definesand conducts its business and how it reports onwhat it has done.

Some of the traditional means of reportingthe performance of public bodies have continuedunchanged; others have changed substantially; stillothers have been abandoned. New mechanismshave been introduced: systematic external andinternal audits of performance, for example, arerelatively new tools, as is professional programevaluation. The concept of management represen-tations on performance is newer still.

Rules governing the self-regulating profes-sions and legislation affecting accountabilityreporting in the private sector have been periodi-cally reviewed and amended to conform to evolv-ing norms and expectations. Canadians can by nomeans be accused of ignoring their accountabilitypractices. Part II discusses the current theory andpractice of accountability reporting. It focuses on

A C C O U N T A B I L I T Y , P E R F O R M A N C E R E P O R T I N G , C O M P R E H E N S I V E A U D I T - A N I N T E G R A T E D P E R S P E C T I V E 9 7

reporting about performance or effectiveness bymanagement to governing bodies and by thosebodies to the constituencies to which they areaccountable; in short, with reporting on the dis-charge of established policy. Accordingly, Part IIdoes not discuss performance reporting at lowerlevels in an organization. Furthermore, in keepingwith the intent of the book, its focus goes wellbeyond the reporting of financial performance,though this is an important subject.

The term reporting connotes much more thanthe physical act of making a report. It involves theconceptual framework underlying reports: thedetermination of the audience and that audience’sneeds; the decisions of what should be reported,how, and how frequently; and consideration ofwhat must be done to obtain the informationreports will contain. Reporting on performance canand should be distinguished from measuring it.While Part II refers to measurement at variouspoints, its emphasis is on reporting, not measure-ment, techniques. It provides a set of principlesthat, when put into practice, can draw on the bestpractices and products of such existing measure-ment and analysis processes as policy research, pro-gram evaluation, and program monitoring.

While the discussion concentrates on thepublic sector, many of the principles discussedapply to the private sector, as well.

The term management as used here refers tothat group of individuals who are the full-time,paid leaders of organizations. Some members ofgoverning bodies have management responsibilitiesand must, in certain circumstances, be consideredin this category. For example, at the federal andprovincial levels of government, cabinet ministershave specific management responsibilities, assignedby law, that they fulfill in collaboration with theirdeputy ministers and public service managementteams. Analogous situations exist in the municipal,health care, and other parts of the public sector. In


A BRIEF HISTORY

THE HISTORY OF THE MACHINERY AND ADMINISTRATIVE

PRACTICES OF THE GOVERNMENT OF CANADA IN THIS CENTU-

RY, ECHOED TO A GREATER OR LESSER EXTENT IN THE

PROVINCES, IS ONE OF ALMOST CONSTANT FLUX. THE

ACCOUNTABILITY REGIME HAS NOT BEEN IMMUNE FROM

CHANGE. PATRONAGE IN PERSONNEL MATTERS WAS SUBSTAN-

TIALLY OVERCOME BY THE INTRODUCTION OF THE MERIT SYS-

TEM IN 1918. THE SUBSEQUENT UNIONIZATION OF PUBLIC

SERVANTS, WITH ITS ACCOMPANYING GRIEVANCE SYSTEM (THE

GROWTH OF COLLECTIVE BARGAINING REALLY TOOK OFF IN

THE 1960S AND 1970S) INTRODUCED WHAT SOME SEE AS A

NEW ACCOUNTABILITY REGIME. SWINGS BETWEEN CENTRAL-

IZATION AND DECENTRALIZATION HAVE BEEN RELATIVELY FRE-

QUENT. A COMPTROLLER OF THE TREASURY WAS CREATED IN

THE 1930S TO HANDLE THE PREAUTHORIZATION OF ALL

EXPENDITURES FORMERLY DONE BY THE AUDITOR GENERAL.

THAT POSITION WAS ABOLISHED IN THE 1960S. THE OFFICE

OF THE COMPTROLLER GENERAL WAS ESTABLISHED IN 1978,

BUT IN 1993 ITS FUNCTIONS WERE ABSORBED INTO THE

TREASURY BOARD SECRETARIAT AND THE SECRETARY OF THE

TREASURY BOARD ASSUMED THE ADDITIONAL TITLE OF

COMPTROLLER GENERAL. VARIOUS MANAGING AND BUDGET-

ING TECHNIQUES HAVE BEEN INTRODUCED (WITH A VIRTUAL

HAIL OF ACRONYMS) AND THEN ABANDONED. GOVERNMENT-

OWNED BUSINESS CORPORATIONS WERE FORMED; PRIVATIZA-

TION PURSUED.

FLUX INDEED. SEEN IN THESE TERMS, ONE MIGHT CONCLUDE

THAT CANADIAN PUBLIC ADMINISTRATION HAS BEEN IN THE

HANDS OF PEOPLE PREPARED TO ABANDON PROMISING INITIA-

TIVES IN FAVOUR OF THE LATEST MANAGEMENT ACRONYM. NOT

SO. THE HISTORY OF PUBLIC ADMINISTRATION IS ONE OF A

CONTINUOUS SEARCH FOR BETTER CONTROL, EFFECTIVENESS,

AND ACCOUNTABILITY IN THE PROVISION OF PUBLIC SERVICES.

INCREMENTAL CHANGE IS THE ONLY OPTION WHERE THERE IS

NO PERFECT SOLUTION.

the private sector, executive officers of corporationsfrequently find places on the company’s boards ofdirectors. By extension, management includes themanagement cadre of an organization: that hierar-chy of managers responsible, as delegated, for theperformance of various functions or programs ofthe organization.

The term governing body refers to the groupof elected or appointed officials who exercise over-sight and general direction of organizations.Examples of governing bodies are:

• federal and provincial legislatures;• municipal and regional councils;• boards of directors, governors, and trustees of

hospitals and school boards;• appointed and elected boards of governors,

trustees, directors, or academic senates of uni-versities and community colleges;

• advisory, planning, and regulatory councilsand agencies;

• boards of social service agencies, charitableorganizations, arts, cultural, interest group,and professional organizations;

• boards of state enterprises, majority owned oreffectively controlled by governments or theiragencies;

• in the private sector, boards of directors ofprivate or publicly held corporations.

It is to such governing bodies that managementmust report.

Performance entails accomplishments as relat-ed to efforts or capabilities and intent. The wordperformance is often accompanied by a contextualmodifier, such as public sector performance, admin-istrative performance, management performance—indeed, the performance of a manager or of theteam, program performance, or organizational per-formance. To be effective means to have the capa-bility to cause an intended result, or to have actual-ly achieved it. The word effectiveness in manage-

ment and administration literature and in dailyparlance is also often modified by the same wordsas performance: program effectiveness, managerialeffectiveness, organization effectiveness, and so on.Thus, in the context in which they are used, per-formance and effectiveness share meanings and, asin this book, are often used interchangeably,although effectiveness is the term most used fromhere on. A more careful examination of the con-cept of effectiveness is found below.

Part II is organized into two sections. Section1 reviews the ways that performance has common-ly been reported and discusses some of the concep-tual issues involved. Section 2 suggests a frame-work and approach for such reporting.

Although contemporary organizations typical-ly produce a wide variety of information (asdescribed briefly in chapter 8) both for internal useand external distribution, it is often difficult toobtain a complete picture of an organization’seffectiveness. A major reason for this is that theconcept of effectiveness is itself difficult to grasp.For management to be truly accountable, and forgoverning bodies to discharge fully their fiduciaryresponsibilities, it is essential that this difficulty beovercome.

As chapter 9 explains, no single definition ofeffectiveness offers a practical basis for determiningthe extent to which an organization is effective. Itis possible, however, to determine what characteris-tics an organization should display if it is to be suc-cessful in fulfilling its mission. The twelve attributesof effectiveness presented in chapter 10 cover themost important aspects of an organization’s success,or lack of it. Depending on the organization orcircumstance, some of those attributes will bemore important than others. Indeed, in certaincases, one or more of them may be irrelevant.Nevertheless, by obtaining reliable informationrespecting these attributes, or all that are relevantin the circumstances, a good overall view of an

P A R T I I . P E R F O R M A N C E R E P O R T I N G 9 9

organization’s effectiveness or performance can beobtained.

The contention is that responsible manage-ment should be prepared to analyze its perfor-mance and report its findings to the governingbody. Management—and management alone—has,or should have, the information upon which tobase such a report. Only by doing this will man-agement be truly accountable. And only with suchinformation can a governing body make theinformed decisions necessary to discharge itsresponsibilities appropriately and with confidence.

In chapter 10, a mechanism for performancereporting is suggested: management representa-tions. These are explicit statements that seniormanagement makes to its governing body about anaspect of the organization’s effectiveness or perfor-mance. If management makes representations onall the relevant attributes of effectiveness, aninformed conclusion may be drawn about thequality of its stewardship of funds and otherresources entrusted to it for the accomplishment ofthe organization’s mission.

Preparing management representations oneffectiveness is no easy matter, as chapter 11explains. Experience shows that few organizationshave all the necessary information upon which tobase reliable representations on the suggestedattributes. Nevertheless, most organizations thathave attempted such performance reporting havefound it possible to draft representations that sig-nificantly improve the information on which thegoverning body can rely in exercising its oversightresponsibilities. Where gaps are found in the infor-mation needed to prepare representations, deci-sions can be made subsequently about whether tofill them and, if so, how.

Certain preconditions should be met beforean organization embarks on making managementrepresentations on effectiveness. Paramount amongthese is a degree of trust between management and

the governing body. Organizations that have beensuccessful in preparing these representations usuallyhave a history of trying to improve their informa-tion and accountability systems. Organizations in astate of major transition or crisis probably shouldwait until senior management, which is vital to thesuccess of the exercise, has sufficient free time todevote to the difficult issues that must be dealtwith in preparing representations.

Chapter 12 offers detailed advice about howmanagement could go about the difficult task ofdrafting appropriate representations. The approachoutlined in chapter 12, however, is not the onlyapproach that can be taken by management.Indeed, it is essential to recognize that different cir-cumstances may call for different approaches.

There is, of course, the danger that manage-ment’s reporting may be either biased or incomplete.Where this is an issue, or where the governing bodywants added assurance that what it is receiving is fairand complete, independent validation can beengaged. The role of auditors in this context is dis-cussed in detail in Part III of this book.

The suggested framework and approach toperformance reporting is not intended as a tem-plate or a panacea. There is no intention to be pre-scriptive. All or part of the framework may be use-ful in particular circumstances. Nor is it being sug-gested that the measurement and analysis processesand systems operating within an organization (forexample, evaluation, internal audit, policy research)and feeding into the production of the necessaryinformation should be reshaped in the likeness ofthe framework. On the contrary, the framework isintended as a flexible tool or broad set of principlesto help management and their governing bodiesconsider the performance of their organization anddetermine which aspects need to be reported, andhow to achieve that so as to convey a fair and com-plete picture of performance. The framework andsuggested approach are designed to provide a

A C C O U N T A B I L I T Y , P E R F O R M A N C E R E P O R T I N G , C O M P R E H E N S I V E A U D I T - A N I N T E G R A T E D P E R S P E C T I V E1 0 0

benchmark against which to apply the complete-ness test for the performance reporting made,accepting that the internal processes used and theform and structure of the actual report eventuallymade may vary from organization to organizationand circumstance to circumstance.

P A R T I I . P E R F O R M A N C E R E P O R T I N G 1 0 1

S E C T I O N 1

THE THEORETICAL BASISFOR PERFORMANCE

REPORTING


C H A P T E R 8

EXISTINGREPORTINGPRACTICES

Managements report on their performance totheir governing bodies through a wide array ofmechanisms. Some are found in both the publicand private sectors, others are specific to the orga-nization concerned.

P U B L I C S E C T O R R E P O R T I N G

M E C H A N I S M S

O R A L R E P O R T I N G

Probably the best-known way that govern-ments account for their performance is through thequestion periods of provincial and federal legisla-tures. Television has given this daily exposure torigorous examination an importance and publicfamiliarity not always comfortable for hard-pressedministers of the Crown. Ministers take these ses-sions seriously, spending time to prepare for thecoming ordeal. Although it has been described asthe essence of parliamentary democracy, severalfactors conspire to make Question Period anincomplete accountability reporting mechanism.

The legislature is a political forum. It is thecockpit of partisan dispute. As a result, questionsand answers tend to focus on what will scorepoints for political advantage. Often concentratingon issues found in the headlines, questions are fre-quently designed as much to embarrass as toextract information. Because topicality plays suchan important role in determining what questionswill be asked, there is no attempt to hold all

aspects of government activity to account, and nooccasion for ministers to tell everything that theymight want to make public.

In answering questions, ministers are con-strained not to lie—the sanction if they are caughtout is usually resignation. Nevertheless, at times, itis expedient, even necessary, to shade or shelter thewhole truth, for any number of reasons, somequite honourable. And apart from what can be dis-covered by investigative reporters or assiduousopposition researchers, there is no independentcheck on the replies that are given to questions inthe House.

Debates on the floor of the House provideother opportunities for ministers to account fortheir actions, and for opposition members to com-ment on performance. In general, debates sufferthe same drawbacks as Question Period: the forumis political, and the subject matter limited.

Somewhat similar constraints apply in meet-ings of legislative committees, although the parti-san tone is usually substantially reduced, and oftenpublic servants are present to provide factualinformation. Since their procedures are usually lessformal than House debates, committees can digdeeper into details than is possible in the House.In addition, most legislatures structure their com-mittees to cover the entire range of governmentactivity, providing at least the opportunity forcomprehensiveness. In reality, however, time con-straints limit the areas than can be explored indepth. Except when explosive issues are being dis-cussed, media coverage of committee proceedingstends to be minimal.

I REPEAT… THAT ALL POWER IS A TRUST—THAT WE ARE

ACCOUNTABLE FOR ITS EXERCISE—THAT, FROM THE PEOPLE,

AND FOR THE PEOPLE, ALL SPRINGS, AND ALL MUST EXIST.153

B E N J A M I N D I S R A E L I


Outside the legislatures, ministers find manyopportunities to explain their actions: in publicspeeches, meetings with constituents or interestedparties, interviews with the media, and so on.Although not perfect vehicles, these occasions are asignificant part of a minister’s accountabilityreporting regime.

Outside the senior governments, nearly allpublic sector bodies provide some opportunity forthe governing body to question senior manage-ment and for managers to give an account of theirperformance. The degree of formality may vary,and the scrutiny given to such reporting may rangefrom the intense to the casual. Where meetings arepublic and the governing body is publicly elected,as with municipal councils, the questioning ofmanagement may sometimes assume a political, ifnot partisan, cast. In other circumstances, as wherethe board is the captive of the CEO, compliantacquiescence may be the order of the day. Giventhe number and variety of organizations, there isbound to be a wide range of practice.

W R I T T E N R E P O R T S

Governments produce a plethora of informa-tion in a variety of forms. Many of these havenothing to do with reporting on performance oraccountability: gazettes with their legal informa-tion, guides to obtaining government services orcompleting government forms, calendars forgarbage collection, volumes of statistical and otherinformation, and booklets promoting such diversematters as health, industry, cookery, tourism, andrecreational activities or events are only a fewexamples. Some reports, however, have a directbearing on our subject.

S E N I O R L E V E L S O F G O V E R N M E N T

Until recently, it was the practice of ministersin the federal government to issue annual reports

of their departments’ activities. The provinces con-tinue to do this. These reports seldom reflect astandardized format and tend to concentrate onactivities undertaken in the relevant fiscal period.In most instances, the reports give no indication ofthe reliability or completeness of the informationthey contain. Nevertheless, they provide informa-tion on government activity that might otherwisenot be readily available. Typically, these reports aremade public after tabling in the House, where theyare available for members to use in debates, ques-tions, and committee discussions. Late filing ofthese reports—in some cases, up to two years afterthe end of the fiscal period—limits their usefulnessas accountability documents.

Reporting on financial matters is of directconcern here. While it is not our intention to pre-sent a treatise on financial administration in gov-ernment, an overview of the various documentsinvolved is germane.

At the two senior levels of government, thefinancial cycle starts with the presentation in theHouse of the annual budget, a document givenwide public attention. This document outlines thegovernment’s spending intentions and indicateshow it will be raising money. Emphasis is usuallygiven to changes from existing levels of outlay andincome. Debate on this document provides legisla-tors with an opportunity to discuss any subject ofinterest to them.

The concrete details of the government’sspending intentions are presented to the House inthe printed document Estimates. Although practicevaries, these Estimates typically provide little in theway of performance information, in terms of eitherwhat has gone before or what will result from theintended expenditures. In an attempt to overcomethis deficiency, the federal government introducedwhat are called Part III Estimates in the early 1980s. Part IIIs, published as one document per depart-ment or agency and written in a way intended to

P A R T I I . C H A P T E R 8 . E X I S T I N G R E P O R T I N G P R A C T I C E S 1 0 5

be easy to read, display forward-looking informa-tion on spending plans, even beyond the year ofappropriation, as well as information on past per-formance. Importantly, they display actual expen-ditures against previous plans. They also explainthe mission of the department or agency, its near-term orientation, structure, and past performance.In effect, Part IIIs have replaced annual reports andbrought the added advantage of presenting infor-mation in a manner consistent with what was actu-ally approved by Parliament. While they havebecome the most important documents for report-ing to Parliament by a department or agency, PartIIIs are still incomplete accountability documents.While they contain some evidence that demon-strates the efficiency of the department, they donot, for example, take into account related taxexpenditures (that bypass the system of spendingproposals and appropriations and are hidden butimportant holes in the public purse). Nor do theyprovide sufficient information upon which aninformed assessment of overall effectiveness can bemade on entire programs at a level that is meaning-ful to parliamentarians.

After the Estimates are tabled, legislation isintroduced and passed giving the government thelegal authority to spend and raise money as it hasindicated it wants to. Expenditures are authorizedby an Appropriation Act; and separate bills, such asthose amending taxation legislation, deal with vari-ous means for raising revenue. Some expenditureprograms, such as federal payments under theCanada Assistance Plan, do not require annual leg-islative approval, as they are governed by statutes ofcontinuing force.

The Public Accounts, published some monthsafter the end of each fiscal year, display the govern-ment’s financial position and detail the ways moneyhas been received and spent. The form in whichthese documents are cast makes it exceedingly diffi-cult for nonexperts to gain a clear insight into events.

Of much more interest to legislators and thepublic are the reports of the legislative auditors—the Auditor General of Canada and his provincialcounterparts. These reports contain the results ofthe auditors’ examination of the Public Accountsand of other such audits they have conducted dur-ing the year. With the comprehensive audit (value-for-money) mandates the vast majority of theseauditors now have, the reports contain assessmentsof various aspects of government programs, activi-ties, and functions, and recommendations for waysin which administration could be improved. Theirlegislation, however, does not allow most auditorsto examine the effectiveness of government pro-grams, constraining them not to comment on poli-cy. Despite this limitation, these reports, combinedwith the Part III Estimates and their provincialequivalents, offer the most detailed, authoritativereports on governmental performance available atpresent.

The comprehensive auditing mandates aresignificant, as they usually mean that governmentsare to be audited by the legislative auditor withrespect to:

• financial management and control;• reporting to Parliament on performance;• attesting to the financial statements and the

authority of transactions;• management controls;• economy, efficiency, and effectiveness; and• adequacy of electronic data processing (EDP)

and other information systems.

And the auditors’ reports, of course, contain find-ings on government performance in these areas.

Both the Public Accounts and the auditor’sreport are typically referred to the Public AccountsCommittee of the legislature for examination.Usually chaired by a member of the opposition,these committees tend to concentrate on the auditreport, as it is more understandable than the Public


Accounts and comments on issues of interest to leg-islators. Committee reports are tabled in the legis-lature and usually contain recommendations foraction by the government. These reports are sel-dom debated in the House.

In addition to these annual publications, overthe years there have been numerous royal commis-sions, parliamentary, ministerial, interdepartmen-tal, internal audit, program evaluation, and otherad hoc reviews to assess many aspects of govern-ment policy and administration. Some of thesereports contain a wealth of information and usefulcomment on the aspects of government operationstheir authors were charged to examine. Many ofthese reports, though by no means all, have had adirect effect on what governments do and howthey go about doing it.

One government initiative, the enactment in1984 of Bill C-24, an Act to Amend the FinancialAdministration Act in relation to CrownCorporations, demonstrated the government’sresolve to take concrete steps in the matter of thecontrol, direction, and accountability of federalCrown corporations.154 The legislation clarifiedthe responsibilities of these corporations in mattersof management, information, and oversight bytheir boards of directors. The act introduced anovel approach to audit for Crown corporations,including some specific attention to the notion ofeffectiveness.

More precisely, the legislation requires thatauditors provide positive assurance that the corpo-ration’s systems and practices “were maintained in amanner that provided reasonable assurance that…the operations of the corporation and each sub-sidiary were carried out effectively.”155 It alsorequires management to report on the performanceof the Crown corporation with respect to theattainment of objectives in the year in question. Iteven specifies the value system to inspire the man-agement of the corporation: the safeguarding and

control of the assets, economy, efficiency, andeffectiveness.

O T H E R P U B L I C S E C T O R O R G A N I Z A T I O N S

It is difficult to generalize about performancereports emanating from the public sector outsidethe two senior levels of government. Typically,there are budgets and audited financial statements.The amount of detail they contain and the distrib-ution given to them vary greatly.

In recent years, there has been a significantincrease in the amount of information that somefunders require from the nonprofit service agenciesto which they give money. Often, agency budgetsmust be accompanied by or incorporated in whatare sometimes called service plans before the fund-ing government will decide on the amounts to begiven. These service plans often include informa-tion on what the organization has accomplished inthe past and what it expects to do in the future.While these plans provide much more informationthan simple financial budgets supply, they tend toconcentrate on inputs—hours of care, number ofstudents taught, and so on—rather than on theoutcome achieved by the goods or services pro-duced. It is impossible to say how many, if any, ofthese plans are distributed outside the organiza-tions themselves and the government departmentson which they rely for funding. It is significantthat nongovernmental funding agencies such as theUnited Way show a similar tendency to ask formore performance information from the organiza-tions they support.

Certain areas call for mandatory performancereports. An example is the quality assurance reportsrequired from various hospital departments. Suchreports are regularly considered by the governingboard and forwarded, in some jurisdictions, to theprovincial health ministry. The following letterindicates how seriously these reports are taken.


There are numerous examples of nonprofitorganizations providing information on theiraccomplishments. In many instances, these areanecdotal accounts of selected programs offered inconjunction with fund-raising campaigns. Fewnongovernmental organizations have attempted toassess and distribute information on the overalleffectiveness of their performance.

Information on performance tends to beselective, partial. A major study was conducted of

not-for-profit institutions in the United States156

(hospitals, colleges and universities, human servicesorganizations, state and local government units,trade and professional associations, philanthropicfoundations) many years ago but is likely still validtoday. Its findings are probably valid for Canada,too. The study reported the following:

• evidence of service effort and accomplish-ments reporting was found in every kind oforganization;


LETTER FROM THE CHAIRMAN OF A HOSPITAL’S QUALITY ASSURANCE COMMITTEE

TO HEADS OF DEPARTMENT, 1993

QUALITY ASSURANCE REPORTS HAVE IMPROVED SUBSTANTIALLY OVER THE LAST TWO OR THREE YEARS. WHILE THE QUALITY

ASSURANCE COMMITTEE INCLUDES HEALTH CARE PROFESSIONALS, IT ALSO HAS VERY ATTENTIVE MEMBERS OF THE BOARD OF

TRUSTEES. AUTHORS ARE GAINING EXPERIENCE IN WRITING CLEARER REPORTS, WHICH IS AT LEAST PARTLY THE RESULT OF THE

IMPLEMENTATION OF GUIDELINES AND THEIR CONTINUOUS IMPROVEMENT. AS WELL, THE TRUSTEES ARE GETTING BETTER AT

READING THE REPORTS, THROUGH THEIR GROWING FAMILIARITY WITH THE SUBSTANCE OF THE HOSPITAL, THEIR CAREFUL ANALY-

SIS OF THE REPORTS, AND THEIR RAISING OF INCREASINGLY INCISIVE QUESTIONS.

THE WORDS CHOSEN BY THE AUTHORS OF THE REPORTS ARE IMPORTANT TO THE TRUSTEES IN HELPING THEM TO UNDERSTAND

THE EVIDENCE OFFERED FOR ASSURING QUALITY. ON OCCASION, HOWEVER, HEALTH CARE SITUATIONS HAVE BEEN DESCRIBED

USING QUALIFIERS WITH STRONG EMOTIONAL OVERTONES. WE CAN UNDERSTAND THE FRUSTRATION OF SOME AUTHORS IN

DESCRIBING SOME SITUATIONS, OR THE APPREHENSION OF OTHERS FROM FAILURE TO TAKE CORRECTIVE ACTION OR IN THE FAIL-

URE TO OBTAIN THE RESOURCES NECESSARY TO SOLVE THE PROBLEM. IF A SITUATION IS SERIOUS, THEN RATHER THAN USING EMO-

TIONAL QUALIFIERS, IT DESERVES TO BE ADDRESSED IN A MANNER THAT EXPLAINS WHY THE SITUATION IS NOT SATISFACTORY,

WHAT CRITERIA OR STANDARDS ARE NOT MET, WHAT HAS INFLUENCED THE JUDGEMENT, ETC. IF THE EMOTIONAL IS USED AS AN

OPPORTUNITY TO CHAMPION A CAUSE WHEN THE INTERNAL PRIORITIES SYSTEM DID NOT FAVOUR THIS PARTICULAR SITUATION, IT

SHOULD NOT BE DONE IN A MANNER AFFECTING OBJECTIVITY OR PREVENTING THE COMMITTEE FROM UNDERSTANDING THE REAL

IMPLICATIONS.

THE QA COMMITTEE WORKS BETTER WHEN THE INFORMATION PROVIDED TO IT:

1. ALLOWS THE COMMITTEE TO PROVIDE OVERSIGHT ON MATTERS IN WHICH IT HAS GOVERNANCE RESPONSIBILITY OR INTEREST;

2. SERVES TO IDENTIFY AND EXPLAIN KEY ISSUES, CHOICES, CHALLENGES WITH WHICH MANAGEMENT AND THE BOARD MUST CONTEND;

3. IS CONVEYED IN SUCH A WAY THAT SUGGESTS THE TYPE OF NECESSARY CORRECTIVE MEASURES TO BE TAKEN; AND

4. ASSISTS AN INFORMED DIALOGUE BETWEEN THE DEPARTMENTS, MANAGEMENT AND THE BOARD ON MATTERS THAT ARE CONSID-

ERED OF CORPORATE SIGNIFICANCE.

WE ARE TAKING THIS OCCASION TO THANK YOU FOR YOUR EXCELLENT COOPERATION IN THIS IMPORTANT ASPECT OF HOSPITAL

GOVERNANCE.

• information was found more readily onefforts than accomplishments, efficiency thaneffectiveness;

• reporting of outputs was common and muchof this information was quantified, whilethere was less information reported on results,and in some cases it was totally absent.


P E R F O R M A N C E R E P O R T I N G

Private sector reporting requirements rangefrom providing information to satisfy the demandsof public regulators and professional financial ana-lysts, to information intended for a more generalaudience. They must satisfy the needs of manyinterested parties, including government agencies,self-regulating bodies such as stock exchanges (wheresecurities are issued publicly), shareholders, credi-tors, potential investors, financial commentators,and any other current or potential stakeholders.

Provincial securities legislation governs theinformation that companies must make public. Toprotect the investing public, the regulators requirecorporations that issue securities to the public toprovide full, timely, and continuous disclosure ofall significant information. These requirements,together with strict rules governing insider trading,are intended to ensure that all investors have thesame information on which to base decisions—alevel playing-field.

Increasing attention has been paid over thelast decade to the potential of providing moreinformation than is found in the financial state-ments. The report of the Commission to Study thePublic’s Expectations of Audits (MacdonaldCommission), made the following recommenda-tion in its report issued in June 1988:

The CICA [Canadian Institute ofChartered Accountants] should look favor-ably on additional financial disclosure of a

softer, more subjective nature in aManagement Discussion and Analysis sectionof the annual report. The CICA should assistand cooperate with securities commissions inthe development of standards for informationin the MD&A.157

In 1991, the CICA published a major researchstudy on the subject of reporting entitledInformation to be Included in the Annual Report toShareholders, in recognition of the need for moreinformation than is offered in the financial state-ments.

Many readers will be familiar with thesetrends. The instruments are referred to as AIF(Annual Information Form) and MD&A(Management Discussion and Analysis), narrativereporting tools supplementing financial statements.The AIF offers background information on thecompany, its products and services, the environ-ment in which it operates, as well as its exposure torisk and uncertainties. In the MD&A, manage-ment explains in narrative form the company’s cur-rent financial situation, performance, and prospectsfor the future. It places emphasis on explainingvariations in the results portrayed in the financialstatements, particularly in the last two years, aswell as known trends, events, uncertainties, andsignificant risks having an impact on the compa-ny’s operations.158

Regulatory bodies such as the OntarioSecurities Commission and the Commission desvaleurs mobilières du Québec have turned theirattention to both AIF and MD&A in recent yearsowing to certain shortcomings, and have issueddisclosure requirements regarding them. The regu-lators found that:

Part of the problem was that narrativeinformation, imprecise by nature, was subjectto less scrutiny. Although financial statementsmust conform to generally accepted account-


ing principles (GAAP), there were no compa-rable standards for other narrative disclosures.This left scope for manipulation, allowingcompanies to present information in a biasedmanner or omit material information.Moreover, professionalism and judgement,which are crucial for reporting narrativeinformation, were often lacking. The resultwas promotional and unreliable disclosure.159

The new disclosure requirements are intended toovercome these weaknesses. And because substan-tial advantages accrue to companies which obtainregulatory approval of their reports, the informa-tion now generally meets these heightenedrequirements. A company that issues securities tothe public would typically have to prepare the fol-lowing reports:

• annual audited financial statements and quar-terly unaudited statements;

• annual report to shareholders, usually includ-ing MD&A, historical financial and opera-tional information, relevant statistics,prospects for the future, and so on;

• Annual Information Form;• notice of annual general meeting, with

accompanying proxy solicitation and infor-mation circular;

• immediate press release regarding any eventpotentially affecting the company’s operationssignificantly, with subsequent report to regu-lator;

• report to the government agency (federal orprovincial) under whose jurisdiction it isincorporated;

• reports to taxing authorities; and• reports to other regulating (such as environ-

mental) and statistics-gathering agencies.

In addition, many companies issue press releasesof information for which the regulators do not

require disclosure. And if the company intends toissue securities to the public, it must prepare, haveapproved, and issue a prospectus containing a widerange of required information.

The foregoing clearly demonstrates that a greatdeal of information is currently being generated anddisseminated to various audiences. Within organiza-tions, information is generated from as many per-spectives as there are levels and diversity of opera-tions. People monitoring performance use indicatorsthat may be significant to one manager but quitemeaningless to others or to senior management.160

As a result, adding together all such reports will notnecessarily satisfy the needs of the senior managersor governors of an organization. After all, the infor-mation needs of a member of a board of directorsare substantially different from those of a plant man-ager; on a functional basis, the perspective on per-formance of the chief accountant will be differentfrom that of the marketing manager.

Despite all the attention given to reportingon performance, the information that is madeavailable is very often limited in perspective andfails to provide a comprehensive view of how wellan organization is doing in all significant respects.

S O M E R E C E N T I N I T I A T I V E S

I N R E P O R T I N G O N

P E R F O R M A N C E

Recent years have seen a number of initiativesby governments and others (public institutions,private sector, professional bodies) to improve thequality of performance reporting within organiza-tions, and from them to those who govern and areserved by them. The reporting frameworks relatingto these initiatives differ somewhat in their scopeor character. Nevertheless, they share a commonemphasis on the importance of bringing forwardinformation about the results (ranging from out-puts, to client satisfaction, to achievement of


broader goals and objectives) that are beingachieved, each commenting that all too often thepreoccupation has been with inputs and process.For the most part, these initiatives also share acommon general purpose: to provide informationthat has greater strategic value for decision makingand that will enhance greater transparency andaccountability.

These initiatives have focused on developingand reporting against key performance indicators,usually within the context of a multi-year strategythat recognizes that finding the right indicators,the means to report against them, and how best topresent and use the resulting information are mat-ters that take time and repeated effort to sort out.

A brief overview of some of these initiativesfollows.

I N I T I AT I V E S I N T H E U N I T E D K I N G D O M

The national Citizen’s Charter is a ten-yearprogram launched in 1991. It is linked to a seriesof earlier initiatives (Financial ManagementInitiative, Next Steps) stretching back over morethan ten years and all aimed at the broad objectiveof administrative reform. The charter was put for-ward as a comprehensive program to improve qual-ity, increase choice, secure better value for money,and enhance accountability. It is based on the fol-lowing six principles:

• Standards—publishing standards and progressagainst commitments enables customers toput pressure on those responsible for publicservices to deliver a high-quality service;

• Information and openness—to make best useof their public services, people need informa-tion about how they are run, what they costand how well they perform;

• Choice and consultation—the charter aims togive members of the public a greater say inthe way their public services are run;

• Courtesy and helpfulness—public servants areexpected to be courteous and helpful to thepublic they serve;

• Putting things right—the charter stresses thatif sometimes things go wrong, public servicesshould offer an apology, a full explanation,and a swift and effective remedy;

• Value for money—providing high-qualitypublic services, focusing on the essentials andensuring that services are delivered in themost efficient and effective way.161

The national Citizen’s Charter applies to allpublic services, and each key public service (health,education, employment, social security, and so on)has its own charter—now forty in all. There arealso many thousands of local charters coveringlocal service providers such as police forces and fireservices. Consultation allows the public to help setstandards by determining the priorities and targetsthat are important to them. The national Citizen’sCharter itself has no statutory basis, but is a matterof agreed government policy.

At the local level, however, the governmenthas enacted legislation (1992) to require all localauthorities to publish details of the performancethey achieve against a set of indicators. In Englandand Wales, this work is spearheaded by the AuditCommission (by the Accounts Commission inScotland), an independent body responsible forauditing local authorities. The Audit Commissionhas developed a wide range of indicators againstwhich individual local authorities must measurethemselves and then publish the results. In addi-tion to providing a basis for reporting to the pub-lic, “[T]he indicators were chosen to give informa-tion which will also be useful to council staff andelected councillors… They can use the Citizen’sCharter indicators to help build their own systemsfor monitoring performance.”163


The indicators developed by the AuditCommission fall into four broad, and to someextent interrelated, groups:

• indicators of performance, including efficien-cy and effectiveness

• indicators of cost• indicators of quality• indicators providing background information,

for example, regarding how much a service isused or about population make-up.164

I N I T I AT I V E S I N T H E U N I T E D S TAT E S

Several initiatives are in progress at the feder-al, state, and local government levels to broadenand enhance performance reporting as a basis forimproved decision making and accountability. Thisinitiatives include those mandated by specific legis-lation.

By way of illustration… at the federal level:

T H E U N I T E D S TAT E S G OV E R N M E N T P E R F O R M A N C E

A N D R E S U LT S AC T

As noted in Part I, the GovernmentPerformance and Results Act (GPRA) was passedin 1993. One of its key features is a requirementfor all federal agencies to submit a program perfor-mance report to the president and Congress. Thesereports will discuss performance achieved againstpreviously stated goals, and describe actions thatthe agency intends to take to address areas wherethese goals are unmet.

The essential purpose of the GPRA is tofocus accountability reporting within and by feder-al government agencies on the results beingachieved by programs, instead of on inputs andprocess. This results-oriented information isexpected to lead to improved decision making(including resource-allocation decisions), betterperformance, and more effective congressionaloversight. Currently, there are several pilot projectsoperating across the government. Beginning in1999, agencies will be expected to prepare annualplans setting performance goals, and then to pro-vide performance reporting against these plans thefollowing year.

…and, at the state and local government levels…


CHARTER-RELATED INITIATIVES

TWO RELATED INITIATIVES ARE THE COMPETING FOR QUALITY

PROGRAM AND THE DEPARTMENTAL EFFICIENCY PLANS SYSTEM.

UNDER THE COMPETING FOR QUALITY PROGRAM, DEPART-

MENTS AND AGENCIES ARE REVIEWING THEIR ACTIVITIES. THE

1994 REPORT ON THE CITIZEN’S CHARTER INDICATES THAT

THE KEY IN THIS REGARD IS TO LOOK AT THE JOB TO BE DONE

AND THEN TO WORK OUT HOW BEST TO DO IT, APPLYING A

NUMBER OF TESTS, WITH THE AIM OF INTRODUCING CHOICE

AND COMPETITION WHEREVER POSSIBLE. THIS ENCOURAGES

PUBLIC SERVICES TO RESPOND TO THEIR CUSTOMERS’ NEEDS,

TO REWARD INNOVATION AND TO IMPROVE THE EFFECTIVE-

NESS AND JOB SATISFACTION OF STAFF.

KEY QUESTIONS ARE:

• DOES THE JOB NEED TO BE DONE AT ALL?…

• IF THE ACTIVITY MUST BE CARRIED OUT, DOES THE

GOVERNMENT HAVE TO BE RESPONSIBLE FOR IT?…

• WHERE THE GOVERNMENT NEEDS TO REMAIN RESPON-

SIBLE FOR AN ACTIVITY, DOES THE GOVERNMENT HAVE

TO CARRY OUT THE TASK ITSELF?…

• WHERE THE JOB MUST BE CARRIED OUT WITHIN

GOVERNMENT, IS THE ORGANIZATION PROPERLY STRUC-

TURED AND FOCUSSED ON THE JOB TO BE DONE?162

A FURTHER RELATED INITIATIVE IS THE SYSTEM OF

DEPARTMENTAL EFFICIENCY PLANS THAT ARE INTENDED TO

PROVIDE DEPARTMENTS AND AGENCIES WITH A BASIS TO HELP

THEM DECIDE HOW PUBLIC SERVICES ARE TO BE DELIVERED IN

THE MOST EFFICIENT WAY.

G OV E R N M E N TA L AC C O U N T I N G S TA N D A R D S B O A R D —

R E P O R T I N G O N S E R V I C E E F F O R T S A N D

A C C O M P L I S H M E N T S

The Governmental Accounting StandardsBoard (GASB) establishes standards for financialaccounting and reporting for state and local gov-ernmental entities. These standards guide thepreparation of external financial reports of thesebodies. The GASB also develops concepts to pro-vide a frame of reference for resolving accountingand reporting issues.

Regarding these concepts, the GASB has con-ducted extensive research on reporting on ServiceEfforts and Accomplishments (SEA). “[Such]information is necessary for assessing accountabili-ty and in making informed decisions; therefore, tobe more complete GPEFR [general purpose exter-nal financial reporting] for governmental entitiesneeds to include SEA information.”165

In 1990, the GASB released a major researchreport summarizing the efforts of two dozenresearchers looking at twelve significant programareas of state and local governments. The reportrecommended a number of indicators that mightserve as a useful starting point for expanding andimproving external reporting.166 Subsequently,these ideas were incorporated in the 1994 GASBConcepts Statement related to Service Efforts andAccomplishments Reporting. The ConceptsStatement itself does not establish SEA reportingrequirements for state and local governments.Instead, it provides a framework of ideas aroundwhich further consideration and experimentationcan be done in relation to such external reporting.

The SEA reporting framework has threebroad categories of measures:

• measures of service efforts (input indicators,both financial and nonfinancial);

• measures of service accomplishments (outputand outcome indicators, focusing, respective-

ly, on what has been produced and on whathas been achieved); and

• measures that relate efforts to accomplishments(efficiency and cost-outcome indicators).

It is anticipated that SEA reporting may alsoinclude explanatory information to “help usersunderstand reported SEA measures, assess the enti-ty’s performance, and evaluate the significance ofunderlying factors that may have affected thereported performance.”167

The Concepts Statement discusses some ofthe limitations of SEA information, recognizingthat these measures are but one component of theinformation used to assess accountability and makedecisions. It also outlines ways in which some ofthese limitations may be mitigated.168

R E P O R T I N G I N I T I AT I V E S AT T H E S TAT E A N D LO C A L

L E V E L S —O R E G O N , A N I L LU S T R AT I O N

A number of states are taking initiatives toestablish and expand performance reporting to thepublic. One such initiative is the OregonBenchmarks program, launched in 1989. At thattime, the legislature created the Oregon ProgressBoard, a panel of statewide leaders charged withmonitoring the achievement of the state’s strategicgoals.

The framework within which such reportinghas occurred consists of an extensive series ofbenchmarks, or quantitative measures, that areused to track the extent to which Oregon’s strategicgoals are being met. The emphasis of these bench-marks is on results.

Altogether, in the 1995 report to the legisla-ture (the third such biannual report), there are over250 such benchmarks, of which approximatelytwenty have been designated as core measures.169

The city of Portland, Oregon, has been veryactive in the area of service-efforts and accomplish-


ments reporting. A public report is producedannually, which contains a wide range of perfor-mance information on the city’s largest andmost visible programs. The information in thereport is independently checked by city auditorstaff.170

A number of other states (for example,Minnesota, Texas, Iowa, Arizona) are also experi-menting with what some have termed “results-ori-ented governance.” Cities such as Phoenix(Arizona), Sunnyvale (California), and Milwaukee(Wisconsin) are examples of other local govern-ments acknowledged to be in the upper tier ofthose doing work in the area of performance mea-surement and accountability.

I N I T I AT I V E S I N W E S T E R N A U S T R A L I A A N D S O U T H

A F R I C A

Initiatives in Western Australia and SouthAfrica are linked to the enactment of specific legis-lation—in the case of Western Australia, theFinancial Administration and Audit Act (1985);and, in the case of South Africa, the Reporting byPublic Entities Act (1992), which has recently beenproclaimed.

In Western Australia, government organiza-tions are required to prepare an annual report that,among other matters, “contains performance indi-cators as measurements of efficiency and effective-ness.”171

In South Africa, the legislation requires theboard of directors to submit to the responsibleminister an annual report that, among other mat-ters, contains relevant performance informationregarding the economic, efficient, and effectiveapplication of resources.

A key issue for many organizations in bothgovernments concerned how best to organize andpresent the required performance indicators andinformation in a way that both those sending and

receiving them will judge to be fair and completeand germane to their needs and interests.

I N I T I AT I V E S I N C A N A D A

Part I of this book includes a discussion ofinitiatives undertaken by the federal government toenhance performance reporting and accountabilityto Parliament. In addition to these, there are sever-al other related initiatives concerning departments,agencies, and Crown corporations and variouslyinvolving members of governing bodies, manage-ment, and auditors. Recently, initiatives are alsospringing from other sources—both governmentaland nongovernmental. For example:

T H E P R OV I N C E O F A L B E R TA

In May 1995, the provincial legislature ofAlberta passed the Government Accountability Act.The act requires the government and individual min-istries to prepare and publish annual reports that,among other matters, discuss actual performanceresults against desired results previously outlined intheir respective business plans. Later that same year,the government released its first annual report using,as a starting point, twenty-two core indicators as abasis for overall government reporting.172

T H E S O C I E T Y O F M A N A G E M E N T AC C O U N TA N T S O F

C A N A D A

In 1994, The Society of ManagementAccountants of Canada (SMAC) issued a guidelineintroducing a framework for performance indica-tors that seeks to bring together historical andfuture-oriented information, information related toexternal relationships as well as internal activities,and information that is financial and nonfinancialin nature.

Arguing for the need for a balanced perspec-tive that is capable of serving top management’s


needs and interests, as well as other levels of man-agement and operations in an organization, theguideline puts forward six categories of perfor-mance indicators:

• environmental indicators, relating to suchmatters as use of recyclable materials, acci-dents and injuries resulting from productsand services, etc.

• market/customer indicators, relating to suchmatters as market share, customer satisfac-tion, etc.

• competitor indicators, relating to much thesame dimensions of performance as are exam-ined for one’s own business

• internal business processes indicators, relatingto such matters as product development andmanufacturing cycle times, sales production,etc.

• human resource indicators, relating to suchmatters as employee development, morale,etc.

• financial indicators, relating to such mattersas revenue growth, profitability, etc.173

Developed for application to businesses that pro-duce and sell a product or service, the SMACguideline states that a performance-indicatorapproach such as this is also useful for not-for-profit organizations, including governments.

The need for comprehensiveness, rigour, andflexibility are given strong emphasis in the SMACguideline. Also emphasized is the notion that “per-formance indicators are just that—signals of per-formance… They can give clues but not theanswers directly.” Indeed, to one extent or another,such considerations are mirrored in the discussionof all the above initiatives.


153 BENJAMIN DISRAELI, VIVIAN GREY, BK. VI, CH. 7.154 CROWN CORPORATIONS RECEIVED A LOT OF ATTENTION IN THE EARLY EIGHTIES. IN 1980, THE AUDITOR GENERAL OF CANADA ISSUED THE REPORT: COMPREHENSIVE AUDITING IN CROWN

CORPORATIONS: A BRIEFING PAPER PREPARED FOR THE STANDING COMMITTEE ON PUBLIC ACCOUNTS BY THE AUDITOR GENERAL OF CANADA, JULY 10, 1980. LATER, THE CANADIAN

COMPREHENSIVE AUDITING FOUNDATION RELEASED FOUR PUBLICATIONS SUCCESSIVELY: AUDITING IN FEDERAL CROWN CORPORATIONS: A DIRECTOR’S INTRODUCTION TO THE AUDIT AND

SPECIAL EXAMINATION PROVISIONS OF THE FINANCIAL ADMINISTRATION ACT (AS AMENDED BY BILL C-24), 1984, 42 PP.; PREPARING FOR SPECIAL EXAMINATIONS OF FEDERAL CROWN

CORPORATIONS: A STATUS REPORT—SEPTEMBER 1985, 30 PP.; SPECIAL EXAMINATIONS IN FEDERAL CROWN CORPORATIONS: A NEW DIMENSION TO RELIANCE ON INTERNAL AUDITING IN

CANADA, 1986, 60 PP.; SPECIAL EXAMINATIONS, INTERNAL AUDITS AND RELIANCE: A PRACTITIONER’S GUIDE, 1987, 221 PP.155 ACT TO AMEND THE FINANCIAL ADMINISTRATION ACT IN RELATION TO CROWN CORPORATIONS, S. 143(1).156 PAUL K. BRACE, ROBERT ELKIN, DANIEL ROBINSON, AND HAROLD STEINBERG, REPORTING OF SERVICE EFFORTS AND ACCOMPLISHMENTS (STANFORD: FINANCIAL ACCOUNTING STANDARDS

BOARD, 1980).157 REPORT OF THE COMMISSION TO STUDY THE PUBLIC’S EXPECTATIONS OF AUDITS (TORONTO: CANADIAN INSTITUTE OF CHARTERED ACCOUNTANTS, JUNE 1988), RECOMMENDATION R-17,

57.158 SEE BRENDA EPRILE AND DOROTHY SANFORD, “BEYOND THE NUMBERS: IMPROVING NARRATIVE FINANCIAL REPORTING IN CANADA,” CMA MAGAZINE, MARCH 1991, 7-14. 159 IBID., 7.160 FOR INSTANCE, A RELEVANT INDICATOR FOR THE MANAGER OF A TYPING POOL MAY BE THE NUMBER OF WORDS TYPED IN A DAY, WEEK, OR MONTH. THIS INFORMATION CAN BE USED IN

THE DEPLOYMENT OF RESOURCES AND IS USUALLY EXPRESSED IN THOUSANDS OR MILLIONS OF WORDS. SUCH MEASURES ARE USUALLY MEANINGLESS TO SENIOR MANAGERS WHO ARE THE

CLIENTS OF THE SERVICE. THE CLIENTS WANT TO KNOW WHAT THE EXPECTED TURNAROUND TIME IS FOR A LETTER, A TEN-PAGE REPORT—ONE HOUR, TWO, A DAY, TWO DAYS?161 THE CITIZEN’S CHARTER—THE FACTS AND FIGURES, A REPORT TO MARK FOUR YEARS OF THE CHARTER PROGRAMME (LONDON: HMSO, 1995), CM. 2970.162 THE CITIZEN’S CHARTER—SECOND REPORT: 1994 (LONDON: HMSO, 1994), CM. 2540, 2.163 AUDIT COMMISSION, WATCHING THEIR FIGURES—A GUIDE TO THE CITIZEN’S CHARTER INDICATORS (LONDON: HMSO, 1994), 10.164 IBID, 13-14.165 GOVERNMENTAL ACCOUNTING STANDARDS BOARD, CONCEPTS STATEMENT NO. 2 OF THE GOVERNMENTAL ACCOUNTING STANDARDS BOARD ON CONCEPTS RELATED TO SERVICE EFFORTS AND

ACCOMPLISHMENTS REPORTING (NORWALK, CT: GOVERNMENTAL ACCOUNTING STANDARDS BOARD OF THE FINANCIAL ACCOUNTING FOUNDATION, 1994), NO. 109-A, I.166 GOVERNMENTAL ACCOUNTING STANDARDS BOARD, SERVICE EFFORTS AND ACCOMPLISHMENTS REPORTING: ITS TIME HAS COME, (NORWALK, CT: GOVERNMENTAL ACCOUNTING STANDARDS

BOARD, 1990). SEVERAL RESEARCH REPORTS, FOCUSING ON KEY PROGRAM AREAS, HAVE SINCE BEEN PUBLISHED.167 GOVERNMENTAL ACCOUNTING STANDARDS BOARD, CONCEPTS STATEMENT NO. 2, 20-24.168 IBID, 28-31.169 OREGON PROGRESS BOARD, REPORT TO THE 1995 LEGISLATURE: OREGON BENCHMARKS—STANDARDS FOR MEASURING STATEWIDE PROGRESS AND INSTITUTIONAL PERFORMANCE (SALEM,

OREGON: OREGON PROGRESS BOARD, 1994).170 SEE OFFICE OF THE CITY AUDITOR, CITY OF PORTLAND SERVICE EFFORTS AND ACCOMPLISHMENTS: 1993-94—FOURTH ANNUAL REPORT ON CITY GOVERNMENT PERFORMANCE (PORTLAND,

OR.: OFFICE OF THE CITY AUDITOR, JANUARY 1995).171 MARGARET STOCKTON, EFFECTIVENESS REPORTING: WA PILOT STUDY—A PRACTICAL APPROACH, (PERTH, WESTERN AUSTRALIA: PUBLIC SECTOR MANAGEMENT OFFICE, 1994), 8.172 GOVERNMENT OF ALBERTA, MEASURING UP—FIRST ANNUAL REPORT BY THE GOVERNMENT OF ALBERTA, JUNE 1995.173 THE SOCIETY OF MANAGEMENT ACCOUNTANTS OF CANADA, “DEVELOPING COMPREHENSIVE PERFORMANCE INDICATORS,” MANAGEMENT ACCOUNTING GUIDELINE 31, IN MANAGEMENT

ACCOUNTING PRACTICES HANDBOOK (HAMILTON: THE SOCIETY OF MANAGEMENT ACCOUNTANTS OF CANADA, 1994), 15-19.

C H A P T E R 9

CONCEPTUALISSUES INPERFORMANCEREPORTING

T H E C E N T R A L I S S U E S

The definition of accountability adopted inthis book makes the rendering of account—report-ing—its central requirement. Meaningful reportingabout effectiveness or performance, however, is eas-ier said than done. The Wilson Report, fromwhich the definition of accountability is taken, rec-ognizes the difficulties:

In simple situations, full accountabilityimplies a detailed disclosure of the way inwhich the responsibility has been carried out.If, however, the responsibility is a complexone, perhaps involving many expenditures fordifferent purposes or the performance ofmany duties, full and detailed disclosure mayactually defeat the purpose of accountability.The party who granted the responsibility maybe overwhelmed with details and may beunable, without extensive and laboriousanalysis, to form any judgement as towhether the obligations accepted by the sec-ond party have been properly met.

Thus, the basic problem of disclosure incomplex situations is to find a manner ofselecting and presenting information that willconvey, in summary form, a fair, objectiveand accurate picture of whether a givenresponsibility is being satisfactorily dis-charged. To avoid misunderstanding or misin-

terpretation of the information provided,clear standards or conventions relating to theappropriate form of disclosure should beestablished and agreed upon by both parties.If this can be done, accountability can bemaintained even in the most complex situa-tions.174

Simply put, there are two main issues: whatto report, and how to report it. This chapterexplores some conceptual aspects of these issues;the following chapter offers practical advice abouthow to go about reporting on performance. First, afew definitional matters.

P E R F O R M A N C E A N D

E F F E C T I V E N E S S

As noted earlier, the words performance andeffectiveness both have to do with the notion ofaccomplishments as related to efforts, or capabili-ties, or intent. In the management and publicadministration literature, the word performance isoften circumscribed to indicate the scope of thematters to which it is being related—administrativeperformance, managerial performance, programperformance, and so on. Similarly, the word effec-tiveness is often modified—operational effective-ness, program effectiveness, and organizationaleffectiveness. Thus, in the context in which theyare used, performance and effectiveness have mean-ings that they share. Effectiveness is the term usedmore frequently from here on.

A more careful examination of the concept ofeffectiveness follows shortly.

V A L U E F O R M O N E Y

Values such as prudence, due diligence, regu-larity (compliance to rules), probity, integrity, andequity have inspired the conduct of generations ofpublic servants. They are often values that individ-


uals have carried into the workplace from their pri-vate lives. Getting value for the money disbursedis a responsible objective and an eventual andtangible source of satisfaction. The term valuefor money, as a desirable characteristic of theconduct of public business, has only recentlycome into vogue.

Value for money is an outcome of the move-ment towards a public service that is more respon-sive to public needs and is more accountable. Thatmovement gained momentum with the stress onresponsibility and responsiveness in the RoyalCommission on Government Organization(Glassco Commission) report in 1962. It was givenshape by the Auditor General of Canada Act in1977, and it was reinforced by the RoyalCommission on Financial Management &Accountability (Lambert Commission) report in1979, which placed emphasis on linking account-ability to financial management.

In contemporary usage, value for money sum-marizes three separate but interrelated values:economy, efficiency, and effectiveness, oftenreferred to as the 3 Es. Taken together, these val-ues include the traditional values of prudence, duediligence, regularity, probity, integrity, and equity.Some observers like to add a fourth and a fifthE—equity and ethics—as being peculiar to gov-ernment and thus deserving of specific mention.Commonly, however, both equity and ethics aregenerally understood to be part of the third E,effectiveness.

The third E, effectiveness, is often thought tocontain the essence of the other two—economyand efficiency—even though much of theCanadian writing on the subject of measuring per-formance deals with each of the three Es separately,such distinctions having been largely inspired bythe legislated mandate of audit offices. The reasonfor this is that each of the three values has some-

P A R T I I . C H A P T E R 9 . C O N C E P T U A L I S S U E S I N P E R F O R M A N C E R E P O R T I N G 1 1 7

THE EXPRESSION VALUE FOR MONEY

WHILE THE EXPRESSION VALUE FOR MONEY MAY HAVE BEEN

HEARD BEFORE IN CANADA, THE REPORT OF THE INDEPENDENT

REVIEW COMMITTEE ON THE OFFICE OF THE AUDITOR

GENERAL OF CANADA (WILSON REPORT) NOTED THAT IT WAS

USED IN 1961. IT TELLS ABOUT WATSON SELLAR, AUDITOR GEN-

ERAL OF CANADA IN THE 1950S, INSTRUCTING HIS STAFF TO BE

ALERT TO EXPENDITURES ASSOCIATED WITH THE CANCELLATION

OF AN AGREEMENT BEFORE THE PURPOSE WAS FULLY IMPLE-

MENTED. MR. SELLAR FIRST CALLED THEM “NUGATORY PAY-

MENTS.” IN 1958, HE BEGAN REPORTING SUCH ITEMS AS “NON-

PRODUCTIVE PAYMENTS.” HIS SUCCESSOR, MAXWELL

HENDERSON, IN DISCUSSIONS AT A MEETING OF THE STANDING

COMMITTEE ON THE PUBLIC ACCOUNTS, WAS CONCERNED

ABOUT THE DEFINITIONAL PROBLEM ASSOCIATED WITH SUCH

PAYMENTS. HE RAISED THE POSSIBILITY THAT A DECISION TO

CANCEL A PROJECT AFTER SOME PRELIMINARY EXPENDITURES

HAD BEEN INCURRED MIGHT BE WISE, AFTER ALL. ARE PRELIMI-

NARY EXPENSES OF THIS KIND REALLY NONPRODUCTIVE, AND

SHOULD THEY BE REPORTED AS SUCH? THE COMMITTEE

AGREED ON THE NEED FOR THE AUDITOR GENERAL TO USE HIS

JUDGMENT, AND MR. HENDERSON, DISCUSSING NONPRODUC-

TIVE PAYMENTS, STATED: “IT IS NOT AN EASY PROBLEM… IT

BOILS DOWN TO A QUESTION OF VALUE FOR MONEY.”175

THE WILSON REPORT ALSO UNDERLINED THE USE OF THE

EXPRESSION VALUE FOR MONEY IN ITS DISCUSSION ON EXTEND-

ING THE SCOPE OF AUDIT: “[T]HE CONCEPT APPEARS TO HAVE

DEVELOPED IN BRITAIN THAT THE GOVERNMENT OBTAINS

APPROPRIATIONS FROM PARLIAMENT IN TRUST AND THAT THE

TRUST EXTENDS BEYOND MERE TECHNICAL COMPLIANCE WITH

THE WORDING OF APPROPRIATIONS TO THE PRUDENT EXPENDI-

TURE OF FUNDS. MORE RECENTLY, THE COMPTROLLER AND

AUDITOR GENERAL HAS BEEN CONCERNED WITH ESTABLISHING

WHETHER VALUE FOR MONEY HAS BEEN OBTAINED THROUGH

THE SPENDING PROCESS. THIS CONCEPT INCLUDES, BUT GOES

BEYOND, THAT OF MERE WASTE. THUS, OVER THE YEARS, THE

COMPTROLLER AND AUDITOR GENERAL HAS QUESTIONED AND

DRAWN TO THE ATTENTION OF PARLIAMENT TRANSACTIONS

THAT REFLECT ON MANY ASPECTS OF THE ADMINISTRATION OF

PROGRAMS, AND HIS COMMENTS GO WELL BEYOND A CONSID-

ERATION OF MERE TECHNICAL REGULARITY.”176

thing at its core that distinguishes it from the othertwo. The following definitions have gained generalacceptance, even though they may not be quitedefinitive. In very simple terms, people concernedwith assessing value for money ask the followingquestions respecting the three Es:

Economy: Are we getting the right inputs atthe best cost (getting a good deal)?

Efficiency: Are we getting the most outputfrom our inputs (getting a lot for theefforts)?

Effectiveness: Are the outputs getting the resultswe want (doing the right things)?177

Economy is essentially a resource-acquisitionconcept with a least-cost notion and is concernedwith the acquisition of resources of appropriatequality and quantity at the lowest reasonable cost(buying resources at the right time, at a favourableprice, in the right quantity and quality).

Efficiency is essentially a resource-usage con-cept, also with a least-cost notion, that is con-cerned with the maximization of outputs at mini-mal cost or the use of minimum-input resources(as evidenced by high productivity, on-time perfor-mance, and so on).

Effectiveness has been defined as an ends-ori-ented concept that measures the degree to whichpredetermined goals and objectives for a particularactivity or program are achieved (the attainment ofthe right results from usage of resources and orga-nizational operations).178

THE 3 ES: THEIR ROLE IN DECISION MAKING AND

AS MEASURES OF PERFORMANCE

THE FOLLOWING EXCERPT IDENTIFIES THE THREE ELEMENTS

OF VALUE FOR MONEY NOT ONLY AS AN ACCOUNTABILITY

FRAMEWORK BUT ALSO AS A WAY TO GATHER INFORMATION

FOR DECISION-MAKING PURPOSES:

“TO FACILITATE THE PROCESS OF DECISION MAKING IN THE

CONTEXT OF THE PUBLIC ADMINISTRATION SYSTEM AND BUD-

GETARY CYCLE, IDEALLY A GOVERNMENTAL ENTITY SHOULD

ESTABLISH AND COMMUNICATE CLEAR, RELEVANT GOALS AND

OBJECTIVES; SET MEASURABLE TARGETS FOR ACCOMPLISHMENT;

AND DEVELOP AND REPORT INDICATORS THAT MEASURE ITS

PROGRESS IN ACHIEVING THOSE GOALS AND OBJECTIVES (MEA-

SURES OF PERFORMANCE). FOR EXAMPLE, FOR GOVERNMENTAL

ENTITIES TO HAVE APPROPRIATE INFORMATION FOR MAKING

DECISIONS AND ASSESSING ACCOUNTABILITY, INFORMATION

NEEDS TO BE PROVIDED ABOUT RESULTS ACHIEVED (SERVICE

ACCOMPLISHMENTS OR PERFORMANCE) THROUGH THE USE OF

THE RESOURCES PROVIDED (SERVICE EFFORTS) AND HOW

THOSE RESULTS COMPARE WITH WHAT WAS PLANNED. THE

TERMS ECONOMY, EFFICIENCY, AND EFFECTIVENESS OFTEN ARE

USED IN THIS CONTEXT TO DESCRIBE THE CATEGORIES OF PER-

FORMANCE INFORMATION NEEDED.

THIS NEED FOR COMPREHENSIVE PERFORMANCE INFORMA-

TION IS HIGHLIGHTED BY A GUIDELINE ON ECONOMY, EFFI-

CIENCY AND EFFECTIVENESS SUPPORTING A BRITISH

CHARTERED INSTITUTE OF PUBLIC FINANCE AND

ACCOUNTANCY (CIPFA) STANDARD IN 1982. THE GUIDELINE

STATES: “ECONOMY AND EFFICIENCY IN THE EXECUTION OF

PROGRAMMES IS OF SMALL CONSEQUENCE IF THOSE PRO-

GRAMMES ARE NOT MEETING THE AUTHORITY’S OBJECTIVES,

AND NO ASSESSMENT OF VALUE FOR MONEY IS COMPLETE

WITHOUT REGARD TO EFFECTIVENESS. IN ORDER TO ASSESS

EFFECTIVENESS, IT IS NECESSARY FIRST, TO DETERMINE AND

SPECIFY THE OBJECTIVES AND SECOND, TO ASSESS (MEASURE)

PERFORMANCE AGAINST THOSE OBJECTIVES SO THAT APPRO-

PRIATE ADJUSTMENT OR REMEDIAL ACTION CAN BE TAKEN.”179


More elaborate definitions of the 3 Es havebeen written. There is no universally agreed-uponwording of these definitions, nor need there be.The formulations in the literature all convey thesame basic meaning. The differences in wordingare influenced largely by circumstances or particu-lar legislative or institutional requirements, whichin some cases affect the boundaries between thethree values.180 Many of these formulations areparaphrases of one another, albeit with a usefulcontextual twist. The following detailed definitionsare found in the Comprehensive Auditing Manual ofthe Office of the Auditor General of Canada:

Economy. This refers to the terms andconditions under which an organizationacquires financial, human, physical, andinformation resources. Economy means get-ting the right amount of the right resource, atthe right level of quality, at the right time, inthe right place, at the right cost. Lack ofeconomy in acquiring resources could resultin a higher than necessary cost of products orservices, or products or services of inappropri-ate quality, quantity, or timeliness.

Efficiency. This refers to the relationshipbetween the quantity and quality of the goodsand services produced (output) and the costof resources used to produce them at arequired service level to achieve programresults. An efficient operation either producesthe maximum quantity of output of a givenquality for any given resource inputs, or usesminimum inputs to produce a given quantityand quality of output.

Effectiveness. Program effectiveness is theextent to which program objectives or intend-ed consequences are achieved. Where unin-tended negative effects occur, effectivenessmust be judged on the balance of positiveand negative consequences.

The above gives an impression of neatness inthe abstract separation of three seemingly differentvalues, but the reality is not so clear. In practice, itcan be very difficult to deal with one value whileignoring the other two. Sometimes, what appearsas effectiveness at one operating level may well beviewed as efficiency at the next level. Moreover, theeffectiveness of public programs is often thought tobe at the core of value for money. Effectiveness, orthe attainment of program goals, outranks issues ofeconomy and efficiency because although theremay be some merit in doing the right things poor-ly, there is no redeeming value in doing the wrongthings well.181

ONE SENIOR ACADEMIC WONDERED IF:

… THE BUREAUCRATIC VALUES OF ECONOMY, EFFICIENCY,

AND EFFECTIVENESS [COULD] BE MELDED WITH THE

POLITICAL VALUES OF REPRESENTATION AND RESPONSI-

BILITY, YET STILL RETAIN THEIR SALUTARY INFLUENCE ON

THE WORK-A-DAY WORLD OF BUREAUCRACY.182

There is no doubt that for measurement pur-poses, distinguishing each of the 3 Es is useful. Theproblem is not at that level. It is when oneattempts to form an overall opinion or report onthe performance of a program that it is not possi-ble, nor useful, for that matter, to proceed as if theconcepts were independent of one another.

In its March 1988 report, the StandingCommittee on National Finance subscribed to theview that effectiveness should be broadly definedto reach beyond the attainment of program goalsand should focus on performance in general.Moreover, the committee recognized that effective-ness cannot be measured independently of costsand productivity. In other words, the committee


thought that the intellectual separation of the 3 Esmay be convenient for some purposes, but whenthe time comes to assess and report on perfor-mance on a value-for-money basis, there is a riskthat the demarcation lines between the three con-cepts will produce gaps and inadequacies.

The committee demonstrated a strong prefer-ence for a seamless gradation of values. It felt thateconomy and efficiency were essential to a mean-ingful understanding of effectiveness, and thatthere was no point in separating resource justifica-tion and program-operational performance fromeffectiveness or the outcomes, the results. Broad-based effectiveness, as the committee called it,could not be limited to the extent to which a pro-gram meets its objectives. Moreover, it could notbe measured and reported on independently ofcosts and productivity.

T H E E F F E C T I V E N E S S

D I L E M M A

THE SHEER INTELLECTUAL DIFFICULTY OF GRAPPLING WITH

EFFECTIVENESS—OF WRAPPING OUR MINDS AROUND SO

AMORPHOUS, SO PROTEAN A CONCEPT—HAS BEEN AMONG

THE GREATEST STUMBLING BLOCKS TO A WIDER ACCEPTANCE

OF EXPANDED ACCOUNTABILITY FOR MANAGEMENT…183

The assessment of effectiveness is by far themost important contributor to an accountabilityregime and at the same time the most elusive. Onereason may be that there is no single, high-level,generally agreed-upon definition of effectiveness.

In daily conversation, being effective describesthe capacity or the ability to achieve results. Often,it goes beyond this capacity, and being effectivemeans having reached one’s goals, to be successful.The word may also be used to emphasize theimpact of one’s efforts or actions. Frequently, it

merely describes that a capability is actually in use,or that a law or a rule is in force. There are otheruses of the word.

In practice, when discussing organizations,the word effectiveness is typically modified by oneof three terms: program, operational, and organiza-tional.

• Program effectiveness relates to the continuingrelevance of a program, the attainment of itsintended objectives, its impact, and its cost-effectiveness;

• Operational effectiveness relates to theachievement of output targets, to the deliverysystems for the goods and services produced,and to the cost-effectiveness of these systems;

• Organizational effectiveness relates to theoverall capability of the organization and theinteractions among strategic planning, man-agement structures and processes, and humanand financial resources—all in relation to themission and goals of the organization and theexternal environment.

Some writers have refined the notion of effec-tiveness by making explicit the consideration ofcost-effectiveness, which is the attainment of theobjectives at the least cost, at a lesser cost, or at leastat a reasonable cost in relation to the value of theoutcome. Some have also included both intendedand unintended results or outcomes of a program asfactors in the measurement of effectiveness. Othersstill, particularly program evaluators, have includedthe assessment of the continued need for the pro-gram in their conception of effectiveness. They askthe question: Are the reasons that gave rise to theprogram in the first place still valid today?

How effectiveness is viewed will depend onwho is looking for it. Different views may be takenby the electorate, specific constituencies, cus-tomers, governing bodies, managers, and so on.Each view has validity in its own context.


EFFECTIVENESS: NOT A SIMPLE CONCEPT

… EFFECTIVENESS IS NOT A SINGLE, INDIVISIBLE CON-

CEPT. IT CONTAINS SUBJECTIVE, VALUE-LADEN COMPONENTS

THAT WILL CHANGE WITH TIME AND WITH VIEWPOINT.

FORMING JUDGEMENTS ABOUT EFFECTIVENESS INVOLVES

WEIGHING MULTIPLE, COMPETING AND SOMETIMES CONTRA-

DICTORY OBJECTIVES AND MEASURES… DIVERSE THOUGH

PERCEPTIONS OF EFFECTIVENESS HAVE BEEN TO DATE, AND

DIFFICULT THOUGH THE QUESTIONS IT RAISES ARE, THE STARK

FACT IS THAT THERE ARE PRESSURES TO DEAL WITH THE ISSUE.

THE GREAT CHALLENGE, OF COURSE, IS TO FIND AN APPROACH

THAT WILL BUILD ON THE STRENGTHS BUT AVOID THE WEAK-

NESSES OF PAST MEASUREMENT, REPORTING AND AUDIT INITIA-

TIVES AND CREATE A REGIME THAT WILL SERVE THE INTERESTS

OF ALL THE PLAYERS IN THE ACCOUNTABILITY STRUCTURE.184

The literature on the subject provides a varietyof approaches to defining effectiveness. Untilrecently, however, there has been no single approachthat accommodates the interests and needs of allparties and suits the requirements of a wide varietyof organizations. There is, nevertheless, consensusthat a high-level interpretation of effectiveness willusually involve the most important elements of thenotions of economy and efficiency.

E F F E C T I V E N E S S A N D T H E

A C H I E V E M E N T O F G O A L S

Of all the meanings attached to the wordeffectiveness, probably the most common is relatedto the achievement of goals. Different authors,with different value systems, have their own con-ceptions of effectiveness, but what brings themnear one another is goal accomplishment or perfor-mance in meeting objectives. This is how legislativeauditors have generally viewed effectiveness fortheir own work.

Although making goal accomplishment cen-tral to effectiveness gives it a clear focus, the prob-lems and implications remain very substantial. Theliterature on effectiveness contains many studiesthat raise issues surrounding this particular per-spective. Among the observations185 on this aspectare the following:

• goals are often ill-defined, complex, changing,and contradictory;

• it is often unclear at what level or withrespect to what units the attainment of goalsshould be measured;

• more than one technology or strategy pro-duces the same outcome;

• goal-based perspectives usually take into con-sideration the preferences of managers, not allconstituencies; and

• goal-based definitions have failed to clarifydistinctions between organizational effective-ness, managerial effectiveness, and managerand subordinate behaviour and attitudes.

In addition, there is the semantic problemregarding the apparent interchangeability of termssuch as outputs, goals, objectives, mission, pur-poses, results, ends, and aims. There is also thedifficulty of distinguishing achievements on acontinuum of purposes, from results of immedi-ate efforts all the way to the ultimate outcomes orconsequence of a policy, using terms that conveyexactly what was achieved against expectations atvarious stages:

A reading of the literature on organiza-tional goals reveals that in study after studythere is now general agreement that the offi-cial, formally stated goals of the organisationare of little value for judging efficiency andeffectiveness. Each of these and many otherstudies demonstrate that official or primarygoals are too vague and too general to be


operational. For example, the primary objec-tives of many public bureaucracies are foundin legislation, but are so general that precisetargets are impossible to specify. Thus theobjective of housing policy is to provide“good housing for the whole populace”,whatever that might mean, or education poli-cy is aimed at ensuring that every childreceives an “adequate education.” Precise defi-nitions of what constitutes “good” or “ade-quate” are never set out and become the focusof debate and, in many instances, controversy.In the absence of adequately specified officialgoals, operative goals are used instead. The lat-ter are defined as goals which designate theends sought through the actual operatingpolicies of the organisation; they tell us whatthe organisation is actually trying to do,regardless of what the official goals specify asthe aims.186

Time is also an important consideration. Agovernment policy or program may achieve its pur-pose or have practical effects for everybody con-cerned (every beneficiary), but not at all the sametime. The impact may be immediate or deferred,ephemeral or permanent. It may still be viewed aseffective:

A goal is some future state of the worldwhich the members of the organisation tryto bring about by the application of a set ofmeans. A time dependent means-endsequence implies, therefore, the existence of aseries of intermediate states which may ormay not be emphasised. To the extent thatthey are, they become themselves goals andmight be called intermediate goals, with refer-ence being made to some global or end statewhich is the primary goal of the system…Whilst the bureaucracy might be successfully

achieving one of its intermediate goals and beregarded as being efficient, it might neverthe-less be far from achieving its primary goal.187

Moreover, as mentioned above, the success ofthe policy or program will be viewed more or lessaccording to expectations that are bound to be dif-ferent from one set of stakeholders to another: thebeneficiaries of the program, the suppliers of ser-vices, the electorate, the opposition parties, thecentral agencies, and so on. Official, publicly statedgoals may not be accepted as satisfactory by peoplewho have some interest in the program.

In its 1975 report, the Wilson Committeeidentified problems with considering effectivenessin terms of goals:

Present problems in measuring the effec-tiveness of spending programs revolve aroundtwo major questions. First, how does onedefine in quantifiable terms what Parliamentexpects to achieve in enacting new legislationor by approving a new program through vot-ing funds in an appropriation act? Many pro-grams defy quantification. For example, howcan one translate the purposes of theDepartment of External Affairs into quantita-tive terms that permit objective measurementof performance? While other programs aresusceptible to some form of measurement,they may be intended to achieve a number ofdifferent objectives, some measurable (forexample, the reduction of unemployment orimprovement in the standard of living) andsome intangible (such as, overcoming a senseof injustice). Even if some of these identifica-tion problems can be resolved, there remainsa second major question: how can the resultsachieved be measured?188


O N E S T U D Y H A D T H I S T O S AY A B O U T M O D E R N

O R G A N I Z AT I O N S , PA R T I C U L A R LY I N T H E P U B L I C

S E C T O R :

[T]HEY ARE OFTEN “ORGANIZED ANARCHY” EXHIBITING SUCH

CHARACTERISTICS AS ILL-DEFINED, COMPLEX, CHANGING AND

CONTRADICTORY GOALS; UNCLEAR MEANS/END, INPUT/OUT-

PUT AND CAUSAL CONNECTIONS; AND WIDELY DIFFERING CRI-

TERIA OF SUCCESS OPERATING SIMULTANEOUSLY IN VARIOUS

PARTS OF THE ORGANIZATION.189

The contradictions and paradoxes of govern-ment also contribute to the difficulty of assessingeffectiveness. On the one hand, stated objectives arenot always those being actually pursued, and some-times the policy objectives of a particular program ordepartment are not symmetrical with that of anotherdepartment. In fact, objectives may be competing.On the other hand, government-wide objectives maybe sacrificed, ignored, or forgotten in favour of moreimmediate departmental objectives that may not bein accord with more global purposes.

WHEN YOU HAVE A GOVERNMENT DEPARTMENT THAT IS MORE

LIKE THE CONVENTIONAL PRIVATE SECTOR MODEL OF A PRO-

DUCTION PROCESS, YOU ARE MORE LIKELY TO FIND THE KINDS

OF INDICATORS AND PERFORMANCE MEASURES THAT FACILI-

TATE A MORE PRECISE DEFINITION [OF EFFECTIVENESS].

HOWEVER AS YOU MOVE IN THE AREA OF COMPETING OBJEC-

TIVES AND SHAPING AND INFLUENCING… WHICH ARE LEGITI-

MATE ROLES OF GOVERNMENT… IT BECOMES SIGNIFICANTLY

MORE DIFFICULT.190

Public policy is very complex. It emanatesfrom politics and involves difficult choices andoften produces dilemmas; it develops broad nation-al initiatives for society according to a vision of a

better future, but it gets involved in local conflicts;negotiations are at once at the center of issues andat the margin. Public policy is frequently fashionedto reinforce power. The resulting translation of pub-lic policy into programs or activities will somehowreflect all of these pursuits, making statements ofobjectives an art form of a rather difficult nature.

REPORT OF THE AUDITOR GENERAL

OF CANADA, 1993

THE STUDY… MAKES IT CLEAR THAT, UNDERLYING

THESE THREE CONSTRAINTS [POLITICAL PRIORITIES,

MANAGEMENT BURDEN OF ADMINISTRATIVE AND PRO-

CEDURAL TASKS, AND DISINCENTIVES AND LACK OF

INCENTIVES FOR PRODUCTIVE MANAGEMENT] AND CON-

TRIBUTING TO THEM IS THE INESCAPABLE COMPLEXITY

OF GOVERNMENT; A COMPLEXITY WHICH NOT ONLY HAS

TO DO WITH BIGNESS BUT WITH THOSE DIVERSE OBJEC-

TIVES WHICH ARE INEVITABLE IN THE PUBLIC SEC-

TOR.… IT MUST BE DEFINITIVELY UNDERSTOOD THAT

THIS OFFICE IS NOT ADVOCATING THAT ALL POLITICAL

AND SOCIAL PRIORITIES SHOULD BE SACRIFICED ON

SOME ALTAR OF A NARROWLY-DEFINED PRODUCTIVITY.

OUR STUDY MAKES IT CLEAR THAT IT IS THE NATURE OF

GOVERNMENT TO FULLFIL A SPECTRUM OF OBJECTIVES.

VALUE FOR MONEY IS ONE SUCH OBJECTIVE AND ONE

WHICH THE MANDATE OF MY OFFICE REQUIRES ME TO

EXAMINE. BUT I RECOGNIZE THAT VALUE FOR MONEY,

IN GOVERNMENTAL TERMS, IS NOT MEASURABLE IN

TERMS OF ANY SIMPLE DEFINITION OF PRODUCTIVITY

ALONE. VALUE IS ALSO ACHIEVED IN SOCIAL, CULTURAL,

AND BROAD POLITICAL IMPERATIVES WHICH BY A NAR-

ROWER MEASURE MIGHT BE ACCOUNTED AS DETRACT-

ING FROM PRODUCTIVITY. I FULLY RECOGNIZE, AS DOES

OUR STUDY AND ALL OUR AUDITING, THAT GOVERN-

MENT BY ITS VERY NATURE SERVES A MULTIPLICITY OF

PURPOSES. IT IS WITHIN THE CONTEXT OF ACHIEVING

ALL THOSE PURPOSES THAT THE PRODUCTIVITY OF THE

PUBLIC SERVICE MUST BE MEASURED.


On a different plane, the actual impact orresults of a government program or activity may beindependent of original intentions. The latter mayor may not have been realized. When they are not,it may be because original expectations have beenmodified, unforeseen problems have been met, orthe necessity of the public intervention has changed.Much has to do with the dynamic, sometimesvolatile, nature of the issue.191

S O M E A P P R O A C H E S T O


In response to the public exhortation for pub-lic officials to be more forthcoming about theresults they have achieved, considerable thought hasbeen given to making the notion of effectiveness amanageable concept for the purpose of assessingand reporting on it. One approach, among others,is to classify effectiveness into such categories asthose mentioned earlier: program effectiveness,managerial effectiveness, operational effectiveness,organizational effectiveness. On examination, thesecategories tend to bleed into one another.

By the mid-eighties, the literature of publicadministration had not found a concise definitionfor the word effectiveness that was evocative of theentire range of meanings attributed to it, eventhough the notion of goal attainment was generallyaccepted as central to the notion. On the contrary,social scientists tended to look at effectiveness aslargely situational, of little use without an explana-tion of the circumstances surrounding it. The liter-ature had not yet produced a convenient and com-prehensive framework that would lead to an overalljudgment on the success or effectiveness of a pro-gram or public policy. But it had succeeded inidentifying a hierarchy of so-called effectivenessconstructs that summarized a variety of notions,attributes, and models. One grouping provides thefollowing perspectives on effectiveness:

• constituency/political perspectives: recogniz-ing that many organizations and individualshave competing interests;

• goal-achievement perspectives: recognizingmultiple stated or unstated goals;

• structure/system perspectives: recognizingthat effectiveness is a function of internalstructures, systems, and attitudes;

• resource acquisition/survival perspectives: rec-ognizing that such attributes as resourceattraction, environmental adaptation, andlongevity may be ultimate indicators of effec-tiveness.192

These four perspectives constitute a fairly commonset of distinctions breaking down the concept ofeffectiveness into useful components.

Much of the literature on effectiveness origi-nates in the United States and is really focused onorganizational effectiveness. This emphasis reflectsthe influence of the for-profit sector and a capacityview of effectiveness. This particular view, whichhas substantial empirical support, is based on thebelief that the effectiveness of the organizationlargely results from the values, behaviour, and skillsof the work force. An organization that scores highon these counts is deemed to be an effective orga-nization and is presumed to be doing the rightthing. Thus, the capacity view of effectivenessimplies that if all the relevant and appropriate con-ditions are satisfied with respect to the input, goodoutput will follow naturally.

Beyond this, however, by the mid-eighties theliterature had not produced a model of effective-ness capable of general application. It had certainlynot produced a technical, mathematically basedmodel similar to those used in such fields as eco-nomics and science. The reason, of course, is thatmany of the multitude of variables involved inassessing effectiveness are not susceptible to accu-rate measurement, and the relationships among


them are not clearly enough understood to beassigned precise values. These problems remainunresolved, although some theorists are stillwrestling with them.

Rather than attempt models, some authorshave found it more worthwhile to develop frame-works for assessing effectiveness. These frameworksidentify the attributes that are displayed whensomething is effective, rather than posit a generaltheory of effectiveness itself.

MEANING OF THE TERM—FRAMEWORK

A FRAMEWORK IS SOMETIMES VIEWED PEDANTICALLY BY

MODEL BUILDERS AS A LESSER FORM OF A MODEL. ALTHOUGH

A FRAMEWORK IS A REDUCTION, A SIMPLIFICATION OF THE

REAL WORLD, AS IS A MODEL, IT HAS THE ADVANTAGE OF A

FLEXIBILITY THAT A MODEL MAY NOT OFFER, AND IT IS MORE

SUITED IN AN AREA NOT EASILY QUANTIFIABLE. IT CAN BE

EXPANDED TO SUIT THE CIRCUMSTANCES. IT CAN BE USED

WITH CIRCUMSPECTION. IT CANNOT BE DEFINITIVE. WHILE

CAUSAL RELATIONSHIPS CANNOT BE ENTIRELY IGNORED, THEY

DO NOT HAVE TO BE IDENTIFIED OR FORMALIZED IN A FRAME-

WORK. A FRAMEWORK DOES NOT REQUIRE MECHANICAL

FUNCTIONING IN THE SENSE THAT A MALFUNCTION WOULD

MAKE IT INOPERABLE. FINALLY, A FRAMEWORK IS LESS TEMPT-

ING FOR ITS ARCHITECT AND MANIPULATORS TO REGARD IT AS

AN END IN ITSELF INSTEAD OF AS A TOOL.

L I S T S O F A T T R I B U T E S O F


The literature has noted that determinants ofeffectiveness have not been well defined, and, forthat matter, indicators of effectiveness have notbeen clearly distinguished from determinants. Itfinds that the challenge is more about knowingwhat to measure than how to measure effectiveness.

Creating a list of attributes that, whenapplied against an entity, would define the qualityof performance is most difficult.193 Compared toidentifying objects in nature that can be sorted outby physical attributes readily perceived through thesenses (color, odor, shape, size, weight, consistency,and so on), looking for evidence of effectivenessis a mental exercise at a much higher level ofabstraction.

Attributes have to match a general expecta-tion arising out of the particular vision of a suc-cessful organization and be capable of identifyingsigns of performance that indicate how well theorganization is doing. The vision is not alwaysclear, the expectations not always well articulated,and the signs are often ambiguous, if not mislead-ing. Frequently, the signs themselves are merelyrepresentations (numerical signs, for instance),often only approximations of the reality. Moreover,expectations and understanding change over time.For example, for decades and decades, the primemeasure of the health and success of an enterprisewas a static one—the balance sheet. Then, in the1930s, attention shifted to a more dynamic render-ing of account—the income statement, whichcould be used to appraise corporate earning powerand future prospects.

Finally, even in the absence of changingexpectations, the greatest challenge is the identifi-cation, from a bewildering number of possibilities,of characteristics that are sufficiently representativeand dynamic to indicate the quality of perfor-mance and that, together, can be used to make anoverall assessment of effectiveness.

There are, in the literature of management, inparticular, several examples of lists of attributesintended to cover sufficient indicators of behaviourto gauge an organization’s effectiveness.


O R G A N I Z AT I O N A L E F F E C T I V E N E S S —

J . P. C A M P B E L L

J. P. Campbell’s list of attributes of organiza-tional effectiveness is frequently cited in the litera-ture.194 The list contains thirty effectiveness criteriaand was intended to account for “all variables thathave been proposed seriously as indices of organi-zational effectiveness.”195 Subsequent authors haveobserved that Campbell’s list is not exhaustive, isnot relevant for all types of organization, and con-sists of a mix of causes and effects.

Campbell’s list of attributes of effectiveness:• overall effectiveness• productivity• role and norm congruence• efficiency• profit • managerial tasks skills• quality• accidents• evaluations by external entities• growth• absenteeism• participation and shared• influences• turnover• job satisfaction • achievement emphasis• motivation • morale • value of human resources• control• conflict/cohesion• training and development• flexibility/adaptation • planning and goal setting • managerial interpersonal skills• goal consensus • internalization of goals

• readiness• utilization of environment• stability• information management and

communication

T H E Q U I N N A N D R O H R B A U G H R E D U C T I O N O F

C A M P B E L L’ S L I S T

Robert Quinn and John Rohrbaugh reducedCampbell’s thirty criteria to seventeen,196 presum-ably in the belief that some criteria were subsumedby others, as their list looks similar to the one theyhad reduced. These two sets of criteria to assesseffectiveness nevertheless constitute useful frame-works of attributes:

• morale • evaluation by external entities• value of human resources• planning and goal setting• training and development emphasis• profits• quality• productivity• conflict/cohesion • efficiency• flexibility/adaptation • stability• readiness• control• growth• information management and

communication• utilization of environment

M A N AG E M E N T E F F E C T I V E N E S S —P E T E R S A N D

WAT E R M A N

Another branch of the literature on effective-ness has to do with attributes of managementeffectiveness. It looks so much like organizationaleffectiveness that, to many, management effective-


ness is a stylistic variation of organizational effec-tiveness. While it is largely centered on the workforce (organizational effectiveness), it concentrateson the behaviour and style of the senior cadre ofmanagement, hence the appellation. To use oneexample, in the early 1980s, to develop the thesisof their best-seller In Search of Excellence197 ThomasPeters and Robert Waterman started by applyingwhat was known at the time as the McKinsey 7-Sframework, which they had developed earlier andwhich identified the following:

• strategy;• structure;• systems;• style;• staff;• skills; and• shared values.

These were all interconnected graphically in a webresembling the structure of an atom.

Peters and Waterman were not entirely happywith the 7-S structure, as it could not distinguish agood organization that has a certain element ofinertia from an innovative one with a great capaci-ty to adapt to rapid changes in the world and inthe marketplace. They needed some distinctivemarks to tell them which among all the good com-panies they were looking at were excellent. Theydecided that, generally speaking, only innovativeorganizations deserved to be called excellent. Moreprecisely, continuous innovation was defined as themark of excellent companies.198 Their study ofdozens of highly reputed companies led them toconclude that a set of eight behavioural attributesor tendencies was present in what they regard asexcellent innovative companies:

• a bias for action;• close to the customer;• autonomy and entrepreneurship;• productivity through people;

• hands-on, value driven;• stick to the knitting;• simple form, lean staff; and• simultaneous loose-tight properties.

As a sequel to In Search of Excellence, ThomasPeters and Nancy Austin produced A Passion forExcellence,199 which reinforced the concentrationon people and management presented in the firstbook. Passion was defined as comprising the fol-lowing elements:

• care for customers;• constant innovation;• leadership (MBWA–management by walking

about); and• people.

T H E A U D I T O R G E N E R A L’ S L I S T O F

C H A R AC T E R I S T I C S O F W E L L - P E R F O R M I N G

O R G A N I Z AT I O N S

In his 1988 report, the Auditor General ofCanada devoted a chapter to what he called “well-performing organizations.”200 In an approach withcertain similarities to that of Peters-Waterman, hehad invited senior civil servants to identify a num-ber of federal government organizations that bygeneral consensus were performing very well. Hestudied eight organizations so identified, trying toisolate the characteristics that made the organiza-tion perform so well. The following characteristicswere common to the well-performing organizationsincluded in the study:

Emphasis on people• the empowering organization• the caring organization• the successful organization

Participative leadership• the becoming organization• the communicating organization


Innovative work-styles• the learning organization• the problem-solving organization• the intelligent organization

Strong client orientation• supported by owners• close to clients• the concretizing organization

The auditor general’s study was in a senseconsiderably more difficult than that of Peters andWaterman because of the absence in the federalgovernment of overall effectiveness indicators suchas those available in the private sector. Because ofthis, much of the assessment of well-performingorganizations was based on criteria largely drawnfrom economy and efficiency values, or organiza-tional and managerial effectiveness.

T H E B A L D R I D G E AWA R D

The Baldridge Award was established by theMalcom Baldridge National Quality ImprovementAct, signed by President Ronald Reagan in 1987. Itwas named after the late secretary of commerce.The act called for the creation of a national awardto promote quality of goods and services producedin the United States and the development of guide-lines and criteria that organizations could use toevaluate their quality-improvement efforts. It wasleft to the National Bureau of Standards (nowcalled the National Institute of Standards andTechnology) to produce a seven-category, 1,000-point scoring system. Those performance indica-tors constitute a sort of effectiveness framework.

Companies submit applications of up to sev-enty-five pages (up to fifty pages, in the case ofsmall business) with descriptions of their quality-improvement practices and performance in each ofthe following seven areas:

• leadership;

• information and analysis;• strategic quality planning;• human resource utilization;• quality assurance of products and services;• quality results; and• customer satisfaction.

These criteria in turn are broken down into manysubcategories. Recently, there were thirty-two suchsubcategories. The Baldridge test is administeredand graded by teams of trained external examiners.

As a framework for effectiveness, theBaldridge test is not comprehensive, most probablydeliberately so, as it places the emphasis on totalquality management and similar thrusts, not onthe entire performance or effectiveness. Where thebottom line is to serve as the indicator of overallperformance, it is fair to say that profitability maynot necessarily originate in superior quality man-agement. In a study of its own in 1990, theGeneral Accounting Office (GAO) in the UnitedStates concluded, however, “that there was a cause-and-effect relationship between the total qualitymanagement practices embodied in the Baldridgecriteria, and corporate performance measured byemployee relations, productivity, customer satisfac-tion, or profitability.”201 Although some critics saythat the GAO study has limitations, it neverthelessdraws highly plausible conclusions. Intercompanycomparisons are the essence of the Baldridge testbecause it is used to determine the winners in anational award program. Thus, much of themethodology and the rigour of application is pecu-liar to this project.

C A N A D A AWA R D S O F E X C E L L E N C E

Similar in focus and approach to the BaldridgeAward with its emphasis on total quality, Canada’sequivalent—the Excellence Award for Quality—isadministered by the National Quality Institute.


One of three separate awards under the rubricCanada Awards for Excellence, the Quality Awardis divided into seven categories that span the busi-ness, government, health care, and education sec-tors. The Canada Awards for Excellence program isbuilt on its predecessor, Canada Awards forBusiness Excellence, which began in 1984 and hadbeen administered by Industry Canada until man-agement of the program was passed over to theNational Quality Institute with the latter’s estab-lishment in 1992-93.

The National Quality Institute has developeda set of total-quality criteria that organizations canuse as a basis for self-assessment and that serve as abasis for adjudicating the Excellence Award forQuality. These quality criteria are integrated undera general “framework for effective organizations”that focuses on six broad areas, further brokendown into twenty-one subareas. The six areas are:

• leadership;• customer focus;• planning for improvement;• people focus;• process optimization; and• supplier focus.

All the above are seen as linked factors that worktogether to drive key outcomes related to customer,employee, investor, and supplier satisfaction.202

From the foregoing, it can be seen that by themid-eighties, despite serious attention to the sub-ject, there was no practical and widely adaptableregime for assessing the effectiveness of the perfor-mance of public sector organizations. Substantialprogress had been made in providing more andbetter information in several regards, but no solu-tions had been found to some of the conceptualproblems involved in this complex subject. Thetools were simply not available to permit compre-hensive assessment of, and reporting on, how wellorganizations were doing.

S E N S I T I V I T Y O F T H E N O T I O N

O F E F F E C T I V E N E S S

Assessing effectiveness is an issue of extremesensitivity in public sector organizations at boththe political and managerial levels.

For some, it is a practical matter. They thinkthat the scrutiny of policies, programs, activities, ororganizations for effectiveness is costly and time-consuming. Many of the criticisms of effectivenessreviews (but by no means all of them) are based,ironically, on the other two Es, the notions ofeconomy and efficiency in conducting effectivenessreviews, applying the test of value for money onthe method of assessing effectiveness itself. Whatthe critics are really questioning is the worth ofdoing what could turn out to be effectivenessassessment which is disproportionately expensive tothe value of the programs under review.

Some see effectiveness information as usefulbut politically inopportune. Information, oncereported, is bound to be discussed publicly by vari-ous individuals—critics, media observers, politi-cians—all promoting their own perspectives, oftenwith the intent of embarrassing rather thanenlightening or supporting. In short, what is notknown cannot be used against one.

Many failures to achieve economy and effi-ciency can be attributed to inattention, lack of pru-dence, negligence, different understanding of ethicsor public morality, and so on. All that can besomewhat embarrassing under the glaring light ofpublic scrutiny, but nevertheless explainable, rela-tively easily forgiven, and in many cases not trace-able to individuals but rather to “the system,” thusallowing a certain disassociation from the events.The failure to achieve effectiveness, however, isoften attributed, rightly or wrongly, to the lack offoresight, judgment, or intellectual ability of deci-sion makers. Negative comments on these facultiesis particularly wounding to people who value intel-


lectual ability above all else. As a result, there canbe an understandable resistance to effectivenessaccountability, even among decision makers whobring a highly responsible attitude to their role andto their work. Sometimes, because effectivenessstudies are in reference to the past, the resistance istowards third-party assessors, who are looked uponas “second-guessers.”

C H A R A C T E R I S T I C S O F G O O D

R E P O R T I N G

No discussion of the conceptual issuesinvolved in reporting on performance would becomplete without a consideration of the nature ofgood reporting.

If it is to be truly useful, information broughtforward must display certain characteristics. Thesecharacteristics are common to both informationconveyed through financial statements203 or infor-mation stated in a narrative and nonquantifiedfashion. They are:

• relevance;• completeness;• predictability;• understandability;• timeliness;• comparability;• consistency;• verifiability;• credibility; and• fairness.

The following discussion briefly explains what ismeant by the above characteristics.

The information reported to the governingbodies should be relevant in that it pertains directlyto the interests, concerns, and expectations of themembers of the governing bodies and the stake-holders they represent. Anything known to be out-side their interests should be left out. Inclusions

and omissions have to be handled with a good feelfor what members of governing bodies and stake-holders expect.

The information must be comprehensive inthat it provides a complete representation of theorganization or activities under review. Thisincludes the identification of any area deliberatelyleft out, and the reasons for it. Incomplete infor-mation, offering only a partial view to an unsus-pecting reader, may be a hindrance to understand-ing if the report is thought to be complete; it mayeven be misleading, if not deceptive.

The information must be understandable tothe readers of the report. Technical matters shoulduse terminology and reasoning that is comprehen-sible for any lay readers to which the report isdirected. The most common problems are excessivedetails and technical explanations using jargon, orpedantic language. For ease of reading by thelargest number of interested people, it is preferableto use plain, everyday language to achieve under-standability without sacrificing the substance orthe facts.

Timeliness of information is also important.Complete, relevant, reliable information is of littleuse if offered at some inopportune time; for exam-ple, when it cannot be considered by the recipient,or when it is too late, or too early, for the recipientto do anything about the information given.Timeliness brings its own dilemma; accuracy is afrequent problem. Often, members of governingbodies or the public will prefer early information,even though imperfect, to perfect information pro-vided too late. Early information, however, mustnot be so inaccurate as to be positively misleading.The information must pertain to a period not toodistant in the past; otherwise, it will be useless fordecision making and will become merely a recordfor historical purposes.


When the information focuses on perfor-mance over a period of time, it is important to pre-sent it in a manner that enables comparison withearlier periods, as the historical record is useful informing a view of progress, or lack of it. Sometimes,it is most important to know how well the organi-zation is doing when compared with some externalstandard, such as other, similar organizations. Inthese cases, the information must be compiled in amanner conducive to drawing comparisons. Thus,comparability, even in the face of changes, becomesvery important. Related to the notion of compara-bility is the necessity of indicating the sources ofthe changes that may affect the comparison.Changes in methods of observation will obviouslyalter any comparison and will have to be noted.

Any rigorous comparison requires consisten-cy, which means that all the information used isprepared on the same basis. Where even small dif-ferences in performance are important, slight dif-ferences in the way in which information is collect-ed can produce misleading results and lead to erro-neous conclusions. Comparable information pre-sented on a consistent basis will help recipients ofperiodic reports become familiar with the material,which is essential for maintaining interest anddeveloping confidence in the information.

The information presented must be verifiable.It is important that the information be producedfrom sources or systems that can be traced and, ifnecessary, attested to by an independent thirdparty, thus ensuring its credibility.

Reports must be fair. Fairness is usually asso-ciated with a balanced view. Sometimes, balance isinterpreted as a negative view of something coun-terbalanced by a positive aspect of the same thing.Such an even distribution of good and bad in areport can only be achieved at the expense ofobjectivity. Probably the best way to be fair is tokeep things in perspective and be totally objective,presenting without bias both the good and the bad

in whatever proportion they present themselves.Deliberate distortion is a violation of the principleof fairness.

A difficult issue arises with presenting infor-mation that is considered to be sensitive. The con-cern is understandable; sometimes it is related tothe competitive nature of the activity beingdescribed—revealing it would give an advantage toa competitor. Material affecting national securityand individual privacy is sensitive. Another exam-ple of sensitive information and a natural reluc-tance to display it is when it might reveal certainweaknesses leading to adverse consequences for theorganization or its leaders. This can frequentlyoccur in a political environment. For those who areconcerned about being embarrassed by weaknessesbeing revealed, there is no alternative but to with-hold or somehow obfuscate the information.Courage and integrity—in short, feeling trulyaccountable—can overcome these understandableconcerns. With the rise of due diligence as a con-cept of liability, these concerns will be overshad-owed by the requirement for more forthcomingreporting.

The foregoing is but a partial list of character-istics of good information and reporting. The char-acteristics are interrelated, with several being a pre-condition for others. The explanations given are farfrom exhaustive and are designed merely to remindreaders of something they already know: good andcredible information is the product of care andintegrity.

Analysts and media commentators (andopposition members in political forums) often sub-ject public information to careful scrutiny. If theinformation does not meet the above conditions, itwill quickly be discredited, as will the individualswho have provided it.

If members of the governing body and otherstakeholders in an organization do not develop ahigh level of confidence in the information with


which they are provided, they will not be able tointervene appropriately to correct performance,nor, for that matter, will they be able to acknowl-edge good performance where it has been achieved.In short, without good information they will beunable to fulfill the duties entrusted to them.

A B O U T R E P R E S E N T A T I O N S

W H AT A R E T H E Y ?

Representations are statements claiming thatsomething is true about something else. They arewritten in the form of a sentence, or series of sen-tences, and are designed to undergo the test oftruth. A representation will achieve its main pur-pose if it states something that is meaningful and infact true, and if the reader agrees that it is. Authorsof representations have to assure themselves of thesoundness of the facts on which their statements arebased and must appreciate that readers will be seek-ing both confidence in the underlying informationand relevance of its substance.

Representations can be powerful in describinga situation, although they do not have the conve-nience of handling and the appearance of verifiablemathematical exactitude offered by accountingusing numbers, such as financial statements. Butby obeying the rules of logic and of exposition, it ispossible to construct a highly credible rendering ofaccount with words alone.

The vocabulary of exposition204 is abundantand contains the many criteria used daily in assess-ing representations made by various people in avariety of circumstances. The test of truth or falsitythat can be applied to a representation is notalways easy, usually for lack of solid information.Compensating for the absence of complete evi-dence, however, is the fortunate characteristic thatrepresentations emit telltale signs of less than full,meaningful, and truthful information. Carefulanalysis will succeed in ferreting out most of them.

S O M E C O M M O N P I T FA L L S

A representation can be grammatically correctand yet be totally meaningless. It may be couchedin a way that defies verification or understandingbecause of lack of necessary context. In some othercases, the basis for making the representation isabsolutely correct but the sentence suffers fromgrammatical or syntactic disorders that preventclear understanding.

Representations can also be so vague as to pre-vent interpretation, particularly when the statementsappear to be so unrelated to facts or experience thattheir truth becomes hard to assess. Vagueness isrelated to lack of clarity and may or may not bedeliberate. Ambiguity is a different affliction alto-gether: ambiguous statements do have clear mean-ings, but more than one, and it is not possible tochoose which was intended. This situation is usuallymore accidental than intentional, as these statementsare commonly the results of bad syntax.

ONE PROFESSOR, WHEN ASKED TO PROVIDE REFERENCES FOR

MEDIOCRE STUDENTS SEEKING EMPLOYMENT, OFTEN WROTE:

“YOU WILL BE VERY LUCKY INDEED IF YOU CAN GET THIS

YOUNG MAN TO WORK FOR YOU.”

When a representation’s relation to the facts isunclear, it is because the statements are meaning-less in the first place, or vague or ambiguous. Butfacts can be distorted. Statements can include two(or more) segments that are absolutely clear andtrue but imply by their juxtaposition a causal con-nection between them that may not be true orcould not be demonstrated; such statements aremisleading. Again, this situation may or may notbe intentional. Sometimes, a strong belief in one’spoint of view carries with it a surfeit of enthusiasmthat results in slanting the facts to buttress it.


A frequent way to distort the facts, deliberate-ly or by excess of enthusiasm, is to exaggerate,using stronger claims than reasonably justified withwords like “all,” “totally,” and “extremely,” orextravagant adjectives like “tremendous” or “extra-ordinary.” Conversely, to understate—sometimesto diminish guilt, or not to offend, or simply as aresult of excessive caution—will lead to similarmisrepresentation.

An exaggerated representation describes thefacts too strongly. An overgeneralization is some-what different; it claims more factual support thanit actually possesses. Similarly, an oversimplifica-tion is different from an understatement: it ignoresthe complexity of a situation. In a very elaboratedemonstration, however, an oversimplification maybe used as a convenience, a link between twoimportant explanations. Authors who use an over-simplification for purposes of convenience oftenadmit that they do so, which makes it perfectlylegitimate.

Representations that are meaningless, vague,ambiguous, misleading, exaggerated, understated,overgeneralized, or oversimplified all suffer fromimproper relationships between the representationsand the facts supporting them. Such perversions,when carried too far, can make the representationsuntrue.

Other faults often observed in representa-tions, such as triviality, irrelevance, obliqueness,and tautology, are quite different from thosedescribed above in that they do not bear on thetruthfulness of the statement; in fact, representa-tions that have these symptoms are accurate. Butthey may not be useful.

A trivial statement contains descriptions soobvious (indeed, so true) as not to be worthy ofmention. Irrelevant statements are easy to spot:they miss the point altogether—an indication thatthe author has not mastered his or her thesis.

Oblique statements are often intentional anddesigned to be more persuasive, or are made out ofa bad habit. They rely on a particular intimacybetween writer and reader for correct interpreta-tion. Oblique statements use metaphors or irony toreinforce the message.

Tautological statements contain a segmentthat, in different words, merely repeats anothersegment. On that basis, some definitions will lookto be tautological but are not. But if the repetitionis intended as an explanation, it will not meet itspurpose.

Self-contradiction is exactly the opposite oftautology. One segment in a representation contra-dicts another segment, in logic or in syntax.

The above enumeration of possible traps, pit-falls, and faults is not intended to discourage themaking of representations. On the contrary, it isoffered to indicate that, by respecting the rules ofgrammar, it is possible to use words in a manner asrigorous and credible as the numbers in financialstatements that faithfully reflect the financial con-ditions of an organization. Such rules may appeardifficult to adhere to entirely, but they are not iftheir authors bring into the formulation of repre-sentations a healthy attitude towards accountabili-ty, a strong desire to inform; it will be easier thento avoid most of the pitfalls discussed above.

This concludes the theoretical discussion ofreporting on performance. The following chaptersof Part II suggests about how an organizationmight go about doing such reporting. Note thatwhat follows are suggestions only, and representone approach only, albeit one that has been founduseful in a number of situations. Anyone imple-menting the suggested approach should be pre-pared to modify it as required.



174 WILSON COMMITTEE, 9.175 IBID., 20.176 IBID., 14.177 COMPREHENSIVE AUDITING IN CANADA, THE PROVINCIAL LEGISLATIVE AUDIT PERSPECTIVE (OTTAWA: CCAF, 1985), 45.178 LEE D. PARKER, VALUE-FOR-MONEY AUDITING: CONCEPTUAL, DEVELOPMENT, AND OPERATIONAL ISSUES (CAULFIELD, VICTORIA: AUSTRALIAN ACCOUNTING RESEARCH FOUNDATION, 1986),

13.179 GOVERNMENTAL ACCOUNTING STANDARDS BOARD, CONCEPTS STATEMENT NO. 2, 8-9.180 THE FEDERAL AND ONTARIO AUDIT STATUTES CHARGE THE AUDITOR WITH DETERMINING WHETHER MONEY WAS SPENT WITH DUE REGARD TO ECONOMY AND EFFICIENCY IN THE ACQUISI-

TION AND MANAGEMENT OF GOODS AND SERVICES, AND WHETHER THE EFFECTIVENESS OF PROGRAMS IS BEING MEASURED AND REPORTED IN ALL INSTANCES WHERE SUCH MEASUREMENT

IS FEASIBLE AND PRACTICAL. (MANY OTHER PROVINCES GIVE THEIR AUDITORS SIMILAR MANDATES.) WITH SUCH TERMS OF REFERENCE, IT IS CLEARLY NECESSARY FOR THEM TO MAKE DIS-TINCTIONS WITH RESPECT TO THE 3 ES.

181 THIS POINT WAS MADE IN THE AUDITOR GENERAL’S COMPREHENSIVE AUDITING IN CROWN CORPORATIONS, 56.182 J. E. HODGETTS, “VALUES IN THE ADMINISTRATION OF PUBLIC AFFAIRS,” CANADIAN PUBLIC ADMINISTRATION , VOL. 25 (4), WINTER 1982, 474, AS CITED BY RICK VAN LOON, “THE TWINS

ARE NOT IDENTICAL: MANAGEMENT AND MANAGEMENT EDUCATION IN THE PUBLIC AND PRIVATE SECTORS,” IN BENOÎT BAZOGE AND GILLES PAQUET, “ADMINISTRATION: UNITY AND

DIVERSITY,” UNIVERSITY OF OTTAWA QUARTERLY, VOL. 56, NO. 4, OCTOBER/DECEMBER 1986, 74.183 WARREN CHIPPINDALE, “SOLVING THE CASE OF THE ENIGMATIC THIRD ‘E’,” CA MAGAZINE, JANUARY/FEBRUARY 1988, 33-45.184 EFFECTIVENESS REPORTING AND AUDITING IN THE PUBLIC SECTOR (OTTAWA: CCAF, 1987), 64.185 SEE THE “GENERAL LITERATURE SURVEY PROJECT” IN EFFECTIVENESS REPORTING AND AUDITING IN THE PUBLIC SECTOR: SUPPORTING RESEARCH STUDIES (OTTAWA: CCAF, 1988), 8.186 P. M. JACKSON, THE POLITICAL ECONOMY OF BUREAUCRACY (OXFORD: PHILIP ALLAN, 1982), 189-90.187 IBID., 189. THE USE OF THE WORD EFFICIENT IN THIS QUOTATION INSTEAD OF EFFECTIVENESS, A TERM MORE DIRECTLY RELATED TO GOAL ATTAINMENT, SHOULD NOT BE CONSTRUED TO

MEAN EFFICIENCY IN CONTRAST TO EFFECTIVENESS IN OUR USE OF THE 3 ES. MICROECONOMICS IS A BRANCH OF ECONOMICS THAT DEALS WITH THE ECONOMIC BEHAVIOUR (THE WAY IN

WHICH RESOURCES ARE ALLOCATED AMONG ALTERNATIVE USES TO SATISFY HUMAN WANTS) OF INDIVIDUAL ENTITIES LIKE PRODUCTION UNITS, FIRMS, CONSUMERS. (MACROECONOMICS

DEALS WITH THE BEHAVIOUR OF ECONOMIC AGGREGATES LIKE GROSS DOMESTIC PRODUCT (GDP), RATES OF INFLATION, LEVELS OF UNEMPLOYMENT). IT IS BECAUSE JACKSON WAS EXPRESS-ING HIS POINT OF VIEW WITHIN THE CONTEXT OF THE FORMAL ANALYSIS OF THE CONCEPT OF EFFICIENCY, ONE OF THE IMPORTANT ACHIEVEMENTS OF MICROECONOMICS, THAT HE USED

THE WORD EFFICIENT. THE CONCEPT OF EFFICIENCY IN MICROECONOMICS ENCOMPASSES EFFECTIVENESS IN THE SENSE THAT ORGANIZATIONS ARE DEEMED TO BE GOAL-SEEKING ENTITIES.188 WILSON COMMITTEE, 34.189 K. S. CAMERON, “CRITICAL QUESTIONS IN ASSESSING ORGANIZATIONAL EFFECTIVENESS,” ORGANIZATIONAL DYNAMICS, VOL. 9, NO. 2, 1980.190 ANDREW MACDONALD, ACTING COMPTROLLER GENERAL OF CANADA, CITED IN REPORT OF THE STANDING SENATE COMMITTEE ON NATIONAL FINANCE, COMPREHENSIVE AUDITING, MARCH

1988, 8.191 SEE EXPLANATION OF THE NOTION OF INCONSISTENCY OF TIME IN PART I, 33.192 “GENERAL LITERATURE SURVEY,” 2.193 SEE RONALD E. HOYT, “EPISTEMOLOGICAL CHOICES FOR THE DEVELOPMENT OF ACCOUNTING THEORY,” UNIVERSITY OF OTTAWA QUARTERLY, VOL. 56, NO. 4, P. 235 FOR AN INTERESTING

AND RARE TREATMENT OF THE TOPIC ON CHOICES OF ATTRIBUTES FOR ACCOUNTING AND, FOR THAT MATTER, REPORTING PURPOSES.194 J. P. CAMPBELL, IN PAUL S. GOODMAN AND JOHANNES S. PENNINGS, EDS., NEW PERSPECTIVES ON ORGANIZATIONAL EFFECTIVENESS (SAN FRANCISCO: JOSEY-BASS PUBLISHERS, 1977)195 “GENERAL LITERATURE SURVEY,” 13.196 ROBERT E. QUINN AND JOHN ROHRBAUGH, “A SPATIAL MODEL OF EFFECTIVENESS CRITERIA: TOWARDS A COMPETING VALUES APPROACH TO ORGANIZATIONAL ANALYSIS,” MANAGEMENT

SCIENCE, VOL. 29, 1983; SEE ALSO “GENERAL LITERATURE SURVEY,” 14.197 THOMAS J. PETERS AND ROBERT H. WATERMAN, JR., IN SEARCH OF EXCELLENCE, LESSONS FROM AMERICA’S BEST-RUN COMPANIES (NEW YORK: HARPER AND ROW, 1982), 11.198 IBID., 13-15.199 TOM PETERS AND NANCY AUSTIN, A PASSION FOR EXCELLENCE: THE LEADERSHIP DIFFERENCE (NEW YORK: RANDOM HOUSE, 1985).200 REPORT OF THE AUDITOR GENERAL OF CANADA TO THE HOUSE OF COMMONS, FISCAL YEAR ENDED 31 MARCH 1988 (OTTAWA: MINISTER OF SUPPLY AND SERVICES CANADA, 1988).201 SEE DAVID GARVIN, “HOW THE BALDRIDGE AWARD REALLY WORKS,” HARVARD BUSINESS REVIEW, NOVEMBER-DECEMBER 1991, VOL. 69, NO. 6, 84.202 NATIONAL QUALITY INSTITUTE, 1995 ENTRY GUIDE FOR THE CANADA AWARDS FOR EXCELLENCE: BUSINESS—EDUCATION—GOVERNMENT—HEALTH CARE (NATIONAL QUALITY INSTITUTE,

1995), 10-15.203 MOST OF THESE CHARACTERISTICS WERE ELABORATED UPON BY THE CICA, IN A BULLETIN ISSUED BY THE PUBLIC SECTOR ACCOUNTING AND AUDITING COMMITTEE TITLED “PUBLIC

SECTOR ACCOUNTING STATEMENT # 2, OBJECTIVES OF GOVERNMENT FINANCIAL STATEMENTS,” NOVEMBER 1984. BY NOW, THEY ARE TO BE FOUND IN MANY PUBLICATIONS OF SEVERAL

REVIEW DISCIPLINES WITH WORDS APPROPRIATE TO THE PARTICULAR CRAFT. THE CICA, ALONG WITH OTHER PROFESSIONAL ACCOUNTING BODIES, HAD BEEN DEVELOPING ATTRIBUTES OF

USEFUL REPORTING MANY YEARS EARLIER FOR USE IN THE PRIVATE SECTOR.204 HAROLD C. MARTIN AND RICHARD M. OHMAN, THE LOGIC AND RHETORIC OF EXPOSITION (TORONTO: HOLT, RINEHART AND WINSTON, 1965), 57FF. CONTAINS A SECTION ON WAYS OF

JUDGING ASSERTIONS, WHICH IS LARGELY THE BASIS OF THIS EXPLANATION OF THE VOCABULARY OF EXPOSITION.

S E C T I O N 2

PRINCIPLES ANDGUIDELINES FOR



C H A P T E R 1 0

ANEFFECTIVENESSREPORTINGFRAMEWORK

B A C K G R O U N D

In late 1985, the board of governors ofCCAF felt compelled to address the issue ofreporting on effectiveness in public sector organiza-tions. In part, this decision reflected the generallevel of interest in the subject throughout theCanadian public sector. In part, it was a responseto the legislation affecting federal Crown corpora-tions that charged their auditors with commentingon whether these corporations’ systems and prac-tices were maintained in a manner that providedreasonable assurance that (among other matters)their operations were carried out effectively.Without a better understanding of just what effec-tiveness means, the auditors would be unable tofulfill this obligation appropriately.

CCAF appointed the Independent Panel onEffectiveness Reporting and Auditing to undertakethis project. The panel comprised a number ofpeople holding senior positions in the public andprivate sectors with extensive knowledge and expe-rience in public sector management, governance,and auditing. The panel consulted with electedrepresentatives, senior officials in public sectormanagement, as well as with eminent practitionersin auditing and other disciplines. More precisely,those consulted included:

• members of Parliament from all political parties;• deputy ministers; • chief executive officers of Crown corpora-

tions;• leading program evaluators and other profes-

sionals engaged in the fields of performanceand organizational measurement;

• representatives of provincial legislative auditoffices across Canada and the Office of theAuditor General of Canada; and

• partners of prominent firms of charteredaccountants and management consultants.

The consultations were supported by threemajor research projects: a review and synthesis ofthe literature on accountability and effectiveness; areview of how effectiveness is perceived by expertson organizational matters; and a review of whatpublic sector managers have actually said aboutassessment of effectiveness in published docu-ments. The results of these studies were publishedin a single volume.205

The panel decided that any proposals it madewould have to pass the following tests:

• they should be useful and practical for all par-ties to the accountability relationship;

• they should capitalize on the considerableprogress and investment that had alreadybeen made to report and audit effectivenessinformation;

• their orientation should be to build confi-dence and promote understanding and agree-ment between those who give responsibilityto carry out policy and those who accept it;

• they should be starting points for takingaction and catalysts for developing consensus,and not be viewed as prescriptive; and

• they should help the audit profession torespond to expectations that are emerging inboth the public and private sectors.


The panel arrived at a number of conclusions.With respect to the reporting of performance, theymight be summarized as follows:

First, “effectiveness” is a variable concept,as evidenced in the literature and in practice.On balance, effectiveness is best conceived as amultidimensional concept incorporating andlinking several facets of performance that keystakeholder groups—taken as a whole—wouldjudge as generally responsive to their interests.

Second, those who govern our public sec-tor institutions are faced with formidable chal-lenges, choices, and decisions. They also have afundamental responsibility to know what isbeing done and accomplished by those to whomthey have delegated authority and resources. Toknow this, these governing bodies need informa-tion that is organized and presented in a waythat responds to their strategic interests. Theyneed to be able to deal with this information ina practical sense, and to have confidence in itsbeing reasonable to the circumstances.

Third, management—as the party towhom authority and resources are delegated bytheir governing body—is in the best positionto report on the effectiveness of their organiza-tion and major lines of business (that is, makemanagement representations or explicit evi-dence-based statements about effectiveness). Asthe party receiving this delegation, it logicallyand first falls to management to demonstratetheir accountability—through reporting. Thepanel judged that—after many years of invest-ment and learning on management’s part inrelation to the theory and technology of mea-suring and reporting on performance—nowwas the time to expect management to playthis fundamental role.

Fourth, to expect management to playthis role—and others to play their part, aswell—a reasonable starting point needs to be

established from which appropriate actions canbe discussed, agreed to, and then taken. To thisend, the panel proposed a framework of twelveattributes of effectiveness that, collectively,cover all the important aspects of a publicorganization’s performance. By reporting onhow well it is doing in respect to each of theseattributes, an organization is answering all thesignificant questions that its governing bodyand other stakeholders are likely to ask.

And fifth, the panel concluded that sim-ply having management report on performanceto the governing body is insufficient. The gov-erning body needs to have confidence that thisinformation is fair and complete. This requiresan audit role that reports directly to the gov-erning body and that provides the necessarychallenge and assurance to the informationprovided by management.206

M A N A G E M E N T

R E P R E S E N T A T I O N S

The panel’s concept of management provid-ing representations on performance establishes adirect accountability link between managementand the governing body, in keeping with the long-recognized reporting obligation. As the panel notedin its report, it “provides an opportunity for man-agers to explain effectiveness in a context that theyand their governing bodies agree is appropriate. Ithelps to prevent wasted effort by ensuring thatinformation about effectiveness focuses on impor-tant matters presented at a meaningful level. It alsohelps managers clarify their accountability to gov-erning bodies and to promote better accountabilitywithin their organization.”207

As we have seen, common practice in Canadahas been for legislative auditors to assess and reporton value-for-money matters, but for auditors to beconstrained in commenting on the effectiveness ofthe organization under review. The management

P A R T I I . C H A P T E R 1 0 . A N E F F E C T I V E N E S S - R E P O R T I N G F R A M E W O R K 1 3 7

representations approach places the responsibilityfor such reporting squarely on the shoulders ofmanagement, who have both the obligation to beaccountable and are in the best position to knowwhat is going on. This approach also overcomesthe need for eschewing effectiveness issues, since itis management, not some third party, that is issu-ing the statements. In effect, it makes reporting oneffectiveness similar to preparing financial state-ments—a responsibility of management.

Management representations have beendescribed in the following terms:

A management representation is anexplicit statement made by senior manage-ment to their governing body… in relation toan aspect of the organization’s effectiveness orperformance.

It is a means by which management dis-charges an important aspect of their account-ability, i.e., to report to those who haveapproved the organization’s mandate, and del-egated or provided the requisite authority andresources.

A management representation providesinput to decision making by the governingbody, and provides a basis for the governingbody to exercise its oversight responsibilities,i.e. holding management to account for theperformance of the organization.

The information included in a manage-ment representation also provides the govern-ing body with an important basis for explain-ing the performance of the organization—and indeed, their own decisions—to the pub-lic whose interests they are elected orappointed to represent, and to other stake-holders in the affairs of the organization.208

The panel recognized that the preparation ofmanagement representations would be undertakendifferently in different organizations. The panel

knew that what is germane to one organization isnot necessarily appropriate for another. “As a result,the decision to use this approach involves a com-mitment to a substantial amount of effort to refinethe concept and make it work satisfactorily in eachspecific context.”209 Moreover, because of the polit-ical environment in which public sector organiza-tions operate, the panel realized that care must betaken in deciding the nature and form of the infor-mation to be disclosed.

A T T R I B U T E S O F


Aware of the literature on the subject, thepanel considered whether the concepts of program,organizational, and operational effectiveness couldprovide useful tools of analysis. It decided that this“three-way categorization commingles so muchinformation of different types that the results arenot meaningful in any practical sense. It is moreuseful to consider a framework at one level lower.It is expected that information organized at…[that] level… can be substantiated.”210

The panel proposed a list or framework oftwelve attributes that captures the major conceptsfound in the literature on effectiveness. The listwas subjected to careful review and criticism bymanagers and measurement and audit practition-ers. It does not reflect any single exclusive view ofeffectiveness, but is intended to respond to thewide interests of members of governing bodies.The relatively high-level understanding of effective-ness that this framework represents subsumes themost significant elements of economy and efficien-cy, and issues regarding these virtues are found inthe attributes.

No single one of these attributes provides afull appreciation of an organization’s effectiveness.But with information on all these attributes, or onall that are relevant in the circumstances, people


can make informed judgments about the organiza-tion and its management. The panel decided thatreporting on the basis of the framework should sat-isfy the needs of the diverse stakeholders (manage-ment, government, legislators, and the public) whohave varied interests and perceptions and, hence,different information requirements.

Apart from providing a reporting framework,the list of attributes can be used by members ofgoverning bodies as an agenda in their discussionswith management to draw out a broader and deep-er range of effectiveness information than they mayhave received in the past.

As mentioned above, certain of the attributeswill be more significant to some organizations thanto others. Some may not apply at all. Time and cir-cumstances may affect the significance of someattributes. Common sense is to be used in theapplication of the framework.

The attributes that the panel thought com-prehended the concept of effectiveness are:211

• management direction• relevance• appropriateness• achievement of intended results• acceptance• secondary impacts• costs and productivity• responsiveness• financial results• working environment• protection of assets• monitoring and reporting

The following discussion explains what ismeant by each of these terms and why the panelthought they are important.

M A N AG E M E N T D I R E C T I O N

Management direction is the extent towhich the objectives of an organization, its com-ponent programs or lines of business, are clear,well-integrated and understood by its employees,and appropriately reflected in the organization’splans, structure, delegations of authority, anddecision-making processes. Management directionembodies both the accountability of an organiza-tion and its agenda for action. It is not enoughthat the agenda be developed; it must be clearlyreflected in the plans and strategies of subordinateorganizational units and understood by thoseresponsible for execution. Management directionaffects an organization’s capacity to implement itsmandate and to adjust to changing circumstances.Without clear management direction, there are noguidelines to help direct change, integrate ongoingactivity, or monitor and report on performance.

Information in relation to this attribute willhelp assess trade-offs, balances, and linkages amongthe following facets of the institution:

• management style, structure, and processesthat recognize the need for professional lati-tude but at the same time assure appropriatesupervision and control over matters centralto the raison d’être of the institution and tothe continuity of its operational capacity;

• an internalized value system that is client-centered and directed towards providing themost cost-effective service achievable withinthe context of professional standards andcodes of ethics, regulations, and availableresources; and

• effective linkages among and between planningand communications processes at the corporateand operating levels of the institution.


R E L E VA N C E

Relevance refers to the extent to which a pro-gram or line of business continues to make sense inregard to the problems or conditions it is intendedto address. It is an attribute that normally wouldbe considered in preparing a management repre-sentation about a major program, activity, or lineof business within an organization. Informationabout this attribute could lead to confirmation,amendment, or elimination of that program, activ-ity, or business.

This attribute is an extremely important,complex, and potentially controversial aspect ofaccountability. Changes in philosophies and poli-cies regarding the role of public institutions insociety, public expectations, the network of alterna-tive programs and services, and the status andneeds of the community all influence decisionsabout the relevance of programs and services, aswell as choices for the future. Information on thesematters can be important:

• to assure the governing body that the activi-ties of their institution are within the organi-zation’s mandate;

• to highlight any matters that do or couldinhibit the performance of this mandate;

• to identify actual and potential requirements,opportunities and choices; and

• to help the governing body set policy that isresponsive to current needs and emergingtrends and events.

Of course, it is not for management to decideon relevance; this is a policy decision that the gov-erning body must make. But to make this decision,the governing body needs a framework for analysis,and it needs information. It will draw on manysources to extract this information. In that man-agement is responsible for implementing the pro-gram, it is reasonable to expect management to

provide information and advice that the governingbody will then use as a basis for considering thecontinuing need for the program and/or the pro-gram mandate.

A P P R O P R I AT E N E S S

Appropriateness measures the extent to whichthe design of a program or its major components,and the level of effort being made to implementthe program, are logical in light of the program’sobjectives. Design considerations may involveinformation about the choices (for example, regula-tion, financial incentives, and service delivery)available to pursue the public policy objective inquestion. This may also involve information aboutconstraints, the needs of those affected by the pro-gram, and administrative strategies (such as make-or-buy, contracting-out, and decision-making prac-tices) that influence the design of the program.Information on this attribute can help manage-ment and the governing body assess the currentand possible alternative targeting, span, depth, andbalance of support/services relative to needs, and toanalyze the nature and consequences of any gaps.

Information on this attribute may confirm ordisprove that services have the capacity to reach theclientele and that there are no major flaws in logicor design significantly inhibiting the achievementof intended results. In general, information onappropriateness might help identify opportunitiesto improve the internal operations or the externaldelivery of a program, and to assess alternativedelivery vehicles.

This attribute is closely related to theattribute of relevance. The distinctions between thetwo are sometimes difficult to maintain and articu-late since, to a degree, they draw on the sameinformation sources. The difference between thetwo might be seen as follows:


• relevance is concerned with questioning thelegitimacy and pertinence of the program’sobjectives;

• appropriateness starts with the program objec-tive(s) as a given and is focused on an analysisof how well the design and delivery of theprogram suit the objective(s). Essentially, theissue is one of determining the extent towhich the means are proportional to the ends.

AC H I E V E M E N T O F I N T E N D E D R E S U LT S

This attribute concerns information relatingto the extent to which goals and objectives havebeen realized. It is important to emphasize that theissue is not one of simply reporting what has hap-pened as a result of a program or activity, butreporting in such a way that helps governing bod-ies judge whether the level of achievement is satis-factory. This can be a difficult task, as many objec-tives are not stated in terms of explicit ends butrather in terms of continuing effort or relativeachievement. Often, objectives are couched in suchqualitative terms as appropriate, adequate, and rea-sonable. Additional difficulties arise where objec-tives for a particular program or line of business arenot mutually supporting or, indeed, may be com-peting or conflicting.

It is important to decide what level of resultsis being addressed: is it the level of production orthe level of intended impact? Both these levels canbe useful to governing bodies. At the level of pro-duction, the concern is with the specific goods orservices a program produces. Generally, informa-tion about this level of results would be generatedthrough internal operational performance manage-ment systems. This information by itself, however,is unlikely to be sufficient for governing bodies.The raison d’être of most programs, after all, is notsimply to produce goods or services, but to do sowith some valued end or impact in mind.

If information on this attribute is focused onthe institution as a whole—or on one of its majorprograms—it will give the governing body a moreintegrated perspective of overall performance thanthey would be able to piece together from reportson individual departments or other organizationalunits, which typically focus at the output level andsometimes only on pieces of a larger whole. Formanagement, this program perspective can also beuseful in directing and assessing the underlyingstrategies, goals, work plans, and activities of indi-vidual operating or organizational units within theinstitution, and in identifying and analyzing possi-ble trade-offs.

AC C E P TA N C E

This attribute concerns the extent to whichconstituencies to which a program or line of busi-ness is directed judge it to be satisfactory.Information on this attribute can bring an addedperspective to management representations onachievement of satisfactory results.

A program may be achieving its intendedresults but still not be considered satisfactory bythe constituency to which it is directed.Conversely, the clientele might be satisfied with alower level of effort than is currently being expend-ed. However, gaps between what is being achievedand the acceptance this receives may not point to aperformance problem per se, but rather to an expec-tations gap that needs to be addressed through vari-ous information and educational activities.

Clients’ acceptance and satisfaction may con-tribute significantly to the intended results (forexample, if taxpayers feel they are being dealt withfairly, they are more likely to comply voluntarily).Similarly, lack of acceptance may undermineintended results. The perceptions of staff regardingthe level of acceptance by clients are a valuableinput but, in and of themselves, are generally an


insufficient basis upon which to make credibleconclusions.

If an institution has an appropriately widerange of information regarding the acceptance orsatisfaction of its clientele, it may avoid the possi-ble problem of being captured by any one con-stituency or interest group.

Willingness on the part of an institution toseek out views of stakeholder groups also improvesthe visibility of the institution and shows it to beboth accessible and answerable to the public itserves. This is particularly important within thecontext of current “service to the public” initia-tives. It is also important within the context of dis-cussions that institutions now feel they need tohave with their constituents on how best and fairlyto respond to fiscal constraints.

S E C O N D A RY I M PAC T S

This attribute concerns the extent to whichother significant consequences, either intended orunintended and either positive or negative, haveoccurred. For instance, knowing the overall eco-nomic impact of a program designed to provideadequate, affordable housing to specific incomegroups; knowing the environmental impact of aneconomic development project; knowing the disin-centive effect of social assistance payments.Information on this attribute can be important todecisions regarding the design and operation of aprogram or, more fundamentally, its continuationor termination. There are probably many sec-ondary impacts that go beyond the ability of anyorganization to appreciate, measure, and, most cer-tainly, to control.

Information on secondary impacts (both pos-itive and unwelcome) can help senior managementand the governing body identify important trade-offs and assess the consequences of specific choices.

It can also provide the means for management andthe governing body to explain their choices to, anddiscuss alternative approaches with, a wide varietyof external constituencies—such as the communityat large, other institutions, and government. Forexample, significant negative side effects to a pro-gram might cause a reconsideration of the overallmerits of having the program in the first place.Alternatively, information on such secondaryimpacts could lead the governing body and man-agement to consider ways to redesign the programto eliminate or reduce unwelcome effects.

CO S T S A N D P R O D U C T I V I T Y

This attribute concerns the relationship ofcosts, inputs, and outputs. Because governing bod-ies are responsible for approving budgets, costinformation is essential to them. Cost informationcan be classified and aggregated in a number ofways, such as the goods and services acquired,responsibility for incurring the costs, or the activitythat is incurring the cost.

The importance of good cost information isall too easily taken for granted. Deficiencies in thisregard were recognized by the Federal GovernmentReporting Study, jointly undertaken by theAuditor General of Canada and the ComptrollerGeneral of the United States, which proposedmajor reforms in the way government accountingis done, particularly in such areas as full-costaccounting.

During times of severe constraints and declin-ing resources, it is imperative that managementand the governing body be able to demonstratethat the institution is using its resources to bestadvantage, and that the organization is constantlylooking for opportunities to free up resources sothat they can be reinvested where they will do themost good.


R E S P O N S I V E N E S S

This attribute concerns how well an organiza-tion adapts to changes in such factors as markets,competition, available funding, and technology.Information on this attribute can help a governingbody judge how well its organization is attuned tothe publics with which it interacts, and to the vari-ous forces that are pertinent to its role, operation,and objectives. Information in this area can alsoprotect the credibility and integrity of the institu-tion. Moreover, it could provide a useful perspec-tive on the capacity of an organization to maintainits responsiveness in the face of any opportunities,threats, or other changed circumstances that mayoccur.

Current information on this attribute cangive management and the governing body both thetime and the basis for redeveloping program strate-gies, and for shifting priorities and resources in away that will not destabilize the operations of theinstitution.

Management representations respecting thisattribute are typically highly individualistic, focus-ing on factors that are key to the ability of theorganization to respond to specific circumstancesor trends or on general preparedness in this area.

F I N A N C I A L R E S U LT S

This attribute concerns the accounting forrevenues and costs, as well as the accounting forand valuation of assets, liabilities, and equity. Thisinformation is usually reflected in periodic finan-cial statements demonstrating financial positionand financial results of operations. Using thesewell-understood presentations, managers, analysts,and other users will describe or examine informa-tion in terms of trends, ratios, and comparisonswith other organizations. In the public sector, acomparison with approved budget may also beimportant.

Since information on this attribute reflectsthe overall financial position of the institution andprovides evidence of its continuing viability,obtaining it is of central importance to both man-agement and governing bodies.

It may be useful to point out that representa-tions on this attribute include discussions of infor-mation that is contained in the organization’sfinancial statements. The ability to form judgmentsabout the adequacy of such items as sales and prof-itability will depend on consideration of informa-tion from this and other attributes.

Management typically keeps governing bodiesinformed of financial performance as the fiscalperiod progresses. Monthly or quarterly reports arecommon. More formal reporting of performanceusually accompanies the budget process, and theannual report usually contains audited informationon the past year’s performance.

W O R K I N G E N V I R O N M E N T

This attribute concerns the extent to whichthe organization provides an appropriate workatmosphere for its employees, provides appropriateopportunities for development and achievement,and promotes commitment, initiative, and safety.

Attention to this attribute can assure a gov-erning body that the organization has adopted con-structive ways and means to manage its personnel.It also ensures that the organization is operating inconcert with any relevant public policy objectivessuch as affirmative action.

Staff who are motivated, well trained, and sat-isfied with the management practices of the institu-tion that employs them are an important determi-nant of the institution’s capacity to achieve its mis-sion. Satisfied and well-trained personnel are alsolikely to stay with the institution, an important con-sideration in maintaining continuity in operationsand protecting the institution’s intellectual capital.


Adequate space and facilities, equipment, andsafety practices are necessary to minimize dangerand/or accidents to clients, staff, and visitors.Attractive and clean facilities enhance the moraleof staff and help create a good public image for theinstitution.

Staff expect that the institution will invest injob training and career development. The trainingemployees obtain significantly affects the quality ofservice they provide, their progress within the insti-tution, and the institution’s ability to retain themand to benefit from its investments.

Since human resources are such a large por-tion of the budget of most organizations, high ratesof turnover or absenteeism and low levels of pro-ductivity caused by poor working environmentscan waste valuable resources.

P R OT E C T I O N O F A S S E T S

This attribute concerns the extent to whichimportant assets are safeguarded so that the organi-zation is protected from the danger of losses thatcould threaten its success, perhaps its very exis-tence. Assets that fall into this category couldinclude key personnel, sources of supply, valuableproperty, agreements, and important information.

Loss of key assets, or their impairment, with-out recourse to reasonable alternatives, can com-promise the credibility of the institution and itscapacity to operate effectively. Failure to protectsuch assets can lead to loss of public confidence,litigation, and other serious problems.

Governing bodies need assurance that man-agement has identified key assets, assessed risk interms of their loss and impairment, and havestrategies in place to prevent, reduce, or counteractproblems as they emerge.

M O N I T O R I N G A N D R E P O R T I N G

This attribute concerns the extent to whichkey matters pertaining to performance and organi-zational strength are identified and carefully moni-tored. The results of that monitoring should bereported regularly and accurately. Issues that mightbe monitored in all organizations include, forexample, the extent to which key objectives areattained, limits of authority and compliancerequirements are respected, and essential systemsand practices are maintained.

The emphasis here is monitoring and report-ing on the areas that are crucial for accountability,for success or failure—not everything. In healthcare, for example, matters pertaining to the preven-tion, identification, and correction of potentialrisks and actual incidents must be monitored if ahospital’s responsibilities concerning quality careand safety of patients are to be met appropriately.The trustees need to have this information to dis-charge their responsibilities.

This information is needed to plan, allocateresources, delegate responsibility, supervise perfor-mance, assure quality, make decisions, and main-tain accountability. It also facilitates a dialoguebetween staff and management, among manage-ment, and between management, the governingbody, and other stakeholders.

Different levels within the organization requiredifferent information at various times. For example,senior management needs information on the twelveattributes of effectiveness in order to manage corpo-rate affairs and to discharge its accountability to thegoverning body through comprehensive reportingon all matters of corporate significance. Althoughelements of these attributes may also be relevant toothers further down in the hierarchy, concerns ofthese individuals are likely focused on importantoperational matters, and they may need to have cer-tain information on a day-to-day basis.


G U I D E L I N E S F O R M A K I N G

R E P R E S E N T A T I O N S O N T H E

A T T R I B U T E S

The panel recognized that in calling for repre-sentations on the proposed attributes of effective-ness, they were asking managers to do somethingwith which they had little or no experience.However, it had a clear idea of the sort of represen-tations it had in mind:

What are clearly required are representa-tions that present sufficient but not too muchinformation about effectiveness… theyshould be sufficiently detailed to allow thereader to follow the reasoning behind eachstatement, thus permitting an assessment ofboth its validity and significance. Accordingly,in applying the principle to a specific majorprogram, for example, a management repre-sentation might contain the following typesof statements:

• The extent to which this program continuesto be relevant is demonstrated by…

• The following intended results have beenachieved by this program…

• This program has such important secondaryimpacts as…

• The quality of the working environment inthis organization is demonstrated by…

While the essence of representations is infor-mation on which judgements can be based,managers may want to include their owninterpretations of the significance of thatinformation. They are completely within theirrights to do this, of course, but it must berecognized that it is the information, not theinterpretation, that constitutes the representa-tion… To help users arrive at their judge-ments, representations may include materialdealing with such matters as the degree of

certainty associated with the information,important assumptions, the manner in whichthe information was generated, managementperceptions, and risks.212

To assist organizations that decide to followtheir proposals, the panel offered several guidelinesfor instituting an effectiveness reporting regime.213

G U I D E L I N E 1

All applicable attributes of effectiveness should be con-sidered in making management representations andan explanation should be given where any attribute,otherwise expected, is not used.

All twelve attributes should be considered asbases for making representations…. however, notall attributes will be relevant in all circumstances sothat it may be inappropriate to make representa-tions on all twelve at all times. In such instances anexplanation should be given for the omission of anotherwise expected attribute.

G U I D E L I N E 2

Representations should reflect the full range of infor-mation generated in their preparation and containwhatever explanations or qualifications about theinformation and analysis that may be necessary toallow users to make informed judgments.

It is important, too, that representationsreflect the full range of information and analysisthat underlie them. If, for example, they reflectonly the positive aspects, they will be misleadingand seen as self-serving. In addition, they shouldclearly indicate any assumptions that were usedand any areas where the information on whichthey were based is either incomplete or tenuous.Certain sources of information may entail inherentbias, certain types or depth of analysis may bemore authoritative than others, or the evidencetaken as a whole may be equivocal. Where factors


such as these might have a significant effect on thereader’s understanding of the statement, suchexplanations should be disclosed in the representa-tion itself. If similar representations have beenmade in the past on a different basis than the cur-rent representation, the change in basis should alsobe disclosed.

G U I D E L I N E 3

Representations should be prepared in such a way asto take into account the interests of governing bodies.

Common sense dictates that managementrepresentations on effectiveness should be framedwith the interests and needs of the addressees—governing bodies—in mind; in other words, theyshould be user-oriented. One way of ensuring thisis to have the representations take into account themajor objectives of the governing body to whomthey are directed. Such objectives are apt to fallinto one or more of the following categories:

• to ensure equity of treatment among differentindividuals, communities and interests

• to advocate the interests of certain individualsand communities

• to preserve or advocate certain values forwhich they entered public office

• to ensure the satisfactory administration ofpublic activities

• to obtain recognition.

The objectives of the governing body willhave a direct influence on the kinds of manage-ment representations that are appropriate. If thegoverning body places primary emphasis on advo-cacy or recognition, for instance, attributes ofeffectiveness that have a strong external focus, suchas relevance, acceptance, responsiveness, achieve-ment of intended results, and secondary impact,will be most important. Conversely, an emphasison objectives concerned with the quality of admin-

istration and, to some extent, equity, might indi-cate that greater emphasis should be placed onsuch attributes as appropriateness, costs and pro-ductivity, management direction, working environ-ment, protection of assets, and monitoring/report-ing. It must be recognized that, especially in politi-cal environments, members of governing bodiesmay have quite varying interests. To the extent pos-sible, all these interests should be accommodatedin framing representations.

A user orientation also implies that the repre-sentations should reflect the governing body’s per-ception of risk. Certain matters carry with themsuch a combination of probability of occurrenceand severity of impact that they are considered tobe high-risk issues by governing bodies, who willbe looking for management representationsrespecting them. In applying this concept to itsvarious programs and activities, managementshould consider the following types of risk:

• life and health risks, particularly in areas suchas transportation, policing, health care, andenforcement of safety standards;

• economic, social and business risks, in rela-tion to such matters as industrial and regionalstability, unemployment, and income securityor rate of return;

• management systems, practices and controlrisks, particularly in relation to care, diligenceand competence needed in making decisions,supervising transactions, and protecting phys-ical, human and financial resources.

Another consideration that managementshould take into account is the capacity of the gov-erning body to decide or to exercise influence overthe issue on which management representations arebeing drafted. If, for instance, a governing bodylacks the power to introduce, terminate or alterprograms, representations with respect to attributes


such as relevance and some aspects of cost and pro-ductivity may not be of great use. Similarly, acapacity to make decisions on or influence the levelof service provided might call for an emphasis onattributes such as achievement of results and accep-tance.

G U I D E L I N E 4

Representations should be made at the highest mean-ingful level, and to accomplish this top managementmust be involved in preparing them.

There is a consensus that management repre-sentations on effectiveness are best made at ahigh—or executive—level. This is in keeping withthe level at which governing bodies can realisticallyexpect and have the capacity to provide steward-ship of public funds and at which they will receivethe greatest value from the information they areprovided for decision making. It is also consistentwith a determination to avoid information over-load, which, as is generally recognized, detractsfrom good accountability and good management.

In order to make representations that respondto the interests of the governing body, a hierarchi-cal approach should be taken. The highest level isthe entire organization; if meaningful, substantialinformation can be provided on that basis, itshould be done. If this is impossible, it is thenappropriate to consider the next highest compo-nents. This could be, for example, the organiza-tion’s major lines of business or programs, perhapsgrouped where it would be sensible to do so. Ifeven this level is too high to allow the productionof meaningful information, it will be necessary togo one step lower. The hierarchical scale should bedescended only as far as necessary to allow appro-priate representations to be made. In all likelihooddifferent levels of aggregation will have to be usedfor different attributes of effectiveness.

Worthwhile representations are likely to beachieved only if the chief executive officer isdirectly involved in shaping them. It is not rea-sonable to expect that junior-level managers orindividual measurement specialists will have thehigh-level perspective that is called for, and if theresponsibility is delegated to them, informationwill tend to mushroom. As one deputy ministerput it, “Effectiveness is too complex and impor-tant a subject to be left to others.” It is inevitablethat there will be instances where only theknowledge, judgment and authority of the chiefexecutive can determine the most appropriateapplication of attributes, either in relation to thetiming of representations on them or to thebreadth of their application.

It is recognized that having the chief execu-tive officer involved from the outset may be diffi-cult to achieve. This is because it runs counter tothe normal pattern of information generation andprocessing in most organizations, which tends tobe from the bottom up. It also places moredemands on the already busy executive. Withoutthis participation, however, there is a danger thatthe whole exercise could run the risk of being lessuseful than it should be, if not entirely wasted.

G U I D E L I N E 5

Each organization should establish a comprehensivestrategy for reporting management representations oneffectiveness over an agreed period of time.

The question of the timing of these manage-ment representations was raised frequently in thecourse of the panel’s study. In the same way inwhich global representations are thought to beimpractical, so too may be the notion that manage-ment could, or ever should, prepare comprehensiverepresentations on everything all at once. Therereally needs to be a comprehensive strategy forbringing this information forward in an organized


way. This may involve cyclical, detailed reportingwith periodic updates or exception reporting ofmajor changes since the last detailed report.

Other factors to take into account whendeciding on the timing of management representa-tions include:

• the decision-making timetable of the organi-zation and its governing body;

• the continuing relevance of previous represen-tations;

• the cost/benefit of preparing new representa-tions; and

• the audit schedule, including any timingrequirements for an audit examination orreport.

Governing bodies usually have specifictimetables to which they work. They may, forexample, want to take certain decisions at specifictimes to fulfil commitments to the electorate.Other matters affecting timing may be routine,such as the annual review of Estimates or the peri-odic detailed review of programs. Still other deci-sions may be required at specific times because ofsunset provisions. It is important for managers toknow what this timetable is and to time the devel-opment of their representations so as to respond toit in a way that will be compatible with the organi-zation’s internal and externally imposed agendas.

A second consideration that may influencethe timing of management representations is thecontinuing relevance of effectiveness informationpreviously provided. Changing circumstances with-in or outside the organization may make recentrepresentations quickly outdated. Where thisapplies, more frequent development of manage-ment representations may be called for than whererelatively little change has occurred.

A third consideration is the benefits and costsinherent in producing the management representa-tions themselves. The cost to management of

preparing the information and analysis to supportcomprehensive representations on a major unit ofbusiness can be substantial. It will be important forboth managers and governing bodies to considerthese costs against the benefits that are expected toaccrue in terms of being able to make better deci-sions or to act in a more timely manner. In general,where the cost of generating them is high and theincremental benefit to the governing body is low,management representations entailing extensiveanalysis should be made less frequently.

N E E D F O R E X T E R N A L R E V I E W

As was seen in chapter 9, there are a variety ofways in which representations can be faultily con-structed. Many of these pitfalls can be detected bylay readers who have little or no knowledge of thetechnical details of the issues on which the state-ments are based. Simple logic and common sensecan often identify flaws in presentations. Thus,members of governing bodies, as reasonable,responsible people, have an innate competence toassess at least some aspects of the validity of repre-sentations made to them by management.

However, governing bodies cannot be expect-ed to assess the validity and completeness of theinformation underlying management’s representa-tions. Many public institutions are so large, it isunreasonable to expect the organization’s directorsto have detailed knowledge of operations.

THE ORGANIZATIONAL DILEMMA IS: AT THE TOP ONE WORKS

ON THE RIGHT PROBLEMS BUT WITH THE WRONG INFORMA-

TION, AND AT THE BOTTOM ONE WORKS WITH THE RIGHT

INFORMATION BUT ON THE WRONG PROBLEMS...214

A R N O L D J . M E L T S N E R


ELECTED POLITICIANS WHO HEAD GOVERNMENT DEPART-

MENTS ARE OFTEN THOUGHT OF AS CONDUCTING AN ORCHES-

TRA WITHOUT A SCORE. THE BEST THAT THEY CAN BE EXPECT-

ED TO DO IS TO HAVE AN IMPROMPTU REHEARSAL. COMPARED

WITH THE INFORMATION HELD BY THEIR POLITICAL MASTERS

THE BUREAUCRACY POSSESSES MUCH MORE TECHNICAL INFOR-

MATION ABOUT THE SUPPLY OF PUBLIC SERVICES. TO THE

EXTENT THAT INFORMATION IS POWER, THERE IS A NEED FOR

COUNTERVAILING POWER TO THAT OF THE BUREAUCRACY.215

P. M . J A C K S O N

It is only to be expected that managers willwant their representations to reflect well on theirown competence and performance. This naturalbias could, either intentionally or unintentionally,distort the information they provide. No matterhow good the relationship is between managementand the governing body, the latter may still har-bour some doubts about the completeness or theaccuracy of the representations brought forward.

How can this suspicion be allayed, this dis-tortion prevented, this potential bias neutralized?The answer is twofold. First, the process ofpreparing the representations should be disci-plined and transparent. Sloppiness and secrecy areideal cultures in which distortion can grow.Second, the representations should be exposed toexpert, outside, independent review. That reviewwill have two immediate benefits: it will reinforcemanagement’s commitment to a disciplinedprocess; and it will give both management andthe governing body confidence in the representa-tions brought forward.

The panel recognized the ability of externalreview to lend authority to management represen-tations and provide confidence in their accuracy,completeness, and fairness. Consequently, it wasrecommended that auditors provide opinions onmanagement representations brought forward togoverning bodies.

The issues involved in auditing managementrepresentations on effectiveness are discussed inPart III.


205 EFFECTIVENESS REPORTING AND AUDITING IN THE PUBLIC SECTOR: SUPPORTING RESEARCH STUDIES (OTTAWA: CCAF, 1987).206 W. DAVID MOYNAGH, REPORTING AND AUDITING EFFECTIVENESS: PUTTING THEORY INTO PRACTICE, CCAF APPLIED RESEARCH SERIES (OTTAWA: CCAF, 1993), 14.207 EFFECTIVENESS REPORTING, 16.208 MOYNAGH, THEORY INTO PRACTICE, 14, 17.209 EFFECTIVENESS REPORTING, 17.210 IBID., 21.211 THE DISCUSSION OF THE TWELVE ATTRIBUTES DRAWS HEAVILY ON THE PANEL’S REPORT AND ON MOYNAGH, THEORY INTO PRACTICE, EXHIBIT 10.212 EFFECTIVENESS REPORTING, 107-108.213 THE DISCUSSION OF GUIDELINES IS TAKEN FROM EFFECTIVENESS REPORTING, 108-114.214 ARNOLD J. MELTSNER, POLICY ANALYSTS IN THE BUREAUCRACY (LONDON: UNIVERSITY OF CALIFORNIA PRESS, 1976), 289.215 JACKSON, POLITICAL ECONOMY (SEE CHAP. 9, N. 34), 3.

C H A P T E R 1 1

PRELIMINARYCONSIDERATIONSIN REPORTINGEFFECTIVENESS

ALTHOUGH EACH INSTITUTION IS LIKELY, TO SOME EXTENT,

TO DISCOVER ITS OWN INSIGHTS AND LESSONS, THERE IS

MUCH THAT WE HAVE LEARNED THAT IS TRANSFERRABLE.216

T H E Q U E E N E L I Z A B E T H H O S P I T A L

Recently, a number of organizations havereported on their performance using the suggestedframework of twelve attributes of effectiveness dis-cussed in the previous chapter.217 This chapter con-tains some of the lessons learned from these experi-ences in implementing the framework.

This summary discussion focuses on fivebroad issues that face any organization seriouslyconsidering using the framework in reportingeffectiveness. They are:

• determining whether an organization and itsmanagement are ready to take on the chal-lenges of the effectiveness framework;

• setting realistic expectations for an organiza-tion’s first effort at effectiveness reporting;

• deciding where to focus initial effectivenessreporting efforts;

• deciding who ought to participate in theprocess and what roles they need to play tofoster successful implementation; and

• managing the overall implementation processin terms of both general strategy formulationand people-dynamics.

This chapter reflects the thinking and adviceof the individuals who were directly involved insuch projects, including the senior managers whogave personal leadership to these initiatives, andthe wide variety of practitioners who supported theinterests of management or the governing bodyduring the process.

O V E R A L L P E R S P E C T I V E

Experience in applying the effectivenessframework demonstrates that it entails much morethan senior management just sitting down to writea performance report. Indeed, it requires that man-agement think hard about the mandate, objectives,and priorities of the program or line of business onwhich the organization is focusing, to build con-sensus on these matters, and to consider how exist-ing processes are related, or ought to be related. Ina sense, therefore, the notion of reporting almostbecomes a by-product of much broader strategicconsiderations and processes.

WE LEARNED THAT APPLYING THE FRAMEWORK CAN BE COM-

PLICATED, AND THAT FULL AND FRANK DISCLOSURE MAY NOT

COME EASILY. WE MADE MISTAKES, WE LEARNED SOME

LESSONS, AND WE ALSO DID MANY THINGS WELL. FROM OUR

PERSPECTIVE, WE CONFIRMED THE VALIDITY AND VALUE OF

THE FRAMEWORK AND ARE CONVINCED THAT WITH

INCREASED USAGE, LARGER DIVIDENDS WILL BE REALIZED.218

G O V E R N M E N T O F M A N I T O B A

The successful implementation of effective-ness reporting demands a constant series of trade-offs from the beginning of the process (when adecision is made about where to focus and why),until the end (when management reports theirrepresentations to their governing body). Thesetrade-offs are intrinsic to the process and involve


fundamental choices that will determine theamount of added value that will result from theprocess and product. For the most part, thesetrade-offs cannot be made by anyone but seniormanagement.

Experience also indicates that, while centralto the success of effectiveness reporting, seniormanagement cannot do it on their own.Ultimately, a partnership of interests needs to beestablished among senior management, their gov-erning body, and the practitioners who serve bothgroups. This is generally a hard-won alliance, as theparties often have to look beyond the traditionalbeliefs, attitudes, and behaviours that have charac-terized their past relationships. The perspectivesand decisions of each party affect the considera-tions and choices of the others.

LOOKING BACK ON THE PROCESS, MANAGERS REALIZE THAT

THEY HAVE HAD THE BENEFIT OF AN EXTERNAL REVIEW OF

THEIR KEY SYSTEMS AND PRACTICES, BASED ON CRITERIA THAT

THEY THEMSELVES DEVISED AND HAD APPROVED… THEIR

OWN PREPARATION FOR THE EXAMINATION UNCOVERED A

NUMBER OF DEFICIENCIES AND ANOMALIES THAT THEY WERE

ABLE TO CORRECT.219

C A N A D A M O R T G A G E A N D H O U S I N G

C O R P O R A T I O N

Appropriate mechanisms for communicationand consultation among these parties must beestablished if the project is to succeed. Equallyimportant is the influence of the corporate culturethat shapes these relationships. Ideally, it encour-ages openness, invites challenge, permits flexibilityand learning, and is based on mutual respect. Atthe least, the corporate culture must not be hostileto these concepts.

D E T E R M I N I N G I F T H E

O R G A N I Z A T I O N I S R E A D Y T O

P R O C E E D

IN SELECTING AN ORGANIZATION… IT SHOULD BE [ONE]

WHOSE SENIOR MANAGEMENT ALREADY USE OR HAVE OPEN

MINDS ABOUT INNOVATIVE PRACTICES IN ACCOUNTING FOR

THEIR ACTIONS… 220

G U Y B R E T O N

Important factors in determining whether ornot an organization is ready to take on the chal-lenges of this process include:

• the predisposition of the organization, and in particular its management, to be self-challenging, forward-looking, and innovativein improving its accountability and perfor-mance;

• the maturity of the relationship betweenmanagement and the governing body; and

• the capacity of the organization’s managementpractices and processes to respond to theinformation requirements of the effectivenessframework.

Where there is a motivation to improve per-formance, and a relationship of mutual respect thatencourages management and governance to dealwith this issue in a straightforward and evenhand-ed manner, application of the effectiveness frame-work has its greatest benefit.

Organizations most likely to benefit substan-tially, or at least have a greater potential for usingthe framework successfully, are those that:

• have already expressed a desire to improveeffectiveness reporting to the governing bodyand/or the public;

• are forward-looking and tend to pursue newapproaches designed to enhance the effective-

P A R T I I . C H A P T E R 1 1 . P R E L I M I N A R Y C O N S I D E R A T I O N S I N R E P O R T I N G E F F E C T I V E N E S S 1 5 1

ness of their operations;• are involved in or increasingly thinking about

long-term planning and are concerned withthe linkages between this planning and theinformation needed to support it;

• have a history of taking initiatives to enhancegovernance and management processes and tofoster the accountability relationship betweenthe two parties; and

• are in or are about to begin the process oflooking at enhancements or alternativeapproaches to improve the quality of man-agement, governance, planning, and/oroperations.

Where an organization is in a period of crisisor instability, however, or where the relationshipbetween management and their governing body isstrained, it is improbable that an attempt to usethe effectiveness framework will be successful. Inthe first case, the urgency or uncertainty associatedwith the organization’s situation will likely keepmanagement from devoting the necessary attentionand time to effectiveness reporting; it just won’thave priority. In the second case, introducing effec-tiveness reporting could simply make a difficultproblem worse by forcing the spotlight on issuesand information about which neither party is pre-pared to be completely open or fair with the other.In both situations, the result is likely to be aprocess that is soon neglected, eventually resented,and finally abandoned. The approach organizationsmust take, experience shows, is to resolve thesefundamental problems before trying to implementthe effectiveness reporting framework.

The current state of an organization’s infor-mation systems is another key factor in assessing itsreadiness to undertake effectiveness reporting.Those most experienced in working with theframework caution against selecting or encouragingan organization with no or minimal demonstrated

information processing capacity to take on thechallenge of an effectiveness reporting project. It ischallenging enough, they argue, for organizationswith comprehensive and reliable information sys-tems to implement the framework; without suchsystems, it would be extremely difficult to producea report in which management could have confi-dence. While good information systems are indeedhelpful, perfection in this regard is not a prerequi-site to undertaking an effectiveness reportingproject.

The shift from the more traditional focus oninputs and activities, to a focus on broader notionsof effectiveness is not without significant challenge.The challenge is not just a technical one relatingpossibly to the need to adjust internal informationsystems and practices. It also involves a fundamen-tal change in mind-set both on the part of thosewho are preparing this information and those whoare receiving it. Where the parties’ thinking is inharmony, the inclination of one to move in thisdirection is reinforced by the appetite of the other.But this may not always be so. For example,requirements and demands that one party feels toshift to a broader orientation may need to be bal-anced against the expectations of others for a con-tinued focus on inputs. Making this shift—be ittechnical or not—is likely to be a gradual process.

It is important that an organization investsthe necessary time and effort at the front end ofthe process to carefully consider all the critical suc-cess factors discussed above, before a final go/no-godecision is made respecting implementation. Asone practitioner put it, it only makes sense to“begin at the beginning,” making sure that every-one who will be involved in the process has anunderstanding of the effectiveness framework andthat they are committed to the value that thisprocess can bring to the accountability relationshipbetween them.


S E T T I N G R E A L I S T I C

E X P E C T A T I O N S

[W]E ANTICIPATE THAT OTHERS WHO PERSUADE SENIOR MAN-

AGEMENT AND GOVERNING BODIES TO ADOPT THE ACCOUNT-

ABILITY FRAMEWORK AND MANAGEMENT REPRESENTATIONS

MAY BE DISCOURAGED BY THE SCARCITY OF HARD DATA TO

SUPPORT THE DEVELOPMENT OF REPRESENTATIONS. DON’T BE.

THE FIRST REPRESENTATIONS MAY NOT BE PERFECT, BUT THEY

WILL BECOME A STRONG ARGUMENT IN SUPPORT OF BETTER

INFORMATION SYSTEMS, AND ULTIMATELY BETTER MANAGE-

MENT PRACTICES.221

C I T Y O F G L O U C E S T E R

Another key to success in effectiveness report-ing is setting realistic expectations for the organiza-tion’s first efforts. These expectations must notonly be reasonable, they must also be sharedamong management and with the governing body.And once set, they need to be consistently rein-forced throughout the exercise, since the intensityof the process can often cause those closelyinvolved to lose perspective or to attempt to forcethe reporting product beyond its practical limits.

Thinking that an initial effectiveness report-ing project has to be or will be perfect places anunrealistic burden on the process, which willinevitably lead to disappointment or, worse, to aproduct that ignores or subverts important infor-mation not in accord with this predetermined out-come.

At the same time, all parties, particularlymanagement, have to recognize that the process ofdeveloping management representations has to be adisciplined one, based on assessment criteria thathave been discussed and agreed to as reasonable bythe parties involved. And it needs to be backed upby management reporting that provides a fair andcomplete picture of the organization’s effectiveness.

Most organizations that have worked with theeffectiveness framework believe that the right bal-ance between rigour and flexibility will be achievedonly over time as management and the governingbody gain further experience in working with theeffectiveness framework and, through this, developa better appreciation of each other’s needs andcapacities.

Most organizations applying the frameworkdeliberately decided to base their initial manage-ment representations on existing and readily avail-able information and data. In doing so, they knewthat the resulting performance story would beincomplete, perhaps even in important respects.They calculated that they owed it to themselves toexploit their existing information and systems totheir fullest potential—and to assess how theystack up to requirements—before investing furtherresources in developing performance information.Even acknowledging the gaps in information thatthis strategy left, organizations consistently reportthat using the framework as a basis for their initialrepresentations has provided governing bodies witha more integrated and strategic picture of perfor-mance than they have received previously.Moreover, the initial application of the frameworkhas helped these organizations assess their informa-tion base and needs and, from this, determinewhat actions are required to support more rigorousand complete management reporting in the future.

D E C I D I N G W H E R E T O F O C U S

I N I T I A L E F F O R T S

IF YOU ARE PLANNING TO DO THIS, CHOOSE A SIGNIFICANT

AREA OF YOUR BUSINESS TO ASSURE RETURN ON THAT LEVEL

OF INVESTMENT.222

M A R Y F E R G U S O N - P A R É


The decision as to where to focus an initialapplication of the effectiveness framework shouldbe taken carefully. In the case of the QueenElizabeth Hospital in Toronto, senior managementdevoted an entire executive committee meeting tothis issue.

Effectiveness reporting projects undertaken todate show a wide variety in practice—ranging froma focus on the entire organization, to a major lineof business, to a small program, to an administra-tive function.

On balance, the lesson that can be takenfrom these experiences is that application of theeffectiveness framework is most cost-effective whenfocused at a reasonably high level, such as a majorline of business or program that is central to theorganization’s mission. This is the level at whichsenior management makes strategic choices, and itis the level most likely to engage the interest andneeds of the governing body.

The decision to focus at a high level can bringcertain difficulties. For example, the institution’sprograms and organization structures may cutacross each other. Similarly, the traditional perfor-mance measurement and reporting systems withinthe organization may not easily lend themselves tothe desired program focus. These complications,however, have generally not deterred organizationsfrom the decision that a program/line-of-businessfocus is the most appropriate one. Nonetheless,these difficulties do have a strong influence onimplementation strategy, as well as on expectationsabout what can reasonably be accomplished in ini-tial applications of the effectiveness framework.

D E C I D I N G W H O O U G H T T O

P A R T I C I P A T E

Of necessity, an effectiveness reporting initiativeinvolves a number of key participants, and the suc-cess of the initiative is contingent on them recogniz-

ing and being prepared to play specific roles. Thisincludes senior management, the governing body,and the practitioners who support both these groups.

S E N I O R M A N AG E M E N T

MUCH OF THE SUCCESS OF THE EXERCISE IS ATTRIBUTED TO

THE INTEREST AND CONTINUING INVOLVEMENT OF SENIOR

MANAGEMENT THROUGHOUT.223



Time and again, organizations that haveworked with the effectiveness framework point tothe leadership and involvement of top manage-ment, particularly at the CEO level, as the singlemost critical success factor. When managementundertake the process enthusiastically and, throughtheir own active involvement, demonstrate theimportance and commitment they attach to thegoal of improved accountability, the governingbody understands that this exercise deserves theirattention. The converse is equally true. If seniormanagement treat the effectiveness reportingprocess like a spectator sport, encouraging orallowing the primary responsibility for developingmanagement representations to be pushed to lowerlevels in the organization, it is unlikely that thegoverning body will perceive the product as rele-vant to their strategic interests.

THE MOST POSITIVE ASPECT OF THIS [APPLIED] RESEARCH

PROJECT WAS THE EXTENT TO WHICH SENIOR MANAGEMENT

ASSUMED OWNERSHIP OF THE RESULTS… SENIOR MANAGE-

MENT SHOULD BE COMMITTED TO THE VALUE OF THE

PROCESS FROM THE BEGINNING.224

C I T Y O F G L O U C E S T E R


The degree and nature of senior manage-ment’s involvement in the process requires carefulthought. On the one hand, there are judgmentsand trade-offs involved that only management canmake. In addition, senior management need tohave a very good sense of the information they areusing in developing their management representa-tions. On the other hand, the identification andamassing of basic information from the wide vari-ety of sources available to the organization can bevery time-consuming. These tasks are done morecost-effectively by technical support staff, such asprogram evaluators or policy analysts.

In particular, CEO leadership and involve-ment need to be strategically focused. The natureof such leadership and involvement, however, islikely to vary with factors like the organization’ssize, management culture, and policy climate. Forinstance, in a large organization where senior cor-porate support is likely to be available, it is reason-able to expect that the CEO will not be directlyinvolved in the process of amassing and analyzingdata and information and the physical writing ofthe report. In a smaller organization, more exten-sive involvement on the part of the CEO may bepossible or necessary.

Regardless of the circumstances of the organi-zation, however, there are four fundamental rolesfor the CEO to play:

• negotiating a shared understanding with thegoverning body about the priorities, issues,and information that need to be addressed inthe performance report;

• creating a climate conducive to such report-ing, up through the organization and, asapplicable, from the organization to variousstakeholder groups;

• ensuring that the reporting product receivesan effective challenge in terms of its fairnessand completeness and the central messages itseeks to convey; and

• claiming ownership of the performancereport.

Experience shows that arrangements for somemanner of management support should be put inplace at an early stage in the process. This helpssenior management keep focused on tasks wheretheir involvement has the greatest value added—that is, deciding what information needs to bereported and why, and how this informationshould be interpreted and conveyed to the govern-ing body. Making these choices can be facilitatedby input from advisers, but the decisions them-selves should not be delegated to them.

G OV E R N I N G B O D Y

THE PARTICIPATION AND SUPPORT OF OUR BOARD OF

GOVERNORS WAS AN IMPORTANT COMPONENT OF THIS EXER-

CISE… AT THE SAME TIME, IT IS UNREALISTIC TO EXPECT

THAT A BOARD WILL ENTER A NEW PROCESS WITH A PREDE-

TERMINED, FULLY INTEGRATED ARTICULATION OF THEIR

EXPECTATIONS AND INFORMATION NEEDS… WE CONCLUDE

THAT IT IS UP TO THE HOSPITAL MANAGEMENT TO BRING

OPTIONS AND ADVICE TO THE BOARD ABOUT THE INFORMA-

TION THEY OUGHT TO RECEIVE. THE BOARD CAN THEN REACT

TO THIS AND MAKE REFINEMENTS AS IT JUDGES NECESSARY.225


The governing body must be involved in theeffectiveness reporting process. This is necessary inorder to gain their perspectives on the matters thatshould be addressed in the report, and to fostertheir willingness and preparedness to deal withmanagement reporting in an evenhanded way. Forreasons mentioned in the quotation above, this is aconsultation process that management needs to ini-tiate and nurture.


Most effectiveness projects entail some man-ner of consultation between management and gov-ernance at one or more stages of the exercise. Ineach case, the governing body played a key role ingiving legitimacy to, or setting expectations for, theeffectiveness reporting initiative.

It is wise to engage the governing body in theprocess reasonably early, once management havehad the opportunity to orient themselves to theeffectiveness framework and have developed a gen-eral sense of the kind of representations they willbe able to produce. Not seeking such engagementincreases the risk that management will produce areport that the governing body will not under-stand, will not know how to use, will not find ger-mane to their interests, and thus will not value.The essential purpose of the effectiveness reportingframework would, therefore, be unfulfilled.

Experience shows that the best approach is formanagement to establish ongoing consultation witha committee of the governing body rather than to tryto engage the interest and participation of the entiregovernance entity at every key stage in the process.This committee may be one already set up for such apurpose (for example, the board’s audit committee)or one that management would encourage the gov-erning body to establish on an ad hoc basis.

A U D I T O R

THE SPECIAL EXAMINER [AUDITOR] WAS INVOLVED IN THE

PROCESS OF DEVELOPING REPRESENTATIONS RIGHT FROM THE

START. BY ENSURING THAT THE EXAMINER WAS PRIVY TO ALL

THE BACKGROUND THINKING AND WORK, AND GIVING HIM

THE OPPORTUNITY TO COMMENT ON THAT WORK AS IT PRO-

GRESSED, A USEFUL SYNERGISM GREW AND PROBLEMS OF VARY-

ING VIEWPOINTS WERE AVOIDED.226



In the final analysis, the involvement of auditis fundamental to the successful establishment ofcredible effectiveness reporting by management totheir governing body. The role of the auditor inthis context is to provide assurance to the govern-ing body regarding the fairness and completenessof the effectiveness information being reported bymanagement. The presence of the auditor and theindependent assurance role he or she plays rein-force the message that it is important and expectedthat management report the complete performancestory.

At the time of writing, there are only a limit-ed number of examples where there has been anaudit attestation component to an effectivenessreporting initiative. The one prominent lessonlearned from these few cases, and from other ongo-ing effectiveness reporting projects where there isan audit component, relates to the value addedthat all parties see in having the auditor involved inthe process from the beginning, and not just fromthe point where management is ready to table theirfinal representations with the governing body.

From the auditor’s perspective, for instance,having exposure to the thinking of managementand the governing body, and to the backgroundwork that has been done, helps the auditor decidehow to approach the examination work. At thesame time, by taking advantage of early opportuni-ties to explain to management—and perhaps to thegoverning body—the type of perspectives andrange of considerations that will be applied in thecourse of this audit work, the auditor helps foster adisciplined approach to the process and a rigorousreporting product.

There is also a lesson to be learned fromthose projects where the decision was made not tohave an audit attest component for the organiza-tion’s first attempt at effectiveness reporting. Thesedecisions were not motivated by a desire to evadeaudit scrutiny as a matter of principle, but by a


concern about the wisdom of trying to wrestle withtoo many unknown factors at the same time.Possible sources of uncertainty could include unfa-miliarity in working with the effectiveness frame-work itself, management feeling that they need toobtain a better understanding of and confidence intheir capacity to produce substantive representa-tions, and entering uncharted territory in terms ofthe focus and level of discussion between manage-ment and the governing body.

In such circumstances, an organization maydecide to use the first application of the effective-ness framework as a diagnostic of their informationsystems, their capacity to make representations oneffectiveness, and the value that such informationhas in supporting the accountability relationshipbetween management and their governing body.With a better understanding of these matters, andwith confidence in the capacity to move forwardwith these ideas, future effectiveness reporting ini-tiatives could then incorporate a formal auditcomponent.

FAC I L I TAT O R

APPLICATION IS NOT EASY. THE MANAGERS OF THESE ORGANI-

ZATIONS NEED HELP. THE UP-FRONT INVESTMENT TO UNDER-

STAND THE FRAMEWORK IS CONSIDERABLE, THEREFORE, IT IS

PREFERABLE TO DRAW ON RESOURCES ALREADY FAMILIAR

WITH THE CONCEPT.227

J . P E T E R G R E G O R Y

Most organizations have sought out someoutside help to advise and assist them in the initialapplication of the effectiveness framework. Thesefacilitators are senior professionals in private sectoraccounting/consulting firms, legislative auditoffices, and so on, who have an in-depth knowl-edge of the effectiveness framework and who, over

time, have developed a substantial base of experi-ence in working with these ideas in a variety oforganizational contexts. They are also knowledge-able about the organization, and comfortable andskilled in dealing with executive decision makers ina workshop-leading, problem-solving situation.

Generally speaking, organizations have soughtout external facilitation for one or more of the fol-lowing reasons:

• they recognize that working with the effec-tiveness framework is likely to require man-agement and their technical support staff tothink and operate outside traditional patterns,and that having someone with an outside per-spective to assist them in these tasks will helpthe transition occur more smoothly;

• they do not have a ready pool of resourceswithin the organization that can be calledupon to provide management with the ongo-ing support required; and

• they want to use the first application of theeffectiveness framework to train staff so thatthey will be able to support management infuture reporting initiatives.

Regardless of whether it comes from outsideor inside the institution, those organizations thathave used the effectiveness framework report thatsuch facilitation is an essential ingredient to suc-cess. Management cannot do it alone.

Organizations report that the facilitator hasplayed a variety of key roles:

• introducing the effectiveness framework tosenior management, helping managementthink through the implications of applyingthe framework, and, if the decision is to pro-ceed, designing an appropriate implementa-tion strategy;

• organizing the necessary background work andmaterial to facilitate management’s deliberationson various matters throughout the process;


• helping management keep focused on thetasks that need to be accomplished and chal-lenging management at various points in theprocess;

• providing expert advice to management vis-à-vis the development of their representationson effectiveness;

• helping management identify points ofdiminishing returns from their efforts andencouraging them to move on to other issues;and

• bringing closure to the effectiveness reportingprocess.

R E P O R T W R I T E R

Although senior management develops thesubstance of the management representations, thefinal report cannot be written by committee.Somebody needs to be designated as responsiblefor writing the report. This is not a trivial task, as abalance needs to be struck in being succinct yetcomplete, and in achieving a common presentationformat yet preserving the distinctive character ofindividual attributes. Whoever is assigned thisresponsibility needs to be a part of the processfrom the beginning, not just brought in at the end.

M A N A G I N G T H E O V E R A L L

P R O C E S S

PROPER MANAGEMENT OF THE FRAMEWORK IS REQUIRED TO

ENSURE THAT ITS APPLICATION NEITHER BECOMES ONEROUS

DUE TO ITS COMPREHENSIVE NATURE, NOR PROMPTS THE

DRAWING OF UNWARRANTED CAUSAL CONCLUSIONS ABOUT

THE EFFECTS AND EFFECTIVENESS OF PARTICULAR SOCIAL

SERVICES.228

R O X Y F R E E D M A N

The various applications of the effectivenessframework have yielded a number of importantlessons regarding the formulation and execution ofproject management strategy. Many of these havealready been discussed and are integrated below.

First, take a measured approach to the wholeprocess. Before a go/no-go decision is maderespecting implementation, it is important thatmanagement invest the necessary time to informthemselves about the effectiveness framework andto think through the implications of using it. Thisis best accomplished through a process that pro-gressively provides management with informationresponding to their interests, and that allows themto give it detailed consideration. Doing this takestime, but it is time that organizations report waswell spent, or that, in retrospect, they wished hadbeen invested.

Second, recognize that an array of interestsand expectations will be brought to the table bythe various participants. These are likely to varybetween—even among—the governing body, man-agement, and those practitioners who support theinterests of both parties. Moreover, the process islikely to open up entirely new areas of dialogueamong these participants. The implementationprocess has to be flexible enough to encouragediscussion on these different perspectives andexpectations, and disciplined enough to bringthese matters to a resolution that all parties areprepared to support.

Third, establish specific consultation/commu-nication networks to keep all key stakeholdersinformed and to encourage dialogue at strategicpoints throughout the process. The most impor-tant network in this regard is the one that linksmanagement and their governing body. From man-agement’s perspective, for example, they need toknow what the governing body wants out of theprocess, that they are supportive of the generalprinciples that underlie the process, and that the


governing body is ready and able to deal with theinformation that they will eventually receive.

Fourth, impose a reasonable timetable on theprocess. Rome wasn’t built in a day and neitherwill effectiveness reporting.

Organizations applying the effectivenessframework often comment on the need to makesure that the process is well paced to maintain thenecessary momentum. At the same time, it shouldprovide sufficient opportunities for reflection andresearch so that management can come back to thetable with new information and fresh perspectives.Although many factors might dictate otherwise,once a go-decision has been made, a useful rule ofthumb would be for management to allow them-selves about nine months to complete the processof developing their representations and tablingthem with the governing body.

The following chapter offers more detail onthe type of step-by-step strategy that could be usedand on the considerations that attend each stage ofthe process.

D E V E L O P I N G M A N A G E M E N T


THE PROCESS CAUSED EVERYONE TO LOOK BEYOND THEIR

OWN AREAS OF RESPONSIBILITY AND FOCUS ON THE OPERA-

TIONS OF THE CHRONIC CARE SERVICE IN A HOLISTIC WAY…

A VERY IMPORTANT LESSON LEARNED WAS THE NEED TO HAVE

A CLEAR SENSE OF THE PROGRAM TO BE EXAMINED BEFORE

BEGINNING THE PROCESS… [WRITING OUR REPRESENTA-

TIONS] WAS NOT A LINEAR PROCESS… PROBING AND CHAL-

LENGING THE LINKS AMONG THE ATTRIBUTES IS WHAT MAKES

THE EXERCISE COME ALIVE… THE INTEGRATION OF INDIVID-

UAL MANAGEMENT REPRESENTATIONS INTO A SINGLE REPORT

IS A SIGNIFICANT TASK… THIS PROCESS IS ONE IN WHICH ALL

SENIOR MANAGEMENT NEEDS TO BE INVOLVED.229


Developing management representations is acomplex process that involves a high degree ofmultidisciplinary thinking and takes a lot of effort.

There is a high level of interconnectionamong some attributes (for example, relevance andappropriateness); sometimes the dilemma is whereto deal with information that is relevant to eachattribute. At the same time, based on the applica-tions to date, there appears to be no compellingargument that the attributes have to be dealt within any specific order.

Keeping the focus on the main messages canbe difficult, particularly when trying to integrate awide variety of information that, in turn, has beengenerated by a wide variety of sources. To a certainextent, this tension is inevitable.

Experience shows that there is no single rightapproach to applying the framework in an organi-zation, nor should anyone think that the suggestedframework is the only way in which to structurethe performance information to be reported.Indeed, CCAF’s applied research projects and otherapplications of the framework have demonstrated avariety of ways in which to structure the perfor-mance report. In some cases, organizations havefound the twelve-attribute framework to be a rea-sonable structure for the performance report.Elsewhere, organizations have grouped the attribut-es in a way they think best projects the nature oftheir business or in a way intended to show keyinterrelationships among subsets of the attributesand relevant performance information. The frame-work offers sufficient flexibility to be adapted todifferent organizational circumstances.

As explained above, it is not a question ofengaging in slavish, lockstep reporting on each ofthe attributes regardless of the circumstances;rather, the framework should be used as a tool thatprovides a set of principles in stretching and orga-nizing one’s thinking about the performance issuesand information that really matter, and in applying


the completeness test to the reporting that emergesfrom this process.

D E T E R M I N I N G W H E R E A N D

W H E N T O R E P O R T


The panel was convinced of the wisdom ofusing existing vehicles wherever possible for report-ing on effectiveness, rather than creating yet anoth-er reporting mechanism.

At the federal level, annual reports, thePublic Accounts and Part III DepartmentalExpenditure Plans appear to have good poten-tial as reporting vehicles for management rep-resentations on effectiveness. Typically, thesedocuments have an introductory sectionwhere the organization as a whole and thechallenges it faces are discussed, and subse-quent sections dealing with the several majorprograms or lines of business. The formermechanisms seem suitable for managementrepresentations dealing with the effectivenessof the whole organization, while representa-tions that are specific to individual programscould fit well in the later sections… In otherjurisdictions there are similar documents thatdeserve consideration in choosing reportingvehicles.230

As noted above, most public sector organiza-tions already have certain reporting requirementsplaced upon them. This usually includes someform of an annual report. It might include morefrequent reporting to a governing body, focusingon performance in relation to certain critical suc-cess factors for the organization. Frequently, suchreporting is supported by some form of ongoingprogram monitoring. It might also include cyclicalreports provided to a governing body, which focuson key program areas or lines of business. Usually,

these are more in-depth reports that tend to besupported by periodic study-based review processessuch as program evaluation.

Implementation of the framework is notintended to push aside existing reporting processes,but to build on them. Some organizations havefound that implementing the framework has givena broader, more integrated focus to their overallgovernance and management-information strategyand has helped identify where adjustments to indi-vidual elements might be made.

Ultimately, in what form and how often anorganization can most usefully report its manage-ment representations on performance or effective-ness is something each organization can probablyjudge for itself. Some organizations that haveworked with the framework have used their annualreports to the governing body as the vehicle bywhich to convey management representations.231

Others have developed a special report to the gov-erning body focusing on the results of a specificmanagement representations project.232 For stillothers, the intention is to operate within the con-text of a more global strategy for reporting man-agement representations to the governing bodyover an agreed period of time, each time focusingon (a) different area(s) of their business.233

A wide number of interrelated factors affecthow and when to report management representa-tions. As discussed earlier, these include: the deci-sion-making timetable of the governing body, thecontinuing relevance of previous representations, andthe cost-benefit of preparing new representations.

Some of the information that forms part ofmanagement’s representations on overall perfor-mance will be useful to shorter-term decision mak-ing (reporting on costs and productivity, achieve-ment of shorter-term operational results, someaspects of acceptance, financial results, and so on).And, where it is useful, advantage ought to betaken of existing reporting vehicles to bring this


information forward. In doing so, however, it maybe important to explain to the reader that thisshorter-term information, while useful, does notconvey a complete picture of overall performance.In all likelihood, the full range of information thatemerges from applying the twelve-attribute frame-work will be most powerful when it is used in con-nection with longer-term planning, policy, pro-gram, and resource decision making.

In the final analysis, reporting on perfor-mance, in a complete way and at a high level, willneed its own cycle, possibly detached from annualappropriations-related reporting but, as notedabove, intersecting with it where sensibly it should.What that cycle is, and the extent to which it isintegrated with other reporting processes and vehi-cles, are matters that should be discussed betweenmanagement and its governing body.

THE PHILOSOPHY WHICH UNDERLIES THE MANAGEMENT REP-

RESENTATIONS APPROACH… IS AN IMPORTANT ONE AND BEARS

REPEATING. IT PLACES UPON MANAGEMENT THE RESPONSIBILI-

TY, AND HENCE THE ACCOUNTABILITY, FOR ITS ACTIONS. IT

IMPLIES A LEVEL OF TRUST BETWEEN MANAGEMENT AND THE

GOVERNING BODIES. AND ULTIMATELY, IT BOILS DOWN TO A

SET OF VALUES THAT REQUIRES MANAGEMENT TO INTERNALIZE

A SENSE OF ACCOUNTABILITY. WITHOUT THE INTEGRATION OF

THIS VALUE SYSTEM, ALL THE SYSTEMS AND PRACTICES IN THE

WORLD WILL NOT BE SUFFICIENT TO PROTECT THE INVEST-

MENT OF PUBLIC FUNDS.234

G E O R G E A N D E R S O N


216 REPORTING ON EFFECTIVENESS—THE EXPERIENCE OF THE QUEEN ELIZABETH HOSPITAL, CCAF APPLIED RESEARCH SERIES (OTTAWA: CCAF, 1991) [HEREINAFTER QEH], 55.217 THE RESULTS OF SEVERAL OF THESE PROJECTS HAVE BEEN PUBLISHED IN CCAF’S APPLIED RESEARCH SERIES.218 REPORTING ON EFFECTIVENESS—THE EXPERIENCE OF THE GOVERNMENT OF MANITOBA, CCAF APPLIED RESEARCH SERIES (OTTAWA: CCAF, 1991) [HEREINAFTER MANITOBA], 10.219 REPORTING AND AUDITING EFFECTIVENESS—THE EXPERIENCE OF CANADA MORTGAGE AND HOUSING CORPORATION, CCAF APPLIED RESEARCH SERIES (OTTAWA: CCAF, 1990) [HEREINAFTER

CMHC], 13.220 FROM A PRESENTATION BY GUY BRETON, AUDITOR GENERAL OF QUEBEC, TO THE 1991 ANNUAL CONFERENCE OF LEGISLATIVE AUDITORS OF CANADA.221 REPORTING AND AUDITING EFFECTIVENESS—THE EXPERIENCE OF THE CITY OF GLOUCESTER, CCAF APPLIED RESEARCH SERIES (OTTAWA: CCAF, 1992) [HEREINAFTER GLOUCESTER], 23.222 FROM ADDRESS BY MARY FERGUSON-PARÉ, VICE-PRESIDENT OF NURSING, THE QUEEN ELIZABETH HOSPITAL OF TORONTO, TO THE 1991 CCAF ANNUAL CONFERENCE.223 CMHC, 12.224 GLOUCESTER, 21.225 QEH, P. 57226 CMHC, 12.227 FROM ADDRESS OF J. PETER GREGORY, EXECUTIVE PRINCIPAL, OFFICE OF THE AUDITOR GENERAL OF BRITISH COLUMBIA, TO THE 1992 CCAF ANNUAL CONFERENCE.228 ROXY FREEDMAN, DEPUTY MINISTER OF FAMILY SERVICES, GOVERNMENT OF MANITOBA, AS QUOTED IN MANITOBA, 47.229 QEH, 59-61.230 EFFECTIVENESS REPORTING, 114.231 SEE COMMISSION DES NORMES DU TRAVAIL [QUÉBEC], RAPPORT ANNUEL 1992-1993 (SEPTEMBRE 1993) AND RAPPORT ANNUEL 1993-1994 (SEPTEMBRE 1994).232 SEE ALBERTA VOCATIONAL COLLEGE—CALGARY, EVALUATION OF EFFECTIVENESS AND EFFICIENCY (CALGARY: ALBERTA VOCATIONAL COLLEGE, MARCH 1994).233 QEH, 61.234 GEORGE ANDERSON, THEN-PRESIDENT OF CMHC, AS QUOTED IN CMHC, II.

C H A P T E R 1 2

IMPLEMENTINGANEFFECTIVENESSREPORTINGPROJECT

This chapter describes a step-by-step processfor undertaking an effectiveness reporting project,steps that would be taken by an organizationembarking on its first application of the effective-ness framework.

The suggested process is a composite ofactual approaches taken by various organizationsin applying the framework. It also incorporatesthe views of those most closely involved withthese initiatives—in terms of what worked, whatdid not work, and what might have been donebetter.

These suggestions should be viewed as gen-eral guidance, not as a template or series of pre-scriptions to be adhered to regardless of the orga-nization’s unique circumstances. Alternativestrategies and steps may be—and in some casesshould be—considered. This chapter identifiessome of these alternatives and discusses the con-siderations that may attend them.

O V E R V I E W O F T H E P R O C E S S

The suggested process involves seven more orless distinct phases and fourteen more or less dis-crete steps, from the first time the subject of effec-

tiveness reporting is introduced, to when seniormanagement’s final report is tabled with the gov-erning body along with the auditor’s opinion, ifapplicable.

Usually, the initial introduction of effective-ness reporting is made to the CEO and seniormanagement, although the process could conceiv-ably start with a presentation to the governingbody. For our purposes, however, we assume theformer scenario.

Experience suggests that the core work ofthe first project (that is, starting after the initialengagement steps described below) can take aboutseven to ten calendar months. An audit compo-nent adds an additional period to cover the auditplanning, examination, and reporting stages,although much of this work might be conductedconcurrently.

After an organization goes through theprocess once, some steps should be unnecessary insubsequent effectiveness reporting exercises and,in general, the process should operate more effi-ciently and expeditiously. This was certainly theexpectation reported by every organization partic-ipating in the CCAF’s applied research program.For instance, Phase I, which provides the organi-zation with an introduction and orientation tothe effectiveness reporting framework, should nothave to be repeated in subsequent exercises. Inaddition, there may be an opportunity to stream-line the process further by dropping, combining,or shortening certain steps. An organizationshould be able to learn from the first applicationof the process and, based on this, achieve a reduc-tion in the overall investment and time requiredfor future reporting exercises.

The following is an overview of the majorphases of an initial effectiveness reporting project:


I Initial engagement of the CEO/senior man-agement

II Deciding where to focus and how to proceedIII Preparing for and beginning implementationIV Refining the initial management representa-

tionsV Writing the management representations

reportVI Tabling the report with the governing bodyVII Audit of management’s representations on

effectiveness

The following pages take the reader througheach of the above suggested phases. Each step inthe process is described and, in several cases, fur-ther information is provided on the considerations,strategies, and tools that may be useful in theapplication of the effectiveness reporting frame-work in a specific organization.

P H A S E I — I N I T I A L

E N G A G E M E N T O F T H E

C E O / S E N I O R M A N A G E M E N T

Step 1: CEO determines whether there isany potential interest in the effectivenessframework

Step 2: Executive presentation to seniormanagement

Step 3: Organizational workshop forsenior management

Step 1: The CEO determines whether theideas and issues on which the effectiveness frame-work focuses are sufficiently important to theorganization for senior management to explore inmore detail.

The following Accountability Checkup isoffered to assist the CEO (or another senior man-ager or member of the governing body) in makingthis determination. The CEO would probablyreceive the Accountability Checkup from someonewho is knowledgeable about the effectivenessframework and who thinks that using it wouldclarify and improve the accountability relationshipbetween management and the organization’s gov-erning body. That person might be the one whoplays the role of facilitator if the decision is toproceed.

The Accountability Checkup poses three keyquestions:

• What questions and issues about effectivenessare important to you and those to whomyour organization is accountable?

• How effective is your organization in relationto these questions and issues, and how do youknow?

• Can the effectiveness framework be useful toyou and your organization in identifying, dis-cussing, and resolving such matters?

The Accountability Checkup illustrates someof the issues that might reasonably be examined inrelation to each of the twelve attributes of effective-ness. The importance the CEO attaches to thesematters and the CEO’s level of comfort with thequality of available information in these regardswill be key determinants in deciding if it would beworthwhile for management to find out moreabout the effectiveness framework.

P A R T I I . C H A P T E R 1 2 . I M P L E M E N T I N G A N E F F E C T I V E N E S S - R E P O R T I N G P R O J E C T 1 6 3

M A N A G E M E N T D I R E C T I O N

Does everyone understand what they are meant to bedoing?

❑ whether our organization’s mission and priorities areclear, and are understood and shared throughout theorganization

❑ whether there are shared values that bind our organiza-tion together and help the diversity of professional,administrative, and cultural backgrounds work together

❑ whether staff have the authority and tools they needto make decisions and take action, consistent with theresponsibilities assigned to them

❑ whether the organization’s planning and communica-tions practices foster the above

R E L E VA N C E

Do our activities continue to make sense in terms of theconditions, needs, or problems to which they are intend-ed to respond?

❑ knowing the nature and extent of these conditions,needs, or problems—now, and as they may havechanged over time

❑ knowing what other organizations are doing in rela-tion to these conditions, needs or problems

❑ understanding the value-added that our products orservices are intended to provide in this context

❑ knowing whether current activities are operating with-in the approved mandate


Are levels of effort and selected methods of pursuingobjectives sensible and sufficient?

❑ degree to which each product or service is necessary tothe accomplishment of stated objectives

❑ whether our products or services are designed anddelivered in a manner that best responds to the natureand extent of the conditions, needs, or problems iden-tified

❑ extent to which our products or services are consistentwith prevailing standards of practice, ethics, etc., asmay exist

❑ whether the overall level and distribution of effortrepresented by our products or services is proportionalto stated objectives, identified needs, etc.


How challenging are our established goals, and havethey been accomplished?

❑ extent to which our achievements in key result areasmeet expectations in terms of (as applicable):

• the conditions, needs, or problems concerned• established performance targets• past organizational performance• the performance of comparable organizations or activities

❑ whether we are meeting our own prescribed standardsof practice, i.e., we are doing the right things right?

AC C E P TA N C E

Are clients and other key stakeholders satisfied with theorganization and its products or services?

❑ knowing the expectations of our clients and other keystakeholders and understanding the basis for theseexpectations

❑ the extent to which our clients and stakeholders indi-cate acceptance or satisfaction with the organizationand its products or services

❑ whether the organization and its products or servicesare respected within its peer network

S E C O N D A RY I M PA C T S

What are the unintended effects of our activities, be theypositive or negative?

❑ understanding the secondary impacts (social, econom-ic, financial, environmental, etc.) that our activities,products, or services could have on our clients, otherkey stakeholders, related organizations and programs,and/or the community at large

❑ being able to explain secondary impacts that signifi-cantly impede or work at cross-purposes to our statedobjectives, or where such information might call intoquestion the value attached to primary objectives

C O S T S A N D P R O D U C T I V I T Y

Are the relationships between costs, inputs, and outputsfavourable?

❑ whether defined product and service standards arebeing met at the least cost


A C C O U N T A B I L I T Y C H E C K U P

Place a number in each box signifying the extent to which each issue is important to you and your organization, usingthe following as guidelines:0—Not applicable/important 1—Noteworthy importance 2—Significant importance 3—Critical importance

❑ extent to which there is an appropriate balancebetween capital investments and operating expenses,overhead and operations expenses, capacity utilizationof major resources, etc.

❑ how we compare to similar organizations with respectto the above matters


How well are we anticipating and responding tochange?

❑ whether the organization has effective networks andprocesses to identify and assess important events andtrends in its environment

❑ the degree to which the organization has a history ofbeing able to adapt or respond successfully to chang-ing needs, circumstances, etc.

❑ how the practices and track record of the organizationin this regard compare to similar organizations


How good are the financial results in terms of matchingcosts with revenues and appropriations, and financialassets with obligations?

❑ whether our books of account, records, and financialmanagement control and information systems are inaccordance with sound financial policies and procedures

❑ how our cost and revenue ratios compare to similarorganizations

❑ the extent to which our organization’s overall financialposition is viable

❑ whether our organization has a history of conductingits operations within approved budgets and fundinglevels

WO R K I N G E N V I R O N M E N T

Is it a happy, healthy, and constructive working environ-ment where staff are motivated to work together, adaptto change, and develop?

❑ whether our staff ’s job descriptions appropriatelyreflect work responsibilities

❑ the degree to which our staff have the ability andopportunity to provide services to clients in a way thatis valued by clients

❑ whether our staff have adequate facilities and equip-ment to complete their tasks

❑ whether we are providing a safe environment for ouremployees and clients

❑ whether our staff are performing to stated and agreedexpectations, and are receiving appropriate recognitionfor their efforts

❑ whether management are aware of staff opinions inrelation to their job situation, the job satisfaction theyderive, and the organization’s management practices

❑ whether our human resources management plan isintegrated into the organization’s culture and operatingplans, thus enabling the recruitment, retention, devel-opment, and replenishment of well-qualified people

P R O T E C T I O N O F A S S E T S

How well do we protect against surprise events or lossesof key personnel, critical occupations, client informa-tion, facilities, equipment, inventories, processes, oragreements?

❑ understanding the risk of impairment or loss of ourorganization’s key assets

❑ whether we have strategies that adequately respond tothe nature and level of risk assessed

❑ whether these strategies and our performance complywith applicable regulations

❑ how these strategies and performance compare withthe industry in general

M O N I TO R I N G A N D R E P O RT I N G

Do management have the information they need to sup-port their decision making and their own accountability,and do they use it appropriately?

❑ whether management receive complete, credible, andfair information that satisfies their decision makingand accountability requirements

❑ whether accountability reporting is done in a trans-parent manner: are the right things reported at theright time and in the appropriate level of detail oraggregation?

❑ the extent to which monitoring and reporting systemsand processes are cost-effective

Now that you have completed the self-diagnostic, consid-er only those issues to which you have a score of 2 or 3;that is, those matters you regard as having significant orcritical importance.

Now ask yourself the following questions:

• How well is the organization/institution doing inthese areas?

• On what information do I base these judgments?• To what extent would my management colleagues

and those to whom we are accountable agree onthese matters?

Obviously, it is up to you and your management team topursue the issues that arise as a result of this diagnostic.


If the CEO, having completed the checkup,finds that the effectiveness framework raises issuesof sufficient importance to warrant further explo-ration, Step 2 will follow.

Step 2: The CEO and senior managementreceive a one-hour Executive Presentation givingthem an overview of the effectiveness framework.

The person who makes the presentationshould be knowledgeable about effectivenessreporting and needs to have a practical sense ofwhat is typically involved in implementing such areporting regime. He or she should also be knowl-edgeable and experienced in the areas of organiza-tional process, change, and behaviour. It stands toreason that the presenter would also be knowledge-able about the organization itself and the issueswith which it must deal. In certain circumstances,these qualities may not be found in one person, inwhich case the presenter might be accompanied orassisted by a colleague.

The purpose of the Executive Presentation isnot to make a “sale” or to have the governing bodyor senior management group come to a final deci-sion to proceed with implementation of the effec-tiveness reporting framework. Its purpose is to pro-vide the audience with sufficient information todetermine whether it is worth additional effort ontheir part to find out more about these ideas. Thepresenter has to be very clear on this reasonableexpectation if he or she wishes to convey it proper-ly to the audience.

To assist the presenter, CCAF has compiled akit containing detailed advice on how to preparefor the presentation and how to conduct the pre-sentation itself. Originally introduced in 1990, theExecutive Presentation kit contains a wide varietyof supporting documentation and video productsthat have since been added to the package. Itemphasizes the following points:

• the need for the presenter/facilitator to setand communicate reasonable expectations forthe Executive Presentation;

• the need for management to understand thatthis approach is about effectiveness reportingby them to those to whom they are account-able—it is not just another form of audit tobe performed by an auditor; and

• the organizational characteristics that favourthe successful implementation of the effec-tiveness reporting framework.

The CEO plays a key role in creating anappropriate context for the Executive Presentation.By virtue of their position, CEOs have a uniqueperspective on the needs of their governing bodiesand the strategic interests of their organizations.The Executive Presentation will likely be the firsttime that senior management as a whole hearabout and have the opportunity to discuss theseeffectiveness reporting ideas, and they will likelytake their cue from the insight, expectations, andenthusiasm which the CEO brings to the table.

The CEO (along with the “sponsor” insidethe organization, if there is one) and presenter/facilitator should meet before the presentation todiscuss expectations, the general approach to betaken, and so on. They should also establish a clearagenda for the presentation. It has been found tobe very helpful for the CEO to introduce theExecutive Presentation, and in doing so to:

• create a context for the presentation thatfocuses on the governance and policy man-agement challenges being faced by the organi-zation and on management’s responsibilitiesand capacity to develop its own agenda indealing with these challenges;

• establish the CEO’s own expectations; and• state any major questions or refer to matters

that the CEO would like the presenter tocover specifically or to elaborate upon.


SAMPLE AGENDA FOR EXECUTIVE

PRESENTATION

1. Introductory comments by the CEO

2. Introduction of presenter

3. Contextual comments by presenterregarding the purpose and development ofthe effectiveness reporting framework

4. Video presentation introducing theeffectiveness reporting framework

5. Bridging comments and discussion onuse of the framework elsewhere

6. Strategic advice concerningimplementation

7. Questions and answers

To put the presentation in context, it wouldbe helpful if the CEO would provide his or herpersonal perspective of the challenges for the orga-nization. These could include:

• changes, or signs of changing patterns, in theorganization’s clients’ needs and/or mattersrelating to clients’ demographics;

• alternative ways of fulfilling the above needs;• challenges imposed by current economic cir-

cumstances;• pressures or trends affecting the resources that

the organization has available to it to fulfill itsmandate;

• the impact of any recent or potential legisla-tive changes or trends;

• changes or trends in expectations of the organi-zation on the part of the government, clients,and other stakeholders (including the public),for example, deregulation, accountability,

reporting, increased authority or flexibility;• the organization’s need for better, not merely

more, information or reporting;• the challenges of managing risk and encour-

aging reasonable public (and government)expectations in this regard; and

• the opportunity for the organization to demon-strate leadership within the public sector.

Experience indicates that CEOs can cover thenecessary opening matters in about fifteen minutes.Doing this helps to assure the relevance of the ses-sion to the CEO and his or her management col-leagues’ interests. The focus of the CEO’s remarksshould not be on the framework itself, but ratheron creating a context whereby those in attendancewill be asking themselves the central question:

Does the effectiveness reporting frameworkhave real potential to help the organization tomeet current and real management issues andchallenges?

The presenter should be prepared to tailorcomments during the presentation to respond toany major questions or specific matters that theCEO would like addressed. These might deal withany number of topics, including:

• conceptual or process-oriented matters;• nature of executive involvement needed

throughout process;• the experience of others;• possible next steps; and• approaches to introducing the concept to fur-

ther levels of management and key practition-ers inside the organization.


After the presentation itself, participants willlikely have several questions regarding the nature,purpose, scope, and characteristics of managementrepresentations. The Executive Presentation kit

available from CCAF contains a handout intendedto help focus discussion and to provide answers tocommonly asked questions:


T H E C O N C E P T O F A M A N A G E M E N T R E P R E S E N T A T I O N

W H AT I S A M A N AG E M E N T R E P R E S E N TAT I O N ?

A management representation is an explicit statement made by senior management to their gov-erning body (e.g., a board of directors) in relation to an aspect of the organization’s effectiveness orperformance.

W H AT I S T H E A I M O F A M A N AG E M E N T R E P R E S E N TAT I O N ?

It is a means by which management discharges an important aspect of their accountability, i.e.,to report to those who have approved the organization’s mandate, and delegated or provided the req-uisite authority and resources.

A management representation provides input to decision making by the governing body, andprovides a basis for the governing body to exercise its oversight responsibilities, i.e., holding manage-ment to account for the performance of the organization.

The information included in a management representation also provides the governing bodywith an important basis for explaining the performance of the organization—and, indeed, their owndecisions—to the public whose interests they are elected or appointed to represent, and to otherstakeholders in the affairs of the organization.

O N W H AT S C O P E O F AC T I V I T Y D O M A N AG E M E N T R E P R E S E N TAT I O N S F O C U S ?

Depending on the complexity of the organization, specific requirements which may be imposedon the situation, or the aspect of performance being examined, management representations couldfocus on the organization as a whole or on a major line of business or program.

W H AT A R E T H E M A J O R C O M P O N E N T S A N D C H A R AC T E R I S T I C S O F A M A N AG E M E N T R E P R E S E N TAT I O N ?

The core of a management representation is information. A management representation is notjust a summary interpretation or conclusion by management, although management and the govern-ing body may wish to provide/receive such summary judgments as a context for further discussionbetween them.

The management representation should reflect the full range of key information needed to pro-vide a complete explanation of performance. The implications of significant gaps or caveats in thisregard should be discussed.


Commonly agreed principles and conventions (between the governing body and manage-ment) do not exist at this stage in the evolution of management representations on effectiveness.Therefore, it would be reasonable for a management representation to include some discussion ofhow the information is important to senior management and the governing body, and the centralfactors or criteria that have been applied by management in assessing performance. This providesan explicit basis for establishing mutually agreed expectations between management and thegoverning body.

It also provides a key point of reference for the work of various practitioners who perform taskson behalf of these two parties. This includes, for instance, the evaluator or analyst who works formanagement and who may be asked to perform information-gathering and analysis tasks to supportmanagement in making their representations. It also includes the auditor who works for the govern-ing body and who may be asked to examine management’s representations and report to the govern-ing body as to the fairness and reasonableness of these representations.

W H AT I S I T R E A S O N A B L E T O E X P E C T AT F I R S T ?

At first, it would be reasonable to expect that neither the board nor management will havea perfect sense of the interests and requirements of the other vis-à-vis management representa-tions. Nor is it likely that all the required information will be available, given the newness ofsome concepts.

Realistically, initial application is aimed at exploiting existing and readily available informationand analysis to maximum advantage, and at identifying and assessing any gaps. The initial product,albeit imperfect, can then be used as a basis for establishing an understanding and consensus betweenmanagement and the governing body as to what is needed, what can be cost-effectively provided, andwhat strategy will be pursued in this regard.

Participants will also likely have questionsabout the key considerations that guide the properapplication of the effectiveness reporting frameworkand the development of implementation strategy.

Moreover, they will want to know where these ideashave already been applied and what was learnedfrom these applications. The following, as a hand-out, may be useful in answering these concerns:


E F F E C T I V E N E S S P R O J E C T — G E N E R A L G U I D E L I N E S ,

I M P L E M E N T A T I O N A D V I C E , A N D L E S S O N S L E A R N E D

K E Y G U I D E L I N E S F O R A N O R G A N I Z AT I O N I N T E N D I N G T O A P P LY T H E CC A F E F F E C T I V E N E S S

R E P O R T I N G F R A M E W O R K :

• Initially consider all twelve effectiveness attributes and explain those that are not deemed useful.• Management representations (statements) on individual attributes should reflect the full range of sup-

porting information as well as any pertinent qualifications.• Representations should be relevant to the interests of the governing body (i.e., the intended receiver of

these management statements).• Top management must be directly involved in preparing representations.• A comprehensive strategy should be established for reporting management representations on effec-

tiveness over an agreed period of time.

S O M E I M P O R TA N T S T E P S A N D C O N S I D E R AT I O N S I N D E C I D I N G O N A N D D E V E LO P I N G A N

I M P L E M E N TAT I O N S T R AT E G Y:

• Expose the concept to senior executives and governing bodies prior to making a final decision toproceed.

• Take a structured approach to deciding whether or not to proceed. At this stage, identify and answermajor questions that affect implementation, particularly as regards to where it will be applied.

• Establish involvement of different players:- executives;- information providers (e.g., evaluators, policy researchers, planners, etc.);- internal audit;- support and facilitation consultants;- external auditor(s); and- coordinating group/task force.

• As applicable, ensure external auditor support and commitment.• The representations process itself:

- ensures clarity of business, mission, aims, goals, objectives;- relates attributes to same;- develops an explanation of relative importance of attributes, strategic choices, and options avail-

able under each;- considers what assessment factors or criteria need to be applied in order to develop a complete

picture of performance with respect to each attribute;


- brings forward/collects information in keeping with the above;- formulates conclusions on effectiveness in relation to each attribute;- refines representations on an iterative basis; and- establishes ongoing dialogue with the governing body, and other key stakeholders as applicable.

• Finalize representations.• As applicable, obtain audit assurance/opinion on representations.

SO M E L E S S O N S L E A R N E D F R O M OT H E R A P P L I C AT I O N S:

• External facilitation role should be understood and should not serve to deflect ownership of productfrom management. However, this facilitation role is crucial.

• The 80/20 role applies—it is important to focus only on what’s needed and avoid paralysis by analysis.Perfection is not the objective.

• Time and dollar costs for developing management representations depend on the nature of informa-tion already available and on the judgments that management reaches regarding the need for and cost-effectiveness of closing gaps in relation to this information. Not all gaps need necessarily be filled.

• Time and cost and, ultimately, success depend on the quality of interaction between, and the commit-ment of, senior executives. A strong sense of corporate cohesion and development is an important partof the process.

• Constraints should be recognized but should not become absolute determinants in applying the frame-work and in the organization’s own decision-making processes.

• The framework demands consideration of the past, present, and future. Management must be commit-ted to exploring all three with a necessary focus on the real world of the present.

• The decision as to where to focus the initial application of the framework is very important. This deci-sion should be taken only after management has had an opportunity to become more familiar with theframework and the individual attributes and how they could be applied to their particular organization(this can be accomplished through the one-day Organizational Workshop—Step 3 below). For example:

- Trying to apply the framework to the entire organization all at once is more than most organiza-tions could handle and, even if they could, it may not be the right or most cost-effective decision.

- Trying to focus on too small a business component of the organization reduces the cost-effective-ness of the exercise, principally by limiting the strategic management value of the result.

- Trying to focus on an administrative support function (planning, finance, etc.) limits the strategicvalue of the result since the main preoccupation of the governing body and senior management islikely to be the organization’s major programs or lines of business. For example, organizations donot exist to manage their human resources, they manage their human resources to support theirprogram objectives. Besides, the framework will automatically bring in any key administrativesupport considerations under various attributes (human resource–management performance con-siderations, for example, could be brought in under a discussion of “working environment”and/or “protection of [people] assets” to the extent that these attributes are relevant to an under-standing of the overall performance of a major program or line of business of an organization).

• Auditors should be committed, informed, and onside.

Soon after the presentation, the CEO andsenior management should decide whether theyhave sufficient interest in these ideas to justify fur-ther investment of their time in learning moreabout how the organization might profit from theirapplication.

Step 3: Within a month or so, senior man-agement attend a one-day OrganizationalWorkshop chaired and sponsored by the CEO andled by a well-informed facilitator, respected bysenior management as someone able to deal withissues at a strategic level.

This workshop gives management more detailon the attributes that comprise the effectivenessframework. It also provides management with anopportunity to apply some aspects of the frame-work to a few of their major programs or lines ofbusiness. Applying these attributes, albeit in a gen-eral way, can provide management with a taste ofthe substance of the representations they mightend up making, and of the challenges and trade-offs they may encounter in this process.

By the end of the workshop, senior manage-ment will have the information and orientationnecessary to make a decision about whether or notto try to apply these ideas. However, they will like-ly not make a final decision at the workshop itself.Instead, they will want to allocate time after theworkshop to discuss the matter more thoroughlythan would be possible in a short period at the endof an intense day.

At this stage in the process, both the facilita-tor and management should appreciate that a deci-sion not to proceed with implementation may be avery reasonable one. Indeed, the possibility of suchan outcome should be understood from the begin-ning. Management may decide, for instance, thatthe framework opens up issues that are too difficultto deal with from a policy perspective. Or they

may have concluded that implementing the frame-work demands more time and effort than they areprepared to invest. In fact, there are instanceswhere, after participating in the workshop, man-agement decided not to proceed and where, underthe circumstances, all parties agreed this was theright decision. The purpose of the OrganizationalWorkshop is not to skirt such matters, but to focuson them.

Recognizing the importance of the issues dis-cussed at these workshops, CCAF has prepared apackage of material intended to assist facilitators inleading sessions to the most satisfactory conclusion.Taken from that material, Exhibit A in the appendixto this part of the book provides a suggested agendafor an Organizational Workshop and Exhibit B pro-vides an overview of the Leader’s Guide.

The facilitator should retain a record of theresults of management’s discussions at theOrganizational Workshop for possible reference atlater stages in the process (for example, at Steps 4and 5).

P H A S E I I — D E C I D I N G W H E R E T O

F O C U S A N D H O W T O P R O C E E D

Step 4: Decision to proceed, and on whereto focus and why

Step 4: Within a few weeks following the one-day workshop, senior management meet to decidewhether and, if so, how the organization will proceedwith implementation. The CEO presides over thismeeting, with the facilitator guiding the senior man-agement group in discussion about a set of key deci-sions. The agenda can be relatively simple:


SAMPLE AGENDA FOR SENIOR

MANAGEMENT MEETING

1. CEO’s comments pursuant toorganizational workshop, decision toproceed or not

2. Facilitator’s overview ofimplementation steps

3. Key issues to be addressed by seniormanagement

4. Other matters

The third agenda item will take the longest tocomplete. The following key issues need to beaddressed after having decided to proceed andbefore actual implementation:

Issue 1—Determine where to focus the firsteffectiveness project.

The following principal questions and consid-erations may be important to a discussion of thisissue:

• How do we (senior management) want totalk about the organization to our governingbody and those outside the organization:

- in terms of its mission roles (usuallystated as part of the organization’s for-mal mandate)?

- in terms of its major programs or linesof business?

- in terms of the different major clientgroups with whom the organizationinteracts?

- in terms of its organizational structure?- in terms of some combination of the

above?

• Should we be trying to apply these ideas:- to the whole organization all at once?- progressively by components of the

organization?

In addressing these questions, the followingconsiderations may apply:

• link to the interests and perspectives of thoseto whom the effectiveness (that is, manage-ment representations) report would be going;

• senior management’s agenda regarding theseinterests and perspectives;

• senior management’s time commitments andavailability;

• importance of the selected component(s);• capacity of the organization to maintain effec-

tive control of the exercise;• analytical capacity within the organization and

availability of outside expertise to facilitate;• importance of the issues that would be exam-

ined;• link to ongoing and planned initiatives within

or outside the organization (for example,reorganization, resource management, or poli-cy initiatives, and so on); and

• inherent risk of such a project versus the addedvalue that it will bring to us and to those towhom we report or with whom we deal.

Issue 2—Clarify expectations for the final man-agement representations report.

The following considerations may apply to adiscussion of this issue:

• what we want the product to accomplish;• what our assessment is of the level of com-

mon understanding, controversy, etc., associ-ated with the area or issues being examined;

• whether there are existing expectations on thepart of others with which to contend and, ifso, whether these expectations are reasonable,and what is our ability to influence them;


• state of existing information, and whether tobase the first project on this information andbe content with identifying weaknesses, gaps,and redundancies for future rectification;

• opportunity to discover and documentlessons learned as a basis for improved, futureapplication of these ideas to other parts of theorganization; and

• opportunity to contribute to the general bodyof knowledge vis-à-vis the component(s)examined.

Issue 3—Identify who else, besides senior man-agement, needs to be brought into the process.

The following might be considered in decid-ing who should be involved when in the process,how, and to what end:

• persons or agencies to whom the effectivenessreport will eventually be directed;

• lower levels of management;• key support staff (for example, policy ana-

lysts, program evaluators, strategic planners,internal auditors);

• staff at large; and• external facilitator, either throughout the

process or at certain points in the process.

Issue 4—Identify the potential audiences for thereport, what their interests may be, and what processesare needed to ensure that they are appropriatelyinformed.

In designing a communications strategy, con-sider the following potential audiences who mayhave an interest in the process or product, andthen consider what they will want to understand,what message you want to convey, and how andwhen this ought to be done:

• governing body and, as applicable, other par-ties to whom there is an accountability rela-tionship;

• staff within the organization;• other external parties (for example, clients,

central agencies of government, accreditationbodies); and

• public at large.

Issue 5—Determine the necessity and conse-quences of having auditor involvement and, if so,how and when the audit component could best beengaged.

In considering whether and, if so, how to havean audit examination of the management represen-tations, the following considerations may apply:

• the consequences of having it (or not havingit) for the first project—from the perspectiveof management, those who will receive thereport, other stakeholders, independent com-mentators, and so on;

• when, where, and how to involve the auditorin the process; and

• how to assure reasonable expectations oneverybody’s part at the beginning, through-out, and at the end of the process.

Issue 6—Determine what the first step in theprocess ought to be, when this will take place, whoneeds to be involved, what preparatory work needs tobe done, and so on.

At this point, it is important to get consensusamong senior management on these matters.Specific details can be left for later. Normally, thefirst step is the Implementation Workshop (Step 6below). The following considerations may apply:

• where management want to be by the end ofthe Implementation Workshop;

• what documentation is needed to facilitatemaximum participation by senior managementat the workshop, and who will prepare it;

• who else (if anybody) besides senior manage-ment needs to be at the workshop;


• what preparation is required of participantsbefore they attend the workshop; and

• when and where the workshop will be held.

It is very important that all these matters are dis-cussed fully and frankly. These deliberations formthe basis for decisions that will influence the entireimplementation process.

There are likely to be several different andpossibly competing perspectives among seniormanagement on many of these issues. A workingconsensus may not come easily. The issues anddynamics of the meeting may present a significantchallenge to the knowledge, experience, and inter-personal skills of the facilitator.

By the end of the meeting, managementdecides whether to proceed and, if so, to:

• proceed in a specific area;• involve specific senior managers (and perhaps

a few other key support staff ) as a steeringgroup;

• start implementation at a specific time; and• take specific lead-up actions.

Following this meeting, the facilitator docu-ments the key decisions taken and their rationale.This document becomes a general communicationsvehicle to those who are as yet outside the process.It also becomes a reminder to senior managementthemselves regarding their original expectations,thus serving as an important benchmark againstwhich to assess progress.

At some convenient time after the seniormanagement meeting, the CEO may want toinform the governing body of management’s deci-sion to proceed and their reasons for doing so. Thegoverning body’s future role in the exercise couldalso be discussed at this stage.

(For the purposes of this chapter, the pre-sumption is that management decides to focustheir initial effectiveness reporting project on one

major area of the organization’s business. In mosteffectiveness reporting projects completed to date,this has, in fact, been the decision taken.)

P H A S E I I I — P R E P A R I N G F O R

A N D B E G I N N I N G

I M P L E M E N T A T I O N

Step 5: Preparation for theImplementation Workshop

Step 6: Implementation Workshop

Step 7: Briefing and securing thecooperation of the governing body

Step 5: The facilitator has several importanttasks to complete in preparing everyone for theImplementation Workshop—the first step in devel-oping management representations. In most cases,key policy and/or review analysts within the orga-nization would assist the facilitator in these tasks.

The timing of the Implementation Workshopis key. If the workshop is delayed a number ofmonths, the danger is that the perspective that par-ticipants have gained to date may be lost, whichwould mean that valuable time would have to bespent at the beginning of the workshop reorientat-ing participants. Conversely, scheduling the work-shop too soon will not allow sufficient time for thefacilitator to complete vital tasks and to developthe necessary background documentation to helpparticipants prepare themselves for the workshopand then work efficiently once they get together.Ideally, the Implementation Workshop could takeplace as early as six weeks after Step 4 (decision toproceed, and on where to focus and why), whichmeans that the facilitator would have about four


weeks to complete these tasks and to get the appro-priate background material into the hands of theparticipants about two weeks before the scheduledworkshop.

The facilitator’s first task is to develop a con-cise and thorough description of the program orline of business concerned. The information andexplanations provided in this program descriptionwill be vital material in developing managementrepresentations in relation to several attributes ofeffectiveness.

Although there may already be an existingprogram description, it will likely require updating.In some cases, the program description may needto be (re)developed entirely. The outline for a pro-gram profile shown in Exhibit C (in the appendix)includes an illustration of the structure and detailfor such a description.

The first part of the outline focuses on theprincipal elements of the program descriptionitself. The second part identifies some of the keyquestions that may be useful in assessing the com-pleteness of the program description and the ade-quacy with which key elements are described. Thethird part provides an overview of key policy, plan-ning, and review initiatives, processes and productsthat may be pertinent to the program. This infor-mation will help identify issues and informationpertinent to a discussion of the performance of theorganization or program in relation to individualattributes of effectiveness.

The second task for the facilitator is to iden-tify and document any key external or internaltrends, initiatives and reports that may bear on thediscussions to follow. These include:

• known pressures on the organization orprogram being addressed;

• known interests of management, governingbody, government, clients, and any other keystakeholders;

• pertinent policy and planning initiatives com-pleted, under way, or anticipated; and

• pertinent performance-related review process-es, activities, and reports.

Exhibit C provides advice on how backgroundinformation on these policy, planning, and reviewprocesses and initiatives might be presented.

The facilitator’s third task is to distill all thisinto a set of preliminary ideas and suggestions spe-cific to the program concerned. Participants canuse these ideas as an initial reference point forworkshop preparation and discussion. Thisincludes suggestions as to:

• why, to whom, and how information perti-nent to individual attributes of effectivenessmay be important;

• what constitutes reasonable key criteria or fac-tors for assessing performance in relation toindividual attributes; and

• what possible sources of information and dataare available or readily obtainable in relationto individual attributes.

At this stage, it is unlikely that all the partici-pants will share a clear and common perception ofwhat a management representation should looklike. It will facilitate future work if this ambiguityis removed before the Implementation Workshop.To assist in this, the facilitator could distribute theElements of a Management Representationdescribed below.

This document describes a structure for amanagement representation and identifies the con-tent and range of considerations that may be perti-nent in developing its constituent parts. It isintended as a general guideline for the develop-ment of the initial drafts of management represen-tations on individual attributes of effectiveness. Itincludes:


• explaining how information pertinent to aspecific attribute might be important to man-agement or the governing body in terms oftheir respective accountability, decision mak-ing, program improvement, or advocacyinterests;

• identifying and explaining what set of keyassessment factors, criteria, and indicatorsseem reasonable as a basis for analyzing per-formance in relation to the attribute;

• documenting key information and findingsthat explain the effectiveness of the organiza-tion or program in relation to the attribute; and

• presenting conclusions on the performance ofthe organization or program in relation to theattribute.

Using the Elements to think through anddocument the initial draft of the management rep-resentations will help bring and maintain a certaindegree of rigour and discipline to the process. Andmanagement representations so drafted can serve asuseful working and reference documents. It is like-ly, however, that the form and content of the finalmanagement representations reported to the gov-erning body will be a condensed version of what ispresented in the Elements. The state of practice inpreparing management representations is still at anearly stage. Examples of such early-developed rep-resentations by several organizations can be foundin monographs published under CCAF’s AppliedResearch Series. Several of these documents areidentified in footnote references in chapter 11.


E L E M E N T S O F A M A N A G E M E N T R E P R E S E N T A T I O N

1 . A T T R I B U T E

This section identifies the attribute of effectiveness that is the subject of this representation. Forexample, it might read: Relevance concerns the extent to which (the organization, program, or line ofbusiness) continues to make sense in regard to the problems or conditions to which it is intended torespond.

2 . I M P O R T A N C E

Here senior management explain why information in relation to this attribute is, or should be,considered important by themselves, the governing body, or others in understanding the performanceof the organization/program being addressed. Such reasons might include the following (clearly, thesegeneric reasons will need to be customized to reflect the particular circumstances, audiences, programand organization concerned):

• the information permits the governing body to provide oversight on a matter in which it has agovernance responsibility or interest;

• the information demonstrates management accountability in a key area of performance;• the information serves to identify and explain certain key decisions, choices, and challenges with

which management and/or the governing body must contend;• the information empowers the governing body and/or management to make decisions, commu-

nicate the basis for these decisions, and exercise influence; and


• the information assists an informed dialogue between the governing body and management onissues of corporate significance.

This section can be helpful in focusing management’s thinking about this aspect of effectiveness.In addition, establishing why this information is important and how it could be used will, in turn,influence and facilitate subsequent discussions about the scope and depth of the assessment requiredand, ultimately, the conclusiveness of the information that will be needed.

3 . B A S I S F O R A S S E S S M E N T

This section identifies and explains the central factors or criteria that are needed to deal ade-quately with the attribute. These could be expressed in a variety of ways, depending on the nature ofthe attribute:

• key subsidiary issues and questions that need to be addressed• specific indicators to be used• specific benchmarks/targets to be applied• specific comparisons or other types of analysis that need to be done

There are a series of questions that management ought to ask of themselves in relation to theadequacy and sufficiency of the overall assessment approach being put forward:

Would we, those to whom we are reporting, and others who are knowledgeable, agree that:• each of the questions/indicators identified is essential to an understanding of effectiveness in

relation to this attribute?• together, these questions/indicators are sufficient in the sense that they are capable of explaining

the complete story?• the logic that connects these questions/indicators is either self-evident or, if not, is adequately

explained to the reader?• where appropriate, it is made clear what specific benchmarks or targets are being applied, and why?

4 . I N F O R M A T I O N A N D F I N D I N G S

This section is the heart of a management representation.Using the previous Basis for Assessment as a reference point, in this section senior management

provide the key information and findings that can be reported at this time. Information and findings are likely to have both qualitative and quantitative aspects. Material provided in this respect is distinct from interpretations or summary judgments by man-

agement, which are addressed in the next section of the management representation statement.Questions to ask regarding the appropriateness and sufficiency of the information and findings

being presented are:


• Are there any central criteria or factors about which no information and findings are presented?

• Is each information item significant in the sense that its inclusion is essential to an understand-ing of the effectiveness of the program or organization in relation to this attribute?

• Is each information item significant in the sense that its exclusion would lead the reader to anincomplete, inaccurate, or wrong conclusion?

• Does the information presented in relation to a specific factor (i.e., subsidiary question or indi-cator) adequately answer/address that criterion?

• Are there any significant gaps or weaknesses in the information/findings and, if so, are theseappropriately acknowledged and explained?

5 . C O N C L U S I O N S

In this section, senior management explains the interpretations they have placed on the informa-tion and findings, and the summary judgments and conclusions they may have come to in relation tothe performance of the organization/program in terms of this attribute. In some cases, managementmay be able to formulate a firm conclusion, in other cases, there may be caveats, and in still others, itmay not be possible to formulate a conclusion.

The purpose of these conclusions is to convey to the governing body management’s view ofwhat all this means. This may facilitate the deliberations of the governing body and its subsequentinteractions with management.

Management’s interpretations and conclusions are just that, however; they do not preclude theneed to provide the necessary key information. Without this information, the governing body willhave no basis upon which to formulate its own judgments.

There may be several reasons that caveats are necessary, including:• a question about the softness of the data or information;• the lack of multiple lines of evidence where this would be seen as necessary or desirable;• a real or perceived question concerning the reliability of the source of the information or the

accuracy of the data;• the lack of key information needed to be able to draw a complete picture of performance; and• the lack of agreed benchmarks or conventions against which to analyze the information or data.

An important question to consider at thisstage is the sort of information that will be used inassessing the organization or program in respect ofeach of the attributes. To answer that question, one

has to determine the basis upon which judgmentscan be made. The following suggestions are intend-ed to provide assistance:


S U G G E S T E D B A S E S F O R A S S E S S I N G A T T R I B U T E S O F


M A N AG E M E N T D I R E C T I O N

Several factors enter into an assessment of the effectiveness of Management Direction. These include:• the extent to which there is harmony between the governing body and management with respect

to the strategic directions and priorities they see for the organization• the extent to which there is a clear mission statement for the institution, supported by an explic-

it value system that is centered on providing good service to the client, and the extent to whichstaff see themselves in all this, and they know what is expected of them and how this supportsthe overall mission of the institution

This could be measured in a variety of ways, some more direct than others:- Is there an explicit statement of mission and values?- How successfully do management and staff communicate with each other on these matters,

through supervisor/staff discussions, staff focus groups, general meetings, house organs,and so on?

- Are there policies, procedures, and processes in place that lend support to this, and are theyaccessible and communicated to and understood by staff?

- What are the views of managers and supervisors in relation to staff knowledge and under-standing of these matters?

- Does management make decisions and act in a way that is consistent with the principlesunderlying the mission and value system of the organization?

- Is there systematic/anecdotal behaviour, events, and so on, to suggest the extent to whichstaff know and understand?

- What are the direct views of staff on these matters?- How do outside stakeholders perceive the corporate philosophy and image projected by the

institution?• the extent to which staff have the authority and tools they need to make decisions and take

action in accordance with their responsibilities, and the extent to which staff understand thelimits of their authority and that matters that transcend these limits are referred to theappropriate person

• the extent to which program, operational, and work plans are in place, are linked, and focus onthe issues most pertinent to the decision-making and accountability interests of those responsible.


R E L E VA N C E

Assessing effectiveness with respect to relevance could include a consideration of the following:• the nature and extent of the problems, conditions, demands and needs to which the program

mandate is directed, and the extent to which these matters have changed since the programmandate was approved or last revalidated;

• what other organizations are doing/not doing vis-à-vis the above, and the extent to which thissituation has changed since the program mandate was approved or last revalidated;

• how the program is linked to prevailing government policy or corporate, professional, and com-munity philosophies (for example, hospital programs in light of philosophy that favours deinsti-tutionalization), and the extent to which these matters have changed since the program mandatewas approved or last revalidated;

• the extent to which experience with the program, or changes that have since taken place in theprogram’s environment, confirm or refute the assumptions, logical argument, and value-addedinitially associated with or used to justify the program; and

• the extent to which the program and its activities and services are operating within the approved mandate.


Assessing effectiveness with respect to appropriateness could include a consideration of the following:• the degree to which each of the program’s major activities is necessary to the accomplishment of

stated objectives;• the extent to which program products or services are designed and delivered in a manner that

best responds to the nature and extent of the conditions, needs, or problems identified;• the extent to which these products or services are consistent with prevailing standards of prac-

tice, ethics, etc.; and• whether the overall level and distribution of effort represented by the program’s products or ser-

vices are sufficient in relation to stated objectives and identified needs, or, for that matter,whether they exceed requirements.


An analysis of the effectiveness of the organization or program in achieving its intended resultswill likely revolve around the following:

• the extent to which the institution/program achievements in key result areas meet expectationsin terms of (as applicable):

- the conditions, needs, or problems concerned- established performance targets- past organizational performance- the performance of comparable organizations or programs

• whether the institution/program is meeting its own prescribed standards (i.e., are we doing theright things correctly?)


AC C E P TA N C E

The attribute of acceptance could be examined from several stakeholder perspectives, including:• the intended clientele;• the actual user (who may be different from, or just a subset of, the intended clientele);• indirect beneficiaries (for example, the public at large);• other related institutions or intermediaries that may form part of the overall network in which

this institution or program operates;• interest groups;• policymakers; and• internal staff.

Central factors or criteria that may form a basis for assessing acceptance include the following:• the nature of the expectations of these stakeholder groups and the basis for their expectations;• the extent to which stakeholders indicate acceptance of or satisfaction with the program or ser-

vices. The following factors may be pertinent:- perceived adequacy of the information they receive about the program;- accessibility of the program/services;- the quality and quantity of products/services they receive;- the affordability of the program/services;- timeliness of the program in responding to client’s demands;- satisfaction of the client with the manner in which these products or services were deliv-

ered (for example, how courteously or fairly they feel they were treated by staff );- their overall assessment of the efficacy or utility of the program/services in helping them

achieve a satisfactory outcome.

S E C O N D A RY I M PAC T S

It is difficult to identify what factors or criteria might be applied in examining secondaryimpacts without knowing the particular institution or program concerned. This is one area whereeach case will be somewhat different.

CO S T S A N D P R O D U C T I V I T Y

One might want to group indicators into logical sets indicating that, where it would be mean-ingful and possible to do so, information will be presented in time series, drawing comparisons tosimilar institutions and relating information to specific standards/targets—either those set by theinstitution itself and/or by an external body.

There are several ways to group the indicators. Here is one possibility:• Costs

- salary, equipment, accommodation, insurance, etc.


- overhead/operations ratio- training and development costs

• Staff utilization and productivity- management/staff/client ratios- ratios that show how staff spend their working time (e.g., working directly with the client,

doing administrative work, etc.)- various $/work load ratios- processing time

• Facilities and equipment- utilization ratios (for example, how much of the effective capacity of the facilities and

equipment is being used)- $/unit ratios


This is the financial statement of the organization. This should be accompanied with a clearidentification and thorough discussion of any key underlying assumptions, caveats, and so on. Someof the factors that may be pertinent include:

• whether the books of account, records, and financial management control and information sys-tems are in accordance with sound financial policies and procedures;

• how cost and revenue ratios compare to those of similar organizations;• the extent to which the organization’s overall financial position is viable; and• whether the organization has a history of conducting its operations within approved budgets

and funding levels.


Three factors that may be at the center of an analysis of effectiveness in relation to this attribute are:• the extent to which the institution has the networks, mechanisms, and processes in place to

identify and assess possible consequences of relevant trends and events;• the extent to which the institution has demonstrated responsiveness or its lack thereof in rela-

tion to trends and events (for example, new programs or services established, realignment ofresources, etc.); and

• the extent to which the organization compares favourably with similar institutions in theserespects.

W O R K I N G E N V I R O N M E N T

The following factors may be pertinent:• extent to which the institution has the number, type, and mix of staff needed to deliver the pro-

gram/services;


• whether staff job descriptions appropriately reflect work responsibilities;• the degree to which staff have the ability and opportunity to provide services to clients in a way

that is valued by clients;• whether staff have adequate facilities and equipment to complete their tasks;• whether the organization is providing a safe environment for employees and clients;• whether staff are performing to stated and agreed expectations, and are receiving appropriate

recognition for their efforts;• extent to which the institution is making adequate investments in relation to job- and career-

development requirements and the aspirations of staff and the institution;• whether management are aware of staff opinions in relation to their job situation, the job satis-

faction they derive, and the organization’s management practices; and• whether the organization’s human resources management plan is integrated into the organiza-

tion’s culture and operating plans, thus enabling the recruitment, retention, development, andreplenishment of well-qualified people.

P R OT E C T I O N O F A S S E T S

Some of the key assets and considerations that may be applicable in this regard are:• key people—compensation packages and career development strategies may attract and retain

these individuals; and, succession plans can make appropriate provision for the smooth replace-ment of them should they decide to leave;

• client and other information—files and computer records;• key property—ensuring that long-term capital plans are tied into the broad corporate strategy;

procedures and practices are in place to assess and maintain the value of the property and itsgood state of operations; and the state of the property meets external inspection standards;

• key equipment and facilities—provision of adequate space to perform work; regular performanceinspections on equipment to assure their effective operation;

• inventories;• financial position—funding received from the government, donations/revenues from the public

and the like;• agreements—insurance, supply, etc.; and• corporate memory (some aspects may link back to key people/information)—establishment of

policies, procedures, and information systems so that the knowledge base of the institution doesnot reside in the heads of a few key people and is compromised when they leave.

An assessment of effectiveness in relation to this attribute involves three central factors:• the extent to which the institution has identified its key assets and assessed their risk of loss

and/or impairment;• the extent to which the institution has strategies in place that adequately respond to the nature

and level of risk assessed; and

On all these matters, the facilitator, assistedby whatever support staff the institution hasassigned, offers ideas and suggestions. The choiceof the phrase “ideas and suggestions” as opposed to“conclusions and recommendations” is important.The facilitator, analysts supporting the facilitator,and the workshop participants themselves all needto understand that it is the last who make the finaldetermination on these matters. The purpose ofthe background documentation prepared by thefacilitator is to help open up the issues and guidedeliberations, not to constrain these discussions orto foreclose conclusions.

The facilitator’s final task, most likely withinput and assistance from key analysts within theorganization, is to synthesize all this material into adiscussion paper. This document is sent to partici-pants at least two weeks before the ImplementationWorkshop to guide their thinking and preparationfor the upcoming workshop. Exhibit D (in theappendix) illustrates the content of such a discus-sion paper. Two assumptions are made; first, man-agement met after the Organizational Workshop to

consider the range of issues noted in Step 4; sec-ond, at this meeting, management decided to focusthe initial project on one major program area.

Although Exhibit D describes a single three-day Implementation Workshop, a variation of thisapproach may sometimes prove useful or even nec-essary. For example, this step in the process mighttake the form of two one- or two-day meetingsheld several weeks apart. The first of these couldfocus on developing the front end of the manage-ment representations (that is, why individualattributes are important, and what assessment fac-tors, criteria, and so on, could be applied in analyz-ing performance). Then, a general analytical frame-work established, the intervening period could beused to amass the needed and available informa-tion. The results could then be distributed toworkshop participants ahead of a second meeting,which would be held to complete the initial set ofmanagement representations by matching, analyz-ing, and integrating the information collected forindividual attributes.


• the extent to which these strategies and their performance compare to industry practice andstandards, as well as comply with external requirements (for example, legal, regulatory, accredita-tion, and so on).

M O N I T O R I N G A N D R E P O R T I N G

The factors that might be applicable to an assessment of effectiveness in relation to this attributeinclude:

• the extent to which senior management and, indeed, other levels of management and key usersreceive complete, credible, and fair performance information that satisfies their decision-makingand accountability requirements;

• whether accountability reporting within the organization, and from the organization to key out-side stakeholders, is done in an appropriately transparent manner. Are the right things reportedat the right time and in the appropriate level of detail and aggregation?; and

• the extent to which monitoring and reporting systems and processes are cost-effective.

Whether or not the ImplementationWorkshop was completed in one or two meetings,the general expectations for and overall logic of theapproach to this step of the process would remainthe same.

Step 6: Participants convene in a three-dayImplementation Workshop and begin the processof developing their management representations.The facilitator leads this workshop.

Because the tasks of the workshop willdemand the full attention of those participating, itmight be advisable to hold it off-site where there islikely to be a more appropriate ambience and fewerdistractions. Holding a reception or dinner at theend of Day 1 can also give a sense of occasion anduseful informality to the workshop. A possibleagenda for this session is shown in Exhibit E (seeappendix).

By Day 3, the end of the workshop, partici-pants will have a written first draft of a manage-ment representation on each attribute in relation tothe program or line of business being examined. Inpreparing these drafts, participants will use thematerial supplied by the facilitator prior to theworkshop as well as the information they broughtwith them.

In contrast to the free-flowing atmosphere ofStep 4, when decisions were made on where tofocus and how to proceed, the ImplementationWorkshop is substantially more complex andintense. Participants are asked to accomplish agreat deal within a relatively short time. They arealso being drawn into discussions that they maynot have had with one another previously and onissues and questions for which there are noabsolute answers.

The facilitator plays a critical role in fosteringan appropriate climate for workshop discussionand in helping participants keep focused on the

expectations they set for themselves. Playing vari-ous roles during the workshop—planner, organizer,catalyst, challenger, integrator, and problemsolver—the facilitator will need to be adept atmoving from one role to another, as circumstancesdictate. In the final analysis, however, it is thegoodwill, enthusiasm to participate, and mutualrespect among participants that will make or breakthe Implementation Workshop.

Step 7: As applicable, and shortly after theImplementation Workshop, the CEO briefs thegoverning body on the status of the project. Thismeeting is intended to get the board’s general per-spective on what management is doing and to reaf-firm the board’s support in this regard. This isneeded to sustain management’s commitment tothe project, to provide a steady focus for manage-ment’s deliberations, and to assure that the finalproduct will be relevant and useful to the board.

Prior to this meeting, the CEO may want toprovide members of the governing body with amemorandum to help them prepare. Exhibit F (seeappendix) contains a typical briefing note.

At this point, the CEO may suggest the use-fulness of establishing a small ad hoc committee ofgoverning body members with whom to consult,now that management has a better sense of whatthey would be able to produce. If there is agree-ment to such a proposal, a committee representinga cross-section of the governing body could bestruck. Subsequently, a joint meeting might beheld between this committee and key members ofthe project steering group to discuss mutual expec-tations, progress, and potential courses of action.The CEO would lead discussion on this matter.


P H A S E I V — R E F I N I N G T H E

I N I T I A L M A N A G E M E N T


Step 8: Refinement of initial set ofrepresentations by management in a seriesof meetings

Step 9: Consultation/discussion with thegoverning body

Step 8: The management steering groupmeets for two or three one-half to one-day sessions(usually four to six weeks apart) to refine theirmanagement representations. These sessions areguided by the facilitator.

The four-to-six-week period between sessionsis appropriate so as not to unduly burden theschedule of the participants over too extended atime. The intervening periods also provide a rea-sonable opportunity for follow-up and preparationwork arising from discussions and decisions takenat the previous meeting.

Refinements to the initial management repre-sentations are made based on thorough discussion—supported by appropriate documentation—as towhy information on individual attributes is impor-tant, the assessment factors or criteria that ought tobe applied to any judgments of effectiveness regard-ing each attribute, and the information availablewith respect to these criteria. As the steering groupconsiders such matters, it is likely that rationales asto why an attribute is important will be modified,assessment factors and information will be added,honed, or dropped, and conclusions on performancein relation to the attribute will be adjusted.

Most of the work is accomplished in smallerworking groups (possibly the same working groupsas were established for the ImplementationWorkshop), with periodic plenary sessions held

during the day to discuss progress and problems,resolve how to handle any overlap or duplicationamong the attributes and information, and dealwith any other problems that may have arisen.

There are many questions that the groupmight ask of itself in relation to the individual rep-resentations they have developed. The followingare some of those questions. (alternative or addi-tional questions may be applicable in individualcircumstances.)

R E G A R D I N G T H E E X P L A N A T I O N T H A T

S E N I O R M A N A G E M E N T A R E P U T T I N G

F O R W A R D A S T O W H Y E F F E C T I V E N E S S

I N F O R M A T I O N I N R E L A T I O N T O T H I S

A T T R I B U T E I S , O R C A N B E , I M P O R T A N T

Content of the explanation given• Are there points being made here that we as

senior management think are not central toour executive responsibilities:

- to our governing body?- to the government?- to our organization?- to our staff?- to our clients?- to other outside stakeholders?

• Is anything important missing?• Most particularly, will our governing body:

- regard the points made in our explana-tion as important to their interests?

- expect anything else to be included inthis explanation?

Presentation style• Does the explanation flow logically from one

point to the next?• Can some points be integrated, or should

others be broken apart?• Can we condense it?


R E G A R D I N G T H E B A S I S O N W H I C H

M A N A G E M E N T A R E A S S E S S I N G

E F F E C T I V E N E S S I N R E L A T I O N T O T H E

A T T R I B U T E

Content of the basis for assessment• Would we as senior management be prepared

to make the following statements to our gov-erning body?

- Each factor or criterion that we areusing in our assessment is central tounderstanding the effectiveness of theorganization or program in relation tothis attribute.

- No major criterion or factor is missing.• Would anything be seen as missing, biased or

inappropriate by:- our governing body?- the government?- our staff?- our clients?- other outside stakeholders and special

interest groups?- the public at large?- independent subject matter specialists or

researchers?• Are there specific benchmarks, targets, and so

on that we should be applying, either onesalready identified or that we ourselves feel areappropriate, or ones that industry practice orstakeholder expectations would suggest?

• Have we incorporated any such benchmarksand targets into our thinking, and have weexplained adequately what we have done, andwhat we have not done in this regard?

Presentation style• Does the presentation of the assessment fac-

tors and criteria flow logically?• Where the logic flow may not be self-evident

to others, have we made reasonable efforts toexplain how it all fits together?

• Can we condense it?

R E G A R D I N G T H E D O C U M E N T A T I O N O F

I N F O R M A T I O N A N D F I N D I N G S O N


A T T R I B U T E

Content of the information and findings• Is each assessment factor or criterion that

should or can be addressed actually addressed?• Is each point of information or finding directly

pertinent to its applicable factor or criterion?• Are sources of key findings and information

appropriately cited?• Is there any reason to suspect a significant

problem with the currency, accuracy, complete-ness, or reliability of the findings and informa-tion? If so, are appropriate caveats provided?

• Would anything said here be seen as con-testable, incomplete, biased, or contradictory:

- in relation to points made in our man-agement representations with respect toother attributes?

- by our governing body (for example, inrelation to information they may be get-ting through other processes)?

- by the government?- by our staff?- by our clients?- by other outside stakeholders and special

interest groups?- by the public at large?- by independent subject matter special-

ists or researchers?


Presentation style• Do the points flow logically?• Can some points be integrated, or should

others be broken apart?• Can we condense it (for example, boil it

down to no more than two or three keypoints per factor or criterion)?

R E G A R D I N G C O N C L U S I O N S M A N A G E M E N T

A R E P U T T I N G F O R W A R D A B O U T


A T T R I B U T E

Content of the conclusions• Is each applicable criterion or factor specifi-

cally addressed, and will this be recognizedreadily by the governing body?

• Is any conclusion, information, or interpreta-tion inconsistent with the criteria and find-ings, or with our representations on otherattributes?

• Is the level of confidence associated with eachconclusion substantiated by the informationand findings presented?

• Have we identified areas where there is animportant information deficiency (missing,unanalyzed, unreliable), and noted whatactions we have taken, are taking, or are con-templating, and explained where we intendno action?

• Will any of these conclusions be viewed asincomplete, biased, inappropriate, contradic-tory, or simply wrong by:

- our governing body?- the government?- our staff?- our clients?- other outside stakeholders and special

interest groups?- the public at large?

- independent subject matter specialists orresearchers?

• Are there any central findings not linked to aconclusion?

Presentation style• Does the presentation of the conclusions flow

logically?• Can some points be integrated, or should

others be broken apart?• Can we condense it?

At what point the group would stop asking thesequestions is essentially a matter of judgment as towhere diminishing returns on further effort beginto set in. The facilitator can be helpful in identify-ing that point.

This type of reflection can be useful in testingthe validity, completeness, and logic flow of therepresentations, not just from the point of view ofmanagement, but also from the possible point ofview of other key stakeholders or readers of thefinal report.

The key stakeholder to be kept in mind is, ofcourse, the governing body. It is often easy to getcaught up in the process when developing the draftrepresentations and, perhaps, lose sight of—orgloss over—important matters. Asking the ques-tions suggested above may be one way of assuringthat the final product adequately responds to theexpectations that were set for the exercise, and that,individually and as a group, the representationsmeet the basic test of reasonableness.

The product of this step is a second, andnow more substantive, draft of the managementrepresentations for each of the attributes.Experience shows that appropriate representationscan be written in two or three pages for eachattribute.


Step 9: :At this juncture, the CEO couldtrigger a joint meeting between the ad hoc commit-tee of the governing body and key members of theparticipating group to discuss and get a reaction tothe second draft of management representations.This discussion may identify the need for manage-ment to do further fact-finding and/or to makefurther refinements to the representations theyhave developed.

P H A S E V — W R I T I N G T H E

M A N A G E M E N T

R E P R E S E N T A T I O N S R E P O R T

Step 10: Development and managementreview of a working draft of themanagement representations report

Step 11: Management consults with theirgoverning body on working draft withfollow-up refinements as necessary

Step 12: Management tables the finaldraft of the effectiveness report with theirgoverning body

Step 10: An individual within the participat-ing group is assigned the responsibility to consoli-date and write the overall effectiveness report. Thegroup meets for a number of half- to one-day ple-nary sessions (usually four to six weeks apart) toreview, challenge, and refine various aspects of theoverall report. The facilitator guides these delibera-tions. Typically, there would be three such sessions.Depending on individual circumstances, however,fewer or more sessions may be held.

The four-to-six-week period between sessionsis appropriate so as not to unduly burden theschedule of busy group members. The intervening

periods also permit a reasonable opportunity forfollow-up and preparation work by the groupmembers, the drafter of the report, and the facilita-tor, arising from discussions and decisions taken atthe previous meeting.

The report is more than the sum of its twelvemanagement representations. Up to this point, theprincipal focus has been on the logic and com-pleteness of individual representations. The groupnow needs to stand back and look at the linkagesand consistency among these representations. Theyneed to consider the central messages these repre-sentations convey as a whole, and how these repre-sentations and the strategic context within whichthis initiative was taken can best be communicatedto the governing body.

These steps result in an initial and completeworking draft of the report. Exhibit G (see appen-dix) suggests a structure for the report. It is basedon actual reports that have been made to governingbodies. This proposed structure can serve as a basisfor discussion among the project’s principal stake-holders. The final structure of a project report mayvary from the illustration that follows, owing tothe nature of the project, how the organizationperceives its business and priorities, the main mes-sages to be communicated, and so on.

It is useful to start the report with a thoroughdescription of its purpose and the governance/management context in which this approach toeffectiveness reporting is being undertaken. For aninitial effectiveness reporting project, it may also beuseful to include a section on lessons learned, as ameans of documenting and communicating a ret-rospective analysis of such things as: what workedwell and what didn’t; what benefits and costs haveaccrued from the project; what other insights havebeen gained; and what should be done differentlyin the future. Essentially, these matters become thebasis for future discussions on whether and how toproceed with future projects.


Subsequent reports may be shorter. Apartfrom a brief description of the area of activity con-cerned, and a detailed discussion of managementrepresentations on the effectiveness of this particu-lar line of business, it may only be necessary toremind the governing body how the current reportfits within the context of past or future suchreports, that is, within whatever strategy is beingproposed to, or has already been established with,the governing body.

Step 11: At this juncture, the CEO couldarrange a joint meeting between the ad hoc com-mittee of the governing body and key members ofthe management group to get a reaction to theworking draft of the report. This may identify theneed for further fact-finding or refinement toaspects of the report.

It may be possible to accomplish this refine-ment through a series of bilateral discussionsbetween the person responsible for drafting thedocument, the CEO, individual members of themanagement group, and the facilitator.

Step 12: The CEO and key members of themanagement group present the final draft report tothe ad hoc committee of the governing body or, inthe absence of such a committee, to the full board.

P H A S E V I — T A B L I N G T H E

R E P O R T W I T H T H E

G O V E R N I N G B O D Y

Step 13: Tabling the managementrepresentations report with the full governing body

Step 13: The CEO tables and discusses thereport with the full governing body.

Before sending the report to all board mem-bers, the CEO will likely make a final review of itto reinforce his or her personal confidence in itand to anticipate possible reactions by the govern-ing body. To this end, the CEO will likely wantthe facilitator and one or two key people from themanagement group to participate in a pretablingdiscussion. Such a discussion might focus on thefollowing questions:

• Are there matters on which the report has notresponded to previous input or feedback pro-vided by the governing body or its commit-tee? If so, do I know why, and am I preparedto deal with members of the governing bodyon these matters?

• Do the main messages in the report fairlyreflect the effectiveness of the organization orprogram? Is the level of conviction associatedwith these key points appropriate, and doesthe report offer sufficient substantiation inthis regard?

• What will the governing body’s reactions tothese main messages likely be? Do I have astrategy to answer or deal with these reac-tions?

• What implications does the report have forspecific decisions or actions that may have tobe taken by the governing body, myself, andmy management colleagues, or others?

• What is my bottom-line assessment of theeffectiveness reporting framework andprocess? How does this accord with initialexpectations? What further potential do theseideas have for the organization? What shouldthe next steps be?


P H A S E V I I — A U D I T O F

R E P R E S E N T A T I O N S O N


Step 14: Audit of management’srepresentations

Step 14: There may or may not be an auditphase to the initial project. It is sensible to expect,however, that the governing body, as well as otherkey stakeholders, may want to receive independent,third-party assurance regarding the reasonablenessand fairness of management’s representations—ifnot for the first exercise, then for subsequent effec-tiveness reporting exercises.

At Step 4, for instance, it may have beendecided not to involve audit right away. Instead,the decision may have been to use the initial pro-ject to diagnose the state of the organization’s per-formance information in relation to the require-ments of the effectiveness framework. Havingobtained a better understanding of or confidencein management’s capacity to make substantive rep-resentations, or what intermediate actions arerequired to get to this point, the organization maydecide that audit scrutiny could become a featureof subsequent effectiveness reporting initiatives.

If audit is involved, it should be recognizedthat Phase VII is not really a sequential stage in thesense that it only begins after management hasfinalized the effectiveness report. Ideally, the audi-tor would be involved from the beginning. It is inthe interest of all parties that this occur. Such earlycooperation allows management to know the kindof audit tests that will be used and to shape theirrepresentations accordingly. At the same time, itallows auditors an opportunity to understand the

nuances of the process from the outset and tomake better-informed decisions about their auditprograms.

A discussion of what is involved in auditingmanagement representations on effectivenessappears in Part III.


CONCLUSION

There are a number of factors that now putand will continue to put increasing pressure onaccountability in both the private and public sec-tors. The economy lacks the buoyancy it had in the1980s, and is unlikely to regain it in the nearfuture. Governments and businesses, in the face ofenormous financial pressures, are reorganizing,merging, restructuring, and downsizing.

To survive, organizations and their compo-nent departments and programs are going to haveto fight for resources. Those that are most account-able and have processes which support and demon-strate their effectiveness will have a much betterchance of survival. They will have to be able todemonstrate:

• the relevance, appropriateness, impacts, andefficiency of their programs, services andproducts; and

• their capacity to respond to change and tosustain effectiveness in the long term.

Informed decision making is an obligationrather than an option, particularly so when limitedresources and the need to serve citizens and satisfycustomers combine to introduce new and higherlevels of risk than have been common in the past.It is therefore necessary for organizations to ensurethat they have appropriate performance manage-ment and information systems.

E L E M E N T S O F A G O O D

P E R F O R M A N C E M A N A G E M E N T

S Y S T E M

There are several elements that must be clear-ly defined in a good performance management andinformation system:

• the needs and obligations of informationreceivers;

• the responsibilities of those who are obligedto measure and report on performance;

• an agreed basis for measuring and reportingon performance;

• organizational arrangements, incentives, andcapacity development;

• processes and mechanisms to collect, analyze,report, and use information about perfor-mance; and

• validation mechanisms.

N E E D S A N D O B L I G AT I O N S O F I N F O R M AT I O N

R E C E I V E R S

Those who are intended to receive information inthe system ought to:

• have clear responsibilities to know what con-stitutes reasonable information to fulfill themandate they have accepted; they need toarticulate clear objectives and set out expectedperformance or achievement regarding theseobjectives;

• be responsible to ensure that appropriatearrangements are in place to obtain therequired information; and

• use the information obtained in a fair andappropriate manner that fosters trust, a posi-tive environment, and the effectiveness of theorganization.

R E S P O N S I B I L I T I E S O F T H O S E W H O A R E O B L I G E D

T O M E A S U R E A N D R E P O R T

Those who are expected to collect and report per-formance information ought to:

• pay due regard to the specific responsibilities,interests, and needs of those to whom theyare reporting, and due regard to the timeli-ness of the information for decision making;

P A R T I I . C O N C L U S I O N 1 9 3

• ensure that the information reported is com-plete enough to be a fair representation of theperformance of the organization; and

• ensure that the performance information isrigorously prepared and that it contains what-ever explanations or qualifications are neces-sary to allow the receiver to make a well-informed judgment about performance.

A N AG R E E D B A S I S F O R M E A S U R I N G A N D R E P O R T I N G

O N P E R F O R M A N C E

The agreed basis should be clearly understood bythe providers and receivers of the information, andshould:

• serve as a bridge between the needs, interests,and responsibilities of information receiversand suppliers;

• recognize that performance is a multifacetedconcept and should focus on results andachievements;

• serve as a basis for both receivers and suppli-ers of information to judge its completenessand fairness, working from a common andpredetermined set of ground rules; and

• allow flexibility, but ensure that departuresfrom what otherwise might be expected areidentified and adequately explained.

O R G A N I Z AT I O N A L A R R A N G E M E N T S , I N C E N T I V E S ,

A N D C A PAC I T Y D E V E LO P M E N T

The following organizational arrangements areessential for the successful implementation andoperation of a good performance management andinformation system:

• the personal and visible support and involve-ment of the most senior officials in the orga-nization (in government, this means politicalleaders as well as administrative heads);

• an ethos and whatever incentives and sanc-tions may be appropriate to encourage theproduction and use of broad-based perfor-mance information; issues involved may bepower sharing, value systems, mind-sets—indeed, the entire organization’s managementculture;

• a commitment to provide the time andresources necessary to help all those involvedto understand the system and play their roleseffectively; and

• a commitment to persevere with the systemover the long term and to continuouslyimprove it.

P R O C E S S E S A N D M E C H A N I S M S T O C O L L E C T,

A N A LY Z E , R E P O R T, A N D U S E I N F O R M AT I O N

A good performance management and informationsystem:

• recognizes that information must come froma variety of sources, not from a single one;

• takes maximum advantage of existing infor-mation sources, such as day-to-day statisticaland operations data, financial informationsystems, internal audit, other review mecha-nisms, evaluation and strategic planningprocesses; and

• assembles information from these varioussources into an integrated performance pic-ture capable of meeting the tests of adequacy,completeness, and fairness.

VA L I D AT I O N M E C H A N I S M S

A suitable validation mechanism can substantiallyincrease the confidence that decision makers havein the information provided by the system. Asdescribed in Part III, audit could be used to pro-vide this assurance.


E L E M E N T S O F G O O D

P E R F O R M A N C E M E A S U R E S

Usually, no single indicator is sufficient toprovide decision makers with the understanding ofthe performance of an organization or programthey need. Different indicators are commonlyrequired to afford insight into the various aspectsof performance. Besides being reliable and timely,information should:

• explain what choices were made and why, andwhat the consequences were or are expectedto be: the information should contain sup-porting data and include explanations of itssignificance, limitations, reliability, and rele-vance to the issues at hand;

• compare actual performance with intendedperformance and outcomes, using predeter-mined targets or goals;

• contribute to determining how performancecan be sustained or improved in the future;

• reflect the actual performance for the timeframe in question;

• balance the need for understanding and sim-plicity with the complexity of the programs,services, or organizations in question; and

• vary to accommodate the responsibilities,needs, and interests of people at different lev-els of the organization.

P A R T I I . C O N C L U S I O N 1 9 5

A P P E N D I X

EXHIBITS

The following exhibits are provided purely fordemonstration purposes. They reflect what hasbeen done in some applications of the effectivenessreporting framework, and it is hoped they will beof interest to people who are considering takingsuch an initiative. Readers are asked to keep inmind that all these matters can and should beadapted to the particular circumstances of theorganization concerned and the interests, skills,and preferences of the people involved.

E X H I B I T A

S U G G E S T E D A G E N D A F O R O R G A N I Z A T I O N A L W O R K S H O P

Subject Duration

Introduction.................................................................................................................................20 minutesKey challenges ..............................................................................................................................25 minutesOverview of the effectiveness framework......................................................................................30 minutesThe effectiveness attributes...........................................................................................................2 hoursIntroduction to group exercise......................................................................................................10 minutesGroup exercise..............................................................................................................................50 minutesReport out by groups ...................................................................................................................50 minutesApplication of the framework.......................................................................................................45 minutesHelping the organization to make a decision........................................................................................30 minutes


E X H I B I T B

O V E R V I E W O F L E A D E R ’ S G U I D E

P A R T I I . A P P E N D I X . E X H I B I T S 1 9 7

O R G A N I Z A T I O N A L W O R K S H O P P A C K A G E

N AT U R E O F T H E O R G A N I Z AT I O N A L W O R K S H O P

This is a one-day workshop.The purpose of the workshop is to give executive decision makers more information on the

framework of twelve attributes of effectiveness and to give them the opportunity to apply the attribut-es to certain of their major programs or lines of business. Applying these attributes—albeit, in a verypreliminary way—can provide management with a foretaste of the substance of the management rep-resentations they might end up making, and of the challenges and trade-offs they may encounter inthis process. Doing this provides management with the orientation they need to be able to determinewhether these ideas are applicable to their interests. They will also be able to gauge the potential con-sequences that implementation of these ideas may have for their organization.

Possible outcomes of the workshop are:• the organization feels that there is merit to these ideas and would like to hold another, more

detailed workshop to explore further the ins and outs of implementation;• the organization is ready to begin implementation and is ready to commit the resources and

effort necessary to do so;• the organization is not prepared to pursue the topic further at this time.

Participants in the workshop are primarily members of the executive management group.Possibly, one or two key members of the governing body might participate, as well. Also, a few keypractitioners who would be involved in supporting management in such a process might also be invit-ed to attend. The workshop is led by a facilitator.

It is important to keep the focus on the senior-level decision makers, as they are in the best posi-tion to understand where these ideas can or should fit into the strategic agenda of the organization.Middle-level managers are understandably more focused on operational issues—which are not thefocus of this framework.

It is best to keep the size of the group to no more than twenty—the ideal size would be closer tothe 10-12 range.

K E Y PA R T I E S

Typically, there are three key parties.

T H E W O R K S H O P L E A D E R / FA C I L I TAT O R

The role of this person is to lead the participants through a process that will orient them to theeffectiveness framework, have them apply these ideas to their organization, and bring them to a point


where they can make a decision to proceed with implementation, or not. To accomplish this, theindividual must be thoroughly familiar with the framework, knowledgeable about the organization,and comfortable in leading workshops with senior-level decision makers.

In certain cases, the facilitator may be familiar with the framework but not have the necessarydepth of knowledge about the organization. In these circumstances, the workshop leader/facilitatormay be accompanied and assisted by another individual, perhaps a colleague or someone from withinthe organization.

T H E S P O N S O R

The involvement of a sponsor from within the organization is critical to the success of the work-shop. This may be the CEO or some other senior and influential member of the executive manage-ment group. This person is someone who believes in the merit of the message being conveyed and whois prepared to advocate the framework to his or her peers. The sponsor plays several important roles:

• working with the workshop leader/facilitator in planning the workshop, providing information,and assisting in logistical arrangements;

• ensuring that the right people are at the workshop;• introducing the workshop leader/facilitator and setting the stage for the event in terms of situat-

ing these ideas within the context of the interests and strategic agenda of the organization; and• assisting in achieving closure, that is, helping to bring discussion to a point where a decision can

be made to proceed, or not.

If there is no senior-level individual inside the organization who is prepared to play this role,this may present a reason to question the viability of an effectiveness reporting initiative. Ultimately,this process cannot be sustained without the direct leadership and involvement of management andwithout a strong sense of personal ownership on their part to the eventual effectiveness report.

T H E PA R T I C I PA N T S

Those attending the workshop must come prepared to participate actively in the discussion. Thesuccess of the workshop is also highly dependent on who these individuals are. The best candidates arepeople who are responsible for forming and directing the affairs of the organization. If few of these indi-viduals are prepared to participate in the workshop, leaving this role to lower-level managers or func-tionaries, this may present another reason to question the viability of an effectiveness reporting initiative.

P U R P O S E O F T H E L E A D E R’ S G U I D E

The Leader’s Guide provides the facilitator with the advice and information necessary to plan andthen lead the workshop. It comprises nine modules that are linked to a typical agenda for the one-dayworkshop. A participant’s package also comes as part of the overall Organizational Workshop kit.

Of course, as was the case for the Executive Presentation, the workshop Leader’s Guide cannotanticipate every possible type of audience or circumstance. The guide is designed in a way that recog-


nizes there will need to be some flexibility and adaptation in the planning and conduct of the work-shop to suit the specific circumstances.

S E T T I N G A N D C O M M U N I C AT I N G R E A L I S T I C E X P E C TAT I O N S

The purpose of the Organizational Workshop is to bring the members of the executive manage-ment group to a point where they can make an informed decision to proceed, or not, with imple-mentation of the effectiveness framework.

One of the most important understandings that MUST emerge from the workshop is that exec-utive management appreciate that—“This is something that cannot be done for us. We must do itourselves!”

While it might be possible to come to a “go/no-go decision” at the end of the day, this decisioncarries with it yet a further set of considerations: when and where to start, who to involve, preciselywhat to expect, and so on. It is unlikely that by the end of the day, there will be sufficient time orwillingness to engage this further set of considerations.

Thus, at the end of the workshop, the CEO/sponsor should indicate that time will be set aside inthe near future for the executive management group to discuss and resolve these matters. TheCEO/sponsor should encourage his or her colleagues to think about these considerations in the interim.

A D M I N I S T R AT I V E I S S U E S

The Organizational Workshop is an intensive exercise for everyone involved. The group dynam-ics of the workshop require that participants be as comfortable as possible within their environment,with the arrangements, and with one another. Therefore, administrative arrangements should facili-tate open dialogue, ease, and comfort. They should represent a strong commitment to the process andbe of the highest possible quality.

Special care should be given to determining the location for the workshop. Ideally, the meetinglocation should be outside of the business premises. There are several reasons for this. It helps todirect participants’ undivided attention and energies to the workshop without the constant reminderof the organization’s day-to-day business going on just outside the meeting door.

Usually, the best venue for the workshop is a hotel. It will have the necessary space for the mainmeeting room, and separate, smaller rooms for the group syndicate sessions that are part of the work-shop. The hotel can also easily provide other necessary items such as coffee, meals, and audiovisualequipment, as required.

As indicated above, it is important to establish a pleasant and congenial working relationshipamong all participants in the workshop and the workshop leader. A good way to achieve this is tohold a dinner the evening before the workshop, preferably at the same hotel in which the workshopwill be held. The occasion can also be used to jog the participants’ memories on their discussions atthe Executive Presentation and to go over the next day’s agenda.

E X H I B I T C

O U T L I N E F O R A P R O G R A M P R O F I L E


K E Y E L E M E N T S O F T H E P R O G R A M D E S C R I P T I O N

M A N D AT E

• State the current mandate for the program and identify the source of its authority• Explain the underlying rationale for this mandate in terms of:

- the conditions, needs, or problems that the program is trying to resolve- what other programs—inside or outside the organization—may be doing or are empow-

ered to do

O B J E C T I V E S A N D G O A L S

• Identify the stated objectives and goals of the program or line of business

I N T E N D E D C L I E N T E L E / P O P U L AT I O N A F F E C T E D

• Describe who the beneficiaries, intended clients, and actual users are, distinguishing each class,as appropriate

• Describe their characteristics• Explain how these may have changed over time

P R O G R A M D E S I G N A N D D E L I V E RY

• Identify the principal activities or functions of the program and, as applicable, explain theirinterrelationships

• Explain how decisions are made and implemented in relation to who is eligible for what mannerand level of program support

• Explain how program activities are delivered (for example, centralized versus decentralized oper-ations, direct contact with client or through intermediaries, and so on)

O R G A N I Z AT I O N A N D R E S O U R C E S

• Describe the organizational structure of the program• Identify the physical, financial, and human resources that are allocated to the program


K E Y Q U E S T I O N S R E G A R D I N G T H E C O M P L E T E N E S S A N D

A D E Q U A C Y O F T H E P R O G R A M D E S C R I P T I O N

G E N E R A L

• Are there any important discrepancies in perception of the program or aspects of it on the partof various key stakeholders? If so, what are the implications of this?

M A N D AT E

• Does the description adequately explain the original circumstances and reasoning upon whichthe current program mandate and objectives have been justified?

O B J E C T I V E S A N D G O A L S

• Are the program’s objectives stated in results-oriented terms? If not, do we nonetheless knowwhat key results are intended in relation to the objectives? And, if we do, have these key intend-ed results been appropriately identified?

• Are there specific expectations or targets as to the level of performance to be attained in key resultareas? If not, do we nonetheless have a general sense of what these levels of performance ought tobe? And, if we do, have these performance expectations or targets been appropriately identified?

• Is there any competition or conflict among the stated objectives that ought to be explained?

I N T E N D E D C L I E N T E L E / P O P U L AT I O N A F F E C T E D

• Have we appropriately described all the pertinent characteristics (social, economic, demograph-ic) of our beneficiaries, intended clients, and actual users?

• Are we making an appropriate distinction between the concepts of beneficiary, intended client,and actual user?

P R O G R A M D E S I G N A N D D E L I V E RY

• Is there anything that needs to be explained in terms of the intended balance or order of prece-dence among the major activities or functions of the program?

• Is there anything more that needs to be explained in terms of the key criteria and processes usedto establish, for example, who is eligible for what manner and level of program support?

P O L I C Y , P L A N N I N G , R E V I E W , A N D R E P O R T I N G

I N I T I A T I V E S A N D P R O C E S S E S

Developing an overview of past, existing, or anticipated initiatives and processes that are perti-nent to the program will help ensure best use of existing systems and practices. It will also provide acommon starting point for the facilitator and Implementation Workshop participants in their ownpreparation and information gathering leading up to the workshop.


R E G U L A R P O L I C Y, P L A N N I N G , A N D R E V I E W P R O C E S S E S

• Identify and describe each major external and internal process in terms of:- the focus of the process- who conducts the process- what is reported- to whom- how often- how report is used

S P E C I F I C PA S T A N D C U R R E N T P O L I C Y, P L A N N I N G , A N D R E V I E W I N I T I AT I V E S

• Identify and describe specific major external and internal initiatives or events in terms of:- the focus of the initiative- the time frame for the initiative- who conducted/conducts the initiative- as applicable, the findings of the initiative, to whom they were reported, and how they

were used

E X H I B I T D

I M P L E M E N T A T I O N W O R K S H O P D I S C U S S I O N P A P E R

T A B L E O F C O N T E N T S

1. Introduction2. Background3. Description of key roles4. Structure of the Implementation Workshop5. Pre-workshop preparation by participants6. Documentation of workshop discussion and outputs

Annexes

I M P L E M E N T A T I O N W O R K S H O P D I S C U S S I O N P A P E R

1 . I N T R O D U C T I O N

O B J E C T I V E O F W O R K S H O P

The objective of the Implementation Workshop is to have participants develop a first cut of theirmanagement representations in relation to [name of program]. This step in the process builds on theresults of the Organizational Workshop, and on decisions taken at a follow-up management meeting onwhere and how to focus initial efforts.

As explained later, participants will accomplish this task through an iterative process of plenary andsyndicate discussion sessions over the three-day period.

E X P E C T E D O U T P U T

It is reasonable to expect that workshop participants will produce the following outputs over thethree days:

• a description of the strategic context for this initiative, i.e., why undertaken, who can benefit, and how?;• a written preliminary set of management representations in relation to key attributes that manage-

ment think are important to communicating and understanding the performance of [name of pro-gram]; and

• a broad outline of a strategy for further development of these representations to a point where seniormanagement would be comfortable presenting them to the board (governing body).

F O L LOW- U P AC T I V I T I E S T O T H E I M P L E M E N TAT I O N W O R K S H O P

Follow-up activity and meetings will be needed to refine the representations, undertake necessary con-sultations with key stakeholders, draft and refine the management representations report, and develop thestrategy for bringing this report to the board.


P U R P O S E O F T H I S D I S C U S S I O N PA P E R

The purpose of this discussion paper is to provide participants with an overview of the agenda andactivities of the Implementation Workshop.

It is important that participants read this paper before coming to the workshop and that each par-ticipant brings available documentation, information, and data that will be used as a basis for developingrepresentations on the twelve attributes of effectiveness. More detail on the nature of the required docu-mentation, information, and data is provided below.

2 . B A C K G R O U N D

On [date], senior management participated in a one-day Organizational Workshop. The broad objec-tive of this workshop was to introduce the concepts and framework behind effectiveness reporting andauditing and to provide participants with an opportunity to begin to work with some of these ideas in rela-tion to [name of organization]. This was intended to help senior management make an informed decisionas to whether and how to proceed in implementing this approach within the organization.

Following the Organizational Workshop, senior management decided to proceed with implementa-tion, which, in turn, led to a series of more detailed considerations as to where to focus initial efforts, whoto involve, and how to proceed. These matters were discussed at a senior management meeting on [date].At that time, senior management decided to begin with a focus on [name of program].

At that time, management also decided that the next step in the process would be a three-dayImplementation Workshop, delivered by [name of facilitator] and involving the same management group(and, perhaps, a few key analysts who will be expected to support management).

3 . D E S C R I P T I O N O F K E Y R O L E S

W O R K S H O P L E A D E R

The workshop leader is [name of facilitator]. Broadly stated, the facilitator’s role is to establish thecontext for participant deliberations, help focus discussion and provide advice, and maintain an appropri-ate pace in the workshop. [Specify whether facilitator will be assisted by anyone else].

W O R K S H O P PA R T I C I PA N T S

The participants are the senior management of the [name of organization] (and, perhaps, a few of thekey analysts whom management expects to call upon for support throughout the exercise). The work anddiscussions of the participants will be the key to a successful workshop.

Participants are expected to bring to the workshop various documentation, information, and dataavailable to them that they will then be using as a basis for developing management representations on thetwelve attributes of effectiveness. Annex 1235 of this discussion paper provides an initial overview of keypolicy, planning, review and reporting initiatives and processes that may be pertinent to the program.Participants can use this overview as a starting point in their preparation for the workshop.


4 . S T R U C T U R E O F T H E I M P L E M E N T A T I O N W O R K S H O P

Attached as Annex 2 is the agenda for the workshop.

F L E X I B L E S C H E D U L E

The timing of activities identified on the agenda may be speeded up or slowed down depending on cir-cumstances. For instance, at the Organizational Workshop and follow-up management meeting, some thoughthas already been given to those matters scheduled for the first morning. Therefore, it may well be possible toresolve these issues fairly quickly. If so, this would allow more time for the development of the managementrepresentations, which would be welcome, as the time proposed for this activity is expected to be very tight.

M O R N I N G O F D AY 1

After a few brief opening remarks from [CEO name], [facilitator name] will set the stage for the threedays and review the agenda and related logistics with participants. It will be very important at this time toidentify and establish agreement on the general expectations and specific outputs for the workshop, and toacknowledge what still will likely need to be done as further follow-up.

After these introductory remarks and discussion, the first task will be for participants to recap/writedown what they see as the broad strategic context for this initiative. This includes consideration of severalquestions, including the following:

• What factors, initiatives, events, constraints, trends, etc., in the organization’s internal and externalenvironment are at play or could emerge that are significant to this initiative?

• What challenges, decisions, and choices do the board and senior management face in today’s or theforeseeable climate, and what benefits can be derived from this initiative in this regard?

• Do staff, clients, the general community, and other key stakeholders also derive specific benefits fromsuch an approach as this?

Participants will recall that several of these issues were discussed at the Organizational Workshop andagain at the follow-up management meeting. Annex 3 lists some of the issues and considerations discussedat these earlier meetings. [Note: The facilitator should have kept a summary record of these discussions and pre-pared this Annex from those notes.]

The second task for the morning is to recap/write down the rationale for choosing [name of program]as the focus of this first application of management representations. We should review, and if necessaryamend, the program profile that has been prepared and is enclosed as Annex 4.236 We should also identifythe relative importance of individual attributes to this activity. Any attributes that participants judge to beof little or no importance will be put aside. However, it will be important to have a specific rationale forany such exclusions in order that it can be brought back into the discussion of the overall management rep-resentations in the morning of Day 3.

The issues surrounding the choice of [name of program] and the general applicability of the attributesto this area were also discussed previously. Annex 3 outlines these matters, as well. [Note: The facilitatorshould have kept a summary record of these discussions.]


Depending on the participants’ level of satisfaction with the results of past discussions on these mat-ters and the adequacy of relevant documentation appended to this discussion paper, this aspect of theworkshop might proceed quite quickly.

A F T E R N O O N O F D AY 1 A N D A L L O F D AY 2

This time will be devoted to the main task of the workshop—development of management represen-tations in relation to individual attributes.

Participants will be divided into two groups. Each group will be responsible for discussing and thenwriting out draft management representations on two different attributes at a time. Each group will bringits written statements back to plenary for further discussion and, as necessary, revision. This process ofshifting back and forth from syndicate groups to plenary session will continue until all the attributes iden-tified as being important have been addressed. Annex 2—the workshop agenda—outlines provisionalblocks of time for syndicate and plenary group discussion, respectively. We may want to alter this timeallocation as we proceed.

General guidelines and a structure for the development of individual management representations isprovided in Annex 5.237 The commonality of structure will assure that all key aspects of a management rep-resentation are addressed in each instance or, at least, to the limits of current thinking and information.This commonality will also help us to identify gaps, overlaps, and so on, among the individual representa-tions (in the morning of Day 3).

At the end of Day 2, the intent is to be able to give each participant a set of all the individual repre-sentations developed over the previous day and a half.

During this one and a half days, it will be important for us to keep our expectations in mind and tomaintain perspective in relation to what we can reasonably accomplish at this stage in the process. Thismay not always be easy. We should push ourselves to be as complete and forthright as we can, but still rec-ognize that we will not have a full level of comfort with the assessment criteria and information we canbring to bear. However, if we can marshall effectively the criteria and information we currently have, andthen identify what further steps we need to take after the Implementation Workshop, we will have accom-plished a lot.

M O R N I N G O F D AY 3

Beginning in the morning of Day 3, participants will again break into their syndicate working groups.Each group will be asked to review and critique the overall set of representations developed, looking forgaps, redundancies, conflicts, opportunities to make links among these statements, and opportunities forgreater precision and better explanation. The working groups will bring the results of their deliberationsback to the plenary in the latter half of the morning for further discussion and adjustment, as required.

A F T E R N O O N O F D AY 3

In the early afternoon, and time and stamina permitting, each syndicate group will be asked to con-sider several questions related to a strategy for further development of the management representations.The results of this will be discussed in plenary session later in the afternoon. If we are pressed for time, and


we may well be, an alternative might be to delay detailed consideration of these questions to a subsequentmanagement meeting in much the same fashion as we did following the Organizational Workshop.

Some of these questions/issues include the following:• What needs to be done to the draft representations in the short term (that is, over the next few

months) to bring them to a point where senior management would be reasonably comfortable in pre-senting them to the board?

• Who should do what and by when in this regard?• What now needs to be done to begin to engage key members of the board?• As applicable, how will audit be engaged? Has our experience over the last few days given us any cause to

reconsider the merit or consequences of decisions made earlier on this matter?• What needs to be done to engage or keep informed other key stakeholders; for example, staff, clients,

other organizations?

5 . P R E - W O R K S H O P P R E P A R A T I O N B Y P A R T I C I P A N T S

In preparing for this workshop, participants should read this discussion paper carefully; this willappropriately prepare them to approach their tasks and to appreciate the intended linkages among theseactivities.

Most important is the pre-workshop preparation of participants in amassing the documentation,information, and data that they will bring to the workshop and then use as a basis for discussing, develop-ing, and reviewing management representations in relation to individual attributes.

This material could involve the following:• measurement criteria or factors to be used in assessing various aspects of performance or effectiveness;• performance benchmarks, targets, or goals that have been set for various aspects of the program;• available performance information on operations, administration, financial and human resources,

health and safety, costs, outputs and outcomes, and comparisons with other programs or organiza-tions;

• reports and reviews such as those related to audit, evaluation, organizational development, health andsafety;

• policy and planning documents developed within or outside the organization and which would bearon the program and its related activities;

• survey or other such reports pertaining to needs and views of staff, clients, and the community atlarge;

• special studies and reports such as those produced in journals, by the media, and so on, that have abearing on the program in whole or in part; and

• annual reports.

As noted previously, Annex 1 of this discussion paper provides a preliminary overview of some of themajor policy, planning, review and reporting initiatives and processes. Participants may find this useful as astarting point to their preparation. Also, Annex 5 provides guidance and a structure for developing draftmanagement representations. This may serve as a further guideline in identifying and thinking about the


range of assessment factors and information that may be pertinent to a particular attribute. Finally, Annex6 provides a series of general ideas and suggestions in relation to individual attributes, based on a prelimi-nary analysis of all the documentation gathered thus far. No doubt, over the course of their preparationand workshop discussion, workshop participants will find reason to reject, add to, and refine these initialnotions.

6 . D O C U M E N T A T I O N O F W O R K S H O P D I S C U S S I O N A N D O U T P U T S

[Describe mechanics of how documentation produced by individual syndicate groups anddiscussed/refined in plenary sessions will get processed during the three days]. [Note: The facilitator willneed to make arrangements to be able to produce this documentation, possibly using hotel resources, or faxing thematerial back to the organization’s offices to have someone type and/or copy this material before sending it back tothe workshop site.]

A N N E X E S

Annex 1: An Overview of Pertinent Policy, Planning, Review and Reporting Initiatives and ProcessesAnnex 2: Implementation Workshop AgendaAnnex 3: The Strategic Context for this Initiative—Results of Previous Meetings and DiscussionsAnnex 4: Program ProfileAnnex 5: Elements of a Management RepresentationAnnex 6: Drafting the Initial Set of Management Representations—Preliminary Ideas and Suggestions for

Consideration


E X H I B I T E

I M P L E M E N T A T I O N W O R K S H O P A G E N D A 2 3 8

SESSION DURATION

DAY 1

Introductory remarks to set the stage for the workshop (CEO and then facilitator) 45 minutes

Plenary discussion Why is the organization interested in pursuing a management 1 hourrepresentations approach?Who can benefit and how?

Plenary discussion Why has the organization chosen to start with a focus on this program? 1.5 hoursAre we satisfied with how the program is currently described?What level of importance can be attached to individual attributes in relation to this program area?

Syndicate group work Development of draft management representations on specific attributes 1.75 hours

Plenary discussion Discussion of draft management representations on these specific attributes 1.5 hours

DAY 2





DAY 3

Syndicate group work Review of consolidated set of draft management representations 1.75 hours

Plenary discussion Discussion of the results of syndicate groups’ deliberations 1.5 hours

Syndicate group work Strategy for further refinement of management representations 1.75 hours

Plenary discussion Discussion of syndicate groups’ deliberations 1 hour

Plenary Concluding remarks


E X H I B I T F

T Y P I C A L C E O B R I E F I N G N O T E T O T H E G O V E R N I N G B O D Y

To: Members of the Board of Directors

From: CEO

Subject: P R O J E C T C O N C E R N I N G T H E D E V E L O P M E N T O F M A N A G E M E N T R E P R E S E N T A T I O N S

O N E F F E C T I V E N E S S

As a follow-up to our previous discussions, and as background to our upcoming meeting, this briefingnote has been prepared to explain:

• the nature and purpose of this project;• the current status of the project and the plan for its completion;• the importance I attach to having your advice and input at this juncture in the project; and• the nature of the advice and input I am seeking.

N A T U R E A N D P U R P O S E O F T H I S P R O J E C T

Over the last number of months, we have been involved in a project whose purpose is to apply acomprehensive “effectiveness reporting framework” and set of guidelines developed by the CCAF to one ofour main lines of business. I should emphasize that this is a senior management–led, not analyst oraudit–led, process.

For your reference, the attached brochure provides an overview of the effectiveness reporting frame-work we are using, identifying the twelve attributes of effectiveness that comprise the framework and therange of questions that are involved.239

This framework is designed to help a CEO report to the governing body and account for the organi-zation’s performance in a broader, yet more focused and meaningful manner than has generally been com-mon practice. CCAF’s background research indicated that, in most cases, governing bodies across the pub-lic sector were receiving only piecemeal data, lacking in strategic context, and of limited value to their deci-sion making and ability to hold management to account in an adequate manner.

As noted below, this work is ongoing; however, ideas are beginning to coalesce and a reporting struc-ture and content are beginning to emerge. As I explain later, I think it would be very timely to have yourperspective and advice at this juncture.

I would like to elaborate briefly on why we are engaged in this project.First, both you as board members and I and my management colleagues are faced with complex, chal-

lenging, and controversial issues. Demands for services are increasing, the resource environment is con-strained, and the public policy context within which we operate is, to say the least, fluid. This situationwill undoubtedly continue. It requires of us thoughtful and difficult choices. To make these choices, good


information is needed, and this information needs to be developed at a level that will help to identify andanalyze trade-offs and explain these decisions credibly and forthrightly to a variety of stakeholders. Suchinformation will also help us to exert influence on these stakeholders.

Second, more comprehensive reporting will provide a basis for continuing dialogue between manage-ment and the board, at a level that we both judge to be of direct interest. I see this project as an opportuni-ty to introduce a high-level, program-oriented basis for discussion as well as a way of providing you with abetter picture of how underlying activities interact in pursuit of the main objectives of the organization.We have chosen one of our larger programs as the first candidate for reporting. If the project is successful,we will, over time, focus on our other major programs.

Third, the project supports the interests and capability of senior management in developing andestablishing consensus on priorities, connecting the professional and administrative cultures of the organi-zation, directing its operations, and providing a focus for important management initiatives. To a person,senior management is convinced that the process has already yielded benefits along these lines.

Fourth, the project provides the organization with the opportunity to demonstrate leadership withinthe broader community of institutions of which it is a part.

C U R R E N T S T A T U S A N D P L A N S F O R C O M P L E T I O N

Senior management has already met on a number of occasions—usually in the form of workshops—to focus on this matter. The purpose of these workshops was to develop an understanding of these ideasabout effectiveness, to assess their feasibility in our organization, to decide on where to focus initial efforts,and to begin the process of developing management representations; that is, the report on effectiveness.

Subsequently, we have met to develop and refine our management representations on individualattributes of effectiveness. This process continues and we expect that an initial draft of the report will bedeveloped in the relatively near future.

What this initial report will do is capture and integrate existing information, as well as data readilyattainable, against the reporting framework. I expect that there will be several gaps in our information thefirst time around, given that certain aspects of the framework involve new thinking and directions. The ini-tial report will assess the importance of these gaps and, where deemed necessary and cost-effective, will out-line strategies to fill them.

Over time, the report will become increasingly more complete, as will reports on other major pro-grams as they are addressed. I visualize the overall picture as one with in-depth reports to the board on dif-ferent individual major programs each year, and with general performance information on each programbeing reported on a more frequent basis, as required; for example, annually, quarterly, and so on.

I M P O R T A N C E O F Y O U R A D V I C E A N D I N P U T

As I have noted above, senior management have already devoted considerable time and effort to thisproject. This has begun to yield a concrete picture of the nature and content of the kind of report weshould be able to produce. I expect to speak to this matter in more detail at our meeting.

We are now at a juncture where your perspective on the project and on the results it could or shouldyield vis-à-vis your needs is required. Your general views on these matters will help us give further focusand shape to our work and to the report under development. Without this perspective, the ultimate objec-


tive—-providing you as board members with information that will facilitate your decision-making andgovernance responsibilities—-will not be achieved.

Assurance as to your interest in the project and its objectives and, ultimately, your preparedness toengage the product of this process is also of vital importance to sustaining management and staff commit-ment and effort to improve the quality of reporting on performance. Your support will also be a key factorin the credibility attached to this project by the government, by other boards, and by other organizationsand stakeholders within the broader community.

N A T U R E O F A D V I C E A N D I N P U T B E I N G S O U G H T

I regard this upcoming meeting as the beginning of a process of substantive dialogue with you in rela-tion to this project and the results I expect it will produce. I anticipate that there will be further occasionsfor discussion over the next few months as this project continues.

At this point, your general advice and assurance of interest is being sought. Only when you have hadan opportunity to review the draft report would it be reasonable to ask you for more specific reactions.

My hope is that this briefing note, supplemented with the comments I will be making at our meet-ing, will provide a reasonable basis for our initial discussion.


E X H I B I T G

F I N A L R E P O R T S T R U C T U R E F O R A N I N I T I A L E F F E C T I V E N E S S R E P O R T I N G P R O J E C T

— A N I L L U S T R A T I O N

E X E C U T I V E S U M M A R Y

• explanation of what the report is and the context within which it should be viewed;• summary description of the organization, activity, line of business or program;• highlights of management’s representations; and• significance of management’s representations in terms of the interests of the governing body, actions

to be taken by management, future such reporting on this or other lines of business.

M A I N R E P O R T

S E C T I O N 1 — I N T R O D U C T I O N A N D B A C K G R O U N D

• Explanation of what the report is and the rationale for it, for example:- linkage to specific external events or initiatives or to a specific aspect of the organization’s strate-

gic agenda- motivation to provide the governing body with better and more comprehensive performance infor-

mation and so facilitate their oversight and decision-making roles- motivation to demonstrate management accountability- motivation to provide a focal point for discussion between management and the governing body

regarding expectations for and understanding of performance - motivation to provide a focal point for similar discussions among management and down

through the organization• Explanation of how the report should be viewed, for example:

- as a pilot project aimed at learning more about the effectiveness framework and assessing itsapplicability

- as providing an initial basis for discussion with the governing body about their needs for infor-mation and their understanding of the overall performance of the organization, activity, line ofbusiness or program

- as the first (or a subsequent) step in a broader and longer-term strategy already discussed andagreed to between the governing body and management

• Explanation of approach taken to developing the report:- general process followed- major players and their roles- nature and level of contributions made by these people


S E C T I O N 2 — D E S C R I P T I O N O F T H E O R G A N I Z A T I O N , A C T I V I T Y , L I N E O F B U S I N E S S , O R

P R O G R A M O N W H I C H T H E R E P O R T F O C U S E S

• Statement and explanation of the mandate• Statement of the objectives• Description of the structure and interrelationship of activities• Profile of the beneficiaries, intended clientele, and actual users• Description of key aspects of the program design (for example, eligibility criteria) and delivery process

(for example, centralized or decentralized, direct or through intermediaries)• Profile of physical, financial, and human resources, as applicable

S E C T I O N 3 — M A N A G E M E N T R E P R E S E N T A T I O N S O N P E R F O R M A N C E

• Introductory subsection that explains and puts in perspective the following or other matters, asapplicable:

- any choices made, such as to exclude certain attributes, to combine or split certain attributes, togive priority to certain attributes over others, to group the attributes or present them in a certainorder

- the overall message and balance that the management representations, taken as a whole, areintended to convey

- the initial report is likely to be based on existing data and information rather than involve exten-sive measures to capture and analyze a lot of new information, and as a result it is likely to havegaps—some perhaps quite important—that future reports might be expected to fill

• Explanation of the following with respect to each pertinent attribute:- what the attribute means and why management thinks information in relation to the attribute is

important to itself, the governing body, and other key stakeholders- a summary discussion of the key assessment factors, criteria, and indicators that management

actually applied in analyzing performance in relation to the attribute- key data, information, or evidence in relation to the above- the overall conclusion that management has reached with respect to performance (in some cases, par-

ticularly in the initial report, there may be caveats because of lack of appropriate benchmarks, insuffi-cient or contradictory information, and so on)

S E C T I O N 4 — L E S S O N S L E A R N E D

• Discussion of the following or other matters, as applicable:- overall relevance and value-added of the process and the resulting product- roles played in the process by management, governing body, and those who support them- managing expectations regarding the process and resulting product- management of the process itself- nature and level of resources needed compared to what was available to do work such as this


- areas where improvements could be made to adjust the focus of such reports to a higher orlower level, clarify expectations, reinforce or streamline roles played by key players, make theprocess more efficient

A P P E N D I C E S , S U C H A S :

• Organization chart• List of the key assessment factors, criteria, and indicators that were used as the basis for analyzing per-

formance with respect to each attribute• Tables related to key data and information cited in Section 3 of the Main Report


235 THIS DOCUMENT COULD BE DERIVED FROM THE WORK THE FACILITATOR HAS DONE IN STEP 5. SEE THE LAST PORTION OF EXHIBIT C, OUTLINE OF A PROGRAM PROFILE, FOR FURTHER SUG-GESTIONS IN THIS REGARD.

236 THIS DOCUMENT IS PREPARED BY THE FACILITATOR. SEE EXHIBIT C, OUTLINE OF A PROGRAM PROFILE, FOR SUGGESTED STRUCTURE AND CONTENT

237 SEE OUTLINE AND DISCUSSION IN ELEMENTS OF A MANAGEMENT REPRESENTATION, WHICH CAN BE FOUND ON PAGES 159-161 OF PART II 238 AS NOTED IN THE TEXT, IN SOME CIRCUMSTANCES IT MAY BE PREFERABLE TO HOLD THE WORKSHOP IN TWO PARTS RATHER THAN IN ONE THREE-DAY SESSION.239 COPIES OF THIS BROCHURE, REPORTING ON EFFECTIVENESS, CAN BE OBTAINED FROM CCAF.

P A R T I I I I

COMPREHENSIVEAUDIT

A C C O U N T A B I L I T Y , P E R F O R M A N C E R E P O R T I N G , C O M P R E H E N S I V E A U D I T - A N I N T E G R A T E D P E R S P E C T I V E 2 1 7

THE CONDITION UPON WHICH GOD HATH GIVEN LIBERTY TO

MAN IS ETERNAL VIGILANCE . . .240

J O H N P H I L P O T C U R R A N

INTRODUCTION

Parts I and II of this book focused on thenature and importance of accountability in thepublic sector and the means by which managementdischarges its accountability responsibilitiesthrough reporting to the governing body. Part IIIdeals with audit, in particular the role that auditplays in assuring the governing body of the qualityof an organization’s performance. It demonstrateshow audit effectively closes the accountability loopby providing an independent, objective and profes-sional opinion in which the governing body canhave confidence.

The emphasis in Part III is on audit thatserves the governing body, the level in an organiza-tion that delegates responsibility for administeringpolicy to top management. Many of the issues andprinciples discussed are also relevant to other audit-ing activities, such as internal audit, which alsoserve an accountability relationship. Sometimesthat relationship is between management and thegoverning body, although it is more frequentlybetween executive management and other levels.As long as such a relationship exists, the principlesexamined here apply.

All auditing requires discipline and rigour.This applies as much to audits concerning perfor-mance as it does to the audit of financial state-ments. Although our focus is on audits that con-cern performance in respect of economy, efficiencyand effectiveness—on what is called comprehensiveauditing—it is important to understand certainprinciples of audit generally; chapter 13 is devotedto that subject.

To provide a historical context, chapter 14outlines the development of comprehensive audit-ing. The three different approaches to comprehen-sive auditing are described in chapter 15, as well as

P A R T I I I . C O M P R E H E N S I V E A U D I T 2 1 9

the factors that determine which approach is taken. Chapters 16 and 17 describe the comprehen-

sive audit process; some general considerations,and the conduct of the audit itself. Chapter 16explains what differentiates this kind of auditingfrom financial statement auditing and the factorsthat drive the audit itself and the resulting report.As with any professional discipline, a number ofdecisions must be made in the course of conduct-ing a comprehensive audit. These choices and theirimplications are explored, as are the major practiceissues that must be dealt with in this type of audit-ing. Particular attention is given to the role thatinternal auditors play and the way that externalcomprehensive auditors can rely on their work andon other studies that are undertaken with respectto the audited organization.

Chapter 19 explains the standards that com-prehensive auditing must adhere to and the man-ner in which practitioners strive to ensure the high-est professional quality of their examinations.Audits are, of course, conducted by people. Thedemands placed on comprehensive auditors are dis-cussed, as are the skills auditors must bring to theirassignments; the role of the clients and the areas inwhich they can and should influence audits isexplained here.

Comprehensive auditing is a dynamic con-cept, one that has grown and changed throughexperience. It most certainly will evolve even fur-ther in the future. While it is not a panacea forgood accountability, management and governance,it can contribute substantially to those ends. PartIII is intended to give readers a clear understandingof how the concept originated, grew and is prac-tised at the time of writing.

It is important to recognize that the conceptof comprehensive auditing is sufficiently flexible toallow different interpretations and approaches indiffering circumstances. In some cases, Part IIIreflects comprehensive audit practice that is cur-rently being uniformily applied. In other cases, itreflects only what some practitioners are doing. Instill other cases, it encourages practitioners toexperiment with new ideas and approaches. Varietyis a hallmark of this kind of auditing. What iscommon to all, however, is the disciplined, objec-tive professionalism that is brought to the work.


240 JOHN PHILPOT CURRAN, SPEECH ON THE RIGHT OF ELECTION OF LORD MAYOR OF DUBLIN, 10 JULY 1790.

S E C T I O N 1

BACKGROUND


THE ORIGIN OF AUDIT

THE NEED FOR FINANCIAL ACCOUNTABILITY HAS EXISTED EVER

SINCE IT BECAME NECESSARY FOR ONE INDIVIDUAL TO

ENTRUST THE CARE OF HIS POSSESSIONS OR BUSINESS TO

ANOTHER. IN PRIMITIVE SOCIETIES, WHEN A MAN FOUND HE

COULD NOT PERSONALLY CONTROL HIS HERDS, CROPS AND

OTHER POSSESSIONS, HE BEGAN TO DELEGATE RESPONSIBILITY

FOR HIS AFFAIRS TO OTHERS AND TO REQUIRE AN ACCOUNT-

ING FROM THEM. ADVANCES BEYOND A RURAL ECONOMY TO

TRADE AND EARLY FORMS OF INDUSTRY MADE IT NECESSARY

FOR PROPRIETORS TO EXACT AN ACCOUNTING FROM THEIR

SERVANTS THROUGH INTERMEDIARIES. AS MANY OF THESE SER-

VANTS WERE ILLITERATE, THEIR ACCOUNTING WAS NECESSARI-

LY ORAL AND THE INTERMEDIARIES WHO HEARD THE

ACCOUNTS BECAME KNOWN AS AUDITORS. IN ADDITION TO

HEARING AND TRANSCRIBING THE ACCOUNTS, AUDITORS

CHECKED THEIR ACCURACY BY VERIFYING THE QUANTITY OF

MONEY, CATTLE OR GOODS THAT SHOULD BE ON HAND AT A

GIVEN MOMENT.

AS THE WORLD OF COMMERCE DEVELOPED AND THE LEVEL OF

LITERACY ROSE, SERVANTS ACQUIRED THE ABILITY TO REPORT

DIRECTLY, AND AUDITORS WERE NO LONGER REQUIRED TO

HEAR REPORTS AND TRANSCRIBE THEM FOR THEIR MASTERS.

AUDITORS DID NOT, HOWEVER, DISAPPEAR FROM THE SCENE.

INSTEAD THEY MOVED INTO A ROLE THAT RESEMBLED THEIR

MODERN ONE: THEY CONCENTRATED UPON VERIFYING THE

ACCURACY OF THE INFORMATION PROVIDED BY OTHERS.241

C H A P T E R 1 3

UNDERSTANDINGAUDIT

R E L A T I O N S H I P T O

A C C O U N T I N G

Historically, the word audit has been primari-ly associated with, though not a part of, financialaccounting. The greater significance of corpora-tions and government in society has created a needfor more and increasingly meaningful financialaccounting in both the private and public sectors.In the business sector, for example, the growingdiffusion of corporate ownership into hundredsand thousands of unrelated shareholders (includinginstitutional investors who may have large holdingsbut tend to keep their distance from management)has created important groups of interested peopletotally unconnected to the management of thesecorporations or to market and other regulators.Effective communication between these groups andmanagement is essential to the proper functioningof the economic system. Financial accounting pro-vides this communication. Without it, much com-mercial activity would be impossible. Lenders, forexample, demand reliable financial statements as aprerequisite to granting credit. In a user-orienteddefinition that stresses this communication role,R. J. Anderson defines accounting as “the processof identifying, measuring, and communicatinginformation to permit informed judgements anddecisions by users of the information.”242

The conventional understanding of the rela-tionship between accounting and auditing is basedon the following three presumptions:


• management, not the auditor, should prepareaccountability reports;

• there are generally accepted bases for prepar-ing such reports (in Canada, GenerallyAccepted Accounting Principles—GAAP);and

• the auditor’s job is to add credibility to thesereports, using generally accepted auditingstandards to examine them, and thus providethe client with assurance whether they havebeen presented fairly (this has been calledattest auditing).

Anderson explains the need for attestation ina manner that really defines and justifies the role ofauditing in society:

To be effective, accounting informationmust be both accurately prepared by thesender and believed and acted upon by thereceiver. To believe and act upon the informa-tion received, the receiver or user must be sat-isfied with its quality. Four conditions pre-clude many users of accounting informationfrom achieving this satisfaction directly andtherefore create the need for an objectiveaudit and attestation.

The most obvious condition is remote-ness… the remoteness may be geographical…or it may be legal: under corporation acts,shareholders have no right of access to thebooks of account… or it may be economic:although the government has a statutoryright of access to company’s records to vali-date tax information, in many cases it willchoose to avoid the cost of exercising its rightif objective attestation of the information isavailable.

A second condition is conflict of interest.Preparers of financial accounting informationwill frequently have interests at variance with

those of many users.… Wherever users offinancial accounting information perceiveactual or potential conflicts of interest, theywill have a natural reluctance to accept theinformation without some objective attesta-tion of its quality by an auditor.

A third factor is complexity. Increasingcomplexity of the information systems andthe preparation of data for financial reportsincreases the probability of error. At the sametime, user satisfaction as to the quality of theinformation, even given access to all underly-ing records, is diminished substantially. Insuch cases, users require someone else, actingon their behalf, to employ an appropriatelevel of expertise in assessing and attesting tothe quality of the information.

A final factor is the consequence of error.If users act upon the information received,imprudent actions based upon poor qualityinformation may have direct financial conse-quences. The greater the potential conse-quence, the greater the need for satisfaction,through attestation, as to the quality of theinformation received.

The role of auditing is therefore to addcredibility to financial statements and thusto enhance the effectiveness of accountingcommunication needed by our economicsystem.243

ACCOUNTING IS PRIMARILY CONSTRUCTIVE AND IS THE

WORK OF MANAGEMENT, WHEREAS AUDITING IS FUNDA-

MENTALLY ANALYTICAL AND IS CARRIED OUT INDEPEN-

DENTLY OF MANAGEMENT IN ORDER TO JUDGE

MANAGEMENT’S PERFORMANCE.244

P A R T I I I . C H A P T E R 1 3 . U N D E R S T A N D I N G A U D I T 2 2 3

D E F I N I T I O N S O F A U D I T

Over the years, writers have devised a numberof definitions of audit, usually with some specificcontext or purpose in mind. The WilsonCommittee, for example, produced the following:

Audit is a process that is superimposedon an accountability relationship. It is carriedout to establish that a report on the responsi-bility assumed is a correct or fair one and isusually performed by a third party, primarilyserving the interests of the party who delegat-ed the responsibility.245

Other definitions—focusing on the audit of finan-cial statements—are as follows:

Auditing can be defined as a systematicprocess of objectively obtaining and evaluat-ing evidence regarding assertions about eco-nomic actions and events to ascertain thedegree of correspondence between thoseassertions and established criteria and com-municating the results to interested users.246

Auditing is defined as an exploratory,critical review by a public accountant of theunderlying internal controls and accountingrecords of a business enterprise or other eco-nomic unit, precedent to the expression byhim of an opinion of the propriety of itsfinancial statements.247

Auditing is the systematic examinationand verification of the accounting records,vouchers, and other financial and legal recordsand documents of a private or business organi-zation. The audit is performed in order toascertain the accuracy, integrity, and authentic-ity of those records and documents. It is madewith the intention of presenting fairly thefinancial condition at a given date and theresults of operations for a period ending onthat date, on the basis of consistency and con-formity with accepted accounting principles.248

Auditing involves a critical analysis andexamination of the transactions and recordsof a concern, the interpretation of the results,and normally an expression of opinion con-cerning the records and financial statementsof the client. This work is done by a personor by persons independent of managementand is intended to determine the reliability ofmanagement’s representations.249

Auditing [is] the objective examinationof records accompanied by the expression of acompetent opinion concerning the financialcondition and operating results of a client’sbusiness.250

While all of these definitions apply in the contextof auditing financial statements, they are too nar-row in focus where the performance informationinvolved goes beyond matching revenues and costsor valuing assets and liabilities. From an examina-tion of the nature of audit, a somewhat broaderdefinition can be derived, as is shown in the fol-lowing chapter.

C H A R A C T E R I S T I C S O F A U D I T

The above definitions of audit, while differ-ing in particulars, are all based on a commonunderstanding of the fundamental characteristics ofaudit.

The first prerequisite of audit is that it servesan accountability relationship. That must be theprimary purpose of an audit engagement. Whileother purposes may be served by an audit, they areessentially of secondary importance. If serving anaccountability relationship is not the prime pur-pose, the engagement is not truly an audit, nomatter that it uses audit methodologies and eventhough there may be some strategic or tactical ben-efits to be gained from calling such a service anaudit.


Other characteristics of audit arise from theneed to ensure that people who delegate responsi-bility receive an authoritative audit report. Thesecharacteristics flow from the basic requirement thatauditors be independent, in fact and appearance,and that they be objective and competent. Theyinclude:

• sufficient appropriate evidence must be gath-ered in an orderly and systematic way;

• auditors have the responsibility and freedomto determine the scope and depth of worknecessary to support their opinions; and

• auditors have an obligation to carry throughan audit engagement to the point of report-ing their opinions: this obligation to report isnot contingent on the nature of the findingsor of discussions that may take place withmanagement in the course of the audit.

Another important characteristic is the exis-tence of appropriate, relevant and agreed criteriaon which to base the auditor’s examination andresulting opinion. Without such criteria, auditorscould not arrive at opinions that a third partywould judge to be objective, and the resultingreports could be considered biased or highly subjective.

Finally, an audit must be conducted in accor-dance with rigorous, established standards. Forfinancial audits in Canada, those standards are theCICA’s Generally Accepted Auditing Standards(GAAS). The purpose of GAAS is to ensure that astringent discipline is both applied by the auditorand expected by the client and that the resultingaudit report is reliable. Auditing is a profession,and audits must be conducted by professionals.

P A R T I I I . C H A P T E R 1 3 . U N D E R S T A N D I N G A U D I T 2 2 5

241 WILSON, 10.242 R. J. ANDERSON, THE EXTERNAL AUDIT, 2ND ED. (COPP CLARK PITMAN LTD., 1984) 3.243 IBID., P. 5. THE FOUR CONDITIONS IDENTIFIED AS CREATING THE NEED FOR AN OBJECTIVE AUDIT OR ATTESTATION WERE THOSE DISCUSSED IN A STATEMENT OF BASIC AUDITING CONCEPTS

(ASOBAC) (AMERICAN ACCOUNTING ASSOCIATION, 1972) SEE ANDERSON’S FOOTNOTE 5 IN CHAPTER 1.244 ARTHUR W. HOLMES, AUDITING, PRINCIPLES AND PROCEDURES. (HOMEWOOD, ILL.: RICHARD D. IRWIN, INC. 1952) 1.245 WILSON, 9.246 A STATEMENT OF BASIC ACCOUNTING CONCEPTS (ASOBAC), AMERICAN ACCOUNTING ASSOCIATION, 1972, AS CITED BY R. J. ANDERSON IN THE EXTERNAL AUDIT, 2.247 ERIC L. KOHLER, A DICTIONARY FOR ACCOUNTANTS, (NEW YORK: PRENTICE-HALL, INC. 1957) 44-45.248 HOLMES, 1.249 IBID.250 IBID., 2.

C H A P T E R 1 4

UNDERSTANDINGCOMPREHENSIVEAUDIT

H I S T O R I C A L D E V E L O P M E N T

Comprehensive auditing was developed pri-marily as a response to client demand for betteraccountability information. In the 1970s, membersof the federal and several provincial legislaturesrealized that they were not getting the performanceinformation they needed. They sensed an account-ability vacuum. They could choose to ask manage-ment to report on performance (the financialaccountability model), or they could have a thirdparty (the auditor being a prime candidate)attempt to fill the gap.

Frustrated by the apparent unwillingness orinability of management to properly report on per-formance, many legislatures asked their auditors tosupply broader accountability information. Thus wascomprehensive auditing born, based on two impor-tant principles of management in the public sector.

The first principle is that public business shouldbe conducted in a way that makes best possible use ofpublic funds. Officials responsible for spending pub-lic funds must ensure that their decisions result ineconomical, efficient and effective public services.The second principle is that people who conductpublic business should be accountable for the pru-dent and effective management of the resourcesentrusted to them. This onus of accountability per-meates the public sector, from elected representativesaccountable to the public, to the officials accountableto their elected or public service superiors.

THERE MUST BE A REALIZATION BY AUDITORS THAT ALL LEV-

ELS OF GOVERNMENT ARE ACCOUNTABLE TO THE PEOPLE FOR

THE EFFICIENT AND EFFECTIVE DELIVERY OF SERVICES. THERE

HAS BEEN A REALIZATION THAT STRICTLY FINANCIAL DATA IS

NOT SUFFICIENT TO INFORM GOVERNMENT MANAGERS AND

DECISION-MAKERS… AS WELL AS THE PUBLIC… ABOUT

ACHIEVEMENTS AND FAILURES IN CARRYING OUT GOVERNMENT

PROGRAMS.251

T H E H O N O U R A B L E E L M E R B . S T A A T S

C O M P T R O L L E R G E N E R A L O F T H E U N I T E D

S T A T E S ( 1 9 6 6 - 8 1 )

The first Canadian legislature to give its audi-tor a broader mandate was the Parliament ofCanada. Following the 1975 report of the WilsonCommittee, Parliament enacted the AuditorGeneral Act in 1977. Among its several provisions,the act enabled the auditor general to reportwhether money was spent with due regard to econ-omy and efficiency in the acquisition and manage-ment of goods and services, and whether the effec-tiveness of programs is being measured and report-ed in all instances where such measurement is fea-sible and practical.

James J. Macdonell was the auditor to whomthis mandate was given. Even before the enactmentof the legislation, Macdonell had been working todevelop the concept of more comprehensive audit-ing. To this end, he enlisted the participation ofsome of the brightest and best of the auditing andconsulting professions from across Canada. Underthe guidance of a high-level advisory committee,seasoned professionals from public accounting andmanagement consulting firms worked on second-ment with staff of the Office of the AuditorGeneral of Canada to lay the foundation for dis-charging this new responsibility.


THE HARD FACT IS THAT A NEW DIMENSION IS BEING FORGED

IN TERMS OF VALUE-FOR-MONEY ACCOUNTABILITY FOR BOTH

MANAGERS AND AUDITORS IN THE PUBLIC SECTOR, AND THE

SOONER WE RECOGNIZE IT AND ADAPT TO IT WITH VIGOUR AND

ENTHUSIASM, THE BETTER SATISFIED WILL BE THE SHAREHOLD-

ERS (THE TAXPAYERS) AND THEIR REPRESENTATIVES (THE LEGIS-

LATORS). IT IS A GREAT OCCASION FOR PROFESSIONAL PEOPLE

TO FEEL IN THEIR HEARTS THAT THEY ARE DOING SOMETHING

SIGNIFICANT TO EXPAND THE HORIZONS AND THE COMMON

BODY OF KNOWLEDGE OF THEIR OWN PROFESSION.252

J A M E S J . M A C D O N E L L

A U D I T O R G E N E R A L O F C A N A D A ( 1 9 7 3 - 8 0 )

The result of this background work wasunveiled in December 1978 at the centennial con-ference of the Office of the Auditor General ofCanada. The new approach was called comprehensiveauditing. The term was chosen to indicate that auditexaminations would take into account not only thetraditional matters of financial records, the safe-guarding of valuables and compliance with authori-ties, but also issues of nonfinancial performance.

Not long after the federal initiative, several ofthe provinces also provided their auditors withsimilar mandates.

An important landmark in the developmentof comprehensive auditing was the 1980 establish-ment of the Canadian Comprehensive AuditingFoundation (CCAF) as a cooperative, nonprofitorganization devoted to research and education inthe field of comprehensive auditing. CCAF, as it isknown today, is supported by Canada’s leadingfirms of accountants and management consultants,together with the federal and all provincial govern-ments through their legislative auditors. The foun-dation is, in essence, a pooling of interests andresources of these parties. Its main purpose is tobuild knowledge for meaningful accountability and

effective governance, management and audit.Accordingly, its focus is on the interests of thethree principal parties to accountability relation-ships—members of governing bodies, managementand auditors. CCAF is interested in the whole issueof accountability for public sector organizations—ranging from the federal and provincial govern-ments, to health and educational institutions,social service providers, municipalities and, ofcourse, state-owned enterprises. Increasingly, pri-vate sector businesses are benefitting from the workof the foundation. These include regulated indus-tries such as banks and trust companies, as well aspublicly traded corporations in the manufacturingand service sectors.

When CCAF was established, professionalbodies such as the Canadian Institute of CharteredAccountants, the Certified General Accountants ofCanada and the Society of ManagementAccountants of Canada quickly became involved.

In assigning to auditors the responsibility ofhelping to fill the perceived accountability vacuum,governing bodies have nevertheless been cautiousin how much of this role they have given to audi-tors. They recognized that the first duty to reportlies with management, not auditors, and that thereare certain areas in which they did not want audi-tors to become involved. For example, legislatedand voluntary regimes of comprehensive auditingseldom explicitly mandate auditors to report effec-tiveness information where management does notdo so. Moreover, they often restrict auditors fromcommenting on the merits of policy.

Comprehensive auditing is now practised invirtually all provincial governments and in the fed-eral government. It is practised by both externalauditors (legislative auditors) and internal auditors.Many medium- to large-size municipalities haveadopted the concept, as have a number of healthcare, educational and social service institutions.Many of the auditors for these institutions are in

P A R T I I I . C H A P T E R 1 4 . U N D E R S T A N D I N G C O M P R E H E N S I V E A U D I T 2 2 7

private practice. Federal Crown corporations are bylaw required to conduct periodic “special examina-tions” that invoke all of the principal elements ofcomprehensive auditing.

Canada is not alone in adopting broad-scopeaudits. Internationally, audits that go beyond tradi-tional financial statement auditing are accepted inmost western countries and, increasingly, in devel-oping nations. United Nations organizations havealso mandated both their internal and externalauditors to report on observations dealing withperformance issues. These audits, while not alwaysreferred to as such, usually cover the same types ofmatters having to do with economy, efficiency andeffectiveness as do comprehensive audits. Therehave been two international conferences on thesubject—1992, in Mexico City, and 1995 inBuenos Aires—and many other international con-ferences deal with it regularly.

T H E C O M P R E H E N S I V E A U D I T

C O N C E P T

What is meant by comprehensive audit? Anexamination of the term will help clarify it.

CO M P R E H E N S I V E

The first word in the term—comprehensive—was chosen to convey the idea that the examinationincludes more than the traditional audit of financialstatements and an examination of how closely anorganization has complied with pertinent statutoryauthorities and regulations. A comprehensive auditlooks at how carefully an organization has givenattention to economy, efficiency and effectiveness asthe terms are explained in Part II.

The literature defines comprehensive auditingas a concept rather than a technique—one thatembraces three related, but individually distin-guishable, aspects of public sector accountability:

• financial reporting; • compliance with authorities; and • the economical, efficient and effective man-

agement of public funds and resources.

When originally devised, the term comprehen-sive audit referred to the entire range of the man-date for broad-scope audits.

NOTE

BECAUSE FINANCIAL- AND COMPLIANCE-AUDITING

PROCESSES HAVE BEEN WELL ESTABLISHED FOR SOME

TIME, THE TERM COMPREHENSIVE AUDITING IS OFTEN

TAKEN TO CONNOTE THE NEWER ELEMENT THAT DEALS

WITH BROADER PERFORMANCE ISSUES. THIS BOOK

ADOPTS THAT USAGE. THE USE OF THE TERMS COMPRE-

HENSIVE AUDIT AND COMPREHENSIVE AUDIT REPORTS

AND COMMENTS ABOUT THEM SHOULD NOT BE INTER-

PRETED AS EXTENDING TO THE FINANCIAL AND COMPLI-

ANCE ELEMENTS OF THE BROADER COMPREHENSIVE

AUDIT CONCEPT.

A U D I T

The second half of the term—audit—distin-guishes the concept from all other review processes.It emphasizes the need for professional standardsand discipline. In this context, it may be instruc-tive to examine how contemporary comprehensiveaudit practice corresponds to the description ofaudit in the previous chapter.

First, it is useful to sketch how a comprehen-sive audit works.

ProcessIn general terms, a comprehensive audit is

conducted in the same way as all audits. Theprocess starts with the audit engagement. This maybe the result of a legislated mandate, as with leg-


islative auditors, or may arise from a contractualarrangement. The work begins with the planningphase.

P L A N N I N G P H A S E

During the planning phase, the decisions aretaken about what is to be audited and how theaudit will be conducted. A great number of issuesmust be addressed at this time, including:

Audit approach: There are three broad approachesto comprehensive auditing, although in practicethese approaches may not always be mutuallyexclusive.

The first approach focuses on the quality ofmanagement systems and practices. These systemsand practices are examined with the view to usingthem as an indicator of the extent to which theorganization pays due regard to economy, efficien-cy and effectiveness. This is the most commonform of practice to date.

The second approach entails the comprehen-sive audit concentrating on delivering an opinionof the completeness and integrity of management’srepresentations (reporting) to their governing bodyor stakeholders on the performance (effectiveness)of the organization. Such representations includesufficient information to allow readers to drawconclusions and for the auditors to substantiate.

The third approach involves the comprehen-sive audit itself undertaking to measure and reporton the organization’s performance. While auditorshave frequently looked at individual aspects of per-formance (for example, economy and efficiencyissues), there are few examples of their using theattributes of effectiveness discussed in Part II orother similar frameworks. Typically, this approach isappropriate where the governing body wants assur-ance about the organization’s performance but man-agement does not have the capacity or is unpreparedto make the representations that satisfy this need.

Understanding the organization: A good under-standing of the organization to be audited is essen-tial if the right things are to be examined andappropriate conclusions are to be drawn. In someinstances, the auditors are already familiar with theorganization, as they have served the client forsome time. In other cases, however, the auditorsmay know little about the organization and mustgain a sufficient understanding of it in order toplan an effective audit.

Audit objective: The audit objective is a corner-stone element of the audit. It flows from the auditmandate and concerns the nature of the auditinformation that the auditor intends to report tothe client. It influences the audit process through-out—from planning to reporting.

Scope: An important decision to be taken at theoutset is the scope of the projected audit. Scopedetermines what the audit will cover: will it be thewhole organization, one or more of its divisions orbranches, or perhaps one or more or its functions,such as personnel, purchasing, and so on? Unlikefinancial audits, where the scope is nearly invari-ably the entire organization’s financial statements,there is usually no fixed scope for comprehensiveaudits. Determining scope is, therefore, an impor-tant issue and has a direct bearing on the ultimatecost of the audit.

Intended level of audit assurance: In financialaudits, there is a generally understood degree ofcertainty that readers may derive from an auditor’sopinion. Not so with comprehensive audits. Thedegree of assurance that auditors have regardingtheir opinions varies with such matters as theintensity with which the examination is made andwith the nature of the issues examined. However,this does not mean that it is an unlimited conceptthat is absolute at either the high or low end of thescale. In planning the audit, auditors have todecide how confident they will want to be about


the opinions they form and the depth of the exam-ination that will be required to achieve this confi-dence level.

Significance: A third variable that distinguishescomprehensive audits from financial audits is thelevel of significance253 that will be used. Thismeans how important a problem has to be to war-rant being mentioned in the audit opinion. Likethe level of audit assurance, the decisions aboutsignificance affect not only the amount of workthat will have to be undertaken—and hence cost—but also the proper interpretation of the report.

Criteria: Criteria are the yardsticks against whichauditors assess actual performance. They are objec-tive benchmarks that help auditors decide whethersomething is acceptable or whether it should beidentified as a deficiency. They may be eitherquantitative or qualitative and are usually process-oriented, such as best practices or generally accept-ed norms. Depending on the subject, criteria maybe drawn from many sources: accepted good indus-try practice, relevant literature, the organization’sown accepted standards, and so on.

Evidence collection: Decisions must be madeabout what evidence to collect, and how to gather,analyze and manage it. When this has been done, aplan for the examination, in which the requiredevidence is actually gathered, can be made.

Administrative matters: Once all these decisionshave been taken, the auditors can decide on whowill do what, how they will go about it, how longit should take and how much it should cost.

That done, the planning phase is complete.

C O N D U C T A N D R E P O R T I N G P H A S E S

During the conduct phase, auditors collect theevidence they need to support their opinions.Frequently, adjustments have to be made to the

original plan drawn up in the planning phase. Thisdoesn’t mean that the planning was necessarily inad-equate, but rather that new information or insightsare gained, or unexpected problems or opportunitiesare encountered either in the operation of the orga-nization or in the collection of evidence.

Once sufficient evidence is gathered, auditorscompare it to the criteria and formulate opinions.These audit opinions, and the evidence supportingthem, are discussed with management to ensuretheir reasonableness.

The last phase is the preparation and presen-tation of the final audit report. Although reportingseems a distinct phase, it is in fact not usually soisolated. If good communication is maintainedthroughout the audit, reporting of one kind oranother will happen regularly. The final report andits discussion with the governing body is usuallythe last, culminating step.

Is it true audit?With the description of comprehensive audit-

ing above, we may examine whether it is a trueform of audit as we have come to understand theterm. We can ask: To what extent do the three pre-sumptions about the relationship of accounting toauditing mentioned in the previous chapter applyto comprehensive auditing?

Regarding the first presumption (manage-ment’s responsibility to report), in the environmentin which comprehensive audit is performed thereare still few cases of management producing fulland rigorous reports to the governing body on keyaspects of the organization’s performance: there isno legally imposed obligation on management todo so. Does this mean that no true audit can bedone on performance issues in such an environ-ment? Clearly not. This set of presumptions neverenvisaged an environment in which an auditor isengaged to perform an audit but management isnot required to report.


The Wilson Committee faced the dilemmaproduced by this environment. In looking at theaudit of effectiveness, the committee concludedthat the primary role of the auditor is to audit thereports of management on their effectiveness. Insaying this and in reflecting this notion in its defi-nition, the committee made two vital caveats. First,it noted that although the systematic evaluation ofprogram results presents problems, these shouldnot be taken as an excuse to question the auditorgeneral’s right of reporting on obvious cases ofineffectiveness uncovered during the course of hisor her examination of the accounts. Second, thecommittee noted that it was important “that thenew legislation be broad enough to ensure that theAuditor General has the right to report on suchstudies and even make his or her own evaluation ofprogram results if there is no other satisfactory wayof obtaining this.”254

As a result of the incongruity between thepremise of management reporting and the environ-ment in which comprehensive auditing is prac-tised, it is not appropriate to apply all the conven-tional presumptions to differentiate between auditand nonaudit services.

There is, however, some evidence to suggestthat these presumptions could become relevant. Inits 1987 report, CCAF’s Independent Panel onEffectiveness Reporting and Auditing in the PublicSector strongly recommended that managers pro-vide information on effectiveness (which theydefined as including the most important elementsof economy and efficiency) in the form of repre-sentations to help governing bodies judge perfor-mance. The panel also recommended that auditorsprovide attestation audit opinions on these repre-sentations, in effect saying that this is a desirabletarget for comprehensive auditing. Since then,there has been a growing number of cases in vari-ous parts of the public sector where managementhas produced accountability reports, although it is

too early to determine if this will become a com-mon practice.

The second presumption in the conventionalunderstanding of audit—the existence of generallyacceptable bases for the preparation of performancereports by management—cannot now fully apply tocomprehensive auditing. Substantial progress hasbeen made in developing appropriate reporting mod-els for use by management in a manner analogous toGAAP. There nevertheless remains a significant chal-lenge to adapt these principles to different parts ofthe public sector. In the meantime, agreed criteria—even though they are determined on a case-specificbasis and usually pertain to management processes asdistinct from performance results—are used as abasis for carrying out and reporting on comprehen-sive audits. That is, in each situation there is anagreed basis for reporting: agreed standards thatmanagement, and its systems and practices, can rea-sonably be expected to meet. Such criteria have beendeveloped and used for years.

The third presumption—of adding credibilityto information and providing assurance on the fair-ness of the reporting—is an implicit facet of cur-rent practice. Clearly, in the absence of a manage-ment report, the manner in which the auditormeets this presumption will be different, but it ismerely another way of achieving the same purpose.As is described in a subsequent chapter, the princi-pal question is how to communicate the intendeddegree of audit assurance.

This assessment takes into account the view,held particularly in the legislative audit communi-ty, that even if management does report on perfor-mance in a complete and fair way, attestation tothese reports by auditors will likely have to be sup-plemented by additional reporting to fulfill theirresponsibilities completely. For example, legislativeauditors believe that additional reporting will stillbe needed to inform legislators about issues thattranscend organizational lines and for which a


number of ministries share responsibility.Individual accountability reports may need to beaugmented by additional information to help gov-erning bodies examine issues in a broader contextthan might otherwise be afforded.

Although not all the traditional presumptions ofaudit may apply to present comprehensive audit prac-tice, the two key aspects of audit are clearly present.

The first is that the engagement is superim-posed on an accountability relationship. It is aquestion of fact whether an accountability relation-ship exists and if the party who has engaged theauditor is also the party who allocates the responsi-bility. If so, and if the auditor’s report addresses themanner in which the undertaking of the responsi-bility has been discharged, then the engagementmeets the test of serving an accountability relation-ship. That it is the auditor, not management, whois reporting does not mean that the accountabilityrelationship is not being served. This form ofreporting is simply a practical means of accom-plishing the same purpose.

The second aspect is whether comprehensiveaudit is done in compliance with objective stan-dards. In 1988, the CICA’s Public SectorAccounting and Auditing Committee issued itsValue-for-Money Auditing Standards, dealing withthe professional qualities of auditors, the conductof their audit examinations and the content oftheir audit reports. It is to these standards thatcomprehensive auditors must conform.

From the foregoing analysis, and understandingcurrent practice, the following definition of audit hasbeen adopted in the comprehensive audit context:

Audit serves an accountability relation-ship. It is the independent, objective assess-ment of the fairness of management’s repre-sentations on performance or the assessmentof management’s systems and practices,against criteria, reported to a governing bodyor others with similar responsibilities.255

N O N A U D I T F O R M S O F

P R A C T I C E

Many practitioners provide their clients withconsulting services in the value-for-money area;indeed, from time to time, members of the legisla-tive audit community undertake research thatrelates to performance in addition to their regularaudit duties. These services are valuable, and it isentirely appropriate for practitioners to providethem when they can be of assistance to theirclients.

Such services, however, are sometimes ren-dered under an audit label. This is troublesome asit may create unjustified expectations and can con-fuse the client and management communities.Therefore, it is in the best interests of comprehen-sive auditing to use the term audit only when anaudit, as described in the preceding chapter, hasbeen conducted.

Beyond offering the above working definitionof audit, it is not the intention here to propose anexhaustive list of value-for-money services thatshould or should not be considered to be audit.Nevertheless, any engagements that are notdesigned to serve an accountability relationship,and/or that do not meet all the criteria identifiedabove should not be called audits. For example,while recognizing that comprehensive audits oftenlead to management improvements, work that isdesigned primarily to achieve cost reductions, orproductivity increases, or improvements to otherelements of effectiveness, would not likely belabelled as such.

Similarly, engagements that are ostensiblyheld out to serve an accountability relationship,but whose terms of reference are so circumscribed(for example, restricted to matters not pertinent toor perhaps operating well beneath that accountabil-ity relationship) that the resulting report would notlikely be meaningful to the party that conferred the


responsibilities that are being audited, should notbe called audits. Another form of value-for-moneywork that is often characterized by rigorousprocess, but which does not fit the working defini-tion of audit, are investigations into alternate formsof management and their potential impact. Thesecould include organizational studies or investiga-tions into different ways of providing services oracquiring goods or services.

Comprehensive audit practitioners have been,and will no doubt continue to be, called upon bytheir clients to help develop effectiveness or perfor-mance reporting frameworks, approaches or sys-tems. They may also help implement suchapproaches or systems, or participate in activitiesthat are intended to demonstrate such approacheswith a view to educating management. In somecases, practitioners may be asked to do this withthe knowledge that they will also be engaged sub-sequently to conduct additional work that moreclosely fits the working definition of audit that isadopted in this book.

Where any of the above is the case, the mes-sage for practitioners is simple: they should ensurethat these services and the reports resulting fromthem are properly labelled to indicate that they arenot comprehensive audits or comprehensive audit-ing reports.

It would be helpful if the client and manage-ment communities take into account the definitionpresented here when requesting practitioners toprovide services respecting their organization’s per-formance.

TO AVOID CONFUSION AND

UNREALISTIC EXPECTATIONS

• VALUE-FOR-MONEY ENGAGEMENTS THAT DO NOT SERVE AN

ACCOUNTABILITY RELATIONSHIP OR THAT DO NOT EXHIBIT

THE OTHER CHARACTERISTICS OF AUDIT SHOULD NOT BE

CALLED COMPREHENSIVE AUDITS.

• WHERE AUDIT REPORTS CONTAIN THE RESULTS OF

RESEARCH OR OTHER MATERIAL THAT DO NOT MEET THE TEST

OF TRUE AUDIT, SUCH MATERIAL SHOULD BE CLEARLY

LABELLED TO DISTINGUISH IT FROM COMPREHENSIVE AUDIT

OR COMPREHENSIVE AUDIT OPINION.

• PRACTITIONERS SHOULD MAKE SPECIAL EFFORTS ON SUIT-

ABLE OCCASIONS TO DISCUSS WITH THEIR CLIENTS OR POTEN-

TIAL CLIENTS (AND THEIR MANAGEMENTS) THE FUNDAMENTAL

DIFFERENCES BETWEEN AUDITING AND THE VARIOUS TYPES OF

SERVICES DESCRIBED ABOVE.


251 ELMER B. STAATS, CENTENNIAL CONFERENCE PROCEEDINGS (OFFICE OF THE AUDITOR GENERAL OF CANADA, OTTAWA, 1978).252 JAMES J. MACDONNEL, IBID.253 READERS WILL NOTICE THAT THE TERM SIGNIFICANCE IS USED THROUGHOUT PART III AND MAY WONDER WHY THE MORE CONVENTIONAL AUDIT TERM MATERIALITY HAS BEEN AVOIDED.

SOME PRACTITIONERS THINK THAT THE TERMS ARE NOT SYNONYMOUS, NOTING THAT IN COMPREHENSIVE AUDITING MORE EMPHASIS IS PLACED ON QUALITATIVE MATTERS THAN ON THE

MONETARILY QUANTIFIABLE MATTERS THAT PREOCCUPY FINANCIAL AUDITS. THEY BELIEVE THAT THE TERM SIGNIFICANCE CONNOTES THIS GREATER EMPHASIS ON QUALITATIVE MATTERS AND

THAT THIS TERM IS MORE ALL-ENCOMPASSING AND THEREFORE MORE FITTING TO THE BREADTH OF MATTERS THAT FALL UNDER THE PURVIEW OF COMPREHENSIVE AUDITING THAN IS THE

TERM MATERIALITY. IN FACT, THOSE WHO TAKE THIS VIEW OFTEN LOOK AT MATERIALITY AS A SUBSET OF SIGNIFICANCE. OTHERS INTERPRET MATERIALITY AS BROAD ENOUGH TO ENCOMPASS

ALL THAT NEEDS TO BE INCLUDED. IS ONE TERM PREFERABLE? SOME MAINTAIN THAT THE USE OF TWO TERMS TO CONNOTE A SINGLE CONCEPT IS CONFUSING. IT IS EVEN WORSE TO USE

ONE TERM TO MEAN TWO DIFFERENT THINGS. IN THIS BOOK, THE TERM SIGNIFICANCE HAS BEEN USED BECAUSE IT IS THE TERM MOST OFTEN USED IN COMPREHENSIVE AUDITING.254 WILSON, 35.255 COMPREHENSIVE AUDIT REPORTING: CONCEPTS, ISSUES, PRACTICE (OTTAWA: CCAF, 1991) 32.

C H A P T E R 1 5

THREEAPPROACHES TOCOMPREHENSIVEAUDIT

A N E V O L V I N G C O N C E P T

The genesis of comprehensive auditingoccurred long before any comprehensive audit leg-islation was enacted in Canada. For many decades,federal, and in some cases provincial, auditors gen-eral reported instances where they thought thatthere had been a lack of due regard256 to the 3Es—economy, efficiency and effectiveness—in theadministration of public funds. Typically, theirreports contained examples of opinions about suchmatters as costs and productivity and the use ofequipment and resources.

The prime characteristics of reporting thesetypes of instances by auditors were the following:

• the observed instance was exactly that—usu-ally an instance of a specific transaction or aspecific decision;

• the instance was observed as part of otheraudit activities: the observation did not resultfrom a systematic, planned approach toaddress value for money; and

• there were no predetermined criteria to helpassess what would or would not merit auditreporting.

Reports that primarily comprise instances aretoday a much less prevalent product of comprehen-

sive auditing. Where such instances do appear inrecent audit reports, they are usually provided asspecific examples to support broader comprehensiveaudit opinions. In any event, the reporting ofinstances of poor performance provided a strongprecedent for a more rigorous approach to this issue.

A S Y S T E M AT I C A P P R O AC H

In the mid-1970s, the enactment of compre-hensive audit legislation called for more systematicapproaches. The new mandates focused the audi-tor’s attention on finding performance deficienciesand on the systems that were, or should have been,in place to ensure that there was due regard foreconomy, efficiency and effectiveness. As a result,systematic approaches were developed to set thescope of the audits, to develop criteria againstwhich to assess management practices and systemsaffecting performance and to prepare reports.

Over time, the approach has become furthersophisticated. Today, the approach most widelyused by both legislative auditors and private firmswho do comprehensive auditing can best be char-acterized as providing opinions based on reasonablecriteria (usually agreed upon with the client’s man-agement) of whether due regard to the 3Es hasbeen demonstrated in the administration ofresources. This approach recognizes that it is notalways the case that auditors are looking only fordeficiencies to report.

The key factor is the availability of sufficient,appropriate audit criteria. These criteria make itpossible for auditors to determine whether or notreasonable standards were being met. Today, thesecriteria are considered such an integral part of thecomprehensive audit process that the Value-For-Money Auditing Standards, issued by the CanadianInstitute of Chartered Accountants, recommendthat the criteria used in specific examinations beidentified in value-for-money audit reports.


The majority of criteria used in comprehen-sive audits have to do with management systemsand practices established to ensure due regard tothe 3Es. By examining the design of these manage-ment systems and practices and how they are fol-lowed and work in practice, auditors can focus onthe factors that may have a significant influence onan organization’s performance. Although these sys-tems tend to be designed to meet the individualcircumstances of each organization, they frequentlyhave many characteristics in common with systemsused elsewhere for the same purpose. This providessubstantial help in devising, and getting agreementon, the audit criteria. Other criteria may be quitespecific to the organization under review.

One of the important results of using thismethodology is that the audit reports emphasizeopinions on whether or not due regard to the 3Eswas evident. To the extent that specific deficienciesare reported, they are reported in support of, or asexceptions to, the opinions expressed in the report.

The nature of what gets reported from thissystematic, criteria-based process, however, canvary across a spectrum ranging from exceptionreporting of deficiencies in management systemsand practices, to providing an overall opinion onperformance, both the good and the bad. What isimportant is that auditors make their intentionsclear in their reports so that the reader understandsthe context of these reports and does not miscon-strue what is said or what can reasonably be con-cluded or inferred.

Many practitioners recognize that if theyfocus their reports solely on negative observationsand opinions without indicating corrective actions,they would not be contributing as much as theycould to the future quality of public administra-tion. Thus, the auditor’s recommendations andmanagement’s responses are usually included in thereports to increase their value and to provide anopportunity for management to express its views

on the desirability of these recommendations.Often, comments are included to recognize what isworking well in the organization, as well as what isdeficient.

This kind of reporting recognizes that manygoverning bodies are looking for more than lists ofproblems—it is a step towards the expression ofbroader opinions on the state of management inaudited organizations. It also provides recommen-dations that can help the organization becomemore efficient and to manage its resources moreeffectively.

Such reporting, however, does not in all casesovercome the possibility of negativity in the result-ing opinions. If an organization does not in largemeasure meet the established criteria, the auditor’sopinion will say so. But when an organization doessome or most key things well, the auditor willreport a generally favourable opinion that creates apositive context for mentioning any specific defi-ciencies found and audit recommendations forimprovement. This results in a report that moreaccurately reflects the organization’s true perfor-mance. This, in turn, is more useful to the govern-ing body in its decision-making and stewardshipresponsibilities.

T H R E E A P P R O AC H E S H AV E E V O LV E D

Comprehensive auditing has evolved consid-erably in the last two decades. It is not a rigid orstatic concept. In fact, there are three very differentmodels for comprehensive auditing, all of whichapproach the question of performance in differentways.

One model for comprehensive auditing focus-es on the quality of management systems and prac-tices as an indicator of the extent to which theorganization pays due regard to economy, efficien-cy and effectiveness. A second approach is for theaudit to concentrate on providing audit assurance

P A R T I I I . C H A P T E R 1 5 . T H R E E A P P R O A C H E S T O C O M P R E H E N S I V E A U D I T 2 3 5

on the reports by management to a governing bodyor stakeholders on the organization’s performance.A third approach is for the audit report itself toprovide performance information where the orga-nization’s management has not reported adequatelyto the governing body.

These approaches differ in focus and degree,but they are not mutually exclusive. In examiningmanagement systems and practices, for example,the issue is not simply one of noting a deficiencyin the system or practice, but may also extend toconsideration of the causes and consequences ofthis deficiency. Similarly, in looking at the perfor-mance of the organization directly, considerationmust be given to the strength and reliability ofunderlying management systems and practices.

The approach which is right for an organiza-tion depends on a number of factors. Theseinclude, but are by no means restricted to:

• the state of the organization’s systems andmanagerial practices;

• the culture and value systems of the organiza-tion;

• the quality of trust between the governingbody and management;

• the interests of the organization’s governingbody members and management;

• the extent to which the organization is com-mitted to demonstrating its accountability toits stakeholders; and

• its history of work in this area.

The main determinant of which approach ismost appropriate, however, lies in the quality ofthe accountability relationship between executivemanagement and the governing body: the extent towhich the governing body is informed about keyaspects of the performance of the organization, themethods by which it is informed and the way ituses the information it has been given.

This following sections describe each of thesethree different approaches to comprehensive auditing.

A U D I T R E P O R T I N G O N

M A N A G E M E N T S Y S T E M S A N D

P R A C T I C E S

In this approach, the focus of the comprehen-sive audit is the quality of management systemsand practices. These are examined with a view tousing them as an indicator of the extent to whichthe organization pays due regard to economy, effi-ciency and effectiveness.

This approach is currently the most prevalentform of comprehensive audit practice. It is typifiedby long-form reports prepared by auditors thatcomment on the adequacy of management systemsand practices relative to the economical, efficientand effective administration of funds and otherresources.

Legislative auditors often call this a form ofdirect reporting, distinguishing it from attest audit-ing. This form of reporting does more than sum-marize an auditor’s opinions. It also containsimportant information, prepared by the auditor,and reported directly to the governing body.

The systems-and-practices approach has limi-tations. This kind of auditing is only a proxy forauditing that can tell the governing body theextent to which the organization is doing its jobeffectively. It focuses on management systems andpractices, not directly on the organization’s perfor-mance. It is one thing for an audit report to saythat management has or has not followed certainreasonable criteria in terms of a management sys-tem; it is a more difficult matter to know whatsuch a finding really means in terms of the bot-tom-line performance of the organization.


W H E R E T H I S A P P R O AC H I S U S E D

This approach is used—though not exclusive-ly—by virtually all legislative auditors with com-prehensive audit mandates. They typically allot acertain portion of their annual budgets to compre-hensive audits, choosing areas for examination afterassessing such matters as risk, importance, timeelapsed since the last examination, and so on. Andthe comprehensive auditing they do usually con-centrates on key systems and practices establishedto ensure due regard to the 3Es.

This approach has also been used extensivelyby auditors for public sector organizations thathave indicated they want the sort of assuranceabout their management systems and practices thata comprehensive audit can bring. Where a compre-hensive audit mandate is not legislated, the under-taking of such an audit requires careful negotiationto ensure that the client understands the nature ofthe assignment and has no false expectations aboutthe eventual outcome. It may be, for example, thata more sharply focused consulting assignment maybetter serve the client’s needs. As mentioned earlier,to avoid confusion and a debasing of the coinageof the term, it is important that assignments thatdo not meet the essential characteristics of auditare not called comprehensive audits, no matterhow expedient it may be to label them as such.

This approach to comprehensive audit assign-ments is typically undertaken where the client organi-zation has established systems and practices to achievegood performance but does not make substantiatedstatements about its achievements in this regard. Asnoted in the previous chapter, audit reporting usingthis approach may also continue to be needed wheremanagement representations on performance do notcover the entire organization. This could happen, forinstance, in a government where representations aremade by departments but do not cover such govern-ment-wide functions as personnel management.

A U D I T A T T E S T A T I O N T O

M A N A G E M E N T

R E P R E S E N T A T I O N S O N


A R E L AT I V E LY N E W A P P R O AC H

While most comprehensive audit practice hasfollowed the approach described above, there isanother approach that comprehensive auditing cantake if management fulfills its role in reporting onits performance; that is, the provision of auditopinions on reports that contain representations bymanagement on matters of economy, efficiencyand effectiveness. Historically, the lack of suchmanagement reports has meant that this approachcould not be used.

More recently, however, the client, manage-ment and audit communities have been workingtogether to develop acceptable frameworks tounderlie such reports, and experimentation hasbegun in preparing management representationson effectiveness. The framework of twelve attribut-es of effectiveness, as described in Part II, is theproduct of that effort.

The notion of management reporting to agoverning body on the performance of the organi-zation is not new. The individual attributes ofeffectiveness are not new. What is new is reportingthat brings together all of these attributes and thatis based on sufficient discipline and rigour that theinformation is reliable and credible.

When management makes such reports, theyare making representations, just as they do whenthey prepare a set of financial statements. It isimportant to note that the act of preparing man-agement representations on performance or effec-tiveness is a management job. It should not beconfused with comprehensive audit. The audittakes place after management has prepared its rep-resentations.


Where management makes representations,the comprehensive audit largely parallels traditionalfinancial statement attest audits. It adds credibilityto what management is saying so that members ofthe governing body and stakeholders can have asufficient level of confidence in the informationthat they are using to make key strategic and policydecisions.

The underlying approach to the audit ofmanagement representations on effectiveness hastraditional roots—like the previously describedapproach, it examines systems and practices. Here,however, auditors are called on to present an opin-ion on what management has to say about howwell those systems are working. In other words, theauditors must be satisfied that the control systemsthat produced the information used by manage-ment in their representations are reliable.Moreover, the audit will extend to ensuring thatthe representations are not only sound insofar asthe information they contain, but that they alsoinclude all relevant information in order to resultin meaningful disclosure.

E A R LY A U D I T O R I N V O LV E M E N T

This approach to comprehensive auditing ismuch closer to the model used for financial state-ment auditing in which management preparesstatements that are subject to audit. In effect, therole of auditors, although not all the work theyhave to do, is the traditional one.

Probably more than in financial attest audit-ing, comprehensive auditors need to be involvedfrom the beginning in management’s approach tomaking representations on performance. Thepreparation of financial statements is governed bystandard rules (GAAP) that are as well known tothe organization’s accountants as they are to itsauditors. Usually, there is little need to consult

with auditors in drawing up these statements.Nevertheless, in valuing certain assets or liabilities,or in taking a new approach to handling items,accountants will often consult their financial audi-tors. With representations on performance, wherethere are not yet established criteria or standardsfor reporting, early consultation with the auditorsis of great help. Particularly in early experimentswith making management representations that areto be audited, the auditors should be involvedfrom the very beginning.

Management need this involvement in orderto understand what the auditors’ approach will be:the type of audit tests that will ultimately beapplied, what factors will be considered important,and so on. The auditors need this early involve-ment in order to have a good sense of what man-agement have done, how it has been done andwhat considerations have been applied. This helpsthe auditors to decide how to approach the exami-nation work, what level of confidence they canattribute to underlying management processes andsystems, and so on. The governing body usuallywants to see this involvement in order to havegreater confidence in both management’s reportand the auditors’ opinion on that report.

Early on, the auditors would have sought to:• achieve a clear understanding of the steps that

have been taken in the development of therepresentations and the underlying rationale;

• communicate to management and the gov-erning body the auditors’ perspectives on theoverall audit role and process, and the generalconsiderations that will apply in the course ofthe audit and, ultimately, as a basis for theattestation opinion; and

• provide comment on the work of the organi-zation as it progresses vis-à-vis the develop-ment of management representations.


The nature of their involvement presents achallenge to auditors. They need to be able tomaintain a careful balance between preserving pro-fessional independence on the one hand, and beingisolated from the whole process of developing rep-resentations on the other. This, however, is some-thing with which they are familiar. After all, thegreat majority of disclosure and financial reportingproblems are discussed with auditors as accoun-tants are preparing financial statements.

A U D I T A P P R O AC H

As mentioned above, the approach that audi-tors take to management representations on effec-tiveness is, to all intents and purposes, the same asthat taken to financial statements, which involvesreliance on systems, practices and controls that sur-round the collection, consolidation, considerationand reporting of information.

The application of this approach to manage-ment representations should not create conceptualproblems. As with financial auditing, auditors mustunderstand, document and test the processes usedto gather and organize information. The tech-niques that auditors have developed to achieve thisare well known and are readily adaptable to thenew environment. And it is worth rememberingthat there is already substantial experience—gainedthrough comprehensive auditing—in auditing sys-tems to measure and report on many aspects ofperformance. That experience is germane to audit-ing management representations on performance,although the end product—assurance about repre-sentations rather than opinions on managementsystems and practices—may be different.

There are, nonetheless, some technical chal-lenges for auditors working in this environment.Probably more than ever before, they will be deal-ing with lower levels of audit assurance and theattendant reporting and communication issues.

They will also have to increase their knowledgeand expertise in relation to the much broader rangeof concepts involved and the methodologies need-ed to measure and analyze them.

Regardless of whether the auditors areinvolved from the beginning, or whether their firstcontact with the process is only after managementhave prepared their representations, the auditorsneed to ascertain at the outset of the assignmentwhat the organization’s expectations are for themanagement representations process and product,whether these expectations seem reasonable underthe circumstances, and whether these expectationsare understood by the parties involved. Wheretheir initial appraisal of the situation indicates aproblem in this area, auditors ought to go back tothe client to address these issues rather than goingforward and doing more audit work. This is animportant matter in which auditors must exerciseprofessional judgment.

Once the management representations aredeveloped, the auditors would then begin an exam-ination to:

• establish, by means of an initial review, thebasic reliability of the data/information, andthe appropriateness of the assessment factorsor criteria being applied. In determiningwhether the assessment criteria are appropri-ate, the auditors would be guided by:

- the discussions and agreements that mayhave taken place between the governingbody and management on this specificmatter

- the general state of thinking and practicein this specific area

- the logical arguments put forward insupport of these factors/criteria

- the auditors’ own professional judgment• assess significance and risk and, from this,

determine the nature, extent and timing ofthe examination of management systems and


practices that need to be done;• conduct the examination accordingly and

then form an opinion on management sys-tems and practices;

• review the information contained in the man-agement representations and form an opinionon its reasonableness and completeness; and

• communicate these opinions to the governingbody in the form of an audit report.

A U D I T E V I D E N C E A N D L E V E L S O F A S S U R A N C E

As has been pointed out, management repre-sentations on effectiveness vary in many ways, andthese variations have implications for the level ofaudit assurance that it is possible to obtain.

In some instances, full and reliable supportinginformation may be readily available or can be pro-vided in a timely fashion at reasonable cost. Inother cases, it may be possible only to make esti-mates. Sometimes the amount of information sub-stantiation will be determined by the state of mea-surement technology with respect to the attributein question. For example, currently it is easier toaccount for costs and productivity than it is fororganizational responsiveness.

All management representations involve adegree of uncertainty, and managers have to dealwith this appropriately when framing and commu-nicating them to ensure that they are meaningfulto the users. Auditors face somewhat differentproblems, as the following questions indicate:

• How can a valid representation be recognized?• How much audit evidence is needed to sup-

port an opinion on representations?• How can one ascertain if a representation is

misleading?• How far should one look for evidence that is

contrary to the representation?• How can one distinguish representations that

can only be disproved through some distantfuture event?

The above questions bear on the issue of evi-dence and significance from the perspective of theauditors. In general terms, the following factorswill have to be weighed in determining what con-stitutes sufficient, appropriate evidence:

• the extent of assurance desired by the client;• the extent to which the representation can be

substantiated;• the cost of obtaining audit evidence; and• the significance level considered appropriate

by the auditors.

Most financial statement attest audits aredesigned to provide relatively high levels of assur-ance, despite inherent imprecisions embedded insome of the figures reported. The level of assurancesuch audits convey is generally well understood,and techniques have been developed to collect asufficient amount of appropriate evidence cost-effectively. Auditors do not give different levels ofassurance on different types of assets, liabilities,revenues and costs. They have developed their craftto the point where they obtain relatively uniformlevels of assurance on the global assertion of man-agement that the financial statements present fairlythe financial position and results of operations.

To date, auditors have been unable consis-tently to give the same level of assurance respectingall management representations on effectiveness; asomewhat different approach is warranted, at leastin the short term. In large part this is because audi-tors have not provided such assurance in the past.It is an area where experimentation and negotia-tion are necessary. This factor, along with the vary-ing degrees of uncertainty that accompany the dif-ferent attributes, ensures that there will be varyinglevels of assurance provided by auditors respectingmanagement representations on effectiveness. Thissubject is discussed further in chapter 18.


In some cases, traditional positive assurancemay be possible. That is the type of assurance thatis provided when auditors declare that in theiropinion the representation “presents fairly”—thusleaving little or no uncertainty in the reader’s mindabout the reliability of the information provided,within the bounds of whatever reservations man-agement may have declared. Assurance of thisnature may be provided where it is vital to do so,or where sufficient, reliable, appropriate evidencecan be obtained at a reasonable cost. It may, in cer-tain circumstances, be appropriate to provide alower level of assurance than is allowed to financialstatement auditors—a negative assurance thatwould inform readers of the audit report that,based on the work done, nothing leads the auditorsto believe that the representations are not fairlypresented.

There are no definitive answers about thelevel of assurance that either can or should beachieved. Auditors and their clients and manage-ment will have to work together to adopt the mostappropriate patterns. Once sufficient concreteexperience is gained, however, suitable conventionsare almost certain to emerge. What is important isfor auditors to be flexible, to be practical and toconsider the options presented above. To helpensure that the results of the audit are clear forusers, auditors should also describe the criteria theyhave employed to determine what is significant.This issue, too, is discussed at some length inchapter 18.

A U D I T O P I N I O N S O N R E P R E S E N TAT I O N S

Comprehensive audit reports that attest tomanagement representations on effectiveness maybe either highly summarized (similar to the auditopinions that auditors usually give on financialstatements), or they may be longer in form and

narrative. The short-form report for financial state-ments evolved over the years as managementaccepted responsibility for preparing the statementsand the accompanying notes that explain theaccounting practices adopted, valuations and othermatters. In either long- or short-form reports, thefocus of the audit opinion is on the informationreported by management and not on the details ofsystems and managerial practices.

Interestingly, it was a federal Crown corpora-tion that was the first organization to prepare man-agement representations on its performance, basedon the framework of effectiveness attributes. Anaudit opinion was issued on the resulting manage-ment report on the corporation’s effectiveness:

[I]n my opinion, based on the agreedupon framework, the CorporateRepresentation as at June 30, 1989 reportson, in a reasonable and complete way, themost important aspects of the effectiveness ofthe Corporation and the Corporation has theinformation, systems and practices in place tosupport the statements contained therein.257

Since this opinion was expressed, similar opinionshave been issued in a number of other organizations.

A U D I T R E P O R T I N G O N


Members of governing bodies often findthemselves in a situation where they need the typeof performance information anticipated in theframework of effectiveness attributes but, for onereason or another, management is not providingthis information or is not supplying it in a suffi-ciently rigorous or comprehensive fashion. In thesecases, audits focusing on management systems andpractices, such as those described above, may beinsufficient to meet the due diligence needs of thegoverning body.


In such a situation, members of governingbodies may consider a third model of comprehen-sive auditing, one that goes further than looking atsystems. In this approach, in the absence of man-agement information on performance, the auditorsobtain and report results-oriented information tothe governing body. This information might bebased on the effectiveness reporting framework.

To a certain extent, this approach has fre-quently been taken by Canadian legislative audi-tors. They have often included in their reports andopinions information in respect of economy andefficiency. Moreover, of late, some legislative auditoffices have moved towards what is termed “results-based auditing,” in which emphasis is placed onidentifying the critical results or products to beachieved by a program or process. These resultsmay take the form of outputs produced in relationto those intended, products delivered or immediategoals that must be achieved for the program orprocess to be a success. This approach has severalimplications. For example, the auditor may need tospend extra time with management of the auditentity to obtain a common understanding of thekey results and what constitutes reasonable perfor-mance in these regards. Indeed, those moving inthis direction view the approach as significantlychanging the starting point of the audit.258

In the United States, the General AccountingOffice regularly reports information on the econo-my, efficiency and effectiveness of the organizationsthey have reviewed. This approach also has strongroots in financial auditing. It was not that manyyears ago that auditors prepared long-form auditreports that included financial information thatmight otherwise not have been included in finan-cial statements. It is common practice for financialstatement attest auditors to provide information,even in their “short-form” audit opinions, wheresuch information is deemed essential to a meaning-ful understanding of the financial position orresults, and where such information has either beenomitted or improperly reported in the financialstatements.

It is unnecessary to elaborate on thisapproach, since the sort of considerations outlinedin the discussion of the first model—opinions onmanagement systems and practices—apply here.Moreover, the kinds of issues and considerationswith which auditors need to deal in accounting forperformance are discussed in Part II.

It is important to mention, however, that inproviding the performance information that wouldotherwise have been reported by management, theauditor is not attempting to form or communicatean overall judgment about the performance of theorganization or the merits of its policies. Rather,the auditor is providing the governing body withinformation it needs to form such aggregate orsummary judgments.


256 DUE REGARD REFERS TO ADHERENCE TO SOUND MANAGEMENT PRACTICES AND SYSTEMS THAT ARE INTENDED TO FOSTER THE ECONOMIC, EFFICIENT AND EFFECTIVE OPERATION OF THE

ORGANIZATION. IT SHOULD BE RECOGNIZED THAT ADHERENCE TO SUCH SYSTEMS AND PRACTICES IS NOT A PROXY FOR SUBSTANTIVE INFORMATION ABOUT ACTUAL PERFORMANCE.257 REPORTING AND AUDITING EFFECTIVENESS, THE EXPERIENCE OF CANADA MORGAGE & HOUSING CORPORATION (CCAF, 1990) 23.258 WM. F. RADBURN AND MARIA BARRADOS, RESULTS-BASED AUDITING, DISCUSSION PAPER SERIES, DISCUSSION PAPER NO. 68 (OTTAWA: OFFICE OF THE AUDITOR GENERAL OF CANADA,

AUGUST 1994).

S E C T I O N 2

GENERALCONSIDERATIONS


C H A P T E R 1 6

COMPREHENSIVEAUDIT—GENERALCONSIDERATIONS259

Because circumstances vary so significantlyfrom case to case, no two comprehensive auditswill likely be the same. The way in which the deci-sion about the assignment is made, the actual sub-ject matter and conduct of the work are alwaysdetermined by the particular characteristics of theorganization and the people involved. Thus, it isnot appropriate to try to identify a single model ora series of completely discrete steps that apply to allcomprehensive audits. For purposes of presentationin this chapter, however, it is convenient to discussthe process under headings that should be thoughtof as interrelated phases of the audit. The processdescribed here is essentially that used when audi-tors want to satisfy themselves on management sys-tems and practices or to support their opinions onmanagement representations on effectiveness.

For many processes, including financial audit,it is common to identify three steps: planning;detailed examination and analysis (or conduct);and communicating the auditors’ opinion.Although this triad can be used to describe a com-prehensive audit, it is important not to let itobscure a feature of this type of work that must bekept in mind: the whole process is the audit. Gainingand exchanging information at the planning stageis just as much a part of conducting the audit asthe detailed collection of data and the formal

reporting. Impressions and insights gained in thepreliminary interviews are often as critical influ-ences on audit opinions and recommendations asthe analysis of detailed evidence. Indeed, much ofthe detailed work may serve only to test the validi-ty of hypotheses developed at a very early stage.The audit starts even before the first interview.Figure 3.1 illustrates the interrelationships of thedifferent parts of the audit process.

F I G U R E 3 . 1 :

I N T E R R E L A T I O N S H I P S B E T W E E N

D I F F E R E N T P A R T S O F T H E A U D I T


Planning

DetailedExaminationand Analysis

Communicating the Auditor’s Opinion

EstablishingAudit

Criteria

EstablishingAudit

Criteria

Defining theAudit Scope

and Objectives

DrawingConclusions and

Recommendations

CollectingAudit

Evidence

EstablishingAudit

Criteria

Defining theAudit Scope

and Objectives

DrawingConclusions and

Recommendations

CollectingAudit

Evidence

It is with these reservations in mind, there-fore, that the following chapter is organized to cre-ate notional, but artificial, divisions in what is inreality a single continuous process. Issues discussedwith respect to reporting, for example, should bean integral part of the auditor’s thinking in plan-ning and conducting the audit.

Before discussing the actual detailed conductof an audit, however, it is useful to indicate thetypes of examinations that can be undertaken, theway in which decisions are made about what toaudit and the variables on which decisions must bemade and which distinguish comprehensive auditfrom other forms of practice.

T H R E E K E Y V A R I A B L E S

Three closely linked concepts influence, inlarge measure, how auditors conduct their workand how they report their opinions: scope, intend-ed degree of audit assurance and significance.These concepts are intrinsic to any form of audit-ing. They flow from basic notions of audit and thegenerally accepted auditing principles that attendthem. Briefly:

• Scope refers to the breadth and depth of theaudit.

• Intended degree of audit assurance refers to theconfidence that auditors have in the accuracyof the opinions expressed in their reports.

• Significance determines what auditors decideto report and what not to report.

Decisions made with respect to any one ofthese factors will ultimately affect the other twoand the audit report itself, particularly since auditresources are finite. For example, an audit designedto provide an opinion on the entire spectrum of agovernment’s or an organization’s activities is bynature very broad in scope. Given constraints oftime and resources, such an audit could not exam-

ine the organization in the same degree of detailthat would be possible, using the same resources, ifthe scope were limited to only a part of the organi-zation or its activities. As a result, the auditors maynot have the same degree of confidence in theirfindings and consequent opinions as they would ifthey had undertaken a more restricted audit. Insuch cases, auditors would not want to have thereaders of their reports take the same level of com-fort from them as they would had more detailedexaminations been done. Acknowledging that theintended degree of audit assurance is variable, how-ever, should not be interpreted as meaning that itis an unlimited concept that is absolute at eitherthe high or low end of the scale. Its aim is to pro-vide confidence, being the product of a disciplinedprocess, based on the application of suitable criteriaand tied to notions of relevant, competent, suffi-cient and appropriate evidence. It may be, howev-er, that in certain circumstances the intendeddegree of audit assurance is medium to relativelylow. Where this may be the case, the practitionerneeds to exercise professional judgment, thinkingcarefully about the appropriateness of using theword audit as a descriptor of the opinion beingexpressed.

Similarly, an auditor’s consideration of signifi-cance in a wide-scope audit will differ from that ofa more narrowly scoped one. Simply stated, what isimportant to report in a detailed examination of asingle activity may well lose its importance in anaudit of the full spectrum of the client’s operationsor a large portion of them.

In addition, significance factors such as a pro-gram or component’s financial size, sensitivity andimpact normally figure into decisions as to what ischosen to audit.

Comprehensive audits have covered virtuallythe full spectrum of potential scope—from opin-ions on entire organizations to opinions on pro-grams or activities and, in some cases, on only por-

P A R T I I I . C H A P T E R 1 6 . C O M P R E N S I V E A U D I T — G E N E R A L C O N S I D E R A T I O N S 2 4 5

tions of activities or functions. All practitionersrecognize that, as a practical matter, audit resourcesare limited. Auditors and their clients thereforehave choices to make in trading off betweenbreadth of scope, intended degree of audit assur-ance and the significance thresholds they use. Theycan, and regularly do, make diverse choices inpractice. These trade-offs in turn significantlyinfluence audits and the opinions they produce.

It is worth noting that the range of choicesregarding each of the three key variables is broaderin comprehensive auditing than in traditionalfinancial attest auditing. How these variables aretreated in practice also tends to vary more fromone audit to another in comprehensive auditingthan in financial statement attest audits.

Almost universally, the minimum scope forfinancial attest audits is not a matter of choice foreither the auditor or the client. The scope is nor-mally fixed by legislation and well-articulated pro-fessional standards: it pertains to the entire organi-zation. In that context, auditors cannot simplyaudit and render an opinion on one or even severallines of a client’s business or activity. They maychoose to do little or no audit work with respect tocertain aspects of a client’s transactions or affairs,but only on the basis that any problems that mightbe found in these areas would not affect their auditopinion with respect to the organization as a whole.

Comprehensive auditing is practised differ-ently. As a rule, legislation does not specify theoverall scope for such audits.260 Comprehensiveauditors, therefore, have to make choices about thebreadth of these audits and, as mentioned, practiceto date has revealed a wide diversity in the choicesthat are made.

The concept of assurance is central toaccountability and audit. The term intended degreeof audit assurance refers to the confidence thatauditors have in the accuracy of the opinionsexpressed in their reports. Generally, when auditors

are associated with information, readers place morereliance on it than if no auditor had been involved.Clearly, the extent to which auditors accept a riskof error in their opinions will have a bearing ontheir reports’ potential value to their clients andthe decisions that are eventually based on them.Careful professional judgment is needed, sincethere is no universally right or wrong level of auditassurance.

The use of the term assurance is not withoutits critics. Some believe that it connotes a guaran-tee, something that is certainly not intended by anaudit. The critics argue that the term is not gener-ally used in public sector auditing and, hence,should be avoided. Others view it as an acceptedpart of auditing terminology and do not equate itwith guarantee, but with the notion of a declara-tion intended to give confidence. It is this latterunderstanding that is adopted here. It is congruentwith the bulk of accepted audit theory, summa-rized in the following definition drawn from theliterature of financial statement auditing:

Audit assurance may be defined in termsof the degree to which an audit effort increas-es the credibility of specified representationsof others—or, alternatively, the extent that itreduces the risk of material errors in the rep-resentations.261

So, is assurance a valid concept in compre-hensive auditing? Of course it is. Unless readerscan draw accurate inferences from comprehensiveaudit reports, those reports, however accurate andinsightful, would be without value. It is obvious,therefore, that assurance is a key concept for com-prehensive auditing.

As with scope, there is a contrast betweenfinancial statement auditing and comprehensiveauditing in connection with the concept of intend-ed degree of audit assurance. Legislation and regu-lations do not fix a universal degree of intended


audit assurance for financial statement attestaudits. Practice is mature, however, and auditors,their clients and knowledgeable users of auditopinions recognize that the intended degree ofaudit assurance is generally consistent and set at ahigh level. Financial statement audit opinions donot mention the intended degree of audit assur-ance partly because of the generally understoodnorms that have evolved.

Because their scopes vary, the degree ofintended audit assurance associated with compre-hensive audit opinions is bound to vary from oneaudit to another. The intended degree of auditassurance is a true variable, and its range can bequite wide.

In the case of significance, the same consider-ations apply as those with intended degree of auditassurance. That which is important to report is aderivative of the scope for the audit and theintended degree of audit assurance. In financialstatement attest auditing, scope and intendeddegree of audit assurance are reasonably pre-dictable. As a result, significance thresholds alsotend to be common and consistent over a widerange of audits. To the extent that scope andintended degree of audit assurance are variable incomprehensive auditing, so will significance thresh-olds vary from audit to audit. Again, no conve-nient rules of thumb are available for either audi-tors or their clients to apply in this regard.

With three interdependent variables at play,the question arises as to whether one variable shouldtake precedence over the other two. Is there a hierar-chy of importance? Again, in both a theoretical andpractical sense, there are no convenient rules ofthumb, and practice does vary. Some practitioners,for example, place primary emphasis on the degreeof intended audit assurance. Their audit policies arestructured such that the audit result will always con-form to a certain degree of intended audit assurance,and they will not compromise on this. In turn, their

trade-offs relate more to scope and significancethresholds. In other words, they have anchored oneof the variables and only the other two actually vary.In other cases, scope is anchored. There is no singleapproach that is right in all circumstances, but itshould be recognized that in practice certain practi-tioners will pick one of the three variables and tendto keep it relatively constant, while introducing truevariability to only the other two.

Decisions about these variables are crucial indetermining the nature and eventual usefulness ofcomprehensive audits. It is essential, therefore, thatfor each audit these decisions be made both care-fully and consciously with the assistance, whereappropriate, of relevant specialists. Moreover, closeconsultation with the client on these issues willhelp prevent any undue expectations and misinter-pretation of the eventual audit report.

F O C U S O F E X A M I N A T I O N S

Comprehensive audits may examine a num-ber of different facets of an organization. Auditsare regularly conducted on:

• organizational units (such as departments orparts thereof ) that deliver programs or ser-vices (sometimes called vertical studies)

• key management functions (such as personnelor purchasing) that affect all parts of an orga-nization and the management of specifictypes of expenditures that are made in severalor all of the program or service areas

There are no hard-and-fast rules that dictatethe choice of the type of audit to be undertaken. Itis important, however, to consider each possibleapproach and candidate program or function inlight of the significance of the area, perceived diffi-culties and potential benefits. Each has implica-tions in terms of the cost of the audit, its makeup,its report and the nature of its findings.


EXAMPLES OF

POTENTIAL COMPREHENSIVE AUDITS

SPECIFIC UNITS ORGANIZATION-WIDE

OR PROGRAMS FUNCTIONS

HIGHWAY CONSTRUCTION.................FINANCIAL MANAGEMENT

OLD-AGE HOMES............ELECTRONIC DATA PROCESSING (EDP)

SPECIAL SCHOOLS..................HUMAN RESOURCE MANAGEMENT

MILITARY RESERVE..............................................LEGAL SERVICES

FIRE DEPARTMENT ..................................................PURCHASING

CHRONIC-CARE SERVICE.......................................TRAVEL COSTS

S P E C I F I C U N I T S O R P R O G R A M S

Audits of this type:• concentrate on information about results and

operations and the systems and practices thatgenerate such information;

• may require auditors to examine, at the levelof the specific unit, those aspects of organiza-tion-wide functions that have a significantimpact on the operation of the program oractivity under review; and

• usually produce recommendations that arespecific to the particular program or unitunder review and point to improvements inprogram design, organization and service-delivery systems and techniques and toopportunities for cost savings.

O R G A N I Z AT I O N - W I D E F U N C T I O N S O R

E X P E N D I T U R E S

Audits of this type:• concentrate on one or more key managerial

processes that affect all programs;• may require auditors to examine issues per-

taining to the function in several programareas in order to gather sufficient evidenceand to draw broad conclusions; and

• may produce observations and recommenda-tions that point to potential cost savings, butare more likely to deal with questions of theadequacy of the systems and procedures.

D E C I D I N G T H E S U B J E C T F O R

A U D I T

There are many possible subjects for audit inan organization of any size. Several criteria are usu-ally considered in deciding what will be examined.

S I G N I F I C A N C E

No auditor will willingly undertake a com-prehensive audit of a part of an organization’s oper-ations that he or she does not regard as significant.Several factors may be considered under this head.

One is financial size. Small programs, even ifthey might be improved dramatically as a result ofan audit, are usually given a low priority. This isnot to say that programs should be ranked solelyon their budgets, but rather that auditors tend towant to address areas where substantial resourcesare involved. As more and more subjects within anorganization have been covered by comprehensiveaudits, it may be that this criterion (almost one offinancial materiality) will become less important.

A second factor may be a consideration of thesensitivity of the potential subject. This involvesthe subject’s impact on the well-being of the pub-lic. While some programs, such as environmentalprotection and ambulance services, may be relative-ly small in terms of overall expenditure, they mayhave a significant social impact and thereby war-rant attention by the auditors.

P OT E N T I A L R E S U LT S / R I S K S

Auditors try to assess the significance of thepotential results that may accrue from a compre-hensive audit, and this involves an assessment of


the risks inherent in an operation of its kind.Although this is in part a function of the size ofthe operations involved, it is also in part a result ofthe nature of the program or function.

Under this head could come an evaluation ofthe risks of fraud, abuse and financial error. Thepotential for lack of economy and efficiency can alsobe assessed. The relative need for superior manage-ment and control systems can be gauged by lookingat the complexity of operations, the variety of pro-grams or functions subsumed in the subject and thedegree to which operations are decentralized. It isgenerally assumed that the greater the complexityand decentralization, the greater the likelihood thatcontrol systems will require attention. Often, knowl-edge of experience in similar operations elsewherehelps in assessing these matters.

Another factor for consideration is the ade-quacy of the information that is made public orprovided to the governing body. An assessment ofthis information can help auditors judge whetherthe client—the governing body— is properlyequipped with the information it needs to fulfill itsown control responsibilities in respect to a particu-lar potential subject for audit.

S U I TA B I L I T Y F O R A U D I T

One factor that must be considered here is theavailability of appropriate methodology to deal withthe subject. It is not uncommon to consider apotential subject that has not yet been examined bypractitioners. What must be determined is whetherthere are particular characteristics of the subject thatare both of central importance and of too great dif-ficulty to audit. If the important features of thesubject can be audited, there should be no problem,but if some essential part of the subject is so special-ized that it would require the development of newtechniques for audit, a decision will have to bemade about the feasibility and cost of developing

the required methodology. Clearly, a good knowl-edge of what has been done in other jurisdictionswill help in assessing this factor.

Another consideration is the availability ofstaff qualified to conduct the proposed audit. Ifparticular skills are required that are either unavail-able or are committed elsewhere, this must betaken into account. Such a deficiency might beovercome by obtaining the necessary skills fromoutside the audit organization, but this option isnot always available nor, depending on the sort ofskill involved, is it always appropriate.

Although sometimes considered as a separatecriterion, the attitude of management of the candi-date subject is seen by some to influence whetheror not a meaningful audit can be undertaken. Theimpact and efficiency of a comprehensive auditlargely depends on the cooperation of those beingaudited. While there are bound to be instanceswhere audits must be done in spite of a suspiciousor even hostile management, a cooperative envi-ronment is much preferred. Experience has shownthat as more and more audits with positive impactsare completed, there is a growing understandingand acceptance of the concept throughout an orga-nization. Other considerations permitting, it is bet-ter, particularly in the early days of the introduc-tion of a comprehensive audit regime, to go wheremanagement will contribute to, rather than hinder,the audit.

A somewhat related factor is the condition ofthe potential subject for audit. If the organizationor function is relatively stable, this does not createa problem. But if a major reorganization is underway, or has only recently been completed, or ifimportant new systems have just been introduced,it may be useful to consider if an audit would assistor exacerbate a difficult situation. Audits are apt tobe at least marginally disruptive to the routines ofsome of the audited staff, and it is best to avoidplacing extra burdens on already pressed personnel.


A U D I T C Y C L E

Many auditors have established a long-termaudit cycle for comprehensive audits. In these cases,the potential subjects will be dealt with more or lessin the order set down in the predetermined cycle,although the other criteria listed above may affectthe actual order ranking. Where there is no estab-lished cycle for this kind of work, auditors still usu-ally consider the time that has elapsed since a thor-ough audit was made of the candidate subject—even if the scope was confined to financial audit. Inany event, unless the previous comprehensive auditdisclosed some particularly compelling reason to dootherwise, such as a high risk of recurring weak-nesses, auditors will not choose to conduct a secondaudit in the same subject area while other impor-tant subjects are left unexamined.

In considering the subject for audit, it isimportant to ensure that management and theclient are kept informed of the auditors’ thinking.Without consultation, auditors may not recognizepotential difficulties or opportunities. Such consul-tation also helps ensure the cooperation of manage-ment, which is so important for a successful audit.


259 A PORTION OF THE MATERIAL IN THIS AND SUBSEQUENT CHAPTERS IS DERIVED FROM PUBLICATIONS OF CCAF, CICA, THE OFFICE OF THE AUDITOR GENERAL OF CANADA, AND THE U.S.GENERAL ACCOUNTING OFFICE (GAO). IN SOME INSTANCES, THIS MATERIAL IS PARAPHRASED TO MAKE IT BETTER FIT THE CONTEXT. WHEREVER POSSIBLE, DIRECT QUOTES FROM CICA,THE OAG OF CANADA AND GAO PUBLICATIONS ARE GIVEN SPECIFIC ATTRIBUTION.

260 AN EXCEPTION IS THE SPECIAL EXAMINATIONS OF FEDERAL CROWN CORPORATIONS PRESCRIBED BY THE FINANCIAL ADMINISTRATION ACT, WHICH DEAL WITH THE ENTIRE CORPORATION

AND SUBSIDIARIES.261 J. ALEX MILBURN, PH.D., C.A., LIMITED AUDIT ENGAGEMENTS AND THE EXPRESSION OF NEGATIVE ASSURANCE (CICA, 1980).

S E C T I O N 3

COMPREHENSIVE AUDITPRACTICE


C H A P T E R 1 7

CONDUCTINGCOMPREHENSIVEAUDITS

P L A N N I N G T H E A U D I T

Once the general subject for the audit has beenchosen, the detailed planning can begin. Responsi-bility for this work must be assigned to qualifiedstaff, and it is usually best to have it supervised bythe person who will have overall responsibility forthe detailed work of the whole project. Some audi-tors find it helpful to use a peer review process forthese audits, sometimes called an advisory commit-tee and sometimes including professionals from out-side the office. This technique has helped to ensurethe appropriateness of the audit objectives andmethodology. If such a process is to be used, itshould be engaged from the start. In addition, it ishelpful for the client to identify a contact person atthe outset to act as principal spokesperson for theclient on all matters that concern the audit.

THE MAIN OBJECTIVES OF THE PLANNING PHASE

OF THE AUDIT ARE:

• THE DEVELOPMENT OF AN UNDERSTANDING OF THE AUDIT

ENTITY AND ITS ENVIRONMENT;

• THE SELECTION OF THE LINES OF AUDIT ENQUIRY;

• THE PREPARATION OF THE SURVEY PLAN;

• THE IDENTIFICATION OF THE AUDIT ISSUES AND THE CRITE-

RIA BY WHICH THEY WILL BE ASSESSED; AND

• THE PREPARATION OF THE SURVEY REPORT AND AUDIT PLAN.

Given the size and complexity of most sub-jects for comprehensive audits, it is usually impossi-ble for the auditors to provide a complete, detailedcomprehensive audit of the entire organization orfunction. This means that the scope of the auditmust be narrowed to those key areas or issues thatare essential to good performance. Accomplishingthis requires a sound, broad knowledge of the orga-nization or function. Gaining this knowledge is aprimary objective of the first part of the planningphase, usually referred to as the overview stage.

T H E OV E R V I E W S TAG E

The prime activity in the overview stage is togather information about the organization. Withthat information, the auditors prepare a report thatis essentially a plan for the second stage of theplanning phase, the survey.

U N D E R S TA N D I N G T H E AU D I T E N T I T Y

To develop a basic understanding of the orga-nization, the auditors should gather the followinginformation.

The mandate of the audit entityThe entity might be an entire organization,

department, or part thereof. It might be a programor an operation. It could be a major responsibilitycentre or a function. It could be within a central,provincial or local government or within an insti-tution such as a hospital or university. The entitycould be a Crown corporation or just a part of it.

Whatever the entity, the first thing the audi-tors should do is determine its mandate—its raisond’être. If the organization is part of a larger whole,then the overall mandate should be established, aswell as the links, accountability and otherwise,between them. Unless this is done, the organizationmight well be examined out of context.


The programs and activities of the audit entityThe auditors should next identify the pro-

grams operated by the organization, the clients itserves, the program objectives and the generalnature of the operations: inspections, advice, main-tenance, and so on. It may be that the organizationand the program are one and the same. This ismore likely to occur where the entity is small or ispart of a larger program, department, operation orfunction.

At this stage, a good understanding of theauditee’s objectives and the strategies adopted toachieve them is important. Several audits havefound, well into the detailed examination, that thekey issue was in reality not operating-and-controlprocedures but the faulty working of the overallplanning mechanism that had failed to identifytruly appropriate objectives and the strategies toachieve them. Often, becoming satisfied on theseissues at the outset can save a great deal of timeand expense and help ensure a better and moreproductive audit.

How the audit entity is organizedThe auditors should then identify the major

responsibility centres within the organization andthe manner in which they relate to the activitiesidentified. In other words, “Who’s in charge ofwhat?”

Larger entities will also operate certain dis-tinct support activities, such as financial manage-ment, materiel management, computer operationsand human resource management, among others.In some cases, the program under audit will bevery dependent on these support activities. Forexample, in entities requiring highly skilled staff,the staff training and recruitment functions arecritical to achieving good performance. A clearunderstanding of the importance of such supportservices will ensure that they are given appropriateattention in the audit.

ResourcesNext, the auditors should determine what

principal resources (capital, materiel, information,people) are used by the organization and their dis-tribution within it. The auditors should also deter-mine what resources are unique to the organization(for example, scientists, laboratory equipment in aresearch environment) and account for the budgetby both program and responsibility centre toensure that none is missed.

Management climateThe auditors should then determine the man-

agement climate in the organization. Is good man-agement encouraged and what evidence is there toindicate that?

SOME INDICATORS OF

A GOOD MANAGEMENT CLIMATE

• EFFECTIVE INTERNAL AUDIT

• THE USE OF PERFORMANCE MEASUREMENT TECHNIQUES

• RECENT PRODUCTIVITY IMPROVEMENTS

• CLEAR SETTING OF OBJECTIVES, GOALS AND TARGETS

• PROGRAM ANALYSIS AND EVALUATION

The auditors must become acquainted withthe key management processes, especially planning,control and evaluation. It is important for auditorsto understand the structure and process for bothlong-term and operational planning.

Other influencing factorsThe term environment is used to describe a

variety of conditions that can influence the opera-tion of a program or activity, and to which audi-tors must be attuned in performing their work.

P A R T I I I . C H A P T E R 1 7 . C O N D U C T I N G C O M P R E H E N S I V E A U D I T S 2 5 3

SOME ENVIRONMENTAL FACTORS AFFECTING

PERFORMANCE

• ECONOMIC FACTORS: PERIODS OF RESTRAINT THAT MAY

HAMPER SERVICE DELIVERY, OR UNCHECKED EXPANSION THAT

CAN PRODUCE WASTE

• SOCIAL FACTORS: HIGH UNEMPLOYMENT, TECHNOLOGICAL

CHANGE, CLIMATIC OR DEMOGRAPHIC FACTORS SUCH AS ISOLA-

TION OFTEN LEAD TO DECISIONS THAT ARE NOT BASED ON CON-

SIDERATIONS OF ECONOMY, EFFICIENCY AND EFFECTIVENESS

• CENTRAL AGENCY REQUIREMENTS: RULES AND REGULATIONS

REGARDING PERSONNEL MANAGEMENT, SIGNING AUTHORITIES

AND APPROVAL FOR CERTAIN EXPENDITURES ARE OFTEN CITED

AS CONSTRAINTS TO EFFICIENT AND EFFECTIVE MANAGEMENT

• PRESENCE OF OTHER PLAYERS: SHARED OR JOINT RESPONSI-

BILITIES, COMPLEMENTARY ACTIVITIES AND THE NEED FOR

COORDINATION OFTEN PRODUCE CONFUSION AND DELAY

• POLITICAL INTERVENTION: FOR EXAMPLE, AN ACTIVE

INVOLVEMENT OF MEMBERS OF THE GOVERNING BODY THAT

MAY NOT BE CONSISTENT WITH GOOD PERFORMANCE

• INTERNAL POLITICS: POWER STRUGGLES OR FREQUENT

REORGANIZATIONS THAT CAN HAVE DETRIMENTAL EFFECTS ON

PEOPLE AND RESULTS

• PUBLIC INTEREST OR CONCERN: VOCAL AND EFFECTIVE

LOBBY GROUPS OR ATTENTION BY THE MEDIA THAT CAN

INFLUENCE DECISIONS AT THE POLITICAL AND BUREAUCRATIC

LEVELS

Prudent auditors recognize these issues earlyin the audit and take them into account in select-ing areas for detailed examination, in assessing per-formance and eventually in forming their opinions.

The amount of information that auditorshave available or can gather in the overview stage isoften limited, especially if the organization to beaudited is located some distance from the auditors’office. In such a situation, the auditors may initial-ly rely on easily available documentation to gain apreliminary understanding of the organization.

The auditors’ understanding can be verified whenthe audit entity is actually visited in the surveystage.

L I N E O F AU D I T E N Q U I RY

Using the information they have gathered,auditors then make one of the first important deci-sions of the audit: they establish the breadth of thesurvey. The term used to describe the key compo-nents for examination in the survey is lines of auditenquiry. Lines of audit enquiry are the matters orareas selected for preliminary examination in thesurvey stage. They define the survey parameters.Possible lines of audit enquiry may include, but arenot limited to:

• organizational activities or programs• key support functions (such as finance,

human resources)• generic issues such as compliance with regula-

tions or safeguarding of assets• measurement of performance and results• resource acquisition and utilization

Lines of audit enquiry will vary from auditto audit and that there is no single best way todevelop them.

FACTORS TO BE CONSIDERED IN IDENTIFYING

LINES OF AUDIT ENQUIRY

• NATURE AND RELATIVE IMPORTANCE OF THE ACTIVITY

• MONEY AND PEOPLE INVOLVED

• IMPACT OF THE ACTIVITY ON THE PUBLIC OR PRODUCT

• AUDIT AND MANAGEMENT PRIORITIES

• KNOWN PROBLEM AREAS

• RECENT AUDITS AND REPORTS


Overall, the lines of audit enquiry shouldprovide the desired coverage of the organization.Choices may have to be made at this stage,depending on the size and nature of the auditentity and the audit resources available. The diffi-cult part is that these choices are often made withsketchy information. The selected lines of auditenquiry will establish the tone for the work to fol-low, as set out in the survey plan.

S U R V E Y P L A N C O N T E N T S

The overview report/survey plan shouldinclude a brief description of the organization,including its key operational characteristics and adescription of the environment in which the orga-nization operates. The broad areas of examinationto be pursued—the lines of audit enquiry—shouldbe included in the survey plan, with reasons forselecting them. The plan identifies individual sur-vey projects. Usually, there will be one project foreach of the lines of audit enquiry. In certain cir-cumstances, however, it is possible to combine twoor more lines of audit enquiry in one project.Survey project plans include the objectives of eachproject, key tasks, proposed resources and expertiseneeded and time budgets for each project to bepursued during the survey stage.

The survey plan should also identify thesources of the criteria that will be used in making apreliminary assessment of the organization underreview (chapter 19 deals with audit criteria).

The survey plan serves as a useful tool toidentify what areas will be examined, to control theexamination of those areas and to communicate toan audit advisory committee (if there is one)—andultimately to the audited organization—theintended plan of action.

T H E S U R V E Y S TAG E

The ultimate purpose of the survey stage is todevelop a detailed plan for the conduct, controland reporting of an audit. During this stage, audi-tors make an analysis and preliminary review of thesubject to design a manageable audit that dealswith significant issues and fulfills the audit man-date. It is in this stage that decisions are takenabout what is useful and practical to audit, andhow that audit will be done.

Gathering information is crucial in both theplanning and conduct phases of an audit. In theoverview phase, the information needed is of a rel-atively general nature and may be found with rela-tive ease. Indeed, if the auditors have had previousengagements with the organization, the informa-tion may already be present in the audit files. Morespecific information is needed as the audit pro-gresses. It is useful to consider the techniques thatauditors employ to gather the information theyneed.

G AT H E R I N G I N F O R M AT I O N

Among the techniques applied to improve theauditors’ understanding of how an organizationfunctions, either in the planning or examinationphases, the following are the most common:

• meetings and interviews with the personnel ofthe audited organization

• physical observation and inspection• review and analysis of documentation

(reports, manuals, and so on)• modelling

Each of these techniques is briefly discussedbelow. The manner in which the information gath-ered is treated is examined in chapter 20.


Meetings and interviews• Initial meetings should be held to establish

contact and to inform officials of the natureand purpose of the audit.

• Individual interviews with senior staff canenhance the auditors’ understanding of theorganization and help identify and explainissues of interest and importance to the audit.These issues can be used for developing ques-tionnaires to ensure completeness and consis-tency in the subsequent detailed interviewprocess.

During the survey stage it may be neces-sary—and in the conduct phase it is almostinescapable—to interview people who are involvedin the detailed delivery or administration of theprogram or function. Significant differences inunderstanding between these officials and theirseniors can indicate a problem that warrants fur-ther examination. Good or poor morale may indi-cate the extent of regard for economy, efficiencyand effectiveness.

Clients of the program or users of the servicesor goods produced often provide a useful perspec-tive, as suppliers to the organization in certain cir-cumstances. Consistent complaints, if deemed tobe reasonable, may indicate areas needing examina-tion. The client, of course, should be informedabout such meetings and, indeed, may be helpfulin identifying useful contacts.

Officials in other organizations with similaractivities often help identify areas for detailedexamination, broaden the auditors’ understandingof the organization and suggest possible audit crite-ria. In some audits, it is important to obtain theperspective of officials in relevant central agencies.In other audits, this may be unnecessary.

Auditors of other, similar organizations canoften share their experience and point to areaswhere significant problems or weaknesses may be

found. They may also suggest, or help refine,appropriate audit criteria and approaches.

Outside experts in the subject matter beingexamined can offer insights and identify areas thatdeserve particular attention. They may also explainissues that puzzle the auditors and confirm oramend impressions about industry practices. Thisexpertise is often so helpful that many comprehen-sive audits include experts from outside the officeon multidisciplinary audit teams. Bringing thiscapability to the team can be important at allstages of the audit process, including: validatingaudit objectives and audit criteria; discussing thesteps of the audit program; gathering and analyzingevidence; and developing the audit report.Choosing these resources, melding them into theteam and providing appropriate supervision oftheir work is no simple task.

Attending special meetings and conferencesthat pertain to the specialized areas under scrutinyis also a practical strategy. As well as the informa-tion that practitioners can glean from such events,this is also a good opportunity to identify expertsor specialists who, as noted above, can play a keyrole in the audit process.

Physical observation and inspectionVisits to the site(s) of actual operations fur-

nish vital information, provided auditors are obser-vant and know what to look for. Physical inspec-tions allow auditors to acquire an understanding ofphysical processes, equipment utilization proce-dures, accounting operations, and so on. Personalobservation can identify lack of regard for value formoney if there is evidence of:

• a significant backlog of work;• idle equipment and staff;• outdated methods;• poor working conditions;• poor equipment condition.


Review and analysis of documentation

Management reportsManagement reports provide a wealth of

information to auditors. An assessment should bemade about whether all the appropriate reports areproduced. Existing reports should be analyzed inrespect of the following:

• frequency and timeliness;• financial results relative to the budget;• output results relative to targets;• efficiency and unit cost trends;• attainment of, or contribution to, objectives; and• distribution.

Internal audit and evaluation reportsInternal audit and program evaluation reports

are invaluable for the auditors. These reports oftenidentify potential areas of overlap in scope andscheduling of the audit. Past problems discussed inthe reports may or may not require further atten-tion. In addition, they can provide valuableinsights into the working of the audited organiza-tion. Chapter 22 deals with the reliance that audi-tors give to these and similar reports.

Literature search A careful review of the relevant literature gives

the auditors an understanding of the historical devel-opments of the organization and the general area inwhich it operates as well as of characteristic manage-ment control issues. The literature may also suggestrelevant criteria that can be used in the audit.

Policy and procedure manualsAuditors should review important manuals

for clarity of instructions, delineation of responsi-bilities, levels of financial authority, and so on.These documents may also point to audit issuesthat should be examined and to audit criteria thatcan be used.

Other sources and documentationIn addition to the above, it is often be use-

ful for auditors to peruse Hansard (the record ofproceedings in the legislature) and any work thathas been done by other auditors on similar organi-zations of activities.

Modelling A useful way of documenting and understand-

ing the organization is to use models or charts.Charts depict in graphic form processes and rela-tionships that might be difficult to describe in writ-ing. In addition to helping the auditors develop abetter understanding of the organization, models area useful tool for communicating the understandingand findings to managers and the audience for theaudit. Three types of models are commonly used:

• program structure model• key activity or program delivery model• environmental model

Program structure modelFigure 3.2 shows the program structure of a

national highway safety program. Note that thelevel below Program Components identifies theprogram outputs. Below Outputs is the hierarchy ofobjectives and intended effects. The diagram showsthat the accomplishment of immediate objectives isexpected to lead to the accomplishment of interme-diate and ultimate objectives/effects.

PURPOSE OF PROGRAM STRUCTURE MODEL

• TO PROVIDE AN UNDERSTANDING OF THE ORGANIZA-

TION FROM THE PERSPECTIVE OF THE PROGRAM MANAGER

• TO DEPICT PROGRAM ACCOUNTABILITY RELATIONSHIPS

• TO FACILITATE UNDERSTANDING THE PROGRAM

• TO FLAG OTHER POTENTIAL AREAS FOR DETAILED

EXAMINATION


The program structure model attempts to pro-vide an understanding of the audited organization fromthe perspective of the program manager. It shows theprogram accountability relationships, including whichobjectives various program components can be heldaccountable to achieve. The primary purpose forpreparing such models is to facilitate understanding theprogram and to identify related performance indicators

of interest to management and the legislature or othergoverning body. Questions to pursue in relation to theelements contained in the model include: Are programcomponents well defined? Are objectives/effects suffi-ciently precise? Are causal linkages plausible?

Finally, the model helps flag other potential auditissues such as potential problems for planning whenobjectives are vague.


F I G U R E 3 . 2 : P R O G R A M S T R U C T U R E M O D E L - H I G H W A Y S A F E T Y P R O G R A M

F I G U R E 3 . 3 : P R O G R A M D E L I V E R Y M O D E L - D R I V E R L I C E N C I N G

PUBLICITY

MEDIA ADVERTISING PATROLLED AREAS RULES & REGULATIONS STANDARDS & TESTS

CHANGE ATTITUDES COMPLIANCE WITH RULES

MINIMIZE THE INCIDENCE AND SEVERITY OF TRAFFIC ACCIDENTS

SAFE VEHICLES SKILLED DRIVERS

HIGHWAY PATROL VEHICLE SAFETY DRIVER LICENCINGPUBLICITY

MEDIA ADVERTISING PATROLLED AREAS RULES & REGULATIONS STANDARDS & TESTS

CHANGE ATTITUDES

PROGRAMCOMPONENTS

OUTPUTS

IMMEDIATEOBJECTIVES COMPLIANCE WITH RULES

MINIMIZE THE INCIDENCE AND SEVERITY OF TRAFFIC ACCIDENTSULTIMATE OBJECTIVE

SAFE VEHICLES SKILLED DRIVERS

HIGHWAY PATROL VEHICLE SAFETY DRIVER LICENCING

No Fail Fail

1 2

5 6

3 41 2

5 6

3 4

Yes Pass Pass

TRAFFIC ACCIDENTANALYSIS

LICENCEAPPLICANTS

ELIGIBILITYASSESSMENT

ADMINISTERWRITTEN TEST

ADMINISTERDRIVING TEST

ISSUELICENCE

DRIVER LICENCINGSTANDARDS

TRAFFIC ACCIDENTANALYSIS

LICENCEAPPLICANTS

ELIGIBILITYASSESSMENT

ADMINISTERWRITTEN TEST

ADMINISTERDRIVING TEST

ISSUELICENCE

DRIVER LICENCINGSTANDARDS

1. CLERICAL STAFF: DATA CHECK, INCLUDING COMPUTER

2. CLERICAL STAFF: COMPUTER ASSISTED

3. DRIVER EXAMINERS

4. CLERICAL STAFF

5. COMPUTER ANALYSIS OF ACCIDENT REPORTS

6. RESEARCH STAFF: PERIODIC PROPOSALS TO REVISE STANDARDS

Program delivery modelThe second type of model that auditors can

prepare depicts the manner in which the programis carried out in pursuit of the program objective.The main purpose of this model is to identify thekey activities of an organization so that a laterassessment can be made of the control exercisedover them by management.

The flowchart (figure 3.3) illustrates the processfor obtaining a driver’s licence in a Canadian province.It depicts the major activities or decision points ingraphic form and in the sequence in which theywould normally occur, and makes it easier to under-stand the process and to communicate it to others.

The model shows:• the source of demand for the organization’s

outputs and how that demand occurs (peopleapplying for driver’s licences); it indicates ifoutput is generated by demand or supply;

• processes/activities from demand to delivery

of output, as well as key support activities;• decision points and related responsibility cen-

tres; and• the key activities—those on which the success

of the program mostly depends (supportingdocumentation should be prepared explainingwhy they are key).

Environmental modelThe third type of model that is helpful in

understanding an organization is the environmentalmodel. For the most part, this model depicts the var-ious factors or elements that may influence the oper-ations of the program. It is not necessary at thispoint to demonstrate the relative importance orimpact of these factors—this will happen at thereporting phase. At this stage, it is sufficient to notethat these factors exist and to take them into accountduring the examination. The example chosen for fig-ure 3.4 is of a federal materials research unit.


F I G U R E 3 . 4 : E N V I R O N M E N T A L M O D E L

RESEARCH INSTITUTE• To assist in the growth of Canadianindustry through materials research• To provide a research environment

distinguished by the quality andapplicability of its efforts

FederalGovernment Departments

EMR • Transport • DNDIT&C • MOSST

ProvincialResearch Organizations

ORF • CRIQ • WIC • ARC

UniversitiesEcole Polytechnique • McGill, Laval • U.B.C. • Toronto

Sherbrooke • McMaster, etc.

International RelationshipsFrance • Belgium • Switzerland • U.S.A.

U.K. • Germany

Central AgenciesTreasury Board • DSS

Support GroupsFinance • PersonnelOfficial Languages

IndustryManufacturers • Retailers

Consumer Groups • Engineers • Scientists

Other DivisionsExchange of Information

Vice-President(Regional Laboratories)

Formal Reporting

RESEARCH INSTITUTE• To assist in the growth of Canadianindustry through materials research• To provide a research environment

distinguished by the quality andapplicability of its efforts

FederalGovernment Departments

EMR • Transport • DNDIT&C • MOSST

ProvincialResearch Organizations

ORF • CRIQ • WIC • ARC

UniversitiesEcole Polytechnique • McGill, Laval • U.B.C. • Toronto

Sherbrooke • McMaster, etc.

International RelationshipsFrance • Belgium • Switzerland • U.S.A.

U.K. • Germany

Central AgenciesTreasury Board • DSS

Support GroupsFinance • PersonnelOfficial Languages

IndustryManufacturers • Retailers

Consumer Groups • Engineers • Scientists

Other DivisionsExchange of Information

Vice-President(Regional Laboratories)

Formal Reporting

I D E N T I F Y I N G AU D I T I S S U E S

When the survey of the organization or por-tion under review is completed, and indeedthroughout the survey stage, the auditors mustassess the information they have gathered andselect specific areas or issues for detailed audit test-ing. The following factors should guide theirthinking at this stage:

• financial significance (materiality)• major activity concerns• adequacy of systems• risk• potential areas of waste/inefficiency or mis-

management• relationship to organization-wide issues• visibility and interest• capacity to influence or control costs• cost-effectiveness of audit

Financial significance (materiality)Certain activities within an organization will

represent a relatively larger portion of expendituresthan will others. The questions of relative costs andmateriality should be asked early, to identify poten-tial areas for audit and to contribute to the audi-tors’ understanding of the organization and itsactivities by focusing attention on areas of signifi-cant expenditure.

Major programs A thorough review and understanding of the

programs and objectives of the organization, andhow they can be achieved, should indicate to theauditors which program components are key to theorganization, especially from the point of view ofthe targeted client group. To identify audit issuesearly in the survey, it is advisable to address theactivities that are or become important underchanging conditions, and to understand why theyare important.

Adequacy of systemsAn evaluation of the planning, budgeting,

control, management reporting and related systemsagainst applicable regulations, guidelines and direc-tives—and against good management practice—should disclose apparent weaknesses and/or non-compliance that may warrant inclusion in the audit.

RiskIn any organization or activity, certain ele-

ments are critical to its survival. These elementsmust be identified. Auditors should pay particularattention to components that could jeopardize theentire functioning or well-being of an organizationor one of its activities. Often these are supportfunctions such as EDP or training. Management’sability to control risks to these elements should beassessed as well as the reasonableness of the costsentailed in reducing risk. External risks, whichmight come from action or inaction by entitiesidentified in the environmental model, should alsobe identified. To the extent that they can be influ-enced by the audited organization, they should beaddressed in management’s forecasting and plan-ning processes.

Potential areas of waste, inefficiency ormismanagement

The auditors should also pay attention toareas that have been identified, either by manage-ment or other sources, as containing known or sus-pected errors or problems. Indicators of such areasinclude:

• financial problems: disparity of budgeted toactual expenditures; heavy year-end expendi-tures; duplicate payments or overpayments;large write-offs of accounts receivable

• human resource problems: poor morale, vari-ously manifested; inadequate or inappropriateorganization and assignment of responsibili-ties; excessive use of consultants or overtime


• physical resource problems: underused, unusedor obsolete equipment; surpluses or shortagesof, or long waits for, materials; inadequate orexcess space; scanty documentation of proce-dures for major acquisitions

• EDP problems: proliferation of equipmenttypes; software/hardware imbalance; highdowntime and maintenance costs; overuse ofconsultants; user complaints

• other areas: complaints from clients; absenceof targets; missed deadlines; lack of perfor-mance measurement.

Relationship to organization-wide issues andconcerns

When attempting to identify audit issues, theauditors should be aware of both current andimpending organization-wide concerns; for exam-ple, particularly intricate labour negotiations.

Visibility and interestCertain elements or activities in an organiza-

tion may have a high profile resulting either inadverse or positive publicity. Management maywish to avoid adverse publicity by carefully control-ling and monitoring these activities. For example,in an industrial situation, management’s actions (orinaction) regarding industrial pollution could resultin increased interest in the organization. Auditorsmust take these sensitivities into account whendeciding on areas for further examination.

Capacity to influence or control costsManagement’s ability to influence or control

costs should be considered when selecting areas orissues for audit testing. Although certain activitiesmay result in significant costs in absolute terms,auditors should focus attention on those compo-nents that management can change. For example,total land acquisition costs may be a large part ofan organization’s budget, but management may

have limited discretion over the bulk of these costs.The auditors could, however, examine whether allreasonable alternatives had been reviewed andreported to authorized decision makers.

Cost-effectiveness of auditSome areas or issues may appear important, but

are very difficult or costly to audit. Auditors mustassess the special skills and resource levels required toconduct the audit in selecting audit issues.

D E V E LO P I N G T H E S U R V E Y R E P O R T

Purpose of the survey reportThe product of the survey stage is the survey

report, which includes an audit plan. The auditplan comprises a number of audit projects, usual-ly—though sometimes fewer than—one for eachidentified audit issue. The survey report serves thefollowing purposes:

• to communicate an understanding of theaudited organization

- among the team members—it is importantthat all team members have an opportu-nity to contribute to the survey report toensure consensus and that each auditor’sknowledge and information about theorganization is shared with colleagues

- to audit superiors—to communicate whatthe audit team has accomplished todate, what further work it is proposingand at what cost

- to the organization—to inform them ofthe work done, preliminary assessmentand areas for detailed examination, andto solicit their reaction to the focus anddirection of the audit

• to identify critical elements that warrant fur-ther examination and the justification forpursuing them

• to display an audit plan


Contents of the survey reportThe survey report should contain the follow-

ing elements:• audit objective and scope;• brief explanation of programs/activities sur-

veyed, including program structure, deliveryand environmental models;

• identification of environmental/organizationalfactors and constraints;

• areas for detailed examination and rationalesupporting the selection; and

• audit plan, comprising a number of audit projects.

Audit projectsEach proposed audit project should be writ-

ten up in detail. The description of each projectshould provide:

• audit objective;• audit criteria;• nature of the evidence to be sought;• methods to be used to gather and analyze the

evidence; and • resource requirements.

It should be noted that the level at which anaudit opinion is expressed varies in practice. Insome cases, an opinion is provided on each auditproject and all the projects are bound together intoone report, but one overall opinion is not pro-duced. In particular, this would be the case wherethe sum of the parts that are being audited doesnot equal or cannot be reasonably interconnectedto form the whole. But where, deliberately so, thesum of the parts does equal the whole, it becomesfeasible to provide an overall opinion.

T H E C O N D U C T P H A S E

To achieve audit objectives, the conductphase involves gathering and evaluating informa-tion to compare actual practices or operations

against criteria, and to obtain sufficient, appropri-ate evidence to support any opinions and recom-mendations that will eventually be reported. Wheresignificant deviations from criteria are identified,the underlying cause and effect will be determinedduring this phase of the audit.

As already mentioned, the preparation of theaudit plan and elaboration of audit projects is thefinal step in the planning phase. The first step ofthe examination phase consists of preparingdetailed audit programs for each project to guidethe work during this crucial part of the audit.

A U D I T P R O G R A M S

An audit program is a list of procedures to beperformed to compare existing systems and/ormanagement practices with audit criteria and tocollect evidence to support audit observations.

In the survey report, the auditors developedaudit projects with specific objectives and criteria totest for. These are the same elements that carry for-ward to the audit program. The task now is to speci-fy audit procedures that will allow determination ofthe actual state or condition of a management prac-tice or control. Comparing the observed conditionto the mutually agreed criteria will generate a find-ing, either positive (criteria are met) or negative (cri-teria are not met). The audit program should indi-cate the time and resources budgeted to complete it.

AU D I T P R O C E D U R E S

Audit procedures are the tests that auditorsperform to give the necessary degree of assurancethat specified audit criteria are, or are not, beingmet. These procedures could consist of interviews,inspections, analyses of data, confirmations, com-parisons, and so on.

Specific audit procedures should provideanswers to the following questions about the taskto be performed:


• when?— time frame, period, sequence, rela-tionship with other procedures

• who? — individual or team that will carryout the audit procedures

• how? — interview, review, examination, cal-culation, analysis, comparison

• where? — location: H.Q., division, inCanada, overseas

The type of audit procedure used will vary,depending on the particular subject, the purpose ofthe procedure and the type of evidence available.For example, to test adherence to the eligibility cri-teria in offering financial assistance to a manufac-turer, auditors would examine the relevant informa-tion given on application forms and compare it tothe eligibility criteria used by the organization ingranting assistance. In other words, a review of theapplications on file would be sufficient for testingin this case and would become the audit procedure.

But to determine the efficiency of buildingmaintenance, auditors would have to examine allthe relevant factors that have an effect on efficien-cy. For example, the number of employees used,the standards established, the maintenance meth-ods used, the level of service and the amount ofwork carried out will all affect the efficiency ofmaintenance. The audit procedures used in thiscase would involve: collecting relevant informationthrough interviews, reviewing reports and guide-lines, on-site inspection and analyzing the informa-tion collected; calculating performance ratios (out-put/input); and comparing these ratios with thoseof other buildings.

D I F F I C U LT I E S I N D E V E LO P I N G AU D I T P R O G R A M S

Until comprehensive auditing becomes a uni-versal practice, auditors will frequently find them-selves breaking new ground. They have to developcriteria that are appropriate and acceptable to man-

agement. The objectives for many comprehensiveaudit projects tend to be specific to the organiza-tion audited, with few established audit criteriaunderstood by both auditors and auditees. In suchcircumstances, auditors must proceed with greatcare in developing audit programs and recognizethat substantial flexibility and judgment will haveto be exercised. In some instances, the auditor maynot be able to establish criteria to which the client’smanagement will agree. If differences cannot beresolved, it may be up to the auditor to select suit-able criteria (using such sources as legislation, poli-cy and program documentation, similar entities/programs, standards of good practice) and proceed.

Most audit programs will have to be tailor-made for each project in order to address properlythe objectives, criteria and operations unique toeach audit. This also implies that the audit pro-grams developed have to be flexible, evolutionarydocuments that are revised and improved as audi-tors gather more information.

Another difficulty with these types of pro-grams is the breadth of the areas that they cover.This increases the possibility of more than oneauditor examining the same subject area. Forexample, two auditors may analyze the same reportor interview the same department head, but fordifferent purposes. This suggests that the seniormembers of the audit team will have to pay partic-ular attention to the possibility of duplication oroverlap when reviewing audit programs. Theyshould seek to make the audit as efficient as possi-ble while minimizing disruption to the auditedorganization.

E V I D E N C E

Much of the detailed examination phase ofthe audit is devoted to collecting evidence.Evidence is defined inter alia by the Shorter OxfordDictionary as “Ground for belief; that which tends


to prove or disprove any conclusion.” Collectingand using evidence is an essential part of theauditors’ task, and is dealt with in some detail inchapter 20.

A U D I T F I L E S A N D W O R K I N G PA P E R S

Auditors keep a detailed record of the evi-dence they collect and the work they do. Thismaterial supports the audit findings, conclusions,recommendations and opinions. The Office of theAuditor General of Canada has adopted the follow-ing policy:

Audit Principals should satisfy them-selves that audit files document all importantmatters which support the content of thereport and which demonstrate that the auditwas carried out in accordance with the auditand examination policies of the Office.262

The following are among the Office’sComprehensive Auditing Manual guidelines:

Audit files should be:• complete and accurate, showing the nature

and extent of the audit work and providingproper support for audit decisions, findings,conclusions, recommendations and opinions;

• concise, clear and complete, without requir-ing supplementary oral explanations;

• pertinent, containing only information that isrelevant, important and useful to the objec-tives of the audit; andAudit files should clearly state the objectivesof the audit, and the reasons behind specificaudit procedures or tasks and their relation tothe audit objectives. Anyone later using work-ing papers kept of file should be able to readi-ly determine their purpose, the nature andscope of the work, and the conclusions.

All team members should ensure thataudit files contain sufficient but not excessiveevidence to support audit findings, conclu-sions and recommendations and that the filesrecord the nature and extent of supervisionand review.

All written communication with theaudited organization should be filed appropri-ately in the current working papers, the perma-nent file or the correspondence file, dependingon the nature of the communication.263

Other offices have similar policies and guide-lines to ensure that high professional standardscharacterize their work.

F I G U R E 3 . 5 : A U D I T P R O C E S S


CONCLUSION

RECOMMENDATION

CRITERION CONDITION

REPORT

ASSESSSIGNIFICANCE

ANDREPORTABILITY

DETERMINE ORCONSIDER

CAUSE AND EFFECTIF NEGATIVE

ASSESSSIGNIFICANCE

ANDREPORTABILITY

DETERMINE ORCONSIDER

CAUSE AND EFFECTIF NEGATIVE

COMPARISON

FINDING (POSITIVE OR NEGATIVE)

A U D I T F I N D I N G S

When sufficient, reliable, relevant evidence ofperformance is obtained, auditors compare thatperformance to the predetermined criteria to cometo an audit finding. Once an audit finding hasbeen developed, two complementary forms of eval-uation take place: first, assessment of significanceand reportability; and second, determination ofcause and effect.


The significance of an audit finding shouldbe assessed using four factors: the size of expendi-ture or revenues; impact (social, economic, envi-ronmental); sensitivity; and risk. The inclusion ofsensitivity and risk as factors indicates that signifi-cance is not absolute. In many cases, a final deci-sion on whether a finding is significant will not bemade until all evidence has been gathered.

Where a variance from a criterion has beenfound, both the size of the variance, if it can bequantified, and the frequency of occurrence mustbe considered in determining whether the auditfinding is significant. It may be obvious that anerror that has been discovered or a minor weaknessin a system that has been identified, is an isolatedcase and are not symptomatic of a larger problem.Unless other related errors or weaknesses are foundduring the audit, insignificant findings will not beformally reported, although in most cases they willbe mentioned informally to management. Theeventual determination of whether a finding is sig-nificant is based on the exercise of sound profes-sional judgment when all the facts are known.

A negative finding may lose significance if thecondition is already known to the audit client andsteps are under way to correct the situation. Recentreporting of the same or similar findings may alsodecrease significance.

R E P O R TA B I L I T Y

Findings may be significant in size andimportance, but may not be reportable.

A finding may be outside the mandate of theauditors. Findings of this nature are often discoveredincidentally, rather than by a deliberate comparisonof evidence to the criteria established for the audit.While it may be an interesting finding, if it does notrelate to the audit subject, auditors are clearly goingout on a limb if they do not respect the establishedaudit boundaries or scope. Such findings should bementioned to management, but probably notincluded in the auditors’ formal report.

Some findings could be sensitive for political,security or other reasons. Auditors should proceedwith extreme caution in such circumstances, ensur-ing that their superiors are advised at all times.

C AU S E A N D E F F E C T

The cause of a finding is the reason that thereis a variance between the condition that the evi-dence shows and the established audit criteria. Itexplains why something happened and forms thebasis for any constructive recommendations forcorrection. Auditors should be able to demonstratea clear link between the cause they have identifiedand the problems (effects) they have observed oranticipate.

There may be several causes for each finding.For negative findings, auditors need to identify thecause that, if changed, will prevent similar find-ings. In some instances, the cause may be outsidethe control of the organization under audit.

The effect of a negative finding is the measureof the problem created by the identified cause. Itcan be either quantitative (for example, dollars, per-son years, time) or qualitative (for example, lack ofcontrol, poor decisions). To warrant reporting, aneffect should be sufficiently serious to justify, on acost/benefit basis, the action required to correct it.


The effect may have occurred in the past,may be occurring now, or may occur in the future.If the effect occurred in the past, it is important todetermine that the situation has not been remediedto prevent it from recurring.

A word of caution: While the analysis ofcause and effect may be helpful in piecing togetherand making sense out of a series of random obser-vations, the analysis should not be regarded as apuzzle with only one solution. Cause-and-effectchains tend to overlap and merge, often making itdifficult to establish clear relationships.

The following is an example of an audit find-ing from a recent audit. The project is highly tech-nical, requiring state-of-the-art equipment.

CAUSE-AND-EFFECT EXAMPLE

THE DEPARTMENT HAS IDENTIFIED $1.5 BILLION IN ADDI-

TIONAL COSTS RELATED TO THE CAPITAL PROJECT TO BUY A

FLEET OF SUBMARINES APPROVED BY TREASURY BOARD. THE

PROJECT IS BEING CONTAINED WITHIN THE APPROVED BUD-

GET BY CHARGING THESE ADDITIONAL COSTS TO OPERATIONS

AND MAINTENANCE.

POSSIBLE CAUSES:

• PROJECT EXCEEDING BUDGET

• LIFE-CYCLE COSTING PROBLEMS

• FIXED-CEILING FUNDING AND TECHNOLOGICAL UNCERTAINTY

• NATURAL DESIRE TO WANT AS MUCH AS YOU CAN GET

POSSIBLE EFFECTS:

• MISLEADING PARLIAMENT

• MISAPPROPRIATION OF FUNDS

• CONTRADICTS TREASURY BOARD GUIDELINES

• LACK OF DUE REGARD BY BUYING MAJOR EQUIPMENT ITEMS

WITHOUT ADEQUATE SPARES

• DRAINAGE OF THE DEPARTMENT’S BUDGET

A U D I T O P I N I O N S

When the cause and effect of an audit findinghave been satisfactorily identified, auditors come toan opinion, either positive or negative. It representsthe evaluation of the audit finding based on theavailable evidence.

In some cases, although they may have founda major weakness, auditors may have determinedthat management has already planned measures tocorrect the weakness. In this situation, if they aresatisfied that the proposed actions will be effective(and may, in fact, be the actions that they wouldhave recommended), auditors would state theweakness observed, the proposed corrective actionand estimated implementation date, but would notnecessarily state an opinion or recommendation.

The opinion should sum up the evaluation ofthe audit finding. It should not be a restatement ofthe audit finding but an argument for a changethat will bring about a needed improvement.

If the opinion is clearly stated, the recom-mendation often becomes self-evident. If unclear,the rationale supporting the recommendation maybe unconvincing.

R E C O M M E N D AT I O N S

Comprehensive auditors may provide recom-mendations where significant adverse results havebeen identified associated with a negative auditfinding. Doing so can help the reader focus onmatters of particular importance and may increasethe likelihood that observed deficiencies will becorrected. Such recommendations will not berequired where the opinion is positive and auditorshave found no significant deficiencies.

The purpose of an audit recommendation isto state what improvements are needed rather thanto indicate specifically how to achieve them. Unlessthey take this approach, auditors may find theyhave a conflict of interest if they subsequently


audit processes they have recommended. Thisapproach also allows management to determine themost effective solution, which is their role.

Auditors must consider the following ques-tions in developing recommendations:

• Can management control the factors or caus-es of the problem?

• Is the recommendation practical and feasible?Has the suggested solution been tried success-fully elsewhere? Are the necessary resourcesand expertise available?

• Is the recommendation consistent with simi-lar recommendations in the past?

• Is the recommendation consistent with inter-nal and centrally imposed guidelines?

T H E C O M P R E H E N S I V E A U D I T

R E P O R T

The report is the end product of the audit.While the process of auditing may itself haveadded value, the report is the most tangible output.It is what the client is most interested in seeing; itis what the auditor has to offer. No matter howcareful and complete the audit examination hasbeen, it will be of little help to the client and nocredit to the auditor if the report is poorly pre-pared and presented.

The CICA’s Value-For-Money AuditingStandards, reproduced in chapter 23 (standards andquality assurance) provides substantial guidance topractitioners in respect of their audit reports. Thereis no attempt to duplicate that material in thischapter, and readers are urged to read the relevantparagraphs of the Standards in conjunction withthe information provided here.

Once the audit examination work is complet-ed and the evidence has been analyzed, the report-ing phase starts. In reality, the report develops andreporting continues throughout the audit as issuesbecome clearer. A full discussion of the report

structure by members of the audit team should sig-nal the beginning of the formal reporting phase.The presentation of the final, formal report is thelast step in an ongoing process. This section, as hasmost of the previous material, is based on the tra-ditional reporting model for conducting compre-hensive audits, although most of the principlesapply to the other approaches.

CO M M U N I C AT I O N W I T H M A N AG E M E N T

Throughout the audit, starting from the firstinterview, the process of reporting is under way. Atleast for those who will have to deal with the resultsof the audit—the auditee’s management—thereports of progress of the audit are an integral partof the reporting process. By the time the auditorsare starting to prepare the formal report, at least themanagers most directly affected should have a gen-eral idea what the final document will contain.

As findings emerge and the audit processmoves closer to the stage of report writing, it isappropriate to shift what earlier may have been arelatively informal mode of communicationbetween the auditor and management, to a pro-gressively more formal one.

If the report is to be of maximum benefit, itmust contain no surprises. This means that seniormanagement should be given an opportunity to seethe draft report before it is either finalized orreleased.

The prime reason for exposing the draft reportto management is to ensure that the facts it presentsare accurate. These facts should be checked with themanagers directly concerned, since senior managers,who are not usually familiar with all the details, willwant to be assured that there are no misrepresenta-tions in the draft. A second reason is to allow man-agement to review the opinions and recommenda-tions and offer their suggestions. This is an impor-tant step, even though the facts, opinions and rec-


ommendations may have been discussed with theminformally beforehand. The actual wording of thereport is important because the manner in whichmaterial is presented in the draft may have implica-tions that are both unknown and unintended bythe auditors. It is important, therefore, for auditorsto get the benefit of management’s views on thepresentation of the material in the final report.Done appropriately, this will in no way compromisethe auditors’ independence.

Another reason for exposing the draft reportto management is to give them an opportunity toprepare their comments on the opinions and rec-ommendations slated for inclusion in the finalreport and to develop plans to deal with thoseitems on which implementation can begin. Whileit is desirable that there be unanimity on opinionsand recommendations, it is too much to expectthat this can always be achieved. Courtesy alonedictates that management should have a chance toreview and comment on the draft and to prepareany rebuttal or comment they think appropriate.Affording management this opportunity alsohelps ensure that they will maintain a positiveattitude towards the audit and its ultimatefindings.

A word of caution: although it should be theobjective of all comprehensive auditors to gainmaximal cooperation from the auditee, they cannotalways overcome some natural resistance. Auditorsmay find that the auditee procrastinates in return-ing comments and may even jeopardize the audi-tors’ reporting schedule. Accordingly, it is best ifauditors allow an adequate but definite time formanagement to comment on the draft report.Once the timetable is established, it should beadhered to unless there are sufficiently convincingreason for auditors to change it.

P O I N T- F O R M R E P O R T

The usual first step towards the final report isthe preparation of a point-form report. A point-form report is not intended to be a polished docu-ment; it is intended to be clear, concise and simple.This report is a listing of audit findings that theproject leader thinks should be brought to theattention of management, including mattersthought to be of sufficient import to interest theclient governing body.

The point-form report presents findings inrelation to audit objectives and criteria, and foreach finding includes a description of audit evi-dence, causes and impact, opinions and recommen-dations. This report should be cross-referenced tothe audit working papers and supporting evidence.The point-form report can be for one audit projector for the entire audit, depending on the size.

It is important that the point-form report bethoroughly challenged internally to ensure thatfindings are sound. One technique is to arrangeformal internal challenges within the audit team(each member challenging the other’s finding). Inaddition, members of an advisory committee orthe head of the audit unit could challenge reports.

It is expedient to use the same point-form for-mat in subsequent oral or initial written presenta-tions of audit findings to management of the auditentity. As mentioned above, the information pre-sented must be clear and concise, and must containconvincing support for the opinions and recom-mendations. It is at this time that the audit teamcan vet its opinions and recommendations and getan initial response from senior management.

T H E F I N A L R E P O R T

From the point-form report, the audit teamleader prepares the draft of the final report, keep-ing in mind the matters discussed earlier in thischapter and the CICA Standards. Findings consid-


ered to be of lesser importance can be written upin a management letter. This allows the formalreport to concentrate on the main points and topresent them in a crisp, succinct fashion.

Writing the final report is often the most dif-ficult part of an audit. Weeks, and often months,of work must be summarized in a short documentthat presents in a clear and objective fashion themajor observations the auditors want to bring tothe attention of the governing body.

Since these pages are often the only visibleand tangible evidence of the auditors’ work, andbecause their credibility and usefulness to the orga-nization will be judged thereon, it is vital to takegreat care in preparing this report.

MAKE THIS REPORT SHORTER—AND MORE DETAILED.

W I N S T O N C H U R C H I L L

The report should be so written as to reflectaccurately a clear understanding of the audit entity,its environment and key people. It should bear thehallmarks of truly professional reporting: it should bewritten in a style and with wording that minimizesthe chance of its being misunderstood by the intend-ed reader: it should be clear, objective, fair, reason-able and persuasive. To the extent possible, auditorsshould try to make their reports interesting—manyaudit reports fail to be read, or are only partially read,because they lack substance and imagination.

In preparing reports, it is well to keep inmind the interests of—and constraints on—theclient. Many members of governing bodies—legis-lators, for example—are exceedingly busy peoplewho receive vast amounts of material to read anddigest. It is unrealistic to expect them to read morethan they actually need to know. Brevity, withoutsacrificing completeness, will increase a report’sappeal and utility.

If good communication has been maintainedthroughout the audit, auditors should knowwhether management is aware of the findings andis planning or organizing to remedy the weaknessesidentified. If management is taking correctiveaction, the draft and final report should say so,even if it is too early to assess the efforts or to pre-dict their results. Such actions should, of course, beverified when the draft report is reviewed withmanagement.

In some jurisdictions, it is the practice tooffer management an opportunity to have theircomments on the findings, opinions and recom-mendations included in the final audit report. Inother jurisdictions, management issues separatereports on the auditors’ comments.

After submitting the final report, auditors areinvariably given an opportunity to discuss it withthe governing body or some of its members. Thismay be a session with the full council or board, ormeeting(s) with a public accounts committee oraudit committee of the board. Careful preparationwill make such meetings productive, and manyauditors make liberal use of visual aids for theirpresentations.

M A N AG E M E N T L E T T E R S

During their work, auditors often—even usu-ally—find matters that they think should bebrought to the attention of management but thatare not sufficiently important to mention in thefinal report. These matters are customarily the sub-ject of a separate report to management. Thesereports are analogous to the management lettersthat are normally provided at the conclusion of anaudit. There are, however, some differences thatdeserve comment.

The traditional management letters are usual-ly concerned with detailed matters of internal con-trol, typically of a minor nature. They are relatively


succinct, providing little if any background, analy-sis or explanation, and they are normally of interestonly to financial officers.

Such is not the case with management lettersof comprehensive audits. These reports deal with awide range of issues of interest to many, if not all,senior managers. The reasons for some opinionsand recommendations may not be self-evidentwithout appropriate background information. Inaddition, most managers will not be as familiarwith the process as are financial officers with afinancial audit.

MANAGEMENT LETTERS

• POINTS OF LESSER SIGNIFICANCE

• MAY BE REPORTED DIRECTLY TO THE MANAGER

CONCERNED

• RESPONSE IS OPTIONAL

As a result, the management letter should beboth understandable and professional. Many man-agers will not be familiar with technical audit lan-guage, and the report should be written with itsreader in mind. Its organization should be suchthat it assists readers to find and deal with the sub-jects that interest or affect them. It should clearlypresent opinions and recommendations framed insuch a way that they can be dealt with expeditious-ly within the audit entity.

There is some variety of practice concerningreports to management. One approach is to pro-vide the chief manager (deputy minister, executivedirector, and so on) with a short summary report.

These officials are interested in the overview, notthe details that they leave to subordinates. Thesesummaries touch on only the most important find-ings and recommendations, and it is these that arediscussed in the exit interview with the auditor. Allthe other more detailed material is submitted tothe managers responsible for the items involved. Ithas been found that this approach has helpedensure that the exit interview does not becomebogged down in a discussion of minutiae and thatthe chief executive will direct his or her attentionto those matters that are truly important. Otherpractitioners provide a single report that covers allitems that the auditors consider suitable.

Management letters may or may not require aresponse by management, depending on acceptedpractice in the relevant jurisdiction.

F O L LOW I N G U P

Most legislative audit offices make a practiceof periodically following up on the recommenda-tions they have made in order to assess what actionhas been taken and to report back to the legislature.The extent and timing of audit follow-up is basedon an assessment of such factors as whether theaudit conclusions continue to be applicable, whatmanagement is saying about the actions they havetaken, and the amount of reliance that can beplaced on the work of others. Frequently, thedetailed follow-up is done by the internal audit staffof the audit entity, with the legislative auditor pro-viding a more cursory check later. Such follow-upand reporting is considered particularly useful bylegislative bodies and public accounts committees.


C H A P T E R 1 8

COMPREHENSIVEAUDIT—REPORTINGISSUES


Over time, auditors have devised a standardformat for their reports on financial statements. Anydepartures from that format signal to the reader thatsomething unusual is involved that is worthy of note.This is not the case for comprehensive audit reports,and may never be. The reason is that there are vari-ables involved in these audits and, unless they arespecifically dealt with, readers might draw the wrongconclusions from the report. This chapter exploreshow auditors should deal with these matters. It alsodiscusses the concepts of fair and balanced reportingand audit recommendations, issues that havereceived careful attention by practitioners.

In chapter 16 (General Considerations) threekey variables were introduced—independent butinterrelated—that affect and distinguish compre-hensive audits: scope, intended level of audit assur-ance and significance. These variables pose seriouschallenges for practitioners as they prepare compre-hensive audit opinions. As explained:

• Scope refers to the breadth and depth of theaudit.

• Intended degree of audit assurance refers to theconfidence that auditors have in the accuracyof the opinions expressed in their reports.

• Significance determines what auditors decideto report and what not to report.

S C O P E

As explained earlier, comprehensive audits maycover a variety of subjects, a variety of scopes. Theymay deal with entities large or small, with organiza-tional units or management functions. Scope mustbe decided by the auditors, preferably in consulta-tion with the client. A further aspect of scope is thetime period covered by the audit. Closely related toscope is the issue of the audit objective, which con-cerns the nature of the audit information the auditorintends to report to the client.

Because each comprehensive audit is unique,it is important for users of a comprehensive auditopinion to know what the scope of the audit was.Indeed, the CICA Value-For-Money AuditingStandards require comprehensive audit reports todescribe the objectives and scope of the audit,including any of its limitations. The standards thusrecognize not only that auditors should exercisetheir judgment as to scope, but also that they havea professional obligation to describe the scope ofthe audit so that the users of their reports are notmisled or left to make wrong inferences from theaudit opinion.

Since the requirements of the Standards areclear, and those requirements are reproduced inchapter 23 (Standards and Quality Assurance), it isnot necessary here to elaborate on the issue ofreporting scope. The reader is urged, however, toread the relevant section of chapter 23 in conjunc-tion with the material presented here.

A S S U R A N C E

As described above, intended degree of auditassurance refers to the confidence that auditors havein the accuracy of the opinions expressed in theirreports. Because the scope of comprehensive auditsvaries, as does the detail in which examinations aremade, there is bound to be variation in the degreeof confidence that practitioners will have in the

P A R T I I I . C H A P T E R 1 8 . C O M P R E H E N S I V E A U D I T — R E P O R T I N G I S S U E S 2 7 1

opinions they express in their reports. One mustconclude that there is no right or wrong intendeddegree of audit assurance for comprehensive audits.There are, however, issues to examine.

The central issues dealt with here are the clar-ity of communication by auditors to clients on theintended degree of audit assurance and the risksthat attach to decisions based on audit opinions.Before examining these issues, it will be helpful toconsider the concept of association, because, asmentioned earlier, when auditors are associatedwith information, readers tend to place morereliance on it than if no auditor had been involved.

A B O U T A S S O C I AT I O N

One of the keys to dealing with the conceptof assurance in comprehensive auditing is the well-established notion of association. Auditors, includ-ing comprehensive auditors, become associatedwith information by their involvement in reportingon or preparing that information.

In the vast majority of comprehensive audits,auditors themselves have actually compiled andsynthesized most of the performance informationthat is presented in their reports. In logic, there-fore, as well as in practice, this results in thestrongest possible association with the information.

Although auditors have no control over third-party assumptions about their involvement withinformation, or how others represent their involve-ment with information, auditors can control howthey associate themselves with information.

The CICA Handbook deals succinctly withassociation:

When a public accountant associateshimself or herself with information by per-forming services in respect of that informa-tion, the public accountant should appropri-ately communicate the nature and extent ofhis or her involvement with the information

when such communication is required by theHandbook, or when he or she determinesthat a communication is necessary to avoidmisunderstanding.264

When public accountants describe the natureand extent of their involvement through theirreports, they are communicating the extent towhich readers should place confidence in the infor-mation contained in those reports or in the infor-mation to which they pertain. This should hold incomprehensive auditing too—comprehensive auditreports should communicate the nature of theauditors’ association with the information referredto or contained in them. If the nature of associa-tion varies from audit to audit, this should be evi-dent from the auditor’s communication. If thenature of association is constant, that too shouldbe stated. This should be done in such a way thatreaders will have a proper appreciation of theextent to which the auditors themselves have confi-dence in the accuracy of the opinions expressed intheir comprehensive audit reports.

I M P L I C AT I O N S

The assurance implications of reports basedon different approaches to comprehensive auditingare worth examining:

Prevailing practice based on audit criteriaIt is common for auditors, in conjunction

with the client and/or management, to set criteriaand then report whether or not, or the extent towhich, they have been met. Where criteria havegenerally been met, the audit opinion will say soand give a positive assessment of the organizationor program audited. Where the criteria have gener-ally not been met, the auditors’ opinion will likelylead the reader to conclude that the organization isnot well managed.


If an organization is poorly managed, audi-tors may find that all their opinions are negative,and the resultant report will resemble an exception-based report. The difference in assurance that thereader legitimately may draw, however, is pro-found. In an exception-based report, the readershould not assume anything beyond the reporteditems. The readers of reports based on performancecriteria can legitimately derive a degree of assur-ance that all significant opinions, good and bad,about the area subject to audit have been reported.

Prevailing practice tends to be for auditors topresent their opinions without addressing the issueof risk of error even in cases where the risk of errorin the opinion is variable.

Over time, auditors have come to have con-siderable confidence in their findings using auditcriteria. This has, on occasion, led them into mak-ing broad statements about the quality of adminis-tration of other parts of the organization that havenot, in fact, been subject to the same detailedaudit. In effect, the auditors assess the results ofaudit tests from various sources and then formbroad opinions at the highest level. The followingis an excerpt from the 1988 report of theProvincial Auditor of Ontario.

It is difficult to establish measurable cri-teria against which the overall administrativeperformance of government can be assessed.Consequently, we do not have the necessaryaudit evidence on which to base such anassessment. Nevertheless, we are not withoutsome conviction in the matter.

We believe that professional trainingand years of experience enable us to recognizegood management and to understand goodmanagement practices. We also believe thatthrough our long audit association with gov-ernment, we have a good understanding ofthe nature of government and its operatingenvironment. Based on these factors, it is our

opinion that the Government of Ontario isbeing satisfactorily administered overall.

While there will always be instances ofweaknesses and deficiencies in operations ofthe scope and size of the OntarioGovernment, it is our view that these are theexception rather than the rule, and that onbalance, administration of public funds con-tinues to improve.265

Note that the opinion is very carefully worded tocaution readers that it is not entirely supported byaudit evidence. This should make the reader realizethat the auditor intends that readers draw some-thing considerably less than normal assurance fromhis or her opinion. This is a good example of anauditor being clear in communicating the natureand extent of the work performed on informationwith which he or she has been associated.

A final point deserves mention concerningthese types of reports. Some practitioners believethat readers will derive more assurance when theauditors do the reporting directly than when theyattest to accountability reports prepared by man-agement. This view is held even though it is gener-ally acknowledged that management is in a betterposition to provide accountability information.The reason for this view is simple: it is the greatertrust that readers presumably place in auditors.Readers expect that auditors are expert in auditing.Add to this the expectation that auditors are objec-tive and independent, and it is logical to assumethat the information they provide is even morereliable than information prepared by managementand attested to by the auditor. The result is thatreaders are probably deriving a high degree ofassurance from the criteria-based reports preparedby auditors. This underlines the importance ofensuring that the reports clearly state the auditor’sintended degree of audit assurance.


The CICA Value-For-Money AuditingStandards require that auditors communicate, intheir opinions, the scope of the audit work under-taken and their audit criteria. Nevertheless, scope isitself not the determinant of the degree of assur-ance that auditors may wish their clients to derivefrom the information contained in this type ofcomprehensive audit opinion.

Because there is no one generally understoodnorm for the degree of assurance that should bederived from a comprehensive audit, and becausethe breadth and depth of such audits vary widely,the opinion pursuant to each audit is unique.Without some form of readily understandable anddistinct communication in the report on this mat-ter, there is a danger that the readers’ expectationsof the opinion will be unjustified.

To avoid this danger, comprehensive auditreports should indicate, as clearly as possible, theintended degree of assurance that auditors wantreaders to derive from those opinions.

S P E C I A L E X A M I N AT I O N S

The 1984 amendments to the FinancialAdministration Act have provided comprehensiveauditors with significant challenges, one of whichis implicit in the legislation: it is looking for con-sistent levels of assurance from the examinations ofall the Crown corporations. Another challenge hasto do with scope and is explicit in the legislation:examiners have been asked to provide opinions onperformance matters that cover the whole Crowncorporation and its subsidiaries.

Despite the breadth of opinion required ofthe examiner, and the legislated obligation that theopinion indicate “whether in the examiner’s opin-ion… there are no significant deficiencies,” thisform of comprehensive audit reporting essentiallyfalls into the same generic category as thatdescribed in the preceding section.

AT T E S TAT I O N R E P O R T S

At the time of writing, experience with com-prehensive audit attestation opinions is limited,although there are some examples.

There is no question that comprehensiveauditors in this mode will be in a position to pro-vide opinions from which assurance will bederived. After all, management will make represen-tations and auditors will attest to their fairness.The scope of the audit will be driven by thebreadth of operations or transactions to which therepresentations apply. This, in turn, will be deter-mined either by legislation or other authoritativeforms of mandating, or through an agreed under-standing with the client.

The key questions in the evolution of com-prehensive audit reporting practice in this area arewhether the intended degree of assurance thatauditors will ascribe to attestation reports will bethe same as for the other forms of reporting theyuse, and the extent, if any, to which it differs fromthe intended degree of audit assurance of financialstatement attest audits.

Again, there is no single degree of intendedaudit assurance that can be postulated at this time.There may never be. The diversity in nature, sizeand complexity of organizations that may eventual-ly come to make representations on their perfor-mance, with corresponding audit opinions, sug-gests that the intended degree of audit assurancewill likely vary from one audit to the next.

It is therefore important for auditors to com-municate the intended degree of audit assurance sothat, by appreciating as clearly as possible theextent to which the auditors have confidence intheir opinions, readers can avoid reaching wrongconclusions.


R E P O R T I N G “ I N S TA N C E S”

As previously described, this approach,although the first to have been used to report find-ings on the 3Es, is now a much less prevalent formof practice.

As used here, the term reporting instances ismeant to have a specific meaning. It refers toinstances of deficiency that an auditor has simplycome upon in the course of his or her work, notthe product of a planned, criteria-based approachas part of a comprehensive audit. Although itmight look to the reader to be the same as excep-tion reporting—in the sense that both may bereporting deficiencies—it is not. The latter is basedon such a systematic process.

Reporting instances was a form of practicemost frequently used prior to the introduction oflegislated comprehensive audit mandates. Onceformal mandates were introduced, however, audi-tors quickly began to take a more systematicapproach to their work in the 3Es area and adopt-ed the use of predetermined audit criteria.Nevertheless, from time to time, the reportinginstances approach is used, and thus it is importantto examine its implications.

In this form of reporting, no opinions arereported beyond those that pertain to specific defi-ciencies identified. An important issue of associa-tion arises, one that is closely linked to audit scope.In fact, the issue relates more to the auditors’ asso-ciation in the context of the scope of the auditthan to the intended degree of audit assurance.The issue is: did the auditors set out to find all thedeficiencies that might exist and report them, ordid they simply report what was found regardlessof the scope of the audit and the rigour of theirapproach?

The assurance derived is from what is explic-itly stated. Certainly the reader will receive explicitinformation that, in the opinion of the auditors,

economy, efficiency or effectiveness was notobtained in certain instances. The reader could alsoreasonably derive assurance that the informationon the reported instances is accurate because theauditors—professionals with expertise, indepen-dence and objectivity—have brought the deficiencyto light and have been rigorous in collecting andanalyzing relevant evidence. Thus, the intendeddegree of audit assurance with respect to the specif-ic deficiencies reported is high.

Readers may, however, draw inferences fromwhat is not in the report. For example, a readermay assume that the auditors have listed all signifi-cant deficiencies or instances in the report and maythen conclude that everything else within the orga-nization is effectively managed. This type of think-ing has been called “nothing said, all’s well” bysome practitioners. There is clearly a danger thatthis message may be given inadvertently. There isalso the danger that a reader might assume (proba-bly incorrectly) that, having read about problemsin one or two areas, the entire organization is in adreadful mess.

Because of these possibilities, auditors shouldmake their intentions clear, especially if they areonly reporting instances of specific deficiencies thathave come to their attention where they have notbeen deliberately looking for them. Readers canlegitimately derive assurance about those areaswhere there are instances, and they can be confi-dent that only those particular problems exist. Theresults of these audits should not be extrapolated.Readers should not simply assume that all is wellin areas where no instances are reported or, con-versely, that unreported problems abound.

These limitations should be stated specificallyin the report. Auditors using this reportingapproach should be particularly careful to describethe scope of the work accurately. They should alsoconsider cautioning readers that the opinions relateexclusively to specific criteria within the audit area


scoped in, that the results should not be extrapolat-ed and that they should not assume that all is wellthat is not reported as a deficiency.

M A K I N G L E V E L S O F A S S U R A N C E E X P L I C I T

To date, comprehensive auditors have largelyignored, in their formal reports, the issue of com-municating the intended degree of audit assuranceto readers. This has occurred even though theyundertake widely varying audits that produce sub-stantial differences in the degree of certainty withwhich opinions can be presented.

The proposition offered here is straightfor-ward: practitioners who prepare comprehensiveaudit opinions should describe, in their reports, theextent to which they have confidence in the accu-racy of the opinions expressed in these reports. Inso doing, they will signal to readers, and help themunderstand, the assurance that they should derivefrom the information contained in the opinion, orfrom the information to which the opinion applies.

The need to communicate the intendeddegree of audit assurance as clearly as possible isunderscored by the reality that the intended degreeof audit assurance often varies. If it does, it shouldbe described. If, in fact, it does not, auditors stillhave a responsibility to ensure that whatever degreeof assurance is intended is well communicated toreaders. This might be done either through anarticulation of the level of intended audit assur-ance, or by comparison to the normal degree ofassurance that those same auditors associate with afinancial statement audit. The major challenge forpractitioners here lies in finding the best words,understandable to readers, to communicate thedegree of audit assurance that they intend. It is notappropriate simply to attribute a percentage confi-dence ranking, as a number of variables will haveto be taken into account.

As a separate, but equally important, matter,providing audit opinions that communicate thedegree of intended audit assurance opens up newand useful opportunities for communication withclients. Auditors and clients can consult on choicesthat are available and the value that clients associatewith these choices. For example, some clients mayprefer receiving a lower degree of audit assurance onmore widely scoped audits, while others may wantto have relatively high degrees of assurance on morenarrowly scoped examinations. For this dialogue tooccur and the appropriate choices to be made, how-ever, knowledgeable clients must understand notonly matters about audit scope, but also the degreesof assurance that attend audit decisions.

The forgoing deals with the high professionalqualities that should be associated with the com-munication of a completed audit. These same qual-ities should also be an important part of the com-munications between auditors and their clientsfrom the outset, for it is in the planning phase thata client’s expectations are developed. Accordingly,to ensure that maximum benefit is derived, it isimportant that auditors address this topic in theconsultations that they have with their clients atthe time of planning audits.


The concept of significance is fundamental toall auditing. It is a user-oriented concept that helpsauditors decide what information will be impor-tant to the readers of their reports. The major diffi-culty in dealing with the concept is that each read-er may have an individual view about what is or isnot important. Such views will reflect the reader’sbackground, experience, expertise and particularinterest in the subject of the audit.

Auditors cannot be expected to write reportsthat will completely meet the needs of all users. Thatwould be impossible, given the number and diversi-


ty of potential users and the purposes for which theywant such reports. Accordingly, auditors must exer-cise professional judgment in deciding what must bereported and what does not need to be.

In making decisions about what to report,auditors are guided by two central criteria:

• information needed to support the auditopinion, and

• information needed or required by clients tofulfill their responsibilities.

Auditors apply these criteria in deciding whatmight influence the judgments that a reasonableand knowledgeable reader might make aboutwhether the responsibilities conferred by governingbodies have been adequately discharged by man-agement. In so doing, auditors take into accountthe breadth of subject matter and the degree ofassurance they want readers to have in the auditors’opinions. To date, there are no norms to guidecomprehensive auditors in the area of significance,and practice varies widely.

Beyond the above two criteria that attendauditors’ judgments on what to report, some fur-ther considerations come into play to support theirdecisions. After all, such judgments are not madein a vacuum. The following section examines thefactors that influence comprehensive auditors’ deci-sions about what is and what is not significant, andhow they go about dealing with them.

I N F LU E N C E O F T H E S U B J E C T M AT T E R O F

C O M P R E H E N S I V E A U D I T S

To understand the factors that influence judg-ments about significance, one must first reflect onthe subject matter of comprehensive auditing:economy, efficiency and effectiveness. These terms,and the twelve attributes of effectiveness, are dis-cussed in detail in Part II.

It is immediately apparent that, to a largerextent than pertains in financial auditing, the sub-jects of comprehensive audits are described in qual-itative, not quantitative, terms. Of course, mone-tary measurement is important—indeed critical—in many instances, but determination of the extentto which there is due regard for economy, efficien-cy and effectiveness will often rest on nonmonetaryfactors. Some of these factors are quantifiable innonmonetary terms, but some are capable of onlyqualitative assessments: they are simply not quan-tifiable. Furthermore, given the range of mattersencompassed in these audits, there is of necessity abroad array of measurement tools used in formingjudgments about the quality of an organization’sperformance.

These measurement tools are the performancecriteria that have been developed over the years.While there is no single set of generally acceptedperformance criteria as such, there is no shortage ofindicators that can be tailored to individual auditcircumstances.

Obviously, the determination as to whetheror not performance criteria have been met requiresprofessional judgment. Once that judgment hasbeen made, however, there still remains the ques-tion: How does one determine whether success orfailure in meeting a performance criterion is signif-icant enough to report? The scope of each audithas a vital influence on the answer to this question.

I N F LU E N C E O F A U D I T S C O P E

As has been seen, under most legislated man-dates, auditors are directed to report matters thatthey consider to be important, including instanceswhere there is a lack of due regard for the 3Es.Simultaneously, the auditors are given responsibili-ty to decide what will be audited; they determinethe scope for each individual comprehensive audit.As a result, they may audit at the government-wide


level, the department level, the program level, theactivity level or the subactivity level. There areexamples of audits at all these levels.

Auditors judge significance in the context ofthe audit scope. Clearly, a finding that is significantto a subactivity may not be significant to the pro-gram, department or government as a whole. Thewider the breadth of an audit, the more likely thatauditors would ignore minor deviations fromaccepted criteria in their opinions. In a govern-ment-wide audit, for example, such minor deficien-cies in subactivities would not be consequential todecision making with respect to the organization asa whole. Conversely, even small deviations from cri-teria could merit reporting where the scope of anaudit relates to only a very small portion of theorganization’s overall programs or activities. Auditscope clearly influences decisions about significance.

The mandate for special examinations ofCrown corporations, for example, calls for an over-all opinion on the systems and practices of the cor-poration as a whole; a very wide scope indeed. Theexaminers were able to provide these opinions, insome cases without qualification. In all these opin-ions, the significance of detailed audit findings wasconsidered in relation to the corporation as awhole, and the only deficiencies reported werethose that were sufficiently important to affect, orpotentially affect, the performance of the wholeorganization.

At the other end of the scale, opinions writ-ten for narrowly scoped audits tend to provideinformation at a level of detail that would have noplace in the report of a government or department-wide examination.

In practice, it would be impossible for theopinion resulting from a widely scoped audit tocontain the same level of detail as one resultingfrom a narrowly scoped audit. This is because thescope influences the design of the audit and thelevel at which the audit effort is made. In effect,

decisions about scope are sometimes made well inadvance of detailed planning. These decisions sub-sequently have an impact on what auditors arelikely to report.

S I G N I F I C A N C E FAC T O R S

Although audit scope obviously affects deci-sions about significance, it does not providedetailed guidance about what should be reported.

FACTORS AFFECTING DECISIONS ABOUT

WHAT TO REPORT

• DOLLAR VALUE

• RISK OF OCCURRENCE OF AN UNDESIRABLE EVENT

• IMPACT ON THE PROGRAM BEING REPORTED UPON

• POTENTIAL FOR SAVINGS

• ENVIRONMENTAL IMPACT

• SOCIOECONOMIC IMPACT

• SAFETY

• RESOURCES COMMITTED

• RISK OF POOR MANAGEMENT

• PREVIOUS HISTORY OF LACK OF DUE REGARD

• ACCOUNTABILITY CONCERNS

• QUALITY OF CONTROLS

• DETERRENT VALUE

• ETHICS, INTEGRITY, COMPLIANCE WITH AUTHORITIES

• ISSUES THAT MANAGEMENT OR CLIENT EXPRESS AN

INTEREST IN

• INACTION ON PREVIOUSLY REPORTED ITEMS


How to report versus what to reportDiscussions with practitioners about signifi-

cance factors revealed that some factors they citedfocus more on how comprehensive audit findingsmight be reported than whether they should bereported in the first place. A useful first step, there-fore, is to distinguish between those factors thataffect the manner in which auditors report on amatter and those that help determine whether amatter is worth reporting at all.

While the list at left is not intended to cap-ture all possible significance factors, it does listthose that practitioners agree most frequently affecttheir judgments on what to report.

A number of other factors initially cited asaffecting an auditor’s judgment about what toreport were, upon discussion with practitioners,determined to be factors that influence how audi-tors report on a matter once they judge it to be sig-nificant. The following list lists these factors.Again, it should be noted that the list simply con-tains those factors that were most frequently men-tioned, not necessarily all that were cited or may beconsidered.

FACTORS CITED AS AFFECTING

HOW TO REPORT

• PUBLIC EMBARRASSMENT TO GOVERNING BODY OR

MANAGEMENT

• POLITICAL SENSITIVITY

• PUBLIC PERCEPTIONS AND INTEREST

• PROBABILITY OF PROMOTING CHANGE

• THE AUDITOR’S THEMES OR PRIORITIES

• CRIMINAL OR CIVIL-LEGAL IMPLICATIONS

• SECURITY CONSIDERATIONS

• PERSONAL AND/OR OTHER PRIVACY CONSIDERATIONS

Comprehensive auditors are acutely aware thatthey operate in a public environment where theimpacts of their reports can be substantial andwhere their opinions can affect a broad range ofindividuals or organizations. To the extent possible,they want to avoid their opinions being misinter-preted; thus, they exercise careful judgment abouthow to report on matters that they deem signifi-cant. For example, in some cases, the factors in theabove list will affect the language that is used in thereport, the context in which the opinion is providedand the level of supporting detail that is included.

For example, auditors pay careful attention tothe topic of partisan political impact. It would benaive to think that auditors would not be aware ofitems that would likely attract intense public atten-tion. Should partisan political implications affectwhat an auditor might report? Clearly it shouldnot. Auditors must remain independent and objec-tive to retain their professionalism and credibility.But they should be particularly careful in wordingan opinion they know to be politically sensitive.

A final word about the factors in both lists isin order—it is readily apparent that there is a grayarea between these factors. Some factors thatappear in the How to Report list are also reflected,to various degrees, in the What to Report list.

A good example has to do with possible crim-inal or civil-legal proceedings resulting from thecontent of a comprehensive audit report. This pos-sibility would certainly affect how auditors presenttheir findings. At the same time, it is altogetherreasonable that important deficiencies in the obser-vance of ethical standards or compliance withauthorities would have to be reported by auditors.

Although it is recognized that the manner ofreporting can affect the usefulness of a comprehen-sive audit report, the focus of attention here is onthe central question of how practitioners decidewhat is to be reported.


What to reportThe discussion of how auditors decide what to

report focuses on the point in the audit processwhere all testing has been completed and the find-ings are being evaluated for possible inclusion in thereport. This is not meant to imply that significancedecisions begin and end at the reporting stage. Onthe contrary, significance decisions arise and areimportant at all stages of the audit. In fact, practi-tioners say that their perception of significance isfluid. As the audit progresses, more and more infor-mation is gathered, and this information affects theauditors’ view of what is important. Nevertheless,the focus here is on how auditors decide what toreport, having completed their fieldwork.

When all the facts have been collected andanalyzed, auditors will find that some matters areclearly reportable, others clearly insignificant. Thegray in-between area requires fine judgments. Allthe factors grouped in the What to Report list influ-ence decisions in this area. These factors can begrouped into the following categories:

• relative size of expenditures or revenues;• trends in an organization’s performance;• comparative indicators;• behavioural matters; and• potential risks.

Relative size of expenditures or revenuesIt is not surprising that the amount of money

involved is an important factor. It has an influence onmost reporting decisions, as well as in determiningscope. It may be used to quantify or explain systemproblems and weaknesses by pointing out the dollarvalue that the problem has already produced or couldpotentially produce. The term relative is used becausethe significance of the amount of money involved willdepend on the context, largely determined by theaudit scope. As already explained, what may be con-sidered significant at the program level may be clearlyinsignificant to the organization as a whole.

Trends in an organization’s performanceTrends in performance will affect auditors’

judgments. For example, that which may not besignificant in an organization that historically has agood performance record may assume reporting sig-nificance in an organization whose track record isuneven or eroding. Expressed in either monetary orother terms, the pattern of findings is important.

Comparative indicatorsCompared to similar operations or organiza-

tions, relative performance will influence whetheran item is or is not reportable. Auditors will takeinto account the constraints under which manypublic sector managers operate. Management maybe performing below a theoretical optimum but,given the environment, may be comparatively highperformers. This will influence reporting decisionsto avoid misleading the readers.

Behavioural mattersThese factors include public perceptions and

interest, the probability of promoting change,potential for greater economy, efficiency and effec-tiveness in operations, social or economic impactsand environmental impacts. These factors areimportant because of the environment in whichpublic bodies operate. Performance is judged not bya bottom-line profit or loss, but by the amount ofbenefit delivered to the public relative to the costs.These benefits tend to be qualitative in nature andtherefore require sophisticated professional judg-ments. It is notable that recent legislative auditreports have increasingly focused on such matters.

Potential risksAn item may be significant and reportable

solely on the basis that there is risk involved: riskof potential loss, or risk of inefficiency or ineffec-tiveness. This notion extends to a consideration ofthe probability of occurrence and the degree of


potential severity of consequences. Thus, risk can,in some instances, help define what is significant inthe view of the auditor.

S U M M A RY O F FAC T O R S

In summary, the five broad areas describedabove must be considered in each case. The weightgiven to them will depend on the particular cir-cumstances. Each decision on what to report willbe influenced by earlier decisions about thebreadth of the audit, the degree of intended auditassurance, the nature of tests applied, the findingsfrom those tests and the auditors’ judgment ofwhat might make a difference if reported. Theweighting will often depend on the terms of refer-ence, with some being definitive and others (as inmost legislated mandates) providing auditors withgreat latitude in exercising their judgment.

In the end, there is no substitute for the audi-tor’s judgment in determining what to report. But noauditor should hope that management and governingbodies will be satisfied with the answer “trust myprofessional judgment” when they ask why some-thing was reported or why certain other matters werenot. Auditors must strive to explain their rationale indetermining how reporting decisions are made.

Of all the issues involved in reporting com-prehensive audits, the concept of significance suf-fers from the weakest conceptual foundation.While practitioners can readily agree on the fac-tors that should influence significance decisions,to date they have given little consideration to dis-tinguishing those that influence the manner ofreporting from those that determine what theyreport. There is an important distinction betweenthese two issues that has a profound impact oncomprehensive audit reporting.

Auditors do not generally discuss their signifi-cance criteria with their clients or explain them intheir reports. And yet, from a client’s viewpoint

there could be little more important to understandabout a comprehensive audit report than why somematters were reported and others were not. Hereinlie the dangers: danger of unnecessarily idiosyncrat-ic practice, and danger of distrust—or at best, lackof understanding—of comprehensive audits byclients.

The listing of factors influencing decisions onwhat and how to report should provide a basis for fur-ther thought about this subject and be of use to practi-tioners in making future reporting decisions. Althoughit is a start, it will not of itself provide a blueprint.There can be no single, simple formula for using thefactors that go into making significance decisions. Likeall else, these decisions need careful professional judg-ment taken in the context of each audit.

Practitioners should pay particular attentionto this subject and clearly understand why theyincluded or excluded matters from their opinions.The next step is to communicate this thinking tothe users of the opinions. To increase understand-ing and acceptance of comprehensive audits, audi-tors should specify in their reports the significancethresholds they are using and the key factors thathave guided their decisions about what they choseto include in their opinions. The test of doing thissatisfactorily is not necessarily being able to expressthese thresholds and factors as precise quantitativedecision points or terms—in some instances thiswill be feasible, in others it will not. Rather it liesin the tightness and clarity of the logic beingapplied, and explained in a way that the reader willunderstand and judge as reasonable and appropri-ate to the circumstances.

It is helpful for practitioners to discuss thistopic with their clients outside the context of theirformal reports. This could include discussion ataudit committee meetings or their equivalent anddiscussion with governing bodies at the time ofdelivering the audit report. This type of communi-cation with the client is not a substitute for clear


treatment in the auditors’ formal report. It can,nevertheless, go a long way in ensuring a reasonablelevel of expectations on the part of the client and anagreed basis of understanding between client andauditors. Such discussions can also help auditorsunderstand what their clients need to know.

A U D I T O R S ’ R E C O M M E N D A T I O N S

Auditors’ recommendations have become anaccepted part of most comprehensive audit reports.Indeed, some legislative audit offices have adoptedthe practice of listing the recommendations theyhave made over the years and the action that thegovernment has taken on them.

This is what the U.S. General AccountingOffice (GAO) has to say about the subject:

GAO ACCOUNTING STANDARDS

7.21 AUDITORS SHOULD REPORT RECOMMENDATIONS FOR

ACTIONS TO CORRECT PROBLEM AREAS AND TO IMPROVE

OPERATIONS.

7.22 AUDITORS SHOULD REPORT RECOMMENDATIONS WHEN

THE POTENTIAL FOR SIGNIFICANT IMPROVEMENT IN OPERA-

TIONS AND PERFORMANCE IS SUBSTANTIATED BY THE REPORT-

ED FINDINGS. RECOMMENDATIONS TO EFFECT COMPLIANCE

WITH LAWS AND REGULATIONS AND IMPROVE MANAGEMENT

CONTROLS SHOULD ALSO BE MADE WHEN SIGNIFICANT

INSTANCES OF NONCOMPLIANCE ARE NOTED OR SIGNIFICANT

WEAKNESSES IN CONTROLS ARE FOUND. AUDITORS SHOULD

ALSO REPORT THE STATUS OF UNCORRECTED SIGNIFICANT

FINDINGS AND RECOMMENDATIONS FROM PRIOR AUDITS THAT

AFFECT THE OBJECTIVES OF THE CURRENT AUDIT.

7.23 CONSTRUCTIVE RECOMMENDATIONS CAN ENCOURAGE

IMPROVEMENTS IN THE CONDUCT OF GOVERNMENT PROGRAMS.

RECOMMENDATIONS ARE MOST CONSTRUCTIVE WHEN THEY ARE

DIRECTED AT RESOLVING THE CAUSE OF IDENTIFIED PROBLEMS,

ARE ACTION ORIENTED AND SPECIFIC, ARE ADDRESSED TO PAR-

TIES THAT HAVE THE AUTHORITY TO ACT, ARE FEASIBLE, AND,

TO THE EXTENT PRACTICAL, ARE COST-EFFECTIVE.266

Recommendations result from a constructiveapproach to comprehensive auditing. It is assumedthat clients want to be told what needs to be cor-rected and that auditors would be remiss not topoint out opportunities for improvement. Theissue here is simply whether or not the auditor’sreport is the best vehicle through which to deliverthe message. The following questions concerningrecommendations are addressed below:

• Why are recommendations provided as anintegral part of the comprehensive auditreport rather than in a separate communica-tion to the client or the client’s management,as appropriate?

• Are comprehensive auditors increasing theiraudit risk by including recommendations inthe audit report?

• How rigorously should recommendations bedeveloped?

• Does providing recommendations create anobjectivity problem for auditors?

W H Y P R OV I D E R E C O M M E N D AT I O N S I N T H E

R E P O R T ?

The usual reason given for including recom-mendations as an integral part of the audit reportis that clients want it that way.

But why not provide this service through aseparate communication? After all, it is commonpractice for auditors to provide management lettersthat are not made public but contain observations,including suggested improvements. Recommend-ations could be provided through such a vehicle.

Auditors give three reasons why recommenda-tions should be contained in the published report.First, a number of practitioners think that clientswould not want what they may view as the prima-ry benefit from a comprehensive audit to be givensecondary reporting treatment and not included inthe most visible output of the audit, the report


itself. Second, auditors think that including therecommendations in the audit report increases thelikelihood of action being taken on them. Third,the recommendations, if included in the auditreport, serve as a visible benchmark against whichfuture progress can be monitored by governingbodies.

While there is general agreement that recom-mendations are not a necessary part of the concep-tual framework of comprehensive auditing, mostpractitioners think it is a useful practice thatshould continue.

Since it seems that recommendations willappear in comprehensive audit reports for sometime, it is appropriate to consider the ramificationsof this practice.

I S T H E A U D I T O R’ S A N D C L I E N T ’ S R I S K I N C R E A S E D ?

First, it is certainly true that recommenda-tions are elevated in status when they appear in theaudit report. The readers will likely infer that therecommendations are important and, if imple-mented, will result in better performance. After all,they are based on a professional’s judgment, and,since they are in the audit report, readers mayderive assurance that the recommendations will beeffective.

Arguably, this is the biggest problem inincluding recommendations in the report. Italmost guarantees the creation of an unrealisticexpectation because in reality the recommendationsare rarely subjected to the same audit rigour that isapplied in the audits themselves. This does notmean that the recommendations are without soundbasis, but it does mean that there is some degree ofrisk associated with them, both for the auditor whomakes them and for the client who acts on them.

L E V E L O F R I G O U R R E Q U I R E D

An auditor’s training and experience shouldprovide a basis for practical and worthwhile adviceto clients simply as a by-product of the audit work.Such advice is not as reliable as the audit opinions,however, because recommendations are future-ori-ented and cannot easily be tested a priori.

Comprehensive auditors must weigh the riskassociated with including the recommendations aspart of the report against the apparent clientdemands for this service. Auditors run the least riskwhen they simply identify a problem and recom-mend that it be corrected. But clients may be dis-satisfied with such a relatively unhelpful suggestionfrom a professional who should know how thatcorrection can be made. But auditors run a greaterrisk when they provide details of how problemsmay be overcome. Moreover, giving managementlatitude to determine how best to implement rec-ommendations often fosters greater ownership andleads to actions instead of defensive posturing.

In the end, however, professionalism mustprevail, and it appears reasonable that comprehen-sive auditors should do everything possible toensure that their recommendations are developedwith a rigour equivalent to the contents of the restof the report. If they are not, auditors shouldappropriately communicate the limitations of therecommendations.

A N O B J E C T I V I T Y P R O B L E M ?

Some practitioners question the merits ofmaking recommendations because of the long-term implications of the practice. They pointout that, over the years, thousands and thou-sands of recommendations have been made, andit is virtually impossible to follow up on them toassess whether or not any impact on perfor-mance was actually achieved. In addition, theythink that auditors could find themselves in an


awkward position when faced with auditingareas where their recommendations were imple-mented. In such circumstances, it would be dif-ficult to avoid the perception that the resultingaudit opinions may not be as objective as theyought to be.

Some practitioners are also concerned thatwith so many recommendations being made, somesuggestions, despite the auditor’s good intentions,will be inappropriate and not actually achieve bet-ter performance. This could have a damaging effecton the responsible auditor’s credibility and that oftheir future findings and opinions.

Recommendations are a valuable by-productof comprehensive auditing, but great care must beexercised to avoid creating a situation where theymay compromise the auditor’s objectivity.

S U M M A RY

Providing recommendations is not an inte-gral part of the concept, nor a necessary support-ing practice, of comprehensive auditing. Themain purpose of the audit report is to providethe client with useful information about the sta-tus of due regard to economy, efficiency andeffectiveness and/or the fairness and complete-ness of such information reported by manage-ment on the 3Es.

As mentioned above, however, it is appropri-ate to provide recommendations when both theauditor and client think it desirable. Comprehen-sive auditors should maximize the value of theirwork to the client by providing recommendations,but there are alternative means of delivering themthat should be considered for example, manage-ment letters.

F A I R A N D B A L A N C E D

R E P O R T I N G

Fair and balanced reporting is a term that hasbeen associated with comprehensive auditing fromthe outset. Although the term is commonly used,there has been little explanation as to what itmeans and how, as a concept, it influences compre-hensive audit reporting. That it has influencedthese reports is indisputable:

I therefore emphasize to readers that theyshould give special weight to those observations thatare positive; for while briefly stated they do indeedbalance—if not outweigh—negative observationswhich, of necessity, must be described in full.267

It is usually in such a context that the termfair and balanced is used: pointing out the positiveas well as the negative findings of a comprehensiveaudit. There are two basic questions:

• Is fair and balanced reporting a supportableauditing concept?

• What is meant by fair reporting?

A S U P P O R TA B L E C O N C E P T ?

There is remarkable consensus about whetherfair and balanced reporting is a supportable con-cept. However, there is little support for the termbalanced to be associated with comprehensive auditreports. This is not negative in any way; it is sim-ply a recognition of the practical realities of com-prehensive audit reporting.

The biggest problem with the term fair andbalanced reporting is the word balanced. It conjuresup the image of a set of scales upon which positivefindings are weighed against negative findings. Inthe world of comprehensive auditing, the likeli-hood of balance in any true sense is remote. Infact, the term balanced raises questions regardingthe whole audit process, not just reporting. Forexample, would an auditor striving for balance be


required to develop audit tests that would lead tobalanced results? One would hope not, as thiswould strike at one of the key characteristics thatauditors bring to their work—objectivity.

Indeed, striving for balance in the literal sensecould lessen the credibility of auditors’ reports.After all, if balancing were a fundamental concep-tual requirement of comprehensive auditing, howwould auditors deal with the reality of unbalanced(positive or negative) findings? The auditor’s credi-bility would be jeopardized if clients and readersbelieved that some findings or opinions were notreported because they would result in an unbal-anced report.

One can therefore conclude that balanced isnot a part of the comprehensive audit reportingconceptual framework. It is generally agreed thatthe term fair is a more appropriate one thatembraces those aspects of balance that are desirablefrom a conceptual point of view.

W H AT I S FA I R R E P O R T I N G ?

Fairness is one of the professional qualitiesthat auditors bring to their work. The quality ofobjectivity embraces fairness in that auditors have aduty to analyze their audit results and report themin an impartial manner.

The concept of fair reporting does not meansearching for positive findings in an attempt to befair to management. Rather, it means developingappropriate criteria and then reporting whether ornot those criteria are met. Positive findings can andshould be reported, not as an end in itself, but sim-ply as the objective outcome of conducting theaudit. Furthermore, it means putting the findingsand opinions in context. Readers should be able tounderstand the significance of audit opinions—both in absolute and in relative terms—so thatthey will come to fair and warranted conclusions.

From the above, one may conclude thatfairness in reporting is an essential concept in com-prehensive auditing, while balance is not.


262 COMPREHENSIVE AUDITING MANUAL (OTTAWA, OFFICE OF THE AUDITOR GENERAL OF CANADA) §6001.02.263 IBID, §6001.04-.06.264 CICA HANDBOOK, §5020.10.265 OFFICE OF THE PROVINCIAL AUDITOR OF ONTARIO, 1988 ANNUAL REPORT (TORONTO: QUEEN’S PRINTER FOR ONTARIO, 1988) 2.266 GOVERNMENT AUDITING STANDARDS, 1994 REVISION, (WASHINGTON, UNITED STATES GENERAL ACCOUNTING OFFICE) 92.267 ANNUAL REPORT OF THE AUDITOR GENERAL OF CANADA, 1986, PARA 1.15.

C H A P T E R 1 9

AUDIT CRITERIA

W H A T A R E A U D I T C R I T E R I A ?

Auditing cannot be done without audit crite-ria. Criteria are the benchmarks against whichauditors compare what they find in order to drawconclusions. In other words, they are the standardsthat management can realistically be expected tomeet. They describe reasonable norms of behaviourin an organization. They are the measurementdevices against which transactions, events or sys-tems are compared in order to judge whether ornot they are acceptable.

OFFICE OF THE AUDITOR GENERAL OF CANADA’S

DEFINITION OF AUDIT CRITERIA

IN VALUE FOR MONEY AUDITING, CRITERIA ARE DEFINED AS

REASONABLE AND ATTAINABLE STANDARDS OF PERFORMANCE

AND CONTROL AGAINST WHICH THE ADEQUACY OF SYSTEMS

AND PRACTICES, AND THE EXTENT OF ECONOMY, EFFICIENCY

AND EFFECTIVENESS OF OPERATIONS, CAN BE ASSESSED IN THE

PARTICULAR CIRCUMSTANCES OF THE AUDITED

ORGANIZATION.268

In financial auditing, the public accountingprofession uses Generally Accepted AccountingPrinciples (GAAP) as standards or benchmarksagainst which to form its judgments. The long-standing requirement for financial audits and thetechnical nature of the subject matter have led tothe development and acceptance of these standards.

Comprehensive auditing is a comparativelynew field and the subject area is much broaderthan financial auditing. Nevertheless, the auditprocess is much the same, and it is based on crite-ria. In the case of comprehensive auditing, howev-er, auditors need criteria to judge whether opera-tions are economical, efficient and effective.

Audit criteria can apply to several aspects ofan organization’s operations: to the quality andtimeliness of information and its use; to the pro-bity of conduct of officials; to any operating sys-tem, from controls over inventories to staff train-ing, and to the organizational structure itself.

At the very highest level, criteria can be verybroad statements of expectations—assertions ofwhat is good management practice anywhere. At alower level, they can describe very specific expecta-tions of performance. Some criteria may deal withthe amount and quality of information generatedand with the suitability of control systems. Othersmay focus on specific expectations of such thingsas output per unit of input, setting and achievingdeadlines for work assignments, and so on.

Auditors recognize that the criteria they applywill vary from one assignment to another: whatmay be a reasonable expectation for informationsystems in a program spending eight billion dol-lars, for example, may be unreasonable in a pro-gram spending $500,000. The criteria to be usedin any audit must be designed for, and appropriateto, the particular circumstances of the organizationunder examination.

The necessity to devise audit criteria that canreasonably apply in each individual audit arisesfrom the fact that for many management functionsthere are as yet no universally applicable andaccepted practices. But audit criteria must bedeveloped to allow auditors to audit performanceinformation against a standard.


AUDIT CRITERIA

• REASONABLE STANDARDS AGAINST WHICH TO ASSESS

EXISTING CONDITIONS

• THEY EQUATE TO GOOD OR ACCEPTABLE MANAGE-

MENT PRACTICES

• AS MEASUREMENT DEVICES, THEY MUST PROVIDE

SOME FLEXIBILITY BUT STILL ALLOW IDENTIFICATION

OF NONADHERENCE

Criteria are what could be referred to as goodor acceptable management practices. Criteria haveto be general enough to allow management someflexibility in style, but specific enough so that non-adherence to them is clearly evident. It is unrealis-tic to expect that activities, systems or levels of per-formance in economy, efficiency and effectivenessareas will always fully meet the criteria used tojudge them. Satisfactory performance does notmean flawless performance.

And since there are no universally acceptedstandards for many management practices, it isessential that auditors discuss with managementthe criteria they propose to use. Auditors whoassess management performance by yardsticks ofwhich management is unaware court disaster.Every effort should be made to gain consensus oncriteria, although this may not be possible in somecases. Where complete agreement is not reached,auditors should seek precedents, such as practicesin similar programs, operations or jurisdictionsthat would support the appropriateness of the sug-gested criteria. Of course, it is preferable that thesuggested criteria be developed in light of suchprecedents in the first place.

S U I T A B I L I T Y O F C R I T E R I A

As has been pointed out in the CICA publicsector auditing guidelines,269 criteria must be suit-able in order to avoid inappropriate conclusionsbeing drawn about the organization’s operations.They must be relevant to the matters being auditedand appropriate to the circumstances. Suitability ofcriteria depends on factors such as:

• The audit objectives. Criteria that are suitablefor matters related to economy are differentfrom those related to efficiency.

• The activity. Criteria that are suitable forhuman resource management are different fromthose related to capital asset management.

• The approach of the audit. Suitable criteria forexamining processes would differ from thosefor examining results. Examinations ofprocesses are usually related to systems, con-trols and practices. Examinations of resultsusually focus on program outputs, outcomesand impacts.

Some characteristics of suitable criteriainclude:

• Reliability. Reliable criteria result in consistentopinions when used by different auditors inthe same circumstances.

• Objectivity. Objective criteria are free fromany bias of the auditors or management orthe client.

• Usefulness. Useful criteria are those resultingin findings and opinions that meet the client’sinformation needs.

• Understandability. Understandable criteria arethose that are clearly stated and are not sub-ject to significantly different interpretations.

• Acceptability. Acceptable criteria may bedeveloped by management or by the auditorsand be derived from standards established byregulatory bodies, professional associations or

P A R T I I I . C H A P T E R 1 9 . A U D I T C R I T E R I A 2 8 7

other recognized authorities.• Comparability. Comparable criteria are consis-

tent with those used in similar comprehensiveaudits within the organization, in similar cir-cumstances.

• Completeness. Auditors should endeavour toensure that all significant criteria have beenidentified.

These characteristics are considered togetherin identifying criteria and in assessing their suit-ability. The relative importance of the characteris-tics in different circumstances is a matter of profes-sional judgment.

U S I N G A U D I T C R I T E R I A

The level of detail of the audit opinions isaffected by the level of detail at which audit criteriaare specified. At the planning stage, criteria may berelatively general. More specific criteria are usuallyidentified for the detailed examination.

In financial auditing, for example, an auditormay wish to examine accounts receivable. UnderGAAP, the general criterion to be used could beexpressed as follows: “Accounts receivable shouldbe presented fairly.” To arrive at an opinion onthis general criterion, auditors would have toexamine a number of characteristics of the listedaccounts receivable, including ownership, evalua-tion, classification, and so on. Auditors gather evi-dence on each of these sub-criteria to see if theyare met. The opinion on the general criterion,therefore, is reached through using more specificsub-criteria.

In comprehensive auditing, the approach issimilar. For example, in examining the acquisi-tion of a piece of equipment, the auditors’ gener-al criterion might be: “The acquisition took placewith due regard to economy.” To arrive at anopinion, the auditors would develop more specif-

ic sub-criteria about which evidence is to begathered, probably based on central agency ordepartmental directives regarding the acquisitionof equipment. The sub-criteria would likelyinclude the following:

• the need was justified;• the technical requirements were appropriately

specified; • available options were considered;• the acquisition process was suitably managed;

and• custody and control of the asset are provided for.

There may, of course, be even more specificcriteria to assess performance against individualsub-criteria. For example, to determine whetherthe acquisition process was suitably managed, audi-tors would look at such matters as bid solicitation,contractor selection and contract administration,including project controls and work-in-progresspayment policy.

Comprehensive audit criteria are used in thesame manner as in financial audits, but because ofthe broad nature of the audit, they may also serveseveral other purposes, such as:

• to form a basis for communication with man-agement, since their agreement to the criteriais solicited;

• to indicate the scope of the audit in the senseof specifying audit objectives;

• to generate findings and help form and struc-ture the observations; and

• to form a basis for the examination phase ofthe audit, developing the audit program andformulating audit questionnaires.

However they are to be used, audit criteriashould be clearly stated to prevent misunderstanding.


CRITERIA RESPECTING INTERNAL AUDIT: TWO

EXAMPLES

THE FOLLOWING ARE TWO SETS OF CRITERIA DEVELOPED FOR

COMPREHENSIVE AUDITS OF INTERNAL AUDIT FUNCTIONS: ONE

IN A PROVINCIAL LIQUOR COMMISSION, ONE IN A GENERAL

HOSPITAL.

THE LIQUOR COMMISSION

• THE AUDIT PROGRAMS USED BY THE AUDITORS ARE RELE-

VANT AND ADEQUATE

• THE CONTENTS OF THE DIVISION’S INDIVIDUAL AUDIT

REPORTS AND ITS ANNUAL REPORT ARE APPROPRIATE

• THE STAFF CHARGED WITH THE RESPONSIBILITY FOR PER-

FORMING THIS FUNCTION ARE COMPETENT AND INDEPENDENT

• ASSIGNED AUDIT RESPONSIBILITIES ARE APPROPRIATE

THE GENERAL HOSPITAL

• THE FUNCTION IS INDEPENDENT FROM OPERATING DEPART-

MENTS

• THE TWO PERSON STAFF CHARGED WITH RESPONSIBILITY

FOR THE INTERNAL AUDIT FUNCTION ARE COMPETENT

• THE AUDIT PROGRAMS AND THE SCHEDULING OF ASSIGN-

MENTS ARE APPROPRIATE TO THE NEEDS OF THE HOSPITAL IN

TERMS OF RISK, IMPORTANCE AND USEFULNESS

• THE DETAILED AUDIT PLANNING FOR SPECIFIC ASSIGNMENTS

IS ADEQUATE AS ARE THE APPROACHES TAKEN WITH RESPECT

TO SAMPLING TECHNIQUES, TESTS PERFORMED AND EVIDENCE

DERIVED

• THE AUDIT FINDINGS, CONCLUSIONS AND RECOMMENDA-

TIONS WERE REVIEWED WITH DEPARTMENTAL OFFICIALS IN A

MEANINGFUL MANNER

• THE AUDIT REPORTS CONVEY ADEQUATELY THE OBJECTIVES,

SCOPE, FINDINGS, CONCLUSIONS AND RECOMMENDATIONS

FOR EACH AUDIT ASSIGNMENT

• FOLLOW-UP PROCEDURES, TO ENSURE RECOMMENDATIONS

ARE BEING ACTED UPON ON A TIMELY AND PROPER BASIS, ARE

ADEQUATE270

S O U R C E S O F A U D I T C R I T E R I A

Since comprehensive audit criteria address avariety of management practices, it is inevitablethat they derive from a variety of sources. Criteriarespecting probity of behaviour, for example, maybe derived from law, from policy statements (suchas any policies regarding conflict of interest) andfrom statements of ethical practice by professionalassociations. Criteria respecting operating systemsand standards may derive from generally acceptedgood-practice models developed by professions orassociations. The sources may be relatively infor-mal, such as those respecting productivity stan-dards for clerks, or they may be quite formal, suchas those applying to probity of conduct.

Probably the first place auditors should lookfor appropriate audit criteria for a specific audit isin their own office files. If they have conductedaudits on the same or similar entities in the past,they may already possess a number of useful—andtested—criteria. Auditors should not, however,apply those criteria without first reviewing theirsuitability in the current circumstances of theaudit; time brings changes.

In some instances, in increasing numbers, theaudited organization itself has adopted specificmanagement standards for performance. In thosecases, the auditors need only assess those standardsfor reasonableness before adopting them as auditcriteria. Gaining agreement of management tosuch criteria should be automatic.

In other instances, there may be few manage-ment standards in place and audit criteria willhave to be developed. The sources auditors willuse to develop them will be partly their ownknowledge of good management practice, partlytheir findings of what seem to be accepted as stan-dards of performance in similar organizations else-where and perhaps partly the expert advice theymay have to seek.

P A R T I I I . C H A P T E R 1 9 . A U D I T C R I T E R I A 2 8 9

Although most entities have their own partic-ular characteristics, there are certain common fea-tures of public (and private) sector organizations. Itmay well be that issues similar to those faced in anaudit assignment have been faced by auditors inother situations. Criteria may have been developedin other audit organizations that could be used,possibly with modifications, to the audit of anorganization that has inadequate, or inadequatelydocumented, management standards. Gainingaccess to such criteria can save considerable audittime and expense.

One source of audit criteria to which all audi-tors should turn are the directives and guidelinespromulgated by any relevant central agencies orfunders. These are often in a form that permits rel-atively easy conversion to criteria. These directivesand guidelines must, in any event, be taken intoaccount since they set out specific expectations ofmanagement and provide a framework withinwhich administration must be conducted. Even ifthere are no such directives or guidelines constrain-ing the organization, it may be that central agen-cies or funders of other jurisdictions have promul-gated standards that could form the basis of criteriafor the audit.

In using any central agency or other sourcematerial to develop audit criteria, practitionersshould ensure that they do not put themselves in aposition where their work will become merelycompliance audits on behalf of the central authori-

ty. Comprehensive auditors need to keep a broadperspective and must concentrate on those issuesthat are truly essential to the success of the organi-zation—they should not allow themselves tobecome mired in administrative detail at theexpense of what is really vital.

There are several other sources for criteriathat auditors often use. Just because the particularorganization under audit has no or few satisfactory,established management standards that could beused as audit criteria does not mean that such stan-dards may not have been developed elsewhere.Similar entities in other jurisdictions may havestandards that could provide suitable criteria. Evenif there are no such formal standards, the perfor-mance of similar entities elsewhere may provide auseful comparison.

Other sources of criteria include the literaturein the field of the audit subject and of related pro-fessions. In some instances, auditors have consultedexperts in the relevant field when their own knowl-edge of a specialized area is inadequate for thedevelopment of suitable criteria. Often, theseexperts are a part of the audit team from start tofinish. At other times, they are used only forspecific tasks.

As more and more comprehensive audits areconducted, the number and range of suitable auditcriteria grow. By becoming familiar with the manyavailable sources, practitioners can greatly increasethe efficiency of their audits.


268 OAG MANUAL, §2307.02.269 PUBLIC SECTOR AUDITING GUIDELINE 1, PLANNING VALUE-FOR-MONEY AUDITS (CICA, 1990), §.27-.29.270 REPORTED IN COMPREHENSIVE AUDITING IN CANADA: A PROVINCIAL LEGISLATIVE AUDIT PERSPECTIVE (OTTAWA: CCAF, 1985) 116 & 132.

C H A P T E R 2 0

EVIDENCE

W H A T I S E V I D E N C E ?

The collection and analysis of evidence is atthe heart of auditing. The Shorter Oxford Dictionarydefines evidence as: “Ground for belief; that whichtends to prove or disprove a proposition.”

For comprehensive auditors, audit evidence isthe facts or information used:

• to come to the opinion of whether an organi-zation’s management/employees have acceptedand carried out appropriate accounting, man-agement or operational principles, policies orstandards for effectively, efficiently and eco-nomically using its resources (that is, whetherthe audit criteria are being met); and

• to demonstrate to a third party that the audi-tors’ opinion is the correct one.

As discussed, auditors use agreed criteria todetermine whether the auditee has indeed paid dueregard to economy, efficiency and effectiveness.

AUDIT EVIDENCE

THE INFORMATION COLLECTED AND USED BY THE AUDITOR

TO ARRIVE AT AN OPINION OF WHETHER AUDIT CRITERIA ARE

BEING MET.

In carrying out the tasks specified in the auditprograms, auditors will obtain information or evi-dence that will help them arrive at an opinion of

whether or not audit criteria are being met. Thisevidence is obtained during the audit workthrough some of the techniques that are discussedlater in the chapter.

S T A N D A R D S O F V F M - R E L A T E D

A U D I T E V I D E N C E

To guide auditors, and to ensure appropriateprofessional approaches to work, various bodieshave established standards for evidence to be usedin comprehensive audits. The Canadian Instituteof Chartered Accountants has established the fol-lowing standard:

Sufficient appropriate audit evidenceshould be obtained to afford a reasonablebasis to support the content of the auditor’sreport.271

The policy standard of the Office of theAuditor General of Canada is virtually the same:

Sufficient, appropriate evidence shouldbe obtained to afford a reasonable basis forthe content of the report.272

The General Accounting Office of the UnitedStates uses the following standard:

Sufficient, competent and relevant evi-dence is to be obtained to afford a reasonablebasis for the auditors’ findings and conclu-sions…273

These standards introduce a number of keyconcepts concerning audit evidence: relevant, com-petent, sufficient and appropriate. It is important tounderstand what these words signify, although itshould be remembered that they are closely related,and audit evidence should be considered in light ofall of them simultaneously.

P A R T I I I . C H A P T E R 2 0 . E V I D E N C E 2 9 1

R E L E VA N C E O F E V I D E N C E

RELEVANCE

INFORMATION OR FACTS USED AS EVIDENCE MUST BE RELATED TO

THE SUBJECT AND PERIOD OF TIME ENCOMPASSED BY THE AUDIT.

To be relevant, evidence used to support afinding must have a logical, sensible relationship tothat finding. For example:

• Auditors trying to verify the labour element ofproduct costs by reference to source documentsshould be sure that the collective agreementthey examine is current. Using an expiredagreement would produce irrelevant evidence.

• If the policy is to recap tires at 60,000 kilo-metres instead of buying new ones, evidenceshowing that steel-belted radial tires lastlonger than bias-ply tires is not relevant.

CO M P E T E N C E O F E V I D E N C E

Evidence is competent if it conforms to fact.In other words, evidence is competent if it is valid.To be competent, evidence must be obtained froma reliable source. In this context, competence canbe thought of as a synonym for reliability.

COMPETENCE OF EVIDENCE

THE RELIABILITY THAT IS PLACED ON THE SOURCE OF INFOR-

MATION USED AS EVIDENCE.

There is general agreement on the followingways of assessing the reliability of evidence:

• evidence obtained from a credible indepen-dent source provides greater assurance of reli-ability than evidence secured from the auditeeorganization;

• evidence developed under a good system ofinternal control is more likely to be reliablethan evidence obtained where such control isunsatisfactory or nonexistent;

• evidence obtained by auditors through physi-cal examination, observation, computationand inspection is more reliable than evidenceobtained indirectly;

• documentary evidence is usually consideredmore reliable than oral evidence, and originaldocuments are more reliable than copies;

• the reliability of evidence increases when it isconfirmed by another source;

• testimonial evidence obtained under condi-tions where persons may speak freely is morereliable than evidence obtained under com-promising or constrained conditions;

• testimonial evidence obtained from someonewho is unbiased and has complete knowledgeof the area is more competent than testimoni-al evidence obtained from someone who isbiased or has only partial knowledge; and

• statements made by officials of the auditedorganization are more reliable when they areconfirmed in writing.

S U F F I C I E N C Y O F E V I D E N C E

Sufficiency refers to the amount of evidencerequired to come to an audit finding. Auditorsshould ask themselves whether they have enoughevidence to persuade a reasonable person of thevalidity of the finding. In some situations, it maybe useful and appropriate to use statistical methodsto assess sufficiency.

SUFFICIENCY OF EVIDENCE

THE AMOUNT OF EVIDENCE REQUIRED TO COME TO AN OPIN-

ION ON THE AUDIT OBJECTIVE.


Factors to consider in assessing the sufficiencyof evidence include:

• the quality of the evidence gathered;• the level of materiality or significance of the

finding;• the degree of risk associated with coming to

an incorrect opinion;• experience gained in previous audit examina-

tions as to the degree of reliability of theauditee’s records and representations;

• known client sensitivity to an issue;• intended level of audit assurance;• persuasiveness of the evidence; and• cost of obtaining the evidence relative to the

benefits in terms of supporting the finding.

A P P R O P R I AT E N E S S O F E V I D E N C E

Appropriateness refers to the suitability of theevidence for the purpose of supporting a finding oropinion. It is really another way of describing thequality of the evidence: its competence and relevance.

Auditors should always remember that theburden of proof—providing sufficient evidence tosubstantiate an audit opinion—is on them. Theymust use their judgment about what is enough.

Evidence factors to consider in planningWhile planning a comprehensive audit, audi-

tors should identify the probable nature, sourcesand availability of the audit evidence they require.They should consider the following:

• The effect of the audit approach. The nature ofevidence needed if the audit’s primary focus ison processes may differ from that required ifthe focus is more on results.

• The ability to integrate audit work with otheraudits or studies. If other audit reports orstudies are available, auditors may choose touse them as evidence in the audit. If the audi-tors are aware of other planned examinations,

they may choose to coordinate work to beable to rely on their reported results. If evi-dence is to be obtained from other audits orstudies, auditors should assess: the relevanceof the objectives, scope and criteria to theirplanned audit; whether the work was con-ducted in accordance with value-for-moneyauditing standards; the competence and inde-pendence of the auditor or reviewer; and thereliability of the conclusions of the audit orstudy. Auditors should also determine theneed for corroborating evidence. (This sub-ject is discussed in depth in chapter 22.)

• The effect of reporting requirements. Theamount of evidence required to form an over-all opinion on an organization may be rela-tively greater than the evidence needed toidentify and report deficiencies.

• The cost of obtaining evidence. Auditors maynot be able to obtain evidence from a particu-lar source at a reasonable cost for the plannedtiming of the audit. In such circumstances,auditors would assess the possibility ofobtaining other appropriate evidence on atimely basis.

Reliance as a form of evidenceThe term “reliance” is used in a variety of sit-

uations in auditing. Each usage reflects the require-ment that auditors base their opinions or rely onsufficient appropriate audit evidence, often involv-ing the substitution of one form of evidence foranother.

For example, auditors may reduce theamount of detailed substantive testing they per-form on financial statements if they have evidencethat the information reflected in these statements isgenerated by well-controlled information systems.They rely on internal controls to determine thenature, extent and timing of substantive auditingprocedures. Similarly, the auditor of a parent com-


pany may, after carrying out certain high-level pro-cedures, rely substantially on the work of anothercompetent auditor who has audited the financialstatements of a subsidiary.

Auditors may also rely on the work of special-ists such as lawyers, engineers, actuaries or gemolo-gists, who, in their capacity as specialists, provideprofessional opinions about matters that have abearing on the financial statements. To the extentthat auditors can rely on this work of others, theirown work can be reduced.

RELIANCE

RELIANCE IN THE CONTEXT OF A COMPREHENSIVE AUDIT IS

THE SUBSTITUTION OF WORK DONE BY OTHERS FOR WORK

THAT WOULD OTHERWISE HAVE TO BE PERFORMED DIRECTLY

BY THE AUDITORS. IN THE CONTEXT OF THIS DEFINITION,

WORK DONE BY OTHERS INCLUDES THEIR ASSESSMENTS AS

WELL AS THE DETAILED SUPPORTING WORK.

E X T E N T O F R E L I A N C E

Regardless of the situation in which auditorsrely on evidence, certain ground rules apply. First,auditors do not rely blindly; they take steps to sat-isfy themselves that reliance is warranted. Thus,before relying on internal controls over key infor-mation systems, auditors analyze the systems andensure that they have been functioning as intendedduring the period covered by the audit. Beforerelying on the auditors of a subsidiary’s financialstatements, the auditors of the parent company sat-isfy themselves that the auditors of the subsidiaryunderstand their objectives, are aware that relianceis intended and have carried out an audit sufficientto contribute to those objectives. Before relying ona specialist, auditors assess the reputation of the

specialist and ensure that the information providedto the specialist is complete, accurate and timelyand that the major assumptions used by the spe-cialist are consistent with other evidence obtainedby the auditors.

Second, auditors consider the cumulativeimplications of evidence from many sources inreaching their opinions. It is the accumulativeand corroborating effect of evidence from a vari-ety of sources that provides the basis for theiropinions. Any significant inconsistencies betweenevidence from different sources must be resolvedbefore an audit opinion can be offered. Evidencefurnished through reliance must be comparedwith all the other information available to theauditors.

In the final analysis, auditors must have suffi-cient direct involvement with, and knowledge of,material aspects of the matters under examinationto enable them to form their own opinions andrender their own reports, for which they bear soleresponsibility. This fact, which means that relianceon others in no way transfers responsibility tothem from the auditors, helps explain the impor-tance attached to reliance decisions.

MAKING DECISIONS ABOUT RELIANCE REQUIRES THE

AUDITORS TO HAVE ACQUIRED AT LEAST SUFFICIENT

DIRECT KNOWLEDGE OF THE SUBJECT MATTER TO

ASSESS THE VALIDITY AND SIGNIFICANCE OF THE

OTHER’S FINDINGS. ACCORDINGLY, RELYING ON THE

WORK OF OTHERS CAN NEVER ELIMINATE THE NEED

FOR AT LEAST SOME WORK BY THE AUDITORS.

The subject of reliance is dealt with moreextensively in chapter 22.


M E T H O D S O F G A T H E R I N G

A U D I T E V I D E N C E

There are a number of techniques auditorsuse to gather the evidence they need. The follow-ing are derived from one authoritative manual:274

Review of documents: Written material—such as cor-respondence, memoranda, minutes, reports, systemsdocumentation, directions to staff and internalaudit reports—is a major source of audit evidence.

Analysis: The analytical process often compares fig-ures, trends, ratios, processes, procedures, and soon. Judgment and competence are vital here, andsuch evidence is best derived by people knowledge-able in the area.

Investigation: Auditors normally investigate the caus-es and effects of significant audit findings. Wherethey find unsatisfactory results, they investigate thesystems, procedures, management practices, organi-zational environment, etc., to determine the causes.

Interviews/inquiries: Significant informationgleaned from interviews should be documented.Careful preparation (including the development ofchecklists where appropriate) maximizes the pro-ductivity of interviews.

Physical observation/inspection/taking photos: These tech-niques have long been used to obtain evidence aboutphysical assets and can be helpful in gathering infor-mation about people, things or events within the auditscope. It should be remembered that a single observa-tion or photo documents affairs at a specific momentand cannot be used to draw conclusions about mattersthat have occurred over a period of time.

Systems review or update: This involves understandinghow the organization carries out particular functions,including the flow of transactions, the methods usedto assure quality and generate data and the controlsthat apply. It can allow auditors to determinewhether a control existed, but will not provide evi-

dence of how control procedures may have operated.

Confirmation: Confirmation is a method of corrob-orating evidence with independent third parties. Itis often used to verify that an asset or liabilityexists, but may also be used to verify that proce-dures applying to the clients of the auditee actuallyoperate as claimed or as intended.

Detailed testing: In general, testing involves applyinga given audit procedure to a sample of items within apopulation. Auditors then project the results to theentire population. Evidence from this technique givesauditors some measure of the assurance needed toconclude whether or not specified audit criteria arebeing met. The more rigorous the design of the test,the greater the assurance it will provide. Methods oftesting vary, depending on the particular operation orarea of activity being tested and the type of evidenceavailable. Statistical methods of selecting the sampleand projecting the results add precision to a test.

Surveys: also could be added to this list.

A U D I T P R O C E S S A N D E V I D E N C E

How does this understanding of evidencerelate to the comprehensive audit process?

F I G U R E 3 . 6 : A U D I T P R O C E S S


CRITERIONEVIDENCE CONDITION EVIDENCE

COMPARISON

FINDING EVIDENCE

The above diagram shows familiar informa-tion—the audit process, including the statement ofcriteria, and its comparison with the actual condi-tion, to arrive at a finding. At each of these criticalstages, auditors require evidence or proof of state-ments they intend to make in the audit report.

Evidence and audit criteria are closely related.Remember the following points:

• Criteria are crucial to developing an auditobservation. They must be objective anddemonstrably practical and applicable.

• With regard to conditions found, auditorshave to ensure that their statements anddescriptions are accurate and well supported.They need appropriate evidence to supportsuch statements.

• To arrive at a fair comparison of condition tocriteria, an understanding of the organizationand its environment is required. Evidencemust be viewed in context.

• Satisfactory performance does not mean per-fect performance. Often, standards for perfor-mance are stated in a way that recognizes thatthe world is not perfect.

• The criteria auditors use and the comparisonsthey make with the evidence they have foundshould provide structure to and guide theirthinking and analysis—they should notreplace them.

In summary, evidence is often persuasiverather than conclusive. The risk, significance, sensi-tivity of the matter to be reported, and the cost ofobtaining evidence should determine the natureand amount of evidence to be collected. In contro-versial areas, the persuasiveness of the evidenceshould be high. The reliability of the source andtype of evidence influence the degree of its persua-siveness and, therefore, the amount of evidencethat will be needed. When based on sufficient,appropriate data, it should be possible for the prac-titioner to advance opinions with the confidencethat they are valid and will withstand critical exam-ination.


271 CICA STANDARDS, §.31.272 OAG MANUAL, §2501.01.273 GAO STANDARDS,§6.46.274 THIS SECTION IS DERIVED FROM THE OAG MANUAL, §2501.28-.40.

S E C T I O N 4

OTHER KEYCONSIDERATIONS


C H A P T E R 2 1

INTERNAL AUDIT

The concept of comprehensive auditing wasfirst developed in the context of supporting theaccountability relationship between managementand governing bodies and the role that externalauditors play in this regard. Comprehensive audit-ing, however, is by no means the exclusive preserveof external auditors. Value-for-money considera-tions can also be an important aspect of internalaudit practice. Internal auditors, and other profes-sionals with internal review mandates, can make animportant contribution to their organizationsthrough comprehensive auditing.

The Institute of Internal Auditors’ (IIA)Standards for the Professional Practice of InternalAuditing,275 for example, outline the scope of inter-nal auditing work as including (among other mat-ters): appraisal of the economical and efficient useof resources, and review of the accomplishments ofestablished objectives and goals for operations orprograms. Indeed, today a significant proportion ofcomprehensive auditing is carried out by internalauditors, serving the accountability relationshipbetween operating managers and their organiza-tional superiors or, in those fewer cases where theinternal auditor reports directly to the governingbody, serving the latter’s accountability interests.

It is not the intent of this chapter to try todescribe the practice of internal audit in all itsdimensions and details. There are many excellentpublications that do this and a good starting pointfor the serious student of the subject would be thepublications of the Institute of Internal Auditors.What is intended in this and the following chapteris to describe and examine the linkages betweencomprehensive audit and internal audit practice.

W H A T I S I N T E R N A L

A U D I T I N G ?

Introduced several decades ago, a constant,prominent feature of internal audit work has beenan assessment of the adequacy of an organization’ssystem of internal control. Originally, internalaudit was used in connection with the organiza-tion’s financial transactions and was essentiallydevoted to determining whether established pro-cedures were being followed. In both the privateand public sectors, internal auditing has evolvedover the years. Today, most internal auditors haveterms of reference that include nonfinancial sys-tems and practices, and it is now common forinternal auditors to make recommendations forchanges that will produce cost savings and otherimprovements.

The 1990 IIA Statement of Responsibilities ofInternal Auditing describes internal auditing as follows:

Internal auditing is an independentappraisal function established within an orga-nization to examine and evaluate its activitiesas a service to the organization. The objectiveof internal auditing is to assist members ofthe organization in the effective discharge oftheir responsibilities. To this end, internalauditing furnishes them with analyses,appraisals, recommendations, counsel, andinformation concerning the activitiesreviewed … The members of the organizationassisted by internal auditing include those inmanagement and the board of directors.

The scope of internal auditing shouldencompass the examination and evaluation ofthe adequacy and effectiveness of the organi-zation’s system of internal control and thequality of performance in carrying outassigned responsibilities.


This definition is sufficiently broad toencompass a variety of arrangements that mayresult in a range of benefits for the organization.Some examples of the benefits of internal audit ser-vices are set out below. These “products” wereidentified in research276 as being delivered by inter-nal audit in different circumstances. They are notmutually exclusive; most internal audit groups pro-vide more than one such product. Few groupswould claim to produce all of them; internal audi-tors cannot be all things to all people—a focus hasto be developed in each particular situation. And itis better for internal auditors if that focus is in linewith the expectations of the person or persons towhom they report. In one situation or another,internal audit delivers:

• assurance of compliance with established poli-cies and procedures for day-to-day operations(as appropriate, not all are always covered):

- financial- personnel- materiel/purchasing- production or service delivery- marketing and sales

• identification of savings in operating costs inany or all of the areas noted above

• investigation of known problems and devel-opment of solutions in any or all of the areasnoted above

• overall assessments of the reasonableness ofthe balance between risk and control in all orpart of the systems noted above

• reduction of external attest audit fees by hav-ing internal audit staff work under the direc-tion of the external auditor or by undertakingspecific tasks—ranging from the preparationof detailed work papers and analysis, to theaudit of branches and subsidiaries—on behalfor under the supervision of the attest auditor

• provision of information or special studies formanagement during periods of major corpo-

rate changes such as those resulting fromundertaking new programs, work-force reduc-tions or deregulation

• development of operating managers throughthe use of short-term secondments to internalaudit to give them a broader perspective

• contribution to a positive environment, or aheightened awareness of control of perfor-mance issues within which cost-effective sys-tems and practices can develop more easily

• assurance of the completeness, appropriate-ness and reliability of management informa-tion

• confirmation that joint-venture operationshave been fully and appropriately accountedfor and that governing agreements have beencomplied with

• validation of valuations used and assetsacquired in takeovers

• the eyes and ears of senior managementregarding suspected problems, bringing afresh, nondepartmental perspective to bear onthe interaction among, and integration of,organizational units

C L I E N T F O R I N T E R N A L

A U D I T I N G

Internal auditing is a key management prac-tice, and usually the function reports to seniormanagement and has management as its primary,if not sole, client.

This is not always the case, however. In someinstances, the internal audit function reportsdirectly to the governing body. In some municipal-ities, for example, the internal auditor reportsdirectly to the city council. In still other instances,such as federal Crown corporations, the internalaudit function is overseen by the audit committeeof the governing body. In these circumstances,active and supportive oversight by members of the

P A R T I I I . C H A P T E R 2 1 . I N T E R N A L A U D I T 2 9 9

governing body can foster internal auditing andbolster its relative independence.

Organizationally, the internal audit functionis usually set apart so that internal auditors can beindependent of and objective about the systems,practices and operations they audit.

In that in the significant majority of circum-stances internal auditors report to senior manage-ment, the remainder of this chapter and the fol-lowing chapter are predicated on this relationship.Take note that, in cases where the internal auditorreports directly to the governing body, the interpre-tation placed on certain subsequent points wouldvary, or be expanded, accordingly.

Three key factors contribute to effective inter-nal auditing and strongly determine the benefitsthat organizations can expect to realize. Seniormanagement support is, by far, the most importantfactor. Senior management sets the expectationsand controls both resources and access. The secondmost important element is the quality of the peo-ple who lead and staff the function. The third keyfactor is the organizational arrangements made forinternal auditing and the structures within whichinternal auditors are expected to operate. Structuresaffect the way that people perceive and performtheir roles and the way these are seen by others.They can facilitate or hamper productivity andaffect the rate at which change takes place.

S E N I O R M A N A G E M E N T

S U P P O R T : T H E


E N V I R O N M E N T

The success of internal auditors depends firstand foremost on the actual and visible support ofsenior officers. Real and recognized support at thehighest levels of the organization gives internalauditors the independence they need to operatesuccessfully far more than formal reporting levels,

titles or mandates. From such support and theinvolvement of senior management, the followingconsequences can be expected:

• appropriate resources will be provided;• respect for the function will be forthcoming;• internal auditors will be better able to acquire

the senior management perspective needed toimplement the kind of internal auditingexpected;

• staff of an appropriate calibre will welcomeassignments to perform internal audits; and

• the attitude to the function of less seniormanagers—whose cooperation in the perfor-mance of audits and attention to the findingsare essential to the effectiveness of internalaudit—will be shaped.

O R G A N I Z AT I O N A L S TAT U S

The credibility of internal audit findingslargely depends on the extent to which the auditorsare perceived to be objective about, and indepen-dent of, the subjects they audit. This means thatthe internal audit function should be separate fromthe operational or support units whose systems,practices and operations it will subject to audit. Inaddition, individual auditors are more crediblewhen they audit areas for which they are not andhave not been responsible. Personal involvementdiminishes the aura of independence and objectivi-ty so necessary for effective results.

To some extent, the status and authority ofthe persons to whom the internal audit functionreports will also reflect on the credibility of thefunction. It is less likely, for example, that a man-ager will take an internal audit of his or her area ofresponsibility seriously if it is done by a unit thatreports to some relatively low-level functionarythan if the function reports to an officer who issenior to the manager in question.


Formal organizational placement is only oneway that senior management can demonstrate itssupport for internal auditing. Visible support canbe given by a demonstrated willingness to haveaudits conducted in the areas for which senior man-agers are directly responsible. It can also be provid-ed by establishing an environment where internalauditors have frequent informal contacts withsenior officers and the audit committee if there isone. When they are known to have access to thetop levels in the organization and to be sufficientlysecure to report honestly without fear of adverselyaffecting their careers, internal auditors will enjoy astatus within the organization that will facilitatetheir work substantially. Along with that status willcome a respect throughout the organization for theindependence and objectivity with which internalauditors approach their work and a growing accep-tance of the auditing function and its reports.

S TA N D A R D S

Another way in which formal support for theinternal audit function can be demonstrated isthrough respect for the implications of standards.Like most disciplines, internal auditing is governedby standards that establish expectations of goodpractice. These standards embody and codify thefeatures that characterize, indeed create, an auditfunction as distinct from other forms of review.Without standards, there really isn’t an audit; norcan there be reliance.

Looking at internal audit practices as a whole,the Institute of Internal Auditors sets out standardsfor internal auditing that deal with such matters asscope of work, independence and objectivity, per-formance of work, and leadership, staffing andmanagement of the function. In specific organiza-tional settings, further standards may be promul-gated for internal audit work in that jurisdiction(an example of this can be found in the federal

government, where IIA standards have been adapt-ed and set forward by the Treasury BoardSecretariat). Translation of such standards into spe-cific, operational terms can provide a frameworkfor clearly understood expectations that will helpensure that the work of internal auditors receivesthe support of senior management and meritsbeing relied upon by external auditors.

L E A D E R S H I P A N D S T A F F I N G

The specific skills and knowledge required ofinternal auditors will depend on the nature of thebusinesses and activities of the organization underaudit. Team building is a key to success. Sources oftalent include both full-time internal audit staffand corporate staff that can be seconded to internalaudit, either for brief projects or for longer termsoften connected with career development. Theseresources can be further augmented by contractingout for professional services. The support of seniormanagement is crucial here, since the functionneeds an adequate budget and executive clout toattract appropriate staff.

L E A D E R S H I P

Good leadership is essential if internal audit-ing is to implement comprehensive auditing suc-cessfully. Experience with comprehensive auditingindicates that particularly heavy demands will beplaced on those who lead the process. Leaders maynot personally possess all the skills and knowledgerequired by their teams, but they must be able toidentify where and when specific skills are needed,acquire people with those skills and mould theminto a team that can credibly complete the audit.

Leaders must display a number of personalqualities:

• insight to identify major issues;• ability to recognize the need for expertise and

the ability to use it;


• interpersonal and organizational sensitivity;• ability to command respect and gain coopera-

tion; and• a flexible, constructive and innovative

approach.

S K I L L S

The credibility of internal audits depends onthe credibility of those performing the work. It isessential, therefore, that the auditing and otherskills available be appropriate to the tasks they arecalled upon to perform. Accordingly, this is one ofthe issues that external auditors will assess carefullyas a predictor of the extent of reliance that can beplaced on internal audits.

There will be instances where different typesand levels of technical expertise will be important,both to deal with the subject matter of the auditand to establish credibility with those being audit-ed. Internal audit teams will need to possess suffi-cient technical expertise to understand industrypractices for the area under review. Without it, it isunlikely that they will complete their projects suc-cessfully or have their reports accepted.

Apart from this sort of technical expertise,there are other areas of knowledge that should bereflected in the team. These requirements arisefrom the peculiarities of this type of auditing andare described in chapter 24. Areas in which at leasta minimum level of competence must be foundamong audit team members are:

• comprehensive auditing concepts and itscomponents and characteristics;

• governance structures;• organizational effectiveness;• management functions and systems;• appraisal processes and controls; and• audit process.

Larger organizations are more likely to haveinternal audit groups with members from a widerange of professional backgrounds, although somesmaller organizations have made significant stridesin this respect. Assembling teams with all the skillsthat may be required, however, may be both diffi-cult and expensive. Seconding personnel to inter-nal auditing, especially on a short-term basis, offersone avenue for attracting special skills to the groupwithout incurring major, ongoing, overhead costs.


A R R A N G E M E N T S F O R

I N T E R N A L A U D I T I N G

Not surprisingly, there is no single pattern usedto organize the internal audit function. Each entitymakes arrangements for its own convenience, reflect-ing its unique characteristics and needs. Four distinctorganizational arrangements for internal audit aredescribed below. The model chosen depends on howfar the organization wants to go with internal audit,how fast, and where it is starting from.

Because external auditors often rely on exami-nations made by internal auditors (as discussed inthe next chapter), the implications for suchreliance is explored for the various organizationalarrangements.

F I N A N C I A L / C O M P L I A N C E PAT T E R N

The traditional pattern, commonly adoptedwhen internal auditing was introduced, and stillfound in some organizations, is devoted to assess-ing compliance with established financial rules andregulations. In this pattern, internal audits addressfinancial systems and practices with a strong orien-tation towards determining whether prescribedprocedures are adhered to in practice. The internalauditors are, effectively, police in respect of suchmatters as limits of delegated authority and record-keeping procedures.


This pattern is now rarely found in a pureform. Several factors encourage a broadening of thescope of internal audit. One arises from the needto secure the cooperation of managers whose oper-ations are subject to audit. This leads internal audi-tors to reflect on the reasons for noncomplianceand to suggest improvements to prescribed policiesand procedures. Another factor is the need todemonstrate the value of internal auditing to seniormanagers. This also provides an incentive for inter-nal auditors to recommend improvements in exist-ing systems and practices.

Taken together, these factors extend the focusof internal audit beyond compliance with proce-dures to the design of systems and practices.Initially, suggestions tend to result from inquiriesinto underlying causes for noncompliance. Only intime do they result from systematic examinationsof the design of systems and practices. Similarly,there may be incidental observations in respect ofnonfinancial operational systems and practices thatcan, over time, result in audits that deliberatelyseek to find improvements in these areas.

So long as these tendencies are neither recog-nized formally in the mandate of internal audit norreflected in its staffing and organizational arrange-ments, internal audit can be regarded as fitting inthe financial/compliance pattern.

Key characteristics

Leadership and staffing: In this pattern, groups aretypically small and staffed by career internal audi-tors with a financial audit background who haverelatively junior status among managers and profes-sionals within the organization. The head of theunit commonly reports to the chief financial officeror controller.

Products: The scope of internal audits under thispattern is typically restricted to accounting systemsand practices and, particularly, to those aspects that

relate to record keeping and safeguarding of assets.The objective is to assess whether prescribed poli-cies and procedures have been followed.

Reports of these units deal with such subjects:• accounting controls over the initiation,

recording and collection of revenues;• physical inventory and related accounting

records;• the accuracy, validity and authority for pay-

roll charges and the related accountingrecords; and

• the authority, validity and accuracy of record-ed expenditures.

Reports result in observations and recommenda-tions such as the following:

• Delegations of authority have not beenrevised since reorganization and are now out-dated. I recommend that the delegation ofauthority be revised.

• Some units which have dealt with the sameclients for several years without collectionproblems implement credit policy informally.I recommend that (the specified unit audited)adhere to approved credit terms and limits.

• The purchase journal at (name of unit) isredundant as invoices are recorded in the jour-nal only upon payment. The number ofinvoices is limited and the unit has efficientcontrol over the small number of invoicesthrough an outstanding invoice file. I recom-mend that the purchase journal be eliminated.

Costs/Benefits: This traditional pattern of internalaudit is relatively inexpensive, both in terms of thedirect costs and the amount of senior managementsupport and involvement required to maintain itand deal with the audits and resulting reports. Itpromotes adherence to laid-down policies and pro-cedures, thereby supporting a centralized financialfunction that provides precise direction in the formof detailed accounting procedures.


Reliance implicationsMuch of the original impetus for this pattern

of internal audit derived from a desire to demon-strate the reliability of an important but limitedsubset of management systems and practices: theinternal accounting controls in which externalfinancial auditors are also interested. Other systemsand practices are seldom, if ever, addressed.Accordingly, internal auditing in thisfinancial/compliance pattern is likely to be morerelevant to the external financial audit than tocomprehensive audits.

E X PA N D E D S C O P E PAT T E R N

A common internal audit pattern is one inwhich the scope of internal audits has beenexpanded to include not only financial controls,but also day-to-day operational and delivery sys-tems and practices. In addition, the focus ofthese internal audits has been widened to consid-er not only compliance with prescribed systemsand practices, but also the possibility of improve-ments in their design. This is the most commonway in which internal auditing has begun toaddress issues bearing on economy, efficiency andeffectiveness.

In many cases, this evolution has taken placeover many years, reflecting repeated successes inattempting to be helpful by producing suggestionsfor savings. The result is a growing recognition ofthe value of internal audits and an increase inrequests for this kind of examination. Typically,this reflects the long-term impact of a constructiveapproach on the part of the leader of internalaudit, rather than a major initiative of senior man-agement. Eventually, this broadening of the role ofinternal audit may be formally reflected in the mis-sion of the internal audit group.

Key characteristics

Leadership and staffing: In this pattern, the internalaudit group has the advantage of building on anestablished auditing discipline. However, new skillswill probably be required to address new issues anddifferent levels of systems.

In general, such groups are dominated bycareer internal auditors with a financial auditingbackground. Depending on circumstances, some ofthese groups include individuals with skills in suchareas as EDP or engineering. Secondments andcontracting for services are sometimes used toensure that the required skills are available for spe-cific audits. Reporting relationships of groups thathave expanded their scope tend to vary, somereporting to the chief financial officer, others to thechief executive.

Costs: The costs of such groups, relative to theorganization’s total work force, assets or expendi-tures, vary considerably. A survey was made in1986277 of internal audit units in federal Crowncorporations. As a proportion of corporate expen-ditures, the budgets of the internal audit groups ofthis pattern that participated in the research variedfrom 0.4 cents per $1,000 to 3.6 cents per $1,000,with an average of 0.7 cents per $1,000. The varia-tions may reflect the needs of the different indus-tries in which the corporations operate. For exam-ple, internal audit budgets of financial corporationstend to use a higher percentage of expendituresthan those of nonfinancial corporations. Size alsohas an impact. The percentage costs tend to belower in larger corporations. There are, however,significant exceptions to these generalizations.

Products: Typically, the formal scope of audit ofgroups following this pattern has been expanded toinclude all systems and practices of the organiza-tion. In practice, the extent to which the mandatedscope is actually covered varies, largely depending


on the stage of evolution of the group and theskills available. Initially, the focus may continue tobe placed on the financial aspects of the systemsand practices audited, with observations on effi-ciency and effectiveness arising in an ad hoc wayrather than as the primary focus of the audit. Asdevelopment proceeds and demonstrable results areobtained, both the internal auditors and managersfeel more comfortable with internal auditorsaddressing systems and practices that are moreoperational in nature. When the internal auditorhas established a degree of credibility, internalaudits may be used to examine and review knownproblem areas and to develop solutions.

In addition to those encountered in the tradi-tional pattern, examples of the objectives of inter-nal audits that have been performed by groupsorganized in the expanded scope pattern include:

• to review distribution system development todate, with emphasis on computerized securityand control features, and to review imple-mentation plans

• to review the acquisition, utilization andmanagement of personal computers, termi-nals and peripheral equipment, including areview of the adequacy of policies and proce-dures in the personal computer area at allmajor centres

• to examine the overall purchasing function andits delegation to ensure that the most effectiveand efficient degree of control and service isbeing obtained (Included in the audit was areview of computer support, policies and pro-cedures and other activities such as bid proce-dures, quality control, and vendor analysis.)

• to evaluate whether or not a specified capitalproject was carried out with due regard for econ-omy, efficiency and effectiveness, and whether ornot measures had been taken that might benefitthe management of future projects

• to evaluate whether the training, managementand organizational development group canand does fulfill its stated role and missioneconomically, efficiently and effectively

• to review the marketing function, includingmarket research, product development, pric-ing and advertising

The recommendations produced by internalauditors in the expanded scope pattern are fre-quently aimed at achieving direct quantifiable sav-ings. Typical examples include:

• recommendations for improvements in cashutilization by deferring mailing of accounts-payable cheques

• recommendations for minor changes in mate-rial handling equipment leading to significantreductions in spoilage

• recommendations that specific systems underdevelopment be halted pending confirmationof, for example, user needs, feasibility,cost/benefit analysis or volumetrics

• recommendation that a particular group bedisbanded as its role is obsolete or redundant

Benefits: A major benefit of this pattern is its grad-ual approach. Internal audit develops from anauditing discipline applied to financial systems andpractices. Expansion into nonfinancial systems andpractices builds on this base as existing internalaudit staff gain credibility in other areas, or asother skills are added to the auditing discipline. Asthe internal audit function develops, it engendersits own support through demonstrating its abilityto assist managers by identifying cost savings andsolving problems.

Expansion can occur, although at a rela-tively slow rate, with only a moderate investmentof senior management time, effort and prestige.Accelerated expansion requires that senior man-


agers make significant efforts to create initial sup-port for the expansion, to assist in securing the ser-vices of appropriate staff, often against competingdemands in the organization, and to encourageother senior managers to give their time and atten-tion to internal audit findings.

The expanded scope pattern tends to be min-imally disruptive to existing internal audit staff. Italso tends to be satisfying to them, since all staffare involved in the challenge presented by theexpansion of scope.

Finally, because there is at least the potential tointegrate all dimensions of audit work in all projectscarried out by internal auditors, there is a possibilitythat the operational impact on managers who aresubjected to internal audit will be minimized.

Reliance implicationsThe reliance implications of this pattern will

depend largely on how far the evolution fromfinancial/compliance auditing has progressed andhow far the capabilities of staff have advanced insupport of expanded scope. External comprehen-sive auditors will be cautious in accepting theassessments of internal auditors with respect to sys-tems and practices governing day-to-day operationswhen there has been a significant expansion ofinternal audit scope but little commensuratechange in internal audit personnel.

External auditors will require extensive cor-roboration of the reliability and completeness ofinternal audit assessments in areas where internalauditors have little previous familiarity and/ordemonstrated competence. Over time, and as bothmanagement and the internal audit group developconfidence in internal audit’s ability to assess sys-tems and practices of various kinds, the potentialfor reliance can be expected to increase.

Before relying, however, the auditors will haveto be confident of the independence and objectivi-ty of internal auditors in those areas in which

reliance is contemplated. With respect to systemsand practices governing day-to-day operations,reliance will depend on the status of the internalaudit group and its leader, and on their credibilityin the eyes of senior management. If the internalauditors have the respect of senior managementand their assessments are known to carry weight atthe executive level, it is probable that they canattain the required level of independence to per-form reliable audits of systems and practices gov-erning day-to-day operations.

As this credibility is established and internalauditors become more deeply involved in operationalmatters, they will likely begin to touch on issues thatrelate to corporate organizational practices.

A possible consequence of an expansion inthe scope of internal audit, especially where thischange is achieved without additional resources, isthat the degree of attention paid to financial andaccounting controls will diminish. The organiza-tion may inadvertently and unknowingly weakenits systems and practices in these areas, and theexternal auditor may find that additional workbecomes necessary in support of the audit of finan-cial statements.

S E PA R AT E G R O U P F O R P E R F O R M A N C E I S S U E S

This pattern has been adopted by some orga-nizations when starting to undertake internalaudits of economy, efficiency and effectiveness. Inthis pattern, a new group is formed to give initialimpetus to this type of auditing, while the existinginternal audit group continues to perform suchinternal audits as they have traditionally beendoing—usually an expanded scope form. In thelonger term, and after economy, efficiency andeffectiveness auditing is suitably established, thetwo groups may be combined.

The main thrust of this approach is to pro-vide faster results through concentration of effort.


The separate group provides a focus for auditingperformance issues. It emphasizes that internalauditing is departing radically from previous prac-tice, and it unmistakably signals senior manage-ment’s support for the initiative.

Key characteristics

Costs: The initial costs of arrangements in this pat-tern can be considerable. The personnel involvedmay be more senior that those found in traditionalinternal audit groups. It requires a substantial invest-ment of senior management time to create supportfor the function, to assist in identifying and assign-ing appropriate staff (usually from within the orga-nization) and to ensure that senior managers givetheir time and attention to internal audit plans andfindings. Moreover, the internal audits performedcost more than the average for other groups, a reflec-tion of the complexity of the issues dealt with in thehigher-level systems they tend to examine.

Sponsorship: This pattern involves a sponsor at thehighest level in the corporation who convincessenior management that the benefits are worth thecosts and persuades a suitable candidate that anassignment to lead the group will be a positivecareer step.

Leadership and staffing: This pattern is also charac-terized by extensive use of term assignments. Theleader may not be a career internal auditor.Similarly, the internal audit staff in these groupsmay be drawn from technical or operating divi-sions for limited-term assignments to bring to thegroup the required familiarity with operations.Auditing skills may be brought in from the tradi-tional internal audit group or by recruiting or con-tracting from the outside.

Products: It may be expected that such groupswould focus on effectiveness and efficiency issuesat a higher level than would groups under other

patterns in the initial stages. Reports might addresssuch topics as the performance of significant opera-tion units, and the sufficiency and appropriatenessof information provided to senior management,and/or to the board, as a basis for ongoing man-agement of the corporation.

Benefits:• concentration of scarce skills on issues of

economy, efficiency and effectiveness, ratherthan on audits that also encompass financialand compliance aspects;

• the benefits of existing internal audit aremaintained.

Reliance implicationsThese groups would be formed and staffed to

address issues that should be directly relevant tocomprehensive auditors. Considerable reliancecould be placed on their work, provided it hasbeen performed objectively and in accordance withappropriate standards.

Such groups will normally be independent ofsystems and practices governing day-to-day opera-tions, as long as staff are not assigned to auditactivities for which they have previously beenresponsible, or in which they expect to becomeinvolved later.

The issue of independence becomes more dif-ficult when audits are done in the area of corporateorganizational practices and information providedto the board. Some of the issues encountered atthis level may be so deeply ingrained in the corpo-rate culture and ethic that it is difficult for a per-son employed by the company to be—and appearto be—objective about them. Accordingly, externalauditors may be concerned about internal auditindependence and may require rather more corrob-oration of such opinions. Organizations that havefollowed the separate group pattern have taken anumber of steps to bolster independence when


undertaking audits in areas in which their indepen-dence or objectivity might be open to question.These include the use of expert advisory groupswith internal and/or external representation andthe use of external resources on the team.

CO N T R AC T E D E X E C U T I O N PAT T E R N

The internal audit patterns discussed aboveare found in larger organizations. Effective compre-hensive auditing depends on the synergy generatedby a multidisciplinary team. Such synergy, howev-er, cannot be generated in a one-person operation.In smaller organizations, the development of sucha team may represent a greater investment thanmanagement is willing to or should make. As aproportion of the organization’s effort andresources, the costs of a self-sustaining, broad-scopeinternal audit group may be unacceptably high.And yet there may be a recognized need for somecomprehensive auditing within the organization.

Two Crown corporations that participated inthe survey mentioned above had a solution for thisproblem. Although they had internal audit groups,they indicated that it wasn’t possible or desirable tomaintain a permanent internal audit staff capableof addressing the full range of issues raised by themanagement systems and practices of the corpora-tions. One-half to one-third of their internal auditbudgets are used to contract for external assistancewith broader-scope audits. While some internalcapacity for auditing is retained, the primary roleof the internal audit leader is to make sure that theright things get audited and that the resultingopinions are appropriate.

To this end, the internal audit leader, in con-junction with the audit committee and seniormanagement, selects and puts priorities on pro-jects, and is involved at the planning stage of eachproject. This includes establishing project objec-tives and approach, selecting contractors and

ensuring that these outsiders understand the pro-ject, the nature of the operations and the relevantcorporate background. During the conduct phase,the internal auditor serves as a resource to the auditteam, assists in the development of opinions andensures that they are fully supported by the evi-dence, that they reflect the corporation’s environ-ment and that they are practical.

Reliance implicationsIn such an environment, external comprehen-

sive auditors would apply the same tests concern-ing reliance as they would to internal audits con-ducted by corporate employees. There appears tobe no reason to suppose that properly conductedaudits performed under this pattern should be anyless reliable than those performed under other pat-terns. It would be necessary, of course, for theorganization to require that contractors provideexternal auditors with access to their personnel andfiles regarding the internal audits performed.

A U D I T C O M M I T T E E OV E R S I G H T

Not all organizations have audit committeescomprising members of the governing body. Wherethey do exist, their oversight, when carried outwith sensitivity and care, can foster internal audit-ing, bolster its relative independence and improvethe prospects for reliance. It is a particularly visiblemanifestation of support. It also has the potentialto alienate the internal auditor from his or her pri-mary client—management.

The audit committee’s interpretation of itsresponsibility to oversee internal audits varies fromorganization to organization. The level of involve-ment will depend on:

• the availability of members of the committeeand their level of interest in internal audit;

• the level of involvement of the governingbody in the day-to-day management of the



NEW GROUND BROKEN IN INTERNAL AUDIT IN FEDERAL CROWN CORPORATIONS

THE FINANCIAL ADMINISTRATION ACT BREAKS NEW GROUND IN REQUIRING CROWN CORPORATIONS TO CONDUCT INTERNAL AUDITS.

THIS IS THE FIRST REFERENCE TO THE INTERNAL AUDIT FUNCTION IN CANADIAN CORPORATE LEGISLATION AND IS PHRASED IN WHAT

IS NOW SECTION 131 AS FOLLOWS:

(3) EACH PARENT CROWN CORPORATION SHALL CAUSE INTERNAL AUDITS TO BE CONDUCTED, IN RESPECT OF ITSELF AND EACH

OF ITS WHOLLY-OWNED SUBSIDIARIES, IF ANY, TO ASSESS COMPLIANCE WITH SUBSECTIONS (1) AND (2), UNLESS THE GOVERNOR

IN COUNCIL IS OF THE OPINION THAT THE BENEFITS TO BE DERIVED FROM THOSE AUDITS DO NOT JUSTIFY THEIR COST.

THE SUBSECTIONS (1) AND (2) REFERRED TO ARE THOSE THAT REQUIRE CORPORATIONS TO KEEP BOOKS OF ACCOUNT AND TO MAIN-

TAIN MANAGEMENT SYSTEMS AND PRACTICES IN A MANNER THAT PROVIDES REASONABLE ASSURANCE THAT ASSETS ARE SAFEGUARDED,

THAT THE REQUIREMENTS OF THE FINANCIAL ADMINISTRATION ACT AND OTHER PERTINENT AUTHORITIES ARE COMPLIED WITH,

THAT RESOURCES ARE MANAGED IN AN ECONOMICAL AND EFFICIENT WAY AND THAT OPERATIONS ARE EFFECTIVE.

THIS PROVIDES A BROAD TERM OF REFERENCE FOR THE CORPORATIONS’ INTERNAL AUDITORS. NO RESTRICTIONS ARE IMPOSED ON

THEM BY THE ACT. HERE IS CLEAR EVIDENCE THAT PARLIAMENT WANTS INTERNAL AUDITS THAT GO WELL BEYOND MORE TRADITION-

AL ASSESSMENTS OF COMPLIANCE WITH FINANCIAL AND ACCOUNTING PROCEDURES.

THE ACT USES THE WORDS “REASONABLE ASSURANCE” AND, IN SO DOING, IMPLICITLY RECOGNIZES THAT THE COSTS OF SYSTEMS AND

PRACTICES SHOULD NOT EXCEED THE BENEFITS LIKELY TO BE DERIVED. AS INTERNAL AUDITS MUST ASSESS COMPLIANCE WITH THE

ACT, THEY MUST ALSO BE CONCERNED WITH THE CONCEPT OF REASONABLE ASSURANCE AND MUST DEAL WITH THE TRADE-OFFS

BETWEEN THE COSTS OF CONTROLS AND THEIR BENEFITS. IT FOLLOWS THAT IT WILL NOT BE SUFFICIENT FOR INTERNAL AUDITS TO

ASSESS WHETHER THE PROCEDURES LAID DOWN BY MANAGEMENT HAVE BEEN FOLLOWED—THEY MUST ALSO ASSESS WHETHER THE

SYSTEMS AND PRACTICES ARE WELL DESIGNED IN THE FIRST PLACE.

THE ACT SPECIFICALLY REQUIRES AUDITORS TO RELY ON INTERNAL AUDITS TO THE EXTENT THEY CONSIDER PRACTICABLE AND TO

REPORT THE EXTENT OF THIS RELIANCE. THIS, TOO, IS A NOVEL STATUTORY PROVISION.

ALTHOUGH THE STATUTORY REQUIREMENT TO CONDUCT INTERNAL AUDITS IS NEW, INTERNAL AUDITING IS A MANAGEMENT PRACTICE

COMMON TO MANY CROWN CORPORATIONS, AS IT IS TO OTHER ORGANIZATIONS. THE ISSUE FACING CROWN CORPORATIONS AS A

RESULT OF THE INTERNAL AUDIT AMENDMENTS, THEREFORE, IS TYPICALLY NOT WHETHER TO IMPLEMENT AN INTERNAL AUDIT FUNC-

TION, BUT RATHER HOW TO ENSURE THAT THEY CAPTURE THE BENEFITS (INCLUDING OPTIMAL RELIANCE) OF BROAD-SCOPE INTERNAL

AUDITING AS CALLED FOR BY THE LEGISLATION. THE ACT HAS INTRODUCED NEW CONSIDERATIONS THAT CORPORATIONS WILL WANT

TO TAKE INTO ACCOUNT IN DESIGNING, STAFFING AND ALLOCATING RESOURCES TO THEIR INTERNAL AUDIT FUNCTION. SPECIFICALLY:

• RELIANCE ON INTERNAL AUDITS CAN REDUCE THE COSTS (IN TERMS OF FEES AND DEMANDS ON STAFF TIME) OF SPECIAL

EXAMINATIONS; AND

• INTERNAL AUDITS BEARING ON MANAGEMENT SYSTEMS AND PRACTICES WILL HELP THE CORPORATION MAINTAIN THOSE SYS-

TEMS AND PRACTICES APPROPRIATELY, THEREBY REDUCING THE POSSIBILITY THAT A SIGNIFICANT DEFICIENCY MIGHT EXIST

AND BE UNCOVERED BY THE SPECIAL EXAMINATION.

BOTH OF THESE, OF COURSE, ARE IN ADDITION TO THE BENEFITS THAT DIRECTORS AND MANAGERS ARE CURRENTLY RECEIVING FROM

INTERNAL AUDITING.

THE ACT GIVES NO DETAILS ABOUT HOW INTERNAL AUDITS ARE TO BE DONE OR HOW THE INTERNAL AUDIT FUNCTION IS TO BE

ORGANIZED; IT DOES NOT EVEN DEFINE THE TERM. CROWN CORPORATIONS, THEREFORE, ARE LEFT TO DECIDE FOR THEMSELVES HOW

THEY WILL ARRANGE THEIR AFFAIRS TO MEET THE STATUTORY REQUIREMENT.

IN SECTION 150, THE ACT GIVES THE AUDIT COMMITTEE THE DUTY TO OVERSEE ANY INTERNAL AUDIT THAT IS CONDUCTED PUR-

SUANT TO THE ABOVE SUBSECTION (3).

organization; and• the importance of internal audit in the overall

control system of the organization.

Where there are statutory responsibilities tooversee internal audits of management systems andpractices (as is the case for federal Crown corpora-tions), audit committee members should beinvolved in the development of an appropriate andhealthy internal audit function for the organiza-tion. They should consider the internal audit mis-sion, charter, resourcing strategy and plans, theinteraction of internal audit with management andwith the external auditor.

Other elements of the committee’s role withrespect to internal audit may involve members in:

• reviewing long-term plans to ensure that theyare appropriately coordinated with the long-term plans of the annual auditor and thecomprehensive auditors;

• reviewing the annual internal audit plans forthe same reasons;

• reviewing the plans of specific projects that areof particular interest to the audit committee,either because they are of importance to thecomprehensive or annual audit, or becausesignificant negative results are anticipated, orsimply because the project is considered to beof particular importance to the board;

• reviewing the reports of these important pro-jects, including management’s responses tothe recommendations;

• receiving and reviewing annual reports fromthe internal auditors that set out the group’sachievement compared to plans, and a summa-ry of its major findings and recommendations.

All concerned should guard against unrealisticexpectations about the oversight role. The auditcommittee is charged with overseeing, not manag-ing, internal audits. Accordingly, they may not

wish to see or react to individual internal auditreports even if time allowed them to do so. Instead,their focus may be on satisfying themselves thatmanagement has instituted and is maintainingappropriate arrangements for internal audit, thatinternal auditing is providing reliable assessmentsof important management systems and practicesand that management gives due consideration tothese assessments.

It is not uncommon for the head of internalaudit to have a right of access to the chairman ofthe audit committee. Where this is the case, howoften and on what issues to exercise this right arematters that require the internal auditor’s keen pro-fessional judgment. On the one hand, it is impor-tant that the audit committee and internal auditorkeep each other informed on matters of mutualinterest; on the other, in this scenario, internalauditors need to maintain an effective balance intheir relationships with both senior managementand the governing body. Establishing and main-taining this balance is usually hard won and, alltoo often, is something that is easily lost.

I N T E R N A L A U D I T R E P O R T S

Historically, internal auditors have tended toproduce long-form reports that present their find-ings and focus on opportunities for cost savings orimproved procedures. Such a constructive approachhelps build support for the audit function, securesthe cooperation of the managers audited and leadsto adoption of improvements.

Benefits associated with recommendations aresometimes quantified and reported so that seniormanagement can assess the cost/benefit of theinternal audit function.

There may, however, be dangers associatedwith the quantifying of benefits. First, it may bedifficult to establish attribution with confidence. Inaddition, the focus on quantifiable savings may


bias auditors towards issues where quantifiableresults are likely to be achieved, at the expense ofsofter issues such as long-range planning that canhave a significant impact on costs. Furthermore,having to quantify the benefits and effectively takecredit for them may not enhance the relationshipbetween auditor and auditee.

In the past, internal auditors have seldomattempted to reach an overall opinion regardingthe adequacy of the systems and practices audited.Several of the groups participating in the afore-mentioned survey, however, were working towardsreporting in this manner in order to improve theirusefulness to senior managers. This trend can beexpected to continue as internal auditors respondto the requirement in the Financial AdministrationAct to assess whether the degree of assuranceafforded by systems and practices is reasonable. Inaddition, without such concise assessments, exam-iners may find it difficult to evaluate the signifi-cance of internal audit findings, and the potentialfor reliance may be diminished. A mere listing offindings, opportunities for improvement or depar-tures from criteria will not be particularly helpfulto an examiner.

Indispensable features of a good audit reportare that it presents its findings fairly in the contextof an overall opinion and provides sufficient infor-mation to give the reader an opportunity to appre-ciate the significance and basis of the auditor’sfindings. Guidelines have been developed, andwidely accepted, for the reporting of comprehen-sive audits in other milieus (discussed in chapter18), and for dealing with such issues as significanceand level of audit assurance. Consensus exists thatthose guidelines would be appropriate for internalaudit reports and that their adoption should proveuseful in facilitating reliance.


275 THE INSTITUTE OF INTERNAL AUDITORS, STANDARDS FOR THE PROFESSIONAL PRACTICE OF INTERNAL AUDITING (FLORIDA: THE INSTITUTE OF INTERNAL AUDITORS, 1978).276 REPORTED IN SPECIAL EXAMINATIONS, INTERNAL AUDITS AND RELIANCE: A PRACTITIONER’S GUIDE (OTTAWA: CCAF, 1987).277 IBID. THIS SURVEY WAS A KEY ELEMENT OF THE SUPPORTING RESEARCH FOR THE SPECIAL EXAMINATIONS, INTERNAL AUDITS AND RELIANCE PUBLICATION.

C H A P T E R 2 2

OTHER REVIEWPROCESSES—COORDINATION& RELIANCE

As discussed in the previous chapter, manyorganizations subject to comprehensive audits arethemselves trying to promote improved perfor-mance through internal audit. Frequently, not allthis kind of work done inside organizations iscalled internal audit. Program evaluations, forexample, are often engaged to assess whether theresults of operations are worth the investment oftime and resources; some organizations have unitsexclusively devoted to this practice. In addition,managers often initiate studies of elements of theiroperations that deal with performance issues. Thesemay be done in-house or by using outside consul-tants, or by a combination of the two.

While internal audit is used as the mainexample here, similar factors apply to all thesekinds of examinations when external comprehen-sive auditors consider relying on their results.

It is important for comprehensive auditors toknow about work of the kind described abovebecause reports of this work can provide them withimportant information that improves their under-standing of the organization. Those reports mayalso lead the external auditors to examine moreintensely, or effectively to ignore, an area that hasalready been studied. Thus, these studies can havean important impact on scope decisions for com-prehensive audits. Finally, if they have been done

with appropriate objectivity, independence andprofessionalism, external auditors may be able torely on the reports, thus economizing on auditeffort and cost.

C O O R D I N A T I O N A N D

C O O P E R A T I O N

A constructive, professional relationshipbetween the external auditors and the internalauditor and others doing work of interest to theformer is essential if the full benefit is to be derivedin the long term. This relationship is not an eventthat happens at one point in time; it involves morethan one evaluation and a once-and-for-all conclu-sion. Reliance is significantly influenced by theongoing relationship that should, through regularcontact over time, become a relationship of trustand mutual respect between the two parties.

External auditors should realize that the waythey approach, treat and refer to internal auditorsand others is going to have an impact on develop-ing a sound working relationship with them andtheir ability to rely on their work in the long term.One key element in establishing trust is that theexternal auditors have a clear appreciation of thestate of development, mandate and value of inter-nal audit or other appropriate function.

As trust and mutual respect develop, the ben-efits derived from the relationship can be expectedto increase substantially. Regular contact betweenthe two parties may result in the identification ofnumerous opportunities to improve audit effective-ness and reduce audit costs. In addition, each partyis likely to benefit from the sharing of methodolo-gy, terminology, training, follow-up efforts andevaluations that will inevitably occur.

Given substantially compatible objectives, itonly makes sense that the two parties coordinatetheir activities. Coordination might include:


• seconding personnel from one group to theother, usually on a project basis

• combining forces on a particular project, butwith ultimate responsibility for the projectresting with one group

• coordinating plans to ensure not only thatreliance is effectively implemented, but also thatthe inevitable demands they make on the timeof the staff of the organization are minimized

• one group performing specific tasks for theother

In addition to minimizing cost and disrup-tion, an important objective of coordinating activityis to make the best use of available resources. Forexample, internal auditors typically have the consid-erable depth of knowledge of the organization thatone would expect of a professional employee, andmay also enjoy knowledge and skills in relevant spe-cialist areas. Similarly, the external auditors maypossess particular skills in relevant specialist areassuch as EDP that are not available within the inter-nal audit group. There may be opportunities to mixand match resources to produce a synergisticimprovement in results for either or both groupswhile, at the same time, containing costs.

It is possible, of course, that over time therelationship between external and internal auditorsor other officials could sour. This is particularlylikely if either party sees a decline in the credibilityof the other in the eyes of the governing body orsenior management. It is in the interest of all to seethat this does not occur.

No matter how good the relationshipbetween external and internal auditors, there are stillseveral inherent differences in the nature of the twotypes of audit, and in the responsibilities of those per-forming them, that will affect the degree or extent ofreliance that can be expected. Where these factors arewell understood by both parties, the relationshipbetween them should be strengthened, not weakened.

R E L I A N C E

As discussed in chapter 20, in appropriate cir-cumstances external auditors may rely on the workof others and avoid having to replicate what hasalready been done. As noted earlier, reliance in thecontext of a comprehensive audit is the substitu-tion of work done by others for work that wouldotherwise have to be performed directly by theexternal auditors. In the context of this definition,work done by others includes their assessments aswell as the detailed supporting work.

The importance of reliance is recognized byall auditors. The Office of the Auditor General ofCanada Manual, for example, contains this policystatement:

Audit teams should rely, to the extentpracticable, on the work of internal auditorsand program evaluators and on other studies,if they have been carried out in accordancewith appropriate professional standards.278

For purposes of convenience, the discussionbelow refers primarily to reliance on the work ofinternal auditors. It should be remembered, howev-er, that there may be other groups in the auditedorganization that produce reports dealing with man-agement systems and practices, program perfor-mance, and so on. The work product from suchgroups should also be assessed for its reliance poten-tial. These studies are considered later in the chapter.

P L A N N I N G F O R R E L I A N C E

External auditors do not place reliance blind-ly. They must satisfy themselves that the work onwhich they rely meets appropriate standards. Asthe OAG Canada Manual says:

Where the auditor relies on entity stud-ies, internal audits or evaluation work, he orshe should obtain sufficient independent evi-dence to judge the quality of that work.279

P A R T I I I . C H A P T E R 2 2 . O T H E R R E V I E W P R O C E S S E S — C O O R D I N A T I O N & R E L I A N C E 3 1 3

External auditors can be expected to look atinternal audits and internal auditing arrangementsin two ways. The first is to evaluate the overallappropriateness of the function and the arrange-ments provided for it. External auditors will dothis where internal auditing is one of the manage-ment practices that falls within the scope of theirinquiry. Even where internal audit is not includedwithin the scope of the comprehensive audit, exter-nal auditors will examine these arrangementsbecause the assessment that results will serve as apredictor of the extent to which they can rely oninternal audits.

The second way is the assessment of individ-ual internal audit projects. External auditors willinitially identify the groups, however named, thatare carrying out internal audit work, and assess thesuitability of the arrangements for internal auditingin the organization.

A S S E S S M E N T O F A R R A N G E M E N T S

The overall assessment can embrace all signif-icant aspects of the arrangements for internalauditing in the context of what can reasonably beexpected in an organization of the size and natureof the specific one under examination.

External auditors will be interested in morethan just the current state of internal auditing inthe organization. To get a realistic appreciation ofthe function, they will have to consider its histori-cal development, stability and the manner in whichits relationships with other parts of the organiza-tion have developed. A mere snapshot of the func-tion might ignore a significant trend in its develop-ment and result in an assessment that might onone hand underestimate the growing credibility ofan internal audit group or, on the other, provide afalse sense of security to directors and managers.

Each organization must determine the level ofinternal audit services that is appropriate to its cir-

cumstances and culture and then establish the func-tion accordingly. In a specific organization at a par-ticular time, the internal audit function might beconsciously established with a mandate, organizationand/or resourcing strategy that does not allow it toaddress fully all aspects of management systems andpractices. This does not necessarily mean that thefunction is inadequate. Rather, it points out theneed for external auditors to take into account therespective contributions of both senior managementand internal auditors towards an effective internalaudit function. Accordingly, the external auditorsshould first explore the role and resources that seniormanagement establishes for the function and com-pare these with the what might be expected in suchan organization. Differences identified are theresponsibility of senior management and should bediscussed with them. External auditors can thenrealistically compare the actual organization scopeand performance of internal audit work to the man-date it has been given.

As in other aspects of comprehensive audits,the ideal situation is one in which the externalauditor substantiates management’s evaluation.Situations where the auditors’ assessment is signifi-cantly different from that of management or wherethe auditors must generate the assessment fromscratch, might indicate that the auditors have failedto understand management’s perspectives, or couldeven suggest that management is not taking itsresponsibilities for internal auditing sufficientlyseriously. Accordingly, in developing their opin-ions, external auditors will need to interview peo-ple other than internal auditors, including mem-bers of the audit committee, if one exists, andsenior management.

The following is a discussion of the key fac-tors that bear on the assessment and the kinds ofquestions that external auditors might first askthemselves in this regard.


Questions bearing on the auditors’ assessment ofinternal audit arrangements

Questions regarding the role of the internal auditfunction within the organization

• Do the general scope and specific plans ofinternal audit make sense in terms of theinherent risks of the operations, the organiza-tion’s strategic plans and the environment inwhich the organization operates?

• Do senior managers, the audit committee, ifthere is one, and the head of internal auditagree on the role of internal auditing and thebenefits it is intended to produce?

• Do organizational, staffing and reportingarrangements for internal auditing make sensein relation to the general scope and specificmission assigned?

• Is internal auditing producing the intendedbenefits?

• How well is internal auditing accepted in theorganization? Do senior mangers and man-agers being audited trust and respect theinternal auditor? Do they consider seriouslyand follow up on internal audit findings?

• Are undesirable potential effects of internalauditing, such as substitution of internalaudits for good management practice or anunduly adversarial environment, recognizedand avoided?

• How well does the internal audit functionadapt to new issues and interact with man-agement to secure resources and obtain man-agement’s cooperation and consideration ofits findings?

• Is the overall cost of internal auditing reason-able compared with industry norms and thework produced? Are there more cost-effectivemeans of providing some of the benefits thisorganization derives from internal auditing?

Questions regarding the internal audit group itself• Are responsibilities for the delivery of the

internal audit program clearly understoodand reflected in the organization, structureand auditing processes of the group?

• Do internal audits and the unit’s supportactivities (such as training) logically con-tribute to the mission of internal audit in theorganization?

• Do all members of the internal audit groupunderstand the role of internal audit in theorganization and how they should act as indi-viduals to further that role?

• Does the morale of internal auditors suggestthat the function is healthy and that compe-tent professionals will be attracted andretained? Are internal auditors challenged,provided opportunities for personal develop-ment and encouraged to use their initiative?

• Is there reasonable assurance that there will beappropriate continuity of key people withinthe internal audit function to provide aneffective “corporate memory”?

• Are key aspects of internal audit performance(quality, timeliness, acceptance) monitored andreported appropriately? Are these reports used?

• Are standards of performance met? Do moni-toring reports accurately reflect the perfor-mance against standards?

• Are appropriate auditing techniques andstaffing patterns used to maximize the outputof the function? Is the level of nondirect timereasonable?

K E Y FAC T O R S I N T H E A S S E S S M E N T

To assess the overall arrangements for internalaudit, auditors must appreciate the factors thatcontribute to effective internal audit and the rela-tionships between those factors. These factors canbe grouped under the following headings:


• supportive senior management and organiza-tional environment

• a clear mission and adequate direction withinthe mission

• appropriate leadership and skills• standards of performance and quality assurance

Supportive senior management andorganizational environment

Checklists for the evaluation of internal auditfunction are typically based on standards. They pro-vide a list of questions to be addressed and a frame-work for organizing the information gathered in thecourse of the review. They must, however, be tailoredto reflect the expectations of internal auditing in thespecific organization; they do not in any way relievethe external auditors of responsibility for assessingthe function in the context of any relevant legislativeprovisions and the organization’s circumstances.

Because of its importance to the function,external auditors must have a solid appreciation ofthe strength of, and limits to, management’s sup-port for internal audit, not only as reflected in for-mal documentation, but also in terms of thebehavioural aspects—the day-to-day interactionbetween internal audit and management. Theimplications of senior management support are dis-cussed in chapter 21.

A clear mission and adequate direction within themission

As described in the previous chapter, the sys-tems and practices subject to internal audit couldrange from the purely financial, to the full array ofmanagement systems and practices; and the level atwhich internal audits are expected to operate canrange from day-to-day operations, to the informa-tion provided to the board. Moreover, the focus ofinternal audits can range from assessing compliancewith clearly defined requirements, to identifyingopportunities for improvement (or deficiencies), to

providing overall opinions on the systems and prac-tices audited. The focus of internal audit in an orga-nization may change over time to reflect changes inmanagement structure, style or priorities.

As internal audit functions can experience somuch change, external auditors will be concernedthat the benefits that internal auditing is intendedto produce are clearly understood by all concernedand that they are reflected in internal audit plansand capabilities. An internal audit charter typicallyprovides a general description of the scope of sys-tems and practices to be subject to internal audits.Beyond this general description, expectationsshould be as clear as possible about the mission ofinternal audit—what it is supposed to do in respectof the systems and practices that it audits, and howit is supposed to benefit the organization.

Internal audits cannot be all things to allpeople in the organization at the same time.Attempts to please everybody usually result injust the opposite: internal audits are not exemptfrom this truism. A clear, realistic and sharedunderstanding of its mission is important for theinternal audit function’s success.

In this connection, it is worth noting that if theinternal audit function is directed unduly to produc-ing reports on which external auditors may rely, theremay be a tendency to pay less attention to otherissues of more direct interest to managers. If leftunchecked, this could result in the eventual loss ofsupport for the internal audit function and in theloss of some of the benefits that the organization hadpreviously been receiving from internal audits.Accordingly, it is important to keep in mind that theprimary purpose of internal auditing is to serve theorganization, not the external comprehensive auditor.

A mission is only a string of words until it isreflected in plans. External auditors should look forinternal audit plans that are realistic and thatreflect a logical relationship between the mission tobe achieved and the specific work to be done.


Typically, internal audit plans are based on athree- to five-year cycle, updated annually. The planfor the current year is usually developed in greaterdetail than the plans for the balance of the cycleand is subject to change as the internal auditorsrespond to changing circumstances, management oraudit committee priorities and specific requests.

External auditors will be interested in theplan and the process by which it is developed.Specifically requested audits and the involvementof senior management in the development of thebasic plan are indicators of support for internalaudit and should yield a plan that focuses onimportant issues. At the same time, numerouschanges in internal audit plans to accommodaterequested audits may suggest either that internalaudit judgments were faulty to begin with, or thatmanagement has drawn the function into the man-agement process to an extent that blurs its properrole. Although flexibility is good, excessive adhocery will be judged a weakness.

A P P R O P R I AT E L E A D E R S H I P A N D S K I L L S

External auditors will consider whether theknowledge, skills and experience of the internalaudit staff mirror the scope of their work. In doingthis, they will consider the attributes of good inter-nal auditors indicated in the previous chapter.

S TA N D A R D S O F P E R F O R M A N C E A N D Q U A L I T Y

A S S U R A N C E

Standards of performance and quality assur-ance are key characteristics of the practice of internalauditing. For reliance purposes, they are essential.

The external auditors will seek evidence thatstandards have been established for the performanceof internal audits and that effective processes havebeen adopted to assure the quality of individualaudits. The degree to which the evidence suggeststhat appropriate standards are applied to all internal

audits will affect the amount of corroborative workthe external auditors will perform in respect of anyparticular audits on which they are planning to rely.

It is important to note that external auditorswill apply the same standards to any specific pro-jects undertaken within the organization by groupsother than internal audit if they are to rely onthem in the audit. It is the standards that definethe audit, not the organizational designation of thepeople performing the work.

These, then, are the key factors in the assess-ment. What might the outcomes be? The next sectionexamines the range of outcomes that might resultfrom this assessment and their reliance implications.

O U T C O M E S O F T H E A S S E S S M E N T O F OV E R A L L

A R R A N G E M E N T S

In preparing their overall assessment of inter-nal audit arrangements, external auditors willaddress the following questions:

• Have an appropriate role and resources ade-quate to the role been given to internal auditin light of the nature and needs of the organi-zation?

• How well is internal audit using the resourcesto fulfill its given role?

Appropriateness of role and resourcesAs noted earlier, effective internal auditing

demands that both senior management and internalauditors play their respective parts. The directionand control of internal audit is a senior managementprerogative and responsibility; senior managementmust balance a number of sensitive factors whenthey exercise their prerogative. This investment ofmanagement effort should prove worthwhile, how-ever. An organization that sets realistic but challeng-ing expectations for its internal audit function, andprovides it with commensurate resources, has pro-vided a firm foundation upon which the internal


auditor can build. The prospects for an effectiveinternal audit function—and therefore forreliance—will have been significantly enhanced.

ORGANIZATIONS CAN ENHANCE THE PROSPECTS FOR

RELIANCE BY PROVIDING INTERNAL AUDIT WITH AN

APPROPRIATE ROLE AND RESOURCES.

A situation that should cause the externalauditors concern would be one in which an undulyrestricted role has been given to internal audit. Forexample, the auditors may find that, notwithstand-ing an apparent capability, internal audits of cer-tain functions are not performed, perhaps not per-mitted. Or they might find that the resourcesmade available for internal auditing are not suffi-cient. Such a situation would suggest that theinternal audit function is either not understood ornot accepted. It might well raise the possibility inthe external auditors’ minds that the organizationis not fulfilling its responsibilities with respect tothe other aspects of the management charge andthat there may be undisclosed weaknesses inunderlying systems and practices. At the very least,the auditors would have to consider reporting asignificant deficiency regarding internal auditing.

ORGANIZATIONS THAT HAVE NOT PROVIDED INTERNAL

AUDIT WITH AN APPROPRIATE ROLE AND RESOURCES

RUN THE RISK OF HAVING A SIGNIFICANT DEFICIENCY

REPORTED.

Internal audit’s fulfillment of its roleIt may be found that internal audit is failing

to fulfill, or to fulfill completely, a role that hasbeen reasonably defined, even though sufficient

resources have been provided to it. In the course ofthe transition to comprehensive internal auditing,for example, internal auditors will, on occasion, failto meet completely the objectives that they set forthemselves. Provided these occasions are acknowl-edged and honestly reported, auditors will recog-nize that it is better to have achieved part of anambitious goal than not to have tried.

In the worst-case scenario, internal audit isfound to provide assurance where it is not warrant-ed. Either by undertaking work for which they lackthe capacity, or through failing to communicate thesignificance of their findings, internal auditors mightconvey a false sense of security to senior manage-ment. In such circumstances, the external auditorswill have to determine whether the situation is suffi-ciently serious to constitute a significant deficiency.

WHERE THE ORGANIZATION HAS PROVIDED AN APPRO-

PRIATE ROLE AND RESOURCES, BUT THE INTERNAL

AUDIT FUNCTION HAS FAILED TO PERFORM UP TO

EXPECTATIONS, THE AUDITORS SHOULD CONSIDER

WHETHER A SIGNIFICANT DEFICIENCY EXISTS.

In such cases, the prospects for reliance areconsiderably diminished. Reliance requires a degreeof trust; a situation in which internal auditorsallow a false sense of assurance to be drawn fromtheir work does not generate that feeling.

Even though their assessment has disclosedsome weaknesses in the overall arrangements forinternal auditing, it may still be possible for theauditors to place some degree of reliance on specif-ic internal audits that are relevant to the audit andthat have been performed in accordance withappropriate standards. In such cases, of course, theauditors may require more evidence regarding theconformity to standards of individual internal


audits than in situations where the overall assess-ments are satisfactory. The depth of this projectreview will be correspondingly greater.

NOTWITHSTANDING DEFICIENCIES IN THE OVERALL

ARRANGEMENTS FOR INTERNAL AUDITING, AUDITORS

SHOULD CONSIDER PLACING RELIANCE ON ANY RELE-

VANT INTERNAL AUDIT THAT IS SATISFACTORILY PER-

FORMED.

D E V E L O P I N G A T E N T A T I V E

S T R A T E G Y F O R R E L I A N C E

I D E N T I F Y I N G P R O J E C T S F O R R E L I A N C E

To rely on internal audits, external auditorswill have to evaluate individual internal audit pro-jects to establish their relevance to the comprehen-sive audit and the competence with which theyhave been conducted. Although the competenceissue cannot be finally assessed until the internalaudit work has been completed, much can andshould be done prior to that time to provide abasis for a realistic reliance strategy.

The first step in any reliance strategy, then, isto identify all those internal audits, whetherplanned or completed, that are relevant to thecomprehensive audit. An auditor’s preferred optionshould be to rely on relevant internal audits unlessthe cost of doing so will clearly outweigh the cost,to the auditor and to the organization, of doingthe work directly.

During the survey of the organization, exter-nal auditors assess the risks inherent in the organi-zation’s operations and identify those systems andpractices that are vital to the audit opinion. Oncethese systems and practices have been identified,auditors should ascertain if internal audits havecovered, or plan to cover, all or part of the groundin which they are specifically interested.

As mentioned earlier, there may be groupswithin the organization that conduct studies that,although given a different name, could be consid-ered internal audits. Auditors should consider thework of all such groups to determine if they, too,produce candidate projects for reliance.

It may well be that a proportion of the workof internal audit is not of interest to external audi-tors. Such a situation may simply reflect differencesin the internal audit priorities of management andthe auditors’ particular selection of issues to beexamined in detail.

P R E L I M I N A RY R E V I E W

The external auditors should investigate thenature of the chosen internal audits to determinewhether they actually address issues relevant tothe audit. Where differences in scope, design orfocus of internal audits appear to present barriersto reliance, the external auditors and the internalauditor should explore whether the latter is will-ing to make adjustments to accommodatereliance. It must be stressed that internal auditorsserve the organization, not the external auditors;similarly, auditors must avoid giving the impres-sion that they are trying to take over internalaudit. In the long run, such a perception coulddiminish the potential for reliance by erodingmanagement support.

Unless there are obvious reasons that theexternal auditors could not expect to rely on inter-nal audits—for example, internal audits are beingplanned that address management systems andpractices of a nature, at a level or in a quantity forwhich the internal audit group clearly has notcommensurate resources—it should, in most cases,be possible to place at least some degree of relianceon internal audits, notwithstanding some differ-ences in scope, design or focus.


D O C U M E N T I N G T H E S T R AT E G Y

The audit plan will typically include a list ofprojects to be carried out by the external auditors.The plan should also describe the overall strategyfor reliance and the extent of reliance contemplatedfor each audit project. All the internal audit pro-jects planned could be displayed and those of rele-vance to the auditors highlighted along with theirexpected impact on the auditors’ work.

The presentation of the reliance strategy willvary greatly among organizations depending, inparticular, on the audit strategy and the approachto internal auditing. The reliance strategy must bedescribed as tentative, especially in the case of amulti-year audit plan, because it could change overtime with changing evaluations of internal auditperformance and other changes in the entity’s cir-cumstances.

Where appropriate, the audit committee canthen approve the strategy and monitor its imple-mentation.

THE FIRST STEP IN DEVELOPING A RELIANCE STRATEGY

SHOULD BE THE IDENTIFICATION OF ALL INTERNAL

AUDITS THAT ARE RELEVANT TO THE COMPREHENSIVE

AUDIT.

O P T I M I Z I N G P L A N N E D

R E L I A N C E P O T E N T I A L

Although reliance may be planned andexpected, the decision to rely on a particular inter-nal audit project cannot be taken before the projectis completed and its performance can be assessed.Much can be done prior to the final decision, how-ever, to ensure the success of tentative reliancestrategies. Perhaps the key for optimizing relianceis for all concerned—audit committees, seniormanagement, internal auditors and auditors—to

work closely in planning and coordinating thecomprehensive audit and internal audits. Two areasin particular deserve close attention: criteria andsignificance.

C R I T E R I A

An important facet of the audit plan is theidentification of the criteria proposed for assessingthe systems and practices to be studied. Criteria arethe yardsticks or standards that auditors use toassess the suitability of a system or practice. Asexplained in chapter 19, criteria describe the char-acteristics of a satisfactory mechanism.

As is also explained, in comprehensive auditingthere are no generally accepted criteria for manage-ment practices similar to those that have been adopt-ed by the accounting profession for financialaccounting practices. Nevertheless, if internal auditsare to be relied on, the external auditors and theinternal auditor must agree on criteria. If differentcriteria are applied, different conclusions will follow.Although the interests of internal auditors may gobeyond the interest of the external auditors, the crite-ria must be compatible where their interests convergeand reliance is contemplated. The earlier agreementis reached, the greater the likelihood that reliancestrategies will be successful, since from the beginningboth parties will know the criteria.

The criteria established for the audit caninfluence the time that will be required for theaudit and, hence, its costs, both direct and indirect.Accordingly, these yardsticks should be examinedcarefully before the audit plan is finally approved.

MANAGEMENT, THE AUDIT COMMITTEE (WHERE APPRO-

PRIATE), INTERNAL AUDITORS AND EXTERNAL AUDITORS

SHOULD REACH EARLY AGREEMENT ON THE CRITERIA

TO BE USED IN INTERNAL AUDITS THAT MIGHT BE

RELIED ON IN A COMPREHENSIVE AUDIT.



As discussed in chapter 18, the concept ofmateriality is fundamental to any form of audit. Inessence, it is the determination of what is importantor significant and what is not. It affects the amountof work that auditors will perform, since an audit isdesigned to detect such material departures fromacceptable criteria as may exist. In addition, it affectsthe auditors’ response to any departures they findfrom the criteria that have been applied. Auditorscan expect directors and managers to be interestedin the interpretation given to the question of signifi-cance because it will influence the costs of auditsand the nature of the matters that will be brought totheir attention in the final report.

It can be expected that, just as the concept ofmateriality is still being clarified and refined in thefinancial auditing field, so will it develop over timein comprehensive audits. Until more experience isgained and guidance becomes available, the primedetermining factor will be the judgment and com-mon sense of auditors and members of governingbodies with whom they discuss the issue.

The issue is important and unavoidablebecause the prospects for successful reliance areconsiderably diminished if the external auditorsand internal auditor are applying different stan-dards of significance. The significance levelsapplied by the auditors and the internal auditormust be compatible if reliance is to take place. Thiswill occur when the level used by internal audit isat least as stringent as that used by the auditors.Each has a separate perspective. Typically, the inter-nal auditor will consider significance in the contextof the specific organizational unit or system orpractice under review, whereas external auditorsmust consider significance in the context of what isimportant to the governing body. But each mustunderstand the other’s perspective for a satisfactoryreliance relationship to be established.

INTERNAL AUDITS SHOULD USE THE SAME CONCEPT OF

SIGNIFICANCE AS THE EXTERNAL COMPREHENSIVE AUDI-

TORS OR A MORE STRINGENT ONE.

The perspectives of users and potential usersof comprehensive audit reports may vary widely, as,indeed, may their motives for wanting to knowabout significant deficiencies in management sys-tems and practices. An understanding of these per-spectives in and of itself, important as it is, will notprovide complete answers to the significance ques-tion for the auditors. They must bring their ownjudgments to bear.

Readers will find a more extensive discussionof the concept of significance in chapter 18.

T I M I N G

There is no general rule that specifies theperiod covered by a comprehensive audit opinionor the timing of the work supporting the opinion.Nevertheless, the work, including internal auditson which reliance is placed, must relate to the peri-od covered by the opinion. Depending on wheninternal audit projects were performed and theperiod they covered, follow-up work may beneeded to confirm that conclusions remain validfor the period covered by the comprehensive audit.

External auditors, internal auditors and auditcommittees, where they exist, should therefore,consider the impact that the period selected for theaudit will have. Factors to examine include the dis-ruption caused for staff, the amount of workrequired to support the audit opinion and theamount of follow-up entailed. Once the period forthe audit has been decided, it may be possible toorganize the various projects in a way that willminimize costs.


COSTS CAN BE REDUCED AND THE PROSPECTS FOR

RELIANCE IMPROVED IF EARLY AGREEMENT IS REACHED

ON THE TIMING OF THE CONDUCT OF THE VARIOUS

ELEMENTS OF COMPREHENSIVE AUDITS AND INTERNAL

AUDITS.

C H A N G E S I N T E N T A T I V E

R E L I A N C E S T R A T E G I E S

A strategy to rely on internal audits will neces-sarily be tentative. For example, a strategy wouldrequire amendment if, for any reason, internal auditperformance falls short of expectations or if itappears that unwarranted conclusions have beendrawn. In addition, comprehensive audits may becarried out over an extended period. It is possiblethat conditions might change between the planningphase and completion of the audit. Accordingly, itis important that auditors monitor the progress andresults of internal audits on which reliance is con-templated to determine if changes are required inplans for reliance. Where such changes are neces-sary, auditors should report this to the client, man-agement and the internal auditors.

EXTERNAL AUDITORS SHOULD INFORM THEIR CLIENTS

OF ALL PLANS FOR RELIANCE ON INTERNAL AUDITS AND

ANY SUBSEQUENT CHANGES.

P L A C I N G R E L I A N C E

T H E N E E D T O E VA LU AT E I N D I V I D U A L I N T E R N A L

A U D I T S

Before external comprehensive auditors can beexpected to rely on internal audits, they will have tosatisfy themselves that the internal audits weredesigned with appropriate procedures and particu-

larly with appropriate criteria. They will ascertainthat each project was devised to provide a level ofdetail and rigour of assessment at least equal tothose the auditors need for their own purposes.

For projects that pass these tests, auditors mustdetermine whether the calibre of personnel involvedin the audit was adequate to achieve satisfactoryresults. Even the best-designed project is apt to fail ifthose who conduct it have insufficient ability, expe-rience and training to complete it competently.

An assessment must also be made of how welleach job was done: whether enough appropriateevidence was collected and whether it supportedthe analysis and conclusions drawn by the internalauditor.

If these conditions are met external auditorswill likely rely on the internal audit, accept its con-clusions and reflect this in the comprehensive auditreport. This assessment will have to be made foreach project on which reliance is planned.

EACH INTERNAL AUDIT PROJECT ON WHICH RELIANCE

IS PLANNED SHOULD BE EVALUATED SEPARATELY.

Focus of evaluationExternal auditors must evaluate each individ-

ual internal audit to determine its persuasiveness asa source of evidence and the significance of itsfindings to the audit opinion. To do this, auditorswill want to understand the internal audit—including its results as well as the auditing methodsused—in sufficient depth to allow them to appreci-ate the basis for the internal audit findings andconclusions, and their reliability. In this respect,auditors should explore, to the depth justified bythe importance of the internal audit in the overallcomprehensive audit strategy, the following:


• the organization, function or program subjectto internal audit;

• the objectives and scope of the audit and,particularly, any limitations on the scope ofthe audit;

• the criteria applied in performing the audit;• the evidence gathered by the internal auditors;• the logical links between the findings, conclu-

sions and recommendations;• the independence, objectivity and qualifica-

tions of the professional staff who carried outthe audit;

• the supervision and quality assurance proce-dures applied to ensure that the audit workmet appropriate performance standards;

• the significance considerations applied inevaluating the detailed findings;

• the processes used to finalize the report andobtain acceptance by management; and

• the formal and informal responses of manage-ment to the internal audit assessment.

External auditors must also compare the evi-dence produced by internal audits with informa-tion from other sources. Inconsistencies betweenevidence from different sources should be exploredso that auditors can reach a reasonably confidentopinion.

Factors influencing the depth of evaluationExternal auditors will make as detailed an

evaluation of individual internal audits as isrequired in their professional judgment. The extentof corroborating evidence required by auditors whointend to rely on an internal audit will vary fromproject to project. The rigour and depth of theauditors’ evaluation of an internal audit project is amatter of professional judgment, reflecting theimportance of the subject matter, the consequencesof being in error about deficiencies in the area andthe likelihood that an erroneous conclusion might

be drawn. Among the factors auditors might con-sider in making such judgments are:

• the importance of the project to the overallaudit strategy;

• the confidence the auditors have in the inter-nal audit function based on their overallassessment and on their experience with eval-uations of other specific internal audit pro-jects;

• the amount of time that has passed and thedegree of change that may have occurredbetween the completion of the internal auditand the date of the evaluation;

• the relative independence of the internalauditors vis-à-vis the manager who is respon-sible for the systems and practices underreview;

• the apparent consistency of the conclusionswith the evidence;

• the manager’s formal and informal reaction tothe report;

• the availability of complementary evidencecollected in other facets of the audit; and

• the rigour of the specific internal audit planand approach, including the calibre of thepeople involved in the audit, and the natureand extent of supervision exercised over thework done and the judgments made.

External auditors can obtain the informationneeded to evaluate an internal audit project in anumber of ways. The techniques they employ mayinclude:

• discussing key aspects of the audit project withthe internal auditor and/or team members;

• considering the consistency of the internalaudit assessment with other evidence available;

• ensuring that internal auditors understand thecontext in which auditors intend to rely onspecific audits and inquiring if the internalauditor knows of any reasons for not relying;


• reviewing audit project files to confirm thenature and extent of the work done and evi-dence gathered, to support the conclusionsand recommendations;

• reperforming some of the detailed testing tosupplement the internal audit work or to pro-vide direct, hands-on knowledge of the sub-ject matter; and

• performing additional work in the same area.

As auditors proceed through this spectrum ofincreasing amounts of effort, they are essentiallyobtaining additional evidence to corroborate orrefute the internal audit assessment, thereby maxi-mizing their level of comfort in their assessment ofthe findings and conclusions of the internal audit.

At the point where the external auditorsdecide to gather additional direct evidence in thesame area, they have essentially concluded thatthey cannot tolerate the possibility that the internalauditor’s assessment might be in error. They havedecided that evidence regarding the validity andrigour of the process followed in the internal auditis not sufficient; the risk of error in the audit opin-ion is too great to accept the internal audit reportwithout direct corroborating evidence. This may ormay not be a reflection on the performance of theinternal auditors. The issues involved may be socentral to the audit that the auditors feel a need fordirect evidence to corroborate the internal auditfindings or to put themselves in a position toappreciate their significance properly. Possibly,though, the auditors may have concluded thatthere is an unacceptably low probability that theinternal audit assessment is valid. To avoid possibleconfusion or misunderstanding, the external audi-tors should clearly differentiate between poor inter-nal audit performance and other reasons for doingwork directly.

EXTERNAL AUDITORS SHOULD PERFORM AS DETAILED

AN ASSESSMENT OF INTERNAL AUDITS AS NECESSARY,

BUT ONLY AS DETAILED AS NECESSARY, TO COME TO A

PROFESSIONAL JUDGMENT ABOUT RELIANCE.

R E P O R T I N G O N R E L I A N C E

External auditors should attempt to bedescriptive as well as declarative in reporting onreliance. They should do more than state that theyhave “relied on internal audits to the extent practi-cable” regardless of the extent to which reliance oninternal audits has contributed to the audit opin-ion. While such a statement might always be fac-tual, it is not informative to the readers of theaudit report.

TO MAKE REPORTING ON RELIANCE MEANINGFUL,

EXTERNAL AUDITORS SHOULD PROVIDE A CONCISE

DESCRIPTION OF THE NATURE AND EXTENT OF

RELIANCE PLACED ON INTERNAL AUDITS AND NOT

MERELY STATE THAT THEY HAVE RELIED ON THEM TO

THE EXTENT PRACTICABLE.

Instead, reports on the extent of reliancewill be more meaningful if auditors provide aconcise statement of the nature and extent of thereliance placed on internal audits. Depending onthe circumstances, it may be appropriate todescribe such matters as the areas in which theassessment of internal audits have been acceptedin reaching the audit opinion and the areas cov-ered by internal audits where reliance could notbe placed. In the latter case, the external auditorsshould differentiate between instances where theyconcluded that doing the work directly would be


less expensive than doing the work required forreliance, where they could not agree with theinternal audit assessment without substantialadditional work, and where they arrived at a con-trary assessment.

The discussion of areas where reliance couldnot be placed should be carefully written to avoidany unintended criticism of internal audit.

O T H E R F O R M S O F R E V I E W

As mentioned earlier, there may be othergroups within an organization—such as programevaluation groups—that produce reports dealingwith management systems and practices or otheraspects of an organization’s performance. Neitherinternal auditors nor comprehensive auditors willwant to cause unnecessary costs by ignoring orreplicating such reviews.


FEDERAL CROWN CORPORATIONS

AS MENTIONED IN THE PREVIOUS CHAPTER, FEDERAL CROWN CORPORATIONS ARE REQUIRED BY LAW TO CONDUCT INTERNAL

AUDITS IN ORDER TO ASSESS WHETHER THE CORPORATION’S SYSTEMS AND PRACTICES GIVE REASONABLE ASSURANCE THAT:

• ASSETS ARE SAFEGUARDED;

• THE REQUIREMENTS OF THE FINANCIAL ADMINISTRATION ACT AND OTHER PERTINENT AUTHORITIES ARE

COMPLIED WITH;

• RESOURCES ARE MANAGED IN AN ECONOMICAL AND EFFICIENT WAY; AND

• OPERATIONS ARE EFFECTIVE.

THE ACT ALSO CHARGES AUDITORS TO RELY ON THE WORK OF INTERNAL AUDIT TO THE EXTENT THEY THINK PRACTICAL AND TO

REPORT ON THE EXTENT OF THAT RELIANCE.

FOR THESE CORPORATIONS, THE SCOPE FOR INTERNAL AUDITING IS POTENTIALLY IDENTICAL TO THAT OF THE SPECIAL EXAMI-

NATION (IN ESSENCE, A COMPREHENSIVE AUDIT) THAT MUST PERIODICALLY ASSESS MANAGEMENT SYSTEMS AND PRACTICES.

THIS SIMILARITY IN SCOPE CREATES THE POTENTIAL FOR SIGNIFICANT RELIANCE: TWO AUDITORS ARE CHARGED WITH EXAMIN-

ING THE SAME SUBJECT MATTER TO MAKE SIMILAR SORTS OF JUDGMENTS. THE LEGISLATION SPECIFIES THAT INTERNAL AUDI-

TORS ASSESS WHETHER SYSTEMS AND PRACTICES PROVIDE REASONABLE ASSURANCE AND THAT EXAMINERS DO THE SAME. TO

THE EXTENT THE INTERNAL AUDITS FULFILL THE STATUTORY EXPECTATION, RELIANCE BY EXAMINERS ON INTERNAL AUDITS

WILL BE FACILITATED.

THIS PRESENTS THE AUDITORS WITH A DILEMMA, HOWEVER. IT IS WELL ESTABLISHED THAT EXTERNAL AUDITORS CANNOT ACCEPT

MANAGEMENT’S ASSESSMENTS WITHOUT CORROBORATION, YET AUDITORS ARE CALLED UPON TO RELY ON INTERNAL AUDITS—A

MANAGEMENT PRACTICE. TO SOME EXTENT, THE ACT MITIGATES THIS DILEMMA AND, AT THE SAME TIME, UNDERLINES THE IMPOR-

TANCE OF THE INTERNAL AUDIT FUNCTION BY PRESCRIBING THAT THE BOARD OF DIRECTORS, THROUGH ITS AUDIT COMMITTEE,

OVERSEE INTERNAL AUDITS. THE BOARD OF DIRECTORS HAS BEEN GIVEN A RESPONSIBILITY TO MAKE SURE THAT APPROPRIATE

INTERNAL AUDIT ARRANGEMENTS ARE INSTITUTED AND, THROUGH THIS RESPONSIBILITY, HAS A STAKE IN RELIANCE ISSUES. THIS

ARRANGEMENT ALLOWS FOR SOME DISTANCE BETWEEN INTERNAL AUDIT AND MANAGEMENT, THEREBY INCREASING THE INDEPEN-

DENCE OF THE INTERNAL AUDIT FUNCTION.

AUDITORS CAN BE EXPECTED TO TAKE INTO ACCOUNT THE NATURE OF THE OVERSIGHT ROLE PLAYED BY THE AUDIT COMMITTEE IN

THEIR RELIANCE DECISION.

Some of these reviews may have the charac-teristics of an audit. These characteristics relate tothe independence, objectivity and competence ofthe auditors and the rigour of the approaches theyemploy in the performance of their work. Providedan internal review possesses the characteristics ofinternal auditing set out above, it can and shouldbe regarded as an internal audit for purposes ofreliance.

ALL AN ORGANIZATION’S INTERNAL REVIEW ACTIVITIES

SHOULD BE TAKEN INTO ACCOUNT IN CONSIDERING

RELIANCE.

Although they may not be regarded as inter-nal audits, reviews that do not adhere to these stan-dards should not be ignored. They may be animportant part of the organization’s systems andpractices and provide information, insights andsuggestions that should be taken into account.Unless internal reviews meet the standards, howev-er, it is unlikely that auditors will be able to rely onthem to as great an extent as on work that meetsall the criteria of an internal audit.

THE DEGREE OF RELIANCE PLACED ON SPECIFIC INTER-

NAL REVIEWS SHOULD REFLECT THE EXTENT TO WHICH

THEY HAVE MET APPROPRIATE ORGANIZATIONAL AND

PERFORMANCE STANDARDS FOR INTERNAL AUDITING.

Some organizations are assessed by outsidereviewers other than external auditors. For exam-ple, hospitals and a number of other institutionsare subject to an accreditation process. This is aninspection by a provincially or nationally recog-nized body to evaluate if established standards arebeing met. An accreditation survey provides asnapshot of the organization’s performance withregard to those standards, and is usually repeatedon a regular basis. Because accreditation is carriedout by recognized experts in the relevant field orfields, the reports they generate are authoritativeand may provide a source of reliable evidence forcomprehensive auditors.

Altogether, there may be a substantial constel-lation of reviews that may provide evidence in acomprehensive audit. By way of illustration, the fol-lowing two tables describe the sorts of reviews thata team engaged in a comprehensive audit of a hos-pital, or a community college in British Columbia,are likely to be able to consider. Depending on theterms of reference of the audit, and the calibre ofthe work that the auditors determine has gone intosuch reviews, these reports may provide valuableevidence upon which some degree of reliance maybe placed.


H O S P I T A L R E V I E W P R O C E S S


Description

Reportrecipient

Performedby

Use ofresults

Frequencyof review

Review cri-teria andprocess doc-umented in

Opinion on the finan-cial statements

Board of trustees ormembers of the hospitalcorporation

External auditor

Funding bodies, boardand senior management

Annual

CICA Handbook givesprocess standards.Canadian HospitalAccounting Manual(CHAM) providescriteria

Inspection surveyReviews all or somefunctions of hospitalagainst national stan-dards.

A “snapshot” of hospitalstructure and process,with objective of pro-moting quality of care.

CCHA survey covers allhospital departmentsand services. Otheraccreditation surveysfocus on specific, esp.teaching areas

Board of trustees

Senior management

Senior medical staff rep-resentative

External survey team

Action to correct defi-ciencies prior to nextsurvey

Re-survey after one, twoor three years dependingon previous accredita-tion award

Standards manuals for dif-ferent types of hospitals.

Forms completed byhospital in advance ofreview.

Survey questionnaire foruse by survey personnel

A system for ongoingmeasurement of quality-of-care delivery, encom-passing both processand outcomes

Action to eliminateproblems and to moni-tor activities to assuredesired results have beenachieved

Required for full hospi-tal accreditation

Senior management

Middle management

Senior medical staff rep-resentative

Quality assurance com-mittee and coordinator

Senior management,departmental and clini-cal staff implementmethods to resolveproblems

Ongoing

Hospital develops quali-ty assurance (QA) man-ual containing standardsset by each departmenthead under the QualityAssurance Committee’sguidance

A professional peer-review process thatexamines the quality ofcare provided to individ-ual patients.

May be retrospective(review of charts) orconcurrent.

Medical audits are oftenundertaken separatelyfrom quality assuranceactivities

Senior medial staff rep-resentative

Clinical departmentheads

Professional audit com-mittee

Identification of contin-uing education needs forphysicians and otherhealth care personnel

Mechanism for developingstandards of clinical care

No specified frequency

No external manual oncriteria or process

Financial Audit Accreditation Quality Assurance Clinical Reviews


A process for evaluating andminimizing risk in order toimprove the quality andsafety of hospital operations

Deals systematically withincidents in order to mini-mize hospital’s liability

Board of trustees

Senior management

Risk management commit-tee and or risk manager

Establish methods to mini-mize risks and limit liability

Ongoing

Manual developed internallyto meet needs of hospital

Insurance industry develop-ing external manuals

Evaluation of the use ofresources against internal orexternal criteria

Often focuses on length ofstay, use of diagnostic andtherapeutic procedures,drugs and complementaryservices

Impact analyses predictsresource utilization of newmedical staff and expandedor new programs

Senior management

Senior medical staff repre-sentative

Utilization review commit-tee and or admissions anddischarge committee.

Hospital-wide resourcemanagement committee

Management and clinicalstaff implement recommen-dations affecting overall useof resources or the practicepatterns of individual clini-cians

Usually monthly

No external manual on cri-teria or process. But certainhospital and medical associ-ations have developedguidelines

Evaluation of the outcomesof a particular program,procedure or productagainst set objectives

Often involves comparisonof established new methodsof providing care

Senior management

Senior medical staff repre-sentative

Evaluation specialists,usually external

Funding bodies, boards andsenior management useinformation to approve,modify or eliminate pro-grams

One time projects, oftenassociated with a trial phase

Established researchmethodology for measuringoutcomes and verifyingcausal relationships

Role studies and Masterprograms assess the rele-vance and capacity require-ments of services providedby a hospital.

Operational and otherreviews assess functioning ofone or more departments,usually focusing on econo-my and efficiency

Board of trustees

Senior management

External agent (typically amanagement consultant)

Funding bodies, boards andsenior management normal-ly require implementationof some or all recommenda-tions

Role studies/Master pro-grams:

Occasional, often associat-ed with redevelopmentplanning

Operational reviews:Occasional, usually whenan operational problemhas been identified bysenior management

Various manuals on criteriaand process developed bydifferent consulting firms

Risk Management Utilization and Program/Procedure/ Other ReviewsImpact Reviews Product Evaluation


College and Institute Review Processes

Review of Instructional Programs

Description

Reportrecipient

Performed by

Strategic PlanReviews

Update & amendinstitutionalobjectives in lightof changingconditions

Board, ministry,public

Administrationand faculty/staffteam

InstitutionalEvaluation Process

Evaluation ofinstitutionalgoals, activitiesand qualitycontrol processes.Usually carriedout in two phases:1. internal self-

study2. external

validation ofthe self-study

Board ofgovernors,ministry,institution

1. Crossinstitutionalinternal team

2. External auditteamappointed byprovincialsteeringcommittee.

InternalProgram Reviews

Review ofprogram content,instructorqualifications,delivery methods,facilities,processes forcurriculumupdates, studentoutcomes

Board,administration,educators,advisorycommittees

Ad hoc programreview team,usually includesexternalcomponent

Accreditation

Review and approval ofspecificeducationalprograms toensure they meetthe requirementsof the accreditingbody

Administrationand theaccrediting body(e.g. registerednurses association,engineeringtechnologistsassociation, etc.)

Varies, usually ateam named byaccrediting body

CourseArticulation

Review of coursecontent to ensurematerial andexpectations ofstudents areharmonized forcourse-credittransfer purposesbetweenpostsecondaryinstitutions

Instructors,departmentheads, deans

Subjectarticulationcommittees (e.g.,mathematics,English,engineering,trades) whichincludeinstructors fromarticulatinginstitutions

Student Follow-up

Survey of studentoutcomes andsatisfaction bymeans ofprovincewidesurveys

Administration,ministry, boards,students, advisorycommittees,educators

Surveyconducted byeach institution

Link File Project

Tracks studentflows into, andthrough full-timeacademic andcareer/techprograms in thepostsecondarysystem. Trackseducation-outcome data, notemployment data

Adminstration,ministry,articulationcommittees,educators

Registrarscontribute data toprovincial steeringcommittee

Support ServiceReviews

Review ofinstitutionalsupport services,e.g., libraryadmissions,finance

Board,administration,staff

Ad hoc reviewcommittee,including externalcomponent

Financial Audit & Enrolment Audit

Opinion on the1. financial

statements and 2. enrolment

statistics

Board ofgovernnors,ministry, public,funding bodies,seniormanagement

External auditors

A C C O U N T A B I L I T Y , P E R F O R M A N C E R E P O R T I N G , C O M P R E H E N S I V E A U D I T - A N I N T E G R A T E D P E R S P E C T I V E 3 3 0

College and Institute Review Processes

Review of Instructional Programs

Use of results

Frequency

Review criteriaand processdocumentedby:

Guides institutionin settingpriorities and inallocatingresources

Annual review,three-year update

Ministry ofAdvancedEducation

Renewal andimprovement ofthe institution

Base for andcheck on strategicplanning

Six years

Provincial steeringcommittee

Institutions canadapt process tosuit their ownneeds. AdvancedEducationCouncil of BritishColumbia(AECBC) throughProvincialInstitutionalEvaluationSteeringCommittee

Renewal andimprovement ofthe program

Part ofinstitutionalevaluation process

Each program isreviewed on a 5-8year cycle

AECBC throughProvincialInstitutionalEvaluationSteeringCommittee

Validate use ofthe institution’seducation andtraining as a basisfor awarding thecredential rightsand/or privilegesof the accreditingbody

As determined byaccrediting agency

Accreditingagency

Granting ofcourse credit tostudentstransferringbetweeninstitutions

Ongoingmonitoring withchange dependingon pace of changein subject areas(e.g. computerscience vs.medieval English)

As agreedbetweenparticipatinginstitutions

Ongoing appraisalof programs, informal programreviews, in overallappraisal ofinstitutionalquality, inidentifyingemployers ofgraduates

Annually, ninemonths afterstudent has leftthe institution

StudentOutcomesWorking Group

Understanding ofstudent educationand trainingpatterns andachievements

Annually

Link File SteeringCommittee

Renewal andimprovement ofsupport services;part ofinstitutionalevaluation process

Each service isreviewed on a 5-8year cycle

Established byProvincialInstitutionalEvaluationSteeringCommittee

Monitor financialneeds, processesand resources

Annual

1. Ministryspecifies formatin consultationwithinstitutions,CICAHandbook

2. Ministry guidefor enrolmentaudits

Strategic PlanReviews

InstitutionalEvaluation Process

InternalProgram Reviews

Accreditation CourseArticulation

Student Follow-up

Link File Project Support ServiceReviews

Financial Audit & Enrolment Audit

Not all organizations, of course, may have aswide a range of reviews as these institutions typi-cally do. Where they exist, however, studies whichare germane to the audit should be considered bythe auditors with a view to reliance. This applies toreviews made in-house, by professional associa-tions, and to studies made under contract by out-side consultants.

ALL RELEVANT STUDIES MADE BY REVIEWERS OUTSIDE

THE ORGANIZATION SHOULD BE CONSIDERED FOR

THEIR RELIANCE POTENTIAL.


278 OAG MANUAL, §6051.01.279 IBID, §6051.05.

C H A P T E R 2 3

STANDARDS ANDQUALITYASSURANCE

No professional discipline can long survivewithout establishing criteria for its practitioners.Prospective clients must be confident that the servicesthey are contracting conform to objective standards ofperformance and must be assured of the competenceof the people who will be performing the work.

The issue of quality assurance has engagedcomprehensive audit practitioners from the outset.They recognized that, young though the disciplinemay be, it is important for its credibility to stake-holders, its acceptance by governing bodies andmanagers and its usefulness in the public sector thatits practice be characterized by true professionalismand due care. To this end, practitioners took actionon two fronts: the establishment of professionalstandards, and the development of techniques toassure the quality of each audit undertaken. It isthese two issues—value-for-money audit standardsand quality assurance—that this chapter addresses.

V A L U E - F O R - M O N E Y A U D I T

S T A N D A R D S

As discussed in chapter 14, in 1987 practitionersof comprehensive auditing turned to the Public SectorAccounting and Auditing Committee of the CanadianInstitute of Chartered Accountants to develop a state-ment of standards. CICA completed its work in 1988,and in March of that year issued Value-For-MoneyAuditing Standards. These standards are a central refer-ence point for comprehensive audit practitioners, andare reproduced in their entirety as follows:


V A L U E - F O R - M O N E Y A U D I T I N G S T A N D A R D S


.01 Value-for-money auditing is one of the three main elements of comprehensive auditing, the oth-ers being the audit of financial statements and auditing for compliance with legislative and relatedauthorities.

.02 In general terms, comprehensive auditing is concerned with accountability for the discharge ofassigned responsibilities. Auditing in the Public Sector, Section PS-5000, discusses comprehensiveauditing and accountability and includes a description of the roles of governing bodies, managers andauditors in the accountability process. The other two main elements of comprehensive auditing areaddressed in Audit of Financial Statements in the Public Sector, Section PS-5200, and Auditing forCompliance with Legislative and Related Authorities, Section PS-5300.

.03 This Section recommends standards for auditors engaged in value-for-money auditing that relateto the professional qualities of those auditors, the conduct of their audit examinations and the con-tent of their audit reports.

.04 These standards are intended to apply to audits carried out for the purpose of examining andreporting on matters related to any or all of the following:

(a) The adequacy of management systems, controls and practices, including those intended tocontrol and safeguard assets, to ensure due regard to economy, efficiency and effectiveness. (b) The extent to which resources have been managed with due regard to economy and efficiency. (c) The extent to which programs, operations or activities of an entity have been effective.

.05 Value-for-money audits may be carried out with respect to an entity or only a portion of anentity, such as a program, management control system or organizational unit. The terms of the audi-tor’s mandate, whether embodied in legislation or established by contract, specify the audit andreporting requirements. The auditor may be asked to examine all or only some of the matters set outabove. For example, regarding effectiveness, some legislated mandates require the auditor to examine,assess and report on the existence and adequacy of procedures to measure and report on the effective-ness of programs but do not ask the auditor to report on the extent to which the programs, opera-tions or activities are themselves effective.

.06 The reporting requirements of value-for-money auditing mandates also vary. Manyvalue-for-money auditing mandates, such as those relating to federal and provincial governmentdepartments and agencies, require the auditor to report deficiencies observed. However, other audit-ing mandates require the auditor to express an opinion, such as whether there is reasonable assurance,based on specified criteria, that there are no significant deficiencies in the systems and practices examined.

.07 The auditor may also attest to written assertions prepared by management to demonstrate man-agement’s due regard for economy, efficiency and/or effectiveness in discharging their responsibilities.

P A R T I I I . C H A P T E R 2 3 . S T A N D A R D S A N D Q U A L I T Y A S S U R A N C E 3 3 3

As practice develops in this area, additional guidance may be required to clarify the interpretation andapplication of these basic standards.

.08 In this Section: (a) “Economy” refers to the acquisition of the appropriate quality and quantity of financial,human and physical resources at the appropriate times and at the lowest cost. (b) “Efficiency” refers to the use of financial, human and physical resources such that output ismaximized for any given set of resource inputs, or input is minimized for any given quantityand quality of output provided. (c) “Effectiveness” refers to the achievement of the objectives or other intended effects of pro-

grams, operations or activities.

G E N E R A L S T A N D A R D S

.09 Auditing is predicated on the audit team’s knowledge, competence and objective state of mind,and on the due care exercised in carrying out the examination. Accordingly, these professional quali-ties are reflected in the following General Standards.

.10 The person or persons carrying out the examination should possess or collectively possess the knowledgeand competence necessary to fulfill the requirements of the particular audit. [March 1988]

.11 The auditor, who is in charge, has overall responsibility for a particular audit engagement,including the responsibility to determine the content of the report. However, in fulfilling the auditengagement, the auditor may delegate portions of the work pertaining to the planning, execution orsupervision of the engagement to other members of the audit team. The ability to make judgmentsand assessments in fulfilling the requirements of the particular audit is dependent on the audit team’sknowledge and competence to perform the tasks necessary. Collectively, they will need adequateknowledge in both relevant value-for-money auditing techniques and the matters subject to audit, aswell as the ability to apply such knowledge. In addition, an overall understanding of the nature of theoperations is important so that the members of the audit team have an appreciation of the context ofthe matters subject to audit.

.12 Knowledge and competence in value-for-money auditing can be obtained through a combina-tion of training and experience. The nature and extent of the training and experience required byindividuals on the team will vary according to the objectives and scope of the particular audit, andthe work responsibilities and functions allocated to each individual. Nevertheless, all audit teammembers require an understanding of the basic objectives of auditing and the audit process sufficientto understand and apply these value-for-money auditing standards.

.13 Knowledge in many fields may be required to carry out specific value-for-money auditingengagements. The audit may focus on any of the entity’s management systems, controls and practicesand/or its operating performance or program effectiveness. Depending on the matters subject toaudit, knowledge of and competence in fields such as engineering, statistical analysis, human resourcemanagement and economics, among others, may be required to make appropriate analyses and com-


petent assessments. The auditor is not expected to possess the expertise of specialists on the auditteam but must have a level of knowledge sufficient to define the objectives and terms of referencegoverning the work assigned to them. When using specialists on the audit team, the auditor shouldobtain reasonable assurance concerning the specialists’ competence in their fields.

.14 The examination should be performed and the report prepared with due care and with an objectivestate of mind. [March 1988]

.15 Due care and objectivity are requisite professional qualities for all auditors, including thoseinvolved in value-for-money auditing. Due care imposes a responsibility on audit team members toperform their work diligently and conscientiously. Audit team members should maintain an objectivestate of mind in order to remain unbiased in forming their conclusions. They should also be free, andappear to be free, from relationships that may bias their professional judgment.

.16 Value-for-money audit reports often include recommendations to address identified deficiencies.The auditor should consider the effect that offering such advice may have on his or her objectivity insubsequent audits of the same entities. The auditor’s recommendations may point to the direction inwhich positive changes can be made; however, detailed plans and implementation of changes are theresponsibility of management.

E X A M I N A T I O N S T A N D A R D S

.17 The work should be adequately planned and properly executed. Audit team members should be prop-erly supervised. [March 1988]

.18 To plan and carry out an audit examination, all members of the audit team need to possess orobtain knowledge of the entity, or portion thereof, being audited and its environment appropriate fortheir assigned responsibilities. Such knowledge would include an understanding of the entity’s rele-vant objectives, constraints, resources, management processes and accountability relationships.

.19 Planning consists of developing an overall strategy and a detailed approach for the expectednature, extent and timing of the examination. In a value-for-money audit, planning involves estab-lishing the objectives, scoping the examination, assessing the suitability of identified criteria anddetermining the specific procedures required to achieve the objectives. Decisions made in the plan-ning stage with respect to these matters may need to be modified as information is acquired in thecourse of performing the audit.

.20 Supervision is designed to ensure that the planned procedures are properly carried out.

.21 When multi-disciplinary audit teams are used, adequate planning and supervision are particular-ly important so that the team members’ different perspectives, experience and specialties are appropri-ately used in the overall audit. It is important that all team members understand the objectives of theparticular audit and terms of reference of the work assigned to them. Adequate supervision is impor-tant so that the work of all team members is executed properly and is in compliance with these audit-ing standards. The auditor needs to have an adequate appreciation of the methods, assumptions and


source data used by all team members, particularly specialists, to be able to assess the reasonablenessand significance of their findings.

.22 In planning the matters to be examined and the work to be performed, the auditor should be cog-nizant of the concept of significance. Significance and materiality are synonymous concepts; however,significance is the term often used in the context of auditing in the public sector because it is embeddedin legislation and practice. Both terms encompass qualitative as well as quantitative considerations.

.23 Significance is judged in relation to the reasonable prospect of a matter influencing the judg-ments or decisions of a user of the audit report. For example, factors that may influence the auditor’sjudgment as to what is significant in a particular circumstance might include the potential public,legislative, economic or environmental impact.

.24 Criteria for evaluating the matters subject to audit should be identified and the auditor should assesstheir suitability in the circumstances. [March 1988]

.25 Auditors need criteria against which to evaluate matters subject to audit. Criteria are reasonableand attainable standards of performance and control against which the adequacy of systems and prac-tices and the extent of economy, efficiency and effectiveness of operations can be assessed.

.26 There is no body of generally accepted criteria for all aspects of value-for-money auditing.Criteria may be developed from various sources, including:

(a) legislation or policy statements; (b) standards of good practice developed by professions or associations; (c) statistics or practices developed within the entity or among similar entities; and (d) criteria identified in similar value-for-money audits.

Criteria identified from these sources may require interpretation and modification to ensuretheir relevance to the entity under audit.

.27 The auditor has a responsibility to assess whether identified criteria are suitable. Suitable criteriaare those that are relevant to the matters being audited and appropriate to the circumstances.

.28 When management has developed criteria for assessing systems, practices and operations, theauditor would use those criteria if in his or her opinion the criteria are suitable. If the auditor believesthat criteria proposed by management are not suitable in the circumstances, the auditor shouldattempt to resolve differences in opinion. If differences cannot be resolved, the auditor should consid-er the alternative courses of action available. Some auditors, such as federal and provincial legislativeauditors, are obliged under their mandates to carry out an examination and report. Therefore, in theevent that differences cannot be resolved, they would select suitable criteria and proceed. Other audi-tors may limit the scope of the examination to those areas in which agreement on criteria can bereached or, depending on the extent and significance of the disagreement, they may have to considerresigning the engagement.

.29 In no circumstances should the auditor perform the audit and report on the basis of criteria thathe or she believes are unsuitable.


.30 If suitable criteria cannot be identified by either management or the auditor for some of thematters subject to audit, the scope of the audit would be correspondingly reduced and the limitationin scope addressed in the report.

.31 Sufficient appropriate audit evidence should be obtained to afford a reasonable basis to support thecontent of the auditor’s report. [March 1988]

.32 In determining the sufficiency and appropriateness of evidence, the auditor is guided by theneed to minimize the risk of reporting erroneous findings or inappropriate conclusions.

.33 The sufficiency and appropriateness of evidence are interrelated concepts and refer to the quan-tity and quality of evidence. The decision as to whether a sufficient quantity of evidence has beenobtained will be influenced by its quality.

.34 Generally, evidence obtained directly by the auditor (for example, through observation or analy-sis) is more reliable than information obtained indirectly, and documentary evidence is more reliablethan oral evidence.

.35 The auditor may find it necessary to rely on evidence that is persuasive rather than conclusive.Thus, the auditor normally seeks corroborating evidence from different sources or of a differentnature in making assessments and forming conclusions.

R E P O R T I N G S T A N D A R D

.36 There is no standard report in current value-for-money auditing practice that is analogous to theauditor’s standard report on financial statements. Value-for-money auditing reports vary betweenjurisdictions and audits depending on differences in audit mandates, and in the scope and complexityof the particular audit and its findings. Regardless of variations in reporting, there are some funda-mental elements that should be included in any value-for-money auditing report. These are set out inthe general Reporting Standard below.

.37 The auditor’s report should:(a ) describe the objectives and scope of the audit including any limitations therein;(b ) state that the examination was performed in accordance with the standards recommended in thisSection and accordingly included such tests and other procedures as the auditor considered necessary inthe circumstances; (c ) identify the criteria and describe the findings which form the basis for the auditor’s conclusions; and(d ) state the auditor’s conclusions. [March 1988]

.38 It is important that the report describe the objectives of the audit examination and what hasbeen audited so that readers can understand and properly interpret the results. The objectives of theaudit are based on the audit mandate and should set out clearly the purposes of the particular audit.The scope of the audit is described by identifying the entity, or portion thereof, subject to audit (forexample, the corporation, department and/or program); identifying or referring to the matters exam-ined; and describing the time period covered by the audit.


.39 Any limitations in scope and the reasons therefor should be described in the audit report. Ascope limitation occurs, for example, when the auditor is unable to audit key organizational units orsystems or to perform necessary audit procedures due to factors beyond the auditor’s control. Thescope of the audit can also be limited by the inability to identify or agree on suitable criteria, as dis-cussed in paragraphs PS-5400.28 and PS-5400.30. The auditor would consider whether it is appro-priate to comment, in the report, on the implications of the lack of suitable criteria for the entitybeing audited.

.40 The general and examination standards set out the professional qualities expected of auditorsengaged in value-for-money auditing and basic standards for the conduct of value-for-money auditexaminations. Thus, the report should inform readers that the examination was performed in accor-dance with these standards and that the auditor made judgments as to what tests and other proce-dures were necessary in the particular circumstances to comply with the standards.

.41 The report should identify the criteria and describe the findings sufficiently to allow readers tounderstand the basis upon which the auditor formed his or her conclusions. Audit findings arise froman examination of the underlying facts, comparisons with suitable criteria and the auditor’s analyses.The criteria upon which the auditor made his or her assessments and formed his or her conclusionsmay be made apparent in the description of findings or may be set out separately in the report.Alternatively, the auditor may refer the reader to an accessible source. In any case, the criteria shouldbe clearly identifiable.

.42 The report should clearly state the auditor’s conclusions. Conclusions should be related to theobjectives and scope of the audit and should follow logically from the description of the criteria andfindings.

.43 Value-for-money audit reports may also incorporate the auditor’s recommendations, as well asmanagement’s responses with respect to the matters reported.

A P P L I C A T I O N O F T H E R E P O R T I N G S T A N D A R D I N

S P E C I F I C C I R C U M S T A N C E S

.44 Reporting requirements vary in accordance with the specific value-for-money auditing man-dates, whether embodied in legislation or established by contract. The following provides generalguidance to legislative and other auditors who are required by current legislation to state their conclu-sions: (a) in the form of an opinion with respect to the entity, or some portion thereof, or (b) in theform of observed deficiencies.

(a) When required by the audit mandate to express an opinion with respect to an entity, or por-tion thereof, the auditor’s opinion should be clearly expressed, or a statement made that anopinion cannot be expressed and the reasons therefor. Adequate explanation should be providedwith respect to any reservations contained in the opinion.

The precise wording of the auditor’s opinion will depend upon the objectives and scope ofthe audit mandate. The auditor’s opinion should be related to the criteria upon which it is


based. In addition, the words “in all significant respects” are often added, to convey to readersthe importance of judgment in relation to significance.

A qualified opinion would be expressed if there is a limitation in scope, unless the limita-tion is so pervasive that there is no basis for an opinion. In the latter case, the auditor woulddeny an opinion. Similarly, there may be matters, such as deficiencies reported, that qualify theaudit opinion but are not so significant as to necessitate an adverse opinion. Qualificationsshould be clearly described in the report.

(b) Auditors, such as those of federal and provincial governments, may be required by theirmandates to report observed deficiencies in management systems, controls and practices, orother instances when there has not been due regard for, or the achievement of, economy, effi-ciency or effectiveness. In these circumstances, the auditor’s conclusions relate to deficienciesobserved, and accordingly the report should, in the context of the scope and objectives of theexamination, identify each deficiency and identify the criteria and describe the findings relatedthereto. In addition, the auditor should consider whether, for greater clarity, it is appropriatewhen describing the scope to include a description of the approach followed to select matters forexamination.


Q U A L I T Y A S S U R A N C E

A set of professional standards does not aloneensure that auditors will in fact comply with them.What practitioners need is a method of assuringthat the standards are maintained in their audits.To this end, the various offices practising compre-hensive auditing have established regimes for quali-ty assurance. Such regimes comprise several ele-ments and document the safeguards that must beobserved at every stage of an audit.

A U D I T M A N U A L S

One element of quality assurance regimes isthe preparation and use of audit manuals, whichdetail how practice in the office will be conducted.Typically, these manuals are based on the variousdocuments issued by CICA respecting auditing,but provide more detail and are designed with thenature, mandate and circumstances of the individ-ual office in mind.

Although they vary, audit manuals usuallycover, amongst other things, the following subjects:

• governing legislation and office mandate;• the ethical standards expected of auditors (for

example, the Code of Ethics for InternalAuditors in the federal government280 that, inturn, were adapted from the Code of Ethicsof the Institute of Internal Auditors);

• office organization, with the responsibilitiesof each area and level described in detail;

• an outline of the steps to be taken in anaudit, and the procedures to adopt;

• the actions that should be taken at each stepto ensure quality, including the requiredsupervision and review;

• the audit methodologies and techniques to beused in various circumstances;

• advice on issues commonly faced by auditors,such as determining appropriate criteria, col-lecting and weighing evidence, and preparing

and presenting reports;• requirements for documentation of audit work;• procedures for dealing with clients and other

organizations; and• policy and procedures for such issues as con-

fidentiality of auditee and office informa-tion, reliance on and coordination withinternal audit and communicating with thepublic.

As offices gain experience with comprehensiveauditing, audit manuals tend to be modified andexpanded.

However adherence to the policies and proce-dures of even the most elaborate manual does notensure competent audits. Comprehensive auditingis an extended exercise in using professional judg-ment. It is conceivable that an otherwise impecca-ble audit could contain such errors of judgmentthat the result would be invalid and an embarrass-ment to the office.

It is for this reason that offices have incorpo-rated checks in their procedures to subject individ-ual audits to peer or other review processes as theyprogress.

R E V I E W P R O C E S S E S

SupervisionOffices ensure that their audits are led and

supervised by experienced auditors. The require-ments of supervision are outlined in some detail inthe chapter 24. This supervision is intended toensure that all major decisions and judgments pro-posed by audit team members are reviewed bysomeone who can subject them to rigorous chal-lenge. Team leaders themselves are supervised bytheir seniors, whose responsibility it is to ensurethat audits under their jurisdiction are properlyconducted.


Peer reviewIn most offices, it is common to have senior

auditors review the plans, evidence collected andconclusions of their colleagues. This provides anopportunity for team leaders to have their judg-ments tested against the collective experience andwisdom of the office. Another benefit of this kindof review is that it keeps those who are doing thereview current with what is happening in otherparts of the office and can contribute to uniformityand improvement in practice through sharing ofinnovative approaches and successful experiences.

Another form of peer review involves auditorsfrom other offices reviewing work in relation tospecific audits or perhaps examining the functionas a whole. This practice might also be seen as aform of independent review.

Independent reviewsSome offices establish audit advisory commit-

tees for individual audits. Typically, these commit-tees are composed of people from inside and out-side the office, with the outsiders bringing specificknowledge or expertise in some vital aspect of theaudit. The OAG of Canada Manual identifies thefollowing three purposes served by the advisorycommittees it requires for all audits except attestaudits and follow-ups to audits:

• Advisory committees help the Office take aconsistent, high-quality approach to its work,by providing advice, guidance and counsel tothe audit Principal and the responsibleAssistant Auditor General.

• The committee is designed to provide a forumin which the auditor can present plans, find-ings, conclusions and recommendations, anddiscuss difficult, ambiguous or contentiousissues and alternative reporting strategies.

• Individual members of an advisory committeewho have specialized knowledge in a particu-lar area may also act as special advisers to the

audit Principal and AAG, on matters relatedto their respective areas of expertise.281

A subsequent paragraph of the same sectionof the Manual identifies when committee adviceshould be sought:

The Assistant Auditor General should obtain theadvice of the committee members:

• before implementation of the survey plan;• at the end of the survey, to review the survey

report and the proposed examination planincluding the scope, approach and criteria forthe audit;

• at the end of the examination phase, toreview proposed observations, findings andreporting strategy; and

• when drafting the audit report with particularemphasis on the conclusions and recommen-dations.

The Manual makes it clear that advice maybe sought from the committee or individual mem-bers at other times if required.

Of course, not all offices use advisory com-mittees to the extent that the OAG of Canadadoes, but where they are used they are extremelyhelpful in assuring effective, high-quality audits.

Practice reviewsThe internal practice review within practi-

tioners’ offices is one method that has been used toconduct sample reviews of audits after they havebeen completed.

In the Office of the Auditor General ofCanada, practice reviews seek to provide assuranceto the auditor general and executive committeeabout how well the office is carrying out itsresponsibilities. Such reviews would, for example,assess audit quality, information on audit-objectiveachievement and reasonableness of audit cost, and


compliance with office and professional standards.The practice review teams also provide audit

teams with constructive suggestions for improvingthe economy, efficiency and effectiveness of the lat-ter’s work.

S TA F F I N G

One thing all offices doing comprehensiveauditing agree on is the need to have competentauditors available to manage and conduct audits. Thequalities they look for are outlined in chapter 24.

Comprehensive auditing cannot be masteredsolely by academic knowledge. Practitioners andpotential practitioners typically have respectableacademic credentials; what they need to becomeproficient at is on-the-job experience and trainingspecifically designed for comprehensive auditing.An emphasis on professional development charac-terizes the offices doing this work. Participation intraining courses is active.

All offices doing comprehensive auditing givecareful consideration to their internal organization.They recognize that knowledge of the auditedorganization is essential for successful auditing, andmost offices are organized so that auditors concen-trate in certain areas (such as on a group of govern-ment departments) so that the time they need togain knowledge of the organization is minimizedand their understanding of the problems and con-straints they are apt to encounter is enhanced.Such organizational arrangements help give assur-ance that the quality of the audits will remain high.

These, then, are the main methods that com-prehensive auditors and their offices apply in orderto ensure that the quality of the work meets stan-dards that bring satisfaction to their clients andcredit to the profession. It can be expected that,over time, there may be further developments inthese regards as comprehensive auditing itself triesto give its clients even more value.


280 TREASURY BOARD OF CANADA, TREASURY BOARD MANUAL—REVIEW, INTERNAL AUDIT, EVALUATION (TREASURY BOARD OF CANADA, MAY 1994) 36-37.281 OAG MANUAL, §6031.03-.05.

C H A P T E R 2 4

PROFILE OF THECOMPREHENSIVEAUDITOR

No audit is better than the people who con-duct it. In a discipline like comprehensive auditing,where so much is expected, it is essential that prac-titioners enjoy the requisite competence to do thework satisfactorily. Sloppy or unsatisfactory workbrings opprobrium not only to those directlyinvolved, but also to the auditors’ professions. Asmentioned in the previous chapter, since the prac-tice of comprehensive auditing is relatively young,particularly in comparison to financial statementauditing, it is important that client acceptance befostered, and that comes through benefiting fromproficient audits. Professional competence, there-fore, is a matter of vital concern.

The need for this competence is clearly recog-nized in the profession. The CICA Standards havethis to say:

The person or persons carrying out theexamination should possess or collectivelypossess the knowledge and competence neces-sary to fulfill the requirements of the particu-lar audit.282

A somewhat similar policy is adopted by theOffice of the Auditor General of Canada:

The audit Principal, in consultationwith the Assistant Auditor General, shouldensure that the team conducting an audit hasthe appropriate disciplines, skills, and experi-ence levels necessary for the assignment.283

The IIA standards in this area are also similar:Internal auditors should possess the

knowledge, skills and disciplines essential tothe performance of internal audits.284

Comprehensive auditing requires such a widevariety of subjects in such a broad range of envi-ronments that no one individual could accrue allthe skills, knowledge and experience required forall comprehensive audits. Indeed, the issue of qual-ifications has interested practitioners since theyfirst began to conduct comprehensive audits. Inconjunction with its professional development pro-gram, in 1983 CCAF asked a group of comprehen-sive audit practitioners to consider the skillsrequired for this kind of work. As a result of thatgroup’s deliberations, the foundation publishedKnowledge Requirements for ComprehensiveAuditing: A Practitioner’s Guide in 1984. This wasconsidered a preliminary document, subject toreconsideration as more and more experience isgained in the discipline. At the time of writing, amajor review of this subject is being undertaken.The material presented in this chapter relies heavilyon the 1984 publication, and readers should recog-nize that it is more indicative than authoritative.

What skills are required to conduct compre-hensive audits? First, it is helpful to make somedistinctions. One is between people who regularlyconduct comprehensive audits and others who areoccasionally involved in this work. Individuals inthe former group are auditors; those in the lattergroup are usually technical specialists who arebrought in for specific audits where their expertiseis necessary for a competent, thorough, compre-hensive audit. A second distinction is betweenthose who lead the audits and those who workunder the leader’s supervision. A third distinctionis between the skills required by individual audi-tors and the collective skills of the comprehensiveaudit team.

P A R T I I I . C H A P T E R 2 4 . P R O F I L E O F T H E C O M P R E H E N S I V E A U D I T O R 3 4 3

What is needed in someone who leads a com-prehensive audit?

C O M P R E H E N S I V E A U D I T

L E A D E R S

The audit team leader is responsible for thequality and cost of the work undertaken. To fulfillthese responsibilities, leaders have to bring a com-bination of diverse qualities to the assignment.They must have an understanding of a wide rangeof management practices and a sensitivity to theirpractical application in specific circumstances.Moreover, they must be able to use, but not bedominated by, the technical and specialist skills ofmembers of their team, and be able to foster theunderstanding and cooperation of managementthroughout the process.

Among other things, team leaders are calledupon to make informed judgments about:

• the importance of the component elements ofthe assignment—without a basic grasp of theconcepts of each element, a practitioner maynot devote sufficient resources to some. Forexample, without a basic understanding ofhuman resource management, inefficiencies inan organization’s use of staff may not be rec-ognized as an area warranting investigation;

• the need for and alternative sources of reliablespecialized staff support—practitioners mustknow enough about a subject area to recog-nize when they are not qualified to deal effec-tively with the issues identified. The leadermust be aware of available sources of expertiseto meet this need; and

• the need to manage and assess the quality ofthe contribution of any specialized advisers orstaff support that may be obtained—theymust be in control of the work done by a spe-cialist. They must have a level of knowledgesufficient to enable them to interpret the spe-

cialist’s work in the context of the overallaudit, and to report the specialist’s findings tomanagement.

In order to make these judgments, audit teamleaders must have a sound grasp of the six follow-ing general subject areas.

CO M P R E H E N S I V E A U D I T I N G C O N C E P T S A N D

P R AC T I C E S

Knowledge of the history of comprehensiveauditing in Canada and elsewhere, the concept, itscharacteristics and limitations, the role, servicesand publications of CCAF, and CICA’s Value-for-Money Auditing Standards, and generally acceptedauditing standards allows practitioners to under-stand this concept’s appropriate application and toexplain it to clients. It provides an understandingof the way in which other audit concepts relate tocomprehensive auditing and the standards of pro-fessional competence, independence, objectivityand fairness required.

G OV E R N A N C E S T R U C T U R E S

For practitioners to apply the concept ofcomprehensive auditing within the context oftheir assignment it is necessary for them to havean understanding of the law and practices of thesector in which the audit entity operates, its orga-nization structure, services and/or products and itsinformation needs and flows. This understandingprovides the knowledge necessary to assess thecompleteness, relevance, timeliness and accuracyof information made available to the governingbody and its impact on accountability issues.Knowledge of the relevant statutes and regulationsensures that the audit effort is directed to mattersthat are genuinely within the competence of theorganization.


O R G A N I Z AT I O N A L E F F E C T I V E N E S S

Team leaders need a sound knowledge of orga-nization and accountability structures, programs,management processes and procedures. Thisincludes a knowledge of behavioural science theoryrespecting organizational behaviour and the impactof working environments on managers, employeesand the entire organization. This knowledge allowspractitioners to choose the most appropriate ways ofassessing the organizational structure and reportingrelationships within the organization. Concerningthe organization itself, leaders must understand:

• its relationship with other organizations orfunctions;

• its accountability relationships;• its procedures and processes for management;• the sources and nature of any constraints that

may affect “due regard”; and• its services, outputs and effects.

M A N AG E M E N T F U N C T I O N S A N D S Y S T E M S

Knowledge of what constitutes good practiceis essential for auditors to assess the systems andpractices used by the organization and to judgewhether they contribute to good accountabilityand to a due regard for economy, efficiency andeffectiveness. Among the several practices affectinginformation management, human, physical andfinancial resource management are the following:

• forecasting—including economic, demo-graphic and market forecasting;

• planning—including the generally acceptedconventions of strategic planning, programand budget planning, and various resourceallocation techniques;

• operations control—including process con-trols and project controls for major capital,research and administrative projects;

• human resource management—including therespective roles of line managers and the per-

sonnel function in accountability and deci-sion making in respect of: manpower plan-ning, staffing and career planning, trainingand development, appraising performance,directing, motivating and sanctioning staff,job evaluation, classification and compensa-tion and staff relations;

• the role of the financial function in account-ability and decision making;

• the form and content of budget documents;• budgetary control and financial reporting;• accounting and control of revenue, expendi-

ture and asset transactions;• asset, cash, property, material, supply and

information management;• EDP resources management;• information sources—including those of polit-

ical, management, physical and social sciences;• mechanisms for assembling, storing, analyz-

ing and accessing information; and• analytical techniques—including financial,

statistical and quantitative analysis.

A P P R A I S A L P R O C E S S E S A N D C O N T R O L S

Practitioners must have a knowledge of howpeer review programs, internal audit programs andother systematic assessments and analyses of organi-zational and program performance operate. With thisknowledge, they can better judge in which circum-stances each of these methodologies is likely to beuseful and the level of effort required to apply them.

A U D I T P R O C E S S

Audit leaders must be well versed in the auditprocess. They must be able to do, and oversee, theplanning, conducting and reporting of fieldworkthat meets the relevant professional standards. Thevarious aspects of the audit process include:


• audit planning;• understanding the organization;• understanding and evaluating systems, results

and value for money;• sources and use of audit criteria;• audit evidence, including a wide range of col-

lection and analysis techniques; and • audit reporting.

Leaders should know how to select and effi-ciently apply all the available tools and techniquesfor assembling and analyzing information into alogical, comprehensive, constructive report thataddresses accountability and due regard for valuefor money.

E T H I C S

Audit leaders should be imbued with theethics required of comprehensive auditors. Honesty,fairness, objectivity, independence and compliancewith the law are necessary values, and these shouldbe combined with a sensitive understanding of theneed for confidentiality of certain information.Because members of comprehensive audit teamsoften come from disparate disciplines, audit leadersshould be able to convey and exemplify appropriateethical standards to other team members.

M A N AG I N G T H E P E O P L E D Y N A M I C S

Much more than financial statement auditing,comprehensive auditing calls for highly developedskills in managing human relationships. There aretwo dimensions to this: dealing with clients, andassembling and managing the audit team.

Dealing with clientsIn some instances, members of governing

bodies and their managers will have a relativelyscanty understanding of comprehensive audit.Indeed, they may have none at all. Whether the

auditors’ mandate for this type of audit is legislatedor not, they should explain to both the governingbody and management the overall concept, itsapproaches, processes and benefits. They mustensure realistic expectations. In addition, theyshould explain the concept of management repre-sentations on effectiveness and, where the govern-ing body decides to proceed with it, assist in thedesign and initiation of implementation strategies.

Particularly where there is no legislated man-date for comprehensive auditing, auditors act asadvocates, advisers and educators to appropriatedegrees. Ideally, they will participate with the gov-erning body and management in deciding whatwill be audited, and (as discussed in chapter 18)explain the implications—both for costs and theultimate report—of decisions about the optionsavailable for scope, significance and degree of auditassurance.

Auditors should encourage clients to participatein the decisions regarding these matters. The greatertheir participation, the greater the chance the auditwill meet their needs and the less chance of disap-pointment arising from unrealistic expectations.

Frequently, management of an organizationthat has little or no experience of comprehensiveauditing will be sceptical, anxious or hostile to theprocess. But experience has shown that the bestresults come when there is full cooperationbetween the auditors and those who manage whatis being audited. It is important, therefore, thatauditors possess the interpersonal skills to gain thenecessary cooperation and to overcome whateverinitial reluctance they find in an organization.

Managing the audit teamAudit team leaders must be just that: leaders.

Their first responsibility is to ensure that all therelevant expertise needed for a specific audit isavailable to, or represented in, the audit team.Some legislative audit offices have professionals


from other disciplines than audit on staff; someprivate audit firms have associated consulting orother units from which nonfinancial expertise canbe drawn. In some instances, it is necessary to con-tract out for the expertise required by an audit.Audit team leaders must satisfy themselves thatteam members from other disciplines truly havethe expertise required. The cost of recruiting thisexpertise must, of course, be taken into account.

The supervision of what is so often a multidisci-plinary team brings challenges the leader must meet.Sometimes the experts from other disciplines may besenior in rank to the leader, a situation that calls forcareful handling. More junior members of the teamalso require supervision. The OAG of CanadaManual says the following about supervision:

Policy requires that audit team membersbe properly supervised. The audit Principal isresponsible for the overall direction, co-ordi-nation and control of all audit work. Projectleaders should monitor all individual tasksand provide support to all team members asrequired.

Supervision of audit work is an essentialand continuous process. It starts with plan-ning and giving clear directions to teammembers about their work, seeing that thework is done, and evaluating the significanceof problems encountered. Supervision shouldensure that:

• audit work conforms to policy;• the audit approach and procedures adopted

are effective and properly carried out;• only essential work is performed;• the audit evidence gathered is sufficient and

appropriate to support and sustain audit con-clusions;

• audit work is appropriately documented inworking papers;

• audit budgets, timetables and schedules aremet;

• personnel with the correct mix of skills areproperly assigned to audit projects;

• audit facts and findings are reviewed withappropriate levels of management in theaudited organization; and

• matters included in audit chapters are signifi-cant, factual, fair and supported by sufficientand appropriate audit evidence.285

Clearly, substantial skills are required in order toaccomplish all these supervisory tasks successfully.

In addition to all the above, audit team lead-ers must ensure that their audits are properlycoordinated with any other audits the office isconducting simultaneously. Any duplication ofaudit effort is not only costly for the audit organi-zation, but can be unnecessarily bothersome anddisruptive to auditees.

T H E A U D I T T E A M

The audit team that the leader supervisesshould have all the necessary capabilities to con-duct a successful comprehensive audit. When thesubject under examination encompasses technicalmatters not within the usual areas of expertise offinancial auditors, the necessary professional assis-tance of other disciplines is required. Multidiscip-linary teams are as much the rule as the exceptionin comprehensive audits.

Often, individual auditors have developedexpertise in a variety of relevant areas, such as EDPor human resource management, that are frequentlyrequired in audits. Where such specialized knowledgeexists in-house and is called for in a specific audit, anoffice will not have to engage outside expertise.Where it does not exist, the team may have to besupplemented by one or more outside experts.

Technical, nonaudit experts may be used atany stage in an audit. They can contribute to theidentification of potential trouble areas in the plan-


ning phase, to the identification of audit criteria,to the design of audit programs to obtain relevantevidence, to the analysis of evidence and to thedrawing of conclusions and framing of recommen-dations arising from them.

As with the team leader, all team membersshould have an understanding of the organizationunder examination, and understand the concept ofcomprehensive auditing and its implications. Theyshould also have a good knowledge of:

• relevant audit techniques;• the standards that must be adhered to in

comprehensive audits;• accounting principles to understand costing

processes; and• the means of gathering and analysis of evidence.

In addition, all team members must embracethe attitudes, values and ethics required of auditors.


282 CICA STANDARDS, §.10.283 OAG MANUAL, §2601.01.284 IIA STANDARDS, 13.285 OAG MANUAL, §1507.12-.13.

CONCLUSION

Although a relatively new concept, compre-hensive auditing has been widely accepted in theCanadian public sector as a valuable means toachieve better accountability and improvedadministration and governance. Initially intro-duced in federal legislation in the 1970s, this kindof auditing has now been adopted in virtually allprovincial governments, many medium-to-largemunicipalities and a number of health care, educa-tional and social service institutions. Althoughdeveloped in the public sector, the concept is find-ing increasing interest and application in the pri-vate sector, as well.

The concern with achieving better account-ability and performance is not restricted toCanada; other countries are demonstrating thesame strong interest. In the United States, forexample, the General Accounting Office has intro-duced performance auditing, the U.S. governmentterm for comprehensive auditing. In Britain, theprocess has been mandated for local governmentsfor many years under the term value-for-moneyauditing. Most developed countries, and many inthe developing world, are taking similar initiatives.

Precedents for auditors reporting on perfor-mance issues stretch back in history. For decades,auditors reported instances where they found thatmistakes were made, money was wasted, and thelike. These instances were simply matters theauditors came across in the course of their finan-cial audits. Comprehensive auditing represents astep beyond this: a systematic approach to theanalysis of whether management is paying dueregard to matters of economy, efficiency andeffectiveness in the administration of theresources entrusted to them.

Comprehensive auditing is not a static con-cept. In fact, there are three very different modelsfor comprehensive auditing, each of whichapproaches the issues of accountability and econo-my, efficiency and effectiveness in different ways.

One model for comprehensive auditing focus-es on the quality of management systems and prac-tices as an indicator of the extent to which theorganization pays due regard to economy, efficien-cy and effectiveness. In the second model, auditconcentrates on providing assurance on reports bymanagement to the governing body regarding theoverall performance or effectiveness of the organi-zation, as viewed, for example, through the prismof the twelve attributes framework discussed inPart II. Still another model is for audit itself toattempt to provide the performance report.

When comprehensive auditing was first intro-duced, it adopted the management systems andpractices model; this is still the most prevalentapproach today. In these audits, the assessments ofauditors are guided by specific criteria, and deter-mine, as objectively as possible, how well thoseindicators of good performance are being met.Where good performance as well as deficiencies arereported, this has done much to alleviate manage-ment’s concerns. No management group likes tothink that auditors will comment solely on prob-lems and deficiencies. Moreover, the fact that a rig-orous set of criteria are being applied to the processis comforting to management, and to the govern-ing bodies that commission the audits.

Nonetheless, there are limitations to thisapproach. For instance, this type of auditing is onlya proxy for an assessment of an organization’s per-formance since it focuses on management systemsand practices, not directly on the performance ofthe organization. It is one thing for an audit reportto say that management has or has not followedcertain reasonable criteria in terms of a manage-ment system, but it is a more difficult matter to

P A R T I I I . C O N C L U S I O N 3 4 9

know what such a finding really means in terms ofthe bottom-line performance of the organization.

This brings us to the second model—auditattestation to management reports on performanceor effectiveness. Historically, the lack of manage-ment reporting on performance has meant that thisapproach could not be used much. And, as men-tioned earlier, one of the main reasons that man-agement didn’t report much was that there was noagreed framework within which to do so. Such aframework has now been developed; managementis beginning to use it in reporting, and auditors arebeginning to issue audit opinions on that report-ing. Experimentation is ongoing with this form ofcomprehensive auditing.

The attestation model offers several key chal-lenges to auditors. Practitioners need to be able tomaintain a careful balance between preserving pro-fessional independence and avoiding isolation fromthe whole process. They also have to assess infor-mation that contains varying degrees of uncertaintyand imprecision. In addition, this model presents aprofessional development challenge to auditors toincrease their knowledge and expertise in relationto the much broader range of concepts involvedand to the methodologies associated with theirmeasurement and analysis.

All this requires that auditors work closelywith the governing body and management todefine an audit approach that is rigorous, yet onethat also remains flexible enough to accommodateand support the learning curve that auditors, gov-erning body and management must experience.

In the third audit model, the auditors arecommissioned by the governing body to examinethe performance of the organization directly, possi-bly applying the twelve attributes framework.

This approach would be used where, for onereason or another, management is not providingthis information in a sufficiently rigorous fashionand where members of the governing body feel

that an audit focus on systems and practices isinsufficient to meet their due diligence needs.Although there is provision for it in some legisla-tion, historically this model has not been used verymuch in Canada. Recently, however, some legisla-tive audit offices have moved towards what iscalled “results-based auditing.”

An approach that would see the auditorexamining the performance of the organizationdirectly would at least partially compensate for thelack of effectiveness information provided by man-agement and, certainly, would go beyond the limi-tations of systems and procedures-based audits.But, it could have its own pitfalls—principally thatauditors might become too involved in mattersrelating to policy determination.

Nonetheless, if management does not fulfillits responsibility to report on its performance, it isnot reasonable to expect that the governing bodywill simply decide to go without. They need thatinformation to exercise necessary oversight, toassess alternatives and make decisions, and then tobe able to explain these decisions to their con-stituencies. In these circumstances, governing bod-ies could very well go to others for this informa-tion, and their auditors would be prime candidatesfor this function.

Whichever model is right for a particularorganization will depend on a number of factors.The state of the organization’s systems and prac-tices, the nature of management’s commitment todemonstrating their accountability and the organi-zation’s track record are a few of these considera-tions. The main determinant of which model ismost appropriate, however, lies in the quality ofthe accountability relationship between seniormanagement and its governing body—that is, theextent to which the governing body is informedabout key aspects of the organization’s effective-ness, the methods by which it is informed and theway in which it uses the information it receives.


The choice of which audit model to adoptshould be based on the individual circumstancesand needs of the particular organization. What isimportant is that this decision not be made in avacuum. It ought to be the product of informeddiscussion between the governing body and man-agement, with the auditors also involved to helpensure that the two parties have sufficient back-ground and information about each of the poten-tial audit model choices.

Whichever approach they are using, compre-hensive auditors must conduct themselves in accor-dance with the standards of their profession. Thedemands on practitioners of this kind of auditingare substantial, and the true benefits of this workcan only be achieved through adherence to thehighest professional principles and practices.

Comprehensive auditing is now an acceptedpart of public administration in Canada. It hasalready demonstrated that its conceptual basis issufficiently strong and flexible to meet emergingchallenges and to adapt to varying circumstances.It can be expected in the future to continue itscontribution to good accountability, managementand governance.

P A R T I I I . C O N C L U S I O N 3 5 1

INDEX

A

Abella, Rosalie, 38Acceptance (attribute), 141-42,182Accountability

administrative, 62answerability, 47,49,61,63,66-67

attitude, 47-53,54-55inappropriate, 55-56

auditor, 40,80-83,100,119,224,273,343-46autonomy, 56

of auditor, 346citizenry, 24corporate governance, 28Canadian government. See Canada, government ofconcepts, 3,44-49consequential, 58context for, 3,4control, 4,20definitions, 44-45,47,115demonstrating, 50dilemmas, 5effectiveness, 4,45,47,58,59,62,81,120,136,

156-57,193factors, 54-56,193private sector, 71,193public sector, 71,193

external, 57,63,73inducement, 50-51

financial, 62goals & objectives, 47,80-81governance, 3,4,8-10,14-26,28,37,55,59,74,

80-83,136,344,351inconsistency of time, 34institution(s), 45internal, 57,63,73language, 33

legal, 58Bhatnager case, 67-68Westland case, 68-69

legislating, 52-53lobbyists, 23managerial, 3,9,47,52,59,80-81,90-92,106,273

commercial, 57control(s), 75-79professional, 58resource, 57

measuring, 45media, 24nonprofit organizations, 73objectives, 48-49performance, 47,50,80

monopoly, 73private sector, 713 Es, 117-18

partnership with governing body, 98-99,140, 142,143-44,151,232

political, 57,62positive democratic value, 44private sector, 71-74,80-83,97procedures, 58public sector, 71-72,80-83,97rendering an account, 44,46,48,55,80,97

external, 57,116,132internal, 57

reporting, 97-98auditor role, 156-57,273financial performance, 98information, 273theory & practice, 97

representations, 132-33,147responsibility, 45,47-48,80-81

control(s), 75rewards & sanctions, 44,50-52social contract, 12subjective, 48-49systems, 45teamwork approach to, 56

I N D E X 3 5 2

theory, 4,44-49traditional view, 62type of, 45visible, 25

Accountability checkup, 163-65Achievement of intended results (attribute), 141,181 Act to Amend the Financial Administration Act.

See Bill C-24AIF, 109Alberta, 114

auditor mandates, 227Anderson, R. J., 222,223Annual information form. See AIFAnnual reports, 114,160Appropriateness (attribute), 140-41,181Appropriation Act, 106Assistant Auditor General of British Columbia, 53Atkinson, Michael, 10Attestation. See Comprehensive audit, attestation Audit

accounting, 222,223,224assessing risk, 80attestation, 156,223,231,236broad-scope, 227,228characteristics & prerequisites, 224-25committee, 37,320effectiveness, 80,136,147-48

reports, 192examinations, 239,240financial, 219,229

accounting, 222attest, 238,246criteria & suitability in, 288statement, 220,224,238,246

findings, 256-66GAAS, 225head, 37internal, 37,97,100,219,298-311,317,318,325legislative, 106,120,137,227-29,231,232,242,

270,280of management representations, 97,163

new techniques for, 249nonaudit services, 232-33performance reporting, 80,97,143,162

auditor role, 156-57,1933 Es, 219

plan, 83,162,320preparation, 81-83process, 80-81,345-46& quality performance, 219recommendations, 37,266report to governing body, 219& representations,schedule & timing, 148,162role, 223true, 230-32validation mechanism, 194See also Comprehensive audit

Audit Commission, 111,112Auditor(s)

accountability, 80-83,100intended degree of audit assurance, 246

attestation on representations, 23CCAF

review of effectiveness attributes, 138determining mandate of audit entity, 252

due regard to 3 Es, 234,236,275external, 97,220,302,312-14

assessment of internal audit, 315-16,317, 318,322,323

reporting on reliance, 324-25GAAP standards, 223independence, 37,225,268,279internal, 37,227,300-1,302-3,304,306-7,309,

313-14,325legislative, 106,120,137,227,228-29,231

comprehensive audit, 227,237due regard to 3 Es, 237

reporting, 231direct, 236information on 3 Es, 242

levels of assurance, 239-41,246-47,275performance reporting, 100,226


preparation, 81-83professional judgment, 239,322-24recommendations, 235

Public Accounts Report, 106reporting instances, 275-76to governing body, 236

Report to the Independent Review Committee of the Office of the Auditor General of Canada. See Wilson Committee

role, 156effectiveness reporting, 156-57formal audit component, 157

value systems, 81Auditor General Act, 226Auditor General of Canada, 67,106,117,119,136

auditor profile, 343Comprehensive Auditing Manual, 118-19,264,313on performance excellence, 127-28role in developing comprehensive auditing, 226report(s), 40,123

on full-cost accounting, 142Standing Committee on National Finance, 119Wilson Committee report, 231

Austin, Nancy, 127Australia, 114Autonomy, 39-40

accountability, 56boards & regulating agencies, 40,56subsidiarity, 56

Axworthy, Lloyd, 25

B

Baldridge Award, 128Bhatnager case, 67-68Bill C-24, 107BNA Act, 16Board(s) of directors

management objectives, 10responsibilities, 9,31stewardship, 31

British Chartered Institute of Finance and Accounting. See CIPFA

British Columbia, 53,329-30British North America Act. See BNA ActBrussels, 40Budget, 105,106,143Bureaucracy, 18

as technocracy, 35bureaucratization, 35British parliament, 36Canadian government, 19

independence in, 36-37,38empowered model, 77government administration, 35political neutrality, 39 private sector, 35street-level, 77-79

policy-making, 78,79Burke, Edmund, 21

C

Campbell, J. P., 126Canada Assistance Plan, 106Canada Awards for Business Excellence, 128-29Canadian Bar Association, 37CCAF, 64

applied research projects, 159,162assisting facilitators, 172benefits for private sector, 227comprehensive auditing, 227executive presentations

preparation kit, 166responses to common questions, 168-69,

170-71professional development program, 343public sector organizations, 227

reporting effectiveness in, 136,159publications, 172,177,343-44reporting & auditing, 136-44,231

I N D E X 3 5 4

Canada, government ofaccountability & governance, 4,9,16,17,18,20,

25,37,85,114agencies, 18,19,114citizen rights, 75Crown corporations, 20,114employee rights, 75Increased Ministerial Authority &

Accountability, 85-92parliament, 44,61,66-67,114judicial, 20

annual report, 160auditor(s), 19,349bureaucracy, 19,20,36,84,cabinet & committees of, 18Commonwealth status, 39Confederation, 62,66conflict of interest guidelines, 18controls, 5,27,37

regulatory agencies, 37Crown, 15,61departments, 19federal court, 20federalism, 25-26governance in, 15,16 House of Commons, 16,17,19,61,64,106,114internal auditing standards, 37jurisdiction

federal & provincial, 19,20-21,84lobbyists, 23management control, 41,84,85managerial culture, 84new expenditure management system, 92

business plans, 92outlooks, 92

ombudsman, 19organizational chart of, 18performance reporting, 105

information, 226prime minister, 16,18privacy commissioner, 19

Privy Council, 16,61,63-64,66Clerk (office of ), 86Osbaldeston report, 63-64

public servants, 19-20accountability, 20anonymity commission, 18,20,69

unions, 19Senate, 16-17shared management agenda, 89-90statutory right of access to company records, 223Supreme Court. See Supreme Court of CanadaTreasury Board, 84,86,89,90,301

IIA standards, 301Canadian Comprehensive Auditing Foundation.

See CCAF Canadian Institute of Chartered Accountants.

See CICACanadian Radio-television & Telecommunications

Commission. See CRTCCEO(s), 28-29

brief to governing body, 186sample, 210-12

chairman, role of, 29corporate directors, 28-29& effectiveness framework, 154-55,163establishing ad hoc committees, 186executive presentations, 166

context of, 166-67implementation process, 172agenda, 173

& internal audit, 303issues facing, 29,105liability, 29methods of selecting, 29nonperformance, 29pretabling discussion questions, 191refining reports, 162,191representations, 147,190responsibilities, 29-30self-diagnostic checklist, 164-66


sponsoring organizational workshops, 172-75Certified General Accountants of Canada (CGCA), 227Charlottetown Accord, 26Charter(s), 10

citizen, 10-11informal, 12See also Constitution

Chief Executive Officer. See CEO(s)CICA, 75,225,227,232,234,332,340,343

auditing committee, 232,332GAAS, 227,287,291Handbook, 272research on reporting, 109Value-For-Money Auditing Standards, 234,267,

268,271,332-33,338,344scope of audit criteria, 274

CIPFA, 118Clinton, Bill, 41Commission des valeurs mobilières du Québec, 109Commission to Study the Public’s Expectations of

Audits. See Macdonald CommissionCommittee on Corporate Governance in Canada,

30-31Comprehensive audit

accepting internal auditor’s recommendations, 306reliance factors, 306,312,314,316,325

assessing information, 249 factors in, 260-61for good management practice, 260risk, 260

assurance, 239attestation, 274auditor’s intended degree of, 246-47,271-72,

273-74,275-76communication clarity, 276exception-based report, 275-76implications of report, 272levels of, 241reporting instances, 275-76variables of, 247

attestation, 231

auditor’s opinion on representations, 231financial statement, 240

long-and short forms, 241model, 350reports, 274testing criteria, 62,273

audit enquiry, lines of, 254-55audit teams, 256

multidisciplinary, 343-44auditee’s objectives, 253

review of, 260strategies, 253-54

auditor profile, 343appraisal processes & controls, 345audit process, 346-47CCAF publications, 343-44knowledge factors, 344-45management functions & systems, 345OAG Manual, 347-48organizational effectiveness, 345role, 220

external, 220,227independence & objectivity, 279

skills, 343-44,347-48choosing subject for, 252community college, 326

table of, 329-30compliance, 227-28

Value-For-Money Auditing Standards, 232conduct, 244

continuous process, 245phases, 256

contribution to good accountability, governance& management, 220,232

coordinationinternal & external auditing, 320

cost, 229,247control by management, 261effectiveness of, 261identifying areas for examination, 260

conduct phase, 230,262

I N D E X 3 5 6

criminal/civil-legal proceedings, 279criteria, 286

acceptable to management, 263appropriate for examinations, 286,320connection to evidence, 296consulting with experts, 290developing, 289,320directives & guidelines, 290due regard to 3 Es, 235,256,277examination, 234,238,241,247,262,

272-74,281general purposes, 288Hansard, 257internal audit, 289

hospital, 289liquor commission, 289management practice, 287

outside expert, 256policy & procedural manuals, 257sources of, 289-90sub-criteria, 288suitability factors & characteristics, 287-88usefulness of, 288,320variance in findings, 265

cycle, 250defined, 228,232,286economic, efficient & effective management, 228engagement, 230,232environment, 253

factors affecting performance, 254evidence, 240

appropriate, 293,296collecting, 264-65competence of, 292,296concepts, 291,323connection to criteria, 296defined, 263-64,291factors in planning, 293

costs, 293,296effectiveness of approach, 293integration of other audits, 293

reporting requirements, 293,296model, 259gathering, 295-96poor internal audit performance, 324reliance on, 293-94standards, 291sufficiency of, 292-93

examinations, 239lines of audit enquiry, 254-55objectives of, 239-40special, 274

examples, 248exception-based reports, 273exit interview, 270factors influencing decisions, 277-96,322-23feasibility studies, 249final audit report, 230

discussion of structure, 267with governing body, 282

establishing timetables, 268fair & balanced reporting, 284-85management letters, 269-70,284point-form, 268preparation & presentation, 230,241preparing final draft, 268-69review by management, 267-68significance threshold, 281writing, 269

financialaccounting, 222records, 227reporting, 228

financial statement auditing, 220attestation, 274audit assurance, 246

findings, 265assessment of

reportability, 265significance factors, 265

cause & effect, 265-66


negative & positive, 265GAAP principles, 231,238history of & development, 226-27hospital, 326,327-28information gathering, 255

by internal auditors, 227,298-311meeting/interviewing personnel, 255-56models, 257-59physical observation/inspection, 255-56review & analysis of documentation, 257

involvement with management on representations, 238

legislation, 234management representations, 237

performance standards, 289management systems & practices, 236mandates, 106,236materiality, 260,321

definition, 321standards & significance, 321

models for, 235-36,257-59audit assurance on performance, 236,350charts, 258-59focusing on due regard to 3 Es, 235,349providing performance information to

governing body, 236,350nonfinancial performance, 227organization of audit entity, 253

management climate, 253principle resources, 253

organization-wide functions, 248attitude of management, 249control systems, 249performance, 236,350stability in, 249

planning phase, 229-30administrative issues, 230advisory committee, 252audit plan, 261,262evidence collection & analysis, 230intended level of assurance, 229-30,231

level of significance, 230nonaudit services, 232-33objectives, 252performance assessment, criteria for, 230quality of management systems & practices,

229scope, 229surveying, 252understanding the organization, 229

policy & guidelines manual, 264practitioners & independence, 220

professionalism of, 276& independence, 239

principles in, 226intended degree of assurance, 245scope, 245significance, 245

problems & issues, 240cost control, 261identifying inefficiencies, 260

indicators, 260-61labour negotiations, 261publicity, 261,279

procedures, 262relevant questions, 263testing criteria, 262

process, 228-29,245communicating auditor’s opinion, 244examination & analysis, 244flowcharts, 264,295

professional development & training for, 342program, 262

audit criteria, 262collecting evidence, 262management agreement, 263supporting observations of, 262

projects, 262writing detailed plans for, 262

quality of assurance, 332CICA, Value-For-Money Auditing Standards,

333-38

I N D E X 3 5 8

regimes for, 340-42recommendations, 235,247,266,282

by management letter, 284GAO standards, 282by internal auditors, 306merits of, 283-84purpose of, 284questions concerning, 267,282-84rationale for, 282-83survey report & audit plan, 261

record keeping, 264reliance, 306,312,317-19,321,322-24,326,331

reporting, 324-25report & survey plan, 255,319reporting issues and instances, 271-85

clarity, 272-74deficiencies, 275-76instances, 275intended degree of audit assurance, 271-74,

283scope, 271,274significance, 271unrealistic expectations, 283

restrictions, 227risk assessment, 249,260,296

abuse, 249financial error, 249fraud, 249management controls, 260recommendations for, 283

safeguarding valuables, 227specific units or programs, 248

attitude of management towards audit, 249control systems, 249significance, 248

standards, 220survey stage, 255,319

report & audit plan, 261-62systems and practices approach, 236

evaluation, 260limitations, 236

Comprehensive Auditing Manual, 119,264,313 Comprehensive auditor’s profile, 343-48

appraisal processes & controls, 345audit process, 346human relationships

dealing with clients, 346managing audit team, 346-47

knowledge of comprehensive auditing, 344good practice & accountability, 345governance structures, 344management functions & systems, 345organizational effectiveness, 345

professional development, 343skills

auditing, 343leadership, 344supervision, 347

OAG Manual, 348teamwork, 343,347

Constitution(s), 10-14bureaucracy, 35Canadian, 15,20-21division of powers, 25European community, 40Great Britain, 13ministerial responsibility, 66United States, 11,35Westminster model, 15written & unwritten, 13See also Charter(s)

Control(s)auditor, 239,345

cost of audit, 261cost to management, 261

CICA, 75design, 75effective, 80goals & objectives, 75government of Canada, 5IMAA, 91improper, 77


street-level bureaucrats, 77-79management administration of, 75management letters, 269-70mechanisms & instruments, 75nature of organization, 75operational, 4polar approaches, 4performance expectation, 75policy making, 79power of authority, 75responsibilities of governing body, 249rights of citizens & employees, 75standards, 75systems, 249

Coordinationagreement on audit criteria, 320developing trust, 312external & internal auditor, 312,315-16factors affecting degree of reliance, 313,316,

317-19,321,322-24improving effectiveness & auditing costs, 312-13problems, 313questions regarding audit group, 315sharing resources, 313timing considerations, 321

Corporate culture, 75-79,80,151See also Bureaucracy, management

Corporate governanceaccountability, 28-29,30board of directors, 28-29CEO, 28-29,30,31decisions, 27issues in, 27-29private sector, 4,9,14shareholders, 28,29,30TSE, 30-31

Costs & productivity (attribute), 142,182-83Covenants. See ChartersCriteria. See Comprehensive audit, criteriaCrown, 14,26,61,63

& broad-scope audits, 308

civil servant allegiance, 39Financial Administration Act, 274internal audit of, 325ministers of, 104sovereign authority, 15special examinations, 278statutory responsibilities, 309survey of internal audit units, 308

Crown corporations, 28,57,107,136,228comprehensive auditing in, 228,241

CRTC, 38,58

D

Democracyaccountability in, 45,48,58,71Canadian, 22characteristics, 13direct, 12parliamentary, 22participatory, 11,22,24media & free press, 24 polls & influence in, 24representative, 12social contract, 11,14

Discretionbureaucratization, 35,78conduct of government, 33decision making, 34,78erroneous decisions, 34implementation of rules & policies, 33ministerial, 34policies, 33powers of, 35preset rules, 33See also Rules

Due regard, 193-95Auditor General Act, 226comprehensive audit planning, 229decision making, 193ensuring information validity & fairness, 194

I N D E X 3 6 0

responsibilities & needs of stakeholders, 193timeliness, 193to 3 Es, 234,235-37, 256,277,284,349using audit information, 288

See also 3 Es, due regard; Comprehensive audit, effectiveness

E

Economy, 117,118,119,128,193,236,242,277, 284,349

Effectiveness, 98-101,116-19,120-28accountability, 130,138as performance, 98,116assessment, 120,123-24,127-29attributes, 125-27,137-44,150,159,

assessing, 180-85twelve, 98-99,190,237

Auditor General’s report, 127-28auditor role in, 156-57,345CCAF panel, 136CEO decisions, 163

accountability checkup,163conceptual problems, 129,136cost, 120defining, 120,121-22,125,136executive presentation, 166

CCAF kit for, 168-69,170-71& external facilitator, 157-58implementing framework, 152-54

strategy guidelines, 170-71,172-75,190-91information systems, 151-52innovations, 127key participants, 154-57management, 126,127-28managerial, 98,116,124,138McKinsey 7-S list of factors, 127operational, 116,120,124organizational, 98,120,124perspectives on, 124program, 98,116,120,124

project strategy, 158-59reporting, 145-49,150-54

final reports, 190-92,213-15framework, 125-26,128,138-39,162-92,242phases & steps, 163suggestions & guidelines, 162writer’s role, 159

timetable, 159value-added, 155Wilson Committee, 122See also Management, effectiveness; Performance,

effectivenessEfficiency, 109,113,117-119,121,128-129,137,236

due regard to, 242,277,284,349control systems, 249

Elected representative(s)dilemmas facing, 22-23issues & philosophy, 21-23& special interest groups, 22

Estimates & Part III Estimates, 105-6,148Ethical behaviour

accountability, 46,54-55,80-83& auditing criteria, 289auditor values & attitudes, 346,348media, 22

Europecommon market, 40technocracy, 35

Executive presentation(s)CCAF kit, 166,168-69& context of, 166-67 facilitator’s role, 166final reports, 191representations, 147sample agenda, 167See also Reporting, CEO’s

Export Development Corporation, 20


F

Facilitator(s)discussion papers, 185,203-8effectiveness, 157,172ideas & suggestions, 176,185& implementation workshop, 175,176,185-87

outline, 200-2sample agenda, 209

management representations, 187-89& organizational workshops, 172,173pretabling discussion questions, 191roles, 154-57,166-67trends & initiatives, 176writing reports, 158,176-79See also Effectiveness, key participants

Fair & balanced (audit) reportingdefining, 284-85duty to impartiality, 285

Federal Court of Canada, 20Federalism

Canada, 24-25Europe, 40

Financial Administration Act, 274,311Financial results (attribute), 143,183Financial statement(s)

audit,220,224,238,246,276 & disclosure, 109-10& reporting information, 130

Flaherty, David, 24France, 11Franks, C.E.S., 48

G

GAAP, 110,223,231,238audit criteria, 286-88

GAAS, 225,287GAO, 128

accounting standards, 282performance auditing, 349

due regard to 3 Es, 242GASB standards, 113Generally Accepted Accounting Principles. See GAAPGenerally Accepted Auditing Standards. See GAASGermany, 40Giametti, Bartlett A., 42Glassco Commission, 117Governance. See Accountability, governance Governing body, 98,140-44,146-48,227

& accountability reports, 232& committees

ad hoc, 186,191audit, 308-9,321

& role of auditor, 156-57,229-30& role of CEO, 154,163,186consulting with, 158,161control, 249decision-making timetable, 160-63& due diligence, 241-42expectations, 153,160,177,235involvement in reporting, 155,158-60management representations, 191-2,236questions for CEO, 191receiving audit reports, 146-49,236,238,269,

281,283,299relationship to management, 350

Government(s)ad hoc reviews, 107comprehensive auditing, 107democratic, 11,22,24,27dictatorial, 27good governance & accountability, 27,34,46,

138,140,142intergovernmental affairs, 25oral reporting, 104privatization, 72-73rules & discretion, 33-34totalitarian, 27Western, 39written reporting, 105-7

Governor General of Canada, 16

I N D E X 3 6 2

GPRA, 54accounting & reporting issues, 113efficiency & cost outcomes, 113

Great Britain administrative class, 41& audit commission, 111bureaucracy & accountability, 36,68-69Civil Service Commission, 39citizen’s charter, 12House of Commons, 68-69Labour government, 11parliament, 21,26,36social compact, 11United Kingdom’s Civil Service Pay &

Conditions Service Code, 39& value-for-money auditing, 349Westland case, 68-69

H

Hansard, 257Hierarchies, 78,91Hobbes, Thomas, 10Hospital(s), 154,159

accreditation surveys, 326bureaucracy & accountability, 36performance reports, 107,154responsibility & liability, 41review process, 327-28

I

IIA, 298,301internal auditor profile, 343

IMAAaccountability, 86,87,90annual management reports, 85checklist, 86control(s), 91-92& cyclical audits, 85& government-wide responsibility, 88

performance measurement, 85,87-88& Treasury Board of Canada, 85,89

Implementation Workshop Discussion Paper, 203-9Increased Ministerial Authority & Accountability.

See IMAAIndependence

agency, 36-38auditor, 36-37,225,268,279judiciary, 37neutrality, 38-40professional, 11,239

Independent Panel on Effectiveness Reporting & Auditing, 136-44

twelve attributes, 98-99,137,138-39,190,237, 277,350

Independent Review Committee on the Office of the Auditor General of Canada. See Wilson Committee

Industry Canada, 129Institute for Internal Auditors. See IIAIntended degree of audit assurance

& association, 272,275CICA Handbook, 272

definition, 245& levels of assurance 271,273,276& internal auditing, 311& scope, 275,276,

Internal audit, 37,97,100,219compliance with rules & regulations, 302,303-4committee role, 310& auditor skills, 302-3budgets, 304charter, 316costs & benefits, 303-5& Crown corporations, 325cycles & annual updates, 317definition, 298establishing credibility of, 305-6execution pattern, 308expanded scope pattern, 304,306-7failing to meet objectives in, 318


intended degree of assurance, 311IIA standards, 298links to comprehensive audit, 298,304objective, 303objectivity & independence in, 300-1,307-8organizational arrangements, 300-2policing authority & record keeping, 302-3products of, 299program evaluation, 312,325quality assurance, 317questions regarding, 315recommendations & reports, 303,305,310-11& reliance, 304,306-7,311responsibilities, 298& scope, 303performance issues, 306-7& senior management support, 300-1,303standards & good practice, 301,317structure of, 300-1team & leadership, 301-4,309value of, 303-4

Iraq, 65Israel, 26

J

Japan, 28Jefferson Thomas, 21Journalism. See Media

K

Kernaghan, Kenneth, 61

L

Lambert Commission, 63-64Legislation

accountability, 52-53,97,107,274budget, 106& governance structure, 10

scope of audit, 246See also specific legislation

Legislature(s)& accountability, 26,37,104forum for political debate, 104elected member accountability, 26Hansard, 257Legislative Assembly, 53& performance information, 226provincial, 21Public Accounts Committee, 106

Light, Paul C., 46Lobbyist(s), 23-24Locke, John, 10

M

Macauley, Robert, 37Macdonald Commission, 109MacDonald, Donald C., 38MacDonald, Flora, 38MacDonell, James J., 226McGrath Commission report, 64McKinsey 7-S (effectiveness) factors, 127Major, John, 11Malcom Baldridge National Quality Improvement Act.

See Baldridge AwardManagement

accepting audit reports, 235,267-8accountability, 100-1,104,137-38,151,154,157,

219,223,231,236,273,351& administration, 41,65,76attitude towards, 249-50,253,263,267audit for Crown corporations, 107,310controls, 75,79,144due regard to 3 Es, 349effectiveness, 98-101,116-19,120-28,136-37,

140,142,144,154-159executive, 219GAAP principles, 231goals, 121,122,146,180

I N D E X 3 6 4

good & acceptable practice, 287,351hierarchy, 91information & systems of, 80,151-52,154,193internal auditing, 299-303,305-7,310,316-19& leadership, 41,42letters, 269-70models, 76,79organizational workshop, 172-75,185partnership(s)

audit practitioners, 151,154audit team, 261,267governing body, 98-99,140,143-44,151,154,

161,236,350performance reporting, 100-1,104,107,141,

144,151-52,154-55,159-61,193financial accountability model, 226internal audits & evaluations, 257

planning process & risk identification, 260private sector, 98public sector, 41,97,136relationship to governing body, 236reporting, 150-53,163,166,172-75,230-31,237representations, 187-89review of attributes of effectiveness, 138review of final audit report, 267-69,281steering group, 186-89,190-91stewardship, 42,100systems, 235-6,260,345-46workplace & employee attitudes, 76,143,194,

236,346See also Accountability, managerial

Management direction (attribute), 139,180 Management discussion & analysis (MD&A) on

financial performance, 109Media

ethics, 22information reporting issues, 129-31journalists & press councils, 24ministerial responsibility to, 24,69-70polls & results, 23-24press releases, 110

reporting on accountability, 69-70& special interest groups, 22,24television reporting, 104

Meech Lake Accord, 126Ministerial responsibility

accountability, 62-64,65-67,104,232Bhatnager case, 67-68CCAF report on, 65concept of, 4constitutional principle, 66doctrine, 65Lambert Commission, 63-64McGrath Committee, 64media treatment, 69-70,104minister(s), 37performance reporting, 104-5prime minister & cabinet, 15,16,17,18Public Accounts Committee, 65Public Service 2000 report, 65Westland case, 68-69See also Canada, government of, accountability

& governance Monitoring & reporting (attribute), 144,185 Mitchell, Jim, 62

N

National Institute of Standards & Technologyeffectiveness framework, 128

National Quality Institute. See Canada Awards for Excellence

Neutralityindependence & accountability, 38political, 39public service, 39,62

Nongovernmental organizationseffectiveness, 107governance, 21performance information, 107-8responsibilities, 21

Nonprofit sector& comprehensive auditing mandates, 106-7


O

Office of the Attorney General (OAG)Manual, 347

Ontario agencies & tribunals, 38auditor’s report (1988), 273governance, 9,21-22government accountability, 37utility commissions, 21

Ontario Commission on Electoral Finance, 38Ontario Hydro, 20Ontario Labour Relations Board, 38Ontario Management Board (OMB), 54Ontario Securities Commission, 109Opinion polls, 23-24Oregon benchmark program, 113Organizational workshop, 196-99Osbaldeston, Gordon, 63-64

P

Parliamentaccountability, 44,56,63-64,114,

answerability, 45,65-66budget, 105independent agency tribunal, 38members, 25,136reporting documents, 105-6See also specific countries

Performance as effectiveness, 98,116audit assessment criteria, 288auditing, 349characteristics of good, 127-29,130-32management standards, 289management system, 193-95motivation to improve 3 Es, 117-19,129,151,

287,349program, 140-41representations, 132-34

Performance reportingassessing effectiveness, 98,127-28benchmark, 101,286budget, 105,143

Estimates & Part III Estimates, 105-6,148characteristics, 130-32,193concepts & issues, 98,116-34environmental factors, 254factors affecting, 254federal, 104-5hospital, 107improving, 110indicators & guidelines, 115information, 130-31,145-49,286by management, 98,104,137,289mechanisms, 100,104ministerial, 105nonprofit agencies, 107,115polls, 23-24provincial, 105public & private sector, 98,104substance & timing, 98

Peters, Thomas, 127Polls. See Opinion PollsPress councils. See MediaPrivate sector

audit committees, 37& comprehensive auditing, 349CCAF role in, 227disclosure, 109-10& legislation affecting accountability, 91,109management, 98performance reporting in, 109-11prospectuses, 110public sector, 290

Program delivery model(s), 258-59profile outline, 200-2structure model(s), 257-58

Protection of assets, (attribute), 144,184-85 Public Accounts Committee & auditor’s examination, 106

I N D E X 3 6 6

Public Accounts Part III Departmental Expenditure Plans, 160

Public sectoraccountability, 88-90,117,226annual reports, 160& anonymity, 67audit committees, 37CCAF role in, 227,231comprehensive audit, 237,290,349delegation of authority, 88effectiveness, 98-99,136empowerment, 88& extension of IMAA, 98-99Glassco Commission report, 117management, 41,42,98,136,226ministerial responsibility, 65-67performance reporting, 104-5,107,160& private sector, 290rules & interpretation, 91workplace culture, 92

Public Service 2000 Report, 45,65-67

Q

Quality assuranceCICA Value-For-Money Auditing Standards,

332-38regimes for, 340-42

Quebec, 109 Queen Elizabeth Hospital (Toronto), 154,159Quinn, Robert, 126

R

Ratushny, Edward, 37Reagan, Ronald, 128Reciprocal responsibility, 41Reliance

& assurance, 317-19& coordination, 312as evidence, 293-94,322-24

degree of, 326external/internal audit information, 312-13factors affecting degree of, 321,322-24,326,331implications, 304,306-7internal auditing information, 319-20internal auditors, 313-14 materiality & significance, 321optimizing, 320-22reporting, 324-25strategies, 319-20

Relevance (attribute), 140-41 Remoteness, need for objective attestation, 223Report of the Independent Review Committee of the

Office of the Auditor General of Canada.See Wilson Committee report

Reportingaccountability, 97-98,232annual, 142,160audit, 123,146-49,156-57,193,230,236,238,

269,273,281,299broad-scope, 227,228CCAF, 136-44CEO, 147,191comprehensive audit, 274,275-76Comprehensive Audit Manual, 119,264,313& disclosure, 110effectiveness, 145-49,150-54,156-57,162,

190-91,213-15,219,242evidence, 293,296exception-based, 273by facilitator, 158fair & balanced, 284-85financial, 98,228findings, 256-66,276-77GASB role in, 113good, 130governing body involvement in, 155,158-60government, 104,105-7information, 273instances, 275-76issues, 271-85


legislative, 231by management, 98,104,137,289by media, 24,69-70,104,129-31performance, 80,97,100-1,104-5,107,

127-28,130-32,143,151-52,154-55,159-62,193, 286

private sector, 98,104,109-11public sector, 104,136,159reliance, 324-25representations, 132-33,147scope, 271significance, 281-82theory & practice, 973 Es, 219,242value for money, 234Wilson Committee, 47,122,226,231

Representationsaccuracy, 132-33attitude towards accountability, 133CEO involvement, 147comprehensive reporting strategy, 147-48description of, 132,138,168& effectiveness, 237exaggerated, 133external review of, 149lack of information for, 132language in, 133management

audit of, 192,237-38,241,274comprehensive strategy for reports, 147,160,

257cost & cost benefit, 148,160Elements of a Management Representation

document, 176-79& executive presentation (questions), 168-69& governing body, 187-89implementation workshop, 175information gathering, 148,185levels of assurance, 238-41preparing, 100-1,141,144,148-49,175,

176-77,185,237

writing, 100,158,186,189,190pitfalls, 132-33,148rendering true account, 132-33See also Governing body, representations

Responsiveness (attribute), 143,183 Research institutes, 8Results-based auditing, 242,350Rohrbaugh, John,126Rousseau, Jean-Jacques, 10Royal Commission on Financial Management &

Accountability. See Lambert CommissionRules, 33-35,76,97

See also Discretion

S

Scopeauditor obligation to report, 271cost & benefit of audit, 280defined, 245,271& intended degree of audit assurance, 274-76internal audit, 302-3,319key areas of audit, 252legislation on, 246meeting performance criteria, 277special examinations of Crown corporations, 278See also Comprehensive audit

Scotland, 111Service Efforts & Accomplishments (SEA), 113Secondary impacts (attribute), 142,182 Shared Management Agenda. See SMAShareholder expectations (accountability), 29,90,222Significance

& audit findings, 265,276-77auditor’s judgment, 281-82behavioural matters, 280comparative indicators, 280criteria, 281defined, 245,271,321factors influencing, 277,278-79& how to report, 281-82

I N D E X 3 6 8

judging, 278,321& materiality, 320in internal auditing, 311,320& potential risks, 280-81& size of expenditures or revenues, 280& trends in performance, 280& what to report, 280-82See also Comprehensive audit, principles

Sinclair, Duncan, 9SMA, 90-92SMAC, 115,227Social contract, 9,10,14

accountability,12as social compact, 11

Society of Management Accountants of Canada. See SMAC

Sopinka, John, 67South Africa, 114Spain, 11Special interest groups, 22-23Stewardship

accountability issues, 45,147administration,42Auditor General of Canada, 40auditor preparation, 81-83Supreme Court of Canada, 20

Bhatnager case, 67-68See also Corporate stewardship

Subsidiarity, 40

T

Technocracy, 353 Es

auditing performance, 219,287definitions of, 118-19,120due regard to, 234-35,236,237,242,256,277,

284,349in comprehensive audit planning, 229in internal auditing, 304,306-7reporting instances, 275-76

twelve attributes of effectiveness, 98-99,190, 237,277

Television. See MediaTreasury Board of Canada, 84,85,86,89,90,91Toronto Stock Exchange, 30,31Twelve attributes of effectiveness, 137,138-39,350

U

United Kingdom. See Great BritainUnited Kingdom’s Service Pay & Conditions Service

Code, 39United Nations, 228United States

bureaucracy, 77-79business director’s responsibilities, 29comptroller general, 142Constitution, 11,26General Accounting Office, 128,242,282,349government, 42,54

organizational chart, 26Government Accounting Standards Board, 113Government Performance & Results Act, 54,113not-for-profit institutions, 108performance measurement & accountability,

27,54,73,114privatization, 73results-oriented governance, 114separation of powers, 17

United Way, 107

V

Value for money accountability relationships, 233assessing & reporting on, 137& audit reports, 234auditing standards, 332,333-38& communication clarity, 274& internal audit practice, 298nonaudit consulting services, 232


publications (CICA), 234,267,268,271,274, 332-33,338,344

Standing Committee on National Finance, 120& 3 Es, 117-19

Value-For-Money Auditing Standards, 234,267,268, 271,332-38,344

Vicarious responsibility, 41

W

Wales, 111Waterman, Robert, 127Westland case, 68-69Wilberforce, William, 22Wilson Committee

auditor general’s right to report, 231definition of accountability, 47,116

effectiveness, 122report, 47,122,226,231

Working environment (attribute), 143-44,183,184World Bank, 9

I N D E X 3 7 0

ACCOUNTABILITY PERFORMANCE REPORTING …Accountability, performance reporting, comprehensive audit -...

Documents

Transcript of ACCOUNTABILITY PERFORMANCE REPORTING …Accountability, performance reporting, comprehensive audit -...