Measuring Public Transport Performance

7/28/2019 Measuring Public Transport Performance

1/50

Division 44Water, Energy, Transport

Measuring Public Transport PerformanceLessons for Developing Cities

Sustainable Urban Transport Technical Document # 9

Information Stockholm Oslo Helsinki Copenhagen Geneva

Easy to get the ino needed when planning a trip 74 % 74 % 81 % 73 % 71 %

The ino is good when trafc problems occur 22 % 21 % 28 % 31 % 49 %

The inormation is good in stops and terminals 37 % 51 % 45 % 43 % 81 %


2/50

AcknowledgementTe author would like to acknowledge the

research carried out by the authors o ran-sit Cooperative Research Program (CRP)Reports No 88 and 144; and the authors othe Research Results Digest 95 o the Inter-national ransit Studies Program (ISP) othe CRP, which provided guidance to buildthe case or introducing and improving per-ormance measurement in developing cities.Te author would also like to express her deepgratitude to experts odd Litman, Heather

Allen and Mohinder Singh or their critical

review and valuable suggestions or improvingthis document.

About the authorChhavi Dhingrais a civil engineer with aMasters Degree in transportation engineer-ing rom the Asian Institute o echnology,Tailand. For the last seven years she has been

researching on issues related to sustainablemobility and urban development policy andplanning, in the developing country context.Her work areas include improving urbanpublic transport services, policies to encourageusage o clean transport modes, transport andequity issues, and capacity development in theurban transport sector. Chhavi is part o GIZsSustainable Urban ransport Project (SUP)team where in addition to research and policyadvisory work or cities, she is involved in the

design and delivery o training courses anddevelopment o course material.


3/50

i

IMPRINTAuthor: Chhavi Dhingra

Editor: Deutsche Gesellschaft frInternationale Zusammenarbeit (GIZ) GmbHP. O. Box 518065726 Eschborn, Germanyhttp://www.giz.dehttp://www.sutp.org

Division 44: Water, Energy, TransportSector Project Transport Policy Advisory Services

On behalf of

Federal Ministry for Economic Cooperationand Development (BMZ)

Division 313 Water, Energy, Urban DevelopmentBMZ BonnDahlmannstrae 453113 Bonn, Germany

BMZ Berlin | im EuropahausStresemannstrae 9410963 Berlin, Germanyhttp://www.bmz.de

Manager: Manfred Breithaupt

Editing: Santhosh Kodukula, Jonathan Gmez Vilchez

Cover photos: Chhavi Dhingra, Santhosh Kodukula

Layout: Klaus Neumann, SDS, G.C.

Eschborn, December 2011

Measuring Public Transport PerformanceLessons for Developing Cities

Sustainable Urban Transport Technical Document # 9

CopyrightTis publication may be reproduced in wholeor in part in any orm or educational or non-prot purposes without special permissionrom the copyright holder, whenever providedacknowledgement o the source is made. TeGIZ would appreciate receiving a copy o anypublication that uses this GIZ publication asa source. No use o this publication may be

made or resale or or any other commercialpurpose whatsoever.

Disclaimer

Findings, interpretations and conclusions

expressed in this document are based oninormation gathered by GIZ and its consult-ants, partners and contributors rom reliablesources. GIZ does not, however, guaranteethe accuracy or completeness o inormationin this document, and cannot be held respon-sible or any errors, omissions or losses whichemerge rom its use.
http://www.giz.de/http://www.sutp.org/http://www.bmz.de/http://www.bmz.de/http://www.sutp.org/http://www.giz.de/


4/50

ii


5/50

iii

AbstractPerormance measures are navigation toolsthat can help public transport authorities andcity governments determine where they wantto go and how to get there. Tey have manypractical applications including trend analysis,comparisons, target setting, system improve-ment and incentives or managers and employ-ees. Tey help identiy potential problems andoptimal solutions. Tis document describesthe role that perormance measurement canplay in public transportation planning andmanagement, the need or developing cities to

start adopting perormance evaluation and thesteps or initiating this. Tis document pro-vides examples o successul public transportperormance evaluation systems rom acrossthe globe, including developing cities that arebeginning to explore these systems, and iden-ties key actors necessary or creating success-ul evaluation systems. Tis inormation willbe useul to policy-makers, analysts, and prac-titioners involved in urban transport planningand particularly public transport planning

and provision in cities, in both developed anddeveloping countries.


6/50

iv

Contents

1. Introduction 12. Performance measurement 2

2.1 Perormance measurement applications 3

2.2 Uses o perormance measurement in the transit industry 4

2.3 Types o perormance indicators 5

2.4 Perormance points o views 6

3. Past efforts on performance measurementand benchmarking in the transport sector 83.1 International initiatives 8

3.2 Regional initiatives 9

3.3 Lessons learnt 11

4. Case studies of public transport performancemeasurement programs from around the globe 124.1 Key fndings rom the case studies 12

5. Customer satisfaction and quality improvement-must havesof any performance measurement program 15

6. Realities and challenges for developing cities 186.1 Service level benchmarks or urban transport in India: Brie overview 18

6.2 Suggestions or urther improvement 20

7. Advice for cities and public transport agencieson setting up performance measurement systems 237.1 Data collection 23

7.2 Policy and/or legal enabler 24

7.3 Reporting 24

7.4 Making course corrections 24

7.5 Outreach, networking and eedback 25

8. Next steps 268.1 Data defnition 26

8.2 Suggested perormance evaluation areas 27

Bibliography 31

Appendix: Case studieson public transport performance evaluation 33


7/50

1

Measuring Public Transport Performance Lessons for Developing Cities

1. Introduction

Modal shares o public transport are on the decline in most developing cities. Deterioratingquality and/or lack o alternative transport modes, e.g. public transport and sae walking and

cycling acilities have orced many to shit rom public transport to using personal vehicles astheir daily mode o commute. Tis shit has translated into increased trac congestion, air andnoise pollution, reduction and deterioration o public spaces and urban orm, social exclusion,increased GHG emissions and many other negative externalities.

Figure 1

Public transportinrastructure inmost developingcities continues tobe o poor quality.Photo: Santhosh Kodukula,

Vishakhapatnam, 2008

Many cities (e.g. in India, Indonesia, China, Malaysia) are recognising this decline in modalshares o resource ecient modes and are attempting to address this issue at a policy level, byencouraging greater usage o public transport and non-motorised transport (NM) modes intheir cities. However, many o these cities are nding it a challenge to implement such a policyvision on ground due to a number o barriers which are mostly political, institutional and evencultural. While there is a lot o emphasis on new and expensive inrastructure creation to meetthe supply gaps in public transport; soter (and oten low-cost) interventions like improvingservice quality, accessibility and taking into account customer perception, are totally neglected.Tere is no mechanism that exists which can assess i the available transport modes are ullling

their expected roles and meeting various the standards regarding environment, saety, equity,comort, etc. In most cases, there are no standards.

It is well acknowledged that in order to improve and manage a service, one has to rst be ableto measure it. Hence in order to make public transport services attractive, and thereby increasetheir modal shares, public transport services in cities not only need to be planned, operated andmarketed well, they also need to be measured and monitored on a continual basis.


8/50

2

Sustainable Urban Transpor t Technical Document # 9

2. Perormance measurement

Perormance measurement is essential or eective planning and management. It includes vari-ous activities that track an organisations ability to achieve its intended objectives:

box 1

Key terms and defnitionsPerformance evaluation reers to a specic monitoring and analysis process

to determine how well policies, programs and projects perorm with regard to

their intended goals and objectives.

Benchmarking reers to a process or identiying best management practices

that an organisation can emulate.

Performance indicators (also calledmeasures of effectiveness) are specicmeasurable outcomes used to evaluate progress toward established goals

and objectives.

Baseline (or benchmark) existing, projected or reerence conditions i

change is not implemented.

Source: VTPI, 2010

Perormance measures are an extension o our personal senses sight, hearing, touch, smell andtaste. Tey are navigation tools that help an organisation determine where it wants to go andhow to get there. Tey have many practical applications including problem identication, trendanalysis, peer comparisons, target setting, evaluating potential improvements, and incentives ormanagers and employees (Litman, 2005).

Perormance measures are widely used in transport planning. Tey can have various namesincluding, sustainable transportation indicators, perormance indicators or just transporta-tion statistics. Regardless o what they are called, every jurisdiction and agency should developan appropriate set o statistics that are collected consistently, suitable or planning and evalua-tion purposes (specic guidance on this is provided in the last section o this paper).

Perormance measurement can support public transport planning in many ways. It allows tran-sit planners and operators to determine i resources are used eciently and equitably, identiypotential problems, and to veriy whether a particular improvement strategy achieves its pre-dicted targets. It paves the ways or course correction which translates into a constant eort at

improving services to match the standards.Public transport perormance evaluation can refect various perspectives. Many commonly-used public transport perormance indicators such as load actor and cost-per-vehicle-kilo-metre, measure operating eciency. Other indicators, such as rider comort, travel speed andreliability, aordability, integration and satisaction, refect the user experience. User-orientedindicators are important or developing public transit systems that respond to user demandsand so are able to attract even choice riders. Tis is an area that needs serious attention in mostdeveloping cities today.


9/50

3


Care is needed when using perormance evaluation and benchmarking to insure that theyare based on appropriate assumptions and quality data. Perormance indicators should not beselected simply because they are considered easy to measure. Important objectives and impacts,such as social equity and user comort, should not be ignored simply because they require moreeort to evaluate. It is also important to monitor the quality and consistency o data used orevaluation, and to ensure that the people who use data understand how it is dened, how it wasobtained, its reliability, and possible sources o bias.

Tis document provides guidance on perormance measurement program development orcities. It discusses basic concepts, describes examples o public transport system perormanceevaluation practices rom around the globe, and identies key actors or their success. Tisinormation will prove to be o value to policy-makers, policy analysts, and practitionersinvolved in urban transport planning, particularly public transport planning and provision incities in both developed and developing countries.

2.1 Perormance measurement applications

Perormance measurement involves the collection, evaluation, and reporting o data that relatesto how well an organisation is perorming its unctions and meeting its goals and objectives(CRP, 2003). Perormance evaluation reers to a specic monitoring and analysis process todetermine how well policies, programs and projects perorm with regard to their intended goalsand objectives.

Perormance measurement can have many specic applications:

Problem identication: It can identiy undesirable conditions, such as wasted resources,trac accidents or vehicle ailures, and help determine their causes.

Trend analysis: It can help identiy changes that are occurring.

Peer comparisons: It allows a particular organisation or group to be compared with peers(similar organisations or groups).

Evaluating changes: It can be used to track the results o specic changes, including newpolicies and programs to determine i they are successul, and or research purposes.

Target setting: It allows managers to set specic targets to be achieved.

Incentives: It can be used to establish rewards or managers and employees.

Benchmarking is the process o systematically seeking out best practices to emulate. Bench-marking involves direct contact with other organisations, delves into the reasons or their suc-cess, and seeks to uncover transerable practices applicable to the organisation perorming theanalysis (CRP, 2010).

Perormance measurement and benchmarking are dierent concepts. A perormance report is notthe desired end product o a benchmarking eort; rather perormance measurement aids bench-

marking by providing a set o indicators that are then used to provide insights, raise questions,compare with/identiy other organisations rom which one may be able to learn and improve.

For example, NOVA, an international rail benchmarking program comprising a consortiumo sixteen middle-sized metros rom around the world, denes benchmarking as a structuredapproach to identiy actions that lead to superior perormance. Benchmarking is not merelya comparison o perormance data or a creation o league tables. Perormance measures, orexample, deliver little benet on their own, but they stimulate productive questions and lines oenquiry or more in-depth analysis and research [1].

Figure 2 describes the step-wise methodology o benchmarking.

[1] http://www.nova-metros.org/GetAbout.do?category=Home&subcateg=Benchmarking , Accessed on 1 Sept. 2011.
http://www.nova-metros.org/GetAbout.do?category=Home&subcateg=Benchmarkinghttp://www.nova-metros.org/GetAbout.do?category=Home&subcateg=Benchmarking


10/50

4


Benchmarking can be used to highlight areas o low or high perormance and to show where anorganisation is in rankings. Perormance measurement orms the initial steps o a benchmarkingexercise.

2.2 Uses o perormance measurement in the transit industry

Perormance evaluation is now increasingly being used by transit agencies all over the world andseveral new tools and inormation systems are being created that make it easy and more ecientto apply this technique. Practitioners particularly nd this useul in the ollowing ways:

Reporting perormance o public transport to the authorities and publicwhich in many placesis required by law or as per a directive o the government. Usually in most places where it is

undertaken, perormance evaluation results are published and shared with all stakeholders.In Hong Kong, or example, this inormation is included in the annual report and in the sus-tainability report that shareholders receive annually.

Monitoring Service improvements, assessing past interventions, attracting more riders and orincreasing the appeal o public transport.

Diagnosing problems and the health o the system, making course corrections and refning strat-egy this means that perormance evaluation helps practitioners identiy areas in the system

which are not unctioning adequately and where service standards are not being met, andrectiy those. As an example, Singapores Land ransport Authority (LA) through itsannual customer satisaction survey to evaluate the service standards set in the license agree-ments, ound that customers expressed dissatisaction with requency o service. Since this

ties directly to LAs goal o making public transport a choice mode, LA decided to cor-rect course by asking transit providers to increase capacity by minimising headways during

Figure 2

Benchmarking steps.(Source: TCRP, 2010)


11/50

5


peak period rom 3 min to between 2 and 3 min and rom 7 min to between 5 and 6 minduring lunch periods (CRP, 2010).

Incentivizing quality improvements.

Responding to user eedback.

Providing decision making bodies with accurate inormation to support the needed actions orinvestments, budgeting, etc.

Providing the public with inormation on transit perormance so they can choose it and use it.

Setting service standards.

Aiding internal communications and management.

Noting community benefts(say increase in modal share o public transport over time, increasedliteracy rates and health improvements due to better access by public transport, etc.).

Te quality and nature o data collected today on public transport perormance in most devel-oping cities is limited and redundant, ocusing largely on operational eciencies and very littleon quality and other user-related parameters. I modal shares or public transport have to bepreserved and increased, it becomes very important that cities engage in perormance evaluationo public transport rom various perspectives-operator, user, local authority and community atlarge. A good starting point or initiating perormance evaluation or public transport services indeveloping cities would be its inclusion in the overall policy ramework. While in many devel-oping countries (or example India), an urban transport policy (or a drat) exists at the nationallevel, it is not complimented by a policy evaluation and monitoring program, nor governed byregulations and not linked to the budgeting process.

2.3 Types o perormance indicators

More than 400 perormance indicators are used in the transit industry today. Each indica-tor is assessed based on its perormance category (availability, service delivery, communityimpact, travel time, saety and security, maintenance and construction, and economic/nancial

Figure 3

ransMilenio, Bogota.(Source: GIZ Photo DVD, 2010)


12/50

6


viability), its data collection needs, and its potential strengths and weaknesses or particularapplications (CRP, 2010).

Perormance measures/indicators can be designed and dened at various levels, namely:a. Sustainable development indicators (general statistics about a citys economic, social and

environmental perormance);b. Sustainable transport indicators (general statistics about a transport systems economic,

social and environmental perormance);c. ransport planning indicators (conventional statistics about transport system peror-

mance, such as travel speed, congestion delays, accidents and uel consumption);d. Public transport strategic planning indicators (long-term public transport perormance

trends);e. Public transport operations indicators (short-term public transport perormance).

Within these, indicators could be again classied as very important, helpul or specialised aswell as environmental, social and economic. Tereore, a right balance has to be maintainedwhile selecting indicators. Indicators should be relevant and analytically sound, and correspond-

ing statistical data needs to be available. Tis data should comply with certain quality standards.In many cases, the availability o statistical data is currently a bottleneck or passenger transportbenchmarking, especially or non-motorised transport. Tese issues are discussed in greaterdetail in the last section o this document.

2.4 Perormance points o views

Perormance measurement can be carried out rom dierent perspectives: rom an enterpriseperspective or rom a customer perspective. It can take place at dierent levels: at policy level(regulatory ramework, inrastructure provision) and at microeconomic or enterprise level(transport companies and operators). Many current public transport system perormance indica-tors ocus on operating eciency (e.g. load actors and cost per vehicle-kilometre) rather than

perormance as experienced by users (convenience, comort, speed, reliability, aordability, inte-gration, etc.).

Measures can be either outcome or descriptive indicators. Outcome indicators describe the per-ormance achieved by the organisation, given a set o inputs, and should be the majority o themeasures used in the analysis. In a public transport context, many outcome indicators are per-ormance ratios that compare an outcome (e.g. ridership) to an input (e.g. revenue hours).

As Rickert (2005) describes, perormance indicators can be direct (e.g. Disabled passengerstook 250 trips in March on Bus route # 17) or they can be proxy measures which are sub-stituted or the direct measure (e.g. Following the deployment o low-foor buses at newlyimproved bus stops on Bus route # 17, Rehabilitation Center A reports that 20 additional per-

sons living near this route are now using their services.). Both the direct and the proxy meas-ures provide helpul data to understand the results. Both measures can be compared a year laterto indicate i usage is increasing or decreasing. Both measures permit comparisons to the situa-tion prior to initiating accessible bus service as well as a comparison to some stated objective oranticipated perormance.

In broad terms, perormance measures could be o the ollowing kinds:

Ratios (e.g. cost per revenue km, passenger per seat);

Indices (e.g. a measure combining capacity, route coverage, and requency);

Level o service (e.g. requency levels);

Stand-alone individual quantitative or qualitative measures (e.g. ridership, requency, pres-ence o digital inormation systems at bus stops);

Percentages (% increase in school/college enrolments ater introduction o bus services in anarea, percentage times when the bus arrived within a 5 minute delay, etc.).


13/50

7


General sustainability indicators can be integrated with other types o accounting statisticsin transport. Indicator sets should be derived as much as possible rom existing accountingdata sets, while existing accounting data should be extended towards sustainable developmentrequirements (Litman, 2005).

Perormance measures should be simple, intelligible to all concerned, and relevant to the mostimportant goals o the agency. Cities should careully look at trade-os between the time andcost o collecting data or perormance indicators, on the one hand, and the utility o the data,on the other. But most importantly, cities should rst dene its goals and targets rom which theperormance measures fow.

An extensive collection o perormance measures (130 amilies o measures and over 400 indi-vidual measures) as a reerence or agencies developing or updating a perormance-measure-ment program can be ound in the CRP Report No 88 (2003), which is available at: http://onlinepubs.trb.org/onlinepubs/tcrp/tcrp_report_88/Guidebook.pd . Te American Public rans-portation Association publishes an annual Public ransportation Fact Book reports (http://www.apta.com/research/stats), the Canadian Urban ransit Association publishes the Canadian

ransit Fact Book, and the US Federal ransit Administration maintains the National ransitDatabase (http://www.ntdprogram.gov/ntdprogram). Te Mobility in Cities Database (2001) othe UIP also contains some 120 indicators under 15 themes related to urban transport. Tis isdiscussed in more detail in the next section.
http://onlinepubs.trb.org/onlinepubs/tcrp/tcrp_report_88/Guidebook.pdfhttp://onlinepubs.trb.org/onlinepubs/tcrp/tcrp_report_88/Guidebook.pdfhttp://www.apta.com/research/statshttp://www.apta.com/research/statshttp://www.ntdprogram.gov/ntdprogramhttp://www.ntdprogram.gov/ntdprogramhttp://www.apta.com/research/statshttp://www.apta.com/research/statshttp://onlinepubs.trb.org/onlinepubs/tcrp/tcrp_report_88/Guidebook.pdfhttp://onlinepubs.trb.org/onlinepubs/tcrp/tcrp_report_88/Guidebook.pdf


14/50

8


3. Past eorts on perormance measurement and benchmarking inthe transport sector

In 1996, the European Commission (EC) identied passenger transport and in particular public

passenger transport, as a sector growing at an annual rate o 3.2 % while the average annualgrowth o GDP in real terms was 2.4 %. Growth largely occurred in private transport where75 % o the total kilometres were by private car, 15 % by conventional public passenger trans-port and the remaining 10 % by bicycle, walking, air and other means. Given this and the actthat urban public transport plays a key role in making cities sustainable, a number o eorts atperormance measurement were initiated, particularly in Europe, over the last couple o decadesto improve the delivery and attractiveness o public transport modes in cities. Some o thesehave been briefy discussed below (R, 2004)

3.1 International initiatives

3.1.1 The Millennium Cities Database (2001); Mobility in Cities Database (2006)Te Millennium Cities project involved the compilation o a database o data rom 100 citiesin order to compare their transport systems. Data was collected or over 200 indicators or the

box 2

The Mobility in Cities Database (2006)Developed by UITP in partnership with several other agencies in Europe, this database consists

o 120 urban mobility indicators or 52 cities worldwide or the year 2001. The CD-Rom contains:

1. A set o urban mobility indicators which can be searched by topic, location or a combina-tion o both;

2. Reports, analyses, recommendations, actsheets, denitions o indicators, and local mobil-

ity policies, available in 4 languages;

3. A tool to compare one city with up to 9 other cities in graph orm.

Indicators or each location are grouped under the ollowing themes:

1. Background inormation on city;

2. Private transport inrastructure;

3. Public transport inrastructure;

4. Private passenger vehicles: supply and use;

5. Taxis and collective taxis: supply and use;

6. Road trac;

7. Public transport supply;

8. Mobility and modal split;

9. Public transport productivity and travel cost;

10. Cost o travel or the traveller;

11. Cost o passenger transport or the community;

12. Energy consumption or passenger transport;

13. Passenger transport polluting emissions;

14. Passenger transport atalities;

15. Private motorised transport and public transport (comparisons).

Each o the above themes in turn has indicators specied under it, which have been reported

or the selected cities. The document on denitions o indicators contains nearly 122 deni-

tions, and is something cities developing perormance evaluation systems would nd extremely

useul as a reerence guide.


15/50

9


cities or the year 1995 and compared with the year 2001. Te database was later updated byUIP in 2005 (see Box 2). Conclusions on the evolution o key indicators between 1995 and2001 can be drawn by comparing both databases.http://www.uitp.org/publications/pics/pd/MILLE.pd,http://www.uitp.org/publications/Mobility-in-Cities-Database.cm .

3.1.2 NATCYP Benchmarking National Cycling Policies (2001)

Te NACYP benchmarking initiative was another o the BES networks case studies whichincluded a national level comparison o cycling policies between Czech Republic, England,Finland, Te Netherlands and Scotland. In addition, the development and delivery o nationalcycling policies was given the much needed publicity, raising ministerial awareness o cyclingspotential, particularly with regard to sustainable transport policies.http://www.velomondial.net/page_display.asp?pid=14

3.1.3 CoMET (1996) and NOVA (1998)

Tese are both international railway benchmarking clubs. Tey are dierent, but related, groupswhich aim to compare the urban rail systems o various cities rom around the world. Memberspay a ee to belong to the clubs. COME groups bring together large sized metros and NOVAmedium and smaller sized metros. In these programs, agencies use insight rom other metrosystems in order to attempt to improve internal working cultures and ones own service levels,based on a set o Key Perormance Indicators (KPIs).http://www.comet-metros.org and http://www.nova-metros.org/Welcome.do

3.2 Regional initiatives

3.2.1 Citizens Network Benchmarking Initiative (19982001)

Te goal o the pilot project o the European Commission, DG Energy and ransport (DGREN), was to test the easibility o comparing public transport perormance across all modes,rom a citizens point-o-view. During the pilot, 132 perormance indicators were tested, which

were rened to 38 indicators by the end o the process. Te overall objective o the project wasto promote the identication and dissemination o good practice in urban transport systems andinrastructure by enabling cities and regions to exchange ideas and experiences and to comparethe perormance o their local and regional transport systems by benchmarking methods. Tethree main questions which the indicators were seeking to answer are:

How do people travel? What transport services do people preer and how well is the systemmeeting these requirements?

How accessible is the transport system? How congested are the roads? What inormation is

available to motorist and transport users? What are the costs o transport? What is the impact o transport on the environment? Howsae is it to travel?

http://www.transportbenchmarks.eu/benchmarking/initiatives-citizens-network.html#

3.2.2 EQUIP (19992000)

Te EU Urban ransport Benchmarking initiative called EQUIP was concerned with devel-oping a sel-assessment benchmarking handbook that proposed indicators or measuring theinternal eciency o land-based local public transport operators in EU nations. Tere are veseparate but compatible versions o the handbook or ve public transport modes (bus, trolleybus, tram/light rail, metro and local heavy rail) plus a short version o 27 super indicators to

provide an entry to benchmarking.http://www.transport-research.ino/web/projects/project_details.cm?id=289
http://www.uitp.org/publications/pics/pdf/MILLE.pdfhttp://www.uitp.org/publications/Mobility-in-Cities-Database.cfmhttp://www.velomondial.net/page_display.asp?pid=14http://www.comet-metros.org/http://www.nova-metros.org/Welcome.dohttp://www.transportbenchmarks.eu/benchmarking/initiatives-citizens-network.html#http://www.transport-research.info/web/projects/project_details.cfm?id=289http://www.transport-research.info/web/projects/project_details.cfm?id=289http://www.transportbenchmarks.eu/benchmarking/initiatives-citizens-network.html#http://www.nova-metros.org/Welcome.dohttp://www.comet-metros.org/http://www.velomondial.net/page_display.asp?pid=14http://www.uitp.org/publications/Mobility-in-Cities-Database.cfmhttp://www.uitp.org/publications/pics/pdf/MILLE.pdf


16/50

10


3.2.3 Scandinavian BEST (1999 onwards)

Set up by the Stockholm public transport authority SL, this initially involved the our Nordiccapitals o Oslo, Stockholm, Copenhagen and Helsinki and took the orm o a survey o 36questions based on 10 categories carried out in spring 2000. Te idea was that, or each o the

10 categories, the city which displayed the best results would present a success story at a semi-nar. As a result, our Common Interest Groups were chosen and each city was made responsibleor one o them. Tese were:

Integrated Public ransport and City Planning Copenhagen

Inormation at rac Disruptions Helsinki

Complaint Management Oslo

Systematic Branding Stockholm

http://best2005.net

3.2.4 BESTRANS (2004)

Benchmarking o Energy and Emission Perormance in Urban Public ransport Operations

aimed at developing an internal and external benchmarking methodology or energy and emis-sion perormance in the urban public passenger transport sector and to carry out a benchmark-ing exercise with a number o European operators. Te indicators used in the BESRANS pro-

ject ocus particularly upon the eciency, energy costs, average travel speeds and extent o routeprioritisation or buses. http://www.tis.pt/proj/bestrans

3.2.5 The Urban Transport Benchmarking Initiative (20032006)

Tis three year initiative benchmarked dierent aspects o 45 participating European Citiestransport systems, with themed working groups each researching individual urban transporttopics in great depth. http://www.transportbenchmarks.eu

3.2.6 CAFs Urban Mobility Observatory (SMO)

Te Corporacin Andina de Fomento (CAF) recently launched the Urban Mobility Observa-tory (SMO). Te observatory is intended or Latin America and the Caribbean and providestechnical indicators and data analysis or 15 metropolitan areas in nine countries o the region.

Also see Box 3. http://www.ca.com/view/index.asp?pageMS=61860&ms=19
http://best2005.net/http://www.tis.pt/proj/bestranshttp://www.transportbenchmarks.eu/http://www.caf.com/view/index.asp?pageMS=61860&ms=19http://www.caf.com/view/index.asp?pageMS=61860&ms=19http://www.transportbenchmarks.eu/http://www.tis.pt/proj/bestranshttp://best2005.net/


17/50

11


box 3

Transit Evaluation in Latin America

Despite having some o the most amous transit systems in the world, Latin America has recog-nised the need or better perormance and maintenance o its systems. Since the 1970s, ever

since the concept o BRT was born in Curitiba (Brazil), public transport systems have been running

inconsistently and independently rom each other, without a common set o standards, reerences

or guidelines. To address the disparity in the perormance o bus systems in the region, a new

industry coalition was established in April 2010, called the Asociacin Latinoamricana de Siste-

mas Integrados de Transporte y BRT(SIBRT) (Latin American Association or Integrated Transport

Systems and Bus Rapid Transit). Weblink: http://www.sibrtonline.org

SIBRT includes 14 o the most infuential transit agencies in Latin America, representing ve

countries: Colombia, Brazil, Chile, Mexico and Ecuador. The key objective o the SIBRT is to pri-

oritise mass transit in Latin American cities by developing standard methodologies to measure

the perormance, impact and management o BRT systems, synchronising data collection across

transit agencies, and creating saety baseline measurements. SIBRT is meant to provide a body

o accessible research and expertise that cities can draw rom in order to develop and build upon

the models that exist in Latin America. The aim is to share a common set o standards, case

studies and best practices, to help improve transit systems across Latin America, and deal with

challenges like overcrowding and are evasion, the encroachment o bus lanes by general trac,

or slow bus speeds.

Based on a set o Key Benchmarking Indicators (KBIs), SIBRT will help promote and standardise

other elements o high-quality BRT systems, including Intelligent Transportation Systems (ITS),

like real-time bus tracking or bus prioritisation at trac lights, exclusive bus corridors, level plat-

orms between the bus and curb, pre-payments and controlled access to stations. Other cities can

monitor this mechanism and adopt it to suit their requirements. The Benchmarking and Innovation

area o SIBRT has the task o coordinating the organisation, implementation and maintenance o

a database o indicators.

Sources: http://www.thecityfx.com/blog/establishing-standards-to-improve-brt-systems-in-latin-america ; Last accessed on

6 September 2011, and http://www.sibrtonline.org; Last accessed on 27 November 2011.

3.3 Lessons learnt

Learnings rom these initiatives indicate issues regarding data collection and the need to haveconsistent denitions o data or comparing cities/regions. Tings like national (or even statelevel) regulations/laws, market conditions and technological changes and sotware upgradesimpact data comparison results. Clear denitions o indicators, good quality, clear and consist-ent data sets and good legal backing and support o public transport operators and authorities is

a must-have or cities to have a successul perormance measurement system. All the key actorsmust understand the process and eorts needed or such a system and design it in a way thatcan make the system applicable and locally relevant. However, given that developing cities areliterally to start this process rom ground-zero, it is assumed that they will realise and addressthese issues as they go along the process, however, where possible they should learn rom theexperiences o the other cities/regions.

Te initiatives mentioned above raised some pertinent questions which are valid and importantor all cities. Te ocus o these exercises, even all those years back, was measuring aspects likeoperations, quality, inormation, user perception, transport access, energy and environmentimpacts, etc. in relation to public transport, which is what developing cities need to ocus on ina big way today.
http://www.sibrtonline.org/http://www.thecityfix.com/blog/establishing-standards-to-improve-brt-systems-in-latin-americahttp://www.sibrtonline.org/http://www.sibrtonline.org/http://www.thecityfix.com/blog/establishing-standards-to-improve-brt-systems-in-latin-americahttp://www.sibrtonline.org/


18/50

12


4. Case studies o public transport perormance measurementprograms rom around the globe

A study o a variety o good practices in public transport perormance evaluation rom Asia,

Australia and Europe was carried out (see Appendix). Public transport perormance evaluationsystems o cities like Singapore, Kuala Lumpur, Sydney and Helsinki have been studied anddocumented. Tese cities were analysed against the ollowing set o parameters which oer othercities a useul perspective on how perormance evaluation systems could be set-up and run:

Basic geographic and demographic inormation;

Existing transport modes;

Share o public transport trips;

Key agencies involved in public transport planning and provision;

Guiding Policy document or public transport provision/management?

Perormance evaluation mandatory as per Government requirement?

Perormance measures and standards used;

Reporting Format; Feedback/evaluation methods;

Legal enabler;

Enorcement;

Outreach eorts.

Te technical sheets detailing public transport evaluation systems (in the above ormat) in selectcities around the world have been provided in the Appendix.

4.1 Key fndings rom the case studies

A review o perormance evaluation systems as presented in the Appendix shows that oten

agencies providing public transport, by law are required to (a) establish goals and objectives orimproving services (b) develop strategies to meet these objectives (c) dene perormance criteriaand targets (d) measure progress and dene inputs or uture improvements (e) report periodi-cally the results o perormance evaluation.

Figure 4

Kuala LumpurPhoto: Andreas Rau, 2006


19/50

13


Figure 5

Singapore adopts asystematic approacho setting measurable

perormance targetsin line with its policy.Source: Derived rom TCRP, 2010

In the Asian context, Singapore presents an example o a well-planned and systematic per-ormance evaluation system, where each policy objective is clearly translated into measurabletargets and perormance indicators. Singapore also oers an excellent example o a well denedpolicy, policy objectives and targets under each objective or ensuring a sustainable urban trans-port system (Figure 5). Tis is something that other cities could emulate and learn rom.

Kuala Lumpur has only recently initiated a perormance evaluation program, which appears tobe less mature than Singapores. It is only recently that bus management in Rapid KL realisedthey needed to make corporate and strategic planning eorts to improve the companys serviceand protability, and it is to Rapid KLs credit that it has taken lessons rom other systemsin order to establish its own processes and is showing signs o success with each step orward.Developed cities like Sydney in Australia and Helsinki in Finland also oer examples o goodperormance evaluation systems, where service parameters are detailed out and measured witha proper eedback system, to meet the agency objectives. Sydneys comprehensive and user-ocussed set o perormance measures and targets present a holistic approach in perormanceevaluation. Helsinki stands out or its highly customer-oriented approach o benchmarking itspublic transport system. In act, most o the Benchmarking European Service o public rans-

port (BES) surveys reveal that Helsinki is a clear winner when it comes to some aspects likevalue or money and customer loyalty. Besides ocussing on quality parameters, Helsinki also


20/50

14


does well in areas like competitive tendering processes or buses, integrated ticketing and pro-viding bus priority. So does Sydney.

Tis approach o packaging various measures to promote public transport is usually what islacking in developing cities today. Also evaluating areas like access, equity, ease o transers andcustomer perception is usually overlooked by public transport providers. As a result, service

levels or the captive riders continue to deteriorate, and the choice users gradually move to per-sonal motorised modes o travel.

Each case study oers a dierent perspective, yet valuable lessons or cities to initiate similarperormance evaluation exercises in public transport. Te presence o a guiding/mandatorypolicy and a legal enabler, a systematic and well-coordinated perormance evaluation system(like BES), clear agency goals and objectives, sound inter-agency coordination, identicationand constant revision o perormance measures, a special ocus on customer care, good report-ing and inormation sharing, and enorcement, emerge as the common and key actors orachieving successul evaluation systems.

Figure 6

SydneyPhoto: Manred Breithaupt, 2008


21/50

15


5. Customer satisaction and quality improvement-must haves oany perormance measurement program

Tis section does not ocus on a particular city, but on a very important aspect o perormance

measurement which is largely neglected in developing cities today: customer satisaction andquality improvement related measures in public transport.

Figure 7

Having strongcustomer ocus is one othe central elements o

creating a good publictransport perormanceevaluation system.Photo: Kaushik Deb, Karnataka, 2009

Quattro (1998)[2] was a research project carried out under the ransport Research and ech-nological Development (RD) Programme o the EUs Fourth Framework Programme orRD and Demonstration. Te geographical scope o Quattro covered the European Union,Norway, Poland, Hungary and the Baltic States.

Recognising that modal shares o public transport have to be increased, and that like any othersuccessul service enterprise, public transport too should ocus more on customer satisaction,

the objectives o the Quattro project were: to identiy current and emerging quality management practices in contracting and tenderingin the urban public transport (UP) sector,

to evaluate these practices and try to gure out how the existing trends in quality manage-ment in other elds than UP could be implemented in UP with identication o bestpractice and well developed total quality management,

to suggest guidelines to authorities and operators involved in UP provision on issues o ten-dering, contracting and perormance monitoring, with a strong ocus on quality.

Te study also recognised that the quality o urban public transport was dependent on a numbero actors like the capacity o the operator to manage his organisation, operating conditions, roleo the public authorities (roads department, trac police, etc.).

At present, in many developing cities, the above ideas and practices are absent. aking theexample o India, where public transport is essentially bus-based and a state level unction, citieshave very little role to play in the planning, provision and monitoring o public transport. Testate bus companies typically run in losses and without any incentives to improve quality andin the absence o a policy directive/law to measure and improve perormance, bus operations incities continue to remain inadequate and unattractive.

Quattros research developed a specic quality management tool, the UP quality loop(Figure 8), which can be applied at the rms level as well as at the whole UP systems level.

[2] http://www.transport-research.ino/web/projects/project_details.cm?id=636&backlink=%2Fweb%2Fcommon%2Fsearch%2Ecm&reerer=searchstring%2Aquattro ; Last accessed on 27 November 2011.
http://www.transport-research.info/web/projects/project_details.cfm?id=636&backlink=%2Fweb%2Fcommon%2Fsearch%2Ecfm&referer=searchstring%2Aquattrohttp://www.transport-research.info/web/projects/project_details.cfm?id=636&backlink=%2Fweb%2Fcommon%2Fsearch%2Ecfm&referer=searchstring%2Aquattrohttp://www.transport-research.info/web/projects/project_details.cfm?id=636&backlink=%2Fweb%2Fcommon%2Fsearch%2Ecfm&referer=searchstring%2Aquattrohttp://www.transport-research.info/web/projects/project_details.cfm?id=636&backlink=%2Fweb%2Fcommon%2Fsearch%2Ecfm&referer=searchstring%2Aquattro


22/50

16


Figure 8

Te UrbanPublic ransport

Quality Loop.Source:

QUATTRO taken rom TCRP 2010

Te report describes the quality loop to be based on our distinctive benchmarks:1. Expected Quality: Tis is the level o quality demanded by the customer. It can be

dened in explicit and implicit expectations. ools or evaluation: revealed and statedpreerence methods.

Figure 9

Good inormationsystems are an

integral part ogood quality public

transport systems.Source: TransMilenio S.A., 2007


23/50

17


2. Targeted Quality: Tis is the level o quality that the transport undertaking aims toprovide or its passengers. It should be dened according to the level o quality expectedby the passengers, external and internal pressures, and budgetary constraints and com-petitor/market perormance. ools or evaluation: customer charters and guarantees oservice, partnership agreements, quality standards and certication, quality contracts,quality tenders and evaluation procedures, etc.

3. Delivered Quality: Tis is the level o quality that is achieved on a day-to-day basis innormal operating conditions. Disruptions to service, whether they are the ault o theundertaking or not, are considered. ools or evaluation: compensation schemes or thebenet o the users, reward/penalty schemes concerning operators and authorities, inter-nal quality measurement, sel assessment methods and benchmarking (KPIs).

4. Perceived Quality: Tis is the level o quality perceived by passengers during their jour-neys. ools or evaluation: customer satisaction index (CSI), customer charter eedbacksystems.

From the above, it is evident that there are many approaches that operators and public authori-ties can adopt to improve public transport quality. Box 4 describes Sydneys experience on this.

box 4

Applying customer satisaction and quality monitoring inSydneys busesThe private bus operators in Sydney (Busways) adopt a rigorous customer

satisaction policy as part o achieving their agency goals. In addition to moni-

toring on-time running (drivers radio-in when delays are longer than 10 minutes

and appropriate action is taken), there is an emphasis on using three dedicated

sta personnel to perorm customer service duties exclusively, on a roam-ing basis throughout the system. These employees are in direct contact with

customers and are used as the eyes and ears o management to ensure

that the operating plan works successully. They also act as troubleshooters

to solve on-the-spot problems (e.g. ticketing issues and missed connections).

The coordinators are also bus drivers at dierent times o the day and this way

get to experience rst-hand the customer issues.

Customer inormation is provided at all bus shelters. Busways uses two

contractors to monitor perormance on buses and at bus/rail interchanges.

By travelling as passengers, these contractors are not recognised by the sta

and are able to report to management on the perormance o drivers (customer

relations) and on any other problem encountered by passengers. Customer

service is monitored by the use o customer surveys which are conducted atregular intervals.

In order to maintain long term loyalty, bus operators also employ an Inra-

structure Planning Manager and an assistant, who consistently work with local

councils, the Roads and Trac authority, and other inrastructure developers

to ensure that all planned developments are bus riendly.

(Source: TCRP, 2003)


24/50

18


6. Realities and challenges or developing cities

Little has been done to establish perormance evaluation systems or public transport in devel-

oping cities. Te reasons or the same are maniold: lack o one body accountable and respon-

sible or public transport planning and provision, loss-making operations and hence dierentpriorities (ironically engaging in perormance evaluation could help make operations viable, but

this is not the ocus at the moment), inadequate policy thrust at state and central levels, lack o

any legal binding or quality improvements and customer service, etc. Given these, the capac-

ity challenges and the act that in many developing cities, public transport services do not all

under the purview o responsibilities o the city authorities, many might eel that there is much

that remains to be done in other areas beore good perormance management systems are estab-

lished. On the other hand, establishing good perormance evaluation as a regular practice can

initiate the needed improvements and ensure long term sustainability o public transport sys-

tems. Te next section discusses an initiative o the Government o India in this direction.

6.1 Service level benchmarks or urban transport in India: Brie overview

Recognising the need or perormance evaluation and to address quality concerns, compare

cities and improve urban transport perormance in cities, the Ministry o Urban Development

(India) introduced Service Level Benchmarks (SLBs) (which resemble perormance indicators),

or urban transport to be adopted by selected Indian cities, to start with. Tese benchmarks

were introduced to set a basic minimum standard o perormance which are commonly under-

stood and used by all stakeholders. Tese benchmarks were meant or introducing accountabil-

ity in service delivery and to help city authorities and other agencies in identiying perormance

gaps and eecting improvements through the sharing o inormation and best practices, ulti-

mately resulting in better services to the people. Te SLBs were intended to provide a common

minimum ramework or monitoring and reporting on service levels o various urban transportunctions.

Te Ministry has introduced SLBs or the ollowing areas in urban transport:

Public transport acilities;

Pedestrian inrastructure acilities;

Non-Motorised ransport (NM) acilities;

Level o usage o Intelligent ransport System (IS) acilities;

ravel speed (Motorised and Mass ransit) along major corridors;

Availability o parking spaces;

Road saety;

Pollution levels;

Integrated land use transport system;

Financial sustainability o public transport.

Each o the above areas has been urther described with a list o indicators/perormance meas-

ures. ypically, our levels o service (LoS) have been specied, viz. 1, 2, 3, and 4; with 1

being highest LoS and 4 being lowest to measure each identied perormance benchmark. A

ormula or calculating the LoS or each indicator has already been provided. Te goal, natu-

rally, would be to attain the service level 1 or each indicator. Te LOS o each indicator is

added up to arrive at the overall LOS or the citywide LOS or the area which is being bench-

marked, which is then compared to a prescribed table indicating the health levels or that area.

At the end o this calculation, one can arrive at a qualitative assessment o how well or poor the

identied area o urban transport is perorming. Tis has been explained with the help o anexample in Box 5, which ocuses on the rst area Public ransport Facilities.


25/50

19


box 5

Analyzing service level benchmarks or Public Transport Facilities in India

Perormance Measures identied under the urban transport unction Public Transport Facilities:1. Presence o Organised Public Transport System in Urban Area

Target: Within the rst year, all JNNURM cities to establish Organised Public Transport System and

by second year all 0.2 million plus population cities (as per 2001 Census) to establish the same.

2. Extent o Supply/Availability o Public Transport

Target: Within the rst two years, all million-plus cities but less than 4 million to increase public

transit supply to service level 3 or above all 4 million-plus cities to increase supply to service

level 2 or above.

3. Service Coverage o Public Transport in the city (Bus route network density)

Target: All million-plus cities but less than 4 million to increase their public transit coverage at

least supply to service level 3 or above; all 4 million-plus cities to increase the service coverage

to service level 2 or above.

4. Average waiting time or Public Transport users

Target: All million-plus cities to maintain average waiting time or public transport users to be

a maximum o 10 minutes or below within 2 years.

5. Level o Comort in Public Transport (Crowding)

Target: In all million-plus cities, within 2 years, the level o service should be 3 or above.

6. Percentage Fleet as per Urban Bus Specications

Target: All million-plus cities to have at least 25 % o their feet as per urban bus specications

by the end o rst year.

For the calculation o LOS or the measure Average waiting time or Public Transport users,

the steps to be ollowed are:

a) Delineate the key corridors or public transport in the city;

b) Calculate the average waiting time (in min) o passengers or each route;

c) Create the requency distribution.

LOS 4 Avg. waiting time

1 < = 4

2 4 6

3 6 10

4 > 10

Calculated LOS o Public Transport = LOS 1 + LOS 2 + LOS 3 + LOS 4 + LOS 5 + LOS 6

Overall Calculated LOS Interpretation for the citys public transport system

1 < 12 Good, wide spread, easily available to the citizens, comortable.

2 12 16Needs considerable improvements in terms o supply o buses/

coaches, coverage and requency. It is comortable.

3 17 20May need considerable improvements in terms o supply o buses/

coaches, coverage, requency o the services and comort.

4 21 24 Non-existent or poorly organised.

Similarly, LOS or other urban transport unctions is also to be calculated.

The complete document on SLBs can be accessed rom:

http://urbanindia.nic.in/programme/ut/Service_level.pd
http://urbanindia.nic.in/programme/ut/Service_level.pdfhttp://urbanindia.nic.in/programme/ut/Service_level.pdf


26/50

20


6.2 Suggestions or urther improvement

Te SLBs or urban transport introduced by the Ministry o Urban Development, Government

o India, is a well thought out and intended technical initiative by the Government. Such anexercise is necessary to set baselines or urban transport in cities, which do not currently exist.Tis initiative also refects the governments vision o having a greater share o well-unctioningpublic transport modes in the cities. Some observations and suggestions on the SLBs are dis-cussed below. Tough these emerge rom the Indian SLBs, they could be equally applicable toother developing cities which are considering establishing perormance measurement or evenbenchmarking systems.

a) It would be a good idea to start with perormance evaluation, develop the requisite sys-tems or monitoring, make improvements, and then gradually move towards creating abenchmarking system, where best practices are emulated rom each other. Focussing onthe latter should come ater the cities have addressed their challenges and proven them-

selves to be competitive in service delivery. Learning rom good practices is no doubtuseul, however bringing the service levels to a minimum acceptable standard shouldremain as the main ocus.

b) Data collection and reporting is a resource intensive activity, and some kind o prioritisa-tion o areas should be done. It has been widely acknowledged that public transport andnon-motorised transport improvements are priority areas or developing cities. Tereore,cities should develop a set o perormance measures in areas relevant or them, ratherthan adopting the entire set o service level benchmarks and reporting on them. Forexample, areas like level o usage o IS, may not be necessary or all cities, especiallythe medium and small order cities. In act, i every city can start to measure the peror-mance o its public transport and non-motorised transport systems properly, and use this

inormation or generating improvements and creating a paradigm shit in policy-makingand public perception these issues, it would be a signicant achievement.

Figure 10

Mumbaissub-urban rail.

Photo: Chhavi Dhingra, 2011


27/50

21


c) Furthermore, the indicators or non-motorised and public transport should include prin-

ciples o universal design and inormation and guidance systems as indicators.d) Having vehicle travel speeds o motorised vehicles and availability o parking spaces

service level benchmarks conveys a wrong signal, as improving these may result in creat-ing induced demand or more personal motor vehicles. Instead o targeting or improve-ment, these areas could be targeted or demand management or discouraging personalvehicle usage, and to compliment public transport and non-motorised transport improve-ment. Instead o measuring travel speeds o personal vehicles, cities should measure howmuch the travel speeds o public transport modes increase every year as opposed to per-sonal modes, and, level, enorcement and eectiveness o parking management measurescould be monitored instead o number o parking spaces.

e) Road saety is an important measure, but conficts with the earlier area o having

increased travel speeds. Increased speeds o say personal vehicles usually would implyincreased road risk or other vulnerable road users and modes, hence reduced road saety.Hence, prioritisation o areas needs to be done or perormance evaluation.

) For air pollution related measures, ocus should also be on monitoring technology usageand mix in feets and tail-pipe emissions. Measuring things like energy eciency opublic transport vehicles, percentage o vehicles running on clean uel, percentage o allurban trips made by public transport, emissions per public transport vehicle, etc. mightbe useul and more pro-active than simply measuring the pollutant concentration levelsin air [3].

[3] The Benchmarking o Energy and Emission Perormance in Urban Public Transport Operations(BESTRANS, 2004) report provides useul insights in this area.

Figure 11

Bangalores air-conditioned busesoer comort andconvenience.Photo: Chhavi Dhingra, 2007


28/50

22


g) Aggregating results as a number in the orm o level o service does not always conveythe correct picture and risks oering an aggregate and sometimes vague diagnosis o theproblem where it becomes dicult to say which improvements matter most. At the endo the day, how we use the valuable data collected or interpreting results will govern howsuccessul a perormance evaluation system is. Review o international case studies revealsthat it is necessary or the city authority/operator to have clear goals, objectives, peror-mance measures and targets which give a clear idea o what and how much improve-ment is needed where. Tis does not imply that indicators need to always be quantitative.Simple yes or no indicators at times may be enough to understand the problem (e.g.do 90 % o the bus stops in the city have disabled ramp access?). However the questionsasked or areas measured should refect actual problem areas o urban transport and leadto ways or improving them, rather than presenting a generic urban transport scenario orthe city in terms o numbers.

h) Comparisons based on the prescribed service level benchmarks and using the same cri-teria o LoS across dierent cities might obscure the cities unique characteristics. Asalready mentioned above, every city/region needs to identiy its priority areas or meas-

urement and develop indicators accordingly. In case o cities with similar public transportsystems, there could be an agreed set o perormance measures that they need to reporton, depending on the reporting requirements o the government. However, all o thisshould be in line with the goals and objectives o a policy guideline/law or voluntarycommitment to undertake perormance evaluation or public transport.

i) Recognising the importance and role that Intermediate Public ransport (IP) vehicleswill continue to play in developing cities, a way to include IP modes in perormanceevaluation and integrating them with existing/proposed public transport modes needs tobe explored. Tis also translates into making the SLBs more conducive to the needs othe smaller cities which might not always have ormalised public transport systems.

A comprehensive report by UIP on statistical indicators o public transport perormance in

Arica (UIP, 2010), oers good insights into using a set o global indicators or evaluatingpublic transport perormance, and identiying the specic challenges applicable in the Ari-can context through case studies. Te report also makes recommendations or bringing aboutreorms in the public transport sector in Arican cities, which is dominated by the so calledinormal transport modes.


29/50

23


7. Advice or cities and public transport agencies on setting upperormance measurement systems

Perormance measurement is a strategic process that will take some time to be ully developed,

and come to a level where it is used or making substantial changes to policies and strategies. Itneeds the coming together o a number o aspects, and as ar as possible should be an independ-ent and transparent process.

Developing cities may need to address several challenges to create these systems (maybe evencreate new laws/enactments) and build the required capacity. Given below are guidelines thatcities and transit agencies should keep in mind while establishing a comprehensive public trans-port perormance measurement program:

7.1 Data collection

Carrying out an evaluation exercise would require time and rigorous data collection eorts,

which would need eective monitoring, coordination and commitment on part o the cityauthorities. Perormance evaluation eorts should not end in surveys, tabulation and report-ing only. Interlinking transport statistics to perormance indicators would also need to bedone.

Cities should apply the ollowing principles when selecting transportation perormance indi-cators (VPI, 2010):

Comprehensive Indicators should refect various economic, social and environmentalimpacts, and various transport activities (such as both personal and reight transport).

Data quality Data collection practices should refect high standards to insure thatinormation is accurate and consistent.

Comparable Data collection should be standardised so the results are suitable orcomparison between various jurisdictions, times and groups. Indicators should be clearlydened.

Easy to understand Indicators must useul to decision-makers and understandable tothe general public. Te more inormation condensed into a single index the less meaningit has or specic policy targets (or example, Ecological Footprint analysis incorporatesmany actors) and the greater the likelihood o double counting.

Accessible and transparent Indicators (and the raw data they are based on) and analysisdetails should be available to all stakeholders.

Cost efective Te suite o indicators should be cost eective to collect. Te decision-making worth o the indicators must outweigh the cost o collecting them.

Net efects Indicators should dierentiate between net (total) impacts and shits oimpacts to dierent locations and times.

Perormance targets select indicators that are suitable or establishing usable peror-mance targets.

In most cities where perormance evaluation is done, most data and inormation regardingmobility trends is placed in the public domain, and is accessible to general public and agen-cies, alike. Tis enables standardisation and avoids spending money on conducting multiplestudies by dierent agencies. Tis kind o inormation sharing enables successul monitoringsystems, and should be adopted by cities. Tis would again require good inter-agency coordi-nation and ree-fow o inormation. Cities and public transport agencies should agree uponcommon measures and data denitions this will provide standardisation and enable com-parison o data with other cities/countries (see more on this in Section 8.1).

Indicators should be prepared early in the planning process. For example, perormance

measures should be included in invitations or transport concessionaires to provide services,or in requests or proposals or supplying transport vehicles or inrastructure. Leaving some


30/50

24


fexibility or minor negotiations with the regulating authorities, agreed upon standards andperormance measures could go into the contractual language as binding criteria during thelie o the agreement (Rickert, 2005).

raining and capacity development to understand the various aspects o initiating and con-tinuing a perormance evaluation program will need to be imparted at various levels or gov-ernmental and non-governmental stakeholders. Tis will include things like understandingdata ormats, conducting surveys, data analysis and suggesting remedial actions. Policy-mak-ers, city planners, transport operators, drivers and even the civil society need to understandthe need and benets o perormance evaluation rom their perspective.

7.2 Policy and/or legal enabler

Experience rom other cities demonstrates that getting cities and operators to agree on engag-ing in perormance evaluation usually requires a directive rom the central government(could be a policy, law, commitment to a Charter or any legal enabler), state level support andthe ull cooperation o all local actors whose work aects public transport planning and pro-

vision, including the civil society. States and cities shall commit to this eort and start with modest goals and objectives tomonitor and improve public transport. It is important to ensure that such an eort is notone-o and the cities are able to sustain and continue with the same in the uture.

7.3 Reporting

Annual/quarterly/monthly reports must be prepared and shared by cities/operators with allkey stakeholders, including the public.

Reporting should not be restricted to nancial and operational measures. Service as per-ceived by the customer and measures like accessibility, saety, comort, reliability, aord-ability, etc. should be made part o a ormal perormance evaluation system (see more on

this in Section 8.2).

7.4 Making course corrections

Tis should be an ongoing process, and over time new and dierent perormance indicatorsshould be tested to seek urther improvements. Te integration o perormance evaluationactively into agency development and decision-making is most critical.

Practitioners should make sure that they link perormance measures back to strategic goalsand objectives and acilitate constant course correction. Experience rom Asian cities likeSingapore, Hong Kong, aipei, etc. shows that this usually needs strong commitment andsupport o senior management, regularly scheduled meetings to evaluate perormance, ocus-sing on a limited number o measures, proper communication o the results internally and

externally and presence o both quantitative and qualitative perormance measures.


31/50

25


7.5 Outreach, networking and eedback

Perormance evaluation should be accompanied by good outreach eorts; e.g. having a help-ul website, call centre support, newsletter. All these enable operators to reach out to thepublic and in turn the public to understand and use their services more and more.

Peer networking, mentoring and proessional support are important actors, especially whencities are about to set up perormance evaluation systems. Agency sta grows proessionallythrough exposure to and discussions with colleagues in similar positions at other agencies.wo key success actors or European transit benchmarking networks have been the use oan external acilitator (e.g. a university or a private consultant) and ongoing nancial support.Te acilitator perorms unctions that individual transit agency sta may not have time orexperience or, including compiling and analysing data, producing reports, and organisingmeetings (e.g. inormation-sharing working groups on a specic topic or an annual meetingo the network participants) (Ryus and Sembler, 2010).

Tere are several organisations working actively in this area; e.g. Volvo Research and EducationFoundations, which could be contacted or advice by cities and public transport agencies.


32/50

26


8. Next steps

8.1 Data defnition

One o the rst things that developing cities need to do in order to engage with perormanceevaluation or even benchmarking, is data collection and beore that, data denition. Most citiestoday do not have baseline data on aspects like urban travel demand, modal shares, trip makerproles, travel patterns, land-use changes, inrastructure capacity and quality, service quality,inclusiveness, equity, etc. Te quality o data that exists is usually poor and the absence o clearand consistent denitions and unreliable collection methods add to the quality problems.

Figure 12

Proper data defnitionand collection arepre-requisites to

successul perormanceevaluation systems.

Photo:

Chhavi Dhingra, Ahmedabad, 2011

In order to address the data gaps, extensive surveys (household, transit, employment centres, etc.surveys) and data mining and consolidation rom existing secondary sources (mobility plans,trac and transportation studies, land-use plans, population census, etc.) might be required tobe undertaken. Also, the database created should be in a standardised and agreed ormat whichcan be used by all stakeholders. A glossary o terms should be developed which should apply toall cities and the terms dened.

Denitions o these terms could ideally be grouped by topics/themes, consistent with the datacollection tables. For example, this could be one possible grouping:

Modes o service related

Demographic, geographic

Financial

Organisational

Legal/regulatory

Vehicle characteristics


33/50

27


Energy and environment related

Universal design

Quality/services received by user

rip patterns in the city (planning)

Services rendered/operations

Vehicle and service related amenities, inrastructure

8.2 Suggested perormance evaluation areas

Relevant parameters, in the orm o questions, and examples o perormance indicators or theseparameters are listed below. Cities and public transport agencies should dene start collectingpublic transport statistics and perormance indicators on these guidelines, as part o setting upa perormance evaluation system. Tis will give a good idea o the health o the system. Oncethis is done and measures or correction identied, then agencies could look at setting bench-marks and start comparisons between cities.

Figure 13

It makes sense tostart with moreobvious and basic

perormance measureslike availability, access,aordability, saety,comort, convenienceand reliability.Photo:

Chhavi Dhingra, Ahmednagar, 2011


34/50

28


Table 1: Public transport performance evaluation areas for developing cities

Parameter Issues to be addressed Examples of possible Indicators*)

Modal share Is public transport the most preerred mode o

travel in the city? Which user groups constitutethe public transport riders?

Share o trips made by public transport

(by user groups);

Availability The presence o a public transport network or

mode in an area/locality;

Number, requency, No. o hours or

which service is available; headways

Accessibility The ability to reach the mode within a reasonable

time period, by a reasonable path (unobstructed

inrastructure) and presence o inormation

systems to access public transport;

Percentage o areas having public

transport accessible within 500 metres

by walking/cycling, walkability in areas

being served by public transport,

availability o user inormation via

phone, internet, sms; inormation

accessible and understood by all user

groups;

Reliability How well does the public transport ollowpublished schedules?

Number o breakdowns, share onon-time trips;

Saety and Security Feeling o saety rom accidents and injury while

using public transport and eeling personal

security;

Accidents and injuries per 100 000

trips, saety o pedestrians accessing

public transport, number o incidents o

thets and sexual harassment, agency

responsiveness to incidents, visibility

and lighting;

Equity (inclusiveness) How easily disadvantaged groups (people

with low incomes, physical disabilities or other

disadvantages) reach and use the system?

Physical accommodation and acilities

or disabled in vehicles and stations,

baggage carrying acilities;

Aordability and

payment

How aordable is using public transport? How

easy is it to pay to use various modes?

Fares as shares o incomes, ares in

comparison to other modes, multiplepayment options available, intermodal

are integration;

Intermodal connectivity How easily can transers be made rom one

mode to another-both physically and are-wise?

Integration between service providers,

other modes, ares;

Quality, speed, attrac-

tiveness and comort

How attractive is public transport to retain

existing users and attract personal vehicle

users? Does it get priority on road? Does it have

dierentiated services? Is it complimented by

supporting measures to discourage personal

vehicle usage? Do operators have incentives to

maintain and improve service quality?

Boarding and alighting ease, availability

o seats, cleanliness, gender separated

seating (in some countries this

makes services more attractive to

women commuters), passenger air

quality, basic amenities at stations, air

conditioning, bus only lanes, courteous

sta, onboard internet acilities;

Environmental impact What is the level and savings o energy and

emissions as a result o increased usage o

public transport;

Emissions per km, uel eciency, share

o feets run on clean uels;

Economic aspects Are investments, are policies, taxation structures,

costs borne by operators, subsidy mechanisms

in the transport sector conducive to improving

the availability and eciency o public transport?

Trends in investment in public transport

and supporting non-motorised

transport improvements, tax and

subsidy burden on public transport

Operational peror-

mance o public trans-

port systems

How well are the services doing nancially,

technically?

Revenue per km, No. o bus

breakdowns, etc.

HR policies and internal

management

Organisational perormance and business

management;

Sta to bus ratio, perormance based

appraisal;


35/50

29


Parameter Issues to be addressed Examples of possible Indicators*)

Presence o policies/

norms at national/state/city levels to ensure:

Public transport services are run as a successul

business enterprise;

Modal shares o public transport are increased

by making it the most avoured and priority mode

o travel in cities;

Perormance evaluation is an integral part o

public transport planning and provision;

Cities are responsible and empowered to plan

and manage their public transport services;

There is complete coordination between various

relevant agencies related to public transport

planning and provision.

Existence o a public transport policy,

policy directive on perormancemeasurement, inter-agency

coordination and cooperation.

*) Could be both qualitative and quantitative

Table 1: Public transport performance evaluation areas for developing cities (II)

able 1 presents a starting point or cities to create an evaluation system. Te mechanism orinstitutionalising perormance evaluation could either be proposed by the Centre or be workedout jointly between the states and cities. Tere also needs to be some lead time or preparationor the cities and or them to develop capacities to initiate and continue such an exercise. oachieve this, there is a need to give local governments both incentives and management toolsto implement policies or more ecient, cleaner and saer urban mobility and encourage a shittowards a more sustainable urban mobility culture. Tere are several initiatives that govern-ments worldwide are taking in this direction, which could oer valuable lessons and directions

in this regard. Box 6 describes one such initiative.


36/50

30


box 6

EcoMobility SHIFT Measuring mobility policies

The EcoMobility SHIFT project (June 2010 May 2013) is an activity o the Global Alliance or EcoMobility which aims to createa certication scheme to assess and help improve local governments sustainable transport policies, where municipalities

will be able to obtain an EcoMobility Label. The EcoMobility certication scheme is meant or local authorities or groups o

municipalities responsible or transport policies in an urban region as a whole. The project denes an EcoMobile City as one

developing, improving and encouraging the use o alternatives to the private car. The project targets that a city, its trans-

portation/mobility department, in cooperation with internal and external partnerships and supported by the policy-makers

and citizens, is unctioning and taking decisions in a way (enablers) as to put the right transport products and services o

high quality in place (products and services) that can take the city towards EcoMobility (results and impacts). This is briefy

explained in the gure below.

Labeling, identiying strengths and weaknesses, benchmarking and measuring: all orm key aspects o such an exercise,

which will require creating denitions and terminologies that distinguish good perormance rom bad, indicators, etc. This will

be supplemented by knowledge o best practices in each o the identied areas or improvement.

In this Project, a list o 36 indicators has been compiled as well as inormation sheets or each o them. The list contains seven

indicators or relevant actors impacting on the EcoMobility criteria (city prole), six indicators measuring the citys approach

in planning or EcoMobility (the Enablers), sixteen indicators measuring the authoritys output in public space (Transport

systems and services), and seven indicators measuring the longer term impact o these eorts (Results & impacts) on the

transport systems as well as on the city at large, such as modal split and energy eciency. For each indicator, ve levels o

perormance have been described to enable scoring. The weighing o the individual scores within the blocks (domains) o

assessment criteria, as well as the weighing o the various blocks against each other (e.g. how do we rate processes com-

pared to results) will be decided.

This approach acilitates the review o mobility policies and their impacts at a city level and thereore promises to enable a

more holistic outcome. It also makes the city authorities more responsible and accountable or its mobility practices. It does

however, require extensive inormation gathering. For more inormation, visit: http://www.ecomobility.org/shit-project
http://www.ecomobility.org/shift-projecthttp://www.ecomobility.org/shift-project


37/50

31


Bibliography

CTE (2008), Improved Methods For Assessing Social, Cultural, And Economic Eects O

Transportation Projects, NCHRP Project 08-36, TRB (http://www.trb.org) and AASHTO; at

http://www.statewideplanning.org/_resources/234_NCHRP-8-36-66.pd. CTOD (2010), Perormance-Based Transit-Oriented Development Typology Guidebook,

Center or Transit-Oriented Development (http://www.reconnectingamerica.org); at http://

www.reconnectingamerica.org/public/display_asset/2010_perormancebasedtodtypolo-

gyguidebook.

CTS (2010), Measuring What Matters: Access to Destinations, the second research summary

rom the Access to Destinations Study, Center or Transportation Studies, University o Min-

nesota (http://www.cts.umn.edu); at http://www.cts.umn.edu/Publications/ResearchRe-

ports/pddownload.pl?id=1426 .

FHA (2004). Transportation Perormance Measures in Australia, Canada, Japan and New

Zealand. U.S. Department or Transportation, Federal Highway Administration. USA

Margareta Friman and Markus Fellesson (2009), Service Supply and Customer Satisac-

tion in Public Transportation: The Quality Paradox, Journal o Public Transportation, Vol. 12,

No 4, pp. 41-56; at http://www.nctr.us.edu/jpt/pd/JPT12-4.pd.

Albert Gan, Feng Gui and Li Tang (2011), System or Transit Perormance Analysis Using

the National Transit Database, Journal o Public Transportation, Vol. 14, No 3, pp. 87-108; at

http://www.nctr.us.edu/wp-content/uploads/2011/10/JPT14.3.pd.

Theunis F. P. Henning, Sugandree Muruvan, Wanhua A. Feng and Roger C.Dunn (2011),

The Development O A Benchmarking Tool For Monitoring Progress Towards Sustainable

Transportation In New Zealand, Transport Policy, Vol. 18, pp. 480488 (http://www.science-

direct.com/science/article/pii/S0967070X10001368).

Robert A. Johnston (2008), Indicators or Sustainable Transportation Planning, Trans-portation Research Record 2067, Transportation Research Board (http://www.trb.org), pp.

146154; at http://pubs.its.ucdavis.edu/publication_detail.php?id=1260.

Kittleson & Associates (2003a), Transit Capacity and Quality o Service Manual, Report 100,

Transit Cooperative Research Program, Transportation Research Board (http://www.trb.org);

at http://trb.org/news/blurb_detail.asp?id=2326.

Todd Litman (2003), Measuring Transportation: Trac, Mobility and Accessibility, ITE Journal

Measuring Public Transport Performance

Documents

Transcript of Measuring Public Transport Performance