Efficiency Measurement of Road Maintenance using Data Envelopment Analysis PRESENTED BY:...
-
Upload
tracy-brooks -
Category
Documents
-
view
220 -
download
0
Transcript of Efficiency Measurement of Road Maintenance using Data Envelopment Analysis PRESENTED BY:...
Efficiency Measurement of Road Efficiency Measurement of Road Maintenance using Data Envelopment Maintenance using Data Envelopment
Analysis Analysis PRESENTED BY: KONSTANTINOS TRIANTIS
AUTHORS:MEHMET EGEMEN OZBEKJESUS M. DE LA GARZAKONSTANTINOS TRIANTIS
DATE:JUNE 28TH, 2007
TENTH EUROPEAN WORKSHOP ON EFFICIENCY AND PRODUCTIVITY ANALYSIS
Today…Today…
MOTIVATION FOR THE RESEARCH
PROBLEM STATEMENT
RESEARCH PURPOSE, OBJECTIVES, AND HYPOTHESES
DATA AND MODELING ISSUES
IMPLEMENTATION EXAMPLE
CONTRIBUTION OF THE RESEARCH
For the last two decades, the For the last two decades, the road maintenanceroad maintenance concept has been gaining tremendous concept has been gaining tremendous attention.attention.
Surveys by National Council on Public Works Improvement and American Society of Civil Engineers indicate a substantial deterioration in the road system from 1988 to 2005.
The Federal Highway Administration endorsed “asset management” to be the future approach of road maintenance for all state departments of transportation.
PERFORMANCE MEASUREMENT
Capital investment to build.
Maintenance to upkeep what is built.
Performance-based:
The risk of deficient design is imposed on the contractor.
The contractor often seeks innovative construction methods to make profit under fixed fee payment.
Asset management:
Comprehensive and well-structured approach to the long-term management of assets to provide effective
and efficient services. Calls for the utilization of
management, engineering, and economic principles in making decisions as to how resources should be allocated.
Virginia Department of Transportation (VDOT) has Virginia Department of Transportation (VDOT) has been one of the been one of the firstfirst state state DOTsDOTs to implement a to implement a pilot performance-based asset management pilot performance-based asset management contract.contract.
Earlier research has developed a Earlier research has developed a five-five-component frameworkcomponent framework to compare to compare effectivenesseffectiveness of the traditional maintenance of the traditional maintenance approaches against the performance-based approaches against the performance-based asset management approaches and asset management approaches and implemented it to evaluate the pilot project implemented it to evaluate the pilot project
(Piñero 2003, p. 47)
Quality of Service(Agency/UsersSatisfaction)
Timeliness ofResponse
Safety
Overall Performance
Cost
Level of Service
Current Road Maintenance Performance Measurement Systems
Not knowing how “efficient” state DOTs are in being “effective” can lead to excessive and unrealistic maintenance budget expectations.
For the cases in which comparative analyses are made, disregarding such external and uncontrollable factors and using pure effectiveness results may lead to unfair comparisons.
Solely focus on “effectiveness” measures, e.g., level-of-service.Disregard the “efficiency” concept,e.g., the amount of resources utilized to achieve such level-of-service.
Do not investigate the effect of the environmental factors, e.g., climate and location.Do not investigate the effect of the operational factors, e.g., traffic, load, design-construction adequacy.
The purpose of this research is, by using the Data Envelopment Analysis (DEA) approach, to develop and implement a generic, replicable, and comprehensive framework that can: Measure the overall efficiency of road maintenance operations. Consider the effects of external and uncontrollable factors on such efficiency.
The specific objectives of this research are, through the use of real data, to identify: 1) The relative efficiency of different units in performing road maintenance services.2) The reasons of the efficiency differences between units.3) The effects of the external and uncontrollable factors on the efficiency of units.4) The benchmarks (peers) and best practices that pertain to the inefficient units.
The hypotheses of this research are as follows:1) State DOTs are more efficient when they utilize the performance-based approach instead of the traditional method-based approach for the maintenance of the roads. 2) A significant portion of the observed efficiency differences between different units can be attributed to the effects of the environmental factors (e.g., climate, location, etc.) and operational factors (e.g., traffic, load, design-construction adequacy, etc.) faced by such units.
This presentation focuses on the This presentation focuses on the data and modeling issuesdata and modeling issues relevant to the relevant to the first first seven tasks seven tasks of the frameworkof the frameworkDefine the comprehensive list of Input-Output
Variables and Uncontrollable Factors
Address the issue of uncontrollable factors
Refine the comprehensive list of Input-Output Variables and Uncontrollable Factors
Perform data mining
Clean the data
Allocate the data to the DMUs
Decide on the size of the DMU and the time unit of analysis
Perform data conversion and data rearrangement
Choose the type of DEA models to be run
Run the DEA models and obtain the efficiency score, targets, and peer(s) for each DMU; and the overall efficient frontiers
Identify (i) the reasons of efficiency differences between the units and (ii) effects of uncontrollable factors on the efficiency of the units
Prepare the data to be used in the DEA models
Validate the results, identify best practices, and perform policy analysis
To define the input-output variables and uncontrollable To define the input-output variables and uncontrollable factors, factors, thethe process process that is under investigation should be that is under investigation should be studied. studied.
Preserving theCapital Investment
Maintenance ofHighways
Maintaining FreeFlow of Traffic
Preventive Maintenance -Planned, Cyclical
Restorative Maintenance -Minor Repairs
Rehabilitative Maintenance - Major Repairs and Replacements, Reconstruction
Types of maintenance performed to meet Level-of-Service requirements
Restorative Maintenance -Minor Repairs
Incident Response
Snow and Ice Removal
Types of maintenance performed to meet Timeliness of Response requirements
Preventive Maintenance -Planned, Cyclical
Works in the favor
OutputLevel-of-Service
(LOS)Works against
Preventive and Restorative Maintenance Deterioration
To define the input-output variables and uncontrollable To define the input-output variables and uncontrollable factors, factors, thethe process process that is under investigation should be that is under investigation should be studied. studied.
Consumes inputs Consumes outputs
Affected by uncontrollable factors
Inputs:Labor
MaterialEquipment
This research deals with This research deals with allall of the of the asset itemsasset items present within the present within the right-of-way fences. However, this presentation will focus only on the right-of-way fences. However, this presentation will focus only on the bridgesbridges..
Defining Input-Output Variables for the Maintenance of Defining Input-Output Variables for the Maintenance of BridgesBridgesInputs: Resources used by DMUs Outputs*: Benefits made as a
result of the process undertaken by the DMUs
*: Undesirable outputs are not considered in this research
Labor
Material
Equipment
Bridges’ Maintenance
Bridges Maintained
to meet LOS Requirements
Inputs OutputProcess
Input
Labor Hours
Quantity of Material
Equipment Hours
Bridges’ Maintenance
National Bridge Inventory Rating
Input Metrics to Establish Technical Efficiency
Output MetricProcess
Cost
Input Metric to Establish Cost Efficiency
METRICS
Once everything is considered, the list of input-output variables and Once everything is considered, the list of input-output variables and uncontrollable factors becomes uncontrollable factors becomes exhaustiveexhaustive..
The DMU is chosen to be the The DMU is chosen to be the county of Virginiacounty of Virginia and the time unit of and the time unit of analysis is chosen to be a analysis is chosen to be a fiscal yearfiscal year-- Both decisions are Both decisions are dictated by dictated by the cost data.the cost data.
1
2
3
4
5
8
7
6
The resulting number of DMUs The resulting number of DMUs dictateddictated the approach to be used to the approach to be used to deal with uncontrollable factors. deal with uncontrollable factors.
One Stage ApproachesUncontrollable Factors treated as Controllable Variables in the DEA Model
Uncontrollable Factors treated as Uncontrollable Variables in the DEA Model
Uncontrollable Factors used to Develop Categories of DMUs to be included in the DEA Models
Continuous Uncontrollable Factors used to Restrict the Peer Reference Set
Multi Stage ApproachesUncontrollable Factors used to Perform Regression Analysis over the Obtained Efficiency Scores
Parameters Obtained by the Regression Analysis used to Build an Overall Environmental Harshness Index
Uncontrollable Factors used to Perform Bootstrapped Regression Analysis over the Obtained Efficiency Scores
There is a number of approaches to There is a number of approaches to refine the comprehensive lists of refine the comprehensive lists of controllable variablescontrollable variables and and uncontrollable factorsuncontrollable factors..
Develop (or adjust) the list of DMUs to be compared
Develop the comprehensive list of variables
Refine the variable list using the Judgmental Process
Achieved a well refined list that can be used in the
DEA models?
Refine the variable list using the Quantitative Methods
Refine the variable list using the DEA Based Analyses
Achieved a well refined list that can be used in the
DEA models?
Achieved a well refined list that can be used in the
DEA models?
Formalize final DEA model using the list of variables obtained at this point
YES
YES
YES
NO
NO
NO
Ø Expert opinionsØ Analytic Hierarchy
Process (AHP)
Ø Aggregation MethodØ Rescaling MethodØ Principal Component Analysis (PCA)Ø Identification of Correlations
Ø Performing trial DEA runs with different combinations of variables
To To refine the comprehensive list of variablesrefine the comprehensive list of variables, , aggregation methodaggregation method was used. was used.
OutputsUndesirable outputs are out of the scope of this researchCritical output variables are Deck, Superstructure, Substructure, and Slope/Channel ProtectionNo cost data for Slope/Channel Protection is provided by VDOTDeck, Superstructure, and Substructure are combined into one variable by using the weighting structure developed and implemented by Virginia Tech as agreed by VDOT
InputsUsing the research by Dadson (2001), 16 input variables (Variable 2- Variable 4 and Variable 6-Variable 18) can be combined into a single input variable that represents all of those to a great extent
Dadson identified the Dadson identified the effects of theeffects of the VA Environmental Regions VA Environmental Regions on the on the deterioration of the bridges.deterioration of the bridges.
TW: TidewaterEP: Eastern PiedmontWP: Western PiedmontN: NorthernCM: Central MountainSM: Southwestern Mountain
As a result of the refinements, As a result of the refinements, total number of variablestotal number of variables (controllable (controllable and uncontrollable) reduced and uncontrollable) reduced from 26 to 4from 26 to 4..
Before showing the Before showing the implementation exampleimplementation example of the developed of the developed framework for bridges, it is essential to discuss the framework for bridges, it is essential to discuss the datadata..
Level-of-Service DataProvided by Federal Highway Administration.Federally-mandated program.Used to establish investment requirements, to develop data summaries at the national level for reports to Congress, and to respond to inquiries from entities.
Cost Data
Provided by VDOT as obtained through Financial Management System II.Per fiscal year, per county.Includes only preventive maintenance and restorative maintenance expenditures. No rehabilitative maintenance expenditure is included.No district or central office overhead is included.
Data miningData mining and and data cleaningdata cleaning were performed. were performed.
The data was needed to be The data was needed to be convertedconverted into the format suitable to into the format suitable to represent the variables. Moreover some represent the variables. Moreover some rearrangementsrearrangements had to be had to be made in the data to make it meet the made in the data to make it meet the structuring requirementsstructuring requirements of the of the DEA model.DEA model.
A number of A number of final refinementsfinal refinements accompanied the accompanied the DEA model selectionDEA model selection and and runrun..
BCC Model is selected to be utilized as it is believed that the bridge maintenance process is more likely to experience variable returns to scale.
Change in Overall Bridge Condition
Total Area Served
Regional Effect Variable
Cost
Modified Change in Overall Bridge Condition
DMU TAS
DMU REGIONAL EFFECT
Cost adjusted for overhead and inflation
Frontier AnalystFrontier Analyst was used to run the chosen DEA model and to was used to run the chosen DEA model and to obtain the results.obtain the results.
Conclusion 1: All counties are efficient relative to each other. Conclusion 2: The model, as structured and run above, is not able to discriminate between the DMUs and thus overlooks the relative efficiency differences (inherent within the DMUs) that exist in reality.
To test the validity of the alternative To test the validity of the alternative conclusions, conclusions, further analysesfurther analyses were were performed.performed.
Reducing the Number of Variables using DEA Based Analyses
Increasing the Number of DMUs
Increasing the number of DMUsIncreasing the number of DMUs changed changed the the efficiency scores to a great extent.efficiency scores to a great extent.
Fauquier
Albemarle
Augusta
Alleghany
Roanoke
Henrico
Spotsylvania
Rockbridge
Given the fact that the value of the cost variable for the DMUs Given the fact that the value of the cost variable for the DMUs belonging to Roanoke County is extremely low, further analysis (this belonging to Roanoke County is extremely low, further analysis (this time time excluding the DMUs belonging to the Roanoke Countyexcluding the DMUs belonging to the Roanoke County) is needed ) is needed to verify that Spotsylvania County is indeed extremely inefficient to verify that Spotsylvania County is indeed extremely inefficient relative to other counties in the data set. relative to other counties in the data set.
Removing the Roanoke County from the analysis Removing the Roanoke County from the analysis affected affected onlyonly the efficiency score of the the efficiency score of the Spotsylvania County.Spotsylvania County.
The results of the models need to be The results of the models need to be validatedvalidated and and verifiedverified by the decision makers at VDOT before by the decision makers at VDOT before actually taking any of the actions suggested by the actually taking any of the actions suggested by the model.model.
DEA does not directly pinpoint the underlying causes of inefficiencies of DMUs.
The findings of a DEA study are intended to be used as guides for managerial actions and policymaking as calculated targets for inputs and outputs indicate potential performance and efficiency increases for inefficient DMUs.
Contribution to the Contribution to the Highway MaintenanceHighway Maintenance literature: literature: A framework that is able to differentiate effective and
efficient maintenance strategies from effective and inefficient ones.
A A frameworkframework that is able to take into account: that is able to take into account: multiplemultiple views (e.g., the driver’s versus the maintenance views (e.g., the driver’s versus the maintenance
provider’s)provider’s) important important societal goalssocietal goals (e.g., safety) (e.g., safety) uncontrollableuncontrollable considerations (e.g., climate, location, etc.) considerations (e.g., climate, location, etc.)
Contribution to the Contribution to the DEA / Measurement ScienceDEA / Measurement Science literature: literature: Application of DEA within the Application of DEA within the engineeringengineering discipline discipline Illustration ofIllustration of
issues associated with issues associated with definingdefining andand refining the listrefining the list ofof input-output variables input-output variables andand uncontrollable factors uncontrollable factors
data availabilitydata availability issues issues modeling issues associated with modeling issues associated with definingdefining the appropriate the appropriate
decision making unitdecision making unit the potential the potential impactimpact that efficiency analysis can have on that efficiency analysis can have on
road maintenance improvementroad maintenance improvement
Proposed research’s Proposed research’s contribution to BOKcontribution to BOK is is twofold:twofold: