1

INFORMATION IS POWER – IN GOOD TIMES AND BADG.A. Christoffersen

City Care Limited, Tauranga, New Zealand

ABSTRACT

Data is the foundation of decision-making in ordinary day-to-day business operations. But in a civil emergency,

when normal operations are disrupted, the focus on robust business process in collecting and distributing data is

suspended in favour of the immediate need to restore service.

Emergency situations demand emergency actions. Data collection responses must be capable of meeting the

needs of unknown future users with unspecified and possibly divergent requirements.

This paper explores approaches to data collection on the repair work to the underground pipe networks in

Christchurch following the earthquakes. In particular, it will showcase a software application used to capture

closed circuit television (CCTV) inspection data. CCTV based data is facilitating the post-disaster repair, design

and management of Christchurch’s underground pipe network.

KEYWORDS

Christchurch, earthquake, CCTV, underground services, wastewater, stormwater, database

1 INTRODUCTION

1.1 BACKGROUND

The Christchurch earthquakes, particularly the events of 4 September 2010 and 22 February 2011, had a

devastating effect on the lives of the residents and continue to have an impact never seen before in New

Zealand. The loss of life and damage to buildings and homes has been well documented through Central

Government and Local Government reports, the media and of course through social media channels such as

Twitter, YouTube and the Internet in general.

The process of rebuilding Christchurch City has commenced, led by the Government Christchurch Earthquake

Recovery Authority (CERA), the agency coordinating the ongoing effort. An unseen component of the rebuild

is the effort of gathering data and information that underpins the design decisions required for the rebuild.

There are many data and information sources being used to inform the design decisions of infrastructure and

buildings before the construction is undertaken. These information sources include land stability, structure

condition and the state of the underground utility services networks.

The investigation and inspection of the condition of the stormwater and wastewater underground utility services

is primarily assessed by the use of Closed Circuit Television (CCTV). The focus of this paper is on the processes

and methods used to collect, evaluate and supply the CCTV inspection data to the Stronger Christchurch Rebuild

Team (SCIRT) to enable informed design decisions to be made.

SCIRT is rebuilding Christchurch’s damaged street level civic infrastructure which includes roads, fresh water,

wastewater and stormwater systems following the earthquakes of 2010 and 2011. The SCIRT delivery teams are

responsible for the day to day rebuild delivery works and consist of City Care Limited (City Care), Downer,

Fletcher Construction, Fulton Hogan and McConnell Dowell. Ultimately SCIRT is responsible to the people of

Christchurch and New Zealand.

2

1.2 CONTEXT

The focus of this paper is on the data processes associated with the CCTV inspection activity, however it is

important to set the scene on the scale of the CCTV operation being undertaken in Christchurch. City Care is

undertaking CCTV work on behalf of SCIRT with the inspections helping designers/engineers determine the

extent of the network that needs to be replaced or rehabilitated as the rebuild progresses.

In post-earthquake Christchurch, undertaking the CCTV inspection on the pipes is often the easy part of the

operation. Preparing the pipes to a state suitable for the CCTV inspection often involves jetting (washing out

silt and debris) and sucking where possible, the jetted material from the pipe to allow the pipe to be successfully

inspected using CCTV.

Photograph 1: “CCTV camera used to inspect pipes

with no or minimal flow”

Photograph 2: “CCTV survey Boat to inspect pipes

that are either flooded or under constant flow”

Currently there are 30 combined camera, jetting, and suction crews operating across the Christchurch network

with 350 people involved. The crews involved consist of City Care and contractor teams. As at 13 July 2012

approximately 30,400 pipes have undergone CCTV inspections, equating to 1100km of the wastewater and

340km of the stormwater networks since 4th September 2010. With the current level of resourcing the extent of

CCTV inspection undertaken averages between 10 and 12.5km per week, to a maximum of 15km sometimes

achieved.

The scale of the earthquake damage being inspected is unlike anything previously recorded in New Zealand,

including pipes now flowing in the opposite direction, significant displacement or complete blockage. In many

cases the same pipes have to be re-inspected after subsequent earthquakes.

Photograph 3: “Example of the pipe damage Photograph 4: “Example of the pipe damage

3

encountered during pipe inspection activities” encountered during pipe inspection activities”

4

2 CCTV DATA AQUISTION

2.1 INSPECTION HISTORY

2.1.1 4 SEPTEMBER 2010 EARTHQUAKE

Following the 4 September 2010 earthquake and up until the 22 February earthquake, the Christchurch City

Council (CCC) managed the inspection work undertaken by CCTV contractors. The inspections were used to

evaluate pipe condition in areas where the network was damaged or where it was in an unusable state. The initial

focus of this work was to understand what emergency repair works were required to get the networks operational.

CCTV of the pipes inspected was undertaken and the detail logged into either computer-based programs,

Cleanflow / WinCan or onto paper-based log sheets. The video and log information was catalogued by CCC in a

Microsoft Excel spreadsheet, with the damage information used to undertake emergency repair works.

The approach used in managing the data during this period was:

Two Microsoft Excel spreadsheets were used (wastewater and stormwater) to manage and catalogue the

inspection data captured and the evaluation of the data – i.e. whether the pipe asset was undamaged or

needed repair works.

The original videos are stored at CCC.

Most of the inspection videos were copied onto a network computer drive at CCC, occupying

approximately 2 terabytes of storage space. Some of the original video disks had errors and could not be

copied onto the network drive.

2.1.2 22 FEBRUARY 2011 EARTHQUAKE

In comparison to the 4 September event, the 22 February earthquake generated significantly more damage to

the wastewater and stormwater underground pipe network. While one of the uses of the CCTV inspection

information was to assist in getting the pipe networks operational, the information collected was seen as an

important input into the rebuilding process of the wastewater and stormwater networks to ensure the

development of a long term, maintainable, sustainable network.

Considering the scale of the damage to the civic infrastructure, the decision was made to transfer the CCTV pipe

inspection priorities and resultant work instructions to SCIRT, guided by the requirements of the design teams

responsible for rebuilding the civic infrastructure networks. The management of the CCTV field operations, data

management and reporting was at that stage transferred to City Care.

Along with the scale of the data collection, there was a realisation that the inspection data would not only be

required for the rebuild process, but would likely to be required for as-yet-unknown future requirements. This

necessitated a change to the process of collecting the inspection data, compared to the period of 4 September

2010 to 22 February 2011.

I t was unknown at the time, but the newly developed process had to be further modified, because of issues of

scale, process and data quality issues of the original approach.

The processes developed are discussed in the following sections and include the work components, the initial

process approach and the current process approach.

5

2.2 THE WORK COMPONENTS

The CCTV inspection work can be broken into six main components.

Item Description Provider

Instruction The CCTV work instruction compiled by SCIRT is received by City Care.

The work package may consist of a handful; tens or even hundreds of

individual pipe assets located within a particular catchment or sub

catchment.

SCIRT

Allocation The work package is allocated to one or more CCTV inspection teams to

undertake the work. Work package details consist of the pipe asset details

e.g. location, asset number, physical attributes e.g. pipe diameter, length,

material, connecting assets.

City Care

Inspection The work package details are transferred by the CCTV contractor into

either a computer based program (Cleanflow or Wincan) or to hard copy

log sheets for each pipe asset to be inspected. Where required the pipe

assets are prepared for inspection by jetting and sucking debris from the

pipes.

The pipes are inspected using CCTV with the surveyor logging and grading

observations on damage or other issues that may be affecting the pipe

asset performance.

The two main outputs are:

Video footage of the pipe survey.

Pipe inspection logs, into a computer software program, template.

Microsoft Excel spread sheets or onto paper assessment sheets.

CCTV

contractor

Review /

Assessment

The video of the pipe and the log sheets are provided to reviewers who

undertake reviews of the pipe video and the log sheets compiled by the

surveyors. The reviewers review the logs based on the video and make

recommendations on the pipe condition e.g. relay, reline, replace.

City Care

Reporting Reporting of work undertaken and the status of each work package in the

overall process is reported weekly.

City Care

Data provision The video footage is catalogued and the pipe information is provided for

loading into SCIRT InfoNet database.

City Care

Table 1: “The main components of the CCTV inspection process components”

2.3 INITIAL PROCESS APPROACH

The initial process approach in dealing with SCIRT CCTV inspection requests corresponding to the components

of Table 1 are discussed in the following section.

2.3.1 INSTRUCTION

Work instructions received from SCIRT generally included a spreadsheet and GIS based map of the of the pipe

assets to be surveyed as determined by the design teams. The format of the spreadsheets was often inconsistent,

although not an issue at the time, as the systems used for validating the data request were inherently manual.

6

2.3.2 ALLOCATION

On receipt of the work instruction, a number of different spreadsheets containing previous CCTV inspections

were manually inspected. Using this information, the work instruction would undergo refinement as necessary –

i.e. if relevant and current CCTV of pipes in the package already existed, this information was used in the review

process. The work instruction was then programmed into the workload and issued to the CCTV contractor for

inspection work.

2.3.3 INSPECTION

The CCTV contractor entered key asset information into their video capture system and undertook the survey.

Inspection information corresponding to the NZPIM was entered into an Excel spreadsheet based on a template

document.

Figure 1: “Template spreadsheet for collecting CCTV pipe inspection information”

In excess of 3,500 Microsoft Excel spreadsheet files were created using this process during this period. The

resultant spreadsheet files were stored in an Internet based Microsoft Sharepoint solution for later retrieval.

7

2.3.4 REVIEW AND ASSESSMENT

A number of third parties were used to review and assess the CCTV video and corresponding log sheets, based on

the NZPIM. The recommendations from the reviews were used to determine the action required at the individual

pipe asset level – i.e. reuse or replace.

Although the NZPIM was used for determining the action on the pipe inspected, minimal quality auditing was

undertaken on the reviews.

2.3.5 REPORTING

Weekly reporting on the status of the work instructions was undertaken manually. Progress by the CCTV

inspection crews was provided by telephone or in person and manually entered into a Microsoft Excel

spreadsheet delivered to SCIRT by email.

2.3.6 DATA PROVISION

The original pipe inspection data was available to SCIRT via the Internet from the Microsoft Sharepoint site

containing the spreadsheets.

The defect data was loaded in SCIRT’s pipe asset management system from either the CCTV contractors

CleanFlow software used to undertake the inspections or from the template Microsoft Excel spreadsheets.

2.4 CURRENT PROCESS APPROACH

2.4.1 NEED FOR CHANGE

With the scale of the CCTV operations escalating and after working with the resulting data, it became obvious

the Initial Process Approach had significant limitations, with a reliance on considerable manual data entry.

CCTV surveyors needed to enter the allocated work package pipe asset information into their software

systems or onto hard copy sheets.

Duplication of effort – i.e. entry of the same pipe asset information by surveyor and reviewers.

Data entry mistakes, with minimal data validation processes.

Data standards and quality were difficult to be adhered to / enforced.

Misplacing of pipe inspection data.

Time delays in providing reviewed information.

Issues in being able to extract data from thousands of reviewed files for use in other software systems.

The limitations of using separate Microsoft Excel spreadsheets and lack of system enforced integrity

checks needed to be a focus area of improvement.

2.4.2 PROCESS DEFINITION

The issues associated with the initial approach, particularly around process, data collection and quality control

drove a change in the process approach.

The CCTV inspection process was reviewed, modified and documented.

Considerable effort was put into standardising data inputs and outputs.

Options were explored to apply automation and database systems to assist in the process.

The improved process approach in dealing with SCIRT CCTV inspection requests corresponding to the

components of Table 1 are discussed in Table 2 and the following sections.

8

Delivered by Outputs System(s)

SCIRT project

definition team

List of projects in priority order SCIRT Geographic

information system

(GIS)

SCIRT asset

team

SCIRT GIS team

INFONET

Work instruction asset listing with

map exported from SCIRT GIS or

INFONET in standard format.

SCIRT GIS

Project centre

InfoNet

SCIRT work

instruction

Electronic work instruction issued in

an Excel spreadsheet.

Project centre

City Care and

contractors CCTV

surveyors

Import work instruction into City

Care job management system.

Work assigned to CCTV team.

CCTV inspection undertaken.

Reviews and recommendations.

Data exported for InfoNet.

City Care’s job

management

system

SCIRT InfoNet team Damage reports.

Data for designers.

InfoNet

SCIRT GIS

Table 2: High level flowchart of the CCTV process

The key of the new process was the application of City Care’s existing database job systems to manage, validate,

report and supply the data associated with the CCTV work instructions. The CCTV module was written in

accordance with the NZ Pipe Inspection Manual 2006 to handle the collection of the CCTV inspection data.

2.4.3 INSTRUCTION

The main output of the process above is the CCTV work instruction (step 3), which is the development of a

work package in the form of an Excel spreadsheet used to program the CCTV work to be undertaken. The base

of this detailed information is from the Christchurch City Council’s GIS system, containing all known pipe

assets.

As a standard format was now being supplied by SCIRT, an automated process was developed to load the

information into City Care’s job management system. Part of the process was the validation of the work

instruction from SCIRT against the master asset database (of all pipe assets) contained in the City Care job

management system.

9

2.4.4 ALLOCATION

With the work instruction in City Care’s job management system, the inspection work was scheduled by City

Care planners, and then assigned to the CCTV surveyors in the field using a cellphone based Internet connection.

The work processes were dynamic and seamlessly integrated into City Care’s job management system.

Figure 3: Screenshot of a pipe inspection job assigned to a CCTV inspection team

2.4.5 INSPECTION

CCTV surveyors recorded the pipe inspection information (in accordance with the NZ Pipe Inspection Manual

2006); using City Care’s job management system, either directly in the field or back at the office after the

inspection was complete. In addition to the pipe inspection information, images of pipe defects were recorded

and attached to the job.

10

Figure 4: “Observations inspection records of a single pipe asset”

2.4.6 REVIEW AND ASSESSMENT

City Care based reviewers use the City Care job management system CCTV module to audit the surveyors’

inspection and make adjustments to the survey where appropriate. The work of the surveyors and reviewers is

regularly audited to ensure consistency and quality standards are maintained.

2.4.7 REPORTING

The automated daily report indicates the status of the current work underway. It is possible to “drill down” to

pipe asset level to see the particular assets that have been captured to date. The report is generated “in real

time” and is a reflection of the status of the asset survey at the time the report is run. i.e. the surveyors will be

using the City Care job management system to record the inspection information, and when they “complete” a

job that information will be immediately reflected in the daily report.

The City Care CCTV management team has real time access to the report as well as SCIRT who use the internet

based job management system to view in real time the work package status / progress.

Figure 5: System generated daily report

2.4.8 DATA PROVISION

A significant output of the data captured through the surveying and the reviewing process is the “raw data” for

inclusion into InfoNET. InfoNet is used by SCIRT for damage assessment, data manipulation and reporting.

The City Care job management system holds the required information that would have previously been on paper

sheets, in Excel spreadsheets or from the surveyors’ computer software. The data collected is made available

once the job is “signed off” by City Care, in the format developed by SCIRT for direct import into InfoNet. In

addition, the data export, a ZIP” package of all documents associated to the job – i.e. photos / images is sent.

This exported information can be supplied as either a large batch of data or down to individual work packages.

11

3 CCTV CATALOGUE

The challenge of requesting new CCTV work in a project area is in understanding what CCTV inspection surveys

have been undertaken, and when. This knowledge helps refine the pipe assets requiring surveying and reducing

the duplication of effort and cost.

Initially there was not a definitive catalogue containing all the CCTV work undertaken to date. It was a time

consuming process to search the information sources to determine whether CCTV pipe inspection surveys had

already been done on the pipe assets identified in the job instruction. This meant CCTV pipe inspection could be

done on pipes that did not require it.

The issues encountered in compiling the single catalogue included:

The existence of multiple catalogues, formatted differently and/or containing different data.

CCTV inspections simply not been recorded i.e. sitting on or under.

Inspection of multiple assets on one row in the spreadsheet.

All fields for a record not being complete.

Data in fields simply incorrect due to data entry issues.

The first step was to merge the various catalogues into a single file and at the same time ensure that each record

was complete. In December 2011 the “Master” CCTV catalogue contained fewer than 10,000 records. As the

various catalogues were merged together and additional data “found”, the catalogue grew to over 30,500 records.

Figure 6: “A sample of the information contained in the CCTV catalogue”

On working with the data it became apparent that minimal quality control on the information entered into the

catalogue had been done, with common issues such as upstream and downstream identification numbers being

transferred and pipe asset IDs incorrect.

An automated process using Safe Software’s Feature Manipulation Engine (FME) was developed to validate at

pipe asset level that the asset IDs in the catalogue matched the asset IDs contained in CCC spatial data and to

ensure the key fields of Survey ID, type, survey date and survey name were populated. Where possible automated

approaches were used to update the catalogue records, however often manual work was required to fix the errors.

As well as the tests on the catalogue, additional automated tests were undertaken to validate the catalogue

against the hard copy records and DVDs containing the inspection detail and to validate the catalogue against

the information on the video file server and vice versa.

The result of this process increased the percentage match of records in the catalogue to pipe assets in the CCC

GIS system from around 75% to 99.6% and 96.7% for wastewater and stormwater respectively. The record

match improvement was obtained even with a considerable increase of records in the catalogue.

12

Figure 7: Graph of the improvement in key data attributes in the CCTV catalogue”

The CCTV catalogue is not 100% complete and miscellaneous data is being discovered constantly. However,

now the CCTV catalogue information is as up to date as possible and available in a single file. It can be used for

validating the information currently contained in InfoNet, making available via GIS viewers a mechanism to

“find” relevant video files / records and for data statistical analysis.

13

3.1 STATISTICS

3.1.1 WASTEWATER NETWORK

As at 13 July 2012 the Christchurch wastewater pipe network (over 100mm diameter) consisted of 1,990 kms.

Of the 18,081 CCTV wastewater pipe investigation surveys documented in the CCTV catalogue, 18,011 (99.6%)

matched to the CCC spatial database.

The 18,011 matching surveys were associated to 12,872 pipelines. This difference (5139) is due to multiple

surveys (two or more) being undertaken on the same pipe at different times after the earthquake events. The

length of pipe inspected during the 18,011 surveys was 1,099 km, carried out on 767 kms of unique pipes.

The 767 kms of unique pipes equates to 39% of the wastewater network of pipes over 100mm diameter having

being inspected.

Figure 8: Wastewater pipe network indicating CCTV undertaken in the period 04/09/2010 to 13/07/2012

14

3.1.2 STORMWATER NETWORK

As at 13 July 2012 the Christchurch stormwater pipe network (over 100mm diameter) consisted of 1,267 kms.

Of the 12,351 CCTV stormwater pipe investigation surveys documented in the CCTV catalogue, 11,943

(96.7%) matched to the CCC spatial database.

The 11,943 matching surveys were associated to 10,357 pipelines. This difference (1586) is due to multiple

surveys (two or more) being undertaken on the same pipe at different times after the various earthquake events.

The length of pipe inspected during the 11943 surveys was 342 kms, carried out on 291 kms of unique pipes.

The 291 kms of unique pipes equates to 23% of the stormwater network of pipes over 100mm diameter being

inspected using CCTV.

Figure 9: Stormwater pipe network indicating CCTV undertaken in the period 04/09/2010 to 13/07/2012

15

3.2 ON GOING WORK

Considerable time and effort has been committed to obtaining the CCTV inspections from various sources and

the rework involved in adopting a standard way of recording and reorganisation of the data. During this process a

number of sub-projects have been identified that are yet to be undertaken.

3.2.1 CATALOGUE DATABASE

The catalogue of CCTV inspections as at 13/07/2012 contains 30,432 records. This catalogue is a valuable

resource, containing the key information on when the pipe inspection was undertaken, the pipe asset number

and the reference to the physical folder containing paper records and the original DVD video footage. The

catalogue is a key element for obtaining CCTV inspection statistics and identifying the pipe assets inspected.

It has been identified the catalogue is of interest to a wide audience. However there are limitations on the Excel

spreadsheet approach, particularly around version control and the catalogue distribution. The other risk is that

the catalogue can be easily destroyed or data deleted without any automated audit trail in place.

A project is under scoping to move the catalogue from the Microsoft Excel spreadsheet to a SQL database with a

web based front end to allow multiple access to the catalogue from any location. The access to the database will

be controlled on a defined role basis – i.e. administrator, editor, viewer, guest.

A component of the change to the database approach for storing the catalogue information is to investigate the

automatic generation of a record in the database based on the information captured during the pipe inspection

process including the file name of the video record.

3.2.2 VIDEO FILE RENAMING

Since the beginning of the CCTV data capture project, the inspection videos have been randomly named. This

approach means it is difficult to associate the video files to pipe inspection data at an asset level. A draft file

naming specification has been developed, based on the unique asset name, and is to be implemented for future

pipe inspections.

A plan with options on the approach to rename the historical video files is being developed, including time and

estimates and the resourcing requirements to undertake the tasks.

3.2.3 ONE ASSET, ONE VIDEO

Generally the video files contain the inspection of multiple pipe assets in one file. Often the use of “chapters”

splits the video file into individual assets, although this is not a consistent approach. To enable the most

flexibility the video files need to be split to represent the individual pipe assets between manholes. This

approach would provide the most flexibility for managing the data and to allow the association to the individual

pipe assets in software applications such as InfoNET.

No scoping has been undertaken on this sub-project, which is likely to be a significant undertaking. The project

will be further complicated by the header information sometimes being incorrect because it refers to the wrong

asset (due to incorrect data entry), resulting in the requirement for further research.

3.2.4 VIDEO FILE COMPRESSION

The CCTV inspection video carried out to 13/07/2012 occupies over 14 terabytes of disk space. The video files

are generally uncompressed, resulting in significant files sizes, making sharing of the video files a challenge and

providing over the Iinternet a significant challenge.

Preliminary work has been undertaken looking at the file compression options. Compressions of up to 20 times

with little obvious loss in quality for evaluating the video are encouraging, although the approach needs to go

through an approval process. It is likely the video compression project would be carried out in at the same time

as the “One asset, one video” project discussed above. The benefits of compressing the video files would mean

16

that it would be easier to provide the data to third parties, data storage (and backup) requirements would not be

as onerous and providing the files over the internet via a web based application should be achievable.

4 FINDINGS

The scale of the CCTV activities undertaken in Christchurch is unprecedented in New Zealand. Managing this

scale of activity, the resulting data outputs and client expectation is a significant challenge. As the project has

progressed, p rocesses, systems and data quality have been under continuous improvement. The improvement

component of the project is a result of the contribution from those involved, including management, designers,

CCTV field crews, reviewers, administration and software designers.

4.1 FIELD OPERATIONS

The Christchurch CCTV inspection project is larger than anybody anticipated. The scale of the CCTV field

operations is unprecedented in New Zealand and with this scale there are significant issues that are “not normal”

in CCTV operations.

These issues include:

The logistics and management of the inspection work allocation and physical operations of the 30

CCTV crews that are operating.

The transient nature of the project, generally operating over the entire City.

Environmental and health and safety compliance.

The high level of attention that needs to be given to traffic management, as a CCTV inspection project

may be in an area for only a few hours.

The impact of infiltration and aquifers on the CCTV inspection works, resulting in increased jetting and

sucking requirements.

Photograph 5: “CCTV daytime operations” Photograph 6: “CCTV night time operations”

CCTV is a costly undertaking, and one of the future goals is to reduce the inspection costs using alternative

assessment methods. One of the alternatives being investigated by SCIRT is a desktop analysis tool that predicts

pipe condition by using data inputs from other damage indicators or data sources including CCTV pipe inspection

surveys completed to date, liquefaction resistance index, pipe material and age, road surface damage, proximity

to waterways and number of connections. The principle is to look for trends between damage indicators and

completed CCTV inspections, to predict trends across the city without having to undertake CCTV inspections.

4.2 DATA

The data collected from the CCTV inspections is a critical input to the SCIRT teams designing the civic

infrastructure replacement. Issues other than the scale of the information collected include:

17

The requirements of a consistently high standard of information, to the New Zealand Pipe Inspection

Manual (NZPIM) standards.

Regular audits of the field data collected and in management of the data.

The timeliness of information.

The use of the data by multiple audiences e.g. SCIRT designers, CCC, possible future use by insurance

companies and other interested parties.

Managing the volume of data collected; over 30,000 pipe inspections to date have resulted in excess of

14 terabytes of video footage.

5 CONCLUSIONS

Good decisions are based on quality process and information. However, in projects such as the Christchurch

CCTV project, often the right people are not involved in project initiation and planning, resulting in design and

process decisions being compromised. Process and quality can get sacrificed because it is seen as slowing the

project down, particularly in emergency situations, when the opposite is in fact true as the project scales and

becomes more significant. This lack of planning generally results in considerable rework, which is costly in time

and money, especially when the different consuming audiences are not considered.

The use of the City Care job management system (with information stored in an SQL database), reinforces this.

The benefit of changing the CCTV process to use the job management system meant that work instructions

were managed in a robust way and the amount of manual data entry significantly reduced. Not only could the

data be validated and analysed in the database, it could be easily reported or exported to any audience.

It is accepted that the scale of the CCTV project was unseen. The knowledge and experience gained through the

CCTV project demands relooking at the approach to data collection projects, of which the CCTV is a significant

one, whether the project is “business as usual” or an emergency event. The considerable rework associated with

the project, and that will continue into the future, is a result of project initiation of the required management

frameworks not being in place, insufficient planning and lack of attention to quality. However, these issues have

been addressed at various times as the project progressed.

It is important not only to recognise the difference between operational and strategic data collection projects,

but also to have the vision to identify when an operational project will turn into a strategic one. Planning “on

the fly” is sometimes acceptable, but not in a project such as this, where such high data standards are demanded

and where the outputs are so critical to other processes.

Table 3 contains the elements to be considered when designing a data collection project.

Project

Type

Audience Direction Design Quality Implementation Delivery

Operational Self/Team Team Team Manual Paper /

Spreadsheet

Reports

Tactical Business *Steering

group

**Design

team

System based

checks

Personal

database

Reports /

Internet

< I

mpo

rtan

ce

Strategic Anybody *Steering

group

**Design

team

System based

checks

Corporate

Database

Reports /

Internet

Table 3: “Considerations when designing a data collection project”

*Steering group. Too often individuals dictate direction based on their own thoughts or preferences, rather than

the requirements of the business. Representatives on the team should have diverse skills; technical as well as

senior management. As projects move towards strategic, the inclusion of experienced Information Technology

or data experts should be manadatory.

18

**Design Team. As the project scales it becomes more important to include Business Analysts and Business

Intelligence resources with expertise in understanding and analysing data. As well as the internal team designing

the systems, audience representatives of those using the outputs need to be considered.

19

During the CCTV project we often asked of ourselves and others; if we were to do the data collection

component of the project again, what would we do differently and what would be the focus? Table 4 discusses the

elements of a data collection framework.

Item Description

Requirements The answer to “what are the project requirements” is often “We don’t

know what we want”.

The answer is simple; with the correct people you have your best shot at

it. Use a design that will evolve i.e. has strong foundations that can be

built on. Involve people with specific Business Intelligence skills, ideally

with statistical experience, and representatives of the audiences that will

be consuming the data and information. Some members of the audience

will want to know time and cost, others will want detail for analysis and

others will just want to have access.

Direction Dependent on the project type (refer Table 3), the formation and make-

up of the group responsible for the project direction is essential.

Members of the group need to be removed from the day to day operations

of the project. There is a significant difference between working in a

project versus working on it. Those working on the project are tasked

with having a business (and other interested parties) view, guiding the

project, whereas those working in the project are completing the work.

The roles and responsibilities of the group need to be clearly defined and

representatives should include:

Management

Operations

Business analysts

Data analysts

Audience representatives

Design People are often asked to design processes or systems without the

necessary background or experience to undertake the exercise. The

selection of the right people, working to the direction of the steering

group, is essential.

The correct choice for system for solution delivery is essential to enable

projects to scale. For example the default usage of Excel to manage data

in strategic projects needs to be more carefully considered.

Data collection needs to be undertaken to the lowest level component

that allows the easy aggregation or manipulation of data.

Quality The focus on data quality is often overlooked. The reliance on people to

ensure quality is maintained needs to be reduced, with more focus on

system based checks. Where possible people need to be released from the

quality checks and data validation to let well designed systems undertake

this work.

Where people are involved the importance of training and regular

refreshers should not be overlooked. Training along with regular auditing

ensures quality standards are adhered to.

20

Item Description

Change process Any change to a part of the process could have dramatic effects on

subsequent steps. Once the process is agreed, there needs to be a formal

process implemented before further changes are made.

The change process should cover:

The scope / elements that it applies to.

The representatives of the “change” group.

The process for change to take place.

When implementing change to existing processes or development of new

processes, it is essential that pilots or test areas are carried out before full

implementation. This should involve all parties associated with the

change and a formal acceptance process.

Table 4: “Elements of a data collection framework”

The Christchurch infrastructure CCTV project has come a long way, the result of a lot of hard work and tears by

staff in SCIRT, City Care and of course the significant number of contractors used to obtain the CCTV

inspection data.

From a City Care perspective the approach to future data collection involving the scale of this project would be

considerably differently, but we hope we will never be presented with an event such as the Christchurch

earthquakes again.

ACKNOWLEDGEMENTS

Special thanks to Richard Walklin, City Care’s Manager of the CCTV field operations and data management

program. Richard provided background information, photographs for this paper and the accompanying

presentation and the critical review of this paper.

Barry Clark, Brendon Silcock, Bjorn

Triplow, Phillipa Webb, and Chris Wrathall

of City Care

For taking the time out of their busy days to review this paper

and the accompanying presentation.

City Care’s Pages Road Administration

Team

The unsung heroes of the CCTV project, putting right the issues

discovered during the project.

Justin Hall, Chris Taylor, Dave Heiler,

Hywel Lewis, Ben Pritchard, Keith Hastings

and Todd Davis SCIRT

Assistance in defining and implementing the processes to deliver

the best possible information as an input into the infrastructure

rebuild design process.

Nigel Pitts and City Care software

development team

The software programming brains behind the current City Care

CCTV data collection software.

Hua Zhong, City Care Providing CCTV inspection figures and statistics for the paper

and the time lapse video for the paper presentation and

technical peer review

The CCTV pipe inspection contractors For their commitment to the task at hand in truly trying

circumstances and working conditions.

21

REFERENCES

SCIRT (Stronger Christchurch Infrastructure Rebuild Team). Retrieved 31 July, 2012 from

http://strongerchristchurch.govt.nz/about/structure

CERA (Christchurch Earthquake Recovery Authority). Retrieved 27 July, 2012 from http://cera.govt.nz/

Innovyze. Retrieved 31 July, 2012 from http://www.innovyze.com/products/infonet/

GLOSSARY

CAM Customer Asset Management. City Cares application for managing job requests.

CCC Christchurch City Council.

CCTV Closed Circuit Television .

CEM Customer Event Manager. City Cares internet based application that provides access to the

data in CAM. A custom module was developed to provide reports and information

CERA Canterbury Earthquake Recovery Authority.

Civic

Infrastructure

Includes roads, fresh water, wastewater and stormwater network systems.

Cleanflow CCTV inspection software used for recording defect information.

FME Safe Software Feature Manipulation Engine. Software to analyse and manipulate large amounts

of data.

GIS Geographic Information System.

InfoNet InfoNet is a purpose-built Infrastructure Management System (IMS) for water distribution,

wastewater collection and stormwater networks ensuring informed, swift and cost-effective

decision making both for day-to-day operational management and for long-term network

planning.

NZPIM New Zealand Pipe Inspection Manual.

Project Centre SCIRT’s software application for managing project information in one place.

SCIRT Stronger Christchurch Infrastructure Rebuild Team.

WinCan CCTV inspection software used for recording defect information.