WTAnalyser - Filtration v3.2

WTAnalyser (Filtration)Version 3.2Developed by the Australian Water Treatment AllianceCompatible with MS Excel 2003, 2007 & 2010

Step 1.Setting up a SCADA Trend for Downloading Turbidity Data

If a filter is off line for a lengthy period the best option is to just not include the trend on the initial trend page.

Step 2. Exporting the Data

1. Highlighting the column.2. Right mouse click3. Select Format cells4. On the Number tab select Custom5. Scroll down and select d/mm/yyyy h:mm6. Click OK

Step 3. Preliminary Inspection of the DataBefore running the WTAnalyser package it is important to have a good look at the data file, even if it is many hundreds of rows.

Pathogens in drinking water pose a significant hazard to public health. The management of pathogens requires the establishment of appropriate barriers (preventive measures) in the water treatment process. Conventional water treatment affords two barriers to bacteria and most viruses however only one barrier to the protozoan pathogens Cryptosporidium and Giardia. The only barrier to these protozoans in a conventional WTP is the presence of well operated media filters. To demonstrate ongoing production of safe drinking water, long term analysis of filter performance is necessary. The WTAnalyser (Filters) allows such long term analysis and presentation of the data in a form that makes it suitable for reporting to senior management Boards or Regulators.

The WTAnalyser allows you to download and extract data from SCADA trends for detailed analysis. Since the trending system samples at short time intervals, very large amounts of data make up the trends for turbidity. The first step is to extract a subset of the data for further analysis. The WTAnalyser package takes an excel spreadsheet with all the downloaded data from the SCADA system and then takes the maximum (highest risk) turbidity value for each 2 hour period. This results in 12 data points for each 24 hours of operation. The analysis package can also reject data where turbidity data should not be analysed, for example when the plant is off line.

Set up a trend on the SCADA system with the filtered water turbidity data for each filter to be analysed. A suitable period for regular downloading is two to four weeks; however it depends to some extent on your exporting system. Some systems limit the number of rows of data that can be downloaded regardless of the time interval chosen. For example many systems allow 900 rows but some may be as little as 300 rows. So if the analysis period is 4 weeks and there are only 300 rows available the interval between data points will be about 2.25 hours which is really not sufficient data for good analysis. Try to choose a download interval that will give about 15 minute intervals between samples. You may need to involve your IT staff to help you to download more rows of data. The best systems allow you to select whatever interval you want and then download however many rows that are necessary and is only limited by the maximum size of an excel spreadsheet.

If the plant runs intermittently then include on the trend page the plant inflow. The package will only extract data when the plant is running.

Each SCADA system will vary somewhat but all have the ability to save the displayed trend to a data file, usually a csv file. Consult your SCADA manual for instructions. For Citect systems there is a small floppy disc icon at the bottom or top of the screen that allows you to save the file.

Once saved as a csv file, open it and save it as an excel spreadsheet. The columns of data will have a SCADA tag for a label. Rename the columns with simple English descriptors e.g. date, time (sometimes date_time), plant inflow, filter 1, filter 2, raw water, settled water.

In most cases very little modification is required. The only column that does sometimes require manipulation is the datetime column. In most cases this can be done by

The data in the column should now be a simple date and time format that is easy to recognise. If this doesn’t work seek help from your IT staff or someone who is good with Excel. The data will be processed if the date and time are presented in seperate columns or as one date_time column.

Have a look at the date and times. Are they displayed with the same time interval between rows e.g. 5 minutes, 10 minutes etc? WTAnalyser will analyse data at variable intervals but for best possible analysis the data should be evenly spaced. Quite a few SCADA systems only record data when the data starts to change, so for example you may have data sampled at 1 hour or 2 hour intervals but when a turbidity event occurs the data may be sampled at 1 minute or less intervals. This will make the data set look worse than it is. Although the analysis produced by WTAnalyser will still generally be OK, it is far better to set up the SCADA system to always sample at a set interval. The interval should be no longer than 10 minutes and preferably 1 minute intervals if the memory on the system is big enough.

Are there any gaps, where are they? (WTAnalyser can handle gaps but not in the time or date column.)

Are there any blank columns? (WTAnalyser cannot handle data with blank columns between columns with data.)

Step 4. Analyse the DataFollow the prompts in the WTAnalyser spreadsheet and generate the statistical data and have a good look at it.

Have a look at the date and times. Are they displayed with the same time interval between rows e.g. 5 minutes, 10 minutes etc? WTAnalyser will analyse data at variable intervals but for best possible analysis the data should be evenly spaced. Quite a few SCADA systems only record data when the data starts to change, so for example you may have data sampled at 1 hour or 2 hour intervals but when a turbidity event occurs the data may be sampled at 1 minute or less intervals. This will make the data set look worse than it is. Although the analysis produced by WTAnalyser will still generally be OK, it is far better to set up the SCADA system to always sample at a set interval. The interval should be no longer than 10 minutes and preferably 1 minute intervals if the memory on the system is big enough.

Are there any zeroes, where are they? (WTAnalyser can handle zeroes BUT it thinks they are real data which they may not be. Lots of zeroes in your data set will make the data look better than it really is.)

Step 1.Setting up a SCADA Trend for Downloading Turbidity Data

If a filter is off line for a lengthy period the best option is to just not include the trend on the initial trend page.

Step 2. Exporting the Data

Step 3. Preliminary Inspection of the DataBefore running the WTAnalyser package it is important to have a good look at the data file, even if it is many hundreds of rows.

Pathogens in drinking water pose a significant hazard to public health. The management of pathogens requires the establishment of appropriate barriers (preventive measures) in the water treatment process. Conventional water treatment affords two barriers to bacteria and most viruses however

. The only barrier to these protozoans in a conventional WTP is the presence of well operated media filters. To demonstrate ongoing production of safe drinking water, long term analysis of filter performance is necessary. The WTAnalyser (Filters) allows such long term analysis and presentation of the data in a form that makes it suitable for reporting to senior management

The WTAnalyser allows you to download and extract data from SCADA trends for detailed analysis. Since the trending system samples at short time intervals, very large amounts of data make up the trends for turbidity. The first step is to extract a subset of the data for further analysis. The WTAnalyser package takes an excel spreadsheet with all the downloaded data from the SCADA system and then takes the maximum (highest risk) turbidity value for each 2 hour period. This results in 12 data points for each 24 hours of operation. The analysis package can also reject data where

Set up a trend on the SCADA system with the filtered water turbidity data for each filter to be analysed. A suitable period for regular downloading is two to four weeks; however it depends to some extent on your exporting system. Some systems limit the number of rows of data that can be downloaded regardless of the time interval chosen. For example many systems allow 900 rows but some may be as little as 300 rows. So if the analysis period is 4 weeks and there are only 300 rows available the interval between data points will be about 2.25 hours which is really not sufficient data for good analysis. Try to choose a download interval that will give about 15 minute intervals between samples. You may need to involve your IT staff to help you to download more rows of data. The best systems allow you to select whatever interval you want and then download however many rows that are

If the plant runs intermittently then include on the trend page the plant inflow. The package will only extract data when the plant is running.

Each SCADA system will vary somewhat but all have the ability to save the displayed trend to a data file, usually a csv file. Consult your SCADA manual systems there is a small floppy disc icon at the bottom or top of the screen that allows you to save the file.

Once saved as a csv file, open it and save it as an excel spreadsheet. The columns of data will have a SCADA tag for a label. Rename the columns with simple English descriptors e.g. date, time (sometimes date_time), plant inflow, filter 1, filter 2, raw water, settled water.

In most cases very little modification is required. The only column that does sometimes require manipulation is the datetime column. In most cases this

The data in the column should now be a simple date and time format that is easy to recognise. If this doesn’t work seek help from your IT staff or someone who is good with Excel. The data will be processed if the date and time are presented in seperate columns or as one date_time column.

Have a look at the date and times. Are they displayed with the same time interval between rows e.g. 5 minutes, 10 minutes etc? WTAnalyser will analyse data at variable intervals but for best possible analysis the data should be evenly spaced. Quite a few SCADA systems only record data when the data starts to change, so for example you may have data sampled at 1 hour or 2 hour intervals but when a turbidity event occurs the data may be sampled at 1 minute or less intervals. This will make the data set look worse than it is. Although the analysis produced by WTAnalyser will still generally be OK, it is far better to set up the SCADA system to always sample at a set interval. The interval should be no longer than 10 minutes and preferably 1

Are there any gaps, where are they? (WTAnalyser can handle gaps but not in the time or date column.)

Are there any blank columns? (WTAnalyser cannot handle data with blank columns between columns with data.)

Step 4. Analyse the DataFollow the prompts in the WTAnalyser spreadsheet and generate the statistical data and have a good look at it.

Have a look at the date and times. Are they displayed with the same time interval between rows e.g. 5 minutes, 10 minutes etc? WTAnalyser will analyse data at variable intervals but for best possible analysis the data should be evenly spaced. Quite a few SCADA systems only record data when the data starts to change, so for example you may have data sampled at 1 hour or 2 hour intervals but when a turbidity event occurs the data may be sampled at 1 minute or less intervals. This will make the data set look worse than it is. Although the analysis produced by WTAnalyser will still generally be OK, it is far better to set up the SCADA system to always sample at a set interval. The interval should be no longer than 10 minutes and preferably 1

Are there any zeroes, where are they? (WTAnalyser can handle zeroes BUT it thinks they are real data which they may not be. Lots of zeroes in your

Go to Menu page to start analysing data.

WTAnalyser (Filtration)Version 3.2Developed by the Australian Water Treatment AllianceCompatible with MS Excel 2003, 2007 & 2010

Operating Guide

The tool uses plant flow to identify when the plant was in service (if plant flow is not available the tool will assume that the plant is always in service).

The tool can handle empty cells except in the time/date columns; it can handle missing hours or days in the data, but not blank time/date cells.

Please ensure there are no empty columns between columns of data in the data set; there can be empty columns at the end of the data set.

There are three steps to follow when using the tool:

Step 1 - Enter/Edit Plant Information

Set the start date for your dataset.Enter the plant maximum flow. This will be used to calculate the plant off flow (5% of maximum).

Step 2 - Get Data

Step 3 - View Results

Follow on-screen prompts to manipulate graphs.Graphs can easily be exported to Word document.

Click on Buttons to start

This tool has been developed to assist you in analysing the performance of filters at your plant. It extracts data from other MS Excel or csv files. Once data is selected it is automatically processed into a graphical and statistical format that will tell you how your filters are performing.

You can use any time interval in your data up to two hours (e.g. 1 minute, 10 mins, 30 mins, 1 hour, 2 hours). The tool will select the maximum turbidity value in every 2 hour period for each filter.

The tool can analyse up to a maximum of 12 months of data. Data can be added in weekly or monthly intervals to build up a 12 month dataset. The maximum number of rows the tool can process in one go is 26,500 (this is equivalent to 6 months of data at 10 minute intervals).

Enter plant information in the user form (click Button 1 below). This information will be used to create a new file. The file will be labelled with your utility and plant name and the date of the first line of data in your dataset.

If you want to combine two data sets, open up the version of the tool saved with the first dataset in it. Press Button 1 and check the merge data box (don't change the start date). Press Button 2 and select new data. Note: Data must be in exactly the same format as the original file i.e. all columns must be entered in the same order and the date/time format must be the same.

Click on Button 2 (below) and the tool will look for any open excel workbooks. Follow the on-screen prompts to select the right data.

The tool will work with all date/time stamp formats that are recognised by Excel. If your format is not recognised ask you IT department to help. You can use a single column with both date and time included or separate date/time columns.

You can also load up other data such as inlet turbidity or clarified water turbidity to help diagnose problems.

The results are displayed in a number of ways to help you identify filter issues. Click on the sheet tabs to view.

(1) Enter Plant Info (2) Get Data

Example Data Set

XYZ Performance SummaryPeriod: 1-Jan-11 Until: 31-Jan-11Selection:

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

0

0.5

1

1.5

2

2.5

Filte r 1 - Cumula tiv e fre q ue n c y d is tribu tion of wa te r tu rb id ity

Cumulative Frequency Distribution

Tu

rbid

ity

(N

TU

)

0

0.02

0.04

0.06

0.08

0.1

0.12F ilte r 1 - Wa te r tu rb id ity time s e rie s

Da

ily

Ma

xim

um

Tu

rbid

ity

(N

TU

)

0.00 -

0.05

0.05 -

0.10

0.10 -

0.15

0.15 -

0.20

0.20 -

0.25

0.25 -

0.30

0.30 -

0.35

0.35 -

0.40

0.40 -

0.45

0.45 -

0.50

0.50 -

0.60

0.60 -

0.70

0.70 -

0.80

0.80 -

0.90

0.90 -

1.00

1.00 -

1.50

1.50 -

2.00

0%

20%

40%

60%

80%

100%

Filte r 1 - Wa te r turb id ity h is togra m

Turbidity Histogram

Pe

rce

nta

ge

(%

)

0.00 -

0.05

0.05 -

0.10

0.10 -

0.15

0.15 -

0.20

0.20 -

0.25

0.25 -

0.30

0.30 -

0.35

0.35 -

0.40

0.40 -

0.45

0.45 -

0.50

0.50 -

0.60

0.60 -

0.70

0.70 -

0.80

0.80 -

0.90

0.90 -

1.00

1.00 -

1.50

1.50 -

2.00

0%

20%

40%

60%

80%

100%

Filte r 1 - Wa te r turb id ity h is togra m

Turbidity Histogram

Pe

rce

nta

ge

(%

)

XYZ Performance SummaryPeriod: 1-Jul-09 Until: 31-Jul-09Selection:

0

0.02

0.04

0.06

0.08

0.1

0.12

Da

ily

Ma

xim

um

Tu

rbid

ity

(N

TU

)

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

0

0.5

1

1.5

2

2.5


Tu

rbid

ity

(N

TU

)

0.00 -

0.05

0.05 -

0.10

0.10 -

0.15

0.15 -

0.20

0.20 -

0.25

0.25 -

0.30

0.30 -

0.35

0.35 -

0.40

0.40 -

0.45

0.45 -

0.50

0.50 -

0.60

0.60 -

0.70

0.70 -

0.80

0.80 -

0.90

0.90 -

1.00

1.00 -

1.50

1.50 -

2.00

0%

20%

40%

60%

80%

100%

Turbidity Histogram

Pe

rce

nta

ge

(%

)

0.00 -

0.05

0.05 -

0.10

0.10 -

0.15

0.15 -

0.20

0.20 -

0.25

0.25 -

0.30

0.30 -

0.35

0.35 -

0.40

0.40 -

0.45

0.45 -

0.50

0.50 -

0.60

0.60 -

0.70

0.70 -

0.80

0.80 -

0.90

0.90 -

1.00

1.00 -

1.50

1.50 -

2.00

0%

20%

40%

60%

80%

100%

Turbidity Histogram

Pe

rce

nta

ge

(%

)


0

0.02

0.04

0.06

0.08

0.1

0.12

Da

ily

Ma

xim

um

Tu

rbid

ity

(N

TU

)

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

0

0.5

1

1.5

2

2.5


Tu

rbid

ity

(N

TU

)

0.00 -

0.05

0.05 -

0.10

0.10 -

0.15

0.15 -

0.20

0.20 -

0.25

0.25 -

0.30

0.30 -

0.35

0.35 -

0.40

0.40 -

0.45

0.45 -

0.50

0.50 -

0.60

0.60 -

0.70

0.70 -

0.80

0.80 -

0.90

0.90 -

1.00

1.00 -

1.50

1.50 -

2.00

0%

20%

40%

60%

80%

100%

Turbidity Histogram

Pe

rce

nta

ge

(%

)

0

0.02

0.04

0.06

0.08

0.1

0.12

Da

ily

Ma

xim

um

Tu

rbid

ity

(N

TU

)

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

0

0.5

1

1.5

2

2.5


Tu

rbid

ity

(N

TU

)

0.00 -

0.05

0.05 -

0.10

0.10 -

0.15

0.15 -

0.20

0.20 -

0.25

0.25 -

0.30

0.30 -

0.35

0.35 -

0.40

0.40 -

0.45

0.45 -

0.50

0.50 -

0.60

0.60 -

0.70

0.70 -

0.80

0.80 -

0.90

0.90 -

1.00

1.00 -

1.50

1.50 -

2.00

0%

20%

40%

60%

80%

100%

Turbidity Histogram

Pe

rce

nta

ge

(%

)

0.00 -

0.05

0.05 -

0.10

0.10 -

0.15

0.15 -

0.20

0.20 -

0.25

0.25 -

0.30

0.30 -

0.35

0.35 -

0.40

0.40 -

0.45

0.45 -

0.50

0.50 -

0.60

0.60 -

0.70

0.70 -

0.80

0.80 -

0.90

0.90 -

1.00

1.00 -

1.50

1.50 -

2.00

0%

20%

40%

60%

80%

100%

Turbidity Histogram

Pe

rce

nta

ge

(%

)

0.00 -

0.05

0.05 -

0.10

0.10 -

0.15

0.15 -

0.20

0.20 -

0.25

0.25 -

0.30

0.30 -

0.35

0.35 -

0.40

0.40 -

0.45

0.45 -

0.50

0.50 -

0.60

0.60 -

0.70

0.70 -

0.80

0.80 -

0.90

0.90 -

1.00

1.00 -

1.50

1.50 -

2.00

0%

20%

40%

60%

80%

100%

Turbidity Histogram

Pe

rce

nta

ge

(%

)


0

0.02

0.04

0.06

0.08

0.1

0.12

Da

ily

Ma

xim

um

Tu

rbid

ity

(N

TU

)

0.00 -

0.05

0.05 -

0.10

0.10 -

0.15

0.15 -

0.20

0.20 -

0.25

0.25 -

0.30

0.30 -

0.35

0.35 -

0.40

0.40 -

0.45

0.45 -

0.50

0.50 -

0.60

0.60 -

0.70

0.70 -

0.80

0.80 -

0.90

0.90 -

1.00

1.00 -

1.50

1.50 -

2.00

0%

20%

40%

60%

80%

100%

Turbidity Histogram

Pe

rce

nta

ge

(%

)

0

0.02

0.04

0.06

0.08

0.1

0.12

Da

ily

Ma

xim

um

Tu

rbid

ity

(N

TU

)

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

0

0.5

1

1.5

2

2.5


Tu

rbid

ity

(N

TU

)

0.00 -

0.05

0.05 -

0.10

0.10 -

0.15

0.15 -

0.20

0.20 -

0.25

0.25 -

0.30

0.30 -

0.35

0.35 -

0.40

0.40 -

0.45

0.45 -

0.50

0.50 -

0.60

0.60 -

0.70

0.70 -

0.80

0.80 -

0.90

0.90 -

1.00

1.00 -

1.50

1.50 -

2.00

0%

20%

40%

60%

80%

100%

Turbidity Histogram

Pe

rce

nta

ge

(%

)