Shaken, not stirred – comparing lab and field filtering for TSS

3 October 2008

Analytical Methods and Quality Assurance Workgroup

Elgin Perry, Ph.D.Statistics consultantandBill RomanoMD Dept. of Natural Resources

A little background

• DHMH currently analyzes samples collected at 53 non-tidal stations that are part of Maryland’s CORE/Trend network (TSS is analyzed at the Cumberland lab for 17 of these stations)

• In July 2005, the sampling protocol changed for nutrients from analyzing whole water samples to analyzing filter pads

• This change did not include TSS, which is still analyzed as a whole water sample that is filtered in the lab at DHMH or Cumberland

A potential problem

• DNR is planning to change the TSS sampling protocol to match what is now done for our tidal program, field filtering for nutrients and TSS

• One would think that the analysis of TSS is so simple that even a statistician could do it

• Experience with the change from DHMH to CBL with the Maryland tidal program and recent problems with the Virginia main Bay program have shown that analysis of TSS is not so straightforward

• Subtle differences in how samples are handled and analyzed have resulted in step trends in the data

Yet another experiment(more work for the AFO and DHMH)

• Compare field vs. lab filtered TSS• Field staff collected extra water at the

53 stations in March and July• DHMH supplied pre-weighed filter

pads for all stations• All field-filtered pads were sent back

to DHMH for analysis• 17 whole water samples were sent to

Cumberland and 36 were sent to DHMH, which is the current protocol

Comparison of lab and field filtered TSS for March

0

100

200

300

400

500

600

700

800

900

Stations

TS

S (

mg

/L)

Field filtered

Lab filtered

storm event

Comparison of lab and field filtered TSS for July

0

5

10

15

20

25

30

35

ANT0203

CON018

0

POT23

86

NBP0326

YOU09

25

CCR0001

LYO

0004

SAV0000

GEO000

9

PAT0285

NPA0165

GUN025

8

JON01

84

DER0015

NBP0103

TOW

0030

WIL

0013

ANA0082

POT11

84

CAC0031

MON01

55

POT14

72

POT15

96

ANT0044

CAC0148

MON05

28

PXT0809

Station

TS

S (

mg

/L)

Field filtered

Lab filtered

Difference between March lab and field filtered TSS

plotted against the mean concentration of the two methods

(lab filtered minus field filtered)

Difference between July lab and field filtered TSS

plotted against the mean concentration of the two methods

(lab filtered minus field filtered)

Difference between TSS replicates for March

(difference calculated as lab filtered minus field filtered)

Difference between TSS replicates for July

(difference calculated as lab filtered minus field filtered)

Why this seems confusing (to us)

• The March difference plot only shows large differences associated with high concentrations

• This is supported by the March WMRL box plot, which had high concentrations of TSS because of recent storms

• It may have been an “order effect” because the lab bottles were filled after the filter pad bottle

Confusion (continued)

• TSS concentrations were much lower in July and we expected the bias to go away

• The July difference plot shows a positive bias in the data (lab filtered exceeds field filtered)

• The July box plot shows a reversal in the bias, with no bias at WMRL and a small positive bias at DHMH

• The mean difference for March and July data is 3.9 mg/L, March only 6.9 mg/L and July only 1 mg/L

Comparing different data sets

• CV split sample among sub-samples - 21%• CV split sample among labs - 37%• CV for DHMH lab replicates - 32%• CV for lab and field filtered - 44%• The CV for methods is a bit larger than the

others and larger than we expected. Note, these CVs measure total variation = to bias + noise.

Questions to consider:

• Should we continue the paired samples study?

• Are there modifications of the experimental design that might yield better information?

Photographer: William Frece