static-content.springer.com10.1007... · Web viewNIST plasma fatty acids NIST collaboration Part of...

Supplementary Material

For

“Quality assurance and quality control processes: A metabolomics community

questionnaire”

Warwick B. Dunn1, David Broadhurst2, Arthur Edison3, Claude Guillou4, Mark R. Viant1, Daniel W.

Bearden5* and Richard D. Beger6*

1 School of Biosciences, University of Birmingham, Birmingham, B15 2TT, UK

2 School of Science, Edith Cowan University, Joondalup 6017, Perth, Western Australia

3 Department of Genetics, University of Georgia, Athens, GA 30602-7223, USA

4 Institute for Health and Consumer Protection, Systems Toxicology Unit, European Commission - Joint

Research Centre, Italy.

5 Chemical Sciences Division, Hollings Marine Laboratory, National Institute of Standards and

Technology, Charleston, SC 29412 USA

6 National Center for Toxicological Research, US Food and Drug Administration, 3900 NCTR Road,

Jefferson, AR 72079, USA

*Corresponding authors:

Rick Beger: [email protected]

Dan Bearden: [email protected]

of 41

We received 97 individual responses from 84 institutions covering NMR, LCMS, GCMS, and other analytical technologies in all fields of metabolomics. There was a vast range of responses concerning the use of QA and QC approaches. The DQTG QA/QC questionnaire showed that QA and QC use was not uniform across metabolomics labs and there is a need to establish minimum standards in the use of QA and QC measurements and reporting in metabolomics.

Question 1. Laboratory Identification (please complete all fields; use N/A if necessary)

Answer Options Response Percent

Response Count

Laboratory Principal Investigator Name 100.0% 97Laboratory Institution 100.0% 97Laboratory Department/Division 100.0% 97Laboratory City/Town 100.0% 97Laboratory State/Province 100.0% 97Laboratory ZIP/Postal Code 100.0% 97Laboratory Country 100.0% 97

answered question 97skipped question 0

Question 2. How many years of experience does this group have in metabolomics?


Response Count

None 0.0% 00 - 2 9.3% 92 - 4 19.6% 194 - 6 15.5% 156 - 8 12.4% 12> 8 43.3% 42


of 41

Question 3. Respondent Identification


Response Count

Respondent Name 100.0% 97Respondent Email Address 100.0% 97Respondent Phone Number 100.0% 97


Question 4. Respondent Position


Response Count

Principal Investigator/Group Leader 36.1% 35Clinician 0.0% 0Staff Scientist 14.4% 14Technician 3.1% 3Postdoctoral Researcher 19.6% 19PhD Student 18.6% 18Masters Student 0.0% 0Undergraduate Student 0.0% 0Other (please specify) 8.2% 8


Question 5. How many years of experience does the Respondent personally have in metabolomics?


Response Count

None 0.0% 00 - 2 11.3% 112 - 4 27.8% 274 - 6 16.5% 166 - 8 13.4% 13> 8 30.9% 30


of 41

Question 6. Are you a current Metabolomics Society Member?


Response Count

Yes 80.4% 78No 19.6% 19


Question 7. What type of work does your laboratory do?


Response Count

Primarily biological/chemical laboratory 23.7% 23Combination of biological/chemical laboratory AND data processing/bioinformatics 70.1% 68Primarily data processing/bioinformatics 5.2% 5None of the above (please specify) 1.0% 1


Question 8. In what area(s) of science are you currently applying metabolomics (select all that apply)?

Plant Science 33.0% 32Nutritional Science 33.0% 32Environmental Science 22.7% 22Clinical and Medical Science 64.9% 63Microbiology & Parasitology 23.7% 23Toxicology 35.1% 34Drug development and discovery 15.5% 15Animal health and model systems 33.0% 32Bioinformatics and computational biology 33.0% 32Systems biology 45.4% 44Other (please specify) 7.2% 7


of 41

Question 9. What type of samples do you work with (check all that apply)?


Response Count

Cell based 70.1% 68Biofluids 78.4% 76Tissues 72.2% 70Other (please specify) 10.3% 10


Question 10. What type(s) of species do you work with (check all that apply)?


Response Count

Microbial 42.3% 41Plant 34.0% 33Mammalian 61.9% 60Human 76.3% 74Fish 19.6% 19Insect 16.5% 16Bird 5.2% 5Invertebrate 14.4% 14Other (please specify) 7.2% 7


Question 11. What type of analytical metabolomics experiments are you currently performing (select all that apply)?

Targeted analyses 73.2% 71Untargeted analyses 87.6% 85Metabolic flux analysis 20.6% 20Nuclear magnetic resonance (NMR) spectroscopy 34.0% 33Liquid chromatography mass spectrometry (LC-MS) 82.5% 80Gas chromatography mass spectrometry (GC-MS) 49.5% 48Capillary electrophoresis mass spectrometry (CE-MS) 6.2% 6Direct-infusion mass spectrometry (DIMS) 16.5% 16None 2.1% 2Other (please specify) 6.2% 6


of 41

Question 12. How many biological samples are analyzed in a single typical biological study across one or multiple analytical batches?

1 - 50 23.7% 2351 - 200 50.5% 49201 - 500 8.2% 8501 - 1000 8.2% 81001 - 5000 2.1% 2Greater than 5000 3.1% 3Not Applicable 4.1% 4


Question 13. Approximately how many samples has the Laboratory analyzed in the last 12 months?

1 - 100 5.2% 5101 - 1000 25.8% 251001 - 5000 37.1% 365001 - 10000 14.4% 14More than 10,000 11.3% 11Not Applicable 6.2% 6


Question 14. Do you have a standardized in-house training program for metabolomics?


Response Count

Yes 35.1% 33No 64.9% 61


of 41

Question 15. Who is responsible for conducting the in-house training?


Response Count

Professional staff 48.5% 16Postdocs/Graduate students 36.4% 12Other (please specify) 15.2% 5


Question 16. Are periodic checks of actual practice conducted to maintain training levels?


Response Count

Yes 72.7% 24No 27.3% 9


Question 17. If the above question is answered "Yes", then who is responsible for conducting the checks?

Self check 16.7% 4Professional staff 58.3% 14Postdocs/Graduate students 20.8% 5Other (please specify) 4.2% 1


of 41

Question 18. Do you have written Standard Operating Protocol (SOP) documentation for sample collection?


Response Count

Yes 89.7% 78No 10.3% 9


Question 19. If the answer above is 'Yes', then what is the source of the protocol(s)?


Response Count

Direct from literature 4.5% 3In-house developed 58.2% 39Both 37.3% 25


Question 20. Please give the literature reference for sample collection.

NMR-based metabolomic analysis of plants Hye Kyong Kim, Young Hae Choi1 & Robert Verpoorte; As per clients samples Dunn et al., Nature Protocols, 2011 Literature search Sample collection SOP depends on the type of project. one example can be found here

http://www.pnas.org/content/112/17/E2120 Bruker SOP Development and validation of a standardized protocol to monitor human dietary exposure by

metabolite fingerprinting of urine samples. Favé G, Beckmann M, Lloyd AJ, Zhou S, Harold G, Lin W, Tailliart K, Xie L, Draper J, Mathers JC. Metabolomics. 2011 Dec;7(4):469-484

e.g., Sellick Nature Protocols Sample preparation, general sample guidelines for metabolon studies, metabolon brochure; http://www.nature.com/nprot/journal/v6/n7/abs/nprot.2011.335.html Polyomics does talk to collaborators on how to set up the experiment - case to case The Handbook of Metabolomics DOI 10.1007/978-1-61779-594-7 Dietmair S, Timmins NE, Gray PP, Nielsen LK, Krömer JO (2010) Towards quantitative metabolomics

of mammalian cells: Development of a metabolite extraction protocol. Analytical Chemistry 404: 155-164.

Fiehn, O., Wohlgemuth, G., Scholz, M., Kind, T., Lee, D. Y., Lu, Y., ... & Nikolau, B. (2008). Quality control for plant metabolomics: reporting MSI compliant studies. The Plant Journal, 53(4), 691-704.‐

of 41

Question 21. Do you have written SOP documentation for sample extraction/processing?


Response Count

Yes 89.7% 78No 10.3% 9




Response Count



Question 23. Please give the literature reference for sample extraction/processing.

Phytochemistry, v. 62, p. 949-957, 2003; HPLC-DAD method for metabolic fingerprinting of the phenotyping of sugarcane genotypes. Analytical methods

http://link.springer.com/article/10.1007/s00216-010-4139-0 Dunn et al., Nature Protocols, 2011 Literature search Beckonert et al., Nature Protocols 2, 2692 (2007); Xiao et al., Analyst 134, 916-925 (2009) Metabolomics identifies a biological response to chronic low-dose natural uranium contamination in

urine samples. Grison S, Favé G, Maillot M, Manens L, Delissen O, Blanchardon E, Banzet N, Defoort C, Bott R, Dublineau I, Aigueperse J, Gourmelon P, Martin JC, Souidi M. Metabolomics. 2013;9(6):1168-1180

http://dx.doi.org/10.1007/s11306-011-0377-1, http://dx.doi.org/10.1016/j.ab.2010.03.003 Smart, K. F.; Aggio, R. M. B.; VanHoutte, J. R. and Villas-Bôas, S.G. 2010. Analytical platform for

metabolome analysis of microbial cells using methyl chloroformate derivatization followed by gas chromatography-mass spectrometry (GC-MS). Nature Protocols 5: 1709 – 1729.

Nature Protocols - Dunn group - 2011 Folch, J., Lees, M. and Stanley, G.H.S., J. Biol. Chem., 226, 497-509 (1957); 2. Viant MR, Methods in

Molecular Biology, 358 229-245: Metabolomics: Methods and Protocol; 3. Wu H, Southam AD, Hines A, Viant MR, Analytical Biochemistry 372 (2008): 204-212.

of 41

Modification of Folch J, Lee M, Sloane Stanley GH. A simple method for the isolation and purification of total lipids from animal tissues. (1959) J Biol Chem 226; 497-509

http://www.nature.com/nprot/journal/v6/n7/abs/nprot.2011.335.html as in: http://pubs.acs.org/doi/abs/10.1021/ac2021823 doi:10.1038/nprot.2006.59; doi:10.1038/nprot.2007.95 J Proteome Res. 2010 Aug 6;9(8):4131-7. doi: 10.1021/pr100331j. NMR-based metabolomic analysis of plants doi:10.1038/nprot.2009.237 Oliver Fiehn papers Initially started with Rabinovitz for cells PLoS Pathog. 2006 Dec;2(12):e132, then modified Fiehn, O., Wohlgemuth, G., Scholz, M., Kind, T., Lee, D. Y., Lu, Y., ... & Nikolau, B. (2008). Quality

control for plant metabolomics: reporting MSI compliant studies. The Plant Journal, 53(4), 691-704.‐ Bligh and Dyer, J Biochem. Physiol. 1959.

of 41

Question 24. Do you have written SOP documentation for sample storage before and/or during analyses?


Response Count

Yes 52.9% 46No 47.1% 41




Response Count



Question 26. Please give the literature reference for sample storage.

Answer Options Response Count

6answered question 6

skipped question 91

Beckonert et al., Nature Protocols 2, 2692 (2007) Smart, K. F.; Aggio, R. M. B.; VanHoutte, J. R. and Villas-Bôas, S.G. 2010. Analytical platform for

metabolome analysis of microbial cells using methyl chloroformate derivatization followed by gas chromatography-mass spectrometry (GC-MS). Nature Protocols 5: 1709 – 1729.

Sample Preparation for Plant Metabolomics DOI 10.1002/pca.1188 Fiehn, O., Wohlgemuth, G., Scholz, M., Kind, T., Lee, D. Y., Lu, Y., ... & Nikolau, B. (2008). Quality

control for plant metabolomics: reporting MSI compliant studies. The Plant Journal, 53(4), 691-704.‐

of 41

Question 27. Do you have written SOP documentation for instrumental analyses?


Response Count

Yes 74.7% 65No 25.3% 22




Response Count



Question 29. Please give the literature reference for instrumental analyses.



skipped question 78 ISO 17025 Thermo Planet Orbitrap Literature search Beckonert et al., Nature Protocols 2, 2692 (2007) http://pubs.acs.org/doi/abs/10.1021/ac9019522 Assorted Fiehn Warwick B Dunn, David Broadhurst, Paul Begley, Eva Zelena, Sue Francis - McIntyre, Nadine

Anderson, Marie Brown, Joshau D Knowles, Antony Halsall, John N Haselden, Andrew W Nicholls, Ian D Wilson, Douglas B Kell, Royston Goodacre & The Human Serum Metabolome (HUSERMET) Consortium, Procedures for large - scale metabolic profiling of serum and plasma using gas chromatography and liquid chromatography coupled to mass sp ectrometry, Nature Protocols 6, 1060 - 1083 (2011)

as in: http://pubs.acs.org/doi/abs/10.1021/ac2021823 (valid for Thermo Exactive)

of 41

Wang et al. BMC Medicine 2013, 11:86 http://www.biomedcentral.com/1741-7015/11/86 (28 March 2013)

doi:10.1038/nprot.2006.59; doi:10.1038/nprot.2007.95 J Proteome Res. 2012 Dec 7;11(12):6231-41. doi: 10.1021/pr3008946. Epub 2012 Nov 26. Bruker Metabonomics training course Agilent user manual For GC-MS we use the Fiehn method, others we have our own. Misra et al., 2015 (In preparation) Agilent-Feihn GC-MS mass spectrometry Agilent MS manual or other instruments' manuals

Question 30. Do you have written SOP documentation for assessment of the quality of data from QC samples?


Response Count

Yes 51.7% 45No 48.3% 42




Response Count



of 41

Question 32. Please give the literature reference for QC sample assessment.



skipped question 82 ISO 17025 Literature search Dunn Nature Protocols PMID: 23240883 Warwick B Dunn, David Broadhurst, Paul Begley, Eva Zelena, Sue Francis - McIntyre, Nadine

Anderson, Marie Brown, Joshau D Knowles, Antony Halsall, John N Haselden, Andrew W Nicholls, Ian D Wilson, Douglas B Kell, Royston Goodacre & The Human Serum Metabolome (HUSERMET) Consortium, Procedures for large - scale metabolic profiling of serum and plasma using gas chromatography and liquid chromatography coupled to mass sp ectrometry, Nature Protocols 6, 1060 - 1083 (2011)

http://www.nature.com/nprot/journal/v6/n7/abs/nprot.2011.335.html We do have a procedure to visually check the QC samples - this is done for every batch that is run at

Polyomics. Wang et al. BMC Medicine 2013, 11:86 http://www.biomedcentral.com/1741-7015/11/86 (28

March 2013) Nature Protocols 6, 1060–1083 (2011) doi:10.1038/nprot.2011.335 Want EJ, Masson P, Michopoulos F, Wilson ID, Theodoridis G, Plumb RS, Shockcor J, Loftus N,

Holmes E, Nicholson JK. Global metabolic profiling of animal and human tissues via UPLC-MS. Nat Protoc. 2013 Jan;8(1):17-32.

AMDIS NIST Dunn et al, Nat Protoc, 2011

of 41

Question 33. Do you have written SOP documentation for deciding when QC data from instrumental analysis has failed and how to correct the instrumental data?


Response Count

Yes 33.3% 29No 66.7% 58




Response Count



Question 35. Please give the literature reference for QC sample decisions.

ISO 17025 Literature search we do have the QC check written and when to decide when the instrument has failed but not how to

correct, usually this falls on myself or the professional staff hired for this purpose. Procedures for large-scale metabolic profiling of serum and plasma using gas chromatography and

liquid chromatography coupled to mass spectrometry. Dunn WB, Broadhurst D, Begley P, Zelena E, Francis-McIntyre S, Anderson N, Brown M, Knowles JD, Halsall A, Haselden JN, Nicholls AW, Wilson ID, Kell DB, Goodacre R; Human Serum Metabolome (HUSERMET) Consortium. Nat Protoc. 2011 Jun 30;6(7):1060-83

I do not understand Q33. If instrument fails, there's no correction of data, but instrument repair/maintenance

Dunn Nature Protocols PMID: 23240883 Instrument manufacturer manuals Again, We do have a procedure to visually check the QC samples - this is done for every batch that is

run at Polyomics. Nature Protocols 6, 1060–1083 (2011) doi:10.1038/nprot.2011.335 Dunn et al., Nat Protoc, 2011

of 41

Question 36. Are you REQUIRED to meet any laboratory accreditation criteria from professional organizations or institutes?


Response Count

Yes 10.6% 9No 89.4% 76


Question 37. If the above answer is "Yes", what is the name of the accreditation agency?



skipped question 89 ISO 9001 Canadian Association for Laboratory Accreditation Banaras Hindu University CAP accreditation standards ISO Unsure University of Florida

Question 38. Are you VOLUNTARILY meeting or attempting to meet any laboratory accreditation criteria from professional organizations or institutes?


Response Count

Yes 25.9% 22No 74.1% 63


of 41

Question 39. If the above answer is "Yes", what is the name of the accreditation agency?

Accreditation Austria CLIA NATA ISO9000 ISO - Swiss accreditation Serive (SAS) Canadian Association for Laboratory Accreditation ISO-9001 Metabolomics Standards Initiative For targeted analyses we attempt to meet FDA requirements for LC-MS assays GLP FDA Not sure about this - I am not the head of the lab.... - but there are recent activities to document

better what is done for different projects conducted at Polyomics. Metabolomics Society Metabolomics Society Data Standards Initiative Unsure NIST

of 41

Question 40. Do instrument operators have to pass a certification test after training?


Response Count

Yes 21.2% 18No 78.8% 67


Question 41. If the above answer is "Yes", who is responsible for conducting the checks?


Response Count

Self check 23.8% 5Professional staff 57.1% 12Postdocs/Graduate students 4.8% 1Other (please specify) 14.3% 3


Other (please specify)

Whomever trained them, most often professional staff but some instruments are being trained by post-docs or grad students.

Staff of the manufacturer Unsure

Question 42. Do you have an ongoing Continuing Education requirement for operators or analysts?


Response Count

Yes 25.9% 22No 74.1% 63


of 41

Question 43. Do you have a protocol for independent review of quality-related results?


Response Count

Yes 29.8% 25No 70.2% 59


Question 44. Do you have a written protocol for QA review criteria?


Response Count

Yes 20.2% 17No 79.8% 67


Question 45. Who is responsible for conducting the QA review? (Check all that apply).


Response Count

Self check 54.8% 46Supervisor 35.7% 30Statistician or Bioinformatist 27.4% 23Professional staff 28.6% 24No QA review is done 16.7% 14Other (please specify) 4.8% 4



collective supervision during meeting reporting All data is publicly accessible at * and goes through an automated analysis. This is first QA all data

has to pass and enables users of the data to spot issues more readily partially done through quality management reviews Often in communication with head of the lab

of 41

Question 46. Do you apply quality control charts in quality assessments?


Response Count

Yes 39.3% 33No 60.7% 51


Question 47. Do you store project instrument data in a data archive?


Response Count

Yes 89.3% 75No 10.7% 9


Question 48. If the above answer is "Yes", what type of archive? (Check all that apply.)


Response Count

In-house archive 94.7% 71Public archive 17.3% 13Other (please specify) 1.3% 1



we are working on creating a public archive but are not there yet

Question 49. Do you store quality-related instrument data in a data archive?


Response Count

Yes 72.6% 61No 27.4% 23


of 41

Question 50. Do you validate your project sample measurements with: (Check all that apply).


Response Count

Repeated sample extraction and analysis 73.2% 60Repeated instrumental analysis on the same sample 86.6% 71Analysis of a set of samples separately collected over time 53.7% 44Sending samples for external extraction/analysis by a separate laboratory 8.5% 7Measure quantitative values with a complementary analytical measurement 24.4% 20No sample measurements validated 2.4% 2Other (please specify) 7.3% 6



correlation matrix analyses repeated instrumental analysis of a standards mixture study pooled sample, synthetic standard mixture sometimes we are adding standards in the sample, this is an efficient QC throughout the entire

project same material, stored at -80 °C, extracted and analyzed with each sample series Occasionally sending to external labs

Question 51. Have you ever participated in an interlaboratory comparison exercise?


Response Count

Yes 32.9% 27No 67.1% 55


of 41

Question 52. If the above answer is "Yes", please list the interlaboratory comparison exercise.



skipped question 71Response Text

As part of the NIH U24 metabolomics centers Ring Trial metaboring study (10.1007/s11306-014-0740-0) ERNDIM Many kind of interlaborator exercise. In metabolomics see: Martin, Jean-Charles, et al. “Can We

Trust Untargeted Metabolomics? Results of the Metabo-Ring Initiative, a Large-Scale, Multi-Instrument Inter-Laboratory Study.” Metabolomics 11, no. 4 (August 2015): 807–21. doi:10.1007/s11306-014-0740-0.

professional ring metabolon sample handling interlaboratory comparison MetabonomicHealth we have done some with several labs but never published Can we trust untargeted metabolomics? Results of the metabo-ring initiative, a large-scale, multi-

instrument inter-laboratory study. Martin JC, Maillot M, Mazerolles G, Verdu A, Lyan B, Migné C, Defoort C, Canlet C, Junot C, Guillou C, Manach C, Jabob D, Bouveresse DJ, Paris E, Pujos-Guillot E, Jourdan F, Giacomoni F, Courant F, Favé G, Le Gall G, Chassaigne H, Tabet JC, Martin JF, Antignac JP, Shintu L, Defernez M, Philo M, Alexandre-Gouaubau MC, Amiot-Carlin MJ, Bossis M, Triba MN, Stojilkovic N, Banzet N, Molinié R, Bott R, Goulitquer S, Caldarelli S, Rutledge DN. Metabolomics. 2015;11(4):807-821

Metabo-Ring NIST lipidomics 2014/2015, NIH ring trial 2015 NIST lipidomics currently ongoing NIST plasma fatty acids NIST collaboration Part of Imperial College interlaboratory reproducibility of both NMR and LC-MS platforms EU-FLAVO; EU-METAPHOR; LCMS profiling TOMATO extracts (with Germany, Israel, Japan) Metabolomics Ring Trial Initiative ERNDIM International NMR-Based Environmental Metabolomics Intercomparison Exercise ENVIRONMENTAL

SCIENCE & TECHNOLOGY Volume: 43 Issue: 1 Pages: 219-225 Analysis of NIST blood sample NMR vs MS platforms NIST

of 41

ABRF Study NIH common fund

Question 53. Would you be interested in participating in an interlaboratory comparison?


Response Count

Yes 47.6% 39No 11.0% 9I need more information 41.5% 34


of 41

Question 54. What types of QC materials do you routinely use in analytical measurements for metabolomics projects? (Check all that apply).


Response Count

We do not use any QC materials outside of the project experimental samples 2.5% 2Solvent blanks (spiked or unspiked) 86.3% 69Pooled project materials (spiked or unspiked) 82.5% 66Pooled materials prepared from another source than the current project (spiked or unspiked) 33.8% 27Standard Reference Materials (SRMs) 47.5% 38Amount of substance standards 46.3% 37Retention time standards 57.5% 46


Question 55. What do you use the QC materials for? (Check all that apply).


Response Count

Evidence of sample preparation consistency 80.0% 64Detection of inadvertent sample contamination during processing 58.8% 47Determination of column integrity for chromatography 76.3% 61Determination of correction parameters for chromatography 48.8% 39Calculation of mass spectral m/z calibration 42.5% 34Determination of m/z trueness (closeness to known m/z values) 47.5% 38Determination of absolute amount of substance trueness (closeness to known amount of substance) 21.3% 17Determination of relative amount of substance trueness (closeness to known relative amount of substance)

41.3% 33

Determination of m/z precision (precision of m/z values) 47.5% 38Determination of absolute amount of substance precision (precision of amount of substance) 27.5% 22Determination of identity 55.0% 44Qualitative comparison with previous internally obtained results 63.8% 51Retention Standards 53.8% 43


of 41

Question 56. Do you perform replicate extractions of experimental samples?


Response Count

Yes 58.8% 47No 41.3% 33


Question 57. If the previous answer was "Yes", then how are the replicate extracts analyzed?


Response Count

Individually 85.1% 40Pooled together 14.9% 7


Question 58. Do you perform replicate analytical measurements on a single extract/sample?


Response Count

Yes 68.8% 55No 31.3% 25


Question 59. Do you extract a blank so that the blank sample goes through the whole analytical process?


Response Count

Yes 87.5% 70No 12.5% 10


of 41

Question 60. If the previous answer was "Yes", when is the blank analyzed during the analytical run?


Response Count

Start and end of study 27.9% 19At regular intervals 44.1% 30Randomized through study 20.6% 14Other (please specify) 7.4% 5



Depends on study Start of Study Start and end and randomized only at the end to prevent LC column disruption Both at start and end and randomly

Question 61. Do you analyze QC materials to assess the retention time characteristics of a chromatographic column?


Response Count

Yes 80.0% 64No 20.0% 16


of 41

Question 62. Do you randomize samples in any purposeful way to avoid unintended correlation with instrument conditions?


Response Count

Yes 91.3% 73No 8.8% 7


Question 63. Do you perform any operations to reduce the possibility of sample-to-sample carry over?


Response Count

Yes 77.5% 62No 12.5% 10Not Applicable 10.0% 8


Question 64. Do you run periodic instrument condition checks?


Response Count

Yes 93.8% 75No 2.5% 2Not Applicable 3.8% 3


Question 65. Do you use Standard Reference Materials (SRMs) like NIST 1950 during metabolomics analyses?


Response Count

Yes 21.3% 17No 78.8% 63


of 41

Question 66. If the previous answer was "Yes", how often are the SRMs analyzed?


Response Count

Less than once a day 21.1% 4Once a day 26.3% 5Twice a day 5.3% 1More than twice a day 10.5% 2Other (please specify) 36.8% 7



Start of study With each of my runs Approximately every 10 samples throughout an assay Run SRMs when individual biological study is runnning samples are carried to other lab for metabolomics studies and analysis Interspersed with every batch of samples

of 41

Question 67. How do you report your quality control data in reports and publications? (Check all that apply).


Response Count

Precision measurement for each reported metabolite (or peak) 33.8% 27Range of measurements variables for a complete dataset (e.g. RSD) 45.0% 36Include QC data on a box plot 23.8% 19Principal Components Analysis (figure) 56.3% 45Principal Component Analysis (calculation of multidimensional variance) 26.3% 21Use QC data as a test set in a supervised multivariate model 12.5% 10Descriptive statement 56.3% 45Reference to adherence to (and passing) a particular published acceptance criteria 16.3% 13Do not report 13.8% 11Other (please specify) 6.3% 5



correlation matrix analysis comparing all features between samples and replicates Tabular format with %CV For the question of 54 to 67 it depedns on the experiment. For example when I do a broad survey

for discovery of new molecules then replicates may not be performed as the target molecules will be followed up with isolation and full structure elucidation and the function of the molecule will be assessed, whereas clinical project where statistics is needed will have multiple replicates. The questions above did not leave different options.

yet to publish none published so far that I know of

of 41

Question 68. What are the biggest issues in quality assurance that you currently face?




Written SOPs are maintained Formalization of protocols; Ensuring we have enough sample to create a pooled control. Having time to write protocols/SOPs

for all the different projects/sample matrices used. high fluctuation of analysts Consistency for all the process We do not have a formal certification for QA. Day to Day variability in process analyst training The lack of consistency and knowledge of staff withholding procedures that are in place; Lack of

knowledge of instrument operation by some users has lead to disintegration of columns, precipitation on source and on ion optics.....; Lack of recording and logging of instrument perform data. "

Finding time to provide adequate training to new users in the lab. unsufficient control on sample collection and storage when outside the lab. CERTIFICATION; TRAINING; DOCUMENTATIONS (SOPs) Long term stability of the analytical platform. We have studied extending over 10s of thousands of

samples running over multiple years. Finding a balance between time investment in quality assurance versus sample throughput.

Ensure traceability of data and quality assurance at the biological steps of the experiments. Collection and reporting of metadata and quality by operators in charge of the biological experiments. Lack of training and knowledge/understanding about risks of contaminations and sources of variability impacting Metabolomics studies.

Ph.D students are only at the lab for a limited period of time; Formation is quite long and they don't really appreciate the “regulatory” aspects

Do not have a guideline for those who are working in and are new to this field; Lack of standardized materials for tissue, lack of industry standard for QA including number of blanks, randomization, pooled sample analysis, etc; Sample preparation from a multitude of people preparing samples according to a SOP

Preliminary Review of Experimental Study Design Protocols - Study Protocols should be carefully reviewed by all members of a study team prior to final acceptance. Study design reviewers should include a diverse multidiciplinary team of scientists which including analytical chemists, biologists,

of 41

toxicologists, pharmacologists, microbiologists, bioinformatics and engineers (as appropriate), each having sufficient educational backgrounds and/or laboratory experience conducting GLP and/or QA/QC experiments. Pilot study experiments may be called for inorder to validate the performance of various aspects of the studies prior to full protocol acceptance.

Establishment of Validated Sample Acquistion Protocols - Biological specimens contain enzymes that may remain active after sample collection. Controlled experiments were recently performed to evaluate the outcomes of identical samples collected and processed for storage by various methods. The results have not yet come in but these types of experiments will be helpful to establish sample collection protocols designed to minimize and arrest sample changes resulting from differences in sample collection. Procedures must show an acceptable degree of stability without compromising the sample or end results. Procedures must also be practical for properly trained staff to perform proficiently and consistently. Sample acquisition protocols will need to be tested and validated for each specific tissue matrix. Other important considerations prior to sample collection include food and water consumption records, feeding time, activity, fasting status, daylight/darkness cycles, vial types used for sample collection, collection time, collection interval(s), subject stress level, etc. Additional considerations include minimum sample size requirments (multiple analyses are often required using different analytical platforms and techniques), minimum number of replicates required to obtain representative sample sets, effect of time on samples maintained at room temperature, the use of preservatives and various freezing techniques (ie. flash freezing with N2), and freezer storage conditions.

Sample Preparation for High Sample Throughput - Enzymes may reactivate when biological specimens thaw which could also lead to potential changes in sample composition. Thawing of frozen samples is not instantaneous, homogenous and may not be equivalent for each sample; solutes may crystalize, become sequestered, separate, or otherwise change within the matrix due to freezing and thawing effects. The amount of time and temperature that samples spend in the thawed state prior to sample processing, and during processing in preparation of the samples for instrumental analysis. Many metabolites have polar functional groups that require derivatization prior to analysis, particularly for GC/MS analysis. These reactions often perform best under dry condition. Solvents, therefore may need to be actively and verifiably dehydrated prior to use for optimal results. Several types of controlled experiments are needed to evaluate the effects of various thawing and processing conditions. Appropriate steps should then be taken to minimize each of these potential sources of variability. Protocols describing accepted sample preparation methods for each specific type of biological matrix are needed. With improvements in engineering and robotics, attempts should be made to develop accurate, automated, high-throughput sample preparation procedures whenever possible. Automated methods however should be adequately tested and shown to offer improved consistency over conventional methods prior to full implementation.

when samples are recieved from outside collaborators and preparation conditions are unknown it can be hard to evaluate/normalize

The commercial availability of the metabolite database is critically needed to identify unknowns based on the MS/MS data.

of 41

Perhaps this is not the quality assurance you were thinking off but my number one problem we spend most of our time on is ID of the molecules and the quality of these assignments. This is true for targeted analysis and untargeted analysis. For example lets say in a targeted analysis of a hexenoic acid. The MS/MS and retention time is very similar to the cis vs trans version as well and different locations of the double bonds etc.another example, I often, even with standards, cannot differentiate the type of sterol. I can say the detected molecule is a sterol based on the retention, fragmentation, even co-migration with standard but not exactly which one. One regiopositional isomer or steroisomer changes its function. In untargeted its even more of an issue. We can on average only annotate 1.8% of the data (i.e. say something about the chemistry) that is collected (of the spectra that should have sufficient information to get matches); There is no specific scoring function that give a statistical value of how correct the match is? If someone tells me its leukotriene, how do we know this is correct. This is a similar problem as in gene function annotation. Minor changes to a gene sequence can have profound biological consequences and change the function dramatically. There is no QA possible for this at the moment.

Define cocnlucsive standards in each trial with each partner, since a QC sample should refelct the trial population and sample volume of the trial should be adatpted to faciliate QC control in a proper way; Prepare a standard procedure, for QC with CV of intra &inter-batch measuremnt, outlier control, plausibility of measured metablite levels and so on wihtout loosing to much analytes.

formalization of protocols Changing goalposts: e.g. for collection and processing of plasma/serum samples previously the

protocol of Beckonert et al. Nature Protocols (2007) was employed. Now new recommendations have come out (Anal. Chem. 86, 9882 (2014)) that are in conflict with the Beckonert conditions, but only vague explanations as to why recommendations have been changed or what the advantage of the new conditions is.

Difficulties in establishing criteria to say “ok, the instrument is as performant as it was for last experiment”. For example, the intensities of the ions in the calibrant solution have changed considerably over the 7 years of use of our micrOTF-Q; Difficulties in standardizing our protocols: too depending on each particular project.

samples are carried to other lab for metabolomics studies and analysis Global strategy and review of QA for a range of analytical strategies with different types of

instruments. The availability of guidance in appropriate quality assurance techniques in metabolomics analysis

has increased since I began my program, but it would be helpful if a single guidance document was available rather than trying to aggregate information and possibly missing some.

accuracy of compound ID Training and continuing confirmation of operator and instrument performance training and documenting training on various instrument platforms and software; large number of

SOPs needed due to project-specific needs formalisation of protocols Pre-analytical processing steps like choice of sample tube, prolonged storage of blood or plasma,

and incubation at improper temperature can have negative effects on human plasma quality, which in turn influences analytical reliability and reproducibility of clinical results. It is especially important

of 41

for systems biology approaches, -omics methods, and biobanking, both to assure and control sample quality.

analyst training Instrument operators don't have any independent checks on instrument performance for

metabolomics samples. We don't use one standard method for LC-MS, due to many different matrices; this makes month to

month comparisons impossible. We don't really want to, because that would mean using the longest method in order to cover all sample types, doubling the analysis time needed for many samples. I have been running a separate external standard frequently to somewhat overcome this.

How should the measured quality data be used to compare studies and simply in review of the data. What does the quality data tell us and how should it be applied to studies?

No procedures to follow. Ensuring that operators are following SOPs Ensuring adequate compliance to protocols between different analysis operators. Certification of operators; Analyst training No true standardization across the community I am not the head of the lab - but based on discussions I would say:

o difficulties in getting collaborators to follow the guidelines for sample preparation and labelling - which can result in bad results or lots of confusion

not enough reference standards available formalization of protocols and analyst training None, since there are institutional quality assurance guidelines, inspections, etc. that guide all of our

laboratory work. As I am relatively new, I am still training in this area. We are currently developing more extensive

SOPs for current and future users. Identifying quality assurance goals that are appropriate for our laboratory. standardization of biological sample collection Periodical certification of instruments and operators. 1. Reference materials used for QA may not be representative for the inherent (biological ,

genetical,etcetera) variation within and composition of project samples: some specific project samples may therefor be out of the QA dynamic range, e.g. saturation effects. This can only be observed after the actual project sample analysis. 2. In case QA shows that instrument performance is different from previous experiment(s) that new project samples have to be compared with, but same performance cannot always be restored completely. 3. QA of project samples not always possible

Batch-to-batch normalization in large-scale studies where unusual tissues are analysed. Pre-processing steps for optimal metabolomic phenotyping in large-scale studies have been extensively discussed and mainly solved by multiple internal standards and pooled calibration sample strategies. If the tissue has some characteristic or specific metabolites it is not so easy to find the correct Quality Control samples. For example, skin has extremely complex ceramides, not present in other matrices. Although “batch-to-batch” normalization could be performed by pooled calibration samples, some additional challenges have to be considered. 1) Lack of commercial standards of skin

of 41

pool. 2) If prepared “in-house”, stability of the ceramides has to be taken into account. 3) Some ceramides are specific to skin, and therefore, different matrices cannot be used as potential pooled calibration samples to correct their “batch-to-batch” differences.

QA is both very time consuming and expensive to do - and critically does not contribute to an assessment of your output by the wider academic community.

>90% of the work in the lab is targeted analysis, which does not face the same challenges as non-target analysis, since we are running a set of standards to check out chromatography and performance of the instrument prior, in-between and after the samples.; Non-target is performed on a QTof MS and here we see that sensitivity vary during day to day basis and also during a sequence.

When you follow strictly QA protocol, you may loose a metabolite that had RSD e.g. 31%, but had high per cent of change between studied groups. We try to do comparisons also based on the whole data set and later check the metabolites which nad very high % of change but did not passed QA protocol.

Standardization and replication of sample prep methods. Communication with biological collaborators.

assessing what does quality assurance mean specifically to the project to be meaningful; how we will measure and report. to what standard?

ways to show the data/ data processing INTERLABORATORY CONTROL; CERTIFICATION - obtaining meaningful samples, with appropriate replication and experimental controls from

biologists (improved significantly in recent years); maintaining instrument performance (particularly MS) over long peroids

correction of instrument drift Challenging to keep QA for all the different platforms and assays that we perform in our

metabolomics center. Analyst training. We have just 2 people that are good in prepare samples. 1. Instrument conditioning and health of HPLC and MS. 2. Batch to batch variations. Huge RSDs for

some/ many vital metabolites. 3. Analyst to Analyst choice of workflows (tools, instrument, workflows, handling, hence a lot of subjectivity) 4. What is doable in top labs and cited in literature is not doable in my lab! (resource availability limits standardization of the whole process!). 5. At Postdoc scale professional training is far too little- not everyone can afford to, and not all labs starting 'metabolomics' provide training at appropriate levels. 6. Sample specific issues: liquid samples (water content can cause quantitative variations in milk, serum, blood etc.); difficult samples hard to macerate and extract; ions in sample of diverse origin and so on.

Consistency in analyst technique from person to person, consistent review of quality assessment data

Individual Standards for Untargeted metabolomics are expensive and hence can not be qualified or quantified with precision

Each person in our lab seems to have his/her own way of conditioning capillary for CE; There is no training session to use the instruments but we try & make errors on our own to learn in a hard way. There must be some discrepancy among us in terms of how we operate the instrument. Also, CE

of 41

instrumental condition changes with humidity and temperature in our lab, which makes it extremely difficult to assure the reproducible conditions.

ion suppression is always a problem using ESI Formalization of protocols, especially for larger clinical studies. Collection/preparation of samples and experimental design. Making sure that data integrity is intact when passing from original source to my hands for data

analysis. Original/Raw data sometimes difficult to obtain.

Question 69. What are the biggest issues in quality control that you currently face?




Lack of proper QC samples that properly mimic real samples and are used through the full sample prep and instrument analysis process.

Don't calibrate the equipment after the analysis. No major issues. Inadequate certified standards and matrix reference materials misidentification Time consuming. day to day variance in instrumentation consistency of sampling No authentic standards being run with batches to validate retention times and general method

performance. Cost and availability of isotope-labelled internal standards to allow quantitative analysis. accuracy of data extraction by vendor or open source software. INTERLABORATORY CONTROL How to use intelligent batch correction methods in large scale untargetted methods. Interpretation of ananlytical data including metadata form the biological steps that are often too

loose. Long term evaluation Columns that are used in the LC-MS. Lack of adequate isotopically labelled standards in analysis, especially lipids. Data review by qualified and trained persons Establishing Quality Control Acceptance Criteria is a big and challenging issue. Many types of quality

control measures can be implemented to provide data reviewers with some degree of acceptability

of 41

with regards to particular experimantal parameters. The challenging thing is to establish some limits or degrees of acceptability for particular experimental parameters that will reflect upon the degree of confidence in the data presented; The following are some points that I had drafted a little over a year ago that may provide some additional incite regarding QA/QC:

o Standards - Chemical standards are necessary for determining instrument signal response and to optimize detection of specific metabolites or derivatives, typically under ideal conditions when there are little or no interferences. Which specific standards are needed, how to prepare them and at what concentrations for a particular investigation should be described. Standard stability and the need for preservation measures should be evaluated. Standards, un-labelled or isotopically enriched, can be used to fortify the biological matrix or for preparing combination standards and synthetic controls. Standards, when compared with spiked samples are useful for determining recoveries, ion suppression and other matrix effects.

o Blanks - Appropriate Blanks are needed, especially “Process Blanks” which are treated like samples but do not contain the biological specimen. Process Blanks are an important parts of any analytical method development and validation process since they are necessary for determining the background signal(s) due to solvents, reagents, and the effects of processing. At least one Process Blank should be included along with each sample preparation batch. The actual number and frequency of Blank analyses to include as part of an analytical sample set should be addressed as well as acceptance or rejection criteria.

o Study Group Replicates - A minimum number of study group replicates (subjects within a specific dose/time framework) should be predetermined for statistical calculations and to allow for outliers.

o Samples - The minimum required sample size should be predetermined for each particular analytical platform. Vial sizes should be appropriate to the actual size of the aliquot to minimize excessive vial surface area contact. Tissue samples should be pre-weighed, homogenized and extracted with the same mass to solvent ratio. In some cases normalization may be required for variations in dilution or to compensate for variation in water or lipid content.

o Analysis Order - The biological specimens should be randomized prior to analysis. If all specimens cannot be prepared at one time, an equal number from each dose group should be included and randomized accordingly.

o Replicate Samples - Replicate portions of approximately the same size from the same biological specimen that are carried through sample preparation and analysis at the same time and in the same way. This is an important way to acquire data regarding sample to sample processing consistency. How frequently should replicate samples be prepared and analyzed?

o Replicate Injections - Replicate injections are multiple injections that are analyzed from the same sample vial, either in sequence or randomly during an analytical sequence.

of 41

This is a way to measure the consistency of the instrumental analysis. At what points and how frequently should replicate injections be made?

o Spiked Samples - The addition of a known amount of a compound to a sample that is expected to have this compound in a sample. Spiked samples are useful for determining recoveries and detection response due to matrix effects by comparison with un-spiked samples and pure standards. Accurate fortifications should be made to the original sample at biologically relevant concentrations and at the earliest practical stage of sample preparation. What compound(s), or classes of compounds, should specimens be spiked with and at what concentration(s) needs to be determined.

o Internal Standards (IS) - A substance or substances, usually isotopically labelled compounds, that are chemically similar but not identical to the analytes of interest, that are added in a constant amount to every sample, blank and standard. IS are used for calibration and normalization purposes. Labeled standards are generally expensive, even those that are commonly available, and much more so if special synthesis are required. Which internal standards are needed and at what concentrations needs to be determined. Unique detection and response curve linearity are important.

o Standard Reference Materials (SRMs) - SRMs consist of various types of samples that are similar to the sample matrix being analyzed. These may be homogenous pooled samples which that are representative of a study group, generally with at least some analyte species known with very high certainty. Include the source, matrix, and certifications. How frequently should SRMs be analyzed? SRMs, non-fortified and with various types of fortification, could be used as Quality Control Check samples for inter-laboratory validation. The types and sources of SRMs needed to be determined, the classes of compounds and the range of concentrations to be used for fortification needs to determined. Which SRMs are fortified and which are not should be unknown to the laboratory prior to analysis.

choice of statistical methodology/appropriateness to data collected There is no widely accepted quality control standards available, which can measure the quality of

metabolomics data from each individual laboratory, and can compare data from different labs. This depends on the experiment. In general QC when build in appropriately in an experiment will

allow you to make a rapid decision on using the data or not and repeating the experiment. In-process control, what are the criterrias; Avalaibility of standard metabolites reflecting the

equipment performace but not common in measured samples sample decay; handling by different persons Getting QC material collected at the point of collection (usually far away from the lab in time and

space, so not much control over procedures). XCMS: parameters setting is a huge issue. samples are carried to other lab for metabolomics studies and analysis Use of stable and pertinent QC samples that are not from each project. Information is lacking or difficult to find in the following areas:

o Practical information regarding appropriate number of samples per analytical run

of 41

o Appropriate use of pooled samples and blankso Appropriate techniques for correcting data for a single experiment that has been

processed and analyzed in different batches on different dayso Guidance on typical lifetimes of chromatography columns and other consumableso Guidance on preventative maintenance schedules to ensure data integrity from run to

run absolute quantifications; peak picking in large LCMS data sets Strict adherence to protocols lack of reported QC results and acceptance criteria in literature. No consensus on acceptable RSD for

global metabolomics; wide disparity in various processing software in the number of metabolite features detected (some software packages report a lot of noise which artificially inflates the number of metabolite peaks detected making inter-study comparisons difficult); lack of ion suppression consideration in quality control exercises

preparing suitable reference matarial None column to column comparison Problems with students not being aware of q-values and validation testing for their Metaboanalyst

results; Problems with lack of involvement by clinicians over sample collection; Problems when overseas collaborators with no mass spec experience tell co-supervised students that blanks are not necessary

The use to which the QCs are put has never been documented or formalised. I'm not sure all data handlers are using them correctly, or in some cases, at all.

The lack of available quality control standards. Other than the NIST plasma, other biofluid standards are lacking. Additionally, what type of standard would be best for tissue analyses. Pooled samples can be used but how are these used to compare across studies?

No SOP to following and self discipline. Ensuring the instrument is at peak efficiency Comparison of data run using different instrument methods and across large numbers of batches. No true standardization across the community I am not the head of the lab - but based on discussions I would say:

o difficulties in getting the correct pooled sample and the correct amount from collaboratorso still working on a more formalized protocol - but it seems to be sample type dependent....

batch to batch variability operators and review of prior quality assessment data. Lack of appropriate standard reference materials. It would be great if there was an automated/software analysis of quality control that could provide

flags if something does not pass. Obtaining consistency in large runs - of hundreds of samples. small batch sizes between instrument cleaning leading to confounding of batch and biological

effects Variations in biofluids of clinical origin from time to time and in different patient groups.

of 41

1. Local shifts in retention times and (accurate mass) mz values, ion suppression effects during LC and peak intensities are mostly specific for the individual sample analyzed (matrix dependent). These effects may be different from the (pooled) QC samples and cannot be corrected properly without having standards for each compound (which by definition is true for untargeted approaches). 2. In case of small amounts of raw materials, impossible to perform QC of complete metabolomics pipeline using repeated extractions of same (preferably pooled) material.

Batch-to-batch normalization in large-scale studies Cost and having enough pooled samples to run enough QC samples throughout the analyses. This is related to non-target analysis, especially how to normalize data to eliminate instrumental

drift Some of the metabolites may be diluted in pooled QC samples, and fail to pass QA procedure. Confirmation of metabolomic results using secondary methods. Need more rigorous statistical

analysis of data from both on-going experiments and historical data. assessing what does quality control mean specifically to the project to be meaningful; how we will

measure and report. to what standard? best choice for quality control CARRY OVER; PRECISION; QUANTIFICATION Not too much of an issue unless there is a delay in analysis and samples need to be repeated; Since

there is no place for additional comments I would like to say that all the previous answers are highly dependent on the type of experiment we are running and protocols vary based on objectives. e.g. with large sample sets we will not do full replicates but select a random subset. In our studies biological replication is much more important than technical replication

correction of instrument drift Given the amount of data we generate on a couple hundred analysis projects/year, it's difficult to

perform detailed quality control on all sample sets beyond the basics of making sure the instrument is working well. We would like to automate this, with control charts and Westegard rules, but we don't yet have the resources.

How to chose the best QC for each project 1. Within batch variations. 2. Carry Over from LC and Column. 3. Lack of compound-class specific

multiple internal standards for normalization (using ribitol/ adonitol as internal standards for sterols or amino acids is funny on a GC-MS data!). 4. Choice of statistical tools and parameters. 5. Pool samples and extract once as large pool for QC versus. extract multiple samples and pool once as a large QC pool ? 6. Retention time shifts from Chromatography (GC or LC) is huge !!!

batch to batch reproducibility and ability to compare Analysis of large number samples from different tissue sources leads to possibility of column

contamination and artifact development Due to the instrumental variation and differences sample sizes for each project, we have rather

flexible cut-off values for %CV of QC. This might lead to accepting false data into the true value. changing column characteristics Additional measures beyond pooled QCs that can be used for LC and MS performance metrics, such

as internal standards that can be representative of the metabolome's performance, rather than a single metabolite

of 41

Lack of standard QC assessment. I am typically doing the data analysis to assess the quality of the data. This can be challenging with

different experimental designs. Some may have biological replicates, but no technical replicates. Some may have used internal standards. This heterogeneity makes it challenging to assess the performance and reliability of the data. Additional efforts to have a uniform set of QC measures will make the true assessment of data reliability much more straightforward.

Question 70. Do you want your name entered for the gift card drawing?


Response Count

Yes 80.5% 62No 19.5% 15


of 41

static-content.springer.com10.1007... · Web viewNIST plasma fatty acids NIST collaboration Part of...

Documents

Transcript of static-content.springer.com10.1007... · Web viewNIST plasma fatty acids NIST collaboration Part of...