Agenda Moderator: Betsy Grim, Director Quality Assurance, OPP

Agenda

Moderator: Betsy Grim, Director Quality Assurance, OPP

Part 1: Quality Basics

1:30 – 2:30 Louis Blume, GLNPO QAM

Quality Basics and Stages of Quality

2:30 – 2:45 Kevin Bolger, Region 5 QAM

Regulations Requiring Quality

2:45 – 3:00 Marcia Damato, GLNPO QA Team

Quality Implementation for grants

3:00 – 3:30 Break

Agenda


Part 2: Integrating and Documenting Quality in Project Management

Tom Huges, QAO, NHEERL and Cheryl Itkin, QAO, NCEA

3:30 – 3:50 Overview of “other” data

3:50 – 4:10 Documenting “other” data

4:10 – 4:20 Break

4:20 – 5:00 Hands on exercise

Agenda


3:30 – 4:30 Louis Blume, GLNPO QAM Quality Basics

2:30 – 2:45 Kevin Bolger, Region 5 QAM Regulations Requiring Quality

2:45 – 3:00 Marcia Damato, GLNPO QA Team Quality Implementation for grants

3:00 – 3:30 Break

Quality Management Basics Quality Management Basics and Advanced Stages of and Advanced Stages of

Quality SystemsQuality Systems

Louis Blume, U.S. EPA, Louis Blume, U.S. EPA,

Great Lakes National Program Office, Great Lakes National Program Office, Project Level QA Training for EPA QA ManagersProject Level QA Training for EPA QA Managers

May 14, 2009May 14, 2009

San Antonio, TexasSan Antonio, Texas

Louis Blume’s QA BackgroundLouis Blume’s QA Background

• QA Manager, US EPA – GLNPO, 1995 – present

• Chief of Planning, US EPA, Region 5, 1990 – 1992

• ORD Regional Scientist, US EPA, Chicago, IL, 1989 – 1990

• Soil Scientist, US EPA Office of Research and Development, Las Vegas, Nevada. 1985-1989

Relevant AccomplishmentsRelevant Accomplishments• Co-chaired EPA’s National Workshop on Graded Approaches for QA

• Presented QA Symposia in Canada and Russia

• QA Manager: National Acid Deposition Soil Survey, Direct/Delayed Response Project, US EPA

• M.S. Soil Science,Taxonomy / Morphology – University of Georgia

• B.S. Forestry – Michigan Tech University

We are here today to talk about

the advantages of a

Quality System,Versus a Quality Management

Plan.

Benefits of a Quality SystemBenefits of a Quality System

• Provides an effective management process that assures fact based environmental decisions

• Applies Quality principles to non-laboratory oriented data collection activities

– policy making, modeling, pollution prevention, economic assessments

• Allows flexibility on how you implement data collection activities

– can result in significant cost savings!

• Provides your organization a management structure that promotes continuous improvement

EPA Order CIO 2105.0: EPA Order CIO 2105.0: Major PointsMajor Points

• Defines requirements and implementation for quality systems which support U.S. EPA environmental programs directly or through extramural agreements

• Defines and broadens the scope of “environmental data” to include all input information to decision processes.

• Identifies QA roles and responsibilities for agency managers and staff

U.S. EPA Order CIO 2105.0:U.S. EPA Order CIO 2105.0:Policy and Program Requirements for the mandatory Agency-Wide Quality System

• Establishes policy and program requirements for the conduct of QA, for all environmentally-related measurements performed by or on behalf of U.S. EPA

Definition of Environmental DataDefinition of Environmental Data

“Any measurements or information that describe environmental processes, location, or conditions; ecological or health effects and consequences; or the performance of environmental technology. For EPA, environmental data include information collected directly from measurements, produced from models, and compiled from other sources such as data bases or the literature.”

EPA Order 5360.1 A2, May 2000

Quality SystemQuality System

• A structured, documented Management System describing an organization’s Policies, Policies, Objectives, Principles, Authorities, Objectives, Principles, Authorities, Responsibilities, Accountabilities, and Responsibilities, Accountabilities, and Implementation PlanImplementation Plan to ensure Quality in its Work Processes, Products and Services

• Provides an organization’s framework through: Planning, Implementing, Planning, Implementing, Assessing, Improving, DocumentingAssessing, Improving, Documenting

Quality Management Systems Quality Management Systems can/should be based on a can/should be based on a Graded ApproachGraded Approach

Quality system requirements are commensurate with:– Importance of work– Availability of resources– Unique needs of organization– Consequences of potential decision errors

GLNPO’s Graded Approach to

Quality System DocumentationQuality System DocumentationReplaced four-tiered category approach with new project categories:

1. State agencies2. Tribal grants3. Consortium grants4. Cluster grants5. Existing data/modeling6. Habitat/ecosystem restoration7. Ambient monitoring and research demonstration8. Volunteer monitoring9. Sediment assessment10. Pollution prevention and environmental education11. Repeating projects of similar scope

1 to 4: QMP 1 to 4: QMP usually requiredusually required

5 to 11: QAPP 5 to 11: QAPP usually requiredusually required

Quality Review vs. Peer Review Quality Review vs. Peer Review

• Quality Reviews of QAPPs focus on:– the process– the chance for success– technical details and validity of results, from a

reproducibility standpoint

• Peer Reviews are primarily driven by the discipline

Peer Review and Information Peer Review and Information Quality GuidelinesQuality Guidelines

GLNPO believes that an effective quality system ensures

– Implementation of Agency peer review policy (EPA document #100-B-00-001)

– http://www.epa.gov/osa/spc/htm/2peerrev.htm

– Adherence to OMB’s federal Information Quality Guidelines or IQGs (67 FR 8451)

– http://www.epa.gov/quality/informationguidelines/– http://www.whitehouse.gov/omb/inforeg/

agency_info_quality_links.html

http://www.epa.gov/osa/spc/htm/2peerrev.htm

What Does OMB Direct Agencies What Does OMB Direct Agencies Do to comply with the IQGs?Do to comply with the IQGs?

• Adopt a basic standard of quality for disseminated information

• Apply the principles of information quality to each step of EPA’s development of information- creation, collection, maintenance &

dissemination

• Enable affected persons to seek and obtain corrections to "Information* disseminated” to the public by filing a Request for Correction (RFC)

*Information, dissemination, distribution, *Information, dissemination, distribution, and influential information are further defined at and influential information are further defined at

http://www.epa.gov/quality/informationguidelines/http://www.epa.gov/quality/informationguidelines/

What is What is QualityQuality??

• Objectivity– Presentation: is accurate, clear, complete

and unbiased (transparency)– Substance: accurate, reliable, and unbiased

..based on sound statistical and research methods

• Utility– Usefulness of the information to its intended

users, including the public• Integrity

– Protection from unauthorized access or revision

Quality Control

QAPP, SOPProject

Quality Assurance

QAPP, SPP (DQO)Project

Quality System

QMP

QMPOrganization/Program

Quality SystemPolicy/Regulations

Specifications for Non-EPA Specifications for Non-EPA OrganizationsOrganizations

• EPA Requirements for Quality Management Plans (QA/R-2) EPA/240/B-01/002 March 2001

• Guidance for Developing Quality Systems for Environmental Programs (QA/G-1)EPA/240/R-02/008 November 2002

• EPA Requirements for QA Project Plans (QA/R5) EPA/240/B-01/003 March 2001

• Guidance for Quality Assurance Project Plans (QA/G-5) EPA/240/R-02/009 December 2002

QMPs vs. QAPPsQMPs vs. QAPPs

• Quality Management Plans reflect activities and policies common to all projects.

• Quality Assurance Project Plans reflect specific projects.

QMPQMP

QAPPQAPP QAPPQAPP

Quality Management Plan (QMP)Quality Management Plan (QMP)

The QMP describes quality requirements and procedures for activities that support the collection, use, and communication of environmental data and information, including:

– Project Planning– Grants, contracts and interagency agreements– Documents and records– Technical Tasks– Information Technology– QA Assessments/Audits and corrective action responses

See EPA Requirements for Quality Management Plans (EPA QA/R-2) http://www.epa.gov/quality/qa-docs/r2-final.pdf

Quality Assurance Project Plans Quality Assurance Project Plans (QAPPs)(QAPPs)

• Mandatory document for any data collection activity

• Describes how data and information will be collected, analyzed and assessed

• Describes the technical details and procedures to be used in the collection of data

See EPA Requirements for Quality Assurance Project Plans (EPA QA/R-5)

http://www.epa.gov/quality/qs-docs/r5-final.pdf

Required Elements in a QMPRequired Elements in a QMP

• Management and Organization• Quality System and Description• Personnel Qualifications and Training• Procurement of Items and Services• Documentation and Records• Computer Hardware and Software• Planning• Implementation of Work Processes• Assessment and Response• Quality Improvement

Required Elements in a QAPPRequired Elements in a QAPP

Guidance Assurance for Project Plans, EPA QA/G-5•Project Management

– Elements A1-A9

•Measurement/Data Acquisition– Elements B1-B10

•Assessment/Oversight– Elements C1-C2

•Data Validation and Usability– Elements D1-D3

Elements Requirements

PROJECT MANAGEMENT

A1 Title and Approval Sheet Title and approval sheet.

A2 Table of Contents Document control format.

A3 Distribution List Distribution list for the QAPP revisions and final guidance.

A4 Project/Task Organization Identify individuals or organizations participating in the project and discuss their roles, responsibilities and organization.

A5 Problem Definition/ Background 1) State the specific problem to be solved or the decision to be made. 2) Identify the decision maker and the principal customer for the results.

A6 Project/Task Description 1) Hypothesis test, 2) expected measurements, 3) ARARs or other appropriate standards, 4) assessment tools (technical audits), 5) work schedule and required reports.

A7 Data Quality Objectives for Measurement Data

Decision(s), population parameter of interest, action level, summary statistics and acceptable limits on decision errors. Also, scope of the project (domain or geographical locale).

A8 Special Training Requirements/Certification

Identify special training that personnel will need.

A9 Documentation and Record Itemize the information and records that must be included in a data report package, including report format and requirements for storage, etc.

MEASUREMENT/DATA ACQUISITION

B1 Sampling Process Designs (Experimental Design)

Outline the experimental design, including sampling design and rationale, sampling frequencies, matrices, and measurement parameter of interest.

B2 Sampling Methods Requirements Sample collection method and approach.

B3 Sample Handling and Custody Requirements

Describe the provisions for sample labeling, shipment, chain-of-custody forms, procedures for transferring and maintaining custody of samples.

B4 Analytical Methods Requirements Identify analytical method(s) and equipment for the study, including method performance requirements.

B5 Quality Control Requirements Describe routine (real-time) QC procedures that should be associated with each sampling and measurement technique. List required QC checks and corrective action procedures.


Elements Requirements

MEASUREMENT/DATA ACQUISITION (con’t)

B6 Instrument/Equipment Testing Inspection and Maintenance Requirements

Discuss how inspection and acceptance testing, including the use of QC samples, must be performed to ensure their intended use as specified by the design.

B7 Instrument Calibration and Frequency Identify tools, gauges and instruments, and other sampling or measurement devices that need calibration. Describe how the calibration should be done.

B8 Inspection/Acceptance Requirements for Supplies and Consumables

Define how and by whom the sampling supplies and other consumables will be accepted for use in the project.

B9 Data Acquisition Requirements (Non-direct Measurements)

Define the criteria for the use of non- measurement data such as data that come from databases or literature.

B10 Data Management Outline the data management scheme including the path and storage of the data and the data record-keeping system. Identify all data handling equipment and procedures that will be used to process, compile, and analyze the data.

ASSESSMENT/OVERSIGHT

C1 Assessments and Response Actions Describe the assessment activities needed for this project. These may include DQA, PE, TSA, MSR/ PR/RR

C2 Reports to Management Identify the frequency, content and distribution of reports issued to keep management informed.

DATA VALIDATION AND USABILITY

D1 Data Review, Validation, and Verification Requirements

State the criteria used to accept or reject the data based on quality.

D2 Validation and Verification Methods Describe the process to be used for validating and verifying data, including the chain-of-custody for data throughout the lifetime of the project.

D3 Reconciliation With Data Quality Objectives

Describe how results will be evaluated to determine if DQOs have been satisfied.

GLNPO Added Note: The above elements are minimum requirements for a QAPP, and therefore, should be used during the QAPP development.


Comparison of Required Elements Comparison of Required Elements in a QAPP versus a QMPin a QAPP versus a QMP

Elements of a Quality Assurance Project Plan (QAPP)

Elements of a Quality Management Plan (QMP)

Group A: Project Management ElementsGroup B: Data Generation and Acquisition ElementsGroup C: Assessment and Oversight ElementsGroup D: Data Validation and Usability Elements

1. Management and Organization2. Quality System and Description3. Personnel Qualifications and Training4. Procurement of Items and Services5. Documentation and Records6. Computer Hardware and Software7. Planning8. Implementation of Work Processes9. Assessment and Response10. Quality Improvement

EPA’s Quality SystemEPA’s Quality System

EPA Organizations• EPA Order 5360.1

• EPA Order 5360 (Quality Manual)

• Contracts Management Manual (EPA Order 1900)

• ANSI/ASQC E4-1994

Non-EPA Organizations• 48 CFR 46 (FAR 46) for contractors

• 40 CFR 30, 31, and 35 for assistance agreement recipients

• ANSI/ASQC E4-1994

Authorizing Requirements Authorizing Requirements

• ANSI/ASQC E4-1994/2004 is a national consensus standard designed specifically for quality systems applied to environmental data collection and environmental technology programs.

• Relates directly to ISO 9000 certification and over 14000 approved organizations

Quality System Requirements for Quality System Requirements for Extramural AgreementsExtramural Agreements

• Conformance to ANSI/ASQC E4-1994/2004 requires:

Documentation must be sufficient to describe the quality system

• ANSI/ASQC E4, Specifications and Guidelines for Quality Systems for Environmental Data Collection and Environmental Technology Programs

– Provides the basis for planning, implementing, documenting, and assessing a quality system

– Standard divided into 3 parts

Part A: Management Systems Part B: Environmental Data Part C: Environmental Technology

– Standard available from ASQ (www.asq.org)

American National Consensus American National Consensus StandardStandard

• Conform with ANSI/ASQC E4-1994/2004

• Document the organization's quality system (QMP) which should be approved prior to initiating environmental work

• Document the application of QA and QC activities to activity-specific efforts (QAPP) which should be approved prior to initiating environmental data collection

• Verify that products/services meet specifications

Basic Quality System Requirements Basic Quality System Requirements for Solicitations and Contractsfor Solicitations and Contracts

Is it Acquisition or Assistance?Is it Acquisition or Assistance?

Acquisition vs. Assistance EPA Order 5700.1

The basic difference between acquisition and assistance is the relationship between the parties.

• If an activity is funded to meet an internal EPA need, e.g., prepare a specific report to Congress or purchase furnishings for EPA staff, then an acquisition (contractual) relationship exists and EPA's direct procurement requirements (e.g., the Federal Acquisition Regulations) apply.

• If the activity is funded to primarily benefit someone other than the Federal government

QA Review Form:

• Required for all solicitations and contracts, work assignments, delivery orders, task orders, and for modifications that involve a significant change to the Statement of Work.

• Documents the necessary quality-related activities to be performed by the contractor.

• May document the quality-related activities to be performed by EPA.

• Standardizes how EPA Contracting Officer's Representatives and QA Managers communicate quality specifications to EPA Contracting Officers to ensure inclusion of the specifications in standard contract clauses.

Contract-Specific Quality System Contract-Specific Quality System ProceduresProcedures

Quality Requirements for Solicitations Quality Requirements for Solicitations and Contracts and Contracts

Objectives:• Identify Authorizing Requirements - where

to find specifics and updates

• Describe requirements and responsibilities that ensure QA in Contracts

Federal GovernmentFederal Government

Federal Acquisition Requirements (FAR)• Regulation - FAR 46.202-4 (February 1999)

– Applies to all Federal Agencies– Allows Federal Agencies to:

• Select a higher-level quality standard and• Tailor this standard to meet Agency-specific needs

• The type and content of the contract quality requirements depends on the nature of the work. It can range from inspection at the time of acceptance to requiring that the contractor implement a comprehensive program (quality system) for controlling quality.

• Contract quality requirements fall into four general categories, depending on the extent of quality assurance needed by the Government.

Contract Quality RequirementsContract Quality Requirements

Higher-level contract quality requirements. For complex or critical items or when the technical requirements of the contract require --

(1) Control of such things as work operations,in-process controls, and inspection; or

(2) Attention to such factors as organization,planning, work instructions, documentation control,and advanced metrology.

(FAR 46.202-4)

Contract Quality RequirementsContract Quality Requirements

• EPA's higher-level quality standard is usually American National Standard ANSI/ASQC E4-1994, "Specifications and Guidelines for Quality Systems for Environmental Data Collection and Environmental Technology Programs."

• Chapters 7 and 46 of EPA's Contracts Management Manual (CMM) contains EPA-specific policy and procedures:– Contract Clause– Step-by-Step Directions for COs and CORs– QA Review Form

EPA’s Adoption of FAR 46.202-4EPA’s Adoption of FAR 46.202-4

Not based on cost but on the type of work to be performed

Covered activities are the same as the activities covered by the EPA Quality System

Applies to simplified acquisitions except those under a purchase order (which should not be used for covered activities)

Application of Contract RequirementsApplication of Contract Requirements

The QA Review FormThe QA Review Form

The QARF provides information: • From the Contracting Officer's Representative to inform the

Contracting Officer what quality requirements should be included in a solicitation

• Enhance QA Manager oversight of the quality-related activities

QARF must be completed for: • Solicitations

• Contract Statements of Work (SOW)

• SOW for work assignments/delivery orders/task orders

• Modifications to existing work assignments/delivery orders/task orders that change the SOW

• Technical directives and actions that do not affect the work performed by the contractor (for example, incremental funding or time extensions) do not require a QARF.

Complete the QARFComplete the QARF

• Check "Yes" and complete the rest of the QARF if project involves: – collection, generation, use, and/or reporting of environmental

data;– design, construction, and/or operation of environmental

technologies;– development and/or use of models; or – other activities that need QA or QC requirements as

identified in your organization's QMP.

• If the work does not involve any of these activities, check "No," skip Quality Related Requirements and complete Signatures.

• Estimate the percentage of costs or level-of-effort allocated to the activities identified.

Stages of Quality System Stages of Quality System ImplementationImplementation

• Quality programs are not implemented with the stroke of a pen upon the approval of a Quality Management Plan

• Functional quality programs do not just happen - they evolve, typically after QMP approval

Value of Defining Stages Value of Defining Stages of Implementationof Implementation

• Provides a metric to measure success• Emphasizes the fact that good programs take

time and continuously improve• Illustrates a quality continuum• Establishes realistic expectations

1 2 3 4

Quality InfrastructureQuality Infrastructure

Stages of Quality System Stages of Quality System Implementation MatrixImplementation Matrix

• Columns = Degree of Implementation– Stage 1: 0-25%– Stage 2: 25-50%– Stage 3: 50-75%– Stage 4: 75-100%

• Rows = Characteristics, Actions, Attitudes, and Keys to Success

See handout orhttp://www.epa.gov/glnpo/quality/stagesofquality

Stages of QualityStages of Quality

11 22 3 3 4 4 BargainingBargaining True Value Added True Value Added

DepressionDepression Cost Savings Cost Savings

Baby StepsBaby Steps

Denial Denial Reluctance Compliance Reluctance Compliance Nirvana Nirvana**

TrainingTraining

** Ideal condition of perfect harmony and peace Ideal condition of perfect harmony and peace

1 – 5 years1 – 5 years

AcceptanceAcceptance

Kubler-RossKubler-Ross11

Stages of grief:

• Denial (No lab analysis in my project!)• Anger (I can not afford to write a QA plan, you write

the plan for me)• Bargaining (Not much guidance! They will publish the

report!, They have a QMP!, They did a QA plan a year ago, they’re world renowned!)

• Depression (I don’t care anymore)• Acceptance (Okay I see the value let’s get together

and talk about the key parts!)

1. Kubler-Ross, Elisabeth. 1969. On Death and Dying. ISBN 0-684-83938-5, Reprinted 1997, Simon & Schuster, Inc.

Stage 1: 0-25%Stage 1: 0-25%Characteristics • External pressure forces development of QS

• One person appointed to QA

• No formal infrastructure for training, review, assessment and inventory

Actions • Appoint additional QA personnel, through management edict, try to harness enthusiastic people showing initiative

• Artful Dodgers (Hide from QM)

• Argue that project is not technical or no data, no sampling

Attitudes • Management views quality as outside their primary focus

• Minimal understanding throughout organization, seen as an insurance policy

• Staff have narrow view of when quality is needed

Keys to Success • Develop generic QMP (not too prescriptive)

• Encourage broader ownership across the office

• Try to document existing processes that relate to Quality (e.g., workload planning, expenditures)

• Avoid using top down logic for selling Quality, vs explanation of the benefits

Stage 2: 25-50%Stage 2: 25-50%Characteristics • QMP approved

• QAPPs developed for some projects

• Polarization of QM and POs

Actions • QA staff identify delinquencies & try to fill gaps

• Training initiated, typically introductory

• Good opportunity for external MSRs

• Develop inventory of projects/expenditures

• Emphasize value of QA

Attitudes • Most see QA as bureaucratic exercise

• Difficult exchanges between QA staff & POs

• Problem: “How will my QM fix this?”

Keys to Success • Management takes ownership

• Develop inventory, capture quality during award phase, build rapport with grants, contract staff

• Build on positive behavior & ignore nay-sayers

QA = Quality AssistanceQA = Quality AssistanceFor For Enterprise AssuranceEnterprise Assurance

We succeed as a team, or we We succeed as a team, or we fail as a team.fail as a team.

Stage 3: 50-75%Stage 3: 50-75%

Characteristics • QMP approved and partially implemented

• QMs involved in management meetings

• Management begins to ask QA questions

Actions • POs employ systematic planning for all projects

• QA staff involved in project planning

• Inventory of projects 100% implemented

Attitudes • Most staff believe QA provides value

• QM feels like part of the team and not tattle-tale

• Problem: “How will we fix this?”

• Management becomes enlightened by Quality status (answers to questions)

Keys to Success • QA staff must stay involved at project-level

• Recognize and reward QA successes

• Orient limited QA $ to high priorities

Stage 4: 75-100%Stage 4: 75-100%Characteristics • Quality system is comprehensive

• QA is a component of daily activities for all staff

• Peer review & info quality key parts of QS

• Managers are actively involved and well-trained

• Office is perceived positively by external clients

Actions • Use QA training & experience in hiring criteria

• Staff use “we” terms instead of “you” terms

• Continually re-evaluate, QM provides data assessments that relate to office-wide goals

Attitudes • Staff seek out QA personnel for assistance

• Staff are empowered to improve quality

• Staff reveal QA concerns - know they’ll be heard

Keys to Success • QM integral part of project development

• PO seen as enforcer and not QM

• Hire people with positive QA attitudes

• Quality system relates to organizational goals

ManagementManagement Questions Questions

Why?Why?

When a manager asks about the When a manager asks about the state of quality in their organization, state of quality in their organization, they emphasize its importance and they emphasize its importance and drive the improvement.drive the improvement.

What holds managers back?What holds managers back?

• Fear of additional resource demands• Narrow view of quality (e.g. focus on lab

data), not seen as their function• Not realizing the management tools

associated with the quality process• They do not have battle scars from poor

quality

Stage 2 Questions Stage 2 Questions

• How many active projects do we support?

• What percent collect environmental information?

• How many of these projects have approved quality documentation?

Focus on InventoryFocus on Inventory

Stage 3 QuestionsStage 3 Questions

• How many projects have been assessed to evaluate key quality concerns and quality implementation?

• Are we focusing quality resources on the most important office decisions?

• Are we prioritizing resources to areas of greatest uncertainty?

• Is this uncertainty relevant to the decision to be made?

Focus on ImplementationFocus on Implementation

Stage 4 QuestionsStage 4 Questions

• Have true environmental outcomes been addressed?

• Have we discussed how these quality issues affect the decision?

• Is the final product disseminated, consistent with IQG and Peer Review (reproducible)?

• Have we discussed recommendations for improvement?

Focus on ReflectionFocus on Reflection

Advanced Stages of Quality SystemsAdvanced Stages of Quality Systems

• Are we there yet?

• If not what can we do to move forward?


• Align quality systems with management priorities– Government Performance Results Act

(GPRA)– Fulfilling the mission– Information Quality Guidelines


• Align key program functions toward GPRA goals– In aligning and coordinating different components must have

awareness of how pieces fit together – This awareness will come from the new policy order

• Use the quality system (i.e., the quality framework) to align and coordinate towards the same goals (GPRA goals)

• Educate and involve all components of the office– Define how quality is part of day-to-day activities– Show value added aspects of the quality system– Management, budget office, funding office, clerical,

administrative, and information management staff all need to be onboard


• Focus on Mission Critical Functions (Enterprise Assurance™)– Where can an effective quality

management system have the greatest impact within the office?

– Don’t sweat the small stuff (unless you have a few low hanging apples)

Advanced Stages of Quality Advanced Stages of Quality Systems: Quality FrameworkSystems: Quality Framework

• All mission critical programs may not need a formal QMP per EPA/QAR2 if a quality process exists that addresses common quality practices for various standards:– Personnel Qualifications and Training,– Assessment and Response,– Documentation and Records,– Systematic Planning, etc.,– Procurement of Key Products and Services, – Continuous Improvement, etc.


• This focus on inventory must be addressed at all levels of influence– Headquarters must ask Regions, who must ask

States/Tribes, who must ask NGOs or local entities

• What is the status of quality (i.e., what is baseline or inventory)? Of the ___$ we provide what % is covered by sufficient quality?

• How is quality implemented?• Provides basis for consistency across programs


• Discuss quality issues as part of National Funding Guidance

• Discuss quality implementation and infrastructure when Regional Program Managers conduct program reviews with States and Tribes

Follow the money!


• For grants, interagency agreements, and contracts– We need to address how these extramural

activities meet Performance Goals as defined in our Agency’s Strategic Plan (i.e., define desired outcome, as opposed to a scope that tells them how to implement quality)

– Delegated programs that implement successful quality systems improve overall effectiveness and ease the burden of the overseer


• The environmental community needs to be able to answer baseline/inventory questions, before we can prepare a quality implementation scorecard– How many active projects are there that collect environmental

information?– How many of them have approved quality documentation?– How many have evaluated the quality of the final decisions?

• If we cannot answer these questions for all projects, we should at least answer the questions that relate to our Strategic Plan/ GPRA goals

Next Steps?????Next Steps?????

• Help our managers to answer baseline quality questions. Utilize the annual QAARWP?

• Utilize the baseline data to evaluate if our quality tools are contributing effectively to Agency goals

• Identify the gaps and orient our limited quality resources

Next Steps????? Next Steps?????

• Help our managers understand the quality framework and reduce confusion how various new initiatives fit together to achieve our mission

• Do whatever we can to empower stakeholders with quality tools processes and resources

Key Points in any Quality System*Key Points in any Quality System*

1. The quality system is not optional.

2. All staff in the office should have a responsibility for the quality of their work and of the organization, and have identifiable roles in the quality system.

*acquired from OW’s Ten Commandments for QA (2004 QMP)


3. There is an individual identified within each organizational unit in the Office whose quality system activities are independent of the line management structure.

4. Quality is a critical responsibility of all levels of management within the Office and all management personnel have identifiable roles in the quality system. Managers are responsible for ensuring the allocation Managers are responsible for ensuring the allocation of funding for quality management activities, including of funding for quality management activities, including intramural, extramural, and travel funds, as well as intramural, extramural, and travel funds, as well as funding for personnel and quality-related training.funding for personnel and quality-related training.


5. Quality can only be achieved through systematic planning, assessment, and corrective action.

6. The importance of the project, the risk of a decision error, the schedule for completion, and the available resources are used to establish the level of quality management applied to a given activity. These considerations must be addressed and documented during the planning phase of the activity.


7. The quality of any environmental data or information used by the Office must be assessed (known) and documented, regardless of the source.

8. All environmental decisions made by the Office must be evaluated relative to the quality of the underlying data and information and these evaluations must be documented.

Suggestions for Enhancement of Suggestions for Enhancement of Our Quality ProgramsOur Quality Programs

• Provide quality support to states through annual funding guidance. State funding is much more dynamic and politically driven than federal funding

• Encourage Regions/States to negotiate Quality System resources through the Environmental Performance Partnership Agreement (ENPPA)


• Discuss Implementation of our Quality System at Regional/State/Tribal levels alleviating Regional ambiguity and inflexibility of quality process

• Allow delegated programs to use the Quality System to provide more efficient utilization of limited resources


• Incorporate Peer Review and IQG programs in revisions to the QMP

• Discuss process for Regional and State/Tribal implementation of goals and quality issues in revision to the QMP

• Provide opportunities for managers to take ownership of quality system

“A true functional value-added quality system is not driven by approved

documentation, but more so, by the activities implemented on a daily basis

that enhance the quality of the environmental decision.”

Great Lakes National Program Office’s QMP: http://www.epa.gov/glnpo/qmp/index.html

ConclusionConclusion

Please send all comments and questions to:Louis Blume, GLNPO QM 312-353-2317; [email protected]

Great Lakes National Program Office’s QMP: http://www.epa.gov/glnpo/qmp/index.html

Soaring Towards QualitySoaring Towards Quality

CONGRATULATIONS!CONGRATULATIONS!

You have successfully completed the You have successfully completed the EPA Quality in Contracts and Grants TrainingEPA Quality in Contracts and Grants Training

Please complete the course review form and be sure to mention potential “Toolbox” items

If you would like to know more about the EPA Quality System or have any questions, please contact:

• Quality Assurance Manager, or Quality System Coordinatorhttp://intranet.epa.gov/quality/documents/qa-community.doc

• Quality staff: [email protected] or 202-564-6830

• Visit the EPA QA Web site at:http://www.epa.gov/quality

ResourcesResources

EPA Quality Requirements for Solicitations and Contractsintranet.epa.gov/quality/contracts.html

EPA Quality Systemwww.epa.gov/quality

Federal Acquisition Regulationswww.arnet.gov/far/(Note Part 46 is "Quality Assurance")

EPA's Acquisition Policy Informationwww.epa.gov/oam/ptod/

FAI COR Mentor Courseintranet.epa.gov/oamintra/trainingwww.faionline.com

ELECTRONIC COPIES of the QARF in Adobe Acrobat, WordPerfect, and Word at:http://intranet.epa.gov/quality/contracts/ctools.html#qarf

Your organizations QA Manager http://intranet.epa.gov/quality/documents/qa-community.doc

Agenda Moderator: Betsy Grim, Director Quality Assurance, OPP

Business

Transcript of Agenda Moderator: Betsy Grim, Director Quality Assurance, OPP