© Copyright 2009 TEM Consulting, LP - All Rights Reserved Presentation To Travis County, TX - May...
-
Upload
chloe-nichols -
Category
Documents
-
view
218 -
download
3
Transcript of © Copyright 2009 TEM Consulting, LP - All Rights Reserved Presentation To Travis County, TX - May...
TTEEMM CCoonnssuullttiinngg,, LLPP
© Copyright 2009 TEM Consulting, LP - All Rights ReservedPresentation To Travis County, TX - May 27, 2009 Rev 1 – 05/22/09 - HSB
US Voting SystemConformity Assessment System
Prepared forTravis County Texas
May 27, 2009
TTEEMM CCoonnssuullttiinngg,, LLPP
© Copyright 2009 TEM Consulting, LP - All Rights ReservedPresentation To Travis County, TX - May 27, 2009 Rev 1 – 05/22/09 - HSB
Conformity Assessment
Conformity assessment is a system designed to assure that products and services meet requirements.
Certification is a common component in a conformity assessment system.
Certification of a product is a means of providing assurance that it complies with specified standards and other normative documents.
TTEEMM CCoonnssuullttiinngg,, LLPP
© Copyright 2009 TEM Consulting, LP - All Rights ReservedPresentation To Travis County, TX - May 27, 2009 Rev 1 – 05/22/09 - HSB
System Certification
Several types of certification systems exist:
•Some comprise type testing only
•Other systems include initial testing and field surveillance
•Still others include initial testing of a product and assessment of its suppliers’ quality systems, followed by routine audits that take into account the factory quality system and the testing of samples from the factory and the open market.
TTEEMM CCoonnssuullttiinngg,, LLPP
© Copyright 2009 TEM Consulting, LP - All Rights ReservedPresentation To Travis County, TX - May 27, 2009 Rev 1 – 05/22/09 - HSB
Voting System Certification
In the US, Voting System Certification includes:
• National testing and certification by the EAC• State testing and certification• Local acceptance testing (recommended)• Assessment of its suppliers’ quality systems• field surveillance by both EAC and state officials
TTEEMM CCoonnssuullttiinngg,, LLPP
© Copyright 2009 TEM Consulting, LP - All Rights ReservedPresentation To Travis County, TX - May 27, 2009 Rev 1 – 05/22/09 - HSB
International Conformity Assessment
Conformity assessment is an international concern to assure the safety of food, pharmaceuticals and other critical items. It is also a necessity to support international trade.
To meet these needs a number of treaties and agreements have been created. These agreements are implemented by processes, created and supervised by organizations such as:
• International Standards Organization (ISO)• International Laboratory Assessment Cooperation (ILAC)
TTEEMM CCoonnssuullttiinngg,, LLPP
© Copyright 2009 TEM Consulting, LP - All Rights ReservedPresentation To Travis County, TX - May 27, 2009 Rev 1 – 05/22/09 - HSB
Purpose of Presentation
This presentation discusses the elements common to conformity assessment systems and how these are used in the US Election System for certification and decertification of voting equipment.
TTEEMM CCoonnssuullttiinngg,, LLPP
© Copyright 2009 TEM Consulting, LP - All Rights ReservedPresentation To Travis County, TX - May 27, 2009 Rev 1 – 05/22/09 - HSB
Components of a Certification System
•Initial Type Testing
•Assessment of the Supplier’s Quality System
•Field Information & Feedback
•User Involvement
TTEEMM CCoonnssuullttiinngg,, LLPP
© Copyright 2009 TEM Consulting, LP - All Rights ReservedPresentation To Travis County, TX - May 27, 2009 Rev 1 – 05/22/09 - HSB
Key Participants•EAC
•Voting System Test Laboratories
•Texas Secretary of State, Elections Division
•Travis County Office of Elections
•Vendors
•Voters
TTEEMM CCoonnssuullttiinngg,, LLPP
© Copyright 2009 TEM Consulting, LP - All Rights ReservedPresentation To Travis County, TX - May 27, 2009 Rev 1 – 05/22/09 - HSB
National Program State & Local Programs
The primary concern of the EAC program is that the system design, as represented in the system submitted for testing, meets the requirements
The primary concern of the state and local election official is that: •Meets the specific needs of a jurisdiction•the units delivered meet and continue to meet the requirements over their useful life.
TTEEMM CCoonnssuullttiinngg,, LLPP
© Copyright 2009 TEM Consulting, LP - All Rights ReservedPresentation To Travis County, TX - May 27, 2009 Rev 1 – 05/22/09 - HSB
Key Issues for Certification System
•What is a minimal acceptable system?
•Are the testing lab/testers/lab assessors qualified?
•Will the vendor deliver units within manufacturing tolerances to those tested?
•How will the election officials know if non-compliant units are delivered and what corrective actions can it take?
•Will election officials and poll workers use the systems as intended?
TTEEMM CCoonnssuullttiinngg,, LLPP
© Copyright 2009 TEM Consulting, LP - All Rights ReservedPresentation To Travis County, TX - May 27, 2009 Rev 1 – 05/22/09 - HSB
EAC Testing and Certification Program
VVSGSpecification
Archived Software
Field Experience
Election System
Conformity Assessment System
Fee
dbac
k
Trusted Archive, e.g. NIST NSRL
Vendor(Candidate
Voting System)
TGDC & NIST
State Certification
Acceptance & Pre-Election Testing
System Testing
System Testing
Verification
Verification
VSTL Evaluation
EAC Determination
Su
bm
iss
ion
Fee
dbac
k
TTEEMM CCoonnssuullttiinngg,, LLPP
© Copyright 2009 TEM Consulting, LP - All Rights ReservedPresentation To Travis County, TX - May 27, 2009 Rev 1 – 05/22/09 - HSB
Processes
Processes are necessary to implement the components of a conformity assessment system for voting systems and to provide reasonable assurance that the key concerns of such a system are carefully addressed.
TTEEMM CCoonnssuullttiinngg,, LLPP
© Copyright 2009 TEM Consulting, LP - All Rights ReservedPresentation To Travis County, TX - May 27, 2009 Rev 1 – 05/22/09 - HSB
Processes
• Technical Standard• Establish normative standard
2002 VSS & EAC VVSG• Refine and Keep standard current
• Laboratory Accreditation• NIST NVLAP Accredit to ISO 17025 to assure:
• Technical competence• Management and quality process control
• EAC Accredit as Voting System Test Laboratory
TTEEMM CCoonnssuullttiinngg,, LLPP
© Copyright 2009 TEM Consulting, LP - All Rights ReservedPresentation To Travis County, TX - May 27, 2009 Rev 1 – 05/22/09 - HSB
Processes
• Technical Reviewers• Review and give recommendations to EAC• TR’s will have specialized qualifications for various topics• EAC will perform routine performance evaluations
• Product Evaluation• Test plan• Testing and test report• Witness testing will be part of the process• Test report review and system certification
TTEEMM CCoonnssuullttiinngg,, LLPP
© Copyright 2009 TEM Consulting, LP - All Rights ReservedPresentation To Travis County, TX - May 27, 2009 Rev 1 – 05/22/09 - HSB
Processes
• Vendor Registration• Review the vendor configuration control system• Review the vendor quality control system
TTEEMM CCoonnssuullttiinngg,, LLPP
© Copyright 2009 TEM Consulting, LP - All Rights ReservedPresentation To Travis County, TX - May 27, 2009 Rev 1 – 05/22/09 - HSB
EAC / NISTTGDC
Technical Guidelines
EAC / NISTTGDC
Technical GuidelinesCandidate System
Accredited
Test Laboratory
Accredited
Test Laboratory EACEAC Technical Review
Team
Technical Review
Team
Certified SystemSystem
Description
System
DescriptionNIST
Software Ref. Lib.
NIST
Software Ref. Lib.
Vendor
Quality & ConfigurationControl Processes
Vendor
Quality & ConfigurationControl Processes
State & LocalEvaluation
State & LocalEvaluation
Field Deploymentand Use
Field Deploymentand Use
Receiving InspectionReceiving Inspection
TTEEMM CCoonnssuullttiinngg,, LLPP
© Copyright 2009 TEM Consulting, LP - All Rights ReservedPresentation To Travis County, TX - May 27, 2009 Rev 1 – 05/22/09 - HSB
EAC / NISTTGDC
Technical Guidelines
EAC / NISTTGDC
Technical Guidelines
Accredited
Test Laboratory
Accredited
Test Laboratory
EACEACTechnical Review
Team
Technical Review
Team
NIST
NVLAP&
Software Ref. Lib.
NIST
NVLAP&
Software Ref. Lib.
VendorsVendors
State & LocalElection Officials
State & LocalElection Officials
CitizensCitizensLocal
Receiving Inspectors
Local
Receiving Inspectors
TTEEMM CCoonnssuullttiinngg,, LLPP
© Copyright 2009 TEM Consulting, LP - All Rights ReservedPresentation To Travis County, TX - May 27, 2009 Rev 1 – 05/22/09 - HSB
TGDC&
NIST
VSTL
VVSGSpecification
Build Executable Code
Submission
Source Code
Executable Code
Build Environment
Election Software
Certification, Delivery & Verification Process
System Testing
Trusted Archive, e.g. NIST NSRLDeposit
Manufacturer(Source Code, Build Environment, Voting
System)
Source Code Review
State Certification
Acceptance & Pre-Election Testing
System Testing
System Testing
Verific
ation
Verification
TTEEMM CCoonnssuullttiinngg,, LLPP
© Copyright 2009 TEM Consulting, LP - All Rights ReservedPresentation To Travis County, TX - May 27, 2009 Rev 1 – 05/22/09 - HSB
Conclusion
A well constructed certification system provides satisfactory answers to central issues:
•What is a minimal acceptable system?
•Are the testing lab/testers/lab assessors qualified?
•Will the vendor deliver units within manufacturing tolerances to those tested?
•Will the election officials know if non-compliant units are delivered and what actions can they take?
•Will election officials and poll workers use the systems as intended?