Questions/Comments:Ed Smith ed.smith@dominionvoting.com

VVSG and Requirements Management

Ed SmithJanuary 13, 2011

Opening Thoughts• VVSG some of the most important requirements

for voting systems and their surrounding development system

• Poorly executed RM is a cause of project and product failure

• Requirements elicitation is important• Organizing

– Development Systems– Product: software, hardware, voter/pollworker

interfaces

Requirements Definition

Before it all starts…• The Introductions

– Both Volume I and Volume II– History, Intent, Summaries of each Volume– Much the same for both Introductions– Overview in Volume I– Glossaries

• TDP (Technical Data Package) productionVolume II, Section 2 Description of the Technical Data Package

Before it all starts…• VVSG requirements for Quality Assurance• VVSG requirements for Configuration

ManagementThe testing also evaluates the completeness of the vendor’s developmental test

program, including the sufficiency of vendor tests conducted to demonstrate compliance with stated system design and performance specifications, and the vendor’s documented quality assurance and configuration management practices. The tests address individual system components or elements, as well as the integrated system as a whole.

• Select a development method• Educate Developers with respect to VVSG

Then Development Starts

• Each section results in inputs to documentation– Requirements Specification– Functional Specification– Engineers’ Documents/Technical Solution– Test Specification/Test Plan

• Test Cases– Manufacturing Specification– Project Plans

VVSG

CERTPROG

MANUAL

EXTREFERENCES

LABPROG

MANUAL

STATESTATUTE

CUSTPREFERENCE

REQUIREMENTSSPECIFICATION

FUNCSPEC

TESTDOCS

CM PLANQA PLAN

PROTOTYPE

Bi-directionaltraceability

Then Development Starts

• Each section results in inputs to documentation– Volumes I and II result in Quality Assurance

Planning• Audits• Product Testing

– Reliability– Accuracy– Durability– Real World Validation vs. Verification

Then Development Starts

• Each section results in inputsOverview Voluntary Voting System GuidelinesSection 1 IntroductionSection 2 Functional RequirementsSection 3 Usability and Accessibility RequirementsSection 4 Hardware RequirementsSection 5 Software RequirementsSection 6 Telecommunications RequirementsSection 7 Security Requirements

REQUIREMENTSSPECIFICATION

FUNCSPEC

TESTDOCS

CM PLANQA PLAN

PROTOTYPE

Bi-directionaltraceability

Requirement: voting machine allows access toa range to needs Sub-requirement: …including low vision andlack of motor control of hands Sub-requirement: machine shall have a tactilebox with buttons for… and a 3.5mm input Sub-requirement: Braille shall beembossed on each tactile button

Function Spec:Tactile box shall allow for left/right,Volume and speed…Tactile box 3.5mm input shall accept5 volts, stereo…Braille dots shall be 1mm hemispherical

REQUIREMENTSSPECIFICATION

FUNCSPEC

TESTDOCS

CM PLANQA PLAN

PROTOTYPE

Bi-directionaltraceability

Function Spec:Tactile box shall allow for left/right,Volume and speed…Tactile box 3.5mm input shall accept5 volts, stereo…Braille dots shall be 1mm hemispherical

Test Spec:With an election containing…Ensure function of all tactile buttonsEnsure function of 3.5mm inputEnsure readability of Braille

Test Script:Code an election…Attach tactile box…Scroll through the ballot using…Attach a power supply and input 5V…

Requirements---Guiding Principles

• Must specify what is needed, not the solution (example of machined aluminum housing)

• Complete to an engineering level of detail• Requirements are developed by engineers,

not by marketing department or users• Capturing the requirements may consume

as much as 30 percent of the entire project time budget

• Unambiguous (objectively verifiable)• Quantitative limits expressed with a

realistic measurement tolerance• Self-consistent• Environment completely characterized• Completeness and relevance of external

references

Requirements---Guiding Principles

Lack of RM Allows

• Scope Creep

• Developing the wrong product

Requirements imprecision

• Problems arise when requirements are not precisely stated

• Ambiguous requirements may be interpreted in different ways by developers and users

• Consider the term ‘appropriate viewers’– User intention - special purpose viewer for each

different document type– Developer interpretation - Provide a text viewer that

shows the contents of the document