Future VVSG Development GoalsOne of the tangible suggestions noted during the June 12, 2014 Election Assistance Commission (EAC) Roundtable Meeting on "Reforming the Testing and Certification Process" http://www.eac.gov/roundtable_- _reforming_the_testing_and_certification_process/ was for the EAC to begin discussions with members of the election community regarding how a future Voluntary Voting System Guidelines (VVSG) document should be developed, designed and structured.Next Steps: This paper is ultimately intended for submission to the EAC Commissioners for review. The Future VVSG Working Group members believe that the Commissioners should then task the TGDC and NIST to use the twelve stated goals contained in the accompanying document as a baseline for developing and implementing the next VVSG standards development effort in order to achieve a standards document that allows for reliable testing and certification of election systems through implementable and technology neutral requirements. United States Election Assistance CommissionUSEAC_46171dd6-b84a-11e6-a8f9-06a9f7e90587Future VVSG Working GroupThe EAC envisioned that the primary job of the Future VVSG Working Group ("The Working Group") would be to provide thoughts and ideas to outline the high level direction for the next VVSG development effort. The EAC began this effort with outreach to States via the annual Conference on State Certification of Voting Systems meeting as well as engaging in conversations with numerous individual State and local election officials. The two primary goals for the Working group were: * To explore how future VVSG efforts can support innovation and allow for flexible product solutions while still maintaining clear and testable requirements within the standard * To define a strategy and to develop priorities for producing an effective VVSG in order to ensure that standards consider the interests of all stake holders as well as the real-time operational needs of election officials. The Working Group intentionally did not discuss specific testable VVSG requirements since that process is delegated to the TGDC process called out in the Help America Vote Act... The EAC brought together a diverse and uniquely experienced group of stakeholders to debate and discuss the goals for future VVSG development efforts. The Working Group included:Ann McGeehanFormer Texas Director of Elections & PCEA MemberJuan GilbertProfessor of Computer Science, University of FloridaChristi CoburnDirector of Elections, Jefferson County, ColoradoLori AuginoDirector of Elections, WashingtonPaul AumayrVoting System Director, Maryland State Board of ElectionsJay BaggaProfessor of Computer Science Ball State University, Indiana VSTOPJack CobbLab Director, Pro V & V LaboratoriesMerle KingExecutive Director, Kennesaw State University Center for Election SystemsCliff TatumExecutive Director, DC Board of ElectionsSteve PearsonVice President, Certification, Election Systems and Software (ES&S)David WallickProduct Manager, Everyone CountsSteve TroutDirector of Election Innovation, Clear BallotMary BradyVoting Project Manager, (NIST)_461720f6-b84a-11e6-a8f9-06a9f7e90587To provide thoughts and ideas to outline the high level direction for the next VVSG development effort._46172254-b84a-11e6-a8f9-06a9f7e90587VotingTestingCertificationPurpose & ScopeDefine and confirm the purpose and scope of the VVSG._4617234e-b84a-11e6-a8f9-06a9f7e905871The purpose and scope of the VVSG must be defined and confirmed. -- The Working Group felt that one of the fundamental purposes of the VVSG was determining what policy objectives the guidelines were trying to achieve and to define and describe a voting system. HAVA has its definition of voting system incorporated into the current VVSG, but the reality of state supremacy in election administration and recognition of state diversity and variety are not recognized. While states currently have different definitions of what constitutes a voting system, they should be recognized as an integral part of this process. EAC's Voting System Testing and Certification Program is only one part of the overall conformity assessment process that includes equally important companion efforts at the State and local levels. The process to ensure that voting equipment meets specific technical requirements is a distributed, cooperative effort of Federal, State, and local officials in the United States. Finally, the Working Group urged the EAC to clearly define what components (e-pollbooks? UOCAVA ballot delivery systems? election night reporting systems??) are to be included in the definition of a voting system and to identify what should be tested and certified. Many Working Group members expressed the need for a true federal standard-, i.e. a confederation of state standards. _4617243e-b84a-11e6-a8f9-06a9f7e90587Statute & RulesEnsure the VVSG (and supporting process) is consistent with Federal statute and rule._46172538-b84a-11e6-a8f9-06a9f7e905872The VVSG (and supporting process) must be consistent with Federal statute and rule. -- While perhaps obvious, the working group felt it was necessary to include the indisputable fact that the next VVSG and its individual requirements must not be in conflict with federal laws and regulations._4617261e-b84a-11e6-a8f9-06a9f7e90587State RequirementsIncorporate common State requirements._46172718-b84a-11e6-a8f9-06a9f7e905873StatesNational Association of State Election DirectorsThe VVSG must reflect the bottom-up reality of election administration by incorporating common State requirements to inform future VVSG development. -- Working Group members reiterated that state and local officials should continue to decide what their rules and requirements are for voting systems and what they should test to. Members suggested that in order to make the next VVSG more relevant and more useful to the states, the EAC should map the VVSG with individual state requirements as step one in the next VVSG development process. It was also noted that states currently have a difficult time looking at Federal test reports to determine where state requirements might overlap with VVSG requirements and that a more bottom-up approach to the VVSG might help to alleviate this problem. EAC staff noted that they are currently working with several states to map requirements to the VVSG and are working on a way to build state mapping into the EAC's Virtual Review Tool (VRT) (The VRT is the secure web-based portal developed and used by EAC to work through and track progress in certification test campaigns. NASED representatives on the working group noted that mapping requirements is one of NASED's recommendations to the EAC and that they would be drafting a follow up letter to EAC Commissioners on this subject. Ultimately, the Working Group agreed that the real utility of the VVSG was as a core set of requirements that would be useful to as many states as possible as part of the overall certification process for voting systems._46172808-b84a-11e6-a8f9-06a9f7e90587Election AdministrationBenefit election administration._46172a06-b84a-11e6-a8f9-06a9f7e905874The application of the VVSG must benefit election administration. -- The Working Group agreed that the next VVSG must be a practical document and must be fit for purpose. For this paper, the general definition of "fit for purpose" should be thought of as: A document well equipped or well suited for its designated role or purpose. Most members agreed that the 2005 VVSG document was not entirely fit for purpose in hindsight. The Group noted that how well the current VVSG is fit for purpose can be debated and that how to determine if a future VVSG is fit for purpose would be a challenge depending upon which individuals or groups were looking at the document. The Group did agree, however that the VVSG serves a larger purpose outside the confines of the requirements stipulated within the document. Group members were adamant that the next VVSG should not only provide for basic voting system functionality but should also mention and explicitly recognize that the end users of the products designed to meet VVSG requirements conduct elections in various ways from State to State, and that the VVSG remains voluntary. It is this emphasis on, and recognition of, the practical application of the VVSG that will better ensure its fit._46172b14-b84a-11e6-a8f9-06a9f7e90587ImplementationEnsure the VVSG is implementable._46172c18-b84a-11e6-a8f9-06a9f7e905875The VVSG must be implementable. -- Any standards development effort is successful only to the extent that the requirements contained within the document can actually be implemented by product manufacturers and accurately tested by laboratory personnel. The group noted that the Help America Vote Act (HAVA) called out 6 general attributes of a voting system. SEC. 301. VOTING SYSTEMS STANDARDS. (a) REQUIREMENTS. -- Each voting system used in an election for Federal office shall meet the following requirements:Privacy_46172d12-b84a-11e6-a8f9-06a9f7e905875.1(1) IN GENERAL.-- (A) Except as provided in subparagraph (B), the voting system (including any lever voting system, optical scanning voting system, or direct recording electronic system) shall -- (i) permit the voter to verify (in a private and independent manner) the votes selected by the voter on the ballot before the ballot is cast and counted; (ii) provide the voter with the opportunity (in a private and independent manner) to change the ballot or correct any error before the ballot is cast and counted (including the opportunity to correct the error through the issuance of a replacement ballot if the voter was otherwise unable to change the ballot or correct any error); and (iii) if the voter selects votes for more than one candidate for a single office -- (I) notify the voter that the voter has selected more than one candidate for a single office on the ballot; (II) notify the voter before the ballot is cast and counted of the effect of casting multiple votes for the office; and (III) provide the voter with the opportunity to correct the ballot before the ballot is cast and counted. (B) A State or jurisdiction that uses a paper ballot voting system, a punch card voting system, or a central count voting system (including mail-in absentee ballots and mail-in ballots), may meet the requirements of subparagraph (A)(iii) by -- H. R. 3295 -- 40 (i) establishing a voter education program specific to that voting system that notifies each voter of the effect of casting multiple votes for an office; and (ii) providing the voter with instructions on how to correct the ballot before it is cast and counted (including instructions on how to correct the error through the issuance of a replacement ballot if the voter was otherwise unable to change the ballot or correct any error). (C) The voting system shall ensure that any notification required under this paragraph preserves the privacy of the voter and the confidentiality of the ballot. VerificationPermit the voter to verify the votes selected by the voter on the ballot before the ballot is cast and counted._7c89e2bc-b8aa-11e6-ab40-81a0f7e905875.1.A(i)ChangesProvide the voter with the opportunity to change the ballot or correct any error before the ballot is cast and counted._7c89e5fa-b8aa-11e6-ab40-81a0f7e905875.1.A(ii)ErrorsIf the voter selects votes for more than one candidate for a single office, provide the voter with the opportunity to correct the ballot before the ballot is cast and counted._7c89e780-b8aa-11e6-ab40-81a0f7e905875.1.A(iii)Records & AuditsProduce a record with an audit capacity._46172e70-b84a-11e6-a8f9-06a9f7e905875.2(2) AUDIT CAPACITY. -- (A) IN GENERAL. -- The voting system shall produce a record with an audit capacity for such system. (B) MANUAL AUDIT CAPACITY. -- (i) The voting system shall produce a permanent paper record with a manual audit capacity for such system. (ii) The voting system shall provide the voter with an opportunity to change the ballot or correct any error before the permanent paper record is produced. (iii) The paper record produced under subparagraph (A) shall be available as an official record for any recount conducted with respect to any election in which the system is used. AccessibilityMake the voting system accessible for individuals with disabilities._46172f74-b84a-11e6-a8f9-06a9f7e905875.3Individuals with Disabilities(3) ACCESSIBILITY FOR INDIVIDUALS WITH DISABILITIES. -- The voting system shall -- (A) be accessible for individuals with disabilities, including nonvisual accessibility for the blind and visually impaired, in a manner that provides the same opportunity for access and participation (including privacy and independence) as for other voters; (B) satisfy the requirement of subparagraph (A) through the use of at least one direct recording electronic voting system or other voting system equipped for individuals with disabilities at each polling place; and (C) if purchased with funds made available under title II on or after January 1, 2007, meet the voting system standards for disability access (as outlined in this paragraph). LanguageProvide alternative language accessibility._46173082-b84a-11e6-a8f9-06a9f7e905875.4(4) ALTERNATIVE LANGUAGE ACCESSIBILITY. -- The voting system shall provide alternative language accessibility pursuant to the requirements of section 203 of the Voting Rights Act of 1965 (42 U.S.C. 1973aa-1a). ErrorsComply with error rate standards._461731a4-b84a-11e6-a8f9-06a9f7e905875.5Federal Election Commission(5) ERROR RATES. -- The error rate of the voting system in counting ballots (determined by taking into account only those errors which are attributable to the voting system and not attributable to an act of the voter) shall comply with the error rate standards established under section 3.2.1 of the voting systems standards issued by the Federal Election Commission which are in effect on the date of the enactment of this Act.Vote StandardsAdopt uniform and nondiscriminatory standards that define what constitutes a vote._461732c6-b84a-11e6-a8f9-06a9f7e905875.6States(6) UNIFORM DEFINITION OF WHAT CONSTITUTES A VOTE. -- Each State shall adopt uniform and nondiscriminatory standards that define what constitutes a vote and what will be H. R. 3295-41 counted as a vote for each category of voting system used in the State.System DefinitionDefine voting and election systems._461733de-b84a-11e6-a8f9-06a9f7e905875.7HAVA also gives a very specific definition of a voting system in Section 301 (b). The Working Group noted that the intent of the legislation as well as the very specific language should be taken into account in future VVSG development. The group asked if a more expansive definition for the VVSG that encompassed "election system" as opposed to "voting system", could become the baseline definition for VVSG development.FlexibilityImplement varying levels of standards into the next VVSG to increase flexibility for jurisdictions with varying needs._46173514-b84a-11e6-a8f9-06a9f7e905875.8The Working Group also explored the concept of implementing varying levels of standards into the next VVSG to increase flexibility for jurisdictions with varying needs. An example of this would be where level A includes testing to all standards, level B is testing to a large portion of the standards , and level C is testing only to a few high level requirements. This concept would be relevant for cost savings and for those states that may not need a complete and comprehensive standards testing process for the requirements of their particular state. Ultimately, the Group dismissed this idea since many members said that states that don’t use the federal testing and certification now aren't likely to do so simply because of the inclusion of varying test levels. Other members thought that the concept was also bad for voters since concerns would be raised from voters regarding their “lower” testing and certification impacting their voting process more so than a system which has undergone a "higher" testing and certification of a neighboring state.PracticalityMake testing, certifying and implementing voting systems practical and implementable._4617362c-b84a-11e6-a8f9-06a9f7e905875.9Finally, several members were concerned about states that accept EAC testing and do little or no further testing before certifying and implementing a new voting system. These members noted that if the document was not practical and implementable from the perspective of those jurisdictions, those states might not use the document at all and consequently be worse off than they were currently.InteroperabilityAccommodate the interoperability of election systems._46173744-b84a-11e6-a8f9-06a9f7e905876The VVSG should accommodate the interoperability of election systems. -- The Working Group agreed that while perhaps not currently practicable; the next VVSG should accommodate the interoperability of the wide variety of election systems and peripheral devices currently on the market and the numerous new peripheral devices sure to be on the market in coming years. Several Working Group members noted that some set of "core" requirements are generally mandatory in other industries. Other members noted that one option might be to include guidance language for optional peripherals in the next VVSG in order to assist State and local election officials without making this a requirement. While this idea was generally well received, many members noted that putting non standards (non-normative requirements language) in a standards document is concerning and that some States might feel compelled to include the guidance as a requirement for their state. Most working members agreed that component testing is a certification/process issue not a VVSG issue. EAC staff noted that current VVSG language already allows and does not prohibit the development of a component certification process. VVSG 1.1, Section 1.5.2 (Implementation Statement) notes that: "An implementation statement documents the requirements that have been implemented by the voting system, the optional features and capabilities supported by the voting system, and any extensions (i.e., additional functionality beyond what is defined in the VVSG) that it implements." One final comment on this issue suggested that while the VVSG would permit component certification, additional standards for integration will need to be developed._46173884-b84a-11e6-a8f9-06a9f7e90587OptionsPermit jurisdictional options._461739a6-b84a-11e6-a8f9-06a9f7e905877The VVSG should permit jurisdictional options by incorporating a planned transition between Standards in a measured and predictable manner. The Working Group was in unanimous agreement that any future VVSG must be compatible with existing standards, professional practice and policy. The group felt that future versions of the VVSG must be operationally and technologically bridged to preceding guidelines and practices. Many members of the Working Group noted that without a planned transition, future VVSG implementation would be as disruptive, if not more disruptive to election administration than was encountered in the transitional years after the implementation of the 2005 VVSG 1.0. The majority of members felt that the EAC should let the market drive future VVSG implementations and that it should be up to the states/counties to make the choice. While all Working Group members understood the need to transition away from old, out-of-date standards documents at some point, they agreed that clear understanding of the transition process was vital. A majority stated that old standards needed to be retired. All Working Group members recognized that jurisdictions may have plans for their old systems (certified to previous iterations of the VVSG) and that they need plenty of time to plan for sunset these old systems. Essentially the group stated that they would be looking for an articulated transition plan from the EAC. _46173ad2-b84a-11e6-a8f9-06a9f7e90587CostsAvoid imposing unanticipated costs onto organizations._46173c30-b84a-11e6-a8f9-06a9f7e905878TGDCThe Technical Guidelines Development Committee, an advisory committee to the U.S. Election Assistance Commission (EAC), provides recommendations on voluntary standards and guidelines related to voting equipment and technologies. It is composed of fourteen members (link is external) selected from various standards boards and for their technical and scientific expertise related to voting systems and equipment. NIST chairs and manages the TGDC.NISTEACThe VVSG should not impose unanticipated costs onto organizations. -- Because budgetary concerns are a reality for almost all election officials in the United States, the Working Group agreed that any future VVSG should not impose unanticipated cost burdens on organizations. The group intentionally used the term "organizations" in goal #8 so that the financial impact on voting system manufacturers, election officials, test laboratories and others would be recognized and considered during the VVSG development process. The Working Group realized that it is unrealistic to expect that future VVSG documents might not contain some requirements that produce unanticipated costs; no matter how carefully the standards development process considers cost impact. The group nevertheless felt that it was imperative to urge the TGDC, NIST and the EAC to carefully consider cost factors when developing standards. By considering cost impact, the TGDC, NIST and the EAC can better evaluate competing standards when addressing a policy objective of the standard._46173d5c-b84a-11e6-a8f9-06a9f7e90587ConformanceEstimate the cost of conformance testing to the standard._46173e88-b84a-11e6-a8f9-06a9f7e905879The VVSG must include a cost analysis estimate of conformance testing to the standard. -- The Working Group felt that a cost analysis estimate of future draft VVSG standards should be included as the next logical goal in order to operationalize goal #8. Because doing a cost analysis is not typically part of the TGDC/NIST standards development process, the EAC agreed that they would develop this cost estimate in conjunction with the Voting System Test Laboratories (VSTLs) and make the results available to the election community during the VVSG public comment process._46173ff0-b84a-11e6-a8f9-06a9f7e90587RequirementsMake the requirements should be performance based and technology neutral._4617413a-b84a-11e6-a8f9-06a9f7e9058710The VVSG requirements should be performance based and technology neutral. -- The Working Group had perhaps its longest discussions around the topic of whether future VVSG standards documents should be primarily performance based or primarily design based documents. A majority of the group felt that future VVSG standards should be functional in nature so that the document can more easily be re-defined as technology changes, or as new approaches are presented. The group noted that technology neutral statements have longer lives and in order to keep a future VVSG document "evergreen" detailed description of any technology should be excluded. The Working Group determined that the most promising avenue for pursuing such a document would be to develop high level (performance based) standard and have details contained in some lower level document that can more easily change and adapt as technology changes. The group did recognize, however, that labs will need guidance on how to test performance based standards. NIST and EAC staff noted that such "lower level" detail was already being done to a large extent in the Test Assertions development working group led by Mark Skall. Similar to a process envisioned by the Working Group, the test assertions are not design requirements but details on how those individual requirements will be tested. _4617427a-b84a-11e6-a8f9-06a9f7e90587New/Revised RequirementsAllow maximum flexibility to incorporate new/revised requirements including those from other Standards setting bodies._461743f6-b84a-11e6-a8f9-06a9f7e9058711Standards Setting BodiesNISTIEEEAt this time the IEEE scope of the work includes: * Exports of election management system databases including for election results * Voter registration data and precinct information processed by electronic pollbooks * Candidate and ballot definition information * Voted ballot information and tabulations * Device logs * Other information that may be produced by election management systemsThe VVSG should allow maximum flexibility to incorporate new/revised requirements including those from other Standards setting bodies. -- Working Group members stated that future VVSG development efforts must allow maximum flexibility to incorporate new/revised requirements, potentially including those from States themselves as well as from other Standards setting bodies. NIST is currently providing support to the IEEE VSSC 1622, with the goal of creating an XML-based common data format (CDF) for election systems. A CDF will facilitate interoperability among voting devices and certain types of automated testing._46174554-b84a-11e6-a8f9-06a9f7e90587Education & OutreachEngage in education and outreach._461746a8-b84a-11e6-a8f9-06a9f7e9058712Election CommunityThe VVSG should be accompanied by education and outreach efforts to the election community. -- Although not a goal directly affecting the TGDC standards development process, the Working Group nevertheless was in unanimous agreement that any future VVSG development must be accompanied by educational outreach efforts for the election administration community, other stakeholders, and the public. The group felt that including this as a goal would serve as a reminder to those writing the standards that people need to understand the requirements contained in the VVSG document at their respective level of technological sophistication in order for it to be most useful to the widest audience._46174842-b84a-11e6-a8f9-06a9f7e905872015-07-152016-12-02OwenAmburOwen.Ambur@verizon.net