Voting systems

A roadmap to ensure the usability and accessibility of next-generation elections

Research and create guidance materials for updated and new requirements in the Voluntary Voting System Guidelines (VVSG 2.0), to ensure that all voters can vote independently and privately.

This work includes

Exploring new election technology

Reports on the usability and accessibility of elections technology

Reports on technologies in use at the polling place

Research on voter verification of ballots

  • Voter review and verification of ballots. This review of the literature and research approaches looked at recent studies of ballot marking devices and whether they provide voters with a meaningful opportunity to verify their ballot before casting it. A final section looks at what is known about how to conduct this research. (Review draft)
  • How voters review & verify ballots from ballot marking devices. A report of qualitative research to gain deeper insights into how voters mark, review, verify, and cast their ballots. It looked at the role the design of the voting interaction and overall voting process plays in encouraging voters to carefully check their ballot before casting it. (Review draft)
  • Legibility of paper ballots: What makes a printed summary-style ballot easy to verify? An exploration of design characteristics of summary ballots and whether they are readable visually and with optical scan recognition (OCR) tools for accessibility. (Review draft)

Guidance for usability and user-centered design testing

These guidance documents and the accompanying sample materials and templates, support the three requirements in the VVSG 2.0 draft requirements for a user-centered design process, usability testing with voters, and with poll workers.

User-centered design (UCD) – VVSG 2.0 principle 2.2

Usability testing – VVSG 2.0 principle 8.3

Usability testing with poll workers – VVSG 2.0 principle 8.4

Download all user-centered design and usability testing materials (ZIP file, 524Kb)
All files are from December 2018, when they were reviewed by the Human Factors and Privacy public working group.

VVSG 2.0 usability and accessibility requirements

Work on VVSG 2.0 started in 2016, with principles and guidelines for the next iteration. Public Working Groups debated and wrote requirements. A draft was completed in February 2020 and released for public comment.

Human Factors Test Approach for VVSG 2.0

These test cases are for the human factors requirements in Principle 2.2 and Principles 5-8 efficiently. Each test case includes a method and pass/fail conditions. Many use structured interactions with the voting system interface often testing several requirements at once.  This version is as of August 31, 2020

Adapting VVSG 2.0 to other election systems

Webinars on the VVSG 2.0 draft requirements: Improving the Accessibility and Usability of Voting Systems

White papers for voting system usability and accessibility topics

Working with colleagues on the Human Factors and Privacy Public Working Group, we wrote a series of white papers reviewing issues identified in a gap analysis of the current voting system standards and collected research to support new or changed requirements (all posted as Word files):

The roadmap: A framework for guidance

The roadmap for usability and accessibility of future voting systems aimed to:

  • Increase the level of knowledge about how to create election systems with good usability and accessibility.
  • Promote consistent levels of usability and accessibility across all parts of the elections process.
  • Make systems more usable for everyone in the elections process, including voters, poll workers, elections staff, and advocacy or support workers.
  • identify the appropriate guidance—including guidelines for best practices, procedural support, and training—rather than a single focus on standards and certification.

Work on the roadmap took a voter-centered approach, rather than limiting the scope to the systems.  We hoped that this roadmap can serve the entire election community: election officials, people who design, build, or test elections systems, voters, and election advocates who share the goal of making elections more usable and accessible for everyone.

The roadmap itself is organized into six priority areas, from the design process to certification of specific systems. Each priority area is a goal for improving elections. Within the priority areas, there are 22 objectives, or goals for work that:

  • Identify gaps in current practice and knowledge
  • Provide an overview of the challenges and risks
  • Outline steps towards meeting the objectives
  • List existing resources to support the work.

In 2018, we looked at progress towards the roadmap goals and found that significant work had been done in all priority areas, both through NIST and EAC projects and in other elections research or activities at universities, by vendors, and in elections offices.

The roadmap suggests a model for how different levels of guidance can work together, starting from principles for elections and including both core standards that apply across all systems, guidelines for specific systems, and useful materials such as training, testing methods, samples, and scenarios.

In the larger work on updating voting system guidelines to address changes in technology and election administration, there is a goal to start from broad principles first, with guidelines and technical requirements clearly supporting those principles, as reflected in the model for guidance shown below.

A stack with principles at the base, then layers with core guidance/test methods, system-specific guidelines, training, evaluation, samples, scenarios, and ending with monitoring and feedback in use.

Developing the roadmap

In 2015,  we created a roadmap to help the NIST voting project team look forward to its work on the Voluntary Voting Systems Guidelines (VVSG) 2.0. The roadmap included a history of work towards accessible and usable voting systems, and explored how to achieve the goals of the Help America Vote Act (HAVA) requirements for independent and private voting for all voters.

Time brings changes to the state of the art and technology for voting systems, as well as public expectations about how voters will participate in elections. There are new technologies, new research, new laws, and new elections procedures since the 2005 Voluntary Voting System Guidelines 1.0 was published. Keeping up with these changes requires a new approach to usability and accessibility guidance for election systems.

In 2020, as in 2015, the reality is that there are still many barriers. Even newer systems show poor accessibility and usability, suggesting a lack of knowledge of best practices and existing standards and guidelines. This is true of both voting systems and related technology. As more jurisdictions have switched to paper ballots, there is even more isolation of the “accessible” voting system.

NIST has worked on voting system standards since the Help America Vote Act of 2002, both establishing requirements for certification test labs and creating the Voluntary Voting System Guidelines (VVSG). The VVSG 2005 included the first comprehensive usability and accessibility standards for voting systems.

Our goals for the roadmap were:

  • Increase the level of knowledge about how to create election systems with good usability and accessibility.
  • Promote consistent levels of usability and accessibility across all parts of the elections process.
  • Make systems more usable for everyone in the elections process, including voters, poll workers, elections staff, and advocacy or support workers.
  • identify the appropriate guidance—including guidelines for best practices, procedural support, and training—rather than a single focus on standards and certification.

How we created the roadmap

The roadmap is the result of discussions with stakeholders such as the Election Assistance Commission, state and local election officials, system vendors, researchers, and others about the needs for better guidance for usability and accessibility of future election systems.

The input for this roadmap came from a review of current research on voting systems and active consultation with experts and other stakeholders in the field.

We sought out diverse viewpoints so many different stakeholder perspectives are included. We held three workshops with experts including election officials, researchers, system designers, policy experts, and disability advocates, each focused on a different aspect of the work.

This work was completed under grants and contracts from the U.S. Department of Commerce, National Institute of Standards and Technology.

Project team

  • Whitney Quesenbery
  • Lynn Baumeister
  • Suzanne Chapman
  • Rachel Goddard
  • Kathryn Summers
  • Caitlin Rinn
  • Jennifer Sutton

Special thanks to students from the University of Baltimore, everyone who participated in the roadmap workshops, and all the people who have helped with the research over the years.