Research with NIST
CCD has partnered with the National Institute of Science and Technology (NIST) since 2003 to conduct research on a range of voting related topics. Much of the research went into the development of VVSG 2.0. It also underpins CCD’s body of work on the usability and accessibility of voting systems.
Ballot design
Legibility of summary-style printed ballots
An exploration of design characteristics of summary ballots and whether they are readable visually and with optical scan recognition (OCR) tools for accessibility.
Legibility of summary-style printed ballots: What makes a printed summary-style ballot easy to verify? (2021)
How voters review and verify ballots
A report of qualitative research to gain deeper insights into how voters mark, review, verify, and cast their ballots. It looked at the role the design of the voting interaction and overall voting process plays in encouraging voters to carefully check their ballot before casting it.
How voters review and verify ballots: A report of the research (2019)
Voter Review and Verification of Ballots
This review of the literature and research approaches looked at recent studies of ballot marking devices and whether they provide voters with a meaningful opportunity to verify their ballot before casting it. A final section looks at what is known about how to conduct this research.
Voter Review and Verification of Ballots: Review of the Literature (2019)
E-pollbooks
Electronic Pollbook usability and accessibility
In 2015, we worked on a landscape analysis, checklist for usability and accessibility and a usability test plan for election officials, state certification programs, and system designers.
Visit our usability of electronic poll books page to explore the entire report and related materials.
Poll workers
Poll Worker Training
Accessible voting systems are a key feature in polling places that enable voters with disabilities to vote privately and independently. This report and companion sample training slide presentation are intended to assist election officials as they develop their poll worker training. In addition to ensuring that poll workers know the legal rules for disability voting rights in the polling place, it includes guidance for helping voters while preserving their independence and privacy. This project took place in 2023.
- Customizable Training Deck: Asisting voters with disabilities as they mark, verify, and cast their ballot (pptx download)
Training Poll Workers for Accessible Voting: Supporting voters with disabilities at the polling place (2023)
NIST Voting Technology Series
NIST VTS 100-2
Remote ballot marking systems
Applying VVSG 2.0 requirements to remote ballot marking systems
This report explores how VVSG 2.0 requirements might apply to a remote ballot marking device in which the voter uses an electronic interface to mark their ballot.
This analysis relies on research from 2015, which looked at the benefits and complexities of remote voting to develop principles and guidelines for these systems.
Applying VVSG 2.0 requirements to remote ballot marking systems (2020)
Principles for Remote Ballot Marking Systems
In 2017, we conducted research on the question can we make remote ballot marking systems both accessible and secure, so voters can use and trust them?
Remote ballot marking systems allow voters to receive a blank ballot to mark electronically, print, and then cast by returning the printed ballot to the elections office.
This project investigated the “voter journey” for using a remote ballot marking system that supports election integrity while ensuring that the system is accessible to all voters.
The project goal was to propose general principles and guidelines that can inform the design, development, deployment or selection of a remote ballot marking system–and possibly be useful for other online election systems.
Visit our remote ballot marking system project page to see the final report and related materials.
Vote by mail
Voting by mail: The impact of signatures for voters with disabilities
Voting by mail has recently grown from an exception to a widely used method of voting. In many states, a signature is used to confirm the voter’s identity. However, many people with a variety of disabilities cannot produce a consistent signature that can be used for verifying their identity. This report examines how signatures and signature comparison are used in elections, explores the uses and types of signatures in other contexts, and discusses alternatives being developed that might – and might not – be appropriate for use in elections.
Voting by mail: The impact of signatures for voters with disabilities (2023)
Administering Accessible Vote by Mail Systems
Accessible vote by mail (AVBM) provides options for voters with disabilities to complete their absentee or vote by mail ballots independently and privately. This report reviews current elections offices’ practices in administering accessible vote by mail and considers their innovations and current challenges.
Recommendations include gathering more detailed information about administrative practices, assessing the risks and benefits for different methods of electronic return, and investigating methods for directly counting an AVBM ballot without replication, including best practices for how voter selections are encoded.
Administering Accessible Vote by Mail Systems: Challenges and Innovations in Elections Offices (2023)
Voting systems
Handbook for VVSG 2.0 Usability and Accessibility Test Strategies
This document provides guidance and resources for how to test voting systems against the usability and accessibility requirements in the Voluntary Voting System Guidelines (VVSG) 2.0. The requirements include Principles 2.2 and 5 through 8. The goal of those requirements is to ensure that voting systems certified to VVSG 2.0 are accessible and usable for election workers and voters—including voters with disabilities—so that every voter can mark, review, cast, and verify their ballot independently and privately. (NIST VTS 400-5)
Handbook for VVSG 2.0 Usability and Accessibility Test Strategies: A report on guidance and resources for how to test voting systems (2023)
User-centered design (UCD) – VVSG 2.0 principle 2.2
- How to design and test a usable voting system (PDF and Word)
- UCD report sample (docx)
- UCD report template (docx)
Usability testing – VVSG 2.0 principle 8.3
- How to conduct a usability test with voters (PDF and Word)
- About the CIF for Voting template (PDF and Word)
- Sample usability testing forms (docx)
- CIF template – ballot and voter instructions (docx)
- CIF template – sample appendixes (docx)
- CIF for Voting report template (docx)
Usability testing with poll workers – VVSG 2.0 principle 8.4
Download all user-centered design and usability testing materials (ZIP file, 524Kb)
All files are from December 2018, when they were reviewed by the Human Factors and Privacy public working group.
A protocol for usable accessibility testing for voting systems
This document is a protocol for testing voting systems for accessibility, with a focus on the voter experience. It provides a test of usability and accessibility functions in use, including how well the ballot presents voters with options and allows them to confirm their choices while marking and verifying their ballot before casting.
The method combines an expert review of the accessibility features with usability testing with voters and poll workers. The ballot and test activities can be adjusted to local election administration rules, such as straight-party voting, fusion nominations, and voting methods including vote-for-N or ranked choice voting.
It was originally developed as part of a state certification program where systems must first be certified to the Voluntary Voting System Guidelines (VVSG).
A protocol for usable accessibility testing for voting systems (2023)
Testing material templates
- Notetaking form (PDF and Word)
- Report template (PDF and Word)
- Pollworker testing moderator script (PDF and Word)
- Voter testing moderator script (PDF and Word)
Designing Usable Audio for Voting Systems
Voting systems must support multiple interaction modes: presenting information both visually and auditorily, accepting navigation and selections from both screen touches and key presses on a tactile controller. The best practices outlined in this document focuses on the audio and tactile controller experience. It also includes an incremental approach to testing the audio with voters with disabilities.
Designing Usable Audio for Voting Systems: Best practices and a test approach (2023)
Considering the number of accessible voting systems needed in a voting location
A summary of approaches to considering how to provide resources so that an accessible voting system can be used on an equal basis to any other method of voting.
Considering the number of accessible voting systems needed in a voting location (2021)
Webinars: Improving the Accessibility and Usability of Voting Systems
These webinars were presented in 2020 before the VVSG 2.0 was adopted to introduce the updated requirements for usability and accessibility.
Developing VVSG 2.0
Progress on the roadmap for usability and accessibility of next generation elections
In 2015, the Roadmap for usability and accessibility of next-generation elections identified 6 priority areas for improvement with specific recommendations for meeting those goals.
In 2018, we published this update on progress, which looks at what has happened in each of the priority areas. In both the work by NIST and the EAC, and in the broader world of elections and elections technology, there has been significant activity towards these goals.
Progress on the roadmap for usability and accessibility of next generation elections – 2015-2018 (2018)
Roadmap for usability and accessibility of future voting systems
As work began on the development of VVSG 2.0, this roadmap identified priority areas where new research is needed, and ways to improve the current guidance, standards and certification process. The roadmap took a voter-centered approach, rather than limiting the scope to the systems.
The priority areas are:
- Support the design process
- Engage voters effectively
- Address the entire voter journey
- Support evolving technology
- Provide useful guidance and standards
- Improve testing in design and certification
A roadmap for future usability and accessibility guidance (2015)
Related research and guidance materials available on our voting systems topic page
VVSG 2.0 white papers
These brief white papers were written to summarize research between 2005 and 2015 as work began on VVSG 2.0. The topics were suggested by a Working Group that met regularly to identify gaps in the VVSG 1.0 and suggest updated requirements for VVSG 2.0
Text size requirements (2015)
Color contrast requirements (2015)
Navigation from the review screen on an electronic ballot (2015)
Scrolling on a ballot (2015)
Digital ballot design studio: Select, deselect + scroll and paging (2015)
Assistive technologies in use in the polling place
A white paper reviewing new and emerging technologies and assistive technologies that might have an impact on how voters with disabilities interact at the polling place.