The top 10 guidelines for conducting usability tests of ballots come from two main sources. The first is a group of documents put together into the LEO Usability Testing Kit developed by the Usability and Voting Project of the Usability Professionals’ Association. (LEO stands for local election official.)
The second source is the years of experience the team behind the Field Guides has conducting usability tests and working with counties and states to help them make ballots, forms, and web sites work better for all citizens.
Usability testing is a tool for learning where people interacting with a design – such as a ballot – encounter frustration, and translating what you see and hear to make a better design that will eliminate those frustrations.
At its essence, usability testing is a simple technique: Watch and listen to people who are like your voters as they use a design as they normally would (or as close to normal as you can get). You can probably already see how this is different from conducting surveys or focus groups.
When it is easy for voters to use a ballot, they are more likely to vote as they intend. That means fewer lost votes, which means wider margins (generally), which means fewer ballots are contested if there is a recount. All of which adds up to better elections for everyone.
It’s really simple. You don’t need recording equipment, but you might want to take notes. So, a clipboard can be handy. And you probably want an envelope or a folder to hold the ballots (or other materials) from the test sessions.
Test ballots to improve the ballot design and to understand training issues for election workers when:
Usability tests can answer questions like these:
How many: 12 to 15 one at a time
How many: 1, you or someone who didn’t design the ballot
Helper / note taker
How many: 1 or 2, someone who can help you review
How many: 1 or 2 from citizen groups (you can turn them into helpers, too)
When you do your first usability test, you might want to practice on somebody else’s ballot. That way, you won’t feel so bad when you test yours.
Otherwise, you can test:
Choose a place where you can find voters:
As the interviewer, you guide the voter through the session, watch what the voter does, and take notes (if you can).
Do not help the voter use the ballot. (Well, until after you have finished learning what you need to learn.)
Ask open-ended questions, like, “How did that go?” Follow up with a statement like, “Tell me about how you marked the ballot.”
Did the voters:
Look at what parts of the ballot caused questions, comments, mistakes, or requests for help.
This should tell you what is confusing to voters, what is unclear to voters, and why. It should also tell you what might need instructions or a different heading or label.