Design, test, repeat: designing a new ballot record form

Election forms get revised for many reasons — usually new procedures, new laws, or, often, because something went wrong. Sometimes, it happens because the form is a pain to use and difficult to complete correctly.

The journey to a new Ballot Record form for Virginia started when Brenda Cabrera, the Fairfax City Director of Elections & General Registrar, promised her poll workers — called election officers in Virginia — they would never have to use the old version again.

Election officers use the Ballot Record form to account for all the paper ballots. Most ballots get cast in the usual way, but some replace an absentee ballot, get cast as provisional ballots, or replace one a voter made a mistake on. And some weren’t used at all. Election officers get a certain number of ballots at the beginning of the day. And at the end, it all has to add up.

This sounds easy, but when the totals don’t match up at the  end of a long Election Day, no one is happy. Not the election officers. Nor the election board who had to sort out the problems and inconsistencies on the forms they received.

This wasn’t just a problem in Fairfax City. People from many localities had been grumbling about the Ballot Record for a long time.  We know from our research that end-of-day procedures are the most prone to errors practically everywhere.

With Brenda’s promise, things started moving.

A better form

First, Maggie Ollove and Taapsi Ramchandani from CCD headed to Fairfax to see the problems first hand in the September 2017 Special Election.

They learned that the form made election officers anxious. Officers know it’s an important form. And that made them make mistakes because it wasn’t clear how to use the form. One of the biggest problems was that they tracked ballots with hash marks, leading to mistakes reading the number at the end of the day. (Two vertical marks might mean “2” or “11” for example.) Every hash mark looks the same; “did we already record this one?”

In a pause for the General Election in November 2017, Brenda started working on a new version. Her goal was to have it ready in time for the May 1, 2018 General Election.

By January, she had a draft of the form, with a lot of changes.  The new design:

• Put  information in a more logical order
• Used headings to make each section easy to find
• Put instructions where they were needed
• Dropped some unnecessary questions
• Created a better way to track special ballots

The next step was usability testing. Maggie headed back to Fairfax City in February  2018 so she and Brenda could test the form. During a quick afternoon of testing, they learned that the changes were definitely an improvement. The overall layout was better, and it was much easier to keep track of numbers of ballots. But, the reconciliation section was still hard to use. The columns didn’t line up visually, so it was hard to see which subtotals had to match.

From the moment the officers started using the form, it was clear the instructions still needed work. This was the hardest moment of the project. Everyone had worked really hard to identify the problems and come up with ideas to improve the form. It was really disappointing to see that it didn’t work as well as they hoped.

Feedback that comes through usability testing can be hard to hear, but the reason to do usability testing is to learn what works for users and what doesn’t before the form goes live in a real election.

So Maggie and Brenda went back to work. They numbered the steps in the process, improved the instructions with checklists, and borrowed a layout from tax forms to show which subtotals need to match. They eliminated more unnecessary questions. They also worked to make it work better with the companion Statement of Election Results form.

This time Brenda ran her own usability test, and found a few more details to correct.  She also used the forms during election officer training. Watching the teams work through scenarios that included typical problems yielded some very specific feedback and design tips.

Finally, the form was done and used as a pilot in 6 polling places during a general election in May 2018. All the forms were returned accurately, without calling the elections office, and without having to refer to the training manual. One of the election officers wondered why Maggie was visiting to watch them fill out this form because it was so easy and quick to fill out.  

That made all the work worthwhile.

A better process

A better Ballot Record form was an important goal. But this was also a chance to try out a quick process for updating election forms, informed by data. We wanted to shift the process from talking about the problems to trying out ideas and testing them until the whole form works.

It’s not just a better form for Fairfax City. Other localities in Virginia will pick it up and tailor it to their size and voting systems. Some larger jurisdictions reviewed the new design and had few comments.

Fairfax City used the new form again in the June 12 2018 Primary Election and had no problems. We would liketo see it used more widely in November and become a de-facto standard for the state.

The (not so) secret recipe:·

• Work with the people who use a form to understand the problems in the current form.
• Try out new ideas early, with quick usability tests that let you see what’s working and what isn’t.
• Use opportunities like election officer training to learn more.
• Keep at it until you get it right.
• Look at how a form or other design works within a larger task or job, an use the process to make the pieces work well together.