Making sure the translation is right: a guide and checklist for reviewing machine translation

Translation is complicated. One word in a language may translate into multiple words in another. Some words might not have a straightforward translation. Can machine translation be used reliably for election information?

Election offices receive hundreds, sometimes thousands, of pages of materials that must be translated, and fast. This is especially true in the weeks before an election when voter guides and ballots are being prepared. It’s a time when departments are overstretched and understaffed. Even translation vendors can turn to machine translation tools to help meet publication deadlines and language access requirements.

The technology is improving by the day and in many cases software like Google Translate is able to generate adequate translations. There’s no question that machine translation is here to stay.

However, the process of accurate translation is nuanced and so much more involved than simply pasting a block of text into Google Translate. Plus, when it comes to elections, the stakes are much higher. Election offices have a responsibility to share accurate information with voters. Mistranslation can give voters the wrong information, or create confusion.

At CCD, we see machine translation as a helpful tool, but we also know how important it is to check the work of a machine. Translation is only truly complete with a final review from a human.

What’s the best way to use machine translation without over-relying on it? What are the most common errors a machine translation tool is most likely to make, and how do we account for them?

We’ve thought a lot about these questions, so this article shares our approach to using machine translation effectively. We highlighted examples of common mistakes and what to look out for to ensure your translations are accurate and fluent. Lastly, we’ve included a checklist to follow to both prepare for and review your translations for accuracy.

Machine translation is a helpful starting tool but human translators need to check the text for accuracy, tone and grammar

Your election website, voter education, and forms are the official source for voting information for voters. The information not only has to be right, but sound competent in all languages: bad translations can reduce trust in elections!

While machine translation can provide a very quick and convenient first draft to work from, a human translator is necessary to check for mistakes. There are several types of mistakes a machine translator can make, from inaccurate translations, awkward or misleading phrases to the wrong tone of voice.

With the stakes as high as they are for elections, accuracy is essential.

All of these issues can not only give voters incorrect information but also lead to overall voters to mistrust elections and feel disconnected from the voting process. Think about the last time you got a spam email and you recognized right away something was off because of the typos, wording or spelling. An inaccurate translation will feel off to a native speaker in the same way.

The basics: how does machine translation work?

It’s easy to use Google Translate: you go online, paste in a block of text and within a few seconds you’ll have a translation generated into the language of your choice. But how does the process actually work?

It’s important to understand the fundamentals of machine translation so that it’s easier to identify the strengths and limitations of the technology.

Machine translation uses a database of previously translated sentences or phrases and then picks the one it thinks is the best match. The more text in a certain language is available online, the more reference points the machine translator can draw on, and the more accurate the translation is likely to be. However, this also means that for less common languages, machine translation will likely be much less accurate. For election information, there may only be a small data pool available. And, of course, it does not include oral language.

This method might not be a problem with single words. In fact, many translators look to machine translation as a thesaurus. But it creates challenges in understanding the syntax and overall meaning of a longer, more complex sentence. Machine translation also does not account for things like readability, context, and cultural differences.

This brings us to the best practices to prepare for machine translation.

Preparing to use machine translation: Best practices

We’ve found that a few best practices can help improve the process of generating accurate translations both for machines and for humans.

Plain language is often a good start for translations because it uses simple sentence structure. Starting with original text in plain language increases the chances that a machine translator will be able to translate accurately.

Write the original text in plain language using simple words

Before you translate text, make sure the English text is in plain language. The simpler each word, the less likely a machine will generate a confusing meaning. Simpler and shorter sentences are also easier for humans and machines to translate.

Review the English version and break up any sentence that has multiple clauses. Use the simplest possible phrasing.

Consider how to include complex legal terms

Translating complex terms is tricky because they are already difficult to understand in English.

Complex legal terms or phrases are almost guaranteed not to be translated properly by machine translation. For example, take a term like provisional voting. If you’re coming from a country that does not have provisional voting, you’re going to have to find a way to not only translate the literal words but communicate the meaning of the term.

If you have a legal code with a complex term like “Involuntary Civil Commitment”, for example, there are several different ways to handle this when preparing for a translation.

Before you translate a complex term in English, ask yourself: do we need to use the term in English? Is there a simpler way of phrasing it? If the word or phrase means something very specific and you cannot change it, consider adding a phrase that describes it in plain language.

A good rule of thumb is that if you have to ask a lawyer about a term, you should not be using machine translation to translate it.

We recommend defining these terms in plain language in English first and keeping that phrase as part of the original text immediately following the term. This has the added benefit of helping English speakers understand complex terms as well. Then you can translate the more colloquial definition.

The example below highlights what can go wrong with machine translation and why it’s so important to start with plain language in the original version. We’ve included the original version, the translation and then the back-translation, the literal meaning when translated back to English.

Translation and back-translation

Original English version
Once the vote is deposited in the ballot box, the secretary of the voting location, assisted by an electoral inspector, stamps the word “vote” in the poll book next to the voter’s name.

Spanish translation
Una vez depositado el voto en la urna, la o el secretario de la casilla, auxiliado por una o un escrutador, sella la palabra “votó” en el apartado del nombre de la persona electoral en la lista nominal.

English back-translation from the Spanish
Once the vote is deposited in the ballot box, the secretary or secretary of the box, assisted by one or a scrutineer, seals the word “voted” in the section of the name of the electing person on the nominal list.

In the example above, the term “electoral inspector” is mistranslated. This is not surprising because the term is specific to elections and relatively complex. To increase the chances of accurate machine translation, it would be helpful to review the English version first and ask: is there an easier way to say “electoral inspector”?

Is the election office obligated to use this specific term, or is there flexibility to use plain language? If the term is a legal one that cannot be changed, then consider including the English version in the Spanish translation, with a translated explanation.

A word bank will help with this as well since you will only have to go through this process one time. Once you have a proper translation, you can simply refer to your word back to see how your office has translated a term like “electoral inspector”.

Another factor that makes this sentence difficult for a machine to translate is the fact that it contains many clauses. For a sentence like this, ask yourself if there is any way to break the sentence into two or even three sentences. Shorter sentences are always preferable to ones with multiple clauses.

Create a word bank of translated election-related terms

Keep a word bank for specific election-related terms in the language your election office translates into. A great starting place is the EAC Translation Glossaries. It contains nearly 1,300 civic terms and comes in 20 different languages.

The EAC glossaries are just a starting point. You may have election terms not included, or have to make a choice of which translation to use for terms that have variations. In these cases, you will want an internal word bank for common election terms. Word banks are an excellent way to create consistency throughout your material both within your own office and in your state.

As you develop your word bank, you may need to consider variations in languages and which dialects are spoken by voters in your community. Knowing your voters is especially important because the Voting Rights Act requirements do not include this information. For example, “Chinese” might mean your voters speak Mandarin or Canonese or read traditional or simplified characters, depending on when they came to the US. Mexican Spanish is different from the Spanish spoken in Spain – and also from Dominican or Puerto Rican Spanish.

A word bank will make sure you are accounting for regional variation and speaking to your community in their native dialects.

Here’s an example of a word bank:

ENGLISH	SPANISH
Absentee Application	Solicitud de papeleta de voto en ausencia
Absentee Voter	Votante en ausencia
Ballot	Papeleta
Ballot Drop Box	Buzón electoral
Ballot Envelope	Sobre de papeleta de votación
Candidate	Candidato
Certify	Certificar

A few terms you’ll want to make sure to include in your work bank are:

Ballot
Polling place
Poll worker
Vote by mail/absentee ballot
And even the name of your office

These terms are helpful to include as they frequently get used and it’s important to ensure you are using the same translated word each time. If you’re stuck choosing between two equally valid choices when putting together your word bank, regional differences and the terms used by others in your state and county can help you decide which term to go with.

Reviewing your translation

Given the realities and demands election offices face when it comes to translation, we’ve put together a list of where machine translation can most easily fall short and how to account for that. Once you have a draft of a translation ready, you can use this list to help proofread and review for quality and accuracy.

Leave time to review and revise translations

The most important step is to do a careful check of any machine translation, preferably with a native speaker who will recognize some of the most common errors easily.

When working with a translation vendor, make sure to leave time for revisions and more than one round of edits.

A quick look at how one election office translates material
We spoke with an election office that frequently has to translate a large number of election materials under tight deadlines, sometimes needing to turn around translations of up to 140,000 words in 700 documents in under three weeks.
Here’s what the election office has said about how they handle the volume while ensuring quality.

The election office tries to follow these steps:
1. Whenever possible, translate in-house with full-time staff.
2. When sending it to a translation vendor, assume you are going to do two rounds of proofing, and rewriting 25-30% of the translation.
3. When it’s not possible to proof everything, try to proof at least 25% of the documents to check quality.

Use your word bank to check the translations of election terms

Machine translation doesn’t understand election concepts or which terms are correct for your jurisdiction, so it may choose the wrong one.

Here is an example of information translated from English to Spanish and back to English. “Polling place” has been translated to “el colegio electoral” and then back into English as “Electoral College.” The translation is 90% accurate, but that single word or phrase changes the entire meaning.

Having a word bank helps reviewers identify mistranslated words – and correct them.

You can also use a word bank to spot-check specific words and phrases. If you’re noticing a word continually gets mistranslated, make sure you update the correct translation to your word bank.

Here’s an example of a translation that went wrong because an election-related term was mistranslated.

English:
Absentee voting means voting in person or by mail before Election Day, instead of at the polling place. Any voter may vote absentee. Absentee ballots are ready 46 days before an Election.

Spanish Translation:
Votar en ausencia significa votar en persona o por correo antes del día de las elecciones, en lugar de en el colegio electoral. Cualquier votante puede votar en ausencia. Las papeletas de voto en ausencia están listas 46 días antes una Elección.

English Retranslation:
Voting absentee means voting in person or by mail before Election Day, rather than at the Electoral College. Any voter can vote absentee. Absentee ballots are ready 46 days in advance of an election.

Review the tone of voice in the translation

Without a human translator, there is no way to assess whether translated text sounds like a person barking orders without a human translator who is familiar with the language.

We tested registration forms in four different languages for California’s New Motor Voter implementation and found that even when the translations were technically accurate, they didn’t always match the friendly, helpful tone of the English version. Korean speakers noted that the translations were written very formally. For example, ”overwrite” was translated into “obliterate”, giving it a harsh, off-putting tone.

In a usability test of the Vietnamese translation of election material with native speakers, we saw that the tone of voice could impact the meaning of the text. We were surprised to find out from participants that some words in the Vietnamese translation were even offensive to voters. While the term đăng kí accurately means registration, the term is associated with forced registration drives for reeducation camps following the Vietnam War. Participants preferred the alternative ghi danh, a term used in the Tarrant County instructions.

These nuances in meaning and language can impact voter participation, which is why a human translator needs to make sure the tone is accurate and coming across the way it’s intended to.

Make sure gendered language and verb tenses are accurate

English does not use masculine and feminine for nouns, pronouns and verbs, but many languages do. We’ve noticed that machine translation can sometimes translate gender wrong. As one of the more fundamental parts of a language, when it’s inaccurate, it can come across as funny, awkward, and simply incorrect in the translated language, leading to decreased trust.

Depending on the language you are translating into, you may also have to adjust for specific verb tenses. Gender and tenses are easy mistakes for a native speaker to detect.

Look for grammar mistakes and mistranslations

When we tested voter registration forms, we found several grammatical errors that changed the meaning of sentences and resulted in incorrect translations.

For example, in Chinese, “I am not currently in a state or federal prison” was translated to “I will not be in a state or federal prison.” In Vietnamese, a missing preposition meant that “I am not in federal prison” was translated to “I am in a federal prison.”

Mistranslations like these can completely flip the meaning of the original sentence.

Checklist to prepare and review machine translation

Preparation

Make sure the original text is written in plain language
Read through for any long or overly complicated sentences
Create a word bank of translated election-related terms

Review

Ensure there is enough time in the translation plan for revisions
Look for mistranslation of election-related terms
Review the tone of voice of your translations
Make sure gendered language and verb tenses are accurate
Review potential grammar mistakes and mistranslations
Check to make sure all words are translated according to your office’s translation word bank