Neural machine translation

Neural machine translation, or NMT, is machine translation done by an artificial neural network that predicts the likely sequence of words in the target language for a sentence in the source language. Where earlier systems translated with hand-written rules or with tables of phrase pairs, a neural system learns the whole mapping from examples of translated text.

How it works#

Most NMT models have two parts. An encoder reads the source sentence and turns it into a numeric representation, and a decoder produces the translation from that representation, usually one word, or part of a word, at a time.

The step that made the approach work well on long sentences is attention. With attention, Wikipedia's article explains, the decoder works out at each step which parts of the source sentence to focus on, instead of relying on a single summary of the whole sentence. In 2017 the Transformer, described by Vaswani and colleagues in the paper Attention Is All You Need, built the whole model from attention, and it quickly became the dominant design for translation systems.

Large services moved over within a few years. Wikipedia's timeline puts Baidu's first large-scale NMT system in 2015, Google Neural Machine Translation in 2016, and the neural systems of Microsoft Translator and DeepL in 2017.

NMT and large language models#

A large language model such as GPT is built from the same Transformer parts but trained differently: on predicting text in general, not on translation in particular. Wikipedia's article notes that such models can still produce fluent and competitive translations, especially between languages with plenty of training data, without being trained on the task.

Both inherit a weakness from neural training: their output reads fluently, so a mistake reads fluently too, and the same article notes that NMT tends to produce fairly literal translations and does less well in languages with little high-quality training data. A wrong number or a dropped "not" can sit in a well-formed sentence, which is why a translation still needs a reader who checks it against the source.

Where LangRelay fits#

LangRelay translates with large language models of the general-purpose kind described above, and wraps them in the parts a website needs. A worked example of what goes with the text: for a Dutch Target locale set to Formal, the request names the variety as Dutch (Netherlands), asks for u, and lists the glossary terms found in the text. When you ask Relay for a review, it reads the translated page against the source and reports a changed number, price or date as a major finding. By default a translation goes live as soon as it is ready, and free checks hold a page with untranslated text, the wrong language or a changed price; to approve every page yourself, choose Ask me first. The machine translation entry covers the field's longer history, and post-editing website copy covers the human review.