RSS feed

Introduction: Post-editing in practice – Process, product and networks

Lucas Nunes Vieira, University of Bristol
Elisa Alonso, Universidad Pablo de Olavide, Seville,
Lindsay Bywood, University of Westminster, London

ABSTRACT

The potential benefits of integrating machine translation into human translation workflows are now widely recognised. In many sectors of the translation industry, translators’ throughput is improved with the use of machine translation as a tool in the translating process. Post-editing of machine translation is also a service in its own right, with specific guidelines and, more recently, an international standard. We introduce this special issue by providing a brief overview of post-editing as a practice, service and research topic. Contributions to the issue are then presented. The issue moves from a magnified perspective of translators’ work to important aspects of translation products, translators’ attitudes and translator training. It has four themes: the post-editing process; reception of post-edited products; attitudes and perceptions; and competence, training and education.

KEYWORDS

Post-editing, machine translation, computer-assisted translation, CAT, MT, MTPE, PEMT, translation technology, translator education, human-computer interaction.

Machine translation (MT) quality has improved considerably in previous years. Recent MT improvements are largely due to the rise of neural MT, a relatively new approach to MT development based on artificial neural networks (Bahdanau et al. 2014). Neural MT has significantly advanced MT research, but it too has weaknesses (Castilho et al. 2017). In professional contexts where translations are public-facing products, therefore, MT usually requires human post-editing. This was the case decades ago when MT systems first appeared, and it is still the case now.

Initially, post-editing was conceived as something akin to a final step in the MT development workflow. Early research on MT development did not see post-editing as ideal because “[i]t leaves the final step in the decoding process, the determination of incident meaning, to a human agent” (Reifler 1952: n.p.). The interaction between humans and MT systems was, from the outset, deemed necessary in various contexts, but in the early days of MT research this interaction was clearly skewed towards unaided MT as a core goal. In 1951, the mixture of humans and MT systems was described as ‘Mixed MT’, a process where the machine had a ‘human partner’:

For those targets in which high accuracy is a conditio sine que non, pure MT has to be given up in favor of a mixed MT, i.e., a translation process in which a human brain intervenes. There the question arises: Which parts of the process should be given to the human partner? (Bar-Hillel 1951: 230).

While mostly a matter of language, it is clear from early records that post-editors – and pre-editors – had a peripheral role in the MT-based translation process. They were merely MT’s ‘partners’. In fact, some early articles show that post-editors were not even expected to have knowledge of the source language. The job could be a target-language-only review. Requiring knowledge of just on one language, it was thought, would make it easier for MT to be scaled by avoiding the need for expert translators, who were scarce and then described as a bottleneck in translation provision (Bar-Hillel 1951: 230).

The possibility of carrying out post-editing based only on the target language (often referred to as monolingual or blind post-editing) has been investigated with some positive results (see e.g. Mitchell et al. 2013). Having non-professionals carry out post-editing is also a possibility in the context of multilingual communicative services, for instance1. However, post-editing is increasingly carried out by professional translators. It is now also a client-facing service with its own international standard (ISO 18587 2017), which requires post-editors to have formal translation training and/or professional experience. In this context, post-editors are no longer passive assistants, but rather experts in charge of a professional task.

From technological and user-experience perspectives, the environment in which post-editing takes place has also changed. Early MT developers were often frustrated with the goal of achieving high-quality translations fully automatically. Computer-assisted translation (CAT) tools gained traction largely out of this frustration (Hutchins 1995). This was a new paradigm where technology assisted professional translators and not one where non-professionals assisted the technology. A key feature of CAT is translation memory, which allows human translations to be stored and automatically retrieved for recycling and re-use. Translation memory remains the flagship of CAT, but MT is now available in most CAT tools and plays an increasingly important role in their use.

The use of MT as a CAT feature brought MT technology to the centre of translators’ working environment. This is the case in situations where post-editing is a client-facing service as well as where MT is simply used as a tool in the translation process (see e.g. Vieira and Alonso 2018). However, after decades of being regarded as little more than a step in MT development, or a solution that could bypass expert translators, post-editing unsurprisingly has a negative reputation. Research investigating translators’ views on the use of MT in professional workflows shows that attitudes to post-editing are particularly negative among experienced translators (Moorkens and O’Brien 2015). The reasons for this negativity range from poor MT quality (Läubli and Orrego-Carmona 2017) to MT’s potential effect on the translation market (Vieira 2018).

However, post-editing’s bad reputation has also acted as an incentive to improve translators’ MT use experience. Projects such as CASMACAT (Koehn et al. 2015) and MateCat (Federico n.d.) have looked extensively into ways of advancing human-computer interaction. Among other aspects, studies linked to these projects investigated the use of MT quality estimation to flag problematic MT passages and examined interactive post-editing with online learning, where the MT output learns from translators’ edits on the fly. Moreover, in some contexts translation memory and MT can now be fully combined so that translation memory content can be used to fine-tune the MT output, which further heightens the degree of integration of the various features and technologies at translators’ disposal.

From a taxonomic perspective, higher integration and the several ways in which MT can be used in the translation process have also blurred the lines between practices and technologies. It is becoming increasingly difficult to differentiate between human- and machine-sourced aids in the translation process – for example in situations where translators can use their translation memories to tune MT systems that can then interact with human edits in real time. Coupled with post-editing’s bad reputation, the recent centrality of MT use in CAT environments is linked to a state of terminological instability where ‘post-editing’ can be seen to represent different tasks.

Lilt, for instance, a tool that integrates translation memory with interactive and adaptive MT states on its website that it “replaces post-editing with interactive and adaptive machine assistance”2. Similarly, regarding the availability of MT in CAT tools as an optional information source, market research company Common Sense Advisory states that “this is not post-editing” [emphasis removed] since “linguists can use MT or not as they see fit” (Lommel 2018). Attempts to avoid the term ‘post-editing’ as initially conceived are to be expected given the previously mentioned negativity associated with the term. Interactive and adaptive machine assistance indeed changes the MT use process. Rather than departing from a static MT suggestion as a default starting point, interactive MT arguably gives translators more freedom in adjusting the extent of MT use as they work through a text. More integration also allows professional translators to remain at the helm while even some aspects of project management can be automated and streamlined as part of a comprehensive interactive framework sometimes referred to as ‘augmented translation’ (Lommel 2018). However, since integrating MT into the translation process still essentially involves human use and editing of machine suggestions (statically or interactively) it is debatable whether interactive machine assistance and higher integration of MT and translation memory are indeed a complete departure from post-editing or, rather, if they simply represent a new post-editing method centred on humans rather than MT.

The terminological instability around the use of MT in translation tasks is, in our view, largely a matter of perspective. Previous research, for instance studies carried out in the context of the CASMACAT project, referred to “interactive post-editing” (e.g. Koehn et al. 2015: 27) without according this editing mode the status of a completely new activity. The ISO 18587 standard also recognises that post-editing can take place in “fully integrated environments” involving translation memories and terminological resources (ISO 18587 2017: 5). Our position, therefore, is that an encompassing understanding of the term is, at present, more productive than a restrictive approach that risks excluding specific contexts. Along with previous initiatives aimed at bringing post-editing closer to CAT (e.g. O’Brien 2016), this special issue provides a comprehensive perspective on post-editing tasks. It regards post-editing as an activity that has evolved since the first days of MT technology to include static as well as interactive and adaptive use of MT in environments that include translation memories and other resources. We also approach post-editing as a productivity-enhancing practice and not just as a full-blown service as per the ISO 18587.

Nomenclature issues notwithstanding, the use of MT in the human translation process is increasingly pervasive, which indeed calls for reflection on broader aspects linked to the role of technology in translation and translator training. From a research perspective, the range of topics addressed by studies in this area has widened. Just five years since a previous special issue was published on this subject (O’Brien and Simard 2014), we have seen increased interest in translation products and their end-users, in the place of MT in translator education, in new technologies such as neural MT and in sociological and affective aspects of technology use in translation. Widening the approach to research on MT and translation technology arguably contributes to further maturing the study of these topics. Post-editing, by its very nature, has been at the intersection of various disciplines including Translation Studies, Natural Language Processing and Cognitive Psychology. As the use of MT in the human translation process becomes more widespread, important links to other cognate areas start to develop, including Sociology and Education. It is our view that this plurality of approaches can only be enriching to the field and we are confident that cross-disciplinary research on post-editing will increase.

We believe the articles in this issue are representative of the ways in which this research area is evolving. Collectively, the articles document current trends and point to important future directions in the investigation of the role of MT in Translation Studies. The article contributions also represent the diversity of research in this field. They cover different technologies; institutional, commercial and educational contexts; different text genres; and, importantly, languages that are still relatively under-researched in relation to post-editing, including Welsh, Chinese and Japanese. Some contributions also leave lab environments to conduct research in professional settings involving on-site data collection and the direct participation of translation professionals, including project managers. On neural MT, the articles point to critical aspects that should be considered in the use of this technology in professional tasks as well as in translator education. In addition, the articles rely on a wide range of methodological approaches – both qualitative and quantitative. They deal with issues pertaining to post-editing processes, products and the social networks where MT and post-editing are taught and put into practice. Given the variety of research angles presented by the articles, we hope that this special issue helps to strengthen this area of research and spark interest in new avenues of enquiry on the use of MT in translation processes.

The articles appearing in the issue are introduced below under four themes: the post-editing process; reception of post-edited products; attitudes and perceptions; and competence, training and education.

1. Contributions to this issue 
1.1. The post-editing process

The articles under this theme focus on a range of aspects of how post-editing is carried out. They examine factors including CAT tool features, text genres, MT quality and MT system architectures. The article opening this section, Is the concordance feature the new black? A workplace study of translators’ interaction with translation resources while post-editing TM and MT matches, is by Kristine Bundgaard and Tina Paulsen Christensen. They carry out a workplace study of post-editing behaviour in a CAT environment where translators make use of translation memory as well as MT matches. Based on tasks carried out by seven professional translators from English into Danish, Bundgaard and Christensen show that the concordance search function is translators’ preferred CAT feature. Furthermore, they show that translators tend only to leave the CAT environment when tool-internal resources prove unhelpful. Based on these results, Bundgaard and Christensen call for more research on the use of concordance searches and conclude that in business contexts this feature has replaced bilingual dictionaries as translators’ first port of call.

The second article in this theme focuses on how metaphors are dealt with in the post-editing process. In Investigating the post-editing effort associated with machine-translated metaphors: a process-driven analysis, Arlene Koglin and Rossana Cunha use eye tracking and retrospective think-aloud protocols to examine the process of post-editing a metaphor-rich text translated from English into Brazilian Portuguese using two MT systems. They observe that metaphors translated with Systran often required a lower level of cognitive effort to post-edit than metaphors translated by Google Translate. The authors stress that further work is required to confirm the effect of MT system architecture in such a context. However, they posit that, compared to Google Translate’s output, Systran’s hybrid technology is likely to produce more literal translations that may in fact be more helpful clues to the appropriate translation of metaphors than statistical phrase suggestions that bear no resemblance to the source text.
The next two articles in this theme focus on neural machine translation (NMT) and how it may change the post-editing process and post-edited products. In How does the post-editing of neural machine translation compare with from-scratch translation? A product and process study, Yanfang Jia, Michael Carl and Xiangling Wang examine the translating process and target-text quality in from-scratch translation and NMT post-editing. The study is based on the English-to-Chinese language pair. They also investigate the effect of general versus domain-specific source texts, a relatively under-researched topic in post-editing. The study was conducted in an educational context, with translation students as participants. They found that post-editing of NMT was only significantly faster than from-scratch translation for domain-specific texts, though cognitive effort was found to be lower for post-editing of NMT across text types. The authors also observed that post-editing of NMT and from-scratch translation produced target texts of equivalent fluency and adequacy. In The impact of Google Neural Machine Translation on post-editing by student translators, Masaru Yamada examines NMT in English-to-Japanese post-editing tasks. He compares post-editing of NMT to post-editing of phrase-based statistical machine translation (PBMT) in terms of students’ error correction rates, their perceived cognitive effort, the amount of editing they carried out and the corresponding products. He found that NMT post-editing required less editing, but similar levels of perceived cognitive effort relative to PBMT post-editing. He also observed that, while NMT post-editing produced target texts with fewer errors, students’ error correction rate in NMT post-editing was poorer compared to post-editing of PBMT. These results led him to posit that NMT requires fewer but more effortful edits and that NMT does not improve trainee translators’ performance or empower them to become better professionals.

The last article in this theme presents a cross-linguistic study with a heavy focus on methodology. In Machine translation errors and the translation process: A study across different languages, Michael Carl and M. Cristina Toledo Baez contrast source text strings, translation quality and translating behaviour in post-editing and from-scratch translation based on English-to-Spanish and English-to-Chinese tasks. They show that from-scratch translation is more difficult for source-text words associated with MT accuracy errors. This was observed in tasks involving very different languages such as Spanish and Chinese. The authors conclude that low MT accuracy can act as a sign of source-text strings that are difficult to translate for both humans and machines, a result that was not observed for fluency. The article suggests accuracy as a more generalisable concept in translation quality and translating difficulty.

1.2. Reception of post-edited products

This theme contains two articles focusing on how end-users receive post-edited products. In Eye-tracking translation quality: What effect does post-editing have on the translation product from an end-user’s perspective? Benjamin Screen presents a reading experiment where end-users are exposed to Welsh target texts produced via post-editing and via from-scratch translation of the same English source text. He contrasts the post-editing and from-scratch conditions based on eye-tracking data and self-reported scores provided by the participants on the comprehensibility and readability of the texts. Based on these metrics, Screen concludes that from the perspective of end-users the post-edited text was of equivalent quality compared to the text translated from scratch and that this further supports the use of MT in professional settings. Another point of interest in this study is the focus on Welsh, a language with an increasing body of MT research, but which is still under-investigated in translation technology. In the second article appearing in this theme, Why go the extra mile? How different degrees of post-editing affect perceptions of texts, senders and products among end users, Gys-Walt van Egdom and Mark Pluymaekers investigate how end-users perceive machine translations subjected to different degrees of post-editing. They asked end-users to judge the translations according to a series of factors including style, usability and text logic, and terminology. The experiments were based on two texts, one informational and one technical, which had been post-edited to four different levels: minimal, light, moderate and full. They observed that texts subjected to a higher level of post-editing were not necessarily preferred by the end-users, which was the case particularly for the informational text. Van Egdom and Pluymaekers call attention to the fact that quality is context-dependent and that clients should carefully consider the consequences of choosing different levels of editing in post-editing commissions.

1.3. Attitudes and perceptions

The two articles in this theme widen the focus and discuss important aspects of how machine translation is perceived and how this may affect translators’ work. In Uses and perceptions of machine translation at the European Commission, Caroline Rossi and Jean-Pierre Chevrot investigate perceptions of MT in the European Commission’s Directorate-General for translation (DGT). Based on a three-week research stay in the French translation department and a subsequent survey distributed to the entire DGT, they show that knowledge of MT is inversely connected to perceptions of MT as a threat. Their results also demonstrate that translators make use of MT systems in myriad ways that have often been ignored by previous research, such as using it for inspiration and lexis. Rossi and Chevrot conclude that current models of human-computer interaction only partially account for the wide range of feelings towards and experiences with MT. In Why do many translators resist post-editing? A sociological analysis using Bourdieu’s concepts, Akiko Sakamoto draws on Bourdieu’s concepts of capital, field and habitus to present a sociological analysis of translators’ attitudes to post-editing. Her study is based on post-editing training manuals, focus-group interviews conducted with sixteen UK project managers and a survey of 155 company websites. She proposes that current perceptions of post-editing place post-editors in a different position from translators across axes representing the volume of capital (i.e. remuneration) and the capital type (cultural or economic). While she acknowledges this discussion will need refinement as the field evolves, Sakamoto presents a hypothetical model where post-editors can be on the same level as translators on the capital volume axis. She concludes by saying that this would represent a healthier model for introducing post-editing services to translators.

1.4. Competence, training and education

The final theme in the issue looks at critical aspects of what MT means for translator training and translator competence. In Machine translation and post-editing training as part of a master’s programme, Ana Guerberof and Joss Moorkens provide a description of post-editing and MT project management modules that are part of the localisation Master’s curriculum at the Universitat Autònoma de Barcelona. They present a detailed analysis of the modules, including aspects relating to assessment, and suggestions on how to deal with the challenges posed by the teaching of this topic. The post-editing module includes components addressing post-editing effort, post-editing levels as well as pricing issues. The MT project management module is based on a hands-on group task where students can apply knowledge and skills developed throughout the programme. Guerberof and Moorkens also discuss NMT and how it is likely to affect post-editing practice and translator training. They argue that the benefits of NMT can often be oversold and that this new technology is unlikely to render post-editing unnecessary. They approach it with scepticism but also optimism. In Risk management and post-editing competence, Jean Nitzke, Silvia Hansen-Schirra and Carmen Canfora apply the concepts of risk management and competence to post-editing practice. They first propose a decision tree including different risk assessment criteria that, they suggest, should be incorporated into considerations regarding the extent of MT use in professional translation provision. Their model includes aspects relating to text types, timescale, MT quality and data security. Subsequently they draw on their decision tree’s risk assessment criteria to present a competence model for post-editors that outlines the competences necessary to make informed decisions regarding risk, MT use and post-editing practice. Their model comprises a set of core competences, including risk assessment, strategy, service and consultancy aspects, as well as a set of subsidiary competences, including aspects relating to research, MT, post-editing, translation and linguistic and extra-linguistic skills. They call for a comprehensive approach to post-editing services that embeds risk management as a key component from the outset. The last article in this theme, and in the issue, takes stock of the current place of MT and post-editing in Master’s programmes from the European Commission’s European Master’s in Translation (EMT) Network. This is a contribution in Spanish, Análisis DAFO sobre la inclusión de la traducción automática y la posedición en los másteres de la red EMT [SWOT Analysis of the Inclusion of Machine Translation and Post-Editing in the Master's Degrees offered in the European Master’s in Translation (EMT) Network], by Cristina Plaza Lara. The article presents a SWOT (Strengths, Weaknesses, Opportunities and Threats) analysis of the inclusion of MT and post-editing in EMT programmes and uses content analysis techniques to examine the institutions’ websites and extra information provided by programme coordinators. The sample includes 66 modules from 46 different programmes where there was a clear mention of MT and post-editing. The strengths and opportunities identified include, respectively, links between post-editing and revision and the emergence of MT and post-editing as a new niche market. The weaknesses and threats include a tendency for the programmes to have only an introductory focus on MT and post-editing and the fact that teaching components concerning post-editing production tasks only, rather than MT development, tend to prevail.

The diversity of research objectives in this issue reflects an extremely dynamic field. By first focusing on the post-editing process then moving to a discussion of post-edited products, the ways in which MT and post-editing are perceived, the competence these topics require and how they are taught, we hope that this issue of JoSTrans helps to consolidate research on MT and post-editing in Translation Studies.

References
  • Bahdanau, Dzmitry, Cho, Kyunghyun and Yoshua Bengio (2014). “Neural machine translation by jointly learning to align and translate.” arXiv preprint arXiv:1409.0473.
  • Castilho, Sheila, Moorkens, Joss, Gaspari, Federico, Calixto, Iacer, Tinsley, John and Andy Way (2017). “Is neural machine translation the new state of the art?” The Prague Bulletin of Mathematical Linguistics 108, 109-120.
  • Hutchins, W. John (1995). “Machine translation: A brief history.” E.F.K. Koerner and R. E. Asher (eds) (1995). Concise History of the Language Sciences: From the Sumerians to the Cognitivists. Oxford: Pergamon Press, 431-445.
  • ISO 18587 (2017). Translation services – Post-editing of machine translation output – Requirements. Geneva: International Organization for Standardization.
  • Koehn, Philipp, Alabau, Vicent, Carl, Michael, Casacuberta, Francisco, García-Martínez, Mercedes, González-Rubio, Jesús, Keller, Frank, Ortiz-Martínez, Daniel, Sanchis-Trilles, German and Ulrich Germann (2015). CASMACAT: Final public report. http://www.casmacat.eu/uploads/Deliverables/final-public-report.pdf (consulted 30.10.2018).
  • Läubli, Samuel and David Orrego-Carmona (2017). “When Google Translate is better than Some Human Colleagues, those People are no longer Colleagues.” João Esteves-Ferreira, Juliet Macan, Ruslan Mitkov and Olaf-Michael Stefanov (eds) (2017). Proceedings of the 39th Conference on Translating and Computer, London, November 16-17. Geneva: Tradulex. 59-69. https://www.asling.org/tc39/wp-content/uploads/TC39-proceedings-final-1Nov-4.20pm.pdf (consulted 01.12.2018).
  • Lommel, Arle (2018). “Augmented translation: A new approach to combining human and machine capabilities.” Proceedings of the 13th Conference of the Association for Machine Translation in the Americas (Volume 2: User Papers), Boston, March 17 - 21, 2018. http://www.aclweb.org/anthology/W18-1905 (consulted 30.10.2018).
  • Mitchell, Linda, Roturier, John and Sharon O’Brien (2013). “Community-based post-editing of machine-translated content: monolingual vs. bilingual.” Sharon O’Brien, Michel Simard and Lucia Specia (eds) (2013). Proceedings of the MT Summit XIV Workshop on Post-editing Technology and Practice, 2 September 2013, Nice, France, 35-43. http://www.accept.unige.ch/Products/2013_wptp2_wp7.pdf (consulted 30.10.2018).
  • Moorkens, Joss and Sharon O’Brien (2015). “Post-editing evaluations: Trade-offs between novice and professional participants.” İIknur Durgar El-Kahlout, Mehmed Özkan, Felipe Sánchez-Martínez, Gema Ramírez-Sánchez, Fred Hollowood and Andy Way (eds) (2015). Proceedings of European Association for Machine Translation (EAMT) 2015, Antalya, 75–81. http://www.aclweb.org/anthology/W15-4910 (consulted 30.10.2018).
  • O’Brien, Sharon (2016). “Post-Editing and CAT.” EST Newsletter 48, 10.
  • O’Brien, Sharon and Michel Simard (2014) “Introduction to special issue on post-editing.” Machine Translation 28(3-4), 159-164.
  • Reifler, Erwin (1952). “Mechanical translation with a pre-editor and writing for mechanical translation.” Paper Presented at the Conference on Mechanical Translation (Massachusetts Institute of Technology, June 1952). http://www.mt-archive.info/MIT-1952-Reifler-1.pdf (consulted 30.10.2018).
  • Vieira, Lucas N. (2018). “Automation anxiety and translators.” Translation Studies. Online First. DOI: 10.1080/14781700.2018.1543613.
Biographies

Vieira_PortraitLucas Nunes Vieira is a Lecturer in Translation Studies with Technology at the University of Bristol (United Kingdom). He researches the use of machine translation in human translation workflows, with a focus on the cognitive aspects that surround the use of technology in translation tasks and how this affects processes, products and attitudes. Lucas has a BA in language and linguistics from Universidade Federal Fluminense (Brazil), an MA in language technology from Universidade do Algarve (Portugal) and Université de Franche-Comté (France), and a PhD on machine translation post-editing from Newcastle University (United Kingdom).

E-mail: l.nunesvieira@bristol.ac.uk

Alonso_PortraitElisa Alonso is a full-time lecturer and researcher in Translation Studies at the Universidad Pablo de Olavide (Seville, Spain), where she currently teaches at undergraduate and postgraduate levels. She holds a PhD in Communication Studies, from Universidad de Sevilla, and a BA in Translating and Interpreting, from the Universidad de Granada. Her research interests include the impacts of technology on sociological aspects of translation and on translators’ training. Before lecturing, Elisa Alonso worked as a software localiser at Lionbridge and as a freelancer.

E-mail: elialonso@upo.es

 

Bywood_PortraitLindsay Bywood is a Senior Lecturer at the University of Westminster and Deputy Editor of the Journal of Specialised Translation. She holds a PhD from University College, London, an MA in German and Philosophy from the University of Oxford and an MA in Translation from the University of Salford. Up until five years ago she worked in the translation industry, most recently as Director of Business Development at VSI, an international subtitling and dubbing company with headquarters in London. She teaches translation, project management, AVT, and professional skills for translators, and her research interests include diachronic variation in the subtitling of German films into English, machine translation, and the didactics of translation.

E-mail: L.Bywood@westminster.ac.uk

Acknowledgements

We would like to express our sincere gratitude and appreciation to the 28 anonymous peer reviewers who evaluated the contributions received for this issue. Thanks also go to Mikołaj Deckert, Łucja Biel and Lucile Desblache for their guidance and support throughout the editorial process.

Notes

Note 1:
See e.g. https://unbabel.com/translators/
Return to this point in the text

Note 2:
https://lilt.com/kb/memory/mt
Return to this point in the text