Uses and perceptions of machine translation at the European Commission
Caroline Rossi, ILCEA4, Univ. Grenoble Alpes
Jean-Pierre Chevrot, LIDILEM, Univ. Grenoble Alpes
ABSTRACT
Current research on translation technology seeks to integrate physical, cognitive and organisational ergonomics, and uses insights from the situated cognition paradigm to bring together social and technical perspectives on fast-evolving human-computer interactions. Even though these trends imply that a wider variety of professional contexts should be considered, studies of institutional translation are still scarce. This paper reports on a three-week research stay in the French language department of the European Commission (DGT-Fr2), aimed at understanding current uses and perceptions of machine translation (MT) and post-editing within Europe’s biggest translation institution. Based on ethnographic data, we established a survey that we tested among French translators before translating it into English and submitting it to all DGT translators. Our quantitative data include 89 respondents from 15 language departments. We perform multiple linear regressions to assess technology acceptance, before focusing on the variance that the model leaves unexplained. Our findings show that perceptions of control, subjective norm and image, as well as insecurity (fear of MT) have an impact on professional MT acceptance.
KEYWORDS
Machine translation (MT), post-editing (PE) Technology Acceptance Models (TAMs), Directorate-General for translation (DGT), human-computer interaction, ergonomics, human factors.
Introduction
Current research on translation technology seeks to integrate physical, cognitive and organisational ergonomics (Lavault-Olléon 2011) and uses insights from the situated cognition paradigm (Martín 2010; Risku 2010) to bring together social and technical perspectives on fast-evolving human-computer interactions (O’Brien 2012; Cadwell et al. 2016). Even though these trends imply that a wider variety of professional contexts should be considered, studies of “translating institutions” are still scarce (Koskinen 2008).
This paper analyses the data gathered during and following a three-week research stay in the French language department of the European Commission’s Directorate-General for translation (DGT-Fr2). Our overarching aim is to understand current uses and perceptions of machine translation (MT) and post-editing (PE) within Europe’s biggest translation service. We used existing scales and models designed to study technology acceptance and combined them with ethnographic data that helped us adjust to the specific institutional setting under study. In what follows, we start by describing the context of MT at the DGT (Section 2), before detailing the theoretical framework and methodological triangulation that we used to design a large-scale survey (Section 3). The results are presented using both descriptive statistics and linear regressions in Section 4, and discussed in Section 5.
2. MT at the European Commission’s DGT
The DGT is one of a kind: not only is it “[a]rguably the largest translation agency in the world” (Koskinen 2008: 69), but its mission is also unparalleled, notably in terms of “the unprecedented scale of their multilingual operations and the legal and political importance of translation” (Svoboda et al. 2017: 2). The production of 24 different language versions of each piece of EU legislation, as per Council Regulation No 1 of 1958, is indeed key in making sure all EU citizens get the information they need about their rights. The DGT is also unparalleled in terms of its human, linguistic and financial resources, which make it a model for translator trainers (Rossi 2017: 50). These elements are central to the following statement of the DGT’s vision:
We aspire to be a full partner in the legislative and communication processes, the hub for all translation-related activities at the Commission and a reference in the world of translation, while contributing to the development of each official language and the translation profession (DG-Translation, 2016b: 3).
However, the DGT currently works under tight budgetary constraints: in accordance with the objective of “reducing the cost of translation”, the DGT implements “continued staff reductions” while “meeting unflagging demand” (DG-Translation 2017: 3). Thus, while the number of translated pages is now well over two million, the number of translators has gone down from about 1,750 in 2015 (Cadwell et al. 2016: 226) to about 1,560 as we write this paper (DGT, personal communication).
Figure 1. Evolution in number of translated pages and DGT staff, including translators and other staff members (Source: DGT).
Action has been taken to bridge the resulting gap, which is also patent in overall DGT staff, as shown in Figure 1. Official documents indicate that this action has been of two kinds. First, outsourcing has increased steadily. The 2016 activity report indicates that the proportion of pages translated externally (as a percentage of the total number of translated pages delivered) has grown from 27% in 2015 to 29.5% in 2016, and is set to reach about 31% in 2018 (DG-Translation 2016a: 6). Second, automation and the continuous improvement of language services and tools – notably the Commission’s MT engine – have made an important contribution. In the next section, we review major changes leading to the current workflow.
2.1. Translation tools and workflow: automation at the DGT
The European Commission’s MT system was developed with a view to building a Digital Single Market: ever since its creation, the chief objective of MT@EC1 has been to “help European and national public administrations exchange information across language barriers in the EU, Iceland and Norway” (European Commission 2016: 5). The Commission’s system has nonetheless been ”offered by the European Commission to translators in the EU institutions’ translation departments” since its initial development in 2010, and MT@EC is now deeply integrated in the current workflow. It has actually been cast as an additional resource, though one that ought to be taken with a “pinch of salt” (Cadwell et al. 2016: 228), i.e. used critically by translators, who were thus encouraged to deepen their awareness and understanding of MT.
As shown in Figure 2 below, automation starts when a translation request is sent (through the “Poetry” portal) to the DGT’s central translation memory (TM) called Euramis: this allows all relevant previously translated content to be automatically retrieved and stored in a local translation project. Managers (heads of units) get projects in ManDesk and translators start from those projects to create the translation file in TraDesk. All projects integrate MT@EC outputs together with TMs, but with a strong penalty automatically assigned to the former. As they select resources, translators can untick the MT output and thus choose not to use it at all. The auto-suggest function in their current translation environment (based on SDL Trados Studio) also enables them to discard the MT output and continue typing. Finally, the circular flow in Figure 2 shows that all final versions of translated documents are sent back to Euramis, which will be used in its updated version for each new round of MT training.
Finally, automation is also applied to outsourcing: part of the streamlined workflow described above has been used since October 2016, enabling freelance translators to work in a similar translation environment and (most importantly) with the same resources as DGT translators: “outsourcing with sdlxliff files was introduced as the standard outsourcing method” (DG-Translation 2017: 16).
Figure 2. The DGT’s streamlined workflow
(Source: European Commission, 2016: 5).
We hope to have shown that technology is currently ubiquitous at the DGT, and that translation tasks are embedded in an extremely well-designed workflow. Within this intricate interweaving of technology and human intervention, machine translation, which appears only in the form of easily dismissed segments, may well be of minor importance and its impact limited. To date, however, very few studies have assessed the impact of MT’s inclusion in translation workflows, which “has led to a need for more in-depth studies on the ergonomics and human factors associated with MT use” (Cadwell et al. 2016: 225). In what follows, we explain how we went about analysing translators’ interactions with MT in terms of both uses and perceptions.
3. Assessing uses and perceptions of MT in DGT translators
In order to take into account the specific institutional environment that we described in the previous section, we started by immersing ourselves in the daily routines of the French language department, which welcomed one of us as a visiting professor. Caroline Rossi did unstructured observation in and around DGT-FR.2 for three weeks in January 2017, and conducted semi-directed interviews to gather qualitative data that would help us ask relevant questions to a larger population of translators. This section describes the different steps that led to the design of a large-scale survey on current uses and perceptions of MT at the DGT. The questions we asked in the ethnographic phase of the study are provided in Appendix 1. The survey questions are provided in Appendix 2 together with descriptive statistics.
3.1. Ethnographic data
The ethnographic data collected during our stay comprises field notes of our daily observations, as well as 10 semi-directed interviews. We noticed that overall MT acceptance was quite high and daily use could be observed in most cases. It was also very clear that whilst many translators did choose to use MT, at least some of them also opted out of using the MT outputs in a number of instances. We noticed that there were individual patterns of use that were at least partly linked with the source language(s) the translator was working with. However, considering the sheer number of language combinations that we were likely to find when surveying the whole DGT (as of 2016, MT@EC covered “552 language pairs with 62 direct language pairs”, Cadwell et al. 2016: 227) we decided not to discriminate between language pairs in our survey.
In observing procedures related to automation, we also considered the relatively recent appointment of three “workflow managers”, who remain active translators among the 30+ members of DGT-FR2: their role is to ease and optimise the assignment of translation projects across the team. This enables everyone to work more smoothly, but some of the translators also noticed that it was much nicer to be able to discuss the translation project with the head of unit, as was the case previously when he/she brought the project in person to each individual translator. Although they apply to a specific situation, such remarks are a clear instance of what Marshman (2012: 10, in LeBlanc 2017: 59) describes as an effect of “the human factors and policies in tools’ implementation” rather than the technology itself.
Semi-directed interviews helped us gather more data on those human factors, and we were especially interested in how translators described their interactions with MT. Based on Cadwell et al.’s landmark study, we expected to gain rich insights into human factors associated with MT at the DGT. This is because DGT translators enjoy “relative job security, recognition, and a freedom not typically enjoyed by professional translators in other settings” (Cadwell et al. 2016: 329) which in turn allows for “the needs, abilities, limitations, and well-being of translators to be matters of central concern when considering MT adoption” (ibid.).
There was diversity in our data, confirming that translation technologies “are moulded by and impact upon humans in all sorts of ways” (Kenny 2017: 1). Among translators voicing neutral to positive attitudes to MT, MT@EC was in most cases cast as a typing aid and/or as a mere time saver, with no mention of annoyance and no affective reaction. On the other hand, more critical depictions of MT were associated with affective reactions. These included humorous personifications of MT as both an annoying salesperson repeatedly offering their silly solution (« la solution qu’elle essaye de te fourguer depuis le début » [the solution that the system has been trying to fob me off with from the beginning]) and a scapegoat that translators were quick to blame when they were dissatisfied and frustrated as a result of time pressure (« il faut bien qu’on passe nos nerfs sur quelqu’un » [we have to take it out on someone]).
One of our questions (see Appendix 1) helped us assess general knowledge about how MT works: all answers indicated good overall knowledge. Interestingly, however, the first reply that we got consistently amounted to confessing ignorance, and there were many expressions of uncertainty even in the most informed answers. Everything translators said about their knowledge helped us design a survey question (Appendix 2, question 25), to see whether similar knowledge would be found across the DGT, and if it could be linked with uses and perceptions in any way.
Finally, the answers we received to our question about future uses of TMs and MT in about ten years’ time were surprisingly emphatic on the amount of change that translators expected. Statements were usually based on an appreciation of the rapid changes that translators had witnessed (including the “irresistible rise” of MT) but they also brought a number of expressions of fear that we had not expected: MT was cast as a threat to the human translator in four cases out of ten. On the other hand, translators with neutral to positive attitudes to MT were rather optimistic and mentioned the “constant progress” that was likely to help them more and more. In order to test for the existence of such polarisation across the DGT, we chose to reuse a survey question that we had recently tested and validated with trainee translators (Rossi 2017): answers on a 5-point Likert scale ranged from an appreciation of MT as very threatening, and up to very helpful (Appendix 2, Question 22).
3.2. Technology acceptance models (TAMs)
The core constructs used in our survey are those that are at the heart of TAMs, a series of models developed from a highly successful proposal: “[t]he most widely employed model of IT adoption and use is the technology acceptance model (TAM) that has been shown to be highly predictive of IT adoption and use ” (Venkatesh and Bala 2008: 274). We chose to use them since they appeared to be the most robust, well-established instruments that could give us a coherent picture of the factors related to an individual’s acceptance and use of a new technology. It has to be said, however, that TAMs have been explicitly built with a view to predicting “individual adoption and use of new ITs” (Venkatesh and Bala 2008: 275) thus helping managers to design successful interventions. While the focus of TAMs is more on the technology itself and the individual factors that are likely to foster acceptance and success, we started out with human factors and used TAMs as part of a larger evaluation of MT acceptance and use. This accounts for the parsimonious addition of factors in the model (namely “fear of MT”2 and “perceived impact of MT”), as well as for the deletion of elements that were not relevant to current uses of MT at the DGT. Figure 3 illustrates our adapted version of the third TAM proposed by Venkatesh and Bala (2008), i.e. the hypothesised interactions between the core constructs used to design the survey.
Figure 3. An adapted version of TAM3 (Venkatesh and Bala 2008: 280).
For each construct in the model, measures are achieved using a series of indicators. Having tested and validated reliability using multiple-question scales in a preliminary version of our survey, we avoided repetition of questions bearing on the same construct in the final survey, whenever we could, so as not to over-burden respondents.
3.3. Critical approaches
Existing critiques of TAMs stress that they are individual models that fail to capture social and organisational dimensions (see e.g. Legris et al. 2003: 202). As for organisational dimensions, we expected DGT translators to be a rather homogeneous social group, and one in which translators enjoyed “freedom, recognition and relative job security”, as explained above based on Cadwell et al. (2016: 329). Indeed, one important feature was that translators genuinely had a choice, so the construct of (perceived) voluntariness was not relevant to our setting. Our ethnographic data clearly confirm Cadwell et al.’s results in this respect: not only were translators free to choose which TMs (including MT), and even which translation environment they worked with, but we did see them choose and adjust a number of times, expressing preferences that were based on their experience and expertise. No current version of TAMs is likely to capture such elements, which we believe are extremely specific to institutional translation, and maybe even to the DGT only. What we may expect as a result is more unexplained variation.
Social dimensions, on the other hand, are only included in TAMs in terms of the social influences constraining technology acceptance – namely, image and subjective norm. The models do not include other aspects, such as the social origins, dynamics and consequences of translation technologies for professional translators. One idea that might help in understanding those social aspects, and which our statistics are not likely to capture, is that technology is not neutral and its impact will depend on the context in which it is developed and promoted. This is what Science and Technology Studies, as well as social-constructivist approaches have shown (Kenny 2017: 2; Olohan 2017: 270). Interestingly, while MT has been cast as a necessity with respect to sustaining a multilingual Europe, especially with the recent development of the CEF platform3, no such determinism appears in discourse about MT@EC development for translators’ use at the DGT, even though considerable effort and attention have been dedicated to it. What has been visible, however, and potentially perceived as a threat by some, was the huge investment in technology development, at a time when fewer translators were hired:
The European Commission has already invested more than €200 million over the last seven years on research and innovation in language technologies that have the potential to break through language barriers (Ansip 2016).
Overall, our ethnographic data revealed good MT acceptance and frequent use, as well as patent variation and polarisation on the assessment of future prospects. In order to capture this variation, taking into account social and institutional aspects, we included questions about the translators’ profile. In the next section, we give an overview of survey categories and constructs before moving on to presenting our results.
4. Results
4.1. Survey categories and constructs
Our survey had 89 responses from 15 language departments. The first series of questions concerned experience (age and date of arrival at the Commission), academic background and/or training and language department. We hypothesised that, taken together or separately, these elements might point us to the existence of subgroups. Language departments were especially likely to represent distinct groups:
With little official guidance of translation policy, and with each unit comprised of just one nationality (or, in the case of languages spoken in more than one member state, a few nationalities), the structure supports the development of separate translation cultures within each unit (Koskinen 2008: 70).
However, at least one question revealed surprising homogeneity, namely question 6 (see Appendix 2) in which we asked translators to choose from a series of 4 definitions of a translator and left one field blank for alternative definitions. The vast majority of translators (N=56) chose the definition given within the framework of the Interpretive Theory of Translation, and the rest overwhelmingly (N=31) opted for a definition that pictured the art of “go[ing] unnoticed”.
Table 1 relates questions to existing constructs and shows that we borrowed mostly from TAM3, and made very parsimonious additions as explained in the previous section.
Table 1. Survey questions based on existing constructs.
Questions we have not mentioned so far were added to the survey to provide more qualitative assessments: we could not discuss them all here owing to lack of space, but they can still be consulted in Appendix 2.
4.2. Participant profiles
Data collection was not easy and it appeared that translators were already asked to complete internal surveys regularly and had little time for it. Our quantitative data did not allow for the delineation of subgroups, and our sample was not balanced for gender, as it included 54 women and 34 men (one respondent did not want this information to be used). We do not see this as a problem: although we were hoping to have a more balanced sample, there are no pointers in the literature suggesting gender may have an impact on technology acceptance and use.
More importantly, our sample was well balanced with respect to age.
Figure 4. Number of respondents per age range.
Although we gathered answers from more than half of the 24 different language departments, we found strong disparities in the number of responding translators in each department.
We put together the answers to questions on actual use of MT (9 and 9’) to derive a 0-5 score (i.e. Never: 0, A few times a year: 1, Monthly: 2, Weekly: 3, Daily: 4 and Systematically 5). Average scores were then plotted by language departments, and we checked these results against existing measures of MT adoption (Foti 2015; Kluvanec 2017) to see how representative our sample was. According to the latest annual evaluation (Kluvanec, 2017: slide 20) the departments with the lowest MT adoption rates are EN, DE, FI, LV and NL. Those with the highest are ES, FR, IT, MT, PT and RO. This distinction seems to apply to our data, with the remarkable exception of French, which based on our data is in the low-use group of departments. It is also worth mentioning that the English language department gets the lowest of all average scores: among 16 surveyed translators, only one used MT almost systematically.
Figure 5. Number of translators who took the survey broken down by language department.
Figure 6. Average scores for actual use of MT by language department.
French and English, the only two language departments that depart from previously established adoption rates are among the three procedural languages at the Commission, i.e. the language departments that deal with the greatest diversity of text types and gather larger numbers of translators. Thus, we may hypothesise that in those departments, survey answers will vary according to diversity of respondents or subtle differences in questions used to assess actual use.
4.3. Linear regressions
TAMs were designed to predict IT use, and in order to achieve this, linear regressions have been the most used statistical models (Legris et al. 2003: 196). Overall, TAMs routinely account for 40% to 50% of variance in usage intentions, and only 30% of the variance in use (Venkatesh and Bala 2008: 291). As explained above, however, we did not assess usage intentions: this is because we expected, based on our ethnographic data and on the existing literature, that usage intentions would be more or less similar to actual use. In line with this expectation, we found strong similarity between actual use, as rated in answers to Question 10, and estimated sustainable use (which was assessed in Question 11): Cronbach’s alpha is very high (0.912).
We followed our model (as described above in Figure 3) and performed three multiple linear regressions. We first sought to account for perceived usefulness based on the following six factors: experience, subjective norm, image, job relevance, MT output quality and perceived ease of use. The regression showed that, taken together, those factors accounted for 60.5% (adjusted R-squared 0.605) of all observed variation. However, only three factors made a significant contribution, namely perceived ease of use (standardised beta coefficient: 0.654, p<0.001), subjective norm (standardised beta coefficient: -0.150, p=0.053) and image (standardised beta coefficient: 0.200, p=0.025). This means that experience, job relevance and MT output quality contributed virtually nothing to the observed variations in perceived usefulness.
In a second regression, we tried to predict perceived ease of use with our measures of computer anxiety and control. This was less successful: we predicted only 35.1% of variation in perceived ease of use. Only control, as measured by Question 15, was significant (standardised beta coefficient: 0.580, p<0.001). A very low Cronbach’s alpha (0.112) showed that our first measure of control, assessing translators’ perception of autonomy (Question 8, based on Marshman 2012), was unrelated to that second measure. Besides, our results suggest that we may have failed to measure computer anxiety properly: one reason for this was that we chose to ask a question with reference to the current translation environment of DGT translators rather than computers in general.
In a final regression, we sought to account for actual use based on the following predictors: experience, perceived usefulness and perceived ease of use, fear (rated on a scale of security) and perceived impact of MT. With an adjusted R-squared of 0.517, the regression significantly predicted slightly more than half of the observed variation. However, two of the factors that are an essential component of TAMs failed to make significant predictions, namely experience (standardised beta coefficient: -0.009, p=0.900) and perceived usefulness (standardised beta coefficient: 0.087, p=0.464). Perceived ease of use did appear to have a mildly significant impact (standardised beta coefficient: 0.257, p=0.034). Above all, the two factors that we had introduced turned out to have more weight than traditional TAM measures: fear made a very significant prediction (standardised beta coefficient: 0.377, p<0.001), and perceived impact was also significant (standardised beta coefficient: 0.257, p=0.003).
5. Discussion
Seeking to understand why fear of MT should relate to variations in actual use as it does, we looked at translators’ knowledge of MT (Figure 7) and found that there was a significant correlation between fear (i.e. a low degree of security) and knowledge of MT, as assessed by questions 22 and 25 respectively (Pearson correlation, r=-0.269, n=89, p=0.011). Thus, the translators who perceived MT as a threat were regularly those with the lowest scores for MT knowledge. Although there was good overall knowledge in our sample, with just over half of the translators getting the right answer, and another 27 having understood that EURAMIS was used, this was obviously not enough to “empower rather than marginalise translators” with MT (Kenny and Doherty 2014: 276).
Figure 7. Translators’ knowledge of MT.
Such findings suggest that it would be worth investigating how much knowledge is needed to alleviate fears. Besides, our respondents were not at all convinced that special skills were needed to use MT (45 thought so but 44 did not). Their hesitation could stem from the perfect integration of MT within the streamlined workflow that we described in the second part of this paper, and the resulting lack of direct interaction of translators with MT@EC. This is coherent with Kenny and Doherty’s (2014) holistic and empowering approach to teaching machine translation, as well as with a recent proposal that human-computer interactions (HCIs) could be enhanced by giving translators more control over their translation environments (Van den Bergh et al. 2015: 115). It would certainly take another thorough study to explore if and how this could be done at the DGT. Fostering translators’ participation in the study as actors rather than mere respondents (see e.g. Ospina et al. 2008) might be a way of achieving this aim, while ensuring better dissemination of the results among DGT translators. The existing DGT CATE Lab (Computer-Assisted Translation Environment Lab) is one such initiative, aimed at “involving DGT users in active technology watch” (Travnickova and Mai 2016), and it would probably welcome new collaborations with academics.
The second element that our study reveals is the need to distinguish translation tasks from the translator’s activity when analysing HCIs. The distinction is at the heart of both ergonomics and functional approaches to translation (Lavault-Olléon 2011), but TAMs are more focused on tasks, which may account for their failure to convey a clear picture of translators’ technology acceptance. We have shown that tasks were integrated in a smooth and automatised workflow that eased management in a number of ways, but this is not likely to have much impact on the translator’s activity. Our first linear regression shows that constructs such as MT output quality are virtually unrelated to translators’ perceived usefulness of MT, suggesting that relatively good MT output may still hinder the translator’s activity in at least some circumstances. On the other hand, cognitive load, as well as different perceptions and conceptions of the systems are an integral part of the complexity that characterises such activity: they are captured by the three constructs that have significantly contributed to variations in perceived usefulness, namely perceived ease of use, subjective norm and image.
Finally, our data contain evidence for rich and creative interactions with MT in post-editing, as can be seen in Figure 8. The diagram is based on our recoding of individual definitions of a usable MT output. It shows that although 37% of translators mostly agree to interact with MT to save time, the remaining translators mostly use MT outputs for inspiration, terminology, or in a restricted set of contexts. We may also note that another 25% of our sample provided negative answers, based on the contexts in which they discarded MT (because of a lack of appropriate structure, or when the output did not make any sense). Such interpretive flexibility is evidence that there is room for creativity and expression of individual preferences at the DGT. The categories that we built, based on translators’ open-ended responses to question 20, are necessarily restrictive: there is variation within those categories, but all answers are evidence that translators know very well when to use MT and why they do when they choose to. Thus, our categories should not be taken as evidence that guidelines could be produced to help translators or companies decide. On the contrary, they show that translators’ expertise can and should indeed be trusted when interacting with MT. Leaving it up to the translator to decide when it is appropriate to use MT and when not also opens up new perspectives for translator trainers. By asking students to choose and comment upon their choices, trainers would encourage them to think critically about MT in very concrete ways, and they could even lead them to question and better understand their own cognitive processes (Lavault-Olléon and Carré 2012).
Figure 8. What defines a usable MT output?
6. Conclusion
In the present study, we sought to understand current uses and perceptions of machine translation (MT) and post-editing (PE) at the DGT, and we relied both on ethnographic methods and on survey analysis to account for technology acceptance. Although technology acceptance is high at the DGT, our results suggest that perceptions of MT have a strong impact on both perceived usefulness and actual use, and that HCIs actually designate a wide range of experiences and feelings that existing models partly fail to capture.
It is our hope that we, as translator trainers, will be able to characterise translators’ roles in relation to MT in very diverse ways, thus opening up perspectives instead of shattering hopes.
Acknowledgements
The research reported here was supported by ILCEA4 and conducted as part of an interdisciplinary project funded by the Pôle Grenoble Cognition (for more details on the project: https://evaluerlata.hypotheses.org/). We are also indebted to Élisabeth Lavault-Olléon, who suggested spending time with institutional translators and observing current practices, and helped us get everything started. Our warmest thanks to the DGT, for welcoming us and giving us permission to conduct our study. We also wish to thank the DGT’s “Development of the translation profession” sector and to the EMT network for making this happen. None of the analyses presented here would have been possible without the active participation of DGT translators and heads of units, to whom we wish to express special thanks. The final version of the paper owes a lot to the anonymous reviewers and special issue editor, whose comments and suggestions were extremely accurate and helpful.
References
- Ansip, Andrus (2016). “How multilingual is Europe's digital single market?” Blog post, 27 May 2016. https://ec.europa.eu/commission/commissioners/2014-2019/ansip/blog/how-multilingual-europes-digital-single-market_en (consulted 24.01.2018).
- Cadwell, Patrick, Castilho, Sheila, O'Brien, Sharon and Linda Mitchell (2016). “Human factors in machine translation and post-editing among institutional translators.” Translation Spaces 5(2), 222 –243. DOI: 10.1075/ts.5.2.04cad.
- DG-Translation (2017). Management plan 2017. https://ec.europa.eu/info/publications/management-plan-2017-translation_en (consulted 25.12.2017) Ref. Ares (2017)1592860 - 24/03/2017.
- — (2016a). Annual activity report 2016. https://ec.europa.eu/info/sites/info/files/file_import/aar-dgt-2016_en_0.pdf (consulted 25.12.2017) Ref. Ares(2017)1826615 - 05/04/2017
- — (2016b). Strategic plan 2016-2020. https://ec.europa.eu/info/publications/strategic-plan-2016-2020-translation_en (consulted 27.12.2017) Ref. Ares(2016)1329034 - 16/03/2016
- European Commission (2016). Translation tools and workflow. ISBN 978-92-79-62204-5. DOI:10.2782/703257.
- Foti, Markus (2015). “The Commission’s machine translation service.” Paper presented at Translating Europe Regional Workshop on the Opportunities and Threats of Machine Translation for Professional Translators (Durham, 25 November 2015). http://www.ciol.org.uk/content/machine-translation-opportunities-and-threats (consulted 20.01.2018).
- Kenny, Dorothy (ed.) (2017). Human issues in translation technology: The IATIS yearbook. London: Routledge.
- Kenny, Dorothy and Stephen Doherty (2014). “Statistical machine translation in the translation curriculum: Overcoming obstacles and empowering translators.” The Interpreter and Translator Trainer 8(2), 276-294.
- Kluvanec, Daniel (2017). “From machine translation at the European Commission to European language resource coordination (ELRC).” Paper presented at EULITA: #TranslatingEurope Workshop (Vienna, 30 March 2017). http://eulita.eu/wp/772-2/ (consulted 20.01.2018).
- Koskinen, Kaisa (2008). Translating institutions: An ethnographic study of EU translation. Manchester: St. Jerome.
- Lavault-Olléon, Elisabeth (2011). « L’ergonomie, nouveau paradigme pour la traductologie » ILCEA Traduction et Ergonomie. https://ilcea.revues.org/1078 (consulted 10.12.2017).
- Lavault-Olléon, Elisabeth and Alice Carré (2012) « Traduction spécialisée: l’ergonomie cognitive au service de la formation », ASp, 62 | 2012, 67-77. https://journals.openedition.org/asp/3117 (consulted 05.12.2018).
- Legris, Paul, Ingham, John and Pierre Collerette (2003). “Why do people use information technology? A critical review of the technology acceptance model.” Information and Management 40(3), 191-204.
- LeBlanc, Matthieu (2017). “‘I can’t get no satisfaction!’ Should we blame translation technologies or shifting business practices?” Kenny (ed.) (2017), 45-62.
- Marshman, Elizabeth (2012). “In the driver’s seat: Perceptions of control as indicators of language professionals’ satisfaction with technologies in the workplace.” Proceedings of the Aslib Translating and the Computer 34 Conference. http://www.mt-archive.info/10/Aslib-2012-Marshman.pdf (consulted 18.07.18).
- Martín, Ricardo Muñoz (2010). “On paradigms and cognitive translatology.” Gregory M. Shreve and Erik Angelone (eds) (2010). Translation and Cognition. Amsterdam: John Benjamins, 169-187.
- O'Brien, Sharon (2012). “Translation as human–computer interaction.” Translation Spaces 1(1), 101-122.
- Olohan, Maeve (2017). “Technology, translation and society.” Target 29(2), 264-283
- Ospina, Sonia, Jennifer Dodge, Erica G. Foldy and Amparo Hofmann-Pinilla (2008). “Taking the action turn: Lessons from bringing participation to qualitative research.” Hilary Bradbury and Peter Reason (eds) (2008). The SAGE handbook of action research: Participative inquiry and practice. 2nd ed. London: Sage, 420-434.
- Pilos, Spyridon (2015). “How can public services benefit from the CEF.AT platform?” Delavnica projekta European Language Resources Coordination (ELRC), Ljubljana. http://videolectures.net/elrc2015_ljubljana/ (consulted 18.07.18).
- Risku, Hanna (2010). “A Cognitive scientific view on technical communication and translation: Do embodiment and situatedness really make a difference?” Target 22(1), 94–111. DOI: 10.1075/target.22.1.06ris.
- Rossi, Caroline (2017). “Introducing statistical machine translation in translator training: from uses and perceptions to course design, and back again.” Tradumàtica 15, 48-62.
- Svoboda, Tomáš, Biel, Łucja and Krzysztof Łoboda (2017). “Quality aspects in institutional translation: Introduction.” Tomáš Svoboda, Łucja Biel and Krzysztof Łoboda (eds) (2017). Quality aspects in institutional translation. Berlin: Language Science Press, 1–13. DOI:10.5281/zenodo.1048175.
- Travnickova, Hana and Katja Mai (2016). “DGT Cate Lab.” Paper presented at the Translating Europe Forum. https://ec.europa.eu/info/sites/info/files/tef2016_travnickova_mai_en.pdf (consulted 10.06.2018).
- Van den Bergh, Jan, Geurts, Eva, Degraen, Donald, Haesen, Mieke, Van der Lek-Ciudin, Iulianna and Karin Coninx (2015). “Recommendations for translation environments to improve translators' workflows.” João Esteves-Ferreira, Juliet Macan, Ruslan Mitkov and Olaf-Micahel Stefanov (eds) (2015). Proceedings of the 37th Conference Translating and the Computer, 106-119. https://uhdspace.uhasselt.be/dspace/bitstream/1942/19963/1/TC3715.pdf (consulted 05.12.2018).
- Venkatesh, Viswanath and Hillol Bala (2008). “Technology Acceptance Model 3 and a Research Agenda on Interventions.” Decision Sciences 39(2), 273–315.
Biographies
Caroline Rossi is a Lecturer involved in translator training at Université Grenoble Alpes, and a member of the ILCEA4 research team. She is a cognitive psycholinguist and her research interests range from first and second language acquisition to the epistemological, social and ergonomic consequences of current MT developments for both trainees and translators.
E-mail: caroline.rossi@univ-grenoble-alpes.fr
Jean-Pierre Chevrot is a Professor at Université Grenoble Alpes and the head of the LIDILEM research team. His research topics include language acquisition, sociolinguistics and language in the social media. His concern is to bridge the gap between cognitive, social and linguistic approaches to language.
E-mail: jean-pierre.chevrot@univ-grenoble-alpes.fr
Appendix 1. Open questions used in semi-directed interviews (all conducted in French)
1.Quelle est votre formation? Depuis combien de temps êtes-vous traducteur à la DGT?
What is your academic background and how long have you been working as a translator at the DGT?
2.Quelles sont vos principales sources de satisfaction et vos principales sources de frustration au travail ?
What are the main sources of satisfaction and frustration that you can currently identify at work?
3.Si vous deviez expliquer rapidement à quelqu’un d’extérieur à la DGT en quoi la TA vous aide et ce qu’elle vous apporte, de quoi choisiriez-vous de parler ?
If you were asked to explain to someone outside the DGT how MT helps you and what it brings you, what would you choose to discuss?
4.Vous souvenez-vous de vous être sentis prisonnier de la sortie de TA ou piégé, forcé à l’utiliser ? Racontez
Do you remember feeling trapped with an MT output or forced to use it? Tell me about it.
5.Est-ce que vous sauriez expliquer d’où viennent les sorties de TA et pourquoi elles ne sont pas toujours fiables ?
Would you be able to explain where MT outputs come from and why they are not always reliable?
6.Comment pensez-vous que vous utiliserez les MT et la TA dans dix ans ? Cela peut vous amener à parler des évolutions que vous avez connues
How do you think you will use TMs and MT in ten years? Feel free to discuss your own experience and the changes that you have witnessed.
Appendix 2. Survey questions (answers in brackets)
1. Which language department do you work in?
(Answer chosen from list of ISO codes for all 24 departments: please see figure 5 for figures)
2. What is your academic background and/or training?
- Translation Masters' Degree / School of Translation (43)
- Modern Languages and Law (2)
- Humanities and / or Languages (38)
- Scientific background (e.g. engineer) (4)
- Other (2)
3. You are:
- A woman (54)
- A man (34)
- You do not wish to answer (1)
4. How old are you? (Use 0 if you do not wish to answer)
(Mean age: 46.4, min. 27 max. 64)
5. When did you start working as a translator at the Commission? (YYYY only please -- year you arrived)
6. Which is the best definition of a translator, according to you?
- “A brilliant translator is one who goes unnoticed” (Chassigneux) (31)
- “The best-case scenario is for translation to defend and illustrate the target language and culture” (Gouadec) (0)
- “Translator traitor” (traduttore traditore) (0)
- The translator “understands what a person says and makes it understandable to another” (Lederer) (56)
- Other (2)
7. Among the changes that you have witnessed recently, would you say that Machine Translation (henceforth MT):
- Has had a major impact (39)
- Has not made a big difference (32)
- Other (please specify) (18)
8. Would you say that MT affects:
- The amount of work you do (Yes: 40 / No: 49)
- The kind of work you do (Yes: 17 / No: 72)
- The quality of the work you do (Yes: 35 / No: 54)
- How you do your work -- i.e. your working methods (Yes: 65 / No: 24)
- Your relationship(s) with your client(s) and/or employer(s)
(Yes: 3 / No: 86)
- None of the above (2)
9. When do you use MT?
- Systematically and regardless of text type (38)
- Only in some cases (41)
- Never (10)
9’ (continued). If you do not use MT systematically, when do you use it?
- Daily (1)
- Weekly (19)
- Monthly (11)
- A few times a year (10)
Please specify if appropriate
10. Which pattern of use best suits your expectations and conception of your job?
- Use MT in half of the cases (25)
- Use MT in almost all cases (34)
- Never use MT (10)
- Other (please specify) (20)
11. Which activities or tasks do you enjoy doing without using MT?
(Text box)
12. I am more efficient when I use MT
- I strongly disagree with this statement (6)
- I mostly disagree with this statement (7)
- I cannot say / I have no opinion (5)
- I mostly agree with this statement (42)
- I strongly agree with this statement (28)
- NA (1)
13. I work better and the quality of the translations I write improves when I use MT
- I strongly disagree with this statement (11)
- I mostly disagree with this statement (24)
- I cannot say / I have no opinion (15)
- I mostly agree with this statement (27)
- I strongly agree with this statement (11)
- NA (1)
14. It is easier for me to work with MT (both in terms of the tasks I handle and of cognitive load)
- I strongly disagree with this statement (7)
- I mostly disagree with this statement (8)
- I cannot say / I have no opinion (7)
- I mostly agree with this statement (38)
- I strongly agree with this statement (28)
- NA (1)
15. I have more control over my work when I use MT
- I strongly disagree with this statement (14)
- I mostly disagree with this statement (31)
- I cannot say / I have no opinion (11)
- I mostly agree with this statement (25)
- I strongly agree with this statement (7)
- NA (1)
16. My current translation environment (TRANSLATOR'S DESKTOP - CAT Client - Studio) does not require any major concentration effort
- I strongly disagree with this statement (32)
- I mostly disagree with this statement (35)
- I cannot say / I have no opinion (8)
- I mostly agree with this statement (9)
- I strongly agree with this statement (4)
- I don't work in this translation environment (1)
17. Influential people around me think I should use MT
- I strongly disagree with this statement (7)
- I mostly disagree with this statement (14)
- I cannot say / I have no opinion (41)
- I mostly agree with this statement (20)
- I strongly agree with this statement (7)
18. Translators with expertise and skills in MT and CAT tools have a higher profile
- I strongly disagree with this statement (12)
- I mostly disagree with this statement (17)
- I cannot say / I have no opinion (33)
- I mostly agree with this statement (19)
- I strongly agree with this statement (8)
19. Machine translation is particularly well suited for the kind of translations we handle at the DGT
I strongly disagree with this statement (7)
I mostly disagree with this statement (15)
I cannot say / I have no opinion (13)
I mostly agree with this statement (45)
I strongly agree with this statement (9)
20. The quality of MT is generally usable
- I strongly disagree with this statement (8)
- I mostly disagree with this statement (27)
- I cannot say / I have no opinion (10)
- I mostly agree with this statement (38)
- I strongly agree with this statement (6)
20. What defines a usable MT output according to you?
(Text box)
21. What does critical or sound use of MT mean to you?
(Text box)
22. Do you feel that MT is:
- Very threatening (2)
- Rather threatening (10)
- I cannot tell or have no opinion (13)
- Rather helpful (49)
- Very helpful (15)
23. Do you think special skills are needed to use MT?
- yes (45)
- no (44)
24. Have you changed the settings in Studio 2015 for it to work just like Studio 2014? I.e. automatically inserting the MT output when there is no TM match.
(Yes: 44 / No: 45)
25. You understand that MT works:
- Using TMs retrieved from EURAMIS (27)
- Using some complex computation (3)
- Both (49)
- You have no idea (10)
26. Judicial texts are often quoted as the most amenable to MT. Could you quote any elements that MT cannot help you with?
(Text box)
27. Same question for press releases and meeting reports
(Text box)
28. Would you like to answer further, more qualitative questions about your experience with MT?
- Yes (12)
- No (77)
(if so, please provide your name and contact details here)
Notes
Note 1:
MT@EC was replaced by eTranslation in November 2017, i.e. 10 months after the investigation presented in this paper, which is why we still refer to MT@EC. eTranslation has enabled a better integration into the Connecting Europe Facility (CEF), and an improved, quicker service. The engine itself is still based on MT@EC and mostly on the phrase-based statistical paradigm (neural machine translation has already been implemented for a few language pairs).
Return to this point in the text
Note 2:
Fear of MT was assessed in terms of the degree of security that translators felt in relation to MT (see question 22 in Appendix 2). The lowest score thus indicated insecurity (if they felt that MT was “very threatening”) and the highest corresponded to the secure feeling that MT was “very helpful”.
Return to this point in the text
Note 3:
“MT is the only viable solution for: quick and cheap access to information in foreign languages, understanding information received in a foreign language that otherwise could not be used or would require substantial time and costs to translate, making multilingual use of websites possible, facilitating cross-lingual information search and analytics.” (Pilos 2015)
Return to this point in the text