Home > Issue36 > Wang article

Building virtual communities of practice in post-editing training: A mixed-method quasi-experimental study

Lyu Wang, Hunan University/Changsha University of Science and Technology

Xiangling Wang, Hunan University

ABSTRACT

Since post-editing of machine translation has become a widespread practice in the language service industry, post-editing training has been increasingly included in translation courses. The purpose of the study is to investigate the feasibility of implementing a virtual community of practice (VCoP) in training future post-editors that meet the market’s needs. A VCoP refers to a social group sharing a common interest in an argument or a problem, interacting with each other by networks (Davis and Goodman 2014). Based on this concept, a VCoP method is postulated to facilitate online post-editing courses. It is a method in which students learn how to post-edit and how to manage a post-editing project by actively engaging in a VCoP. To test the feasibility of this method, a quasi-experimental study was carried out at Hunan University in China. Thirty first-year translation postgraduates participated in the study. Fifteen students were taught virtually using the VCoP method, while fifteen students were taught by the same tutor in a face-to-face classroom. A comparison of students’ performance in the two post-editing classes indicated that the VCoP method fostered students’ self-reflection and self-assessment, and ultimately helped improve post-editing quality. Students’ perceptions of the VCoP method were generally positive, showing its usability.

KEYWORDS

Virtual community of practice (VCoP), post-editing training, online learning, mixed-method, quasi-experimental study.

1. Introduction

The increasing need for well-trained post-editors in the translation market and ever-changing technologies have brought new opportunities and challenges to translator training. Translation trainers have started to incorporate post-editing in their curricula, as it becomes a requirement of translators in the digital age (e.g., O’Brien 2002; Pym 2013; Guerberof and Moorkens 2019). Meanwhile, the explosion of the Internet, cloud storage services, and virtual space, places the social aspect of the translation profession within the reach of students and teachers. Translation courses thus have to take into account and adapt to these technological changes, to bridge the gap between the academic and professional worlds of translating. However, although the benefits of using these technologies in the translation classroom have been widely recognised (Massey 2005; Desjardins 2019; Kiraly et al. 2019), how to integrate them into translator training has not been sufficiently explored (Gambier 2012; Díaz-Millón et al. 2020), particularly in the post-editing training contexts.

Building a virtual community of practice (VCoP) has been considered as one of the effective strategies that facilitates online learning (e.g., Wenger 2010; Smith et al. 2017). The concept of a VCoP finds its origins in the term Community of Practice (CoP), which is rooted in social learning theory that regards learning as a process of social interaction (Lave and Wenger 1991). It refers to a social group sharing a common interest in an argument or a problem, interacting with each other through networks (Kirschner and Lai 2007). A VCoP is characterised by the web-based nature of the social interaction, as members communicate and share knowledge through forums, social networking, chat rooms, e-mails and discussion boards (Behal 2019). Compared with face-to-face communities, a VCoP can eliminate both geographic barriers and time limitations, and allows a more heterogenous composition of joiners, thus facilitating communication in virtual learning environments (e.g., Yang 2009; Davis and Goodman 2014). However, despite increasing evidence of the benefits of a VCop in diverse educational settings, there seem to be few studies on what effect this method may have on translation e-learning more specifically.

In fact, trainers and practitioners have trialed e-learning approaches in the teaching of translation for a number of years (e.g., Massay 2005; Robinson et al. 2008; Talaván and Ávila-Cabrera 2021). Many authors suggest that translation e-learning can help train instrumental-professional competence, by offering contexts in which students work on authentic activities and solve problems collaboratively (Prieto-Velasco and Fuentes-Luque 2016; Nitzke et al. 2019a; Kiraly et al. 2019). Relatively few authors, however, have adopted e-learning as a teaching method to train future post-editors (Nitzke et al. 2019a). Problems associated with e-learning may also arise, such as declining motivation, minimal participation, heterogeneous learning needs, and high resource investment (Pym 2001). It is hypothesised that building VCoPs can overcome many of these challenges and facilitate a more effective online course for post-editing trainees. Hence, this study postulates a VCoP method, in which students learn how to post-edit and how to manage a post-editing project by actively engaging in a VCoP. To test the feasibility of the VCoP method, a quasi-experimental study with a control group was conducted at Hunan University in China. A class of first-year translation post-graduates studying post-editing participated in the study, 15 taking the virtual course using the VCoP method and 15 attending the face-to-face class. Both classes received identical course content and were taught by the same instructor. The current study was therefore designed to compare the performance of post-editing trainees during and following both online and face-to-face learning experiences. The specific research questions were as follows: do students using the VCoP method achieve the same level of self-assessment and self-reflection as students in the face-to-face class? Are there differences in student learning outcomes in terms of their post-edited outputs? How do students engage with the VCoP method?

2. Related research

2.1. Post-editing training

Post-editing is often known as a revision-related activity, involving the correction of machine translation (MT) output to ensure that it meets a level of quality negotiated in advance between the client and the post-editor (e.g., O’Brien 2002; TAUS 2016). It is a different task from human translation, requiring a set of specific competences such as risk assessment competence, strategic competence, consulting competence, and service competence (Nitzke et al. 2019b). Translators, therefore, need to be trained to maximise the potential benefits of MT.

O’Brien (2002) was the first scholar to propose the inclusion of post-editing (PE) training as part of translator training programmes. As O’Brien claims, the aim of such a course is to raise novice translators’ awareness about MT systems and their place in translation practice, and introduce them to the practice of PE. To achieve this goal, O’Brien presents an outline of PE courses, including a theoretical and a practical module on MT and PE knowledge. The author also notes the importance of a positive attitude towards MT, which is also stressed by other scholars (Doherty and Moorkens 2013; Pym 2013). Since then, translation researchers, practitioners and trainers have followed O’Brien’s proposal and explored how to train post-editors in educational institutions. Considerable research has been devoted to discussing syllabus designs in particular, covering the goals, objectives, content, process, resources and means of evaluation (e.g., Flanagan and Christensen 2014; Guerberof and Moorkens 2019).

In recent years, trainers have started to make the transition from teaching PE courses face-to-face to teaching these courses online. For example, Nitzke et al. (2019a) introduced a DigiLing programme to train digital competences of translation students, by presenting an example of a post-editing course through an e-learning platform called DigiLing. This e-learning platform allows for more diverse and flexible translation curricula, and offers additional training for students. Likewise, Díaz-Millón et al. (2020) support the use of e-learning in the training of future post-editors, claiming that it helps to foster the cross-curricular competencies demanded by new professional profiles. Although the cited research has addressed the merits of e-learning in post-editing training, no study has undertaken a comparison of student performance in a face-to-face and an online course, to validate these presumed e-learning benefits.

2.2. Virtual community of practice

The term Community of Practice (CoP) was first proposed by Lave and Wenger (1991), and defined as a group of people who engage in a process of collective learning in a shared domain of human endeavour. It can be traced back to social constructivism which stresses the importance of social interaction for cognitive development (Vygotsky 1978; Rogers 2000; Wenger 2010; Risku 2016). According to Lave and Wenger (1991: 91), three characteristics of a CoP are crucial: domain, community, and practice. First, a domain of knowledge creates common ground, inspires members to participate, and guides their learning. Second, a community creates the social bonds for their collective learning, supporting interactions, discussions, collaborative activities, and relationship building. Third, the practice is the specific focus around which the community develops, shares and maintains its core of knowledge (1991:91-117). These three characteristics constitute a community of practice, making it a good fit for professional training. The notion of CoP has been extended to refer to learning communities by later theorists and researchers, whereby a group of students with common academic goals and interests meet regularly to collaborate on classwork and engage in discussion (e.g., Davies et al. 2005; Li et al. 2009). Benefits of CoP for student learning communities have included developing new ways of thinking, creativity, and enhanced problem-solving skills, amongst other benefits (Lenning et al. 2013: 93).

Technological innovations, particularly computer-mediated communication (CMC) tools have opened up new possibilities to enhance and apply CoPs more successfully in virtual learning environments (Hildreth et al. 1998). A CoP combined with CMC tools is defined as a virtual community of practice (VCoP), where students share knowledge, communicate, discuss, and collaborate through networks (Davis and Goodman 2014; Risku and Dickinson 2017). In contrast to a traditional CoP, a VCoP offers more opportunities for distance learners to analyse, negotiate and collaborate, by establishing collaboration across geographical barriers and time zones (Winne et al. 2013). Research suggests that a VCoP can enhance personal and professional development, through online interactive, collaborative, and reflective activities (McConnell 2006; Lewis et al. 2011).

Recently, VCoPs have attracted the attention of translation practitioners, trainers and researchers thanks to the emergence of new web-based translation communities (e.g., Jiménez-Crespo 2015; Risku and Dickinson 2017). For example, O’Hagan (2011) points out the potential benefits of crowdsourcing communities in helping students get acquainted with working conditions in the real labour market. Along with O’Hagan, Sánchez Ramos (2018) recommends turning a translation classroom into an online community, by integrating Web 2.0 tools in students’ learning programs. Although prior studies provide a conceptual basis for applying a VCoP in translator training, they do not provide any detailed accounts of how a VCoP can be implemented in practice, or how to assess students’ performance in a VCoP.

3. Implementing VCoPs in post-editing training

Nowadays, universities in China are increasingly including PE training as part of Master of Translation and Interpreting (MTI) Programmes. In a typical face-to-face PE course, the teacher provides information to students applying a transmissionist approach in the classroom (e.g., Maor and Taylor 1995). This helps to familiarise students with conceptual knowledge on MT and PE. However, the scope of these courses is usually not sufficient to gather enough practical PE experience for students to become professional post-editors (Nitzke 2019a). In addition, teleworking and collaborative skills demanded by the market cannot be easily developed in the face-to-face class (Wang et al. 2017). Bearing this in mind, we propose a VCoP method to facilitate an online PE course, aiming at creating a collaborative learning environment for PE learners and engage them in an authentic post-editing project.

3.1. The VCoP method

Starting from the social constructivist view that learning is a constructive, self-directed, collaborative and contextual activity (Kiraly 2000: 23; Kiraly et al. 2019), we propose a VCoP method in post-editing training. The primary goal of this method is to train future post-editors in the digital age, by providing a learning environment that is more accessible, interactive, and engaging. The VCoP method is a method in which students learn by actively engaging in a VCoP supported by synchronous and asynchronous tools. This method typically is grounded in the following three elements: authentic problems, online collaboration, appropriate tutorial instruction and feedback. Primarily, problems are embedded in an authentic post-editing project commissioned by real clients, which triggers students’ learning motivation. In addition, active community participation is the key to building an empowered community. It allows community members to actively contribute to group discussion and take responsibility for community improvements. It is worth noting that effective online tutoring and feedback are crucial for active engagement and participation.

In implementing the VCoP method, a forum needs to be set up primarily for the course, where students can discuss issues relating to PE and MT, collect information, and share knowledge. Synchronous class meetings are also conducted regularly, using video conferencing software or online chatting rooms. Moreover, the client, target text readers, authors of the source text, and domain experts are continuously available for questions and feedback throughout the project. By engaging in these collective activities, students are more likely to become self-directed learners, interacting with their peers and working in a virtual PE team, rather than passively receiving knowledge from the tutor (Kiraly et al. 2019).

To further illustrate the features of the VCoP method, the following table makes a comparison between the VCoP method and conventional face-to-face teaching currently being applied in a number of universities in China.

Table 1. Features of the VCoP method versus the conventional method

As presented in Table 1, the VCoP method has some distinguishing features, in contrast to the conventional method in use in many Chinese institutions. In implementing the VCoP method, the tutor shifts from a transmitter of knowledge to a facilitator, a coach or a co-learner, providing scaffolding for students (e.g., Kiraly 2000). Also, students adopt significantly different roles from those in face-to-face class. They are placed at the centre of their own learning, working collaboratively with their peers, the tutor, and domain experts from outside the institution. Compared with face-to-face teaching, the VCoP method provides a more interactive environment for learners, with the aid of synchronous and asynchronous tools. The tools enable students to access resources more easily, as well as feedback from target text readers, authors of source texts, and clients.

3.2. Implementing the VCoP method

We developed a 16-week PE training program for MTI students in Hunan University, using the VCoP method. The objectives of this course were: (1) to equip students with PE skills, (2) to enable students to acquire knowledge on PE project management, and (3) to develop students’ teamwork skills for large PE projects. The course consisted of the following four modules: introduction of conceptual knowledge on MT and PE, practice exercises on PE, implementation of a real-life PE project, and reflection on the learning process and the product. Each module involved student participation in weekly online activities and discussions.

A summary of the online course modules is shown in Table 2, which illustrates VCoP-related issues, the mediating web-based tools, and intervention teaching strategies.

Modules	Issues	Web-based tools	Intervention teaching strategies
Conceptual knowledge	MT PE	Tencent Meeting1 Forum E-mail	Send a course introductory pack one week before the course starts; Offer shareable e-learning content; Provide weekly updates to remind students about the week’s topics; Use different scenarios to generate discussion on relevant issues; Post web links to technical assistance
Practice exercises	Productivity Quality Evaluation	Tencent Meeting Forum E-mail	Appoint community leaders; Develop sub-communities; Grab volunteers to make presentations; Prompt peer feedback in group discussions; Foster purposeful discussions
Project management	Terminology PE project management Quality assurance assurance	Tencent Meeting QQ2 WeChat3 Baidu Cloud4 Forum E-mail	Set clear quality expectations and deadlines; Facilitate a real-life project; Post questions that trigger a meaningful online discussion; Prompt feedback to student queries; Provide timely and useful feedback; Foster detailed discussion of feedback from the client, readers, and domain experts
Reflection	PE project PE quality	Tencent Meeting Forum E-mail	Structured class discussions Use self-assessment reports Use peer review scales Use online portfolios

Table 2. Online PE course modules in the VCoPs method

The course started with a module on conceptual knowledge, which aimed at developing a basic knowledge of MT and PE. The issues covered basic principles of MT, MT errors, types of PE, PE guidelines, and PE quality metrics. In this module, students attended live classes with Tencentmeeting, which is one of the most widely used video conferencing applications. This software can also record the course, so that students are able to watch it back as many times as necessary. They also took part in asynchronous learning through the use of e-mails and forums. To facilitate their learning, the teacher offered shareable e-Learning content and used different scenarios to generate discussion on relevant issues.

In the next module, students were asked to do some PE practice exercises, including trying different MT tools, comparing PE with human translation, evaluating quality based on TAUS’s quality metrics and self/peer reviewing. These exercises were designed to help improve problem-solving and critical thinking skills in post-editing. The teacher appointed community leaders, who maintained order and motivated other members of the group. Sub-communities were also developed, to provide more flexible and interactive collaborations. Students were asked to share their learnings with their peers in the virtual class while the teacher shared additional tips on their learning and prompted peer feedback in group discussions.

Over the following ten weeks, students were asked to complete a large real-life project using networks. This module aimed at teaching students how to solve specific problems with specific tools when managing a PE project. Students were randomly allocated to three QQ groups composed of five to six members. They met virtually twice a week in QQ chat rooms, to negotiate and discuss problems arising during the workflow, ranging from terminology management to quality assurance. Their discussions were facilitated by the teacher, who helped to keep discussions focused, provided feedback, posed conflicting views to elicit reflection, and drew conclusions. Students could receive help, support and feedback from the client, readers, and domain experts, by using the WeChat public platform.

The final module offered students an opportunity to reflect on what they learned from the course, particularly in terms of project management and PE quality control. In structured reflection sessions, students shared their learning with peers, by making a class presentation through a video or slide show. They were also asked to write reflective reports and submit online learning portfolios (i.e., group discussion).

3.3. Assessment of the VCoP method

Assessment plays an important role in student learning and is perhaps a driving force underlying effective learning in virtual learning environments (Vonderwell et al. 2007). In this study, we combined continuous assessment with end-of-term assessment, in order to promote effective online learning. Table 3 illustrates the objectives, schedules of the two types of assessments, and typical assessment activities that were used.

Table 3. Assessment in the VCoP method

Continuous assessment was embedded in students’ learning throughout the course. The major purpose of the assessment was to enhance learning engagement and track students’ progress, providing ongoing feedback to improve teaching and learning (Galán-Mañas and Hurtado Albir 2015). A further purpose was to encourage students’ self-assessment skills and reflective thinking, by engaging them in ongoing online assessments. In the course, students were expected to submit their post-edited texts as well as their e-portfolios every three or four weeks of the term. They were asked to evaluate their PE outputs in each phase of the digital project, which were further compared with the teacher’s assessment. They also participated in evaluating their own work and engaged in peer review in a work group, thus developing dynamic peer-assessment skills. This process required the teacher to provide feedback and assistance in order to improve group dynamics.

After completing the continuous assessments by the end of Week 15, end-of-term assessment was carried out in the last week of term. Students were given a post-editing assignment in Translog II, a keylogging software for recording the translation process. In addition, they submitted reflective reports about their learning in VCoPs, based on interactive processes among the members of the group. It was equally important to measure whether students developed a positive attitude towards the teaching method and tasks, in line with the previously-mentioned need for a positive attitude towards MT.

4. A mixed-method quasi-experimental study

This study adopted a quasi-experimental design with a control group to test the feasibility of the VCoP method in the context of post-editing training. This research design, which aims at evaluating an intervention with a non-random group, is widely used when randomisation is impractical or unethical (Cook and Campbell 1979). It can minimise threats to ecological validity, since the research findings are tested in more practical or real-life situations (Gopalan et al. 2020). The methodology of this study is presented in the following sections.

4.1. Participants

The participants comprised 30 translation students in the second semester of an MTI programme, working from English into Chinese, their native language. All of them had passed TEM-8, a standardised English proficiency test for English majors in Chinese universities, ensuring a certain level of proficiency in English. Prior to the course, they had acquired basic translation skills, by completing several courses in scientific and technical translation. However, they had never engaged in any real collaborative translation or post-editing project. They were divided into two groups, an experimental group and a control group. Both groups had the same instructor, attended two PE classes per week, and were taught exactly the same content. The only difference was that the control group met with the instructor in the face-to-face class, while the experimental group was taught online via online learning platforms. All participants were informed of the purpose of this study and signed a consent form at the beginning of the course. Ethical approval was also obtained from the Ethics Committee of the College of Foreign Languages at Hunan University.

4.2. Translation assignment and the client

The students were required to translate some English documents on the topic of deep space exploration into Chinese over a period of ten weeks. The source text was approximately 100,000 words in length, covering knowledge in the domains of defence, commerce, labour, energy, etc. The entire text was pre-translated by Google NMT free of charge. The target text was expected to achieve publishable quality (TAUS 2016) and would be used as reference for researchers and policy makers on space exploration. The students’ client was the Deep Space Exploration Research Centre of Hunan University, which would publish the documents, though without the translators’ names. The research centre was represented by a contact person, who provided a detailed brief and came to our institute to present the project to the students. Students would not get paid, but they would receive a signed statement from the research centre confirming their participation in the project.

4.3. Methods

Prior to the course, one raw MT output from the deep space materials was given to students as a pre-test, along with a translation brief and full post-editing guidelines (TAUS 2016). No significant difference was found in the number of errors remaining in both groups’ PE outputs, suggesting an initial equivalence in post-editing skills before the experiment. The pre-test helped to rule out many of the threats to internal validity, such as history and maturation (Cook and Campbell 1979). Another raw MT output containing a very similar number of MT errors was given to both groups as a post-test.

During the course, students used a rubric (see Appendix 1) adapted from TAUS’ Dynamic Quality Evaluation Framework (2013) to assess their own PE assignments at Week 4, 8, 12 and 15. Their scores were compared with the scores assigned by the teacher. Students were also asked to write reflective reports to see how they learned from the course.

After the course, all 15 students from the virtual class were interviewed in Tencent QQ, which recorded all the statements automatically. The other 15 students were interviewed in the classroom or in the teacher’s office with voice recording apps. The structured interviews (Appendix 2) were designed to explore student satisfaction and perceived effectiveness of the course. Finally, all the students in the online course were asked to respond to a questionnaire measuring their perceptions of the VCoP method, which was adapted from McConnell’s (2006) questionnaire on students’ experiences of learning in e-groups (Appendix 3).

4.4. Data coding

Errors were categorised into five types based on error types suggested by TAUS (2016) and Hsu (2014). As shown in Table 4, incorrect punctuation was counted as a punctuation error. Errors on word choice, terminology, collocation, and fixed expressions were defined as lexical errors. Syntactic errors referred to errors at the syntactic level. If the content that should have been translated had been left untranslated, it would be marked as an error. The errors beyond the sentence level were macro-level errors, related to coherence, target reader, and purpose.

Table 4. Coding errors remaining in PE outputs

A rubric adapted from Kember et al. (2008) was used to assess the level of reflection in students’ reflective reports. This rubric measured the levels of reflection on a 3-point scale. ‘0’ was the lowest mark, suggesting that no reflection occurred, while the highest mark ‘3’ would be assigned if a written report highlighted a high level of critical reflection.

5. Results and discussion

This section presents and discusses results of the study from three aspects: (1) to analyse student performance either in VCoPs or face-to-face from a process perspective, (2) to compare their learning outcomes from a product perspective, and (3) to measure their attitudes towards the VCoP method.

5.1. Process-oriented assessment

A process-oriented assessment is conducted to monitor students’ learning process. This assessment takes the form of learning portfolios, including self/tutor assessments and self-reflective reports.

5.1.1. Self/tutor assessment of post-edited texts

Both the tutor and students used an identical five-point Likert scale at Week 4, 8, 12, 15 to assess performance on PE assignments for both experimental and control groups. The results of the self-assessment (SA) and tutor-assessment (TA) mean scores are presented in Figure 1. As shown in Figure 1, in both groups, SA scores tended to be higher than those given by the tutor at Week 4. This result revealed that at the beginning of the course, both groups tended to over-estimate their performance. As the project progressed, SA scores assigned by the experimental group tended to be more consistent with TA scores, while SA grades assigned by the control group were consistently higher than TA scores. These results indicate that the accuracy of students’ self-assessment improved over time in the online class using the VCoP method. Such a finding echoes previous research conclusions which highlighted that an e-learning environment can promote a greater sense of responsibility and an increased awareness of positive and negative elements in translations (Robinson et al. 2008; Galán-Mañas and Hurtado Albir 2010).

The improvements seemed to be closely associated with online collaboration activities in VCoPs, such as online interactive discussions and collaborative reviewing, according to the interviews. As many students reported, online interactive discussion helped develop a better understanding of criteria and standards for PE tasks. In addition, the VCoP method provided students with a dynamic, interactive learning environment for online collaborative reviewing, which is known to reduce self-assessment bias and promote self-reflection (Tai et al. 2018). As students claimed, online peer feedback motivated them to reflect on their own translations. More importantly, the access to online feedback from target text readers, authors of source text, and clients helped refine the judgement of students about their own work and identify translation errors. The results are in line with Massey and Brändli’s (2016) finding that collaborations and peer reviewing in a digital space lead to an increased level of reflection on students’ own performance, and a higher standard of outcomes and responsibility for their own learning.

5.1.2. Self-reflections of learning process

Both the experimental and the control groups were asked to write reflective reports describing their development in interpreting the PE guidelines, solving problems, gathering information, sharing knowledge, assuring quality, and collaboration. As shown in Figure 2, overall, the mean level of reflection of the experimental group was 2.38, which was much higher than that of the control group (1.66). This result highlights that greater levels of reflection occurred with the VCoP method, compared with the face-to-face method. This phenomenon was also observed by Yang (2009), who discovered high levels of critical reflection in a blog-based community of practice. The scholar concluded that, due to the benefits of the virtual platform, online blogs should be used more widely as a medium to provide and promote critical reflection amongst learners.

A detailed examination of the findings showed that students in the online class achieved a much higher level of reflection in terms of knowledge sharing than their peers in the face-to-face class. This might be due to the fact that VCoPs can facilitate knowledge sharing among students and support various forms of knowledge exchange (Risku and Dickinson 2017). Further, the experimental group displayed a higher level of reflection regarding the development of their collaborative and problem-solving skills. A possible interpretation of these results is that online learning platforms provide students with flexibility to collaborate with each other on their own terms and solve real-life problems jointly (Prieto-Velasco and Fuentes-Luque 2016). In brief, the obtained results suggested that the VCoP method could foster students’ self-reflection. This finding is in line with the wider literature in the domain of distance learning that has long mooted the benefits of online learning for promoting reflective and critical work (e.g., Winne et al. 2013; Tian et al. 2019).

5.2. Product-oriented assessment

The product-oriented assessment that was carried out in the present study sought to assess students’ performance via their finalised products. In this case, errors in students’ post-edited artefacts were calculated based on the error typology outlined in Section 4.4.

5.2.1. Post-editing artefacts

To examine how well students completed the post-editing project, the number of errors remaining in their post-edited artefacts were calculated. Overall, the total number of errors remaining in the experimental group’s post-edited artefacts was fewer. A detailed look at distributions of error types showed that more syntactic and macro-level errors in MT outputs were corrected by the experimental group. As presented in Figure 3, the number of syntactic errors remaining in the experimental group’s post-edited artefacts was 99, which was 20.2% lower than that of the control group. 10.8% fewer macro-level errors were also found in the experimental group’s post-edited artefacts.

These results suggest that the experimental group performed more successfully in the post-editing project. One possible reason underlying the different outcomes was that students in the virtual class received far more feedback from the teacher and their peers than students in the face-to-face class (Smith et al. 2017). Furthermore, students using the VCoP method received additional comments in text-based chats and on the forum from online target readers. The comments maintained much of the driving force underlying their reflection, and ultimately improved their confidence in self and peer-assessment of translation quality, as a number of students mentioned in the interviews. Another possible explanation was that students using the VCoP method were found to be much more engaged in group discussion and collaborative reviewing, in contrast to students in the face-to-face class. These collaborative activities fostered a greater sense of community and increased community engagement, thus increasing the support for a more successful collective product (Wenger et al. 2011).

5.2.2. Post-test achievement

Students’ performance in the post-test was compared in order to measure their achievements after the course. Overall, as shown in Table 5, the experimental group produced significantly fewer errors in their post-edited outputs than the control group (p=.034, p<.05). A further look at error types found that there were significant differences in the average number of syntactic (p=.046, p<.05) and macro-level errors (p=.041, p<.05) remaining between the two groups’ post-edited outputs. Further, more lexical and untranslated errors going unnoticed by the control group were corrected by the experimental group, although the differences were not statistically significant. These results suggested that, by the end of the semester, students using the VCoP method had greater improvements in post-editing skills.

A possible interpretation of this phenomenon is that social learning in VCoPs might trigger students’ metacognition in terms of identifying and correcting errors, helping them make more accurate decisions in post-editing processes. Students using the VCoPs method received more bilateral written feedback via the forum or in chat rooms. This type of feedback was considered by students to be more effective in triggering reflection on decision-making than multilateral verbal feedback in the face-to-face class (Massey and Brändli 2016; Díaz-Millón et al. 2020). In addition, students’ improvements in post-editing skills seemed to be associated with the interactive nature of e-learning, which renders peer feedback loops more timely and effective. Students in the online course tended to be more active in providing peer feedback, rather than rely on the teacher’s feedback as was the case in the traditional classroom. The continuous peer reviewing that took place allowed students to look at their own post-editing process and translation choices, and compare them to those of their peers. By doing this, students were more likely to engage in reflective thinking about translating and making revisions for better quality (Heine 2019; Kiraly et al. 2019).

5.3. Students’ perceptions of the VCoP method

It is equally important to understand students’ perceptions of the VCoP method to see if the method proposed needs further refinements (Galán-Mañas and Hurtado Albir 2010). In order to simplify interpretation, response categories ‘strongly agree’ and ‘agree’ were combined into the ‘Agree’ category, while response categories ‘strongly disagree’ and ‘disagree’ were combined into the ‘Disagree’ category (See Table 6). The table therefore more clearly demonstrates whether students’ attitudes towards the course were positive or negative, which helps to have a general view of strengths and limitations of the VCoP method.

Table 6. Student questionnaire and responses

Students’ perceptions of the VCoP method were generally positive regarding the learning process. As presented in Table 6, 12 out of 15 students agreed that group discussions had given them the information they needed. Such a result revealed the potential benefits of online discussion in VCoPs. Yet, two students disagreed that online discussions could be more productive than face-to-face discussions. They felt that the online medium did not necessarily allow them to contribute more. However, for the vast majority (11 out of 15), the medium seemed to allow them to contribute more compared with face-to-face contexts. This is a significant result that reveals the potential of the online medium for supporting learning in the translation classroom based on discussion and communication. In addition, nine students felt that learning in VCoPs turned out to be better than they had expected, further suggesting the effectiveness of the method. A majority (10 out of 15) said they had not found it difficult to learn effectively in VCoPs, though four students thought it was trickier to learn this way and thought that they had not contributed as much as they would have liked to online discussions. To some extent, participation in online discussions could be related to individual differences (e.g., self-confidence, motivation and learning style) as well as translators’ levels of emotional engagement (Hubscher-Davidson 2009; 2017), all of which can affect how translation students perform in both online and face-to-face classrooms. The psychological factors linked to engagement in VCoPs in the context of translation are worthy of further investigation for improving learning effectiveness.

The results of the study also revealed that students were generally satisfied with the teaching in the VCoP method. Almost all students (14 out of 15) agreed that the VCoP method was a good way of running e-courses for post-editing. Nine students went as far as to say that it was one of the most interesting courses that they had taken. Moreover, 10 out of 15 students said that the way in which the tutors facilitated the project was effective. These results were encouraging, showing the feasibility of the VCoP method in the context of post-editing training. Interestingly, 9 out of 15 students believed that the tutors were much more influential than anyone else, though five students thought that peers had more impact on their learning. The results shed light on the complex role of the tutor and peer learning in VCoPs, something which is also extensively discussed in prior research (e.g., Fulton 2020). Notably, students were divided on the issue of whether the tutor should adopt a more directive approach in participation. Seven students expected more directive tutoring, whereas seven other students preferred more facilitative tutoring. Although context-bound, such a finding suggests that there is a critical requirement for providing effective facilitation and interventions tailored to meet individuals’ needs (e.g., Massey et al. 2019; Brown et al. 2020). Altogether, students’ perceptions of the VCoP method were generally positive, however, further research is required regarding the role of tutor facilitation.

6. Conclusion

The study set out to train post-editors to meet market needs in the digital age. A VCoP method has been proposed, in which students learn by actively engaging in a VCoP supported by synchronous and asynchronous tools. To examine the feasibility of the VCoP method in the context of PE training, a quasi-experimental study with a control group was conducted in an educational institution in China. The results showed that students using the VCoP method were more engaged in self-assessment and reflective thinking about post-editing than their peers in the face-to-face classroom. More active engagement in collective learning also helped them achieve higher post-editing quality. Moreover, students’ perceptions of the method were generally positive, demonstrating the feasibility of applying the VCoP method in this specific context.

Although exploratory in nature, this study paves the way for further research on the use of VCoPs in translation teaching. The present study was conducted in a single institution in China with small samples. Therefore, it would be desirable to extend the research to other regions in order to improve the validity of the conclusions with a larger sample of participants. It would also be interesting to make qualitative comparative analyses of students’ performance in online and face-to-face discussions, especially in terms of group dynamics, collaborative patterns, and responses to feedback. These analyses may help to obtain a better understanding of the findings and shed light on the specific benefits of the VCoP method in teaching post-editing. Consideration should also be given to individual differences (e.g., self-efficacy, motivation), in order to better understand and enhance student engagement, and ease the tension between student dependence and autonomy in virtual learning environments.

Acknowledgements

Thanks go to every participant in the study. Particular gratitude is extended to Dr Anabel Galán-Mañas who offered us constructive and thought-provoking suggestions about the research design and methodology. We are also very grateful to Dr Séverine Hubscher-Davidson, Dr Jérôme Devaux, and the anonymous reviewers who provided thorough, constructive and thought-provoking comments.

This research was supported by the National Social Science Foundation of China (19BYY104).

References

• Behal, Anil (2019). “Leading and managing virtual communities of practice (VCoPs).” Contemporary Management Research 13(2), 15-37.
• Brown, Alice et al. (2020). “A conceptual framework to enhance student online learning and engagement in higher education.” Higher Education Research and Development. Advanced Online Publication. Advanced Online Publication. https://doi.org/10.1080/07294360.2020.1860912 .
• Cook, Thomas and Donald Campbell (1979). Quasi-Experimentation: Design & Analysis Issues in Field Settings. Boston, MA: Houghton Mifflin.
• Davies, Alison, Ramsay, Jill, Lindfield, Helen and John Couperhwaite (2005). “Building learning communities: Foundations for good practice.” British Journal of Educational Technology 36(4), 615–628.
• Davis, Carrie and Harriet Goodman (2014). “Virtual Communities of Practice in Social Group Work Education.” Social Work with Groups 37(1), 85-95.
• Desjardins Renée (2019). “A preliminary theoretical investigation into [online] social self-translation: The real, the illusory, and the hyperreal.” Translation Studies 12(2), 156-176.
• Díaz-Millón, Mar et al. (2020). “Disruptive methodologies and cross-curricular competencies for a training adapted to new professional profiles: the undergraduate program in translation and interpreting.” Sophia Palahicky (ed.) (2020). Enhancing Learning Design for Innovative Teaching in Higher Education. Pennsylvania: IGI Global, 83-104.
• Doherty, Stephen and Joss Moorkens (2013). “Investigating the experience of translation technology labs: pedagogical implications.” The Journal of Specialised Translation 19, 122-136.
• Flanagan, Marian and Tina Paulsen Christensen (2014). “Testing post-editing guidelines: How translation trainees interpret them and how to tailor them for translator training purposes.” The Interpreter and Translator Trainer 8(2), 257-275.
• Fulton, Crystal (2020). “Collaborating in online teaching: inviting e-guests to facilitate learning in the digital environment.” Information and Learning Sciences 121, 579-585.
• Galán-Mañas, Anabel and Amparo Hurtado Albir (2010). “Blended learning in translator training methodology and results of an empirical validation.” The Interpreter and Translator Trainer 4(2), 197-231.
• Galán-Mañas, Anabel and Amparo Hurtado Albir (2015). “Competence assessment procedures in translator training.” The Interpreter and Translator Trainer 9(1), 63-82.
• Gambier, Yves (2012). “Teaching Translation/Training Translators.” Yves Gambier and Luc van Doorslaer (eds) (2012). Handbook of Translation Studies. Volume 3. Amsterdam/Philadelphia: John Benjamins, 163–171.
• Gopalan, Maithreyi, Rosinger, Kelly and Jee Bin Ahn (2020). “Use of quasi-experimental research designs in education research: growth, promise, and challenges.” Review of Research in Education 44(1), 218-243.
• Guerberof, Ana and Joss Moorkens (2019). “Machine translation and post-editing training as part of a master’s programme.” The Journal of Specialised Translation 31, 217-238.
• Heine, Carmen (2019). “Student peer Feedback in a translation task: experiences with questionnaires and focus group interviews.” Elsa Huertas-Barros, Sonia Vandepitte and Emilia Iglesias-Fernández (eds) (2019). Quality Assurance and Assessment Practices in Translation and Interpreting. Pennsylvania: IGI Global, 337-357.
• Hildreth, Paul Michael, Kimble, Chris and Peter Wright (1998). “Computer Mediated Communications and Communities of Practice.” Paper presented at Proceedings of Ethicomp'98 (Erasmus University, The Netherlands, 25-27 March 1998).
• Hsu, Jiuan-An (2014). “Error classification of machine translation. A corpus-based study on Chinese-English patent translation.” Studies of Translation and Interpretation 18, 121-136.
• Hubscher-Davidson, Séverine (2009). “Personal diversity and diverse personalities in translation: a study of individual differences.” Perspectives 17(3), 175-192.
• Hubscher-Davidson, Séverine (2017). Translation and Emotion: A Psychological Perspective. London/New York: Routledge.
• Jiménez-Crespo, Miguel, Ángel (2015). “The Internet in translation education: two decades later.” Translation and Interpreting Studies 10(1), 33-57.
• Kember, David et al. (2008). “A four-category scheme for coding and assessing the level of reflection in written work.” Assessment and Evaluation in Higher Education 33(4), 369-379.
• Kiraly, Don (2000). A Social Constructivist Approach to Translation Education. Manchester: St Jerome.
• Kiraly, Don et al.(2019). “Enhancing translation course design and didactic interventions with e-learning: Moodle and beyond.” Don Kiraly and Gary Massey (eds) (2019). Towards Authentic Experiential Learning in Translator Education (2nd Edition). Newcastle: Cambridge Scholars Publishing, 101-127.
• Kirschner, Paul and Kwok-Wing Lai (2007). “Online communities of practice in education.” Technology Pedagogy and Education 16(2), 127-131.
• Lave, Jean and Etienne Wenger (1991). Situated learning: Legitimate Peripheral Participation. Cambridge: Cambridge University Press.
• Lenning, Oscar Thomas et al. (2013). Powerful Learning Communities: A Guide to Developing Student, Faculty, and Professional Learning Communities to Improve Student Success and Organizational Effectiveness. Sterling, VA: Stylus Publishing, LLC.
• Lewis, Laura et al. (2011). “Virtual communities of practice: bridging research and practice using Web 2.0.” Journal of Educational Technology Systems 39(2), 155-161.
• Li, Linda et al. (2009). “Evolution of Wenger's concept of community of practice.” Implementation Science 4(11). https://link.springer.com/article/10.1186/1748-5908-4-11 (consulted 10.03.2020).
• Maor, Dorit and Peter Charles Taylor (1995). “Teacher epistemology and scientific inquiry in computerized classroom environments.” Journal of Research in Science Teaching 32(8), 839-854.
• Massey, Gary (2005). “Process-oriented translator training and the challenge for e-learning.” Meta 50(2), 626-633.
• Massey, Gary, Kiraly, Donald and Maureen Ehrensberger-Dow (2019). “Training the translator trainers: an introduction.” The Interpreter and Translator Trainer 13(3), 211-215.
• Massey, Gary and Barbara Brändli (2016). “Collaborative feedback flows and how we can learn from them: investigating a synergetic learning experience in translator education.” Don Kiraly (ed.) (2016). Towards Authentic Experiential Learning in Translator Education. Göttingen: Mainz University Press, 177-199.
• McConnell, David (2006). E-Learning Groups and Communities of Practice. Berkshire: Open University Press.
• Nitzke, Jean, Tardel, Anke and Silvia Hansen-Schirra (2019a). “Training the modern translator–the acquisition of digital competencies through blended learning.” The Interpreter and Translator Trainer 13(3), 292-306.
• Nitzke, Jean, Hansen-Schirra, Silvia and Carmen Canfora (2019b). “Risk management and post-editing competence”. The Journal of Specialised Translation 31, 239-259.
• O’Brien, Sharon (2002). “Teaching post-editing: a proposal for course content.” Paper presented at Proceedings of 6th EAMT Workshop Teaching Machine Translation (Centre for Computational Linguistics, Manchester, England, 14-15 November 2002).
• O’Hagan, Minako (2011). “Community translation: translation as a social activity and its possible consequences in the advent of Web 2.0 and beyond.” Linguistica Antverpiensia, New Series–Themes in Translation Studies 10, 11-23.
• Prieto-Velasco, Antonio Juan and Adrián Fuentes-Luque (2016). “A collaborative multimodal working environment for the development of instrumental and professional competences of student translators: an innovative teaching experience.” The Interpreter and Translator Trainer 10(1), 76-91.
• Pym, Anthony (2001). “E-Learning and Translator Training.” https://usuaris.tinet.cat/apym/on-line/training/2001_elearning.pdf (consulted 10. 03. 2020).
• Pym, Anthony (2013). “Translation skill-sets in a machine-translation age.” Meta 58 (3), 487-503.
• Risku, Hanna (2016). “Situated learning in translation research training: academic research as a reflection of practice.” The Interpreter and Translator Trainer 10(1), 12–28.
• Risku, Hanna and Angela Dickinson (2017). “Translators as networkers: the role of virtual communities.” Hermes 22(42), 49–70.
• Robinson, Bryan, López-Rodríguez, Clara I. and Mª Isabel Tercedor Sánchez(2008). “Neither born nor made, but socially-constructed: promoting interactive learning in an online environment.” Traduction Terminologie Rédaction 21(2), 95–129.
• Rogers, Jim (2000). “Communities of practice: A framework for fostering coherence in virtual learning communities.” Educational Technology and Society 3(3), 384-392.
• Sánchez Ramos, Maria del Mar (2018). “Translation learning environments for the future: online collaborative practice.” Birthe Mousten, Sonia Vandepitte, Elisabet Arnó and Bruce Maylath (eds) (2018). Multilingual Writing and Pedagogy in Virtual Learning. Pennsylvania: IGI global, 268-290.
• Smith, Sedef Uzuner, Hayes, Suzanne and Peter Shea (2017). “A critical review of the use of Wenger’s community of practice (CoP) theoretical framework in online and blended learning research.” Online Learning 21(1), 209-237.
• Tai, Joanna et al. (2018). “Developing evaluative judgement: enabling students to make decisions about the quality of work.” Higher Education 76, 467–481.
• Talaván, Noa and José Javier Ávila-Cabrera (2021). “Creating collaborative subtitling communities to increase access to audio-visual materials in academia.” The Interpreter and Translator Trainer. https://doi.org/10.1080/1750399X.2021.1880305 (consulted 10.03.2020).
• TAUS (2013). Adequacy/Fluency Guidelines. https://taus.net/academy/best-practices/evaluate-best-practices/adequacy-fluency-guidelines (consulted 10. 03. 2020).
• TAUS (2016). TAUS Post-Editing Guidelines. https://www.taus.net/think-tank/articles/postedit-articles/taus-post-editing-guidelines
  (consulted 10. 03. 2020).
• Tian, Luo, Smruti, Shah and Helen Cromptom (2019). “Using Twitter to support reflective learning in an asynchronous online course.” Australasian Journal of Educational Technology 35(3), 31-44.
• Vonderwell, Selma, Liang, Xin and Kay Alderman (2007). “Asynchronous discussions and assessment in online learning.” Journal of Research on Technology in Education 39(3), 309-328.
• Vygotsky, Lev (1978). Mind in Society: The Development of Higher Psychological Processes. Cambridge: Harvard University Press.
• Wang, Xiangling, Wang, Lv and Hui Yin (2017). “The construction and application of the translation teaching model of mobile learning community.” Technology Enhanced Foreign Language Education 4, 31-38.
• Wenger, Etienne (2010). “Communities of practice and social learning systems: The career of a concept.” Chris Blackmore (ed.) (2010). Social Learning Systems and Communities of Practice. London: Springer, 179-198.
• Wenger, Etienne, Trayner, Beverly and Maarten de Laat (2011). Promoting and Assessing Value Creation in Communities and Networks: A Conceptual Framework. Heerlen: Ruud de Moor Centrum.
• Winne, Philip, Hadwin, Allyson and Nancy Perry (2013). “Meta-cognition and computer-supported collaborative learning.” Cindy Hmelo-Silver et al. (eds) (2013). The International Handbook of Collaborative Learning. London/New York: Routledge, 462-479.
• Yang, Shih-Hsien (2009). “Using blogs to enhance critical reflection and Community of Practice.” Journal of Educational Technology and Society 12(2), 11-21.

Appendix 1: An evaluation rubric of PE quality

Appendix 2: Structured interview questions

Appendix 3: Student perceptions questionnaire

Please rate the extent to which you agree or disagree with the following statements.

Biographies

Lyu Wang is a lecturer of Translation Studies at Changsha University of Science and Technology. In 2020, Lyu obtained her PhD in Translation Studies from Hunan University. Her research interests include translation education, translation and cognition, and machine translation post-editing. She has published in TS journals, such as Current Trends in Translation Teaching and Learning E and Journal of Foreign Languages.

E-mail : wanglv66@126.com

Xiangling Wang is a Professor, Director of Centre for Studies of Translation, Interpreting and Cognition of Hunan University. She is currently hosting a few projects both at national and state level on translation process and interactive machine translation. Her research interests include translation education, translation and cognition, and machine translation post-editing. She has published widely in journals, such as Translation and Interpreting Studies, Journal of Specialised Translation, Computers and Education, and Interactive Learning Environments.

E-mail : xl_wang@hnu.edu.cn