Home > Issue35 > Lin article

Translation or creation? A case study of signed Chinese poetry from the perspective of multimodality theory

Hao Lin, Shanghai International Studies University

ABSTRACT

The translation of a written text into a sign language text is a plurisemiotic practice, as the written text is a two-dimensional system inherent to the relevant spoken language, while sign language has three dimensions, in which multimodal elements interact simultaneously and merge into the wholeness of production. This paper will focus on exploring poetry translation, one of the most challenging aspects of Translation Studies. We report a case study of translating a poem called We are looking for a light (1982), which is originally presented in Chinese written text, into Chinese Sign Language to create a signed poem. An in-depth comparison between the source and target texts shows that 1), multimodal elements co-exist and interact in the process of translation and 2), various strategies are adopted for intersemiotic translation, including re-ordering, merging, addition, deletion, neologism, and repetition. The study further confirms the value of the theory of multimodality and outlines the challenges associated with it.

KEYWORDS

Plurisemiotic, multimodality, Chinese Sign Language, poetry translation.

1. Introduction

In recent years, Translation Studies has shifted its focus from monomodality to multimodality. As Kaindl (2013: 257) reports, “multimodality is the rule.” Based on the theory of multimodal communication, defined by Kress and van Leeuwen (2001) and further refined by Kress (2010), translation is also regarded as a plurisemiotic practice in general. Texts, as the basis for translation, are composed of a combination of different modes rather than only linguistic texts (monomodality). Apart from the traditional verbal texts, other non-verbal modes should also be highlighted. The practice of translation is a process of correlation and interaction between different semiotic resources which allows people to overcome cultural barriers. As the focus is shifted to the cultural-semiotic dimension of texts, two types of translations are identified: ‘inter/intra-lingual’ translation and ‘inter/intra-semiotic’ translation (Jakobson 1959). Inter/intra-lingual translation refers to translation between or within verbal languages, while inter/intra-semiotic translation refers to different communicative systems, for example, verbal vs non-verbal. Kaindl (2013) points out that the current challenge for multimodal Translation Studies is to develop investigative instruments appropriate for the research of non-language modes. This is of particular focus in this paper, as the current study aims to apply the basic principle of multimodality to an in-depth analysis of the translation practice of a particular mode: that of signed translation of poetry.
Poetry exists as a genre across various languages, even sign languages. In their ground-breaking work, Klima and Bellugi (1979) initiated the linguistic analysis of sign language poetry by introducing the analysis of facial features in sign language. In general, research on sign poetry is sporadic but widespread in the world: see American Sign Language (‘ASL’, Rose 1993; Valli 1993; Ormsby 1996), British Sign Language (‘BSL’, Sutton-Spence and Woll 1999, Sutton-Spence 2005), French Sign Language (‘LSF’, Blondel and Miller 2001, Chateauvert 2014), Italian Sign Language (‘LIS’, Russo et al. 2001), or Chinese Sign Language (‘CSL’, Li and Wu 2014).
Sign language translation, as one of many subfields of Translation Studies, is related to the theory of multimodality. This idea has been explored by a number of studies in recent years. For instance, Eddy (2004) describes the translation of American Sign Language as a combination of the modes and the media. Similarly, Pollitt (2014) explores sign language poetry as Gesamtkunstwerk. Moreover, the multimedia dimensions of sign language translation and theatre performance has further been examined by Weir and Kelstone (2006) and Richardson (2017). Additionally, Remael et al. (2016) discuss the relationship between Translation Studies and audiovisual translation, as well as media accessibility. From ideology to education, Spooner (2016) looks at deaf students’ language ideologies through English-to-ASL translations of literature. Wurm (2014) conducts a qualitative case study analysing one practitioner’s strategies in the translation of an academic text from written English into British Sign Language.
Despite this plethora of research, there is, however, very limited research so far which discusses the translation of signed poetry from the perspective of multimodality. The significance of such a study is of particular importance for the area of sign language translation. Specifically, firstly, signed poetry (the target text) is non-verbal and uses specific language which differs from the other non-verbal modes. Secondly, the signed poetry genre is larger than any other genres of sign language, such as narratives. As signed poetry blends both linguistic features and non-linguistic elements, its translation makes full use of the elements it shares with other non-verbal modes. Therefore, signed poetry can be classified as a plurisemiotic practice. It involves translating the two-dimensional and fixed texts in print (source) to three-dimensional dynamic texts. In order to fully understand intersemiotic translation, it is worthwhile to look into a case study which analyses this mechanism in depth. Therefore, in this paper, we adopt a modern poetry ST with written Chinese as the source, and signed form in Chinese Sign Language as the target.

1.1 Background of Chinese Sign Language: Deaf people and poetry

There are more than 20 million deaf Chinese people and 688 deaf schools across China. It is estimated that there are currently 600,000 deaf students at various stages of their education¹. Literary education is compulsory for Chinese students, and therefore deaf students are taught Chinese poetry. Ever since Confucius (551 BCE – 479 BCE) edited the Shi jin (the literal meaning of shijin is ‘poetry classic’) in his late years, Chinese poetry has long been regarded as the foundation of culture and education. Though sign language has been used as a medium of instruction for more than one hundred years, oralism has still dominated deaf education, and Chinese Sign Language (CSL), at best, only played a secondary role in deaf education. Thus, for a long time, some deaf students preferred to show their competence in composing traditional Chinese poetry (written poetry), and it was not known if there was any practice of signed Chinese poetry.

Over the years, however, CSL’s status has risen and this stimulated the creation of signed Chinese poetry. The reason for this increase in status is twofold. Firstly, sign language is increasingly acknowledged in China as a language equivalent of Mandarin. In particular, advocating CSL in government policies and the advance of CSL linguistic research has helped the language to awaken in the deaf community. Secondly, the progress of both the economy and technology, especially the accessibility and popularity of filming among the deaf, has encouraged deaf poets to produce more sign language pieces. As visual media becomes more and more friendly to the production and communication of sign languages, it is often the case that these amateur poets use social media to share their creations and win their own audiences. Signed Chinese poetry can sometimes be produced in the deaf classroom when deaf students learn about written Chinese poetry. In such an instance, sign language interpretation is necessary to allow students to understand the poems. However, conscious translations of Chinese poetry into CSL is a relatively recent development.

The China Poetry Recital Tournament (Zhonghua Shici Dahui) is a very popular cultural and nationwide activity. Many contests are hosted in different cities and are even broadcast on television. During this event, poetry is recited in Mandarin. In 2018, the Shanghai Deaf Association initiated a poetry recital contest in sign language among the deaf. This was very unusual, as it is custom for the deaf taking part in such contests to recite poems in spoken Mandarin. During this tournament, deaf contestants were asked to express the beauty of written Chinese poetry in sign language for the very first time in a public event. Christie Ni, the translator and performer of signed poetry in the current case study, was born deaf and comes from a deaf family. She received her literary education in deaf schools. As mentioned above, deaf students learn Chinese poetry as a compulsory part of their literary education. However, it is difficult for deaf students to fully understand and appreciate Chinese poetry simply from written text. Ni has successfully translated a number of written Chinese poems into signed poetry, and these translations are widely acclaimed by the deaf community. This made Ni a famous amateur deaf poet among the deaf community in Shanghai, and possibly throughout China.

1.2 Research questions, methodology, and arrangement

Keeping the framework of multimodality in mind, the research question underlying this work is to examine how a particular plurisemiotic translation is realised in its exact form. The exploration of this question will be broken down into two steps. Firstly, the original written text and signed translations will be compared as this will allow the similarities and differences in their organisation to be determined. Second, the layers of signed poetry will be further investigated to identify the mechanisms within the signed text which allow for the combination and interaction of different modes. These modes can be linguistic or non-linguistic, and translations of figures of speech and suprasegmental elements will also be included.

The method we employed in this study was to select a piece of signed poetry which was initially analysed in detail. We have chosen one of the sign language poems which was interpreted by Ni, namely We are looking for a light (1982; abbreviated hereafter as Light). Light (1982) is a short and rich piece of poetry, full of vivid images; under its ostentatious simplicity is ambiguity and infinite interpretations. All of these features make it an ideal test case for sign language translation. The original poem can be found in the Appendix, together with a literal English translation and an English annotation of the signed translation. A recording of Christie Ni’s performance in CSL (Lin 2019) can be viewed online². Because CSL does not have a well-established writing system, poetry in sign language exists as live performances or on video. Therefore, we will use stills from a video to analyse the performance of a signed Chinese poem. Traditionally, three basic parameters of sign language words are identified: handshapes, location, and movement. However, hand orientation and non-manuals (facial expressions) have also been taken into consideration as building blocks of sign words (Klima and Bellugi 1979). Movement and handshape/location are believed to be comparable to vowels and consonants respectively, and the so-called syllables of signs are identified by complete movements of a handshape over certain locations. In this paper, words spelled out in capital letters refer to sign-language words (i.e. FLOWER, see Figure 1). Words with a plus sign (+) next to them denote repetition, for example ‘WALK+’ means repetition of the sign for ‘walk’. Hyphens are used to show that multiple words in the written language are shown as a single word in CSL, for example, ‘SUNRISE-OVER-SEA’ is a single word in CSL.

Figure 1. ‘FLOWER’ (Left), ‘SUNSHINE’ (Right) (Lin 2019).

Furthermore, Sutton-Spence (2005) offers some reliable dimensions for looking at typical features of signed poetry, such as symmetry, morphing, and neologism; a description of these features is also echoed in the present analysis of Light (1982). Because the signed translation is based on a visual-spatial dimension, the annotation tool ELAN (The Language Archive no date) will be used to annotate the piece of signed poetry in a way which allows for a detailed comparison with the original piece (see Figure 2).

Figure 2. A snapshot of the ELAN annotation.

The paper will begin by introducing the original poem and will then describe the features of the signed translation in relation to the original in Section 2. Then, Section 3 will further identify the linguistic mechanisms adopted in the translation at different levels of the text. In Section 4, features of the translation from written Chinese text to signed poetry will be discussed while taking into consideration the use of special plurisemiotic practice and its challenges, as well as contributions to the theory of multimodality. Finally, the Conclusion (Section 5) will provide a summary of the study’s findings.

2. Analysis of We Are Looking for a Light (1982)

With poetry translation, the translator must strive to successfully meet the expectations and sensibilities of the original text. The greatest challenge with translating written Chinese poetry into CSL is that it involves the transfer of a written form (spoken language) to the visual form (sign language), i.e., an intersemiotic translation. Before discussing the analysis of the signed translation of the poem, a short introduction of the original poem is first given, and then the original and translated versions are contrasted.

2.1 Short introduction to the original

We are looking for a light was written in 1982 by Gu Cheng (顾城，1956-1993), one of the most famous poets in modern Chinese literature. Light is a short free verse, with 23 lines and seven stanzas (see Appendix). It was written in modern Chinese. Close analysis reveals that it consists of 146 Chinese characters, or 85 words, as many modern Chinese words are formed out of two characters. For example, 我(wǒ)们(men) means ‘I-PLURAL’. The poem has its own rhyme and tempo, such as the marked repetition of the first two lines of Stanza 1 (S1), which appears four times in S1, S3, S5, and S7:

Having walked so far
We go looking for a light

The repeated lines function both as boundary markers between three different contexts and as a refrain for introducing and emphasising the theme of the poem.

2.2. A general analysis of the signed translation of Light (1982)

The signed translation of the poem keeps the main structure of the original form. Both the original and the sign language translation are composed of seven stanzas and 21 lines in total. However, the translation consists of only 39 signs, whereas the original poetry is composed of 85 words/146 Chinese characters. This difference highlights the importance of exploring the sign translation both internally and externally. It is possible to see that the translation follows the original stanzas. For the refrain-like stanzas, S1, S3, S5, and S7, the equivalent expression is the repetition of the following:

WALK+ SHADOW-DISAPPEARING

Figure 3. ‘Walk further and further with the shadow shrinking’ (Lin 2019).

As can be seen in Figure 3, the handshapes are only one aspect of the performance: instead, the performer bends over, looks in different directions, and even plays with her lips. All of these elements are combined together to form the wholeness of the translation. For example, ‘WALK+’ means that we ‘walk, walk, and walk’, and ‘SHADOW-DISAPPEARING’ is a vivid expression of disappearing shadows (Figure 3). Therefore, as we walk further and further, our shadows become smaller and smaller from afar behind.

LOOK-FOR LIGHT-GAZE

Figure 4. ‘(We are) looking for a light’ (Lin 2019).

‘LOOK-FOR’ is signed by the arc movement, with a ‘V’ handshape, and it is realised with two hands moving in a symmetrical way, followed by the transition of the non-dominant hand (left hand) into the shape of ‘a light’ while the dominant hand (right hand) keeps its movement and handshape (Figure 4). ‘LIGHT-GAZE’ is not a typical sign, but rather a constructed action, as it seems to violate the phonological principle that if a sign involves the movement of two hands, both hands will move in a symmetrical way (Battison 1978). Here, the translator takes advantage of the two articulators to illustrate two different signs simultaneously: the non-dominant hand for signing ‘a light/lamp’, and the dominant hand for signing ‘look for.’ Thus, two signs are articulated as a simultaneous construction. ‘LIGHT/LAMP’ is signed fractionally earlier and then held, and at the same time, ‘LOOK-FOR’ is signed. This creates a strong visual image through two classifiers. These two lines appear four times in the translated version, in positions S1, S3, S5 and S7.

Figure 5. ‘LOOK’ in S2, S4, and S6 respectively (Lin 2019).

However, when translating ‘you say’, the interpreter turns her hands into a pointing sign, which is completed with an aligned eye gaze (Figure 5). In this context, pointing can have many meanings, as it can be used to direct attention or initiate a conversation. In the world of sign language, vision is always the main form of media, which is why the translation of the pointing sign is considered equivalent to ‘you say’. Specifically, by using the pointing sign, the translator turns ‘you say’ into ‘you look’. Furthermore, in this interpretation, the interpreter adjusts the position of ‘you look’. All three expressions of ‘you look,’ which are in the first line of the original stanza, are placed in the middle of the stanza. They have been degraded into a sign within a line, rather than presented as an independent line which begins the entire stanza. The adjustment is made to conform to the poetic structure of sign language and to the customs of deaf signers. Furthermore, the loci (positioning) of the pointing in the three stanzas differs. The pointing in S2 is directing the view to the windows upwards, while that of S4 is to the train, and that of S6 is directing downwards to the children. In line with Schlenker’s (2017) argument that pointing in sign language presupposes some properties of the anaphora, invisible in spoken languages, it can be understood how semantic information is encoded in the three different pointing positions.

Let us now consider S2, S4, and S6, the three main scenes. S2 is rendered into:

TWILIGHT FLOWER
LOOK
HOUSE WINDOW SHINE
FLOWER-BLOOM-MERRILY GRATEFUL

The literal meaning of the above is: ‘at dusk there are flowers on the earth of the court and a lamp shines into the window of a house, which makes one feel quite joyful. The flowers bloom with a sweet smell, making one full of gratitude and felicity’. The author makes several adjustments to the original text in order to adapt to the plurisemiotic target text. First, the interpreter introduces time by using an ‘O’ handshape, moving in an arc downwards, which shows that it is in the evening. Then, she follows with the sign of ‘flowers on the earth’. While in the original there is a reference back to ‘a lamp’, this is realised in another way in the signed translation. As topic-comment construction is much more common in Chinese Sign Language, and for constructing poetic CSL structures in general, the background is often expressed first and the reference needs to be given a location in the signing space.

‘LOOK’ is signed as a shift of perspective, to a series of images in an order that works better in sign language: ‘house-window-light shine passing into indoors through the window’. Then, with the light shining in, the flowers refer back to the first line of S2 ‘bloom with smell’ rather than to ‘turn into another colour’ as it is in the original. The translator directs the audience from a visual image of the flower to its smell because the image of the flower is non-specific and the colour that the flower is turned into under the non-specific light is also vague. For hearing readers, the meaning of the written form in the original poetry mainly resorts to activating the verbal features. However, when it is translated into a signed form for the deaf, the text needs to be seen directly in a three-dimensional space. Therefore, the author singles out the most prominent feature of the flower in general, its odour, and connects it with the mood of stanza 2: ‘bright’, ‘warm’, and ‘sweet’, which is suggestive of a happy childhood and the gratitude for a home and a mother. The translator makes the mood explicit by adding the sign ‘GRATEFUL’, which can also mean ‘memory.’
The second main scene is S4, which suggests a life of struggle and loneliness. The translator initiates this stanza with the background again: the evening. The expression varies, with the same arc movement and different handshapes to show variance within the same rhyme:

IT-IS-DARKENING

Then, as the evening slides in and darkness covers the sky, we see the wild grass is blown all over vehemently by the wind, and nearby trains keep on passing-by quickly and competitively, as shown by the following two lines:

WILD-GRASS-BLOWN-BY-THE-WIND
TRAIN-RUNNING+

As the audience can see facial expressions in ‘TRAIN-RUNNING+’, they can anticipate the next line and the observer is caught by a mixed mood, which creates both a painful and a warm memory. There are two articulators – the left and right hands are divided again, the same handshape is used (the clawed palm or bent ‘5’ handshape), but the hands are in different locations (the left or non-dominant hand is close to the chest while the dominant hand moves close to the head). The hands form an antithesis in movement and meaning: the memory is warm to the heart while painful to the head.

HEART-WARM-WEARISOME-MEMORY

This line is followed by a pointing sign (see Figure 5):

LOOK

This shifts the scene right away, as ‘LOOK’ is signed by the left hand and the pointing handshape transforms smoothly into the classifier of a train (the ‘V’ shapes resemble the wheels of the train). The left hand, which keeps its shape, slowly moves to the top of the train classifier, and away from the interpreter’s chest. This movement denotes that the trains take away all of his or her pains. Again, the translator creates a complex simultaneous sign expression:

TRAIN-CARRYING-AWAY-MY-PAIN

Figure 6. ‘Train carries away my pain’ (Lin 2019).

A discussion of S6, the third scene, will now follow. In the original poem, the image of the light in S6 is still very dim. The light is set beside the sea and is presented as beautiful as a golden orange. In the signed version, the light is set up against the sea over the horizon (the sea is signed first as a background), rather than beside the sea as in the original version. The time is moved to dawn, with the sun rising from the sea and then shining over it brightly. The two lines are signed with very smooth transience, presenting a continuous moving image.

SEA SUNRISE-OVER-THE-SEA
SHINE-ALL-OVER

Again, to allow for a transition, the word ‘LOOK’ is introduced which shifts the perspective to the children playing by the sea, bouncing along joyfully. The word ‘BEAUTIFUL’ can refer to the children or to the scene overall. In the translation, ‘COME-ON’ is added to express the imperative, while the line that all children who love the light will grow up is omitted.

LOOK
CHILDREN-BOUNCING
HAPPY+ BEAUTIFUL COME-ON

In the next line, it seems that the translator deviates from the original poem and makes her own interpretation. Importantly, however, this distinction is made on purpose. Specifically, the translator makes this deviation in order to create coherence and unity within the signed poem. Thus, it could be considered that the signed version of Light (1982) makes some parts of the original more explicit. First, the timeline moves ahead as they walk: it is dusk in S2, evening in S4, and early morning in S6. On the other hand, in the signed translation the present tense has replaced the mixed tenses of the original poem, thus ‘the children that love it will grow up’ is intentionally disregarded. Secondly, the symbolic image of light is very rich and ambiguous, and the image of the light/lamp is also non-specific and dubious in the signed version. For example, in S2, the handshape of ‘LIGHT,’ a fist-handshape opening to 5-shape, can be denoted either as a lamp or as the shine of a lamp; however, the light in S6 has been explicitly referred to as the sun. Together, these indicate that Light (1982) has been made more concrete in the signed version.

Furthermore, as the mood of the original poem is hidden, any interpretation is very implicit, because the subject ‘I’ is hidden, and all of the scenes are told in a third-person narrative. The poem is more like a conversation between two friends along a journey and the other side, the ‘me’ side, is silent. However, the signed translation changes this ambiguity, so that the pronoun ‘I’ emerges ‘from behind the curtain’ and leads the talk along the journey. This ‘I’ has mixed feelings when gazing at the passing train in a lonely station surrounded by wild grass, and then relaxes as if the trains take away his or her pain. The subject ‘I’ even ‘comes on’ to you in S6 to participate in the appreciation of the sunlight over the sea, with children playing nearby. As sign poetry is face-to-face communication embodied with communication, especially when performed, it is easier to involve the audience in empathy when taking a first-person perspective. However, the poem is based on written Chinese, which naturally distances itself from the present. The written form of this poem is supposed to render empathy into sound when it is recited orally, which is more akin to music; however, it is more like a dance when it is translated into signed poetry. S1, S3, S5, and S7 are all formed by two recurring lines, which function as a refrain.

3. Strategies of signed translation

After comparing the two texts, this section elaborates on the strategies adopted at the different levels of organisation to meet the challenges faced by intersemiotic translation. According to Klima and Bellugi (1979), the form of poetry can be analysed from macro-level, mid-level, and micro-level perspectives. Macro-level perspectives look above the lines or between the stanzas, medium-level perspectives consider the lines within a stanza, and micro-level perspectives focus on the words. At the macro-level, the signed translation has kept the frame of the original poem but makes substantial changes to its form to adapt to the target multimodal text. While it apparently follows the pattern of the original faithfully at the macro level, there is actually one exception: the positioning of ‘YOU SAY.’

3.1 Strategies of signed translation at the medium level

Many more adjustments are made at the medium-level, as demonstrated above. These strategies include the following six items: re-ordering, merging, addition, deletion, neologism, and repetition.

1) Re-ordering

All of the main bodies of S2, S4 and S6 have been drastically re-ordered in the signed form. Let us take S2 as an example:

As we can see from the above, the basic sequence of events has been kept: the first three lines mainly describe the light and the last line describes the flowers. However, the translation has made a drastic change between the lines in S2. In the original, ‘the light’ appears first, then the curtain, wall, and flowers. In contrast, what appears first in S2 of the translated version is twilight (which denotes the time), then the flower, house, window, light (shine), and then the flower again. The sequence of the presented images is different between the original and the translation. The motivation behind this re-order is that CSL poetry has a different organisational model which is constrained by the visual-spatial modality. As in S2, the passive construction in the original has been replaced by the equivalent expression in CSL, which can be translated back into ‘there are windows in a house, and the light shines into the house through the windows.’

2) Merging

As demonstrated in the previous section, far fewer signs are required to present the original poem in CSL. In the translated version, some neighbouring words have been merged into one sign. For example, ‘You say’ has turned into ‘LOOK’; in this case, the meaning of ‘you look’ can be derived because it is embodied by the signer herself in the performance.

The same goes for S1:

Having walked so far
WALK+ SHADOW-DISAPPEARING

We go looking for a light
LOOK-FOR LIGHT-GAZE

The first-person narrative is merged into ‘LOOK-FOR,’ because ‘WALK’ has incorporated the plurality of the noun. Therefore, ‘we’ in the original is invisible in the translation.

3) Addition

In order to improve the translation, the translator creates new signs for her interpretation. For example, ‘IT’S-DARKENING’ (L1, S2), ‘TWILIGHT’ (L1, S4), and ‘SUNRISE-OVER-THE-SEA’ (L1, S6) are all poetic creations. The regular addition of these signs functions as a time indicator and creates a background for the changes of time and place. For example, ‘TRAIN’ in ‘TRAIN-CARRYING-AWAY-MY-PAIN’ (L6, S2) is added as an imaginative extension, derived from the translator’s understanding of the original text. The addition of ‘COME-ON’ in L5, S6 also functions as a connector for introducing the last stanza, whilst giving weight to the end of the poem. Therefore, addition is an important mechanism in sign language translation and can be used for various purposes.

4) Deletion

Deletion can be found in many places throughout the translated poem. It mainly falls into two categories: entities and modifiers. The former includes ‘wall’ (Line 3, S2 in the original), ‘another colour’ (Line 5, S2), ‘station’ (L2, S4), and ‘orange.’ The latter includes some colour words such as ‘pure white’ (L3, S4) and ‘golden’ (L3, S6). Furthermore, two action events are ignored in the signed version: ‘flowers transplanted’, (L3, S2) and ‘Children will grow up’ (L5, S6).

5) Neologism

Neologism is often used to extend the expressiveness and creativity of sign language poetry. This poem is filled with neologisms, for instance:

SHADOW-DISAPPEARING
LIGHT-GAZE
HEART-WARM-WEARISOME-MEMORY
TRAIN-CARRYING-AWAY-MY-PAIN
SUNRISE-OVER-SEA

All of these words require two hands to be used, as this can maximise expression in a limited time. As there are two main articulators in sign language, this feature is often explored in sign language poetry for constructing simultaneous sign words. They are all also very iconic; for example, in ‘LIGHT-GAZE’, the left handshape is a classifier for a lamp and the right handshape is that of ‘LOOK-FOR’, while its direction is pointing at the lamp. Together, this makes a complete unit which is translated as a compound sign. All of the words are also context-dependent and the interpretation of the specific new ‘signs’ depends on the understanding of the poem in general, as well as on the previous or following signs. For example, the meaning of ‘SUNRISE-OVER-SEA’ can only be understood if we know that the text is a poem about looking for a light; that the sunshine could be either singular or plural; and that the previously introduced ‘SEA’ provides a background for us to interpret the compound.

6) Repetition

Repetition is used to keep the tempo or to place emphasis. There are two occurrences in the translation:

WALK+
HAPPY+

‘WALK’ is repeated three times while ‘HAPPY’ is repeated twice. These repetitions of ‘WALK’ are an iconic imitation of walking for a long time. Thus, the mood of the original poem is truthfully conveyed using this strategy of repetition.

3.2 Strategies at the Micro-Level

Lastly, let us consider the micro-level of the poem’s structure. As we can see, it only takes 39 CSL words to translate the written Chinese poem, originally composed of 85 words. This implies that sign words are more efficient per unit. The basic parameters of sign words can be modified for an artistic effect so that they are different from the sign language words used in everyday life. In contrast with its citation, the movement of ‘LOOK-FOR’ in S1 is smooth, circular, and symmetric, as it changes from one-hand sign to a two-hand sign. Almost all of the parameters are modified, except for the handshape.

Table 1. Parameters of poetic and citation forms of ‘LOOK-FOR’.

In order to establish an overall poetic effect, almost every sign word in the translation is larger than its every day counterpart. The sign words in the poem often require larger and slower movements, which is similar to an increase in volume and perseveration of the vowels in spoken language. At the same time, there are also pauses between the words or lines when locations are introduced. Handshape in signed poetry is purposefully manipulated to meet the needs of translation. For example, ‘TWILIGHT’ (L1 of S2) and ‘IT-IS-DARKENING’ (L1 of S4) are identical, except for the handshapes. The former adopts a ring handshape and the latter employs changing the handshapes from open to closed hands, thereby representing the closing of the day. The ring handshape is less conventionalised as the sign for TWILIGHT, which is intended to make the sign more iconic, and to also add variety to the poem and to draw a comparison between the two lines.

Phonological iconicity is also exploited for effect in poetry translation. Let us compare an unmarked 5-handshape, which is often associated with width, joyfulness, or ease of life, to the clawed-5-handshape (marked handshape), which alludes to the passion of nature, hardship, or sadness. We can see that the 5-handshape is shared by ‘SUNSHINE’, ‘SEA’, and ‘HAPPY’ in S6 as this is a stanza which describes the future, hope, and the beauty of dreams. In contrast, the claw-5-handshape repeatedly appears in L4-L6 in S4, a stanza which alludes to the cruelty of life in a lonely place and to the regrettable memories of the past. The greatest change occurs at the micro-level due to the morphological differences between Chinese and CSL. More often than not, a signed word contains more information than a Chinese word, because CSL signs can be stacked together simultaneously while Chinese words can only appear in linear form and cannot be spatialised. All the parameters are reconstructed or deconstructed to extend the potentiality of visual-spatial expressiveness.

4. Signed poetry: Inborn multimodality

We have seen that the translator has made profound changes to the original poem in order to adapt it to the plurisemiotic system. All of the features of signed poetry support its natural multimodality. As shown in the above analysis, sign language poems contain not just CSL ‘words’, but also other non-verbal elements, such as constructed action, facial expression, eye-gaze, lip-shaping, and body movement (meaning whole-body movement, rather than the movement parameter within the signed word). Kaneko and Mesch (2013) highlight the significance of eye-gaze in the creative expression of sign language. To illustrate this, in the signed version of Light (1982), the eye-gaze of the signer is directed at the classifier of light when signing ‘LIGHT-GAZE’ (Figure 4). Similarly, in ‘TRAIN-CARRYING-AWAY-MY-PAIN’ (Figure 6), the eye-gaze of the signer follows the movement of the two hands closely, which helps to direct the focus of the eye to the visual images presented manually. Figure 5 also shows the coarticulation of pointing and eye-gaze. Facial expressions are prevalent and salient throughout the whole signed version of the poem.

In addition, body movements and velocity of the hand movements all function to convey the rhythm and tempo of the poem. Interestingly, Napoli and Liapis (2019) have looked into effect reduction, shared by both dance and sign language. They concluded that signed poetry, as a genre of art, can be compared to dance not only in its visual appeal, but also because of some shared basic elements, like the rhythm, tempo, and symbol. These elements work together in a poetic orchestra with sign words to make the translation more expressive. Interestingly, a popular international written system named signwriting originates from dance-writing. All elements or modes, either linguistic or non-linguistic, interact and merge with one another seamlessly and simultaneously to form a harmonious unity in signed poetry translation.

From a different perspective, the genre of signed poetry is more comparable to oral poetry or a poetry recital. This is because both modes interpret the written texts of the poetry and translate them into multimodal texts each time the poems are recited. Signed translation of written poetry turns it into both intersemiotic and interlingual practice if we assume that sign language is a language, despite being also non-verbal at the same time. However, one difference is that sign language mainly resorts to our visual sense, whereas a live oral recital makes use of the audio-visual channel: audio for the verbal/linguistic mode and visual for the non-verbal/non-linguistic modes.

5. Conclusion

In this paper, we have explored the translation of a written poem into a signed poem. It was concluded that such a translation is a plurisemiotic practice, in which different modes or sub-modes co-exist and co-operate for the ultimate unity of translation. Both linguistic (sign words) and non-linguistic elements (constructed action, gesture) merge into a wholeness which cannot be divided in its function. Various strategies at different levels are applied in translation: re-ordering, merging, addition, deletion, neologism, and repetition, though the main frame of the original text is kept. Such a case study helps us to understand more about the modality, and expands our views on the richness of multimodal texts. It also offers a methodology for the analysis of poetry translations, from the written mode to the signed mode.

Acknowledgements

This research is supported by the Ministry of Education in China (MOE)- Project of Humanities and Social Sciences (No. 20YJC740027), the China Disabled Persons’ Federation (CDPF)-project of Braille and Sign Language (CLS2019—05) and Innovative Research Team of Shanghai International Studies University. We would like to thank Yingjie (Christie) Ni for sharing her images and poetry, and above all, for the inspiration. We also are very grateful to the anonymous reviewer(s) for the sparkling comments, and finally for the patience and helpfulness of the editors.

Bibliography

• Battison, Robbin (1978). Lexical borrowing in American sign language. Silver Springs: Linstok Press.
• Blondel Marion and Miller Christopher (2001). “Movement and Rhythm in nursery rhymes in LSF.” Sign Language Studies 2, 24-61.
• China Disabled Persons’ Federation (CDPF). (2019). “Education.” 2020/11/19. http://www.cdpf.org.cn/sjzx/ (consulted 15.11.20).
• Chateauvert, Julie (2014). Poétique du mouvement: ce que les langues des signes font à la littérature. PhD thesis. Université du Québec à Montréal.
• Eddy, Shauna (2004). Signing identity: Rethinking United States poetry, acts of translating American sign language, African American, and Chicano poetry and the language of silence. PhD thesis. University of Southern California.
• ELAN (no date).The Language Archive. https://archive.mpi.nl/tla/elan (consulted 15.11.20).
• Jakobson, Roman (1959). “On Linguistic Aspects of Translation.” Reuben A. Brower (ed.) (1959). On Translation. Cambridge, MA: Harvard University Press, 232-239.
• Kaindl, Klaus (2013). “Multimodality and translation.” The Routledge handbook of Translation Studies. London and New York: Routledge, 275-288.
• Kaneko, Michiko and Johanna Mesch (2013). “Eye gaze in creative sign language.” Sign Language Studies 13(3), 372-400.
• Klima, Edward and Ursula Bellugi (1979). The Signs of Language. Cambridge, MA: Harvard University Press.
• Kress, Gunther (2010). Multimodality: A social semiotic approach to contemporary communication. Abingdon: Taylor and Francis.
• Kress, Gunther and Theo van Leeuwen (2001). Multimodal Discourse: The Modes and Media of Contemporary Communication. London: Arnold.
• Li, Heng and Ling Wu (2014). “A review of sign language poetry (Shouyu shige yanjiu Zhong shu).” Chinese Journal of special Education 166(4), 31-34.
• Lin, Hao (2019). “We look for a light by Christine NI.” https://www.youtube.com/watch?v=rAlD9L--Bj0&feature=youtu.be (consulted 15.11.20).
• Napoli, Donna and Stephanie Liapis (2019). “Effort reduction in articulation in sign languages and dance.” Journal of Cultural Cognitive Science, 1-31.
• Ormsby, Alex (1996). The poetry and poetics of American Sign language. Stanford University.
• Pollitt, Kyra (2014). Signart: (British) sign language poetry as Gesamtkunstwerk. PhD Thesis. University of Bristol.
• Remael, Aline, Nina Reviers and Reinhild Vandekerckhove (2016). “From Translation Studies and audiovisual translation to media accessibility: Some research trends.” Target: International Journal of Translation Studies 28(2), 248-260.
• Richardson, Michael (2017). “Sign language interpreting in theatre: using the human body to create pictures of the human soul.” TranscUlturAl: A Journal of Translation and Cultural Studies 9(1), 45-62.
• Rose, Heidi (1993). A critical methodology for analyzing American Sign Language literature. PhD Thesis. Arizona State University.
• Russo, Tommaso, Rosaria Giuranna and Elena Pizzuto (2001). “Italian Sign Language (LIS) Poetry: Iconic Properties and Structural Regularities.” Sign Language Studies 2, 24-61.
• Schlenker, Philippe (2017). “Sign language and the foundations of anaphora.” Annual Review of Linguistics 3, 149-177.
• Spooner, Ruth Anna (2016). Languages, Literacies, and Translations: Examining Deaf Students' Language Ideologies through English-to-ASL Translations of Literature. PhD Thesis. University of Michigan.
• Sutton-Spence, Rachael and Bencie Woll (1999). The linguistics of British Sign Language: An introduction. Cambridge: Cambridge University Press.
• Sutton-Spence, Rachael (2005). Analysing sign language poetry. Basingstoke, Hampshire: Palgrave Macmillan.
• Valli, Clayton (1993). Poetics of American Sign Language poetry. PhD Thesis. The Union Institute Graduate School.
• Weir, Aaron and Aaron Kelstone (2006). Sign language theatre: expression, language, and transformation. Master’s Thesis. Cleveland State University.
• Wurm, Svenja (2014). “Deconstructing translation and interpreting prototypes: A case of written-to-signed-language translation.” Translation Studies 7(3), 249-266.

Appendix

1. The original poem

S1/L1走了那么远

我们去寻找一盏灯

 S2你说
它在窗帘后面
L5被纯白的墙壁围绕
从黄昏迁来的野花
将变成另一种颜色

 S3 走了那么远

我们去寻找一盏灯

S4/L10 你说
它在一个小站上
注视着周围的荒草
让列车静静驰过
带走温和的记忆

S5/L15走了那么远

我们去寻找一盏灯

S6 你说
它就在大海旁边
像金桔那么美丽
L20所有喜欢它的孩子
都将在早晨长大

 S7 走了那么远

L23 我们去寻找一盏灯

S1/L1走了 那么 远

我们 去 寻找 一盏 灯

 S2你 说
它 在 窗帘 后面
L5被 纯白的 墙壁 围绕
从 黄昏 迁来的 野花
将 变成 另一种 颜色

 S3 走了 那么 远

我们 去 寻找 一盏 灯

S4/L10 你 说
它 在 一个 小站 上
注视着 周围的 荒草
让 列车 静静 驰过
带走 温和的 记忆

S5/L15走了 那么 远

我们 去 寻找 一盏 灯

S6 你 说
它 就在 大海 旁边
像 金桔 那么 美丽
L20所有 喜欢 它 的 孩子
都 将在 早晨 长大

 S7 走了 那么 远

L23 我们 去 寻找 一盏 灯

2. Literal English translation by Hao Lin of We are looking for a light (1982)

We are looking for a light

Having walked so far
We go looking for a light (S1)

You say
It’s hidden behind the curtain
Surrounded by the pure white wall
The wildflowers transplanted here
Will turn into another colour (S2)

Having walked so far
We go looking for a light S3

You say
It’s hanged above a small station
Gazing at the wild grass around
And a train passed by peacefully
Taking away all the warm memory S4

Having walked so far
We go looking for a light S5

You say
It is besides the sea
Beautiful like a golden orange
All the kids that like it
Will grow up in the morning S6

Having walked so far
We go looking for a light S7

3. English annotation of the signed CSL translation

WE GO LOOK-FOR ONE LIGHT

WALK+ SHADOW-DISAPPEARING
LOOK-FOR LIGHT-GAZE

IT-IS-DARKENING FLOWER
LOOK
HOUSE WINDOW SHINE
FLOWER-BLOOM-MERRILY GRATEFUL

WALK+ SHADOW-DISAPPEARING
LOOK-FOR LIGHT-GAZE

TWILIGHT
WILD-GRASS-BLOWN-BY-THE-WIND
TRAIN-RUNNING+
HEART-WARM-WEARISOME-MEMORY
LOOK
TRAIN-CARRYING-AWAY-MY-PAIN

WALK+ SHADOW-DISAPPEARING
LOOK-FOR LIGHT-GAZE

SEA SUNRISE-OVER-SEA
SHINE-ALL-OVER
LOOK
CHILDREN-BOUNCING
HAPPY+ BEAUTIFUL COME-ON

WALK+ SHADOW-DISAPPEARING
LOOK-FOR LIGHT-GAZE

Disclaimer: The author is responsible for obtaining permission to use any copyrighted material contained in this article.

Biography

Hao Lin is an assistant professor at the Institute of Linguistics at Shanghai International Studies University, China. He holds a PhD in Applied Linguistics, specialising in Chinese Sign Language in Fudan University. His research interests include sign language linguistics, sign language culture studies, sign language translation, and education. He also works part-time as a sign language interpreter.

E-mail: linhao@shisu.edu.cn

Notes

Note 1:
These statistics are taken from the official website of the China Disabled Persons’ Federation (CDPF), the government-funded organisation for deaf affairs (2019).
Return to this point in the text

Note 2:
Since CSL poetry is still in its infancy, and is often ignored by the mainstream, recital recordings emerged recently as smartphones began to become more popular and cheaper. There are only a few CSL poets and poems circulating in the public domain or on the Internet. Further, some social media platforms are blocked in China, including YouTube, Facebook, etc., and therefore such recordings are almost impossible to access, unless a user can connect to such platforms through a VPN. Most deaf people in China do not have the technical skills to use a VPN, and therefore cannot upload or access videos of CSL poems.
Return to this point in the text

Home | Past Issues | Keyword Search | Editorial Board | Events | About | Contact | Help