Coherence part 3: what did the students tell us?

Neil Allison presents part 3 of his 3 part investigation to uncover more about the problems of poor coherence in student writing, particularly East Asian writers. This post summarises the experiments/tasks he used in class and the observations and implications for student awareness of and improvement in writing coherent texts.  

You’ll recall, if you read my previous 2 posts on Coherence (Part 1 and Part 2), that I have been puzzled for some time about the ‘logical’ structuring of ideas in east Asian student writing in English. I mentioned at the beginning of post 2 on coherence that in order to solve the puzzle I was following (slightly clumsily) a sort of action research process of: plan – act – develop – reflect. Here’s a bit more information:

Plan: in basic terms, a kind of puzzle identified from observing hundreds of Chinese English learners’ academic writing where their level is 5 to 6.5 i.e. the type of students I encounter most on pre-sessional or in-sessional University courses. However, I didn’t read the literature at this stage (more on this in a minute).

Act: carry out an experiment in my lab (classroom) on the guinea pigs (the students), being careful to ensure no one is harmed. Then, read the literature.

Develop: Based on a comparison of findings between the two sources of information in the ‘action’ stage above, develop an approach to teaching or improving student coherence in academic writing.

Reflect: ongoing!

OK, so why no literature review during the planning stage? Two reasons: The first one will resonate with most of you, readers:-

  1. On a busy teaching load it’s extremely difficult for university tutors/teachers to take a highly structured approach to any of their research. It’s possible, but my feeling is that you’d be at it so long, the moment would be lost or you’d move onto something else before you completed it.
  1. Kolb’s learning process and the styles developed by Honey and Mumford lead me to label myself (if required) in the ‘accommodating’ category, meaning I’m the kind of person that tries to build things without instructions and learn the hard way.

Let’s look at what I actually did and what I found out. If you’re in a hurry, jump to the bold statements at the end of each activity, entitled ‘general comments and coherence remarks’. For succinctness I’ll skim over the experiment design, which was slightly ad hoc but made some attempt to isolate independent and controlled variables, so the dependent variable – COHERENCE – was measurable. This attempt wasn’t particularly pure though because I allowed other things to drift in, so that I ended up with a lot of material for further research (primary and secondary).

Question/puzzle:

  1. What seems to cause poor coherence in Chinese students’ writing

Other questions:

  1. How best to explain to students what coherence actually means and why it’s important?
  2. How interesting for students are the activities that are supposed to improve coherence?
  3. Do students think their writing coherence is improving?

For Development:

  1. What can the above tell me so that the next writing course I teach can more efficiently train students in improved coherence, assuming it is even possible to train such a thing.

The experiment

Here’s what I did. I’ll arrange this as follows: Brief summary of what I did in class (without any theory or justifications; feel free to ask about that if you want); key finding.

The class, by the way, had 12 students: 7 Chinese, 1 Japanese, 4 Arabic speakers.


Activity 1 (week 2 of the course)

  1. Students did reading on learning styles in English. They read a text in their student book and also similar info from a website.
  2. Students wrote on the same topic in L1 (inspired by statement by student in class that he would be able to explain to another Chinese student what the English texts were about in Chinese, but not in English). They were to bring these L1 texts to the next class.
  3. Next class, we looked at linking words and how they related to coherence and cohesion in IELTS assessment and then students identified their use of linking words (adverbs, prepositional phrases e.g. however, in terms of) in their L1 texts.
  4. We counted the linkers in their texts. In 100 word texts, average number was 6 though just Chinese it was less than 5 average.
  5. Students considered the meaning of these discourse markers (translation to English) and whether
    1. there were a variety of options in their language as with English e.g. but/however/whereas;
    2. there was a difference in ‘register/style’ between these options.
  6. We discovered that students weren’t really able to see a distinction for register/style in their translation media e.g. between consequently and so.
  7. We used wordandphrase.info to consider formality/academic style
  8. Students translated all their text into English.
  9. We highlighted examples of informal grammar that came out of the translation.
  10. I used socrative to gather feedback on the activity. 11/12 enjoyed translating and 12/12 thought it was useful.

General Comments:

Students enjoyed writing in their language and translating. They also thought it was useful. Reasons for it being useful were thinking about formality and risks with translation. Some thought translation was interesting and useful because they were getting writing practice.

Coherence remarks:

Nothing major. Linking words seem to be used frequently in their own languages.  Really, this activity became an academic register comparison with L1 domain focused on linkers (cohesion).


Activity 2 (week 3)

  1. Students read text about bees.
  2. I gave students an outline (basic notes) of the bee text (to control the type of information they took from the text).
  3. Students wrote a paragraph based on the notes but translated (into their language) and made it as ‘academic’ and well written as possible.
  4. I presented an example of theme and rheme (and constant and linear progression) and topic sentences (to be first sentence of paragraph).
  5. Students identified theme rheme (known – new) and topic sentences in their translated (to L1) texts to compare the pattern in their text with the presented one.
  6. I asked students on socrative if they had an idea how to take ideas from this lesson to improve their writing. I gave them the following options
    1. I don’t need to change my writing process/approach
    2. I need to change my approach and the best option would be
      1. I should plan my writing in detail, making notes first and at that stage thinking about coherence
      2. I should write as normal but then review each paragraph for theme and rheme
      3. I should write/think in my language first, then translate
      4. Something else

General Comments:

Students enjoyed translating (10/12).   7/12 students thought that reviewing theme/rheme and topic sentences after they’d written something was the best thing for them to improve coherence. Other felt they have no particular issue with coherence.

Coherence remarks:

I imagined that thinking first in L1 might improve coherence as it would reduce demands on short term memory/reduce levels of cognitive load. Sadly, I can’t judge this based on one experiment and would possibly want to try this activity with different groups, sometimes thinking in L1, sometimes L2.


Activity 3 (week 3)

  1. Students wrote a fairly controlled text: data description with speculation. It was controlled in the sense that the first part was describing two items in the data, the second part we brainstormed on the board so they can chose key points/ideas from there.
  2. After writing, revision/ presentation on theme – rheme
  3. Students placed a ‘C’ above any part of their text where they were concerned they hadn’t followed the thematic progressions constant or linear.

Coherence remarks:

Students often put C’s in places where there were no coherence issues and generally coherence was fine.  Perhaps this was because the text was pretty controlled, so cognitive challenges were less and lower demand on working memory.


Activity 4 (week 4)

Its aim was to test whether higher cognitive challenge saw decrease in coherence.

  1. Students wrote an essay at home on effect of technology on young people.
  2. I selected two Chinese students (Student Tea and Student Coffee) for follow-up interview because both these students had weaker coherence from classroom activity 3 and an earlier essay task prior to any of these activities, and also because one student, Tea, now had good coherence, the other, Coffee, still not.
  3. I interviewed these students.

Coherence remarks:

Tea had good coherence. She brainstormed and thought in Chinese, then wrote in English. She spent 1 hour. Student Coffee spent 40 minutes, thought and wrote in English. Neither student reviewed their texts for coherence or even gave it explicit thought. Neither did a detailed plan/outline.

The biggest issue with Coffee text was poor topic sentences. We discussed whether topic sentence should be a detail/more specific than the rest of the paragraph. Both she and I thought it should be more general. However, we disagreed what her ‘topic sentence’ was: I thought it was more specific than the second sentence, she disagreed!


Activity 5 (week 6)

I gave a writing task that a) was conceptually complicated and b) was brainstormed and written without translations (no use of L1 in brainstorming or writing, although dictionaries used in actual drafting stage) c) written under time pressure i.e. increase pressure and cognitive load.

  1. Students brainstormed on a complicated essay topic.
  2. Students wrote paragraph 2 i.e. body paragraph 1.
  3. Students put under time pressure: 15 minutes to write their paragraph
  4. Students wrote again on separate pieces of paper each of their sentences but removing any discourse markers and replacing any referencing and substitution with repetition. Repetition to be highlighted, underlined, so that later it was easier to follow theme rheme, though focus for this activity was mostly topic sentences.
  5. Essay passed in pieces to partner who put them in order. Then check with writer if it was correct.
  6. Investigate whether there is agreement on the topic sentences.
  7. Feedback via socrative.

General comment:

10/12 students enjoyed the activity, one neutral, one negative. The neutral one felt they didn’t learn anything new and the negative one had become more confused.


Activity 6 (week 8)

Students asked to make notes to help define coherence 6/12 were able to do so adequately i.e. including theme/rheme; order of ideas; topic sentences. 3 were unable to do anything. 3 were confused with cohesion and wrote phrases such as ‘discourse markers’, ‘linkers’, ‘pronouns’.

Coherence remarks:

3 of the Chinese students were slightly (or very) confused about suitable topic sentences – they often selected something that was too detailed.


Activity 7 (week 9)

Students produced a timed writing that they had the opportunity to brainstorm on at home. Topic was There are many ethical and legal issues associated with information technology and the use of computers. Briefly summarise a range of these issues and then outline one specific issue and evaluate the impact it has on different stakeholders. You must focus only on 1 context for this essay e.g. a particular type of workplace (hospital, government agency, law firm, medical research laboratory etc), university, school.

Students were given the macro structure so the remaining independent variables were writing under pressure and producing grammar and vocabulary. Students were asked to put a ‘C’ next to sections of their completed essay that had C&C concerns about and give a key word or phrase e.g. topic sentence, linker, etc.

Coherence remarks:

Students put lots of C’s but mostly related to topic sentences. Chinese students (5/7) said they were not confident their topic sentences were good. All students produced essays that showed good coherence.


Conclusion

This is where we have to cross reference with the literature review, which I summarised in my 2nd post of coherence a few weeks ago.

Although I learned from these experiments that Chinese students were often confused about topic sentences, they appeared to benefit from a variety of activities that asked them to think about them either before or after drafting. I also learned that it takes a lot of repetition of coherence related activities to embed an awareness and understanding. Thirdly, I learned that explicit teaching of thematic progression appears to speed up the process of writing coherently but that I need to experiment and read more on paragraph functions and topic sentences in Chinese texts. Finally, I learned that thinking in L1 or translating may be a strategy students can employ to reduce cognitive load and improve coherence. I imagine this last ‘conclusion’ will prove the most controversial and superficially shaky, but they certainly seem to like it!


References

HONEY P. and MUMFORD A.  Manual of Learning Styles,  London: P Honey 1982.

 

 

Advertisements

4 thoughts on “Coherence part 3: what did the students tell us?

  1. Olga

    Neil, thank you for a very interesting and detailed description of the ‘experiment’ you conducted. I particularly liked your involvement of the translation and language comparison. Although you refer to your final concluding remark (‘Finally, I learned that thinking in L1 or translating may be a strategy students can employ to reduce cognitive load and improve coherence’) as ‘superficially shaky’, this is a very important observation, and it is not ‘shaky’ at all. To the contrary, it is well supported by extensive research. The first language is a very rich resource and needs to be viewed as a ‘friend’ rather than the ‘foe’; it is unfortunate that the significance of language comparison is frequently overlooked.

    I think you arrived at a very noteworthy point when you prompted your students to acknowledge that there is coherence to be found in their own language. I tend to believe that ‘coherence’ is not an invention of the English-speaking world and that there is coherence of expression in other literary and linguistic realms. This partially addresses your query: is the ‘logical’ structuring of ideas in East Asian student writing in English different from the ‘logical’ structuring of ideas in English speakers’ writing? My experience of teaching students from different linguistic backgrounds (and this includes English-speakers) indicates that coherence is a common problem and that the lack of coherence is not, by any means, more pronounced in writing produced by East Asian students. Production of a coherent text is a skill and, like any skill, it requires development.

    I will share a few thoughts on how we can approach the process of developing the skill. Prior to that, I would like to refer to Graeme’s response to your first entry (1 of 3 in this series of posts) where he notes that ‘cohesion’ is easy to define and ‘coherence’ is not so easy to define. Indeed, cohesion can be described in simple terms as ‘links among surface elements’. The examples you use in your blog entry 3 (whereas, consequently, however) help accomplish cohesion between the functional units of language; they do not necessarily assist in the production of a coherent text. Yet, Graeme’s rendering of the concept of coherence as ‘a guarantee to the reader that there is an underlying order in your text’ is rather appealing, and the following is my attempt to delineate some properties of this underlying order.

    This is the understanding I try to instil in my students:
    (1) The importance of writer-reader relationship. Writers and readers do not meet. This means that you will not have a chance to explain what you REALLY meant. Your only chance to make it clear to the reader is to put in writing HERE AND NOW, in this text.

    (2) Academic essay is not a detective story. Nothing should be hidden from the reader. The reader should know at every single point (from the very beginning) who does what, how, where, when and why.

    (3) There is ONE CENTRE in your writing. This centre is your argument and everything you write helps you develop and support this argument.

    I also resort to, what I call, ‘Stanislavsky method’ or, to put it simply, ‘I don’t believe it’ method.

    Footnote: Konstantin Stanislavsky (1863 – 1938) was a prominent actor and theatre
    director who created a system of acting that has received worldwide acclaim. At every
    rehearsal Stanislavsky kept repeating “I don’t believe it” in order to make the actors ‘live
    through’ their roles and make them realistic, or believable.

    The students who undergo training under my Stanislavsky method know that they need to make their claims believable. This requires explanation of everything they commit to their text (e.g. this can be illustrated by an example…, the example helps understand why…, this means that…, presented evidence indicates that…). What comes out as believable is usually coherent.

    I will not elaborate on specific techniques I employ (e.g. tracking the route of the argument with a coloured highlighter, designing a symmetrical Persian carpet pattern made of key points of the essay, and so on). This brief response is no match to your very thorough record of activities you undertook in the process of your ‘experiment’. So, I hope we can continue sharing our experiences outside this blog entry.

    Like

  2. Hi Olga. Thanks for your reply.

    It’s funny but I’ve said much the same thing about ‘detective story’ (or a mystery, or magical realism, or whatever students might understand i.e. that we, the readers, are not supposed to be guessing or on the edge of our seats waiting for the point or working out the relationship between points).

    I think if I’ve learned anything particularly corporeal from my experiments, it’s not to be afraid of using translation in class. Hopefully it’s reasonably clear above that the main reason, in the beginning, for using translation was to monitor the effects of cognitive load, but that by the end of the course, I had the impression that it can be 1. fun 2. motivating and 3. useful for awareness raising, apart from the more obvious points such as the fact that we all know students do it whether we forbid it, encourage caution, encourage against it, whatever.

    Like

  3. Steve McGill

    Just a quick note of support for Neil’s conclusion that L1 usage has an important role to play in this kind of learning process. I have absolutely no problem with this conclusion and it has informed a great deal of my teaching for many years now. This in fact also links nicely with Neil’s concerns about when (and indeed if) it is necessary or important to engage with the literature since the literature on L1 usage goes back at least to the 1980’s and appears to clearly support the view that it is a valuable resource which can make a significant contribution to learning if used appropriately. Early examples would include Auerbach (1933), Atkinson (1987) and Hopkins (1988) but the literature has progressively come down in favor of L1 use up to the present day (Cook, 2001; Mart, 2013). Schweers (1999) pointed out that this position is often seen as something close to heretical within EFL circles while Atkinson and Cook both point out that in fact the ‘authorised’ position never had any firm foundation in research or anything beyond the application of some kind of ‘common sense’ dogma. All of this would seem to suggest that engagement with the literature should possibly be the first thing on the list and might usefully inform subsequent methodology.

    Auerbach, E. (1993) Reexaming English only in the ESL classroom. TESOL Quarterly, 27, 1, pp. 9–32.
    Atkinson, D. (1987) The mother tongue in the classroom: A neglected resource? ETL Journal, 41,4, pp. 241–247.
    Cook, V. (2001) Using the First Language in the Classroom. The Canadian Modern Language Review, 57(3), 402-423.
    Hopkins, S. (1988) Use of mother tongue in teaching of English as a second language to adults. Language Issues, 2, 2, pp. 18–24.
    Mart, C. T. (2013) The Facilitating Role of L1 in ESL Classes. International Journal of Academic Research in Business and Social Sciences. 3(1). pp. 9 – 14.
    Schweers, C. W. (1999) Using L1 in the L2 Classroom. English Teaching Forum. 37(2). pp. 6 – 13.

    Like

    1. Thanks for that Steve. I’m not entirely sure how we got to the position in much of the EFL where translation or use of L1 became overlooked, ignored, banished, or whatever but speaking for myself, I used it very very rarely until the last couple of years of my teaching. In fact I’m still cautious in its use but try to provide some for reading and writing classes periodically.

      Your point about reading ‘the literature’ links somewhat to Riccardo’s earlier post about scholarship:
      https://englishforacademicstudy.wordpress.com/2016/02/29/sotl-when-scholarship-becomes-research/

      and perhaps can serve as a reminder that in the position many EAP teachers are in, we need to be a bit more creative about fitting in the literature and being realistic about the short term outcomes of that literature ‘review’. Actually, there’s a scholarship opportunity right there: designing a new template methodology for EAP teachers to be scholars: methodology that works in a loop (observe, read, dabble, observe, read, dabble, read (the good stuff), experiment, observe, publish) and not only the loop approach but one that keeps the scholar interested by having lots of mini-outputs, most likely in the class. That’s actually what I thought was most satisfying about my experimenting (above) – dabbling with new classroom tasks that I was unashamed to seek feedback on and were always getting good feedback.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s