Two Very Different Plenaries: Norris and Thornbury

The Seventh International Conference on Task-Based Language Teaching, held at the University of Barcelona, kicked off on Wednesday 19th April with a splendid, critically acute plenary by John Norris. Norris examined a number of meta-analyses on TBLT and gave a “synthesis of syntheses” which combined a succinct, well-organised and well-presented summary of research findings with an insightful discussion of their implications for teachers. Very good, nourishing stuff!

(Note: I rely on Twitter feeds and 3 eye witnesses for this account.) The ELT Research In Action conference, also held at the University of Barcelona, spluttered into life on Friday 21st April with a lack-lustre, re-hashed, badly-considered plenary by Scott Thornbury. Thornbury asked how methodology writers mediate between researchers and practicing teachers. To “explore” this question, he asked four writers of teaching methodology texts to complete a short, poorly-designed questionnaire, thus collecting a meagre amount of vague data from which a few unremarkable yet dubious conclusions were drawn. The flimsiest “study” I’ve ever seen presented in a plenary shone the dimmest of lights on an important issue, and through the gloom one could just discern Thornbury giving an unexplained thumbs up to the motley views of his puny collection of participants. Very thin soup!

Norris on TBLT

I’m waiting for the slides of Norris’ plenary to arrive, and so I won’t attempt to give a summary of what he said (but I promise that I will soon). Instead, I’d like to sketch the background to his talk and then comment on how he went about it, because, in stark contrast to Thornbury, Norris gave a demonstration of focus, scholarship, critical acumen, and the ability to present and analyse complex data.  It was well-conceived, well-organised, well-considered, and well-executed.

The talk reflects an increased interest in meta-analysis, which was very much “flavour of the conference” , with a surprising number of presentations examining different meta analyses of TBLT research. Meta analyses gather together a collection of published papers on a particular area and synthesise the results. Given the growing number of published studies on various aspects of TBLT, it’s not surprising that increased attention is being paid to ways of accurately synthesising results. The Journal Language Learning is good at publishing meta analyses, and Luke Plonsky has a good review of Meta-analysis in Applied Linguistics.

When it comes to studying TBLT, as Skehan (2016) makes clear, you can take an inductive or deductive approach. Regarding the former, task features are identified and then explored in terms of their relationship with performance, which is typically judged by measures of complexity, accuracy, lexis, and fluency (CALF). For example, Foster and Skehan (1996) contrasted the effects of planning with personal, narrative, and decision-making tasks. The study indicated specific connections between personal tasks and both increased accuracy and increased complexity, and between narrative and decision-making tasks and increased structural complexity.

In contrast, Robinson’s (2011) cognition hypothesis is an example of a deductive approach. Robinson (2011) argues for a principled sequencing of tasks in a syllabus, from less to more complex tasks, so as to help L2 development. To that end he uses the construct of “task complexity” and proposes a distinction between resource-directing factors and resource-dispersing factors, with each of these overarching categories then leading to specific performance predictions. Resource-directing variables make cognitive conceptual demands (e.g. + intentional reasoning) which direct learner attention and effort at conceptualization in ways which the linguistic L2 system can help them meet. In contrast, resource-dispersing variables make performative procedural demands which increase task complexity but without directing learner attention and effort at conceptualization to any particular aspects of language code (e.g. – planning time). Just as an example, resource-directing tasks are likely to give more opportunities for focus on form.

Notice how many terms and constructs are in use here. To simplify, we can say that tasks are seen as complex or simple and that performance is judged in terms of CALF measurements. Then, we can distinguish between Robinson’s daring hypotheses (where the distinction between resource-directing factors and resource-dispersing factors is vital) and the more cautious inductive work of Skehan & Foster and others, where, particularly for Skehan, task conditions are key. Skehan claims that the more deductively motivated studies have tended to generate results that focus on task characteristics, but the results are not consistent and don’t give clear support to the hypothesis. In contrast, he says, the more inductively oriented studies, whether of tasks or conditions, have tended to produce more consistent results.

In brief, when we look at TBLT, if we want to assess the efficaciousness of a task or a sequence of tasks, we have to agree on measurements of key factors (CALF); articulate constructs such as task complexity; clarify what we mean by “complex”; and “task difficulty”, for example; pay more attention to task conditions; distinguish between performance and learning (immediate and delayed); agree on the size of Cohen’s d factor required to calibrate results; and other things besides. And then we have to agree on procedures for putting together meta analyses. All of which means that some TBLT studies are better than others, that there are some contradictory results, and that there are disagreements among those working in the field. Nevertheless, precisely thanks to the demands for clarity, common terms, rigour, etc., progress is being made, and, as Norris showed, meta analyses are a good way of highlighting that progress.

Norris was talking to an audience of academics: they all knew about Robinson’s hypothesis, and task complexity, and CALF and Cohen’s d measurement of effect, and all that and all that. But I repeat, it’s the conception, organisation and execution of the plenary that’s worthy of comment. What Norris did in his talk was to take a well-focused question (What does research tell us about the factors that influence the outcomes of different kinds of pedagogic tasks?), and then present the hugely complex and conflicting evidence in an unusually clear, well-organised way, which included his own well-argued analysis.

  • This, he said, is a synthesis of what research is going on in TBLT.
  • These are the measurements and terms and constructs that we use.
  • This is what we’ve found.
  • These are the limitations.
  • This is what we still don’t know.
  • This is what we should pay more attention to.
  • This is the pay off. If we further clarify our terms and methods, we’ll find out more about the effects of different types of task on performance and learning, and that will have important implications for syllabus design and teaching practice. We’ll be better able to provide an evidence-based and well-argued case for what works and what doesn’t in TBLT, and to give better advice to course designers, materials writers and to teachers.

Let’s look now at how Thornbury gave his plenary.

Scott Thornbury’s view from the bridge

Thornbury used the same presentation slides as those we saw at the IATEFL conference (you can get a pdf file of them at this link.). As I’ve already said, I wasn’t there on Friday, but I’m using Jessica Mackay’s report and the notes of 2 members of the audience to help me, and as far as I can judge, nothing much new was said the second time round.

So, having noted that most ELT practitioners don’t read about research in their field, Thornbury wonders whether those who write methodology books about ELT see it as part of their job to take responsibility for bridging the gap between researchers and teachers. This leads to his main research question:

“How do methodology writers mediate … between researchers and practitioners?”

To find out, he sent a questionnaire to 4 authors of books on teaching methodology. Here are the questions:

  1. How did you get into writing methodology texts?
  2. How important is it, do you think, to link research and classroom practice?
  3. How have you kept/do you keep abreast of new developments in research, e.g. SLA, corpus linguistics, neurobiology etc?
  4. Given that most research is somewhat inconclusive, how do you select from – and prioritize – the research findings that inform your texts?
  5. Do you feel you have an ‘agenda’, i.e. a bias towards a particular theoretical (or a-theoretical) position? If so, do you think this matters?
  6. If not (or even if so) do you attempt to be balanced/impartial/non-prescriptive? How do you achieve this?
  7. Does it concern you that you might be ‘dumbing down’ or otherwise misrepresenting research findings? How do you guard against this?
  8. To what do you attribute your success? (Don’t be modest!)

Note that Q.1 and Q.8 are irrelevant to the question, and that Q.5 and Q.6 are poorly articulated, overlap and in fact elicited almost no relevant data (check the pdf, file for yourself). At the IATEFL conference, Q.7 wasn’t even mentioned, but perhaps it was dealt with this time.

So the “findings” of Thornbury’s study consist of 4 people’s short responses  to 3 or 4 questions.

Here are their answers to Q.2:

  1. DH: Imperative! Teachers need to ground their teaching in research-based findings and assumptions. And, more importantly, teachers themselves should not shrink from engaging in their own classroom-based “action research.” It’s an all-important interaction.
  2. JH: I simply fail to understand people who deny the role of research in helping us understand our practice and improve it. Research is, after all, what all good teachers would do if they had the chance.
  3. PU: It’s sometimes a useful support and can provide interesting insights, but it’s certainly possible to write helpful and valid professional guidance for teachers with no research references whatsoever.
  4. JS: I’ve never found much formal “research” very helpful to my own classroom work. I am not “anti-research” but I do carry a suspicion of many statistical studies in teaching. … I more often look at the literature to see if it can help me understand what I have already noticed myself.

And Q.3

  1. DB I do university courses myself and I read state of the art articles.
  2. PU I’m sure I miss a lot but really important things get cited in stuff that I read, so I get most of the major stuff.
  3. JS Twitter has alerted me to interesting articles and websites.
  4. JH: Teachers journals, published books etc -though I fear that I do not have enough time to do as much of that as I should. […] The large number of teachers’ conferences and seminars that I attend […] News media, magazines and, increasingly, social media where news about new research often breaks.

And Q.4

  1. JS: Mainly, I think I write what I do and what I see other teachers doing. Informed ideas that may or may not work for others. These need to fit in with my own internal schema for how I think people learn, study, behave etc.
  2. JH: I go for what seems plausible to me. But I have to be careful (and suspicious) of my own unreliable instinct … There IS an element of fashion in this too, of course. Readers of a general methodology book need to know what is most ‘current’.
  3. PU: One criterion is, obviously, that I feel the research is reliable –well-designed and carefully executed, with convincing evidence and logical conclusions. Another is that it’s not on a trivial or very limited subject.
  4. DB: The selection of findings to inform my writing is based on degrees of (1) validity through triangulation of findings, (2) relevance of findings to pedagogy, and (3) practicality of those findings for classroom teachers.

From this, Thornbury summarises his findings:

  1. Methodology writers have an interest in keeping abreast of developments in research, but largely as filtered through their own experience and ‘sense of plausibility’.
  2. Methodology writers use research findings less to promote new practices than to validate existing ones.
  3. Methodology writers are sensitive to, and respectful of, prevailing trends, while, at the same time recognizing their inherent weaknesses.
  4. Methodology writing is not ‘applying linguistics’ so much as ‘particularizing theory’.
  5. Methodology writers present options rather than prescriptions.
  6. Methodology writers adopt a voice that is non- academic and practitioner-oriented.

In his abstract, Thornbury promised “to draw some principles that might guide others – such as teacher educators – who [play a similar bridging function]”.  Unless my informants misled me, the guiding principles were not actually “drawn”, rather, Thornbury simply recommended others to follow the good example set by his participants.

A Few Comments: 

  • On the basis of the data from 4 participants, Thornbury is not entitled to make any generalisations whatsoever about what methodology writers believe or do.
  • Mind you, given the vagueness of the 6 points in Thornbury’s summary, they could easily apply to everybody. It’s hardly surprising that his participants “have an interest” in keeping abreast of developments in research; or that they “try not to be biased”; or that they are “sensitive to, and respectful of, prevailing trends” (though I’m not sure there was much evidence of “recognizing their inherent weaknesses”); or that they use a non-academic style to talk to teachers; or even that they “present options not prescriptions”.
  • Even these blurred generalities come from a fairly liberal interpretation of the data. Thornbury smooths over the important differences among the 4, uses one single sentence in the whole data to support the generalisation that “Methodology writing is not ‘applying linguistics’ so much as ‘particularizing theory’” (whatever that might mean); and presents the 6 points as if they represented some fine distillation of the common wisdom of his participants.
  • Thornbury’s discussion of the answers given to Questions 2 and 4 is extraordinarily complacent. These are, in fact, the only interesting and significant glimpses we get into his participants’ beliefs and practices, and they suggest that at least two of the four writers care little for research findings, make scant use of them when writing their books, cherry pick the findings, and see nothing wrong with using research findings as a way to support their own hunches, feelings and beliefs about teaching. Thornbury offers no criticism whatsoever.

In brief, on the basis of the most cursory examination of a flimsy collection of data, Thornbury confidently concludes that methodology writers do a good job and that they serve as a model for others.

A Few Questions:

Why didn’t Thornbury ask a few more people to take part in his study?

Why didn’t he ask more focused questions about research? It’s mostly treated as some kind of homogenous lump, the only exception being when he refers to “new developments in SLA, corpus linguistics, neurobiology etc?” (BTW, “new development in neurobiology” is code for complexity theory; you know, like when a flock of birds part when approached by a predator and then they re-group. A new level of complexity arises, emerges, out of the interaction of the parts and a pattern emerges from the interaction of the parts. That’s just what happens in language learning. Not clear? Well, have you ever noticed how one bit of broccoli looks just like the whole plant? Cont. p.89 )

Why didn’t he ask them what they thought the most important research findings in instructed SLA research were and how teachers should be informed about them?

Why didn’t he check to see if there was any difference between what the participants said in reply to the questions and what they said in their books?

And what about his own case: Why didn’t he tell us about how he himself performs “the bridging function”?

  • How does he link research and classroom practice in his 4 books on how to teach grammar, and in his books on how to teach speaking, how to teach vocabulary, and how to get through the CELTA training course? How important does he think it is to act as a mediator between researchers and practising teachers? How well does he carry out this function?
  • How does he explain the contradictions between what he says in these books and what he says elsewhere about classroom practice based on teaching McNuggets?
  • How do his methodology books chime with what he tells MA students at the New School in New York about the classroom implications of Larsen-Freeman’s version of emergentism (grammar for free through the miracle of “Affordances” and other hogwash)?
  • How does he reconcile the advice he gives in these books with his passionate promotion of Dogme?

And finally, and crucially: Why didn’t he ask his participants if they knew about the robust findings that have emerged from 40 years of research into interlanguage development, which show that teaching can’t influence the route of L2 learning, and which thus challenge any methodology based on the presentation and practice of a succession of bits of grammar and lexis?

Which brings us to the two general questions:

  1. What should teachers know about research findings in applied linguistics?
  2. Who should tell them?

In reply to Q.1, first, they should know about interlanguage development; that’s without doubt the most important issue. All serious scholars of applied linguistics (including Larsen-Freeman last time I heard) agree that “the route of development (the nature of the stages all learners go through when acquiring the second language) remains largely independent of both the learner’s mother tongue (L1) and the context of learning (e.g. whether instructed in a classroom or acquired naturally by exposure)” (Myles,2009). If teachers really appreciated the implications of these findings, it could lead to a profound change in ELT.

Next, the concept of “focus on form” needs to be more generally understood. It’s generally agreed that second language teaching should be meaning-focused, but that it can be improved by some attention to form. This doesn’t mean a return to discrete-point grammar instruction, but that “focus on form…overtly draws students’ attention to linguistic elements as they arise incidentally in lessons whose overriding focus is on meaning or communication” (Long, 1991, pp. 45-46), and “focus on form often consists of an occasional shift of attention to linguistic code features–by the teacher and/or one or more students – triggered by perceived problems with comprehension or production” (Long & Robinson, 1998, p. 23).

Then there’s the work going on in the area of TBLT. As we’ve seen, it’s complex and complicated and “more research needs to be done” as they always say. But, nevertheless, information about the on going work can be made accessible to teachers and has the potential to greatly improve their practices.

Lots of other important work is also going on that can help pronunciation teaching, vocab. teaching, and teaching of the 4 skills.

Regarding Q.2, given that it’s completely unreasonable to expect most teachers to keep up with developments in the areas of research mentioned above, it’s obvious that methodology writers (among others) should act as mediators. These writers can’t be expected to read more than a fraction of the research, which is why reports of meta analyses are so useful. By keeping an eye on what’s going on in the journals, reading a few “state of the art” articles, and reading a few meta analyses, such writers can at least get a good general feel for what’s happening and then be in a position to dig deeper when necessary.

Pace Thornbury’s comfortable conclusions, it’s obvious that 3 out of 4 of his participants (not to mention him and various others) are not doing a very good job as mediators. If they were, there would be more widespread doubts expressed about the way ELT is currently practiced. Just for example, despite Harmer’s replies to Thornbury’s questions, his 500 page methodology book devotes more time to classroom seating arrangements than it does to discussions of how people learn an L2, and nowhere does it deal with the findings mentioned above. Douglas Brown comes out best, but even he fails to properly discuss the implications of the research findings on interlanguage  development.

As for Thornbury himself, his work continues to be inconsistent both in its quality and in its underlying principles. How can the man who slams “McNugget teaching”, who rails against coursebooks, who supports the mad rantings of Larsen-Freeman, and who proselytises the radical principles of Dogme, deliver this same threadbare talk twice in as many months? How can he claim that the 6 points he cobbled together from the limp replies he got from 4 people to a few poorly-articulated questions are sufficient evidence to persuade us that those “on the bridge” are doing a great job of mediating between researchers and practicing teachers?

References

Harmer, J. (2015) The Practice of English Language Teaching. London: Pearson. 

Long, M. (1991). Focus on form: A design feature in language teaching methodology. In K. de Bot, R. Ginsberg, & C. Kramsch (Eds.), Foreign language research in cross-cultural perspective (pp. 39-52). Amsterdam: John Benjamins.

Long, M., & Robinson, P. (1998). Focus on form: Theory, research, and practice. In C. Doughty & J. Williams (Eds.), Focus on form in classroom second language acquisition (pp. 15-63). Cambridge: Cambridge University Press.

Myles, F. (2009) Second language acquisition (SLA) research: its significance for learning and teaching issues. Downloadable here: https://www.llas.ac.uk/resources/gpg/421 Note: Good References and links to some very good resources.

Plonsky, L. (2016) Meta-Analysis in Applied Linguistics. See http://oak.ucc.nau.edu/ldp3/bib_metaanalysis.html

Skehan, P. (2016) Tasks Versus Conditions: Two Perspectives on Task Research and Their Implications for Pedagogy. Annual Review of Applied Linguistics, 36 pp. 34–49.

Skehan, P. & Foster, P. (2008) Complexity, accuracy, fluency and lexis in task-based performance: A meta-analysis of the Ealing research. In Van Daele, S., Housen, A., Kuiken, F., Pierrard, M. & Vedder, I. (Eds.). Complexity, Accuracy, and Fluency in Second Language Use, Learning, & Teaching. pp. 207-226. Brussels: Contact forum.

Thornbury’s books can be viewed here: http://www.scottthornbury.com/books.html

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s