In this post, I’d like to suggest that Jeremy Harmer’s public pronouncements on testing fail to say anything new or interesting and demonstrate a regrettable lack of knowledge of the matters discussed.
At the 2015 IATEFL conference, Harmer gave a talk which you can see by clicking its title here: An uncertain and approximate business? Why teachers should love testing. Harmer’s basic thesis is that teachers should love testing because it’s a necessary part of their job, but it’s an opinion that’s brashly asserted rather than a proposition that’s reasonably argued. Harmer begins by listing objections to testing:
- Tests don’t measure creativity.
- Chomsky says “testing is an anathema.”
- Some people on Facebook don’t like testing.
- Testing 4 year olds is weird.
- Testing is only a snapshot.
- Some people are good at testing, some aren’t.
Note that none of these points is developed and that no coherent argument is attempted.
Harmer then gives reasons why teachers should love testing:
- He got a Grade 1 in playing the tuba because there was a test, and he performed badly in a concert because there wasn’t a test. Testing is thus a powerful motivator.
- Neurosurgeons and pilots must be tested. So we need tests.
- Tests tell us where students are. ”A test if it’s well done will tell you how well your students have done.”
- Tests are getting better. “The Pearson test of academic English is bloody wonderful. I’m saying that because I believe it, not just because they pay me.” The designers claim that their speech-recognition software evaluates speech “as reliably and accurately as any human being can. And I have no reason to doubt that, because the research behind it is er.., er.., massive.”
- Lots of tests are bad. If you want to change testing you can moan or do something; so learn about tests and do something. .
I suggest that this talk makes no worthwhile contribution to our understanding of language testing and that we should expect more than this from a presentation held at prime time in the biggest room in the entire conference centre, and streamed live on the conference website.
Harmer’s second presentation was a videoconference given as part of of the 2015 TOBELTA Online Conference. This is something of a volte-face, since here Harmer asks “Should teachers love tests or hate them?” and begins by confiding that the question is so knotty that it drives him “to schizophrenia”.
Harmer devotes the first 20 minutes of his talk to saying that while he agrees with Luke Meddings that testing is badly-affected by big business, and that the commodification of language is a bad thing, he still thinks that neurosurgeons and pilots should be properly assessed. Harmer spends the rest of the hour variously stating the view that teachers need to become “test literate” experts in the field of testing. At one point Harmer says that teachers need to know about concepts of validity, reliability, and test item types, and at another point he says that knowledge of the two “profound concepts” of content validity and construct validity is vital if teachers are to “get inside the test.”
One other point that can be identified in the talk is that teachers and students should explore testing together. Students and teachers should “discuss together what it is they need to do and want to do with the full understanding of how a test works.”
How do you stop a huge corporation dominating the testing world? How do you stop tests being designed that are absurd and ridiculous? And, guess what? I have no easy answer to that… but I know perfectly well that there’s no merit in, or virtue in complaining about this in private, and, by the way, I say this absolutely genuinely, the reason why listening to Luke and others is so important is that it was not a private event, it was a public event and the more of us who are public about what we think, the greater the opportunity is that, er, things might change.
In evidence here is Harmer’s hallmark combination of gushing sincerity and jibberish.
I suggest that we should expect more than these empty words from an invited speaker at an international conference. The presentation is poorly-structured, seriously lacking in coherence and cohesion, and very low in substantial content. Watching the video, it becomes clear that Harmer doesn’t have a good grasp of even the basic vocabulary of testing, and that he’s unable to offer anything informative or well-considered to a discussion of the uses and abuses of language testing.
Finally, Harmer’s website offers some thoughts on testing in the post Testophile or Testophobe? I leave it the viewer to decide on its merits.
To seriously address the question of the pros and cons of language testing, we have to look not just at how tests are designed, but at how they’re used, an area which Harmer hardly mentions. I would argue that classroom teaching should be 100% test-free, while the use of large-scale tests should be carefully restricted. As Fulcher (2009) says, large-scale testing is a social tool used to ration limited resources and opportunities, and it’s currently being used “to carry a larger social burden than it can reasonably bear.” It should not, for example, as Fulcher (2011) argues, be used to implement immigration policies, to evaluate teachers, or to rank order schools. A place remains for testing, of course, but, on the basis of his latest offerings, Harmer is unlikely to be of much help in deciding what that place might be.
The 2 references to Fulcher can be found on Glenn Fulcher’s excellent website here: http://languagetesting.info/gf/glennfulcher.php Scroll down 4 pages till you come to “Selected Papers”. I particularly recommend his 2009 article “Test use and political philosophy” which you can download from his website. At the end of the article, Fulcher proposes an “effect-driven test architecture” which he hopes can serve “as a method for testers to proscribe unintended uses of their tests.”