Some Advice on Doing an MA Dissertation Part 1

Introduction

For an MA Applied Linguistics / TESOL you’ll be expected to write a substantial (15 – 20,000 word) piece of academic writing in which you:

  • Identify a clear focus and develop an area of interest in a coherent and rational way. This is a serious bit of academic work: it’s not journalism, or an opportunity to display your own prejudices and preconceptions!
  • Show a deep awareness and appreciation of other (academic) work done in the area, and have a point of view on it.
  • Demonstrate the ability to reflect critically on the area you explore.
  • Show you are able to collect, analyse and interpret empirical data (although a theoretical study is also possible).
  • Show you are aware of the implications of your findings, often, but not necessarily, for professional contexts.
  • Say something new – something that nobody has said before!

It normally involves a small-scale study but it can be literature-based, written predominantly as a result of reading and thinking. Universities prefer dissertations in MA TESOLs to have an applied perspective, wherever possible. By “applied perspective” most universities mean that the study should relate to a particular context and should draw on your professional experience. The application may be to the classroom, but this is not compulsory. For instance, a sociolinguistic investigation may have a wider application for a group of learners in a particular society. An SLA investigation could relate to a naturalistic setting, for example, a study of early child bilingualism. In each case, there may be implications for the TESOL classroom. As you have done with previous modules, you are expected to link theory to practical implications in a manner that is appropriate to your study. Dissertations should show in-depth acquaintance with the literature of the subject area, but should also be a vehicle for the expression of personal views.

I’m going to assume that you’ll do a small-scale study. More than 90% of all MA students do this and I strongly advise you to follow suit. If you want to base your dissertation on a literature review, you can do so, of course, but it’s really quite unusual and markers tend to be a lot more demanding of those dissertations where no study is involved.

 Deciding on the topic and the focus of the study

By the time you come to the dissertation part of the MA, you should already have some idea of what you want to do your research on. Play to your strengths; if you did a particularly good paper on any prior part of the course, then consider expanding on it, as long as you don’t use any parts of the text again. If you like grammar, or IT, or teaching young learners, or designing materials, or sociolinguistics, or whatever it is, then look there, and don’t think that you should do something you know nothing about, just to learn more

Maybe you already have a clear idea of what problem you want to explore, in which case, all you have to do is articulate the question and decide on the research tools. But if you’re still groping for a topic, then start with the “ball park”: what area, out of all those you’ve looked at so far in your studies, do you find most interesting? And inside this area, can you think of a problem, a knotty question, or a divise issue, or a weakness, that needs addressing?

Let’s say that you work in Spain teaching English in a primary school. You’re aware that results are poor: most Spanish children finish primary school with very little ability to communicate in English, as measured by results in the Cambridge PET and CET exams, for example. You believe that a contributing factor in these poor results is that most primary school teachers of English in Spain use a rather old-fashioned “PPP” (Presentation, Practice Production) approach to teach a grammar-based syllabus. A variety of issues present themselves here, but let’s say you focus in on the teachers themselves. Why are the teachers not embracing a more up-to-date, “Communicative Language Teaching” (CLT) approach?

Once you get this far, you need to consider your study.

  • What are your resources?
  • Can you use yourself as a teacher trainer or trainee in the study?
  • Can you ask teacher trainers and teacher trainees to fill in a questionnaire? How many?
  • Can you interview some of them?
  • Can you observe teacher training sessions?
  • Can you record them?
  • Can you get access to other data, like teacher journals?

Before you decide on the final Thesis Question, carefully consider what kind of data you can feasably collect in the time frame, and what your limitations are. Just for example, it’s unlikely that in 6 months you’ll be able to do a study where significant improvement in overall language proficiency is measured.

So, to return to your study, how can you investigate what you suspect is teachers’ resistance to CLT? Well, surely the best thing to do is to ask them. But ask them what, precisely? In order to make a questionnaire, and /or an interview template, you need to articulate an overall Thesis Question and then 2 or 3, more specific, research questions.

You could start with the question “Why don’t primary school English teachers in Spain adopt a CLT approach?”. For your own purposes here (and later because you’ll need to address the question more fully), you should be clear what you mean by “a CLT approach”. After that, you could generate these research questions:

  1. What are the perceptions of primary school English teachers in Spain of the CLT approach?
  2. What do primary school English teachers in Spain think are the obstacles to the implementation of the CLT approach?

No matter how you finally go about your study, you have to start by identifying a problem; not an interesting area, not a theme, not a subject, .. a problem! All good research starts with a problem. The problem then has to be articulated as a general Thesis Question, from which Research Questions follow.

These research questions must be formed in such a way that data collected in various ways can answer them. Finally, you have to decide what research tools you’ll use to gather data. It’s important to be clear about the distinction between qualitatitive and quantitative research.

Quantitative Research tries to be objective and is the preferred method of the hard sciences. It relies on empirical observation and it usually reduces the evidence from observations to numbers, focusing on counting and classifying features, and on constructing statistical models and figures to explain what is observed. It makes use of tools such as questionnaires, surveys, measurements and other equipment to collect numerical or measurable data.

Qualitative research, on the other hand, is primarily subjective; researchers often get subjectively immersed in the subject matter. It uses text rather than numerical data, and aims to give a complete, detailed description of the research topic. It is usually exploratory in nature. It uses data-gathering strategies such as individual in-depth interviews, structured and non-structured interviews, focus groups, narratives, content or documentary analysis, and participant or non-participant observation. The data is collected by the researcher himself or herself instead of through the inventories, questionnaires and other means used for quantitative research. This can be both a good thing and a bad thing. The researcher can gain an empathetic understanding of the behaviour of others, but the data collection is bound to be subjective and can even include bias.

For the last 15 years or so there’s been a lively discussion going on in the field of applied linguistics, and particularly in SLA research circles, about the best way to do research. The two opposing sides in this discussion can be seen as supporting predominantly quantitative or qualitative research programmes. In a special edition of TESOL Quarterly in 1996 devoted to ethnographic, subjective research in SLA, the contributors argued for a position totally opposed to the those who adopted a rationalist, empirical approach, and presented research papers that struck the rationalists they were attacking as being almost incomprehensible. In the Applied Linguistics journal, Block (1996) wrote a critique of the papers by Long, Beretta, Gregg, Crookes, and others which had appeared in a special issue of the journal devoted to theory construction in SLA (1993), suggesting that they were guilty of “science envy”, that there was no need to recognise accepted findings, no need for replication studies, and an urgent need for SLA researchers to throw off the oppressively restricting constraints of the “scientific” approach, so as to embrace a more relativist, more ethnographic approach.

Interesting though this debate is (I enthusiastically participated on the rationalists’ side), I don’t think it should bother you much as you prepare for your small study. Both types of research have their place in small studies, and very often quantitative data gathered from a questionnaire are combined with qualitative data from interviews. This is actually the path I will recommend.

Research Strategies

We usually distinguish between three research strategies:

  1. Experiment. Here we measure the effects of manipulating one variable (e.g. class size) upon another variable (e.g. results in an end of term proficiency test). The approach usually involves hypothesis testing.
  2. Survey. Here we concentrate on the collection of information in a standardised form from a group of people. The approach usually employs a questionnaire or structured interview.
  3. Case Study. Here we aim to assemble a detail picture, including in-depth insights, about a single ‘case’, or a small number of related ‘cases’. The approach may involve collection of information via a range of data collection techniques including observation, interview and documentary analysis.

Although these three strategies might look very different, in fact they don’t provide a logical partitioning covering all possible forms of enquiry, and there’s no need for you to agnonise about which category your study falls into. As I’ve already hinted above, I encourage you to use a hybrid strategy falling somewhere between these ideal types, e.g., some survey data combined with a small number of case studies, perhaps focusing on selected variables.   The survey provides a general, representative, picture; the case studies, chosen often on the basis of the survey, illuminate, enrich and bring to life the survey findings.

Questionnaires

By far the most popular and most frequently-used research tool for M.A. students is the questionnaire. A questionnaire is defined as “a self-report instrument used for gathering information about variables of interest to the investigator”. They are referred to as postal or email questionnaires. They consist of a number of questions or items which respondents answer by themselves. The questions or items can be structured or unstructured. That is, the categories of response may be specified or left unspecified. The key element in the self-completion questionnaire is that the researcher is not present when the questionnaire is being filled in. As Dornyei and Taguchi (2009) point out (Questionnaires in Second Language Research: Construction, Administration, and Processing, Routledge) questionnaires are used to gather three types of data:

  • factual (who the respondents are),
  • behavioural (what the respondents have done or are doing),
  • attitudinal (what the respondents think about things. In this, the most important category, Dornyei and Taguchi (2009) distinguish between attitudes, beliefs, opinions, inteterests and values.

The advantages of the questionnaire as a research tool are:

  • they make efficient use of time (the questionnaire can be completed by respondents in their own time and questionnaires consisting of closed questions allow for quick summaries and analysis of responses);
  • anonymity (the questionnaire provides the security of anonymity that few other research techniques offer);
  • standardised questions (in a questionnaire all respondents are presented with the same questions; there is no interviewer coming between the respondent and the question, but there is no scope for negotiating or clarifying the meaning of the question.

These are also the reasons why so much care should be taken drafting questions and why piloting is essential.

The limitations of questionnaires are:

  • they tend to lead to description rather than explanation
  • there is no interviewer to interpret or explain the meaning of questions, or to probe or explore answers
  • resulting information can be superficial. When designing a questionnaire :

These limitations can be overcome to somer extent by following a questionnaire with structured interviews to elicit further clarification.

When designing a questionnaire,

  • Keep it simple. Too many questionnaires are too long, too complicated, and too demanding of the respondent.
  • Make the questions specific. More general questions lead to a wider range of interpretations by respondents and are poorer predictors of behaviour. So, for example, the general question “What newspapers do you read?” should be replaced by the question “Which of the following newspapers have you read this week?).
  • Third, and this is by far the most important issue, use closed questions. Closed questions are often criticised because they force people to choose among stated alternatives rather than replying in their own words. But, precisely because closed questions give the response options, they are more specific and more likely to communicate the same frame of reference to all respondents. There will be times when open questions are preferable, of course, but if you use them,try to keep them to a minimum.

And when it comes to writing the questions, keep these ponts in mind:

  •  When in doubt, leave it out. When you finish the first draft of the questionnaire, go through it and ruthlessly discard any questions that are repetitive or redundant. Ask yourself “Is this question realy necessary?” Redundant items waste everyone’s time, mostly your own.
  • Be careful with the language level: questions have to be phrased in a way that matches the vocabulary of your respondents. Don’t use language that patronises or puzzles your respondents.
  • Be clear. Questions should be clear and unambiguous. Avoid double negatives and long-winded questions. Don’t ask double-barrelled questions. Categories of response should be clear so that the difference among categories is obvious to the respondents.

The Likert scale Questionnaire

The Likert scale questionnaire is the most widely-used questionnaire format among MA TESOL candidates. It was developed by Rensis Likert, a sociologist at the University of Michigan from 1946 to 1970 and is designed to measure attitudes. A typical item in a Likert scale questionnaire looks like this:

The president is doing a good job.       1  2  3   4  5

1 = Strongly disagree

2 = Somewhat disagree

3 = Neither agree or disagree

4 = Somewhat agree

5 = Strongly agree

To make a questionnaire that explores attitudes towards a defined construct (in this case, participants’ attitudes towards the president’s performance in office), you first generate items and then select from among them those that in pilots of the questionnaire proved to be valid, unidimensional (all measuring a common trait), and well discriminating.

The most important thing to note is that a proper Likert scale questionnaire measures the person’s attitude towards a given construct by combining (adding or averaging) their responses across ALL ITEMS. A Likert scale is NOT an individual item; it is always a set of several items, with specific format features, the responses to which are added or averaged to produce an overall score or measurement.

Many MA students make questionnaires using some response scale such as

1 = Strongly disagree

2 = Somewhat disagree

3 = Neither agree or disagree

4 = Somewhat agree

5= Strongly agree”

or

1 = Extremely Poor

2 = Below Average

3 = Average

4 = Above Average

5 = Excellent

using a variety of items that are do not explore the same topic. For example, an MA student used the 5-interval response scale “strongly disagree to strongly agree” in a questionnaire that asked teachers in Portugal to respond to 20 items which included

  • My boss is a good manager
  • My students are polite
  • My classroom is well-equipped.

While each item might say something about how satisfied the teacher is with his or her job, the sum of the responses to all 20 items can’t be used to measure anything very interesting.

So the summing or averaging across several items is essential for a true Likert scale questionnaire. In order to report on such a questionnaire, you sum the values of each selected option and create a score for each respondent. In the example above about an online software purchase, the scores can be from 4 to 20. This score is then used to represent a specific trait – satisfied or dissatisfied, for example – and to create a chart of the distribution of opinion across the population. For further analysis, you can cross tabulate the score mean with contributing factors. But it’s important to stress that for the score to have meaning, each item in the scale should be closely related to the same topic. I should add that the questionnaire can have different sections, each one a set of closely-related items.

Note several characteristics or features that define a Likert scale:

  1. The scale contains several items.
  2. Response levels are arranged horizontally.
  3. Response levels are anchored with consecutive integers (e.g. 1,2,3,4,5).
  4. Response levels are also anchored with verbal labels which connote more-or-less evenly-spaced gradations.

It is absolutely essential that you get a good understanding of Likert scale questionnaires, and of questionnaires in general. I recommend that you get a copy of Dornyei and Taguchi (2009) Questionnaires in Second Language Research: Construction, Administration, and Processing . This has everything you need andf more to design, write, administer and analyse a Likert scal questionnaire, or, indeed any questionnaire. Dornyei’s personal website http://www.zoltandornyei.co.uk/ is a great source for help on questionnaires.

Interviews

This is the second most popular tool used by MA students in their studies. They’re often used to back up questionnaire data, but they can be the main tool used. Interviews are commonly divided into three types

  1. Structured Interviews consist of a set of carefully-formulated of fixed questions which are both systematic and sequenced.
  2. Semi-structured interviews are much more open. Tthe inteviewer has some questions prepared on key issues, but feels free to follow other leads and to let the interviewee digress.
  3. Unstructured interviews, as the name suggests, are more like a conversation, where the interviewer goes with the flow. It’s quite likely that you will interview people you know, and if you do, you have to be careful to that this doesn’t affect or compromise your ability to remain impartial.

When conducting interviews, make sure your questions are clear and non-threatening ; don’t ask leading questions (ones that lead interviewees to respond in a particular way); show interest; and don’t speak too much – your job is to listen and probe.

Observation

In this MA, some students choose to do classroom observations, with the goal of recording the behaviour of the teacher, or of the learners, or of the interactions among them. The observations are done by the researcher, and a video recorder is often used. In order to count as a good research tool, the observation must be recorded systematically, related to well-articulated research questions, and subjected to checks and controls on validity and reliability; so this is really quite a demanding research tool. Observations can be structured or unstructured, both having their own advantages and disadvantages.

In unstructured observation, the researcher can be a non-participant observer, sitting at the back of a classroom taking notes, or a participant. Unstructured observation has the advantage of not imposing concepts or frames of reference, so that the researcher is free to record behaviour as it occurs, making special notes of ‘critical incidents. On the other hand, as with unstructured interviews, it is much more difficult to make organise and analyse the data collected, and it requires a great deal of the observer (validity vs. reliability issues).

In structured observations, the researcher must carefully define what behaviour is to be observed; for example, the number of times a teacher gives negative feedback, or the proportion of classroom time dedicated to explicit grammar instruction, or the turn-taking language used by students when engaged in group tasks. This implies that the researcher has a good understanding of of the phenomenon under study, so as to be able to determine, in advance of the observation sessions, what kinds of behaviour should be monitored to get the relavant data. Furthermore, a research tool must be individually devised for recording the observations – some sort of template or score sheet – and, if there is more than one observer, this must be used by all the observers, following the same criteria. So the tool needs to specify whose behaviour is to be observed and the categories of behaviour to be noted, and might also have to specify affective elements (attitudes), cognitive elements (intellectual components, learning strategies) and psychomotor elements (posture, gesture, movement); activity (repetition, etc.).

The problems of structured observation include inadequate definitions of what kinds of behaviour correspond to a given concept; lack of confidence of observers in their own judgement; and the possibility of observer effect i.e. the presence of the observer changing the environment.

Case Studies

A case study is not a method or even a type of research – it’s defined quite simply by its interest in individual cases. Whether there is only one case, or there are multiple cases, the crucial feature of a case study is the individuality of the case: it is not a sample and the numbers of cases which fall into any category is not important. The cases in question may be individual people, small groups, organisations, communities, classroom groups or events. An individual case study might look at a senior executive’s experience of classroom English in a Barcelona private language school; and an organisational case study might examine how the English department of a university in Boston tried to improve English language support over a four year period.

Case studies make no attempt to generalise; no hypothesis, or theory, or general explanation of a phenomenon is made (although they can be useful in the early stages of research on a subject, in suggesting hypotheses, and in providing a trial run for research methods). So why use them? First, intrinsic case studies are used when one is interested in a case not because we want to generalise from it, but because it’s interesting in its own right. The purpose of the study is is not theory building, but rather intrinsic interest in a particular case. Second, instrumental case studies are undertaken to provide insight into an issue, or to refine a theory. The case itself is of secondary interest. While It is still looked at in depth, the case is studied to help us pursue an external interest. In the context of an MA dissertation, the instrumental case study is thus not appropriate. So if you want to do a case study, you have to explain to your tutor what the intrinsic interest of the case is for you, and you then have to persuade him or her that it’s interesting enough to warrant the effort. Case studies use qualitative methods more often than not, unstructured interviews, observations, field notes, diaries, that sort of thing, They can be very interesting, but they’re actually very difficult to do well. It’s certainly not a soft option – only consider it if you’re really fired up about a case, and if you have a tutor who can guide you carefully through it.

Getting The Study Underway 

Let’s go through the process of starting a small study for your MA dissertation.

1) Decide on an area you want to study. Play to your strengths: choose an area you already know about, and maybe that you’ve already done well on in an earlier part of the MA. Recall that earlier in this chapter, I invented the example of an MA student like you who is an English language teacher working with young learners in Spain. You want to look at classroom methodology , an area you are particularly interested in.

2) Define the problem. Articulate a Thesis Question which focuses on a particular problem. As we saw earlier in the example, in your school, you’re aware that results are poor, and you have the impression from anecdotal evidence and things you’ve read that most Spanish children finish primary school with little ability to communicate in English, as measured by results in the Cambridge CET exam, for example. You think that teachers’ widespread use of the “PPP” approach to teach a grammar-based syllabus is a significant factor, and you formulate the question “Why are the teachers not embracing a more up-to-date, “Communicative Language Teaching” (CLT) approach?

3) Consider Your Resources

  • Can you participate actively (as a teacher trainer or trainee) in the study? If you do, does this mean your study will be broadly speaking qualitative?
  • How many teacher trainers, teacher trainees, and practicing teachers can you count on to participate in a questionnaire? Remember that you can use online questionnaires, so how many particiapnts can you get from your internet, social media, connections?
  • Who can you interview?
  • Can you observe teacher training sessions? Can you record them?
  • Can you get access to teacher trainer notes /teacher journals / learner diaries etc.?
  • What IT equipment can you get access to?
  • What software (e.g., SPSS) can you get access to?
  • Can you travel?
  • What’s the time frame?

4) Articulate Research Questions. Note first that you have to be clear about what you mean by “a CLT approach”. This will be an important part of the literature review, but you need to begin with a definition that you think you can work with, or define the term yourself. Next, you have to decide if you’re going to take a case study approach, or widen it a bit so that the study has some generalisability. Let’s say you decide that this is going to be a small study not a case study. OK, now you can write the research questions. Suppose you come up with these:

RQ 1: What are the perceptions of English teachers working in primary education in Spain of the CLT approach?

RQ 2: What do these primary school teachers think are the obstacles which stand in the way of the implementation of the CLT approach ?

You might also want to use one or two hypotheses to help the study stay focused. For example:

H 1: The teachers in the study claim to be favourably disposed towards the CLT approach, but feel insecure about implementing it.

H 2: The teachers think that resistance from superiors and parents is the main obstacle to implementing a CLT approach.

Obviously, there will a lot of other factors to consider, but these 2 predictions ,whether they turn out to be supported or not by the data, indicate the main thrust of your argument and help to position you.

5) Decide on the research tools. The most usual, and in my opinion, the best, tools to use for such a study are a Likert scale questionnaire and follow-up interviews.

6) Find examples of the chosen research tools that have been used in previous studies. Whatever your topic, it’s important that there is already a body of literature to consult. This literature will include studies, and it’s quite likely that one or more of the studies will have used a Likert scale questionnaire. If you can use a Likert scale questionnaire that has been used in a previously published study, this is a tremendous boost to your own study and will save you a great deal of work. It’s difficult to exagerrate the value of doing a replication study, but a full, faithful replication study is usually beyond the resources of MA students. Never mind – you can still use a published questionnaire, or an adapted version of it, and I strongly advise you to do so.

7) Design the questionnaire and the interview form.

8) Find particiapnts for the questionnaire and the interviews. Generally speaking, 50 respondents to a questionnaire and 6 interviews is fine.

9) Get the ethics permissions sorted out. You can’t ask people to fill in questionnaires or take part in interviews for a study without getting their permission and addressing ethical questions like privacy. Each university handles these matters in its own way, so make sure you comply with your uni’s requirements before doing any field work.

10) Show your tutor what you’ve done so far, get feedback and revise as necessary. You should be in regular contact with your tutor when doing the dissertation, and it’s really important not to take any major decision without consulting him or her. After some less formal consultations, you should finally write a dissertation proposal, which will be submitted to your tutor. The proposal will be 2 or 3 pages long and will include:

11) Write the Proposal

  1. Decide on a provisional title. Don’t spend much time on this; here the title is just to give a “ball park” indication of the diss.
  2. The background. A brief explanation of why you find this area interesting, and what particular aspect of this area you will focus on. This is NOT a literature review! You might mention a few sources, but the idea here is to tell your tutor what the area is, why you’re interested, and why it’s interesting.
  3. The Thesis Question which will inform your research. I think it is much better to articulate a question, rather than an “overall aim”, for example, because this is the
  4. Two or three Research Questions and hypotheses if you choose.
  5. Your research methods and participants.
  6. The value of your intended research. What do you think the results of your study will contribute to the problem that you’ve tackled?
  7. Ethical issues which arise from the proposal. If you intend to ask students or teachers or administrators for their opinions on colleagues, for example.
  8. References. Give your tutor an indication of the literature you will include.

Once you’ve got your tutor’s feedback and final approval, and once you’ve got the ethics issues sorted, you’re ready to start the field work, which I’ll discuss in Part 2.

Advertisements

10 thoughts on “Some Advice on Doing an MA Dissertation Part 1

  1. This is very helpful! Regarding pilot studies, it’s my understanding that they can be done before getting ethical approval but the data, if relevant, can be analysed and used in the report. Is my understanding correct and, if so, what makes pilot data exempt from (prior) ethical approval?

    Like

  2. Are they still allowing you to participate in the MA at Leicester? Despite your minor role in their programme, it must be obvious to anyone who has witnessed your outbursts on this and other blogs that you have few, if any, of the behaviours required of a lecturer. And let’s not get started about the intellectual requirements.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s