Sandrine Zufferey - Introduction to Experimental Linguistics

The construction of a good research hypothesis is the result of different stages, among which the most important are conceptualizing the hypothesis, on the basis of knowledge acquired in the field, and then operationalizing the hypothesis. We will discuss the specific stages for conceptualizing a hypothesis in Chapter 6, which is devoted to the practical aspects of an experiment. We will discuss the stages involved in the operationalization of a hypothesis in Chapter 2.

Let us now go back to the example of the influence of working memory on reading comprehension. In this example, the variable verbal working memory can be observed in two ways. The first possibility would be to measure the skills of the people taking part in the experiment by using a verbal working memory test. According to this evaluation and its results, participants could be sorted into groups. By doing so, every participant is included under a variable modality (e.g. high competence or low competence) depending on his/her own characteristics, as some people have better working memory capacities than others. In this case, the variable is simply observed during research.

A second possibility would be to manipulate the variable verbal working memory , by implementing conditions within the experiment where this variable has different modalities. In our example, the manipulation of the independent variable would aim at restricting the use of verbal working memory in some of the participants, in order to see the impact of such manipulation on reading comprehension, as compared to other participants whose working memory has not been restricted during the reading. A common task used for manipulating verbal working memory is to ask people to momentarily memorize different series of letters while reading the text, to report them and then to memorize others. Having to remember a series of letters while reading the text reduces the verbal working memory storage capacity used for reading and makes it possible to show a connection, if existent, between working memory and comprehension.

In general, in experimental research, the aim is to manipulate all the variables involved in the hypotheses. However, due to practical or ethical reasons, this is not always possible. For example, age, socio-economic level, bilingualism, etc., cannot be manipulated because they are inherent in people. When variables can be manipulated, the decision to manipulate them, as well as the way in which to manipulate them, must follow ethical principles, ensuring that research will not harm the participants during the test. The cost/benefit relationship must be clearly considered when pondering the possibility of manipulating a variable or not. For example, imagine that you formulate a hypothesis stating that in stressful situations, people tend to speak faster than in non-stressful situations. In order to study the influence of stress on articulation rate, you could decide to manipulate the participant’s stress level. To set up a stressful condition, you could imagine putting some of the participants in a dark room in front of an audience booing at them. In experimental terms, such manipulation would be adequate, in the sense that a high level of stress would most likely result from your manipulation. On the other hand, it would be totally inappropriate from an ethical point of view. Actually, this type of manipulation would affect the participants to a much larger extent than needed, and they would probably not leave the experiment unscathed. Although this is an extreme example, it illustrates the fact that an experiment should not leave an impact trace on the participants once the experiment is over. We will develop this point in Chapter 6, which is devoted to the practical aspects of an experiment.

We have seen that when operationalizing research hypotheses, variables need to be defined with accuracy. The main purpose of such a definition is to isolate the variables studied within the experiment, in order to reach a reliable conclusion as to the relationship between them. In parallel, and for the same purpose, it is necessary to control the other variables, known as external variables , which could influence the variables and the results obtained in the experiment. External variables can be multiple and we will return to them in Chapters 2and 6, where we will discuss hypotheses and the practical aspects of an experiment. However, it is generally acknowledged that the characteristics of the participants are variables which may interfere with the variables investigated in an experiment.

Going back to the example of the influence of memory on reading comprehension, we may assume that educational level, general cognitive abilities, age, reading habits, etc., can influence both memory and reading comprehension. Likewise, the characteristics of the material used in the experiment may have an influence on the results. If, in the above-mentioned example, we use very simple text and questions, it is possible that everyone answers the questions perfectly well, regardless of their memory skills. On the contrary, if the text and the questions are very complicated, it is possible that very few people will be capable of answering. In these cases, we risk not finding a connection between memory and reading comprehension, not because the link doesn’t exist, but because the material used for the experiment is not suitable for evidencing such a link.

To attenuate these potential problems, and to reduce the importance of the characteristics of the participants or the material employed, experimental research is based on data collected from a large number of people, using a broad palette of materials. Referring back to our example, it would be necessary to test a large number of people by means of a comprehension test. This test should contain multiple texts and different questions for each of them. In general, the material used in an experiment is defined as a set of items (the texts or the questions in our example are items). The ideal number of participants, as well as the number of items necessary to undertake proper research, is a complex question, which we will address in Chapter 6.

Furthermore, experimental research is generally carried out by recruiting naive participants, who ignore the goals of the experiment and who have zero expertise in the subject under study. This precaution aims to try to control certain cognitive biases that could influence research results. The first bias is related to the fact that the participants who know the research hypothesis may try to base their answers on this hypothesis. Should this happen, the results obtained could suffer from what is called confirmation bias . Rather than answering naturally, participants could provide answers based on the hypothesis to confirm it, not because the assumption is correct, but rather because it seems adequate to them (even if this is not the case). The second bias is related to the fact that participants may want to help the researcher. If the participants know or suspect the goal of an experiment beforehand, the results obtained in this second scenario may not correspond to reality, but rather to the answers that the participants presume are expected.

Finally, in experimental research, participants are generally assigned to conditions in a random manner. This means that every person has the same chances of being included under one condition of the experiment or another. This random assignment offers additional protection against the effect of uncontrolled external variables. In addition to testing a large number of people, randomly distributing them to the different conditions reduces the probability that external variables could systematically influence the results. However, this random assignment is only feasible when all variables are manipulated. When one or more variables are simply observed, participants must be included in one condition or another on the basis of their own characteristics, such as gender or age, for instance. In this case, we speak of quasi-experimental research , since it is not possible to control all the variables. Leaving this question aside, experimental and quasi-experimental research is very similar, and the elements developed in the following chapters apply to both types of research.