Carol A. Chapelle - The Concise Encyclopedia of Applied Linguistics

These questions remain important in the assessment of grammar despite the fact that L2 educators have always acknowledged the importance of linguistic resources, specifically the grammatical resources of communication. Fundamental questions remain because of lack of agreement on how to represent linguistic resources as well as how they can best be taught, tested, and researched. As a result, for decades, testers have proposed and refined models of L2 knowledge, each specifying an explicit grammatical component (e.g., Lado, 1961; Canale & Swain, 1980; Bachman, 1990; Bachman & Palmer, 1996; Purpura, 2004, 2016). These models are introduced as reflecting two distinct conceptualizations of language proficiency: one based on knowledge of grammatical form and the other based on a set of linguistic resources for creating contextualized meaning. Both conceptualizations of L2 knowledge are effectively used today as a basis for designing assessments for a range of purposes, but their differences are important because they affect score interpretation and use.

Drawing on structural linguistics and discrete‐point measurement, Lado (1961) proposed a L2 proficiency model in which L2 knowledge was conceptualized in terms of linguistic forms, occurring in some variational distribution, that are needed to convey linguistic, cultural, and individual meanings between individuals. While his model highlighted the relationship between grammatical forms and their communicative meaning potential, he prioritized form over meaning, thereby operationalizing proficiency as the accuracy of discrete, linguistic elements (phonology, syntax, lexicon) of language use (reading, listening, speaking, writing). In other words, L2 proficiency was defined solely in terms of discrete grammatical forms, with no categorization of the forms and little explicit acknowledgment of their relationship to meaning. This “traditional” approach to L2 assessment (i.e., grammar assessment) is usually based on a principled list of possible forms that might be measured. Figure 1displays a traditional list of grammatical forms. Figure 2shows a list of phonological forms based on Ellis and Barkhuizen (2005) and Bonk and Oh (2019).

Figure 1 List of grammatical forms based on Celce‐Murcia and Larsen‐Freeman (1999)

Figure 2 List of phonological forms

The form‐based approach to assessing knowledge of grammatical forms was the dominant paradigm in L2 assessment until the 1980s and remains even today the basis for test development in many contexts globally. Assessments rooted in this model are organized around the elicitation of discrete grammatical forms, whether they are assessed via selected‐response (SR), limited‐production (LP), or extended‐production (EP) tasks. Finally, the measurement of discrete grammatical forms is currently the mainstay of automated speech and writing assessments, as will follow.

To make the form‐based approach more useful for assessment, Purpura (2004) proposed a model of grammatical knowledge identifying how forms can be organized on the (sub)sentential and discourse levels. This model specifies graphological, phonological, lexical, morphosyntactic, cohesive, information management, and interactional forms (see Figure 3). These categories can be further elaborated to develop content specifications for grammar assessments, and to ultimately provide evidence of content representativeness. Figure 3also shows how these discrete forms have typically been measured in SR and LP tasks.

Figure 3 Grammatical knowledge as form (adapted from Purpura, 2004, reproduced with permission of Cambridge University Press through PLSclear)

The form‐based approach to grammar assessment is useful for providing fine‐grained information about a range of structures across proficiency levels. This approach is useful in diagnostic assessment, in assessments designed to provide individualized feedback, or in automated scoring protocols. Assessments based on the measurement of one or more discrete grammatical forms, however, should not be interpreted as a measure of L2 proficiency, as has been the case in many studies. L2 proficiency involves much more than knowledge of grammatical forms. That said, grammatical knowledge is, in fact, a fundamental component of L2 proficiency, and several studies (e.g., Grabowski, 2009; Kim, 2009; Liao, 2009) have produced consistently strong evidence of a relationship between grammatical knowledge and the ability of learners to use the L2 in context.

In the end, a form‐based conceptualization of L2 knowledge is narrow in scope because in language use a grammatical form is rarely disassociated from its meaning potential. Consequently, this conceptualization fails to address the semantic dimension of grammar, where, for example, an ‐ ed affix encodes past time. It also fails to capture how a sequence of forms in an utterance contribute to the conveyance of propositional meaning, or even how forms used in certain contexts can extend meanings by encoding social status, formality, culture, affect, stance, or other implied pragmatic meanings. The limitation of a form‐based conceptualization is especially evident in cases where meaning extensions can only be derived from context and are dependent upon an understanding of shared norms, assumptions, expectations, and interlocutor presuppositions. Thus, the semantic dimension must be considered if meaningful communication is prioritized over grammatically flawless communication. Finally, the semantic dimension must also be addressed if assessment is to provide comprehensive feedback, since some learners have mastered the form, but not its meaning, or vice versa. Or learners might understand the forms and associated semantic meanings but be incapable of expressing or interpreting meanings extended in context.

Linguists acknowledge that almost all linguistic forms are associated with semantic meaning , and that semantico‐grammatical knowledge is a critical component of the ability to comprehend and generate literal propositions (Purpura, 2016). In other words, graphological/phonological, lexical, morphosyntactic, cohesive, information management, and interactional forms all encode a semantic dimension, and when these form‐meaning mappings are arranged in syntax, they produce utterances which carry propositional meaning (Purpura, 2004). In assessment, the semantic dimension needs to be taken into account since response data vary in terms of semantic meaningfulness. Imagine a context in which a patient calls his doctor for a consultation, and the doctor replies: “I work with a patient now” (literal meaning, my current job is to do this work). This response reflects a knowledge gap in the connection between the morphosyntactic form work and its associated meaning (habitual time), and in this context, such an error may produce a misunderstanding. However, if the doctor responded: “I working with a patient now” or “I 'm working with a patient now” the meaning in both cases, despite the form error, would be: “I am currently occupied—and unavailable to talk.” The meaning of the proposition would have been conveyed. Note that assessment decisions related to all three responses would obviously be different, again underscoring the importance of the meaning dimension. To highlight the importance of semantic meaning as a resource for L2 proficiency, Purpura (2004, 2016) extended the form‐based model of grammatical knowledge to include a semantic dimension. Figure 4presents a characterization of semantico‐grammatical knowledge, along with items typically used to measure this dimension.