Lecture 2(a)_Describing language__ ability

Test method
Examinee’s language Test score capacities
Examinee’s language Use capacities
Decision about examinee
Fundamental components of testing: process, interpretation and use
2. Defining language test construct
• Cronbach & Meehl (1955)
– A postulated attribute of people, assumed to be reflected in test performance.
– A quality or feature of people
• Chapelle (1998, in Bachman & Cohen) – Trait theorists attribute consistencies to characteristics of test takers, and therefore the define in terms of the knowledge and fundamental processes of the test taker. – Behaviorists attribute consistencies to contextual factors, and therefore define constructs with reference to the environmental conditions under which performance is observed. – Interactionalists see performance as the result of traits, contextual features, and their interaction.
– Something different
Different perspectives of construct definition
• Messick (1981)
– Theorists’ various perspectives of construct definition can be understood by identifying how they explain response consistency.
• Brown (2000)
– A construct, or psychological construct as it is also called, is an attribute, proficiency, ability, or skill that happens in the human brain and is defined by established theories.
– Language skills (Listening, speaking, reading and writing) – Language knowledge
• Problems with the skills and components model:
– It’s not clear how skills and knowledge are related. – The failure to recognize the full context of language use.
• Test takers’ performance is affected by a large number of factors. • What are these factors? • The learners’ language capacities (construct) are assumed to be mainly responsible for their language performance on a test. • A clear definition of the construct (capacities) we wish to measure and the means by which we observe and measure these abilities is very important.
3.1 language ability models
• Traditional approach
– Language knowledge (grammar, vocabulary, phonology / graphology)
• The skills and components model (Lado & Carroll)
• Savignon (1983) characterizes communication as:
– dynamic rather than … static …. It depends on the negotiation of meaning between two or more persons. … [It] is context specific. Communication takes place in an infinite variety of situations, and success in a particular role depends on one’s understanding of the context and on prior experience of a similar kind. – (Savignon 1983: 8-9)
• See Bachman (1990: 82)
– Halliday, van Dijk and Hymes’ discussion on the context of language use.
• Interactive or communicative approach
– Language use is a dynamic interaction between the context and the discourse itself.
Fundamental issues in language testing
Examinee’s language capacities
Construct
Examinee’s language performance
Examinee’s language Inference capacities
Validation
3.2 Models of communicative language ability
• Hymes’s L2 construct saw its implementation in the work of Morrow (1977), B. J. Carroll (1978), and Munby (1978), Canale and Swain (1980). – Morrow (1977):
• Kramsch’s (1986) description of communication interaction:
– Interaction always entails negotiating intended meanings, i.e., adjusting one’s speech to the effect one intends to have on the listener. It entails anticipating the listener’s response and possible misunderstanding, clarifying one’s own and the other’s intentions and arriving at the closest possible match between intended, perceived, and anticipated meanings. – (Kramsch 1986: 376)
3. Development of theories of L2 construct
• Chalhoub-Deville (2003)
– ‘Since the late 1960s, the language testing field has earnestly occupied itself with the nature of the L2 construct.’
• Canal and Swain (1980, 1983)
1. Introduction
• Language test as a form of measurement. • Construct: the ‘something’ that we want to measure in a test. • An important step in measurement: to define the construct both theoretically and operationally. • The definition affects the ‘look’ of a test, the reporting of the test scores, and the interpretation of test performance.
Lecture 2: Towards a clear understanding of the language test construct
Han Baocheng
National Research Center for Foreign Language Education Beijing Foreign Studies University