Free TESOL Quarterly Article
This article first appeared in TESOL Quarterly, Volume 45, Number 4, pgs. 628–660. Subscribers can access all issues for free here. TESOL members can subscribe to TESOL Quarterly here. To become a member of TESOL, please click here, and to purchase articles, please visit Wiley-Blackwell. © TESOL International Association.
|
Abstract
The English Language Proficiency Assessment (ELPA) is used in the state of Michigan in the United States to fulfill government-mandated No Child Left Behind (NCLB) requirements. The test is used to promote and monitor achievement in English language learning in schools that receive federal NCLB funding. The goal of the project discussed here was to evaluate the perceived effectiveness of the ELPA and to see if those perceptions could meaningfully contribute to a broad concept of the test’s validity. This was done by asking teachers and test administrators their views on the ELPA directly after its administration. Two hundred and sixty-seven administrators took a survey with closed and open-ended questions that aimed to tap into the consequential dimensions of test validity. An exploratory factor analysis identified five factors relating to the participants’ perceptions of the ELPA. Analysis of variance results revealed that educators at schools with lower concentrations of English language learners reported significantly more problems in administering the ELPA. Three themes (the test’s appropriateness, impacts, and administration) emerged from an analysis of qualitative data. This article discusses these results not only as a means of better understanding the ELPA, but also to contribute to larger-scale discussions about consequential validity and standardized tests of English language proficiency. It recommends that broadly defined validity data be used to improve large-scale assessment programs such as those mandated by NCLB. |
The study discussed in this article explored the concept of test validity (the overall quality and acceptability of a test; Chapelle, 1999) and how teachers’ expert judgments and opinions can be viewed as part of a test’s validity argument. It did so in the context of a statewide battery of tests administered in Michigan, a large Midwestern state in the United States, to English language learners (ELLs) from kindergarten through 12th grade. Data were gathered in the weeks after the testing by surveying educators who had been involved in administering the tests.
Using these data, in this article I examine the validity of the tests by analyzing what the teachers believed the tests measure and what they believed the tests’ impacts are. In essence, this paper investigates the social consequences of a large-scale testing program and scrutinizes ‘‘not only the intended outcome but also the unintended side effects’’ of the English language tests (Messick, 1989, p. 16). Because the testing in Michigan was required by the federal government’s No Child Left Behind (NCLB) Act, I begin by summarizing that law.
THE NO CHILD LEFT BEHIND ACT AND LANGUAGE POLICY TESTS
Although the United States has no official national language policy (Crawford, 2000), NCLB is sometimes viewed as an ad hoc federal language policy that promotes an English-only approach to education (Evans & Hornberger, 2005; Menken, 2008;Wiley &Wright, 2004). NCLB has created stringent education requirements for schools and states. Title I of NCLB,1 which provides federal funding to schools with low-income students, requires those schools to meet state-established Annual Yearly Progress (AYP) goals and to achieve 100% proficiency relative to those goals by 2014. (For details about how states define proficiency, see Choi, Seltzer, Herman, & Yamashiro, 2007; Lane, 2004; Porter, Linn, & Trimble, 2005.) If a school fails to meet the AYP goals, it suffers increasingly serious consequences, eventually including state takeover.
The law has special requirements for seven identified subgroups: African Americans, Latinos, Asian/Pacific Islanders, American Indians, students with low socioeconomic status, special education students, and ELLs. Before NCLB, achievement gaps between subgroups and the overall student population were often overlooked (Lazarı´n, 2006; Stansfield & Rivera, 2001). The law addresses this problem in three ways. First, there are special testing requirements for the subgroups. Now 95% of students within each subgroup—including ELLs who have been in the United States less than 1 year—must be tested for a school or district to meet its AYP (U.S. Department of Education, 2004b). Second, since each subgroup must achieve the same school and statewide AYP goals that apply to the general population, ELLsmust alsomeet English language proficiency benchmarks through additional tests. Finally, schools must report the scores of these subgroups separately, enabling stakeholders such as funders, test developers, teachers, test takers, and parents (McNamara, 2000) to hold the educational system accountable for discrepancies in scores.
This article first appeared in TESOL Quarterly, 45, 628–660. For permission to use this article, please go to http://www.copyright.com/.
doi: 10.5054/tq.2011.268063
The December 2011 TESOL Quarterly contains a wonderful study by Dilin Liu analyzing the frequency of phrasal verbs across several corpora. This is a really useful article for teachers and materials writers, with lots of great information in it and a valuable companion to his 2003 corpus study on the most common spoken American English idioms.
