PAIS Newsletter - March 2014 (Plain Text Version)
|
||
|
In this issue: |
Articles LANGUAGE ASSESSMENT AND EVALUATION: A PERSPECTIVE ON COURSE GRADES, FORMAL TESTING, AND LEVEL PROMOTION Nicholas Ferdinandt, University of Arizona, Tucson, Arizona, USA
In our work there is great satisfaction in knowing that we’re doing a good job. And while car mechanics and housepainters have great satisfaction in the work they do because they can literally see the results of their toils at the end of the day, we teachers and administrators don’t have the same immediacy with the effects of our work. So how do we know we’re achieving our aims? Do we wait 20–30 years for that student or two to come back and tell us at the end of our careers how much we affected their lives with what we did in our classroom? As a practical matter we can’t wait that long to get feedback that helps us do our work better. A good system of assessment, evaluation, and level promotion based on sound practice goes a long way to help us not only improve our teaching, but also improve our programs while justifying what we do to those who make us account for ourselves. And speaking for myself, there is great satisfaction in that. The U.S. Department of Education, which accredits the accreditors, has made classroom-based assessment and evaluation a key element in achieving and maintaining accreditation. The government regulations regarding student achievement and assessment clearly require student learning outcomes, measurement of those outcomes, and a systematic data feedback loop that all reinforce student learning. Analysis of data gathered through a sound system of assessment is the basis of any program’s claim to quality instruction, which should be based on real data, whether quantitative, qualitative, or both. Approaches to language learning assessment and evaluation range from standardized proficiency exams—that take a snapshot of a student’s proficiency at a given point—to teacher-made tests. Standards of accreditation clearly lay out the necessary good practices for measuring student learning and, frankly, ESOL programs are out front in many of these areas due to the leadership of TESOL International Association and the accrediting bodies, such as the Commission on English Language Program Accreditation, in advancing teacher development and industry standards that guarantee attention and assurance of student language learning. As postsecondary programs and private institutions teaching ESOL have come into compliance with the new accreditation law, they have had to contend with the definitions and standards of student achievement that have challenged the quality claims for their programs. After all, how do you link student learning with grades, formal testing instruments, and promotion to the next level? Any good program has a mixture of all of the necessary elements for good evaluation, whether qualitative or quantitative measures. A course grade can be both a qualitative and quantitative measure of student performance. However, the course grade is only valid for student advancement to the next level insofar as that grade is devoid of subjective measures, such as attendance, participation, and other nonobjective measures. The course grade, when the assessments are directly aligned with the student learning outcomes for the course, have a more direct relationship with achievement without these other factors. However, someone in the program needs to ensure that the assessments are in alignment, and administration must provide training on teacher-made assessments to teachers, who have not traditionally been exposed to good practice in assessment in their TESOL teaching certificate or graduate degree programs. Advancing students based on grade promotion alone is problematic in that many times the promotion of an ESOL student to the next level is as much a result of “being a good student” (i.e., knowing how to acquire knowledge and demonstrating good study skills) as it is a result of memorizing declarative knowledge or demonstrating skills of language production. This is the qualitative essence of the grade, and no matter how much you control for other factors, it is difficult to use as the sole basis of advancement and the program’s quality claims, because some element of social promotion will occur. The quantitative aspect of the grade often is the numeration of the final percentage that generates the final grade. However, this aspect alone is also problematic when a student misses a test and cannot make it up. A teacher’s ability to create a fair assessment instrument also makes promotion on grades alone problematic. Recently, I had a student getting straight As in all his other classes come to me to dispute a failing grade on an exam in his grammar class. He showed me the exam and told me why he had failed. His misinterpretation of the directions was based on the prompts, which were misleading. The teacher had merely recreated a poorly written exercise from a popular grammar book, and when the student misinterpreted the directions given the nature of the prompts, the teacher refused to revise the grade, even though the student knew the material and got all other questions correct on the other sections of the test. His advancement to the next level was not compromised by this issue, but his motivation and his relationship of trust with the teacher were compromised. Thus, grades alone are problematic as a means of advancing students. To address this issue, many programs have instituted level exit tests that standardize assessment across all courses and levels. Teaching to the test remains a problem in these programs. Formal testing, such as any popular proficiency exam used for entrance to university that may or may not align with the Common European Framework (CEF), provides an external check of proficiency gains over time, but is not a valid measure of student achievement or learning that can be directly linked with classroom instruction. A student can conceivably be coming to class, but studying for the proficiency test at home, and achieving gains on the test but not achieving the learning outcomes in the classroom. Thus, students fail their classes, but earn admission to the university, which is often the ultimate goal anyway. Any system of advancement, then, that relies on formal testing instruments alone also falls short because no formal, external system can claim a direct effect to what the teacher is doing in class. As stated earlier, in the realm of accreditation, a program must link what it is doing in the classroom with student gains in performance in order to make a claim that it has a quality program. So a combination of the two, grades and formal testing, would provide internal and external referents that can serve for both advancement within the program and the student’s ultimate goals. Our program at the University of Arizona, like many other programs in the United States, is exploring a combination of grades and external, formal gatekeeping exams that take into account both the qualitative and quantitative aspects of assessment in the program. We have provided a lot of training to our teachers in the area of language assessment by hiring an expert in classroom-based assessment, who also oversees a committee that ensures that all teacher-made testing instruments align with the student learning outcomes of our program. This approach was one aspect of a two-pronged plan to also align the program with CEF, which in itself has posed a few challenges worthy of another article. Our program’s transition to better grades upon which to promote students, which is based on direct assessment of student learning outcomes devoid of all other factors, has not stopped “social promotion” and has met with resistance among faculty, who like the old way of doing things. Instituting an external gatekeeping exam at crucial levels can provide an external check as well as a way of aligning with CEF. But most important, it can also provide a reset button that takes into account the student’s own path of learning that is unique to that individual and is an iterative process of acquisition. Our proposed new system will be a combination of grades, formal testing, and an algorithm that will allow students to track into classes and levels that reinforce and advance their language gains. This system is not ideal, nor perfect, but it takes into account the learner, second language acquisition, sound assessment practices, and the social aspects of language learning. This system will never work for all students or all teachers, that’s for sure. One teacher, while I was giving a lecture on this topic, lamented the insistence on measuring learning at all. “But what about the magic?” she asked. And while I don’t deny one’s right to bring about magic in the classroom, as far as I know, “magic” cannot be measured. And whether using magic to teach or any other method, results can and ought to be measured so that we can not only prove to others that we have a good program with good teachers and good resources for learning, but also prove that students are attaining their goals through what we are doing. Furthermore, we can see the fruits of a labor that is much less concrete than painting a wall. Nicholas Ferdinandt holds an EdD in educational leadership from the University of St. Thomas, in Minnesota. He serves as the associate director at the University of Arizona’s Center for English as a Second Language and also serves as the director of Academic Bridge Programs there. Nick’s research interests revolve around language program evaluation and ESL program accreditation. |
|