|
In our work there is great satisfaction in knowing that we’re
doing a good job. And while car mechanics and housepainters have great
satisfaction in the work they do because they can literally see the
results of their toils at the end of the day, we teachers and
administrators don’t have the same immediacy with the effects of our
work. So how do we know we’re achieving our aims? Do we wait 20–30 years
for that student or two to come back and tell us at the end of our
careers how much we affected their lives with what we did in our
classroom? As a practical matter we can’t wait that long to get feedback
that helps us do our work better. A good system of assessment,
evaluation, and level promotion based on sound practice goes a long way
to help us not only improve our teaching, but also improve our programs
while justifying what we do to those who make us account for ourselves.
And speaking for myself, there is great satisfaction in that.
The U.S. Department of Education, which accredits the
accreditors, has made classroom-based assessment and evaluation a key
element in achieving and maintaining accreditation. The government
regulations regarding student achievement and assessment
clearly require student learning outcomes, measurement of those
outcomes, and a systematic data feedback loop that all reinforce student
learning. Analysis of data gathered through a sound system of
assessment is the basis of any program’s claim to quality instruction,
which should be based on real data, whether quantitative, qualitative,
or both.
Approaches to language learning assessment and evaluation range
from standardized proficiency exams—that take a snapshot of a student’s
proficiency at a given point—to teacher-made tests. Standards
of accreditation clearly lay out the necessary good practices
for measuring student learning and, frankly, ESOL programs are out
front in many of these areas due to the leadership of TESOL
International Association and the accrediting bodies, such as the
Commission on English Language Program Accreditation, in advancing
teacher development and industry standards that guarantee attention and
assurance of student language learning.
As postsecondary programs and private institutions teaching
ESOL have come into compliance with the new accreditation
law, they have had to contend with the definitions and
standards of student achievement that have challenged the quality claims
for their programs. After all, how do you link student learning with
grades, formal testing instruments, and promotion to the next
level?
Any good program has a mixture of all of the necessary elements
for good evaluation, whether qualitative or quantitative measures. A
course grade can be both a qualitative and quantitative measure of
student performance. However, the course grade is only valid for student
advancement to the next level insofar as that grade is devoid of
subjective measures, such as attendance, participation, and other
nonobjective measures. The course grade, when the assessments are
directly aligned with the student learning outcomes for the course, have
a more direct relationship with achievement without these other
factors. However, someone in the program needs to ensure that the
assessments are in alignment, and administration must provide training
on teacher-made assessments to teachers, who have not traditionally been
exposed to good practice in assessment in their TESOL teaching
certificate or graduate degree programs.
Advancing students based on grade promotion alone is
problematic in that many times the promotion of an ESOL student to the
next level is as much a result of “being a good student” (i.e., knowing
how to acquire knowledge and demonstrating good study skills) as it is a
result of memorizing declarative knowledge or demonstrating skills of
language production. This is the qualitative essence of the grade, and
no matter how much you control for other factors, it is difficult to use
as the sole basis of advancement and the program’s quality claims,
because some element of social promotion will occur. The quantitative
aspect of the grade often is the numeration of the final percentage that
generates the final grade. However, this aspect alone is also
problematic when a student misses a test and cannot make it up.
A teacher’s ability to create a fair assessment instrument also
makes promotion on grades alone problematic. Recently, I had a student
getting straight As in all his other classes come to me to dispute a
failing grade on an exam in his grammar class. He showed me the exam and
told me why he had failed. His misinterpretation of the directions was
based on the prompts, which were misleading. The teacher had merely
recreated a poorly written exercise from a popular grammar book, and
when the student misinterpreted the directions given the nature of the
prompts, the teacher refused to revise the grade, even though the
student knew the material and got all other questions correct on the
other sections of the test. His advancement to the next level was not
compromised by this issue, but his motivation and his relationship of
trust with the teacher were compromised. Thus, grades alone are
problematic as a means of advancing students. To address this issue,
many programs have instituted level exit tests that standardize
assessment across all courses and levels. Teaching to the test remains a
problem in these programs.
Formal testing, such as any popular proficiency exam used for
entrance to university that may or may not align with the Common
European Framework (CEF), provides an external check of
proficiency gains over time, but is not a valid measure of student
achievement or learning that can be directly linked with classroom
instruction. A student can conceivably be coming to class, but studying
for the proficiency test at home, and achieving gains on the test but
not achieving the learning outcomes in the classroom. Thus, students
fail their classes, but earn admission to the university, which is often
the ultimate goal anyway. Any system of advancement, then, that relies
on formal testing instruments alone also falls short because no formal,
external system can claim a direct effect to what the teacher is doing
in class.
As stated earlier, in the realm of accreditation, a program
must link what it is doing in the classroom with student gains in
performance in order to make a claim that it has a quality program. So a
combination of the two, grades and formal testing, would provide
internal and external referents that can serve for both advancement
within the program and the student’s ultimate goals.
Our program at the University of Arizona, like many other
programs in the United States, is exploring a combination of grades and
external, formal gatekeeping exams that take into account both the
qualitative and quantitative aspects of assessment in the program. We
have provided a lot of training to our teachers in the area of language
assessment by hiring an expert in classroom-based assessment, who also
oversees a committee that ensures that all teacher-made testing
instruments align with the student learning outcomes of our program.
This approach was one aspect of a two-pronged plan to also align the
program with CEF, which in itself has posed a few challenges worthy of
another article. Our program’s transition to better grades upon which to
promote students, which is based on direct assessment of student
learning outcomes devoid of all other factors, has not stopped “social
promotion” and has met with resistance among faculty, who like the old
way of doing things.
Instituting an external gatekeeping exam at crucial levels can
provide an external check as well as a way of aligning with CEF. But
most important, it can also provide a reset button that takes into
account the student’s own path of learning that is unique to that
individual and is an iterative process of acquisition. Our proposed new
system will be a combination of grades, formal testing, and an algorithm
that will allow students to track into classes and levels that
reinforce and advance their language gains. This system is not ideal,
nor perfect, but it takes into account the learner, second language
acquisition, sound assessment practices, and the social aspects of
language learning.
This system will never work for all students or all teachers,
that’s for sure. One teacher, while I was giving a lecture on this
topic, lamented the insistence on measuring learning at all. “But what
about the magic?” she asked. And while I don’t deny one’s right to bring
about magic in the classroom, as far as I know, “magic” cannot be
measured. And whether using magic to teach or any other method, results
can and ought to be measured so that we can not only prove to others
that we have a good program with good teachers and good resources for
learning, but also prove that students are attaining their goals through
what we are doing. Furthermore, we can see the fruits of a labor that
is much less concrete than painting a wall.
Nicholas Ferdinandt holds an EdD in educational
leadership from the University of St. Thomas, in Minnesota. He serves as
the associate director at the University of Arizona’s Center for
English as a Second Language and also serves as the director of Academic
Bridge Programs there. Nick’s research interests revolve around
language program evaluation and ESL program
accreditation. |
