ITAIS Newsletter - June 2018 (Plain Text Version)

The Intercultural Communication Center (ICC) at Carnegie Mellon University (CMU) has been researching correlations between the TOEFL Speaking score and international teaching assistant (ITA) assessment scores since 2005. We use this research to give guidance to the ITAs themselves and also to guide departments in their admissions of potential ITAs as well as their expectations of those ITAs once they have started in their programs.

Since 2005, the ICC has been collecting the TOEFL Speaking subscores for incoming graduate students. We verify the scores, then compare them to the students’ scores on CMU’s in-house ITA test. After 13 years of collecting scores, we feel confident that we have a clear understanding of what the TOEFL Speaking subscore means in reference to the CMU environment. Our center tests more than 600 students per semester, thus over several years we were able to assemble more than 1,000 quality samples for our internal analysis. In conjunction with our data, we also used data from Educational Testing Service, the makers of TOEFL. We have begun to use our research to assign the language certification score for many ITAs for their teaching assistant jobs.

Table 1 shows which TOEFL scores are correlated with which ITA certification scores. The students or departments can choose to use the student’s Speaking score for the certification, or they can ask for a performance test. The table helps the students and departments decide which assessment is best for each student

Table 1 shows the correlation between TOEFL Speaking scores and, based on that score, what we anticipate their ICC ITA test score would be. In this chart, Pass is the top score; Restricted I allows ITAs to teach recitations; Restricted II allows ITAs to teach office hours or assist in labs: Not Ready means students are not yet able to work with students.

Table 1. Correlation Chart

Note: ICC = Intercultural Communication Center; ITA = international teaching assistant.

So how do we use this research to inform ITAs and their departments?

1. Admitting new students: We use this research to advise departments before they admit new students. Because there is no graduate dean at CMU, each graduate department makes its own admissions guidelines, including which cut-scores are acceptable for their specific program. The ICC has developed TOEFL guidelines that are revised each January that give both recommended and minimum scores for admission. We also include the chart (Table 1) so that departments will be able to predict if a potential student will be able to fulfill their ITA requirements. With the information in the TOEFL guidelines, departments can make informed decisions.

2. Realistic language expectations: We also give departments a way to help students set their expectations about language before the students accept their admissions. In the past, students have complained that they did not know before they came to campus that they were expected to take an ITA assessment, or that they may have to take classes to prepare for it. Students would typically say “I thought when I was accepted to CMU that my English was good enough to study here. I didn’t know that I would have to do a lot of extra work.”

ICC has developed a template for departments to include in their admission letters that explains what kind of fluency is needed for success in U.S. graduate programs; announces the student will have to complete an ITA certification; gives them information about the ICC, as we are the ITA training and testing center on campus; and recommends that the admitted student go to the Incoming Student page on our website, which outlines work that they can do to prepare themselves before coming to CMU.

3. Certification options: The ICC also dedicates a significant portion of its website to getting information to preservice ITAs. We give the students the same information that is presented in Table 1 on the Registration page for the ITA certification. They are given a choice between using their TOEFL Speaking score or our in-house test to be cleared. We spell out clearly what each score means and which score band they will fall into if they use the TOEFL option. Note that we have been fairly conservative in selecting these TOEFL scores; students always have the option to ask for a test if they think their fluency has changed. We also include a chart that details the language capabilities expected in each score band.

4. Orientations for incoming students: Another way that we get this information out to the wider campus is through sessions called Language Support Orientations. These are one-time 1-hour sessions that are run multiple times throughout the beginning of the semester. The students bring a verified copy of their TOEFL score (e.g., their score report), which we then use to collect verified scores. The orientation offers guidance about what their TOEFL scores can tell them about their own readiness for academic work, a discussion of common language and cross-cultural issues for graduate students, and reviews what support the ICC provides.

We have gotten very good feedback from the departments on our correlation chart; they feel it makes admission decisions more robust and gives valuable information about the expectations they can have about incoming students. Using the TOEFL Speaking scores to clear students to be teaching assistants empowers the students to make decisions about their own needs. The ICC will continue to collect the speaking subscores to ensure that the information on recommended scores and the levels of scores needed for ITA clearance are accurate.

Rebecca Oreto is the associate director of the Intercultural Communication Center at Carnegie Mellon University and has held a variety of leadership positions in TESOL’s ITA Interest Section. Currently, she is the strand coordinator for the Listening, Speaking, and Pronunciation Strand in TESOL. She is the founder of the annual ITA Professionals Symposium and hopes you will attend in 2019.