A Case for Practice-Based Research in ITA Assessment: Co-Constructing a Research Agenda

Kate Yaw, University of South Florida, Tampa, FL, USA

Tania Ferronato, University of Mississippi, Oxford, MS, USA

Jihye Kim,University of Mississippi, Oxford, MS, USA

In recent years, there has been growing interest in bringing language researchers and practitioners into contact through a process known as researcher-practitioner (R-P) dialogues. Indeed, when researchers and practitioners actively collaborate, they co-construct knowledge that is theoretically sound, practically relevant, and applicable to real-world contexts (Sato & Loewen, 2022). Within the TESOL community, which encompasses many stakeholder groups, a context that could benefit from growth in R-P dialogues is ITA assessment. Practitioners, as the ‘face’ of ITA assessment, have meaningful hands-on experience that offers value to researchers working in this area, while researchers have dedicated time to focus on studying phenomena that directly impact practitioners and ITAs.

One model for such collaborations is practice-based research (PBR; Becker, 2024), which centers practitioners’ contributions across the research cycle (Sato & Loewen, 2022). In the five-stage PBR cycle proposed by Becker (2024), the process initiates with practitioners 1) noting a question or issue in their professional practice and 2) working with researchers to review what is already known about this issue from existing research. Once a research gap has been established, researchers 3) collaborate with practitioners to design and carry out a study that addresses the practitioner-identified problem. Practitioners and researchers then 4) evaluate how the study has been implemented, making any necessary adjustments to materials and instruments. Finally, researchers and practitioners 5) use their findings to determine any recommendations or implications that can be applied to future practice beyond the current research study. Importantly, Communities of Practice that foster R-P dialogues also help to facilitate PBR (Becker, 2024), pointing to the value of spaces in which ITA practitioners and language researchers come together to learn from one another.

In this spirit, we have engaged with the topic of ITA assessment research over the past three years in an effort to co-construct an agenda for practice-based ITA assessment research. Briefly, we began with a scoping review of 48 published ITA assessment research studies (Kim et al., under review) to see what work already exists. We paired this with an analysis of publicly-available ITA testing information from 100 R1 US university websites as an indicator of what potential ITAs might find when exploring US university options. With a better understanding of publicly-available information about ITA assessment research and practices, we then turned to ITA assessment practitioners, who shared valuable insider perspectives via a survey (n = 14) and follow-up interviews (n = 4) (Ferronato et al., in preparation). At TESOL 2025, we shared our findings across these three project pieces in a dialogue session with ITA practitioners, with an aim of developing an agenda for ITA assessment research that is informed by practitioner and researcher perspectives. We turn now to presenting a brief overview of this proposed research agenda (see Table 1).

Table 1

Overview of Proposed Co-Constructed Research Agenda

We have organized this into three broad categories for future work on ITA assessment. The first, who we assess, relates to work that focuses on test-takers, considering issues of fairness, consequences for those who pass and those who fail the assessment, and experiences with the assessment itself. The second category, what we assess, proposes extending existing construct-related research into new dimensions, as well as considering how rubric design relates to construct definition. The third, how we assess, encompasses a broad range of focus areas, including validation of tasks and tests, work on modality and stakeholder involvement, and the role of AI and other technology to support assessment activities. We note that the topics presented in Table 1 are not an exhaustive list, but they represent some areas of interest among ITA practitioners who have participated in our surveys, interviews, and/or dialogue sessions. Thus, there may be fruitful opportunities for future PBR projects in some of these areas.

Based on our scoping review of ITA assessment research studies (Kim et al., under review), we also note some methodological opportunities for ITA assessment research. Out of 48 total studies, 32 (67%) employed quantitative methods, while 7 (14%) were qualitative and 9 (19%) were mixed-methods. Notably, none of the studies reviewed included longitudinal data. There is an opening, therefore, for expanding the methodological toolkit that has traditionally been used to include more qualitative, mixed-methods, and longitudinal approaches as a complement to the more robust body of quantitative work that already exists.

In conclusion, we advocate for the value of future research that incorporates ITA practitioners into and across the research cycle, and we view the PBR model (Becker, 2024) as one way of supporting this. Incorporating practitioners could help to ensure that research is directly applicable to testing practices, policies, and curricula for ITA support programs. Furthermore, fostering these R-P connections brings practitioners’ voices and needs to the focus of inquiry, which validates the core purpose of research to be more meaningful and practically useful. At the same time, we acknowledge the potential challenges for such collaborations. These projects take time and require R-P connections that may not exist in all institutions. There also are questions about the perceived value of this type of work, with projects being viewed by some stakeholders as internally useful only and therefore not easily publishable. While these challenges are real, we have been inspired by the interest and enthusiasm of members of the ITA-IS community for work that provides research-based evidence for practitioners to make their case to their administration, as well as research that addresses the real-life challenges that practitioners face in their assessment-related work. Our ITA practitioner population may be relatively small, but their impact ripples broadly across US university and college campuses. For these reasons, we believe that supporting practice-based ITA assessment research is a valuable scholarly investment.

References

Becker, A. (2024). Applied linguistics communities of practice: Improving the research practice relationship. Applied Linguistics, 45(2), 272-286. https://doi.org/10.1093/applin/amad010

Ferronato, T., Kim, J., & Yaw, K. (in preparation). Assessing International Teaching Assistants: An overview of assessment practices and practitioners’ perspectives.

Kim, J., Ferronato, T., & Yaw, K. (under review). A scoping review of empirical studies on International Teaching Assistant assessment in the North American English context.

Sahranavard, N., & Du, Q. (2022). Reimagining ITA training: Promoting student-scholars’ agency. ELT Journal, 76(4), 476-486. https://doi.org/10.1093/elt/ccab092

Sato, M., & Loewen, S. (2022). The research-practice dialogue in second language learning and teaching: Past, present, and future. The Modern Language Journal, 106(3), 509-527. https://doi.org/10.1111/modl.12791

Dr. Kate Yaw is an Assistant Professor of Applied Linguistics at the University of South Florida. She has worked in language education since 2007, with experience in teaching, teacher training, and program administration. Her research interests include perception/production of speech and the role of listeners in oral communication, along with research methodology and quantitative research ethics.

Dr. Tania Ferronato is an Instructional Assistant Professor at the University of Mississippi. She recently earned her PhD in Linguistics and Applied Language Studies from the University of South Florida. Her research relates to second language speech perception, computer-assisted language learning, and language assessment.

Dr. Jihye Kim is an Instructional Assistant Professor at the University of Mississippi. Her research interests include Mobile-Assisted Language Learning (MALL) among older adults, language assessment, and (multimodal) digital discourse.