Uncanny Writing: Reader Responses to LLM Text Generating Tools

Ryan Morrison, George Brown College, Toronto, Ontario, Canada

For those writers using large language model (LLM) generative platforms, including ChatGPT, composing and editing written messaging is becoming increasingly normalized in professional and personal contexts. While fraught with ethical (Bender et al., 2021) and educational (Kumar et al., 2023) issues, nothing short of legislative policy is likely to hinder the continued proliferation and improvement of such technologies. In the interim, as language teachers, we are both learning and creating the norms of this new language paradigm together.

For language learners, this technology can appear as an outright boon for writing tasks in several different languages. Capable of achieving the aims of a large portion of written modes, auto-generated writing typically allows for the intended message to be understood by the receiver by accounting for style. Needless to say, there have been significant uptakes amongst language learners.

My Concerns

However, it is becoming clearer that in some situations, auto-generated messages are often not received in the way the sender intends. While the message is comprehensible, the meta information surrounding the message (i.e., the message seems automatically generated) hinders the reception of the message. For example, after a mass shooting at a Michigan university, an administrator was publicly admonished for using auto-generated text in their response (Levine, 2023). The fact that a person used this technology in specific writing contexts is becoming newsworthy. Anecdotally, when someone expresses their gratitude in a well-wishing email that contains many of the hallmarks of auto-generated text, I experience an involuntary perception of insincerity, despite the intention of the sender. I can also attest to several instances of parallel feelings from teachers and students who are themselves familiar with the output of text-generating platforms.

A Known Reader Response

This emotional phenomenon parallels other reactions to tech-enhanced mimics of humanity, such as photoshopped images and customer service chatbots. Perhaps the most apt alignment to this nuance is found in computer-animated video media —the uncanny valley. This feeling is particularly evident when the creator attempts to recreate a realistic rendering of a human, rather than an impressionistic or cartoon-style animation. This feeling in the viewer of the content occurs as the mind, for whatever reason, does not accept a computer-generated interpretation, and thus the suspension of disbelief is interrupted. As language teachers, knowing that some readers of auto-generated messages might have these ‘awkward’ feelings is an important point worth demonstrating to our students who are using text generators.

Another perspective worth considering is video-game AI—when a human plays against a program rather than another human. These types of games are famously ‘hackable’ in the sense that the player can eventually ‘beat the game’ once they understand the patterns of the programming. For example, a recent achievement in speed runs was achieved when a player completed Super Mario Brothers only a fraction of a second slower than a programmed version (Orland, 2023). This indicates the probability that a reader can strongly suspect that text was auto-generated if they are familiar enough with the output of this technology. Language teachers, in turn, should start becoming familiar with text-generating tools so that they are better able to engage in discussions of how and when to employ these tools.

What it Means for Language Teaching

With the above concerns in mind, language teachers should also consider the contexts in which we encourage learners to use LLM technology. What writing skills will change value? Beyond the obvious non-development of important language skills, what can we tell students about the effects of automating their language tasks? We will need to confront these uncomfortable, nebulous questions.

Context and Access

For one, teaching that ‘context matters’ may become a more valuable aspect of writing, automated or otherwise. We will always need to be thoughtful in our writing for practical reasons, including its relative permanence to speaking. There is also the concern that disruptive writing technology will affect the quality of the creativity of language arts. Historically, past writing technology innovation has not noticeably dimmed the utility of literary arts; rather, it oftentimes broadens access to a wider community of writers. Still, there are communities in the world that lack access to text-generating tools, but as they become a more common part of language teaching and learning, these gaps will hopefully lessen.

Impetus for LLM Tool Integration in Language Teaching

Being aware of the myriad of available tools that use this technology is important to knowing the state-of-the-art abilities. Due to the profusion of development of this technology, maintaining an ongoing comprehensive list at this point is not a reasonable endeavour. Further, knowing the output patterns of more popular tools will be an asset for teachers. For instance, language suggestion applications, such as Grammarly, have entirely different idiosyncrasies in their outputs than those with a greater generative focus (e.g., ChatGPT). Prompting a generative tool to “fix the grammar problems” requires the tool to rewrite the entire text, and it will often change important proper nouns if not given contextual information through a process known as prompt engineering. For suggestive tools, there is less of a risk of these major changes as prompts like “consider shortening for clarity” give the user a choice to accept or ignore the suggestion. These are but two of the many idiosyncrasies of current tools that language teachers should consider when guiding students on the use of this technology.

A Need for Increase in Research

To address these concerns, language teachers will need to investigate these tools from our unique perspective, and this, in turn, will likely lead to an increase in scholarly research into this phenomenon. Of course, it is inevitable that—as in qualitative research—a large aspect of subjectivity in this emotional phenomenon will be present. Knowing how people feel will give us greater insight into the current state of our social literacy as well. For some readers, this feeling may barely exist, and for others, this feeling may be more prevalent. Knowing that there are both those who have entirely positive and those who have entirely negative opinions indicates that answers will exist on spectrums.

To this end, there are several other spectrums that will intersect with the ones mentioned above, including a continuum ranging from highly emotional messaging to purely administrative texts. In our digital age, there are entirely new ways of communicating through text, and each one of these ways will likely develop its own relationship with auto-generated text. Certainly, the spectrums of user identity will play a role. In short, a framework or multiple frameworks for how and when writers benefit from enhancing their writing with LLM tools should be the ultimate goal.

Preliminary Research Questions

Below is a non-extensive list of preliminary research questions that we can use to further develop an accountable framework:

• How can people effectively identify an algorithmically generated text?
• How much exposure to LLM-generated text is required to develop the ability to identify that a text was generated?
• How does learner identity affect their perception of LLM-generated text?
• How does access to technology affect identifying and responding to LLM-generated text?
• What constitutes the continuum of meaningful written messages?

Again, these are just a few of the possible approaches to this topic, and there certainly are more worth investigating from the perspective of language teaching and learning. Knowing the answers to these questions or at least considering them in the context of increased LLM tool development and usage will undoubtedly serve to help learners navigate these tools.

Final Thoughts

As someone who is exposed to a large amount of unedited, auto-generated text, I can also say that authenticity will continue to matter for readers. When presented with the “blurry” feeling of auto-generated text (Chiang, 2023), it further fortifies the fact that writing will remain a valuable communication skill. Regardless of our sentiments on the topic of LLM text generators, language teachers will need to integrate them into their teaching toolkits. Doing so will help learners to better engage with these tools in a thoughtful way that will leverage the benefits and minimize the pitfalls that are slowly becoming more apparent.

References

Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? Proceedings of the 2021 ACM conference on fairness, accountability, and transparency (pp. 610—623). Association for Computing Machinery. https://doi.org/10.1145/3442188.3445922

Chiang, T. (2023, February 9). ChatGPT is a blurry jpeg of the web. The New Yorker. https://www.newyorker.com/tech/annals-of-technology/chatgpt-is-a-blurry-jpeg-of-the-web

Kumar, R., Eaton, S. E., Mindzak, M., & Morrison, R. (2023). Academic integrity and artificial intelligence: An overview. In S. E. Eaton (Ed.), Handbook of academic integrity (2nd ed., pp. 1–14). Springer. https://doi.org/10.1007/978-981-287-079-7_153-1

Levine, S. (2023, February 22). Vanderbilt apologizes for using ChatGPT in email on Michigan shooting. The Guardian. https://www.theguardian.com/us-news/2023/feb/22/vanderbilt-chatgpt-ai-michigan-shooting-email

Orland, K. (2023, September 8). Record-breaking Super Mario Bros. speedrun approaches robotic perfection. ARS Technica. https://arstechnica.com/gaming/2023/09/record-breaking-super-mario-bros-speedrun-approaches-robotic-perfection/

Ryan Morrison is a cross-appointed English language and Communications professor at George Brown College in Toronto.His research interests focus on technology and its effects on effective teaching and learning.