Would your campus host credit card companies or Big Tobacco? Then why Big Tech?

Last week, my university held a wellness day for students: a day without tests, homework, or classes. Introduced in 2022 in response to campus mental health challenges (including the COVID-19 pandemic), wellness days are held once a semester “to provide staff, faculty and students an opportunity to reflect on their health and well-being.” As an English professor and academic advisor, my role relative to wellness day is to help our students access the mental health resources and opportunities for connection available to them on campus. 

Since the pandemic, things have gotten better, but the bleakness of those years still feels quite vivid to me. It was harrowing to work with students facing mental health emergencies and heartbreaking when students confided that they didn’t have a friend on campus. While I still regularly work with students struggling with mental health concerns, I now find that many students are more aware of the support services available.

But lately I’m concerned about another threat to our students’ mental health. Since the rollout of publicly available generative AI chatbot products, we have growing evidence that these products have negative mental health impacts. Nevertheless, NC State is welcoming representatives from Google Gemini to spend two days on our campus,  October 13th and 14th, touting their products to our students as part of their “college tour.” 

The more we learn about these products, the easier it is to recognize their threat to mental health. Our current perceptions of AI products are heavily influenced by Big Tech marketing. For example, the term “artificial intelligence” is a marketing slogan, not an accurate definition. As many scholars point out, large language models (LLMs) and other AI products do not think and are not capable of making meaning or communicating with intention. They use a variety of computational and linguistic models trained on data (much of it stolen) to predict a pattern of text that is likely to address the user’s prompt.

By incorrectly suggesting that LLMs and other “AI” products possess “intelligence” or the potential to think like human beings, we devalue human beings. We also devalue the human work of thinking, reading, and writing. When our students are given the opportunity to develop these skills, they also develop confidence in their own voices and abilities, a deeply important part of mental health and wellbeing. When we minimize the worth of these skills and devalue the labor required to develop them, we take away the opportunity for our students to develop this confidence, expertise, and persistence. 

The stability of our students’ mental health is also threatened by the false sense of emotional safety LLM chatbots have been designed to create (see the ELIZA effect).  Because tech companies are trying to sell the idea that these products are “intelligent” and “human,” they are created to simulate social interaction (for example, through sycophancy and the use of first person pronouns and other anthropomorphizing language).

While there are increasing cases of these products causing intense emotional attachment that leads to eventual distress, a more widespread concern is the dependence and unearned trust that chatbots engender. The false emotional safety created by chatbots can encourage users to assign too much credibility to generated outputs. Because of this, users may be more easily swayed by poor quality outputs and false information.

Additionally, a recent literature review suggests that use of chatbots could lead to an erosion of social skills called “social deskilling.” Also, these “conflict-free” AI interactions may be less satisfying than human interaction: a study found that texting with a random human did more to reduce loneliness than texting with a chatbot. Encouraging our students to use generative AI products may actually undo some of the difficult work we’ve done to reduce loneliness in our student population.

Generative AI products pose additional threats to mental health including perpetuation of societal biases like racism and chatbot-related suicides and murders.  Researchers are also concerned about the use of products like ChatGPT for mental health therapy. A study conducted at the Stanford Institute for Human-Centered AI found that chatbots encouraged suicidal ideation and enabled delusional and dangerous lines of thought. The study also found that chatbots perpetuated cultural stigma about certain mental health conditions and that this problem did not improve as LLM models progressed, contradicting the constant claims from tech companies that models will continue to get better with more training.

Like many other universities, NC State is introducing generative AI products into workspaces, classrooms, and learning management systems, but, to my knowledge, they are not informing students and parents of potential risks. We need to be honest about the ways that these products could undermine our mental health progress, and we should also acknowledge that the high financial costs of these products may take funding away from other services for our students, like more counselors, teachers, tutors, and advisors. More human beings with more time and energy to authentically engage with our students might be exactly the thing they need. 

I’ve been working with the Genuine Intelligence Project, a grant-funded initiative of the North Carolina conference of the AAUP, to ensure that universities like mine consider the needs of faculty and students when making decisions related to generative AI products. We’ve surveyed students and faculty and sponsored critical AI literacy events, and  now we’re speaking out against Gemini’s college tour, which seems intended to push these unsafe products on our students.

We no longer allow credit card, tobacco, and alcohol companies to target students on campus; I suspect one day we’ll look back and recognize that Gemini’s presence is not much different. If they’re serious about protecting the mental health and wellbeing of our students, NC State’s leaders will say no to Gemini on our campus. 

Elizabeth Wagner is a lecturer and academic advisor at North Carolina State University.