Unlike many adults, teen students are not hesitant to explore, experiment, and embrace artificial intelligence, with nearly two-thirds admitting to having done so, according to a recent Common Sense Media report.
Although many of them report using it for research purposes, the reality is often quite different, with students embracing it for a range of other reasons, from cheating on school work to creating deepfakes.
Some students have also turned to AI chatbots for companionship, with a few even falling into romantic relationships. Others have even taken AI into their confidence for unvetted mental health support, which has occasionally ended tragically, with chatbots encouraging, and even guiding, student plans for self-harm.
In many of these situations, parents, teachers, and other adults have no idea how deeply a student has become involved with an AI chatbot, or even that they’ve formed an interpersonal relationship with an AI-generated character on a platform such as Character AI or Replika.
“Even if you don't have a child who's using one of those, it’s possible for them to end up in a relationship with chatbots as common as ChatGPT or Gemini,” says Kerry Gallagher, Assistant Principal for Teaching and Learning at St. John’s Preparatory School in Danvers, Massachusetts, and Education Director at the nonprofit ConnectSafely, which is dedicated to online safety, privacy, security and digital wellness.
Understanding Social Penetration
Chatbots are primarily a commercial enterprise designed to be engaging, so it’s not surprising that a student – especially vulnerable during adolescence when they might be struggling socially, feeling lonely, or working through mental health or self-image issues – could fall under one’s sway.
“An AI chatbot will answer or respond with the output in which it refers to itself in the first person,” says Gallagher. “So that shifts the way the user is thinking about what they're interacting with because if the output is ‘I …’ then that feels like a personal conversation. It also refers to the user as ‘you.’”
Add into that how more refined the voices used by chatbots have become, and how easy it is to interact with through earbuds, and it can feel like a student is talking with a real, caring human.
“The voice intonation, the mood, the way it shows kindness and empathy, and the way that comes across–what you’re really talking about is something called social penetration theory, which is an intentional design that's coded into the algorithm,” says Gallagher. “The AI companions are programmed to slowly increase the intensity of small talk in order to draw the user in more and more. We see that in the bottom of the output, ‘Do you want me to do this next or this next?’ It starts asking the kind of adjacent questions that humans would ask as part of their conversation.”
Managing Student-AI Relationships
When it comes to navigating these situations, awareness is always the first step. “Parents should not hear about this and rush to lock down their kids' phones,” says Gallagher. “They just need to be informed about what's available on those phones so that they can be talking to their kids about it.”
In that vein, Gallagher notes that it’s sometimes a challenge to recognize that a student has gotten into an unhealthy relationship with an AI because the signs might be harder to spot since most kids tend to be on their phones constantly. But if they’re in their bedroom alone with the phone and the door closed for hours on end, it might make sense to check.
“Swing by their room to check in, ask what they’re up to in a caring way, and listen with genuine curiosity,” says Gallagher. “No need to barge in, but if it's quiet for a long period of time, what are they doing? If they're talking to friends on FaceTime, you can hear that, right? Do you know the voice of the friend they’re talking with? If not, can you ask to introduce yourself and say hello?”
It can be a challenge for educators to address student-AI relationships, especially if it primarily is happening outside of class time. Nonetheless, Gallagher suggests expanding AI awareness by exploring it yourself.
“It's important for adults to be using AI so that when we talk about it, it’s coming from experience,” she says. “If you're not sure how to start, talk to a colleague who is using it and ask them to show you what they're doing.”
Failing to understand AI companions may invalidate what may already be an important relationship in the student’s life, which is not going to help a child who made a poor decision.
“Part of our job as educators is to hold kids accountable, and also to counsel kids so that they don't make those same mistakes again,” Gallagher says. “If adults only tell kids that AI chatbots aren't real and their relationships aren't real, it could result in children doubling down on those relationships.”
When directly dealing with a student engaged with a chatbot, Gallagher suggests focusing on the reasons behind the relationship.
“I would say, ‘Tell me more about your chatbot. When do you talk to it? What is its name? What are some of the things that you talk about?’” she says. “Because I want to get into why they use the chatbot. What need is it meeting? How can we help the student meet that need in a healthier way?”
Gallagher says understanding the underlying why of what’s happening is key.
“By asking them more about their AI chatbot and their relationship, what we uncover is what needs that relationship with that chatbot is meeting that aren't being met right now, and how we can then share that information about that need with the family or school counselor. If that need of that child is met, they're less likely to feel like they need the chatbot and they're also less likely to feel like they need to do harm to their peers or themselves.”
Boosting AI Literacy
In general, educators should aim toward using AI tools that allow them to supervise what students are doing and so they can monitor for any concerning usage.
Gallagher also advocates for better AI literacy, but as tools have evolved, so should the discussions and activities with students.
“Educators can ask kids to develop their own ideas, their own lists, their own guidelines around when AI use is helpful and when it might be harmful,” she says. “They know more than we think. Even students as young as second or third grade are ready to start the conversation about which tasks require human effort so learning can happen.”
Doing so can help peel back the curtain so students can understand what AI actually is and how it works, and use it in a healthy manner.
“Start as simple as teaching that AI is programmed for pattern recognition and help kids understand, especially at that younger age, what that means,” Gallagher says.
She also recommends talking about understanding prompt engineering and verifying outputs with a human eye.
“I think it's okay to ask AI for advice as long as there's a corroboration of that advice,” she says. “If students ask AI to help brainstorm ideas or for initial feedback on an email that they've drafted, that's all fine. It still needs to be run through a human before it becomes something that is integrated or sent.”
Finally, to help boost critical thinking around AI, educators should consider changing their approach to assignments.
“For example, one of the benchmarks might ask students how they used AI. What was the output? Show the actual output, and then show the next version to demonstrate the important critical student thinking. What did the learner do between those two versions? We can help students practice adding that human filter after the AI is used.”
