Google Search has fully integrated AI through a feature called AI Overviews. For some time now, if you phrase your query as a question such as, “When is the next train from Brewster to Grand Central?” the tool will tell you the times the train is leaving . . . except I wouldn't trust that information.
According to one recent analysis conducted for The New York Times, Google AI Summaries are accurate about 9 out of 10 times, which tracks with my experience. Whenever I ask a series of questions around a subject I'm well-versed in, I almost always find minor errors and often notice outright falsehoods, not to mention the aforementioned inaccuracies with train schedule-related questions in my area.
This led me to realize that Google’s AI summaries can be a great tool for teaching AI literacy and giving students tangible insights into AI hallucinations, limitations, sourcing, and more.
To this end, I’ve designed a short assignment. I ask students to use Google’s AI Summary tool for a subject they know well. Then they have to ask it to provide a bio of them or information on their school or the time of their local train or when their favorite sports team is playing. I require that students keep querying until they find falsehoods, which can sometimes take a little time. Then I ask them to verify the sources and explain why and how AI got it wrong.
Though I’ve only recently implemented it, I find this AI exercise enforces the lessons discussed below.
1. Google AI Overviews (And AI Outputs In General) Need Verification
This is an obvious lesson, and one that students already know in theory, but in practice, I find they don’t apply it. Students, and many people in general, will insist something is correct because AI told them, even though they know AI gets things wrong frequently.
Forcing students to confront AI accuracy with this assignment can cut through some of the cognitive dissonance. It’s harder to dismiss AI mistakes when those mistakes are about you or something you know very well, such as your town or school.
2. Source Verification
When I asked Google what nonfiction author David Grann’s next book is, it said he is writing a new book about the Nobel Oil empire. When I clicked on the source to verify this, I was led to Grann’s social media page. As I explored, I realized he was not working on these books but had posted about books on these topics from other authors.
Having students go through a similar process with source verification provided by Google’s AI summary can teach them not only to better spot AI falsehoods but also about source verification more broadly. Even with human-generated work, sometimes when you go back to a cited source, you find it didn’t frame the claim the same way or, in some cases, that the author is making claims the source material can’t support.
Therefore, this type of fact-checking is a valuable skill for AI verification and beyond.
3. AI's Inner Workings
The mistakes this type of tool makes can hint at larger problems and concerns with AI. Any student who knows how to use the internet can look up train times and report back on arrivals and departures.
Powerful AI tools know how to search the internet with an efficiency beyond what any human could dream of achieving, yet these also can’t be trusted with the same simple task. This happens because AI engines don’t verify facts the way a human might. Instead, most current-generation models predict phrases and statements through pattern recognition. This has led some to refer to them as “stochastic parrots,” though that terminology is sometimes derided, correctly, I believe, as an oversimplification.
Either way, the fact that many AI models can’t be trusted to perform a relatively simple task is a good starting point for discussions about the strengths and limitations of these tools, and a direct example of how something about how these operate very differently than humans.
4. Learning When And How To Use AI
Despite all this, I’m not advocating for students to avoid using Google’s AI Overview or some other AI tool. When Google's AI Overview works, it can be a helpful. In fact, one of the main AI uses I’ve found personally useful is as an alterative to a search engine.
Google AI Overviews often get me the information I’m looking for quicker than a traditional search might, and I like seeing what it comes up with before I embark on a traditional search. I’ve just learned not to trust it and always to verify what it says, even on small, seemingly trivial items, such as when the next train is arriving at the station.
Finding these uses and learning the limitations of AI is something important for students to realize as well. Whether we like it or not, we're all interacting with AI content frequently, and becoming more skilled at critically absorbing that content is a necessity for everyone.
Engaging in this type of conversation with students has helped me better focus my own AI use, and I hope has done the same for them. Now, I just need someone to help me figure out my local train schedule.
