Can AI Detect AI? UNLV’s AI Podcast Takes A Closer Look

A UNLV student demonstrated that artificial intelligence (AI) detectors were unable to distinguish fact from fiction in various writing samples as part of a class project.

During a recent episode of The Real BS About AI podcast, cohost Bob Soulliere shared how an undergraduate student tested three documents in GPTZero and several other AI detectors as part of a research project. AI fully generated the content for one document, another blended AI-generated and human writing, and the last one was written entirely by the student.

The results? “They were all widely inaccurate,” Soulliere said.

At the same time, faculty concerns about students using AI inappropriately remain valid, and many instructors want to understand what the university is doing to address AI detection.

Detectors Missed Detection During Testing
Over the last few years, UNLV’s information technology team has actively tested a range of AI detectors and continues to monitor developments in this space. However, the team hasn’t found a solution that meets the university’s standards yet, according to Soullliere.

In fact, cohost Alethea Inns reinforced those findings, experimenting with different writing styles to see whether genre impacted detection results. Her experience was consistent with what the student found.

"I'm clocking on average about 49 percent [human]," Inns said after detectors flagged her original writing to be mostly AI-written.

Inns explained that these tools are designed to err on the side of caution, which can result in false positives and unintended consequences for students who may have a specific writing style that fits a pattern.

“I can’t invest in tools that have a high probability of falsely identifying student work as AI-written,” Soulliere said. “That puts the student immediately in this place that has very little recourse.”

What Faculty Can Do Now
The hosts also emphasized that paying for third-party detection tools doesn't guarantee data protection. Tools that aren't vetted or supported by UNLV may store or reuse student work, raising concerns about privacy and intellectual property.

“Without somebody’s consent, you’re taking somebody’s work. That information is now being stored, processed, and distributed in different ways without a student’s knowledge,” Soulliere said.

Rather than relying on imperfect AI detection tools, the hosts encourage faculty to engage students directly when questions arise. Conversations can clarify understanding in ways a score cannot. Faculty can also get support from the Teaching & Learning Commons, which offers resources on meaningful assessment and responsible AI use.

In her observations, Inns said that most students want to succeed academically and are using AI responsibly to brainstorm, refine, edit, or enhance their work. Students also expect the same transparency from faculty, as instructors increasingly use AI for course content and grading.

A Focus On Ethical, Fair, and Reliable Tools
Looking ahead, there’s optimism about the future of AI at UNLV. The hosts reiterated the commitment to explore detection tools that are ethical, fair, and reliable.

“It’s a priority for us to actively investigate these tools,” Soulliere said. “When we can find one that is ethical and fair to our students and equitable for you as a faculty member, know we’re going to be all in. We want to make sure we’re helping you deliver on your academic goals.”

The Real BS About AI publishes new episodes every two weeks, blending humor, accessibility, and practical advice to make artificial intelligence approachable for the UNLV community. View the AI Technology webpage for past podcast episodes or more information about AI at UNLV.

Short on time? Grab The Afterwards, a quick read about the episode, complete with prompts and notes you can copy and paste for your own use and workflow.  

Content for this story was drawn directly from the podcast and has been converted to this platform with the assistance of AI. Our communications team verifies all content for accuracy.