Moments before signing a landmark partnership impacting millions of teachers, our partners at the American Federation of Teachers asked us a question.
What does it mean to use your tool responsibly?
Ever since GPTZero started the AI detection movement last year, our team has been grappling with how we can continue to lead responsibly and set global standards for this technology. Our answer returns to a core mission — to restore human connection, in spaces strained by AI. It means not defaulting to teachers against students, instead, building for teachers and students navigating AI use together.
We don’t pretend to have the answer key.
But here are our first steps forward.
The Syllabus (5 steps towards responsible AI detection)
Addressing ESL bias in AI detection
Confidence scores for every result
More Interpretability
Safeguards for false positives
Entering the AI Academy
Addressing Biases
Six months ago, a Stanford university study showed that many AI detectors exhibited high false positive rates when analyzing text by ESL students. This biased false positive rate was absolutely not acceptable, and not fair to ESL students.
After learning of this problem, GPTZero immediately began sourcing text by second-language speakers and refining our model to eliminate this bias. We were the first and only AI identifier that directly addressed this problem.
When we re-ran the numbers and analyzed our progress, we succesfully mitigated detection bias, and achieved a target 1% false positive rate for ESL writing.
Confidence Scores
In this scan, you’ll notice something different than any other AI detection result. It comes with a red/yellow/green score assessing the result confidence.
Green, and you can scan with the knowledge that our platform has handled thousands of similar scans with high accuracy. For a Red score, even if the output is 100% AI, more information is recommended.
Red scores indicate that the input data is out-of-distribution. Simply put, this means that the text being scanned is not similar to writing our AI detector has seen before. As an example, while ESL writing is no longer out-of-distribution for GPTZero, it remains out-of-distribution for most AI detectors. Other domains of writing could also be remain out-of-distribution.
That’s why, even with the significant improvements to our core detection model, we were committed to launching a second confidence model.
Confidence scores are launching next week for everyone on the platform. You can think of it as adding a safety belt to AI detection.
More Interpretability
When attempting to identify AI text, an unexplained “score” or “probability” isn’t enough to help people understand the origin what they are reading. Interpretability is a pre-requisite.
As part of our commitment to interpretability, we were the first in the industry to launch sentence by sentence AI highlighting in February. We’re continuing to be at the forefront in interpretability research through our latest feature release, deep analysis.
As you can see above, the Deep Analysis view allows users to examine how specific sentences contributed to the overall AI probability.
This is just one of many initiatives to significantly improve our understanding and experiences with AI identification.
Writing Reports
Vox News recently featured the GPTZero Writing Report, showcasing how it analyzes writing patterns to give additional document insights. As a unique and complementary tool to AI detection, it serves as a power safeguard against false positives by giving writers the power to prove and certify their writing was original.
Learn more about our writing reports here.
Learning Together
The most important step towards responsibly adopting AI identification continues to be having human conversations — especially on when and where it should be applied.
In the past month alone, we joined CourseHero in running their AI Academy for teachers, and started organizing workshops and training materials with our partners at the American Federation of Teachers.
During the AI Academy, teachers were assigned to research and make their own informed decisions before introducing AI tools like ChatGPT, Bard, and … GPTZero. One-quarter of teachers entered the AI academy saying they didn't feel prepared at all regarding AI in the classroom.
By the end of the course, 92% of educators left feeling prepared.
Thank you so much for reading this update.
Overall, we are committed to continuing being a leader in both the responsible adoption of AI generation and AI detection technologies.
These are only the first steps.
Sincerely, and until next year,
Edward
Congrats on the milestone! 👏