Teach with expert insights on AI, curated by your trusty Teacher’s AIde

Welcome back to Teacher's AIed, where we share knowledge just like the know-it-all 5th grader sharing the definition of “antidisestablishmentarianism.”

Well, actually, today, we are not the know-it-alls (I hope we never come across as know-it-alls). Today, I’m sharing two important AI terms I hadn’t heard about.

Ever had that eerie feeling that a piece of writing was crafted by AI?

A colleague once claimed they have a "sixth sense" for AI detection. Intriguing, right?

I recall my initial encounter with GenAI through a New York Times article challenging readers to distinguish between paragraphs penned by fourth graders and those by ChatGPT.

Spoiler alert: I flunked the quiz.

Generative AI is deceptively good! This bears the question - how can teachers truly tell if something was written by Generative AI or by a real, breathing human student?

This isn’t a new question in the slightest. In fact, AI detection was a hot topic in the education world in the Spring of 2023.

There was a time when AI detection was all the rage. The train of thought was that if our conventional plagiarism detectors can identify copied content, why can’t the same system work for Generative AI?

However, AI detection proved to be less precise and faced criticism for bias against writings by English language learners.

This seemed like a proverbial nail in the coffin for AI detection, and it faded into the background.

Yet, to my surprise, AI detection resurfaced - maybe even resurrected - just recently. During a demo of an education-focused LLM, it was presented to me as a tool as part of a suite of AI-enhanced products.

This piqued my curiosity:

• What exactly do AI detectors look for?

• What triggers our AI "sixth sense"?

The representative mentioned two intriguing terms: burstiness and perplexity. 

If these terms are new to you, join me on this exploratory journey.

1. Burstiness and Perplexity

Burstiness and perplexity relate to the randomness in writing.

Perplexity measures the unpredictability of the next word in a sentence, something Generative AI calculates based on word patterns. Humans, on the other hand, introduce more randomness.

Here’s an illustrative example from a blog post I found:

Burstiness, conversely, looks at the overall stylistic shifts in a piece. 

I sometimes write extremely long sentences while utilizing — with pin-point grammatical precision (hopefully) — every punctuation mark available on my keyboard.

Sometimes not.

Over the course of an entire piece of writing, the variability in sentence structure, tone, and other stylistic elements contributes to its burstiness.

AI-generated writing tends to have a more consistent tone, so a varied tone could indicate human authorship.

2. Potential Implications for Writing Teachers

At face value, understanding burstiness and perplexity might help you develop the ultimate AI detector or at least get you some kudos at your next cocktail party.

Will our conventions of writing — or just our understanding of “good writing” — shift because of Generative AI?

3. Head’s Up: Next Week’s Posts

With next Monday being President’s Day - the Teacher’s AIed team is taking the day off from sending out our regularly biweekly newsletter.

However, don’t despair; we will be back in your inbox on Thursday with our next Premium post! Our third and final post in the AI for Special Education series will cover specific AI tools that can revolutionize how you meet the needs of your students with learning differences.

We will send out a short email on Thursday with a snippet of the premium piece to all subscribers!

Class dismissed!

Lewis Poche & Kourtney Bradshaw-Clay