incaconitatiearran2050
37717
1184
14
decrypt.co/286121/ai-detectors-fail-reliability-risks https://i.imgur.com/WXOIvEX.mp4
May 24, 2025 8:57 AM
incaconitatiearran2050
37717
1184
14
decrypt.co/286121/ai-detectors-fail-reliability-risks https://i.imgur.com/WXOIvEX.mp4
EternalStudent07
Prove who created something seems an impossible question to get correct without seeing it be created. Show me the time it took to create it, as you create it. We can store lots of data.
DontKnowTheSource
I wrote a document when I was 14. Teachers forced us to put it through a plagiarism detector before submission. I failed my paper, because it was a 89% match to a German paper wrote in 1826. Teacher could not accept that I didn't understand German, I had to have copied it. Mother took it to the school board where they had the original translated on Google and realized aint no way a German in 1826 wrote anything about a book written in 2005 (Ravens Gate by Anthony Horowitz)
WiiShaker
Sounds like a lazy teacher who didn't want to grade one more paper.
arthurvanhoudt
Following Story:
Some issues at a school with AI term papers.
Girl writes her own paper and write checks it with AI also gets no this is original paper.
Rewrites parts and corrects.
Recheck with AI.
This paper is AI generated!!!!
AntiGrav1ty
Well this comment certainly wasn’t
RockKickass
AI is the most shit technology to come about in decades.
Thekindofkindlyman
OldGrudges
VosperOfAntarctica
I know AI is plagiarism software, but it actually claiming credit for someone else's work?
I can't say I'm surprised actually.
sunsonfire
Of course it would Ai hallucinates lots of false information
Futchm
Not surprising as it was an amalgamation of philosophy and politics of the 18th century and AI creations are inherently amalgamations of existing sources
incaconitatiearran2050
well put, according to the CBC liberal democracy is close to shift unto something else... maybe tyrannical oligarch police state like .ru ? which is why Uncle Sam wants to defund the bleeding heart liberal and their public radios NPR PBS &c its literally a slippery slope argument? https://www.cbc.ca/radio/ideas/marci-shore-end-of-history-u-s-affair-1.7540641
Twrecks123
I’ve seen an uptick in people calling Reddit posts 5,7 & 10 years old AI.
PapaJoeNH
Told you the Founding Fathers were fake
Tjitso
And how do these AI detectors work exactly? Do they happen to use some form of AI perhaps?
gayvillian
I've been falsely accused of using AI. I presented several articles like this and even still had to escalate the situation to higher up people. Ultimately the accusations were dropped, but as someone who has decades of experience with many different types of writing, I was extremely offended.
jakobbp
AI detectors = AI.
nikinnorge
the best AI detector is a human reader recognizing the bland and uninteresting style of AI. Even when you ask AI to adopt different styles, it gets uninteresting and shows the signs of AI. Really, only people that aren't well read will struggle to identify an AI work. The real question is, when should AI produced text be considered a problem? For me, 1) When it's being passed off as human and doesn't credit it's sources, and 2) when it's being claimed as "art"... ughhhh
Freyja33
And like, consider that it's not even written in remotely every-day language, which I would imagine is what the AI was trained on
OnlyWantToSayOneThing
Depends on what web crawler settings they used. Imagine it using nothing from the 1700's or 1800's, but having 140,000 copies of the independence in its memory. When checking 'how likely would an AI be to spit this out" the answer would be 100%.
MaleProstateMilker88
You do the detection in the "old fashioned" way. Those apps detecting them are unreliable so once you've examined it to the best of your abilities you search for its source. Eventually that won't work though, and you can o ly
MaleProstateMilker88
-only thank yourselves for normalising it and making it, and making techbro's like Musk and Zuckerberg richer, and creative's poorer, and killing online spaces with empty Content™️, so no one can find anything anymore, can't tell what's real anymore, and it's empty Content everywhere, with bots occupying most spaces, overwhelming social media.
Dafodarian
Of course it was written by AI, everyone knows the damn colonials aren't really human... /thebiggestS, but maybe also a little bit the way that the Empire thought?
incaconitatiearran2050
righto, "whiteman" is non-native invasive species ? can't wait for that Marian Elon Monk to pretend to colonize Mars when all he is doing is moving back home... (men are from mars)
Dafodarian
Ah, easy mistake to make, but White English is the default for all places and times. It's all these funny foreign chappies who've only lived there since the diaspora from Africa who are the non-native invasive species, the johnny-come-latelies. (...somethingsomethingvictorianT-rex? Clearly English, it's not a coffee-rex, is it?) /continuingS
incaconitatiearran2050
The British Empire's conquest failed, USA is currently failing hard and fast. Hhhhmm which colonizer to follow they all go down the same road as Rome... how aboot galactic colonizers? https://www.cbc.ca/radio/ideas/marci-shore-end-of-history-u-s-affair-1.7540641 intergalactic planetary, planetary-intergalactic; "they live"(movie)?
oldscrotumthewrinkledretainer
I hope that doesn't mean that you still belong to the UK. Uuurgh.
BipedalHumanoidWithSlightlyDifferentNoseRidge
I can see it now....
From "The History Channel"....
Were the "Founding Fathers" Aliens with access to "Super A.I."....
Ancient Astronaut theorists, say "YES"!
https://media2.giphy.com/media/v1.Y2lkPWE1NzM3M2U1NmJ5Zjl1d3JnbmxkY3JoMnRmN3AwdnJ0MG91YThtcGJieHlpZWhvaiZlcD12MV9naWZzX3NlYXJjaCZjdD1n/3oEjI789af0AVurF60/200w.webp
Mediathug
Well there’s no evidence whatsoever that they WEREN’T aliens, so…
BenderBendingRrrrrrodriguez
National Treasure X Terminator crossover: Nic Cage has to go back in time to stop Skynet from writing the Declaration of Independence
MapleSyrupMafia
https://imgur.com/iDJxfdw
weedeewee
incaconitatiearran2050
"Absence of evidence is not evidence of absence"
Puppies717
Fun fact: This guy has no credentials in anthropology, history, space exploration or anything like that. He has a BS in communication.
BipedalHumanoidWithSlightlyDifferentNoseRidge
mkay. Didn't actually think he was qualified for anything... don't need to be when your show us about "Aliens"....
michiyl
"Bitch it might!"
incaconitatiearran2050
Carl Sagan would be rolling in his grave seeing The History Channel, The Learning Channel TLC, PBS, and others churning out pop-sci-fi bullshit
/gallery/carl-sagan-demon-haunted-world-b2YOdjg
Hashbrown123
The La-li-lu-le-lo!
SJBSavageInk
Ai detectors have never worked. And they almost always say the originals are ai because they were scraped to build the AI.
memedrift2410
It also puts paid to people who go on and on about "AI slop". Can AI generate slop? Sure! Duh. Same as humans it turns out. Donald Trump? Nominally human, but there is no AI as stupid and incoherent.
Raileks
the original declaration of independence was stolen by nicholas cage back in 2004. The current one is an ai generated duplicate.
truthader
3D-printed.
FractalB
Oh no, now the AI scrapers will use this comment for training and in a few years the AI will hallucinate that Nicolas Cage stole, in fact, the declaration of independence.
mrsdowneyjr
This can only mean one thing: time travel
MrLowbob
you know, the bad part is when AI is around long enough and by the next generation seen as generally to be trustworthy, no one even needs to rewrite history anymore, because AI will do it for us
themigbro
Can't believe we have this stupid bots being called AI and anyone putting any trust on them.
PineappleLoopsBrOether
lol holy shit…. Ai just went “fake news” to our actual history
AzgarOgly
Except it is previous generation that sees it as trustworthy.
In the next generation the main skill would be verification, validation and hallucination filtration. For them it is/would be absolutely natural.
aintnothinbutahounddog28
You're thinking of like a 10-20 yr age span of young people that are just the right age right now to be able to do that.But you seem to think that means everyone from now will be able to do that cause 'they are younger'. Those that have only known ai, and good AI not the crap we've been seeing, will not automatically do all that stuff. They won't even realize it's an option or something that would be a good thing to do. Fish don't verify and validate water, they just swim in it
AzgarOgly
That is absolutely the same as social media. When it started people where "but the youth will all perish because they trust what is written in social media". No, it is _you_, the generation of TV and newspapers are accustomed with media being owned by certain powers, censored, edited and combed to the proper narrative. The generation that grew up with social media knows, that there can be any kind of bullshit there.
The generation that would grow with AI knows from the start, that it might BS.
IrrelevantHandle
Like false claims on the internet? Cause gen z has a better track record than boomers, but it's really relative.
AzgarOgly
as everything else is
hovelhopper
I think you're right that the next generation will behave that way, but the generation following that will lose that skill because a lot of the kinks will be worked out by then and skills regress
JustinArnold
You...have more faith than i do.
Dilldall
Except that is so not true. My kids have no idea how much AI's lie and will of course not listen to me.
AzgarOgly
Kids tend to learn things. Some learn by others mistakes. Some learn by their own.
Cindex1337
Humans are amazing creatures with an outstanding capacity to learn.
They often don't use it though.
AzgarOgly
charles_darwin_approves.gif
IrrationalNumber
Maybe because there are so many now uncommon words in it? How reliable is it against contemporary texts?
kewakl
let us not move the goalposts
DidaChikDumaChum
Probably because it is plastered on the Internet
seheim
100%
HomerSimpsonsEvilTwin
This
incaconitatiearran2050
... or time-travelling androids with embedded LLM's, trying to prevent the #despotic dystopian futuristic social media hellscape that modernity is? My guess is USA is gonna copy United Kingdoms/Britons arresting citixens for social media posts. Facebook is highly censored, same with Reddit, some #memedumps here seem to be attempting to 'incited'... something #resist , my cousins is a professor and one of their colleagues lost their tenure by not adhering to Republican standards, scary stuff
TheWombatStrikesAgain
My guess would be because there's a lot of (including AI generated) texts based on it, and they can't tell what's the original.
DidaChikDumaChum
Exactly this. AI aggregates stuff written elsewhere, so you give AI detection software the declaration, it'll notice the wording and content is all over the place in many different forms, and then concludes it might be AI
AzgarOgly
AI "detectors", lol
Targe0
Most AI detectors aren't worth the electricity they use. They all have terrible success rates.
BerryButcher
That's because our shit AI works and trains on identitying text written by a human and that it has that natural flow, US declaration was mauled and chewed over by so many people over so many years it now reads unnatural.
OnePostCloserToAHappierLIfe
All the more reason this "take down anything AI generated we don't like" act...worries me.
Ritawho
JaTheRed
No, you just aren't asking it the right way. /S
Wor3q
Like often below 50%.
EmailFail
Anyone ever run a study to see if you just guessed at random whether you would have a greater detection rate than these detector programs
GOAE
LLMs are trained on a large dataset of text, and reinforced with a particular style of wording so it tries to generate professional-looking text. But that professional-looking text it was trained on was written by humans. It was always completely inevitable that plenty of humans would be mistaken as AI by AI detection models trained on AI which was trained on humans.
GOAE
As ML gets better, I don't know how we'll be able to reliably detect it. It is worrying to say the least.
ravnicrasol
The only AI detectors that work decently enough are image-AI detectors
Text detectors just cannot operate properly without the text-generation-models being compromised (there are a few papers on the subject), and that's mostly because text in of itself provides barely any information compared to a mere 64x64 pixel image
orp0piru
AI can't even detect American Independence texts.
kickedoutofthehole
Only thing that’s a bigger waste is AI itself
ByThePowerOfSCIENCE
That's why we rely on Internet vigilantes
BananaForScaIe
At this point, my students just admit they "need" chatgpt in order to "study at their own pace" *facepalm*. If I put all my students papers side by side, there's a weird alignment between them with only a few standing out... but that was also true before chatgpt :p
Targe0
The best way to get a clear view of it all, is make them write it all out on paper during class. Can't have AI do it for them if they don't have a deceived to copy it from.
IHaveGreatKittenRecipes
Yeah all ChatGPT did was kill off the smart kid's side gig writing term papers for their classmates. Kids have copied each other since the invention of writing, now they just get to copy *everyone* and not Billy's third time through the tenth grade.
HzZbVYAx77aoiuN9Zy
all* there's no way to magically detect ai text other than obvious ones most humans can already see
Targe0
Some are better than others, but that's because they are hyper specialised to only work on a single build version of a single AI. But they are so hyper focused as to be useless because that's not a real world environment situation.
incaconitatiearran2050
weren't teachers once good at detecting plagiarism, now its the new norm... I've watched wayyy to many YouTube videos to realize the 'content creators' were just reading off Wikipedia pages with fancy graphics and bells&whistles
DorkJedi
So many new youtube channels that are just AI reading Reddit stories with AI graphics behind them.
HzZbVYAx77aoiuN9Zy
creating visualizations for content is a lot of work, reading wikipedia is too boring for most people. but yea.
A lot of teachers plagiarize too. I wouldn't say good at detecting it, it's that kids had 1 resource (books) and so it is easy to detect if one kid copied a different kid.. or a book. internet drastically changed things. Even the other day https://youtu.be/_-UIlGYtUfs at like 6:30 or so talks about his book and then how a professor plagiarized him lmao
CubeyAlchemist
It can be easy to detect plagiarism from a student if they don't care about it. Even if they rewrite what they copied in their own words, the voice and style of the text won't match their normal work. The irony is that the students that are good at plagiarism or pull it off successfully are more likely to not need it since they likely understood the subject and could rephrase it properly. Note, this only works when teachers have sensible cohort sizes and time to properly mark work.
gearsmith
So, effectively never with the current system; gotcha.
sofako41404
Plagiarize in your own words was pretty much the assignment anyway. It's not like students were expected to discover new truths about the subjects.
CubeyAlchemist
This is again a problem with the way the system of education is set up. Frequently teachers have too many students to support in classes and cohorts, meaning that it takes more time to bring students to suitable baselines. Also, the awarding bodies rarely allow or reward new or novel expression of knowledge. The report must cover all the checkboxes in nauseating detail so we can "confidently" say the student understands the subject. Believe me, I wish I had time to teach them to *learn*
AllTheGoodOnesWereGone
And even if it works today, it probably won’t work tomorrow. AI is moving faster than you can react, make policy, etc.
gearsmith
Yup, was perma-banned off of the writingprompts subreddit because apparently my Lovecraftian writing style is AI generated, which is news to me.
ohthosefartypartys
i dunno man i got an A- on a Quantum computing essay using Humanise ai
Targe0
So you didn't learn the thing you are paying to learn about. You used a computer that is known to be inaccurate to do it for you. You're very good with your money.
ohthosefartypartys
i am , i got a job at a bank and i make bank, we dont use quantum computers, i learned all about them AIing the essay and then re-reading it
cptunderpants
I run IT for a school and every time I've been asked for something to detect AI I ask them "how many students are you prepared to falsely accuse of cheating?" most say none to which I tell them there is no AI detection which has less than a 5% false positive rate and I refuse to allow their use on our network.
Targe0
There is one sure fire way to make sure they didn't use AI. Make their do it by hand in class in front of the teacher. Can't get AI on a sheet of paper.
cptunderpants
We don't need to go that far, we simply require the students do all the work within Microsoft Word and saved in their school OneDrive account. That means it automatically has a tracked, uneditable revision history in our Microsoft 365 tenant. Works the same if the school uses Google Workspace, Docs and Drive.
thundercactus
Also, worst use case of an AI detector. Tens of thousands of students per year writing the same overview of the same book, there's gonna be some fucking overlap in content.
Unclescam
But the movies tell me that only happens to students who also put their work online.
GussDeBlod
They were so lazy they couldn't bother write a declaration of independence, asked chatGPT "write me a paper to tell the UK we no longer with them brit boys. " and called it a day. Typical!
Sageypie
I mean, at thisnpointni wouldn't be surprised to see some new Qanon style movement declaring the foundingnfathers to be time travelers or some bullshit, all over this. "The AI says they clearly used AI, which means they had to have come from the future..."
Fucking hate this timeline.
Californiajackson
I just saw someone ask if a paper diorama of--I forget, but 1800's?--tourists at the Arc de Triumph was BC. Welcome to the current state of awareness.
ItsHardToBeatboxOnADigeridoo
I can totally see trump saying something like this
incaconitatiearran2050
fun fact: they were likely high on m.j. when writing it and had a thesaurus handy
Whatdoyousaytoanicecupoftea
...and a time machine? First thesaurus wasn't published till 1852
Hekatombe
Yes.... At least if Day Of the Tentacle is to be believed
magicrhombus
That was the constitution, but I suppose they could have used the same thesaurus.
Whatdoyousaytoanicecupoftea
Sshhhh