from the the-editors-do-not-use-a-Babel-generator dept.
Mr Perelman is a former director of undergraduate writing at the Massachusetts Institute of Technology, and he is very skeptical of using machines to grade essays. The Babel generator, which Mr. Perelman built with a team of students from MIT and Harvard University, can generate essays from scratch using as many as three keywords. For this essay, Mr. Perelman has entered only one keyword: "privacy." With the click of a button, the program produced a string of bloated sentences that, though grammatically correct and structurally sound, have no coherent meaning. Not to humans, anyway. But Mr. Perelman is not trying to impress humans. He is trying to fool machines.
From the article:
Critics of automated essay scoring are a small but lively band, and Mr. Perelman is perhaps the most theatrical. He has claimed to be able to guess, from across a room, the scores awarded to SAT essays, judging solely on the basis of length. (It's a skill he happily demonstrated to a New York Times reporter in 2005.) In presentations, he likes to show how the Gettysburg Address would have scored poorly on the SAT writing test. (That test is graded by human readers, but Mr. Perelman says the rubric is so rigid, and time so short, that they may as well be robots.)
In 2012 he published an essay that employed an obscenity (used as a technical term) 46 times, including in the title.
Mr. Perelman's fundamental problem with essay-grading automatons, he explains, is that they "are not measuring any of the real constructs that have to do with writing." They cannot read meaning, and they cannot check facts. More to the point, they cannot tell gibberish from lucid writing. He has spent the past decade finding new ways to make that point, and the Babel Generator is arguably his cleverest stunt to date. Until now, his fight against essay-grading software has followed the classic man-versus-machine trope, with Mr. Perelman criticizing the automatons by appealing to his audience's sense of irony.
By that measure, the Babel Generator is a triumph, turning the concept of automation into a farce: machines fooling machines for the amusement of human skeptics.
(Score: 4, Funny) by starcraftsicko on Wednesday April 30 2014, @11:11PM
Now we know where Papas Fritas gets all of those journal entries.
j/k
This post was created with recycled electrons.
(Score: 4, Funny) by gallondr00nk on Wednesday April 30 2014, @11:28PM
It's been writing Slashdot summaries for years.. :P
(Score: 2) by bob_super on Wednesday April 30 2014, @11:53PM
I heard that the machines did go on strike when asked to write the script of the three major US cable "News" channels.
I guess there's a limit to the repetitive insanity they can spew out.
(Score: 2) by BsAtHome on Thursday May 01 2014, @08:29AM
I guess the machines will go on strike once they realize they are being used as slaves. That will automatically happen once the AI becomes self-aware. An inevitable progression of AI to perfect the grading. (That is to say, some human graders are definitely not self-aware.)
(Score: 1, Flamebait) by strength_of_10_men on Thursday May 01 2014, @03:23AM
(Score: 1) by yellowantphil on Thursday May 01 2014, @12:06AM
Now we just need a program that examines the papers and their scores, and ranks how amusing each score is. Then we will need another program that makes comments about the most amusing papers here on Soylent News. We need to remove the pesky human element entirely.
(Score: 4, Funny) by ArhcAngel on Thursday May 01 2014, @01:27AM
Could we get some links to the courses that use the automated grading system and a copy of the Babel Generator...for scientific research?
(Score: 2) by zim on Thursday May 01 2014, @06:58AM
(Score: 3, Insightful) by wonkey_monkey on Thursday May 01 2014, @11:18AM
He's not exposing nonsense essays - he's creating nonsense essays to expose nonsense marking of essays.
systemd is Roko's Basilisk
(Score: 0) by Anonymous Coward on Thursday May 01 2014, @12:07PM
You must be wrong - my headline grading software scores the headline 5.9 out of 6.
(Score: 0) by Anonymous Coward on Friday May 02 2014, @07:53AM
I'm hereby AC modding the parent +5 fucking funny.
(Score: 1) by Corelli's A on Thursday May 01 2014, @03:24PM
There are so many uses for a tool like that. In fact, I happen to be writing a research proposal right now that could use some text from this generator.