The Danish Ministry of Education has developed a "digital exam invigilator" to be used by students in the equivalent of high schools ("gymnasiale uddannelser"). The purpose is to be able to detect cheating during written exams. The program:
- captures all keystrokes (keylogger)
- captures a screenshot every minute and whenever you switch tasks
- a list of all open webpages
- network configuration
- which programs are running
- whether it's is running in a VM
- contents of the clipboard
and sends this to a central server during the exam. The data is kept for 4 months.
The initiative is getting a lot of criticism.
- In 2017 there was 229 suspicions of cheating out of more than 200.000 students, so this initiative may be out of proportions.
- The program is only available for Windows and MacOS. No support for Linux or ChromeOS.
- It may be possible for a 3rd party to do a MITM-attack and take over the students' PCs.
- If a student is unable or unwilling to install the program he can perform the exam under "extended surveillance" (good old-fashioned humans watching) at the school's discretion. Some schools deny students this option and instead just fail them.
- The program will likely collect private information.
The schools do not provide computers for students because they cannot afford it. So its BYOD. On some schools (eg. some vocational schools) Linux is quite common. Some schools have trouble affording the extra human invigilators.
So soylentils: what would you do given the constraints? What do other countries do? Ignore the risk of cheating? Spend money on human invigilators?
All sources are in Danish as this news has not hit the international scene (yet). Sorry.
Danish Ministry of Education page on the program: https://www.stil.dk/uvm-dk/gymnasiale-uddannelser/proever-og-eksamen/netproever/den-digitale-proevevagt
Short analysis by security expert Peter Kruse: https://www.dr.dk/nyheder/indland/den-digitale-proevevagt-overvaagnings-kritiske-elever-faar-ministeriet-til-rette
A Reddit thread on the subject: https://www.reddit.com/r/Denmark/comments/avovqx/staten_har_nu_krav_om_at_vi_installerer_et/
A discussion on version2 (an EE and CS site): https://www.version2.dk/artikel/digitale-proevevagt-totalovervaagning-elevers-computere-midlertidigt-trukket-tilbage-1087609
(Score: 2, Insightful) by Anonymous Coward on Wednesday March 06 2019, @12:33AM (3 children)
The curmudgeon point of view: No devices in the exam room. I had to take tests this way, why should this new generation be any different?
OK, you can have a 4-banger calculator--if it's a test that requires arithmetic. If you try to sneak in something more powerful it's back to the slide rule for you, buddy.
(Score: 1, Interesting) by Anonymous Coward on Wednesday March 06 2019, @01:10AM
I had an exam where they actually gave you a theoretical subject book you had to read and understand along with the test, on a subject not real. It made sure you were actually smart enough for the certification. There was no way to cheat and 90% failed, but I passed.
(Score: 2) by isj on Wednesday March 06 2019, @01:13AM (1 child)
I believe that it was on the Reddit thread someone pointed out that their exam involved CAD drawings, and some other schools used CAS (Computer Algebra System), so you need a computer to do the exam. Paper&pen won't cut it.
Also, many schools switch from paper&pen to computers around the age of 12 because noone cares about penmanship anymore - all writing is done on computers nowaday.
(Score: 2) by jb on Wednesday March 06 2019, @06:59AM
I believe that it was on the Reddit thread someone pointed out that their exam involved CAD drawings, and some other schools used CAS (Computer Algebra System), so you need a computer to do the exam. Paper&pen won't cut it.
Then give them a keypunch and plenty of card stock ;)
Seriously though, if the exam is testing whether students know how to use a specific piece of software (which will undoubtedly be obsolete by the time those students get into the workforce anyway) rather than the principles of the subject (which, once learned, will remain useful for life), then the exam itself (not the system of administering it) has a fatal design flaw so should be rewritten afresh before considering any higher-level reforms.
(Score: 2, Interesting) by EJ on Wednesday March 06 2019, @01:12AM (7 children)
I'm not from Danishland, so my opinion doesn't really matter, but I never was allowed to use anything other than a simple calculator during tests.
If computers are necessary, then they should create a Linux live image on USB keys that they require to be used to access any course materials or even the network. That image can be as locked down as they like.
(Score: 2) by isj on Wednesday March 06 2019, @01:29AM (6 children)
I think would have been a wiser choice. A locked-down bootable USB could be circumvented by booting it in a VM and doing your cheating on the host. You also have the problem if the required tools aren't available for Linux, but that is a separate can of worms.
I'm old enough to also have had the same restrictions as you (nothing more than a calculator during exams), but I do _not_ have fond memories of the 4-hour written exam in programming (pascal) using pen&paper. Yes, that was a thing back then.
(Score: 1) by EJ on Wednesday March 06 2019, @01:36AM (5 children)
They can always just require you to let them boot the system for you, and use their own monitoring software to detect any reboots. The specific software they mentioned is not the only option. There are Linux tools if you own the image. Another consideration is LAN only. No outgoing connections.
(Score: 4, Insightful) by Runaway1956 on Wednesday March 06 2019, @01:57AM (4 children)
I'm more of the opinion that if the school requires a computer for testing, then the school should supply the computer. The Danes are doing something wrong here. Of course, it isn't just the Danes - most of the rest of the world is doing wrong in different ways.
(Score: 4, Interesting) by isj on Wednesday March 06 2019, @02:29AM (3 children)
Note: The age group affected by this are 15-19 years old.
Most schools do not _require_ the student to have a computer, but you'll be at a disadvantage if you don't have one.
Providing personal computers would not be a good idea - you can guess how they would be treated. Instead it's BYOD, but that means multiple OSes and capabilities, and students would be set back if the school provided common computers during exam (and those computers would probably only be used during exam time so ... meh.). Just ask the follow soylentils about using a government-provided keyboard... And you cannot reasonably force the students back to pen&paper because they probably haven't used those devices for the past 7-8 years.
If you try to restrict access to the internet then it can be easily bypassed with a 3G/4G/LTE card in the laptop. And apparently some exams require access to web-only CAD or CAS.
So there your have it: Students bring their own laptop. With full internet access. How to you prevent/detect cheating during exams? That's the conundrum.
(Score: 2) by c0lo on Wednesday March 06 2019, @08:59AM (2 children)
It's called open book exam [google.com].
When properly done, it's one of the hardest to take (and grade).
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 0) by Anonymous Coward on Wednesday March 06 2019, @03:49PM (1 child)
I hated open book exams. My professors designed them so that if you needed to research anything in the books you brought with you, you wouldn't have time to finish the exam. The books were meant to act as reference material where you looked up a formula or data value. Anything more and you ran out of time.
(Score: 2) by c0lo on Wednesday March 06 2019, @11:12PM
That exactly the idea. If do need to study in advance and understand the matter but don't need to bother memorising a bunch of formulas (and be afraid of not getting them right).
Pretty much as in the real professional life - when you have a concrete problem to solve, it's more important to understand the problem and plot the course of action than to actually apply formulas (understand why you are doing first, know how to do it second).
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 5, Funny) by Hartree on Wednesday March 06 2019, @01:32AM (3 children)
"digital exam invigilator"
That sounds vaguely obscene and possibly painful. Like something aliens might use on you when they capture you.
(Score: 2) by isj on Wednesday March 06 2019, @01:40AM (1 child)
Invigilator:: (British) A person who keeps watch over students at an examination.
Yes, that word was new to me too, but it best fits the exact goal of the software, so that's why I chose that translation.
(Score: 3, Interesting) by Mykl on Wednesday March 06 2019, @02:19AM
One of my best friends was an invigilator for a few years. He had a few good stories - some students came up with highly ingenious ways to cheat. Most often, they would give themselves away by looking suspicious though.
Interestingly, virtually all of the cheating students belonged to one specific culture - it appeared that cheating had become somewhat normalised in that culture and was not looked down upon.
(Score: 2) by realDonaldTrump on Wednesday March 06 2019, @06:27AM
Otherwise known as the Prostate Check. Word to the wise, get a woman doctor to do that one. Hopefully one with SHORT FINGERNAILS!
(Score: 2) by The Mighty Buzzard on Wednesday March 06 2019, @01:42AM (10 children)
I mean, it's only going to catch dumbasses and dumbasses weren't going to get a good grade to begin with, so what's the problem? Anyone with a lick of sense is just going to ssh over anywhere else and pull up lynx. Or run the browser in a VM instead of the test. And if they're smart enough to figure out how to circumvent lameass protections, they're smart enough to pass to without cheating.
My rights don't end where your fear begins.
(Score: 1) by optotronic on Wednesday March 06 2019, @02:31AM (5 children)
Smart enough, perhaps. Knowledgeable enough, perhaps not. Knowing your way around a computer doesn't mean you've mastered biochemistry or any other subject.
(Score: 2) by The Mighty Buzzard on Wednesday March 06 2019, @04:06AM (3 children)
If you're capable of finding out what you don't know within time constraints of the test, does it matter? You're at the very least going to be better at whatever you do than the dipshits who wrote this piece of garbage.
My rights don't end where your fear begins.
(Score: 2) by c0lo on Wednesday March 06 2019, @09:06AM (2 children)
What if one picks the correct answer by chance from the internet?
E.g. assuming you could now, how willing would you be to have a surgery with a doctor that passed exams like this?
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 0) by Anonymous Coward on Wednesday March 06 2019, @10:35AM (1 child)
Would you be willing to undergo surgery by a doctor whose only "experience" on that is information he digested from books or digital resources, even if that knowledge is reliably tested in exams?
I certainly would hope that whoever does their first surgery has before assisted many other surgeries led by experienced doctors. After passing that test, I wouldn't care too much whether a single question of an exam years ago was answered correctly by pure luck.
(Score: 2) by c0lo on Wednesday March 06 2019, @11:58AM
Not pure luck, mate, but cheating before picking an answer at random.
So, not a care you say? Even after knowing that, deep down inside his subconscious, that surgeon has the willingness to cheat when it comes to what he perceives is a risk to his skin in the game?
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 0) by Anonymous Coward on Wednesday March 06 2019, @07:06AM
Do you even need to know your way around? Having your sister's iPhone on the desk next to you seems to be out of the realm of possibility .. lol is my old man protecting someone elses network?
(Score: 0) by Anonymous Coward on Wednesday March 06 2019, @06:14AM (3 children)
Some of these systems will check for signs of a VM and simply not allow you to start the exam if they suspect anything odd. There are many ways that to detect the possibility of running in a VM and they only have to be "mostly" sure to get suspicious and not allow the exam to start.
When looking into this not too long ago for a school, my first thought was to have paper notes on the floor and practice looking down without looking down. Perhaps even with a foot on a few buttons of an electronic device (much like devices that aid in card counting). These systems do use a webcam to watch you and ask you to move your camera around to check the room initially, but with some creativity it wouldn't be too hard to hide something out of view.
(Score: 2) by c0lo on Wednesday March 06 2019, @09:09AM (2 children)
Yeah, right. Like it's very hard to start the exam on the host OS and fire up a VM for "research" purposes.
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 0) by Anonymous Coward on Thursday March 07 2019, @07:51PM (1 child)
No. During the exam the invigilatior service monitors a real-time screen capture, your webcam pointing at you, and the software is constantly checking the process list and other information for violations. Any software that hasn't been pre-approved by the course instructor. Presumably one invigilator is monitoring several people at once and something small might slip through for a few minutes, but anything suspicious like a new process will be flagged immediately and shown as an alert to the invigilator.
Reiterating what I said before, the technical side is quite good because they can check for anything and prompt the invigilator to intervene, and they don't have to worry about false positives just to raise an alert. Any meatspace cheating is much more likely to succeed at home, and why I question how remote invigilator can be trusted.
(Score: 0) by Anonymous Coward on Friday March 08 2019, @04:33PM
I don't understand these weird ass solutions. What I'd do is make a loader and hook all these critical function calls to display a very submissive young person taking an exam... or send pictures of goatse instead of screenshots if I'm in that mood.
(Score: 0) by Anonymous Coward on Wednesday March 06 2019, @01:57AM (1 child)
https://www.ncbi.nlm.nih.gov/pubmed/2299306/ [nih.gov]
(Score: 2) by isj on Wednesday March 06 2019, @02:02AM
The ultimate disproof of Intelligent Design.
(Score: 1) by progo on Wednesday March 06 2019, @05:30AM
So... all this fuss, and possibly corrupting or opening security vulnerabilities on the student's computer ...
And AFAICT, they still have no way to detect that the student is consulting sources paper books, or any other computer or handheld device, or a coach.
(Score: 4, Interesting) by bradley13 on Wednesday March 06 2019, @05:50AM (3 children)
Cheating, exams, ah, the tales... It is a serious problem, and internationalization doesn't help: some cultures really don't seem to see the problem. In one extreme case, I had a student ask me why I didn't distribute the answers ahead of time, like he was used to from home.
I teach programming exams, and I would immensely prefer to have students, you know, actually write programs. Solving programming problems on papers is just stupid. Years ago, I would just do this - and leave wireshark running to encourage honesty - but really, it was an honor system.
That doesn't work anymore. More and more people are going into IT, and I frankly can no longer trust my classes to contain zero cheaters. In the grading conference this week, it was literally a question: how do you catch the students who are paying someone in Eastern Europe to do all their work? Even at exams, we can check IDs, but - given large classes and bad pictures - one is still not always sure: "is this really the student, just having a bad-hair day?"
I've used a locked down Linux image that the students boot their own computers from. The Linux image is great, but you'd be amazed how many computers are incapable of booting from it (Microsoft Surface, for example). Also, if you're going this direction, then the students need to be able to bring their online references with - on another USB, and some computers have only the one USB plug. It's not as easy as it ought to be.
We have computer labs, where we can provide the computers, but that's has it's own limitation: the labs are small, because no one ever uses them - everyone has their own machine. So classes would have to take the exam in shifts, um...also not great.
Dunno what the long-term solution is, but we surely haven't found it yet. I'm giving this year's exams on paper, again...
Everyone is somebody else's weirdo.
(Score: 0) by Anonymous Coward on Wednesday March 06 2019, @10:24AM
Actually you have:
There's absolutely no need that exams are done on a computer.
(Score: 0) by Anonymous Coward on Wednesday March 06 2019, @01:37PM
I've had programming exams and also made one. All open book, on a school provided computer, but that wouldn't matter.
When the student finished the tasks you go sit beside them, look over the work together and ask some questions.
A. Some questions on the task to make sure you understand what he/she was trying to do so you can properly grade. (Make some notes that you can use afterwards when you're reviewing their code)
B. Additional follow up questions around the task to see if they also understand why they were doing some things. This also helps to refine grading further.
C. Some small quick questions on things that are in the curriculum but weren't on the test. To refine your grade even further.
If they used a East-European cheat partner, they will fail miserably at answering your questions.
(Score: 0) by Anonymous Coward on Wednesday March 06 2019, @02:05PM
I guess it depends on what the problem is. Having answers is great to verify if you are doing things correctly.
(Score: 0) by Anonymous Coward on Wednesday March 06 2019, @07:55AM (2 children)
Allow them to use internet and any and all sources they can find, you know like real life. Just limit the time to do the exam. This way clueless people who go surfing will surely lose to people who know the things asked.
I definitely wouldn't install a proprietary keylogger on my computer, even if I ran Windoze or macos.
(Score: 2) by bradley13 on Wednesday March 06 2019, @11:54AM (1 child)
Yeah, that's great: they send the problems to their paid friend in Hungary or wherever, get the answers back, and voila: an unqualified nitwit scores a top grade.
Everyone is somebody else's weirdo.
(Score: 1, Touché) by Anonymous Coward on Wednesday March 06 2019, @02:57PM
Sound like what the Ministry did when they got someone to make the program.
(Score: 1, Insightful) by Anonymous Coward on Wednesday March 06 2019, @10:22AM
Instead of monitoring the network, disallow it. It shouldn't be too hard to detect devices trying to connect to a network (they emit certain signals, after all), and if your device does so, you're disqualified.
(Score: 2) by SemperOSS on Wednesday March 06 2019, @01:04PM
To put it in context: Gymnasiale Uddannelser do correspond to the American High School, but more specifically to the senior years, i.e. the ages from 15 to 18. The best corresponding English equivalent would be Sixth Form. Gymnasiale Uddannelser are normally considered more academic than alternative (often vocational) education possibilities for the same year group and are often seen as a precursor to going to university.
I don't need a signature to draw attention to myself.
Maybe I should add a sarcasm warning now and again?
(Score: 0) by Anonymous Coward on Wednesday March 06 2019, @02:02PM (1 child)
The Windows version is a shitty clickonce .Net app, so I bundled it for a nice download of all the dependencies without having to install.
https://0x0.st/zowf.7z [0x0.st]
The student who analysed this used: https://www.jetbrains.com/decompiler/ [jetbrains.com]
This is his blog post (in danish): https://www.alexandernorup.com/DigitalProvevagt [alexandernorup.com]
This is the Mac version:
https://0x0.st/zowV.7z [0x0.st]
(Score: 0) by Anonymous Coward on Wednesday March 06 2019, @02:11PM
By the looks of the amount of people doing analysis on it in the reddit thread. This tool is going to be worthless.
(Score: 2) by DannyB on Wednesday March 06 2019, @03:17PM
Chromebooks are winning big in education.
The school fully controls the device. Including policies.
They're cheap. (School can negotiate volume pricing with all different manufacturers. I hear some even get $67 / unit.)
They're easily replaced if lost, stolen or eaten. When replaced, all your work in the cloud, and in the school district's cloud is still there.
They're secure.
Cloud Print is convenient and wireless.
To transfer files: right-click on file, pick Copy. Unplug mouse, plug mouse into other computer. Right-click, paste.
(Score: 2) by PiMuNu on Wednesday March 06 2019, @05:26PM
I got an A grade in high school physics exam without a calculator because I left it at home. Anyone worth their salt can do it in their heads - or at worst, long multiplication is not hard.
(Score: 1, Interesting) by Anonymous Coward on Wednesday March 06 2019, @06:18PM (1 child)
Our school uses Examsoft [examsoft.com]. We are required to pay a $22 license per semester for the ability to use it, and it is installed to our personal laptops. We register it to ourselves as members of our school, and the instructors then take the rolls and push out their exams to it. We download the exam file, then take the exam, then the results are uploaded.
During the test it ensures on a deep level that the computer taking the test runs the exam full screen during the test and one cannot break out of the exam mode until the exam is submitted for grading. There is also a console that a proctor can use to verify which students are actually currently taking the exam during a test, that the connection is active, and it provides analytics to the instructor after the test. It is also recommended that virus scanning be turned off while running it, and it has screwed up enough people's laptops to the level of requiring OS reinstall that it is universally reviled by the student body. This is a minority of students affected, somewhere between 1 in 20 and 1 in 60. But enough that the word has spread.
That said, I'd trust it more than their invigilator - secure the testing environment and one doesn't have to worry about keyloggers and screenshots.
Besides.... phones.
(Score: 0) by Anonymous Coward on Thursday March 07 2019, @07:48AM
... and smartwatches with predownloaded PDF's - no need for network access.