Submitted via IRC for AndyTheAbsurd
n February 2013, Eric Loomis was found driving a car that had been used in a shooting. He was arrested, and pleaded guilty to eluding an officer. In determining his sentence, a judge looked not just to his criminal record, but also to a score assigned by a tool called COMPAS.
Developed by a private company called Equivant (formerly Northpointe), COMPAS—or the Correctional Offender Management Profiling for Alternative Sanctions—purports to predict a defendant's risk of committing another crime. It works through a proprietary algorithm that considers some of the answers to a 137-item questionnaire.
COMPAS is one of several such risk-assessment algorithms being used around the country to predict hot spots of violent crime, determine the types of supervision that inmates might need, or—as in Loomis's case—provide information that might be useful in sentencing. COMPAS classified him as high-risk of re-offending, and Loomis was sentenced to six years.
He appealed the ruling on the grounds that the judge, in considering the outcome of an algorithm whose inner workings were secretive and could not be examined, violated due process. The appeal went up to the Wisconsin Supreme Court, who ruled against Loomis, noting that the sentence would have been the same had COMPAS never been consulted. Their ruling, however, urged caution and skepticism in the algorithm's use.
Source: https://www.theatlantic.com/technology/archive/2018/01/equivant-compas-algorithm/550646/
(Score: 3, Informative) by Runaway1956 on Sunday January 21 2018, @07:00PM (8 children)
I skimmed over it. No way I would even answer the questions. FFS, they want you to incriminate yourself, as well as psychoanalyzing yourself for them. Fall back on the old "Never talk to the police!" mentality. Never, ever give the opposition ammunition with which to execute you.
I'm going to buy my defensive radar from Temu, just like Venezuela!
(Score: 3, Informative) by wonkey_monkey on Sunday January 21 2018, @08:00PM (6 children)
If you'd actually read the questionnaire, you would have realised that it's not given to the convicted person. What did you think? That they asked the convict if he was going to be good in future?
systemd is Roko's Basilisk
(Score: 3, Insightful) by SparkyGSX on Sunday January 21 2018, @08:39PM (2 children)
If YOU had actually read the questionnaire, instead of accusing someone else of not reading it, you would have realised it IS actually given to the convict.
For example, on page 7:
"The next few statements are about what you are like as a person..."
On page 8:
"The next statements are about your feelings and beliefs about various things. Again, there are not 'right or wrong' answers. Just indicate how much you agree or disagree with each statement"
And there are no right or wrong answers? Really?!?
"A hungry person has a right to steal"
"When people do minor offences or use drugs they don't hurt anyone except themselves"
"When things are stolen from rich people they won't miss the stuff because insurance will cover the loss"
I'm getting a feeling there are at least "desirable" and "undesirable" answers to those questions.
If you do what you did, you'll get what you got
(Score: 4, Insightful) by sjames on Sunday January 21 2018, @09:13PM
Not to mention a few of "those" questions. "When people do minor offences or use drugs they don't hurt anyone except themselves". I presume they mean illegal drugs. They want a yes/no question when only an essay will suffice. Single guy smokes a joint then goes to bed, no problem, so answer true. Guy smokes crack when watching the baby, answer to question false. Shoplifts a pack of gum, false. Jaywalks at 3 A.M. with zero traffic, true. Throws trassh in the gutter, false unless the street sweeper is going to collect it 10 seconds or so later.
Like Mitch Hedberg's stand-up "Have you ever tried sugar....or PCP?"
(Score: 3, Interesting) by frojack on Monday January 22 2018, @12:00AM
If YOU had read what you accuse others of NOT reading you would have noticed it switches mid stream (at about question #31) from an assessor's list of questions to a defendant's list of questions.
Who knows how they finesse that change.
Like bullet lead analysis, this whole thing seems like junk science.
No, you are mistaken. I've always had this sig.
(Score: 2) by Runaway1956 on Sunday January 21 2018, @10:13PM (2 children)
Actually, the questionaire is in sections. The first couple sections are for cops, or corrections officials, it seems, then the other sections are for the defendant. Wonder why there isn't a section for significant people as well - wife, mother, siblings, ex-wife, etc. Everybody gets a say, and sentencing will be democratic? Sounds fun, doesn't it?
I'm going to buy my defensive radar from Temu, just like Venezuela!
(Score: 1) by tftp on Sunday January 21 2018, @11:52PM (1 child)
Much fun. If the esteemed Mr. Eric Loomis was to stand the truly democratic trial by the whole population, he'd be sentenced to hanging. The supply of thieves would be exhausted in no time. If someone believes that it's too harsh, the alternative could be 500 years in uranium mines on Charon.
(Score: 2) by maxwell demon on Monday January 22 2018, @09:16AM
And there would be a huge supply of punished innocent people because the general public is generally quite fast in forming their opinion, typically without taking all (or often, any) facts into account.
The Tao of math: The numbers you can count are not the real numbers.
(Score: 3, Interesting) by JoeMerchant on Sunday January 21 2018, @09:34PM
I can see this as a defense selected option to allowing the judge full discretion on sentencing. Sure, it's no better a than random people, but you know some judges "have it in" for certain profiles of crimes and offenders, so if the defense has the option to go with the algorithm, hopefully they are bright enough to only take that option when they know the judge will do worse.
🌻🌻🌻 [google.com]
(Score: 2) by bradley13 on Sunday January 21 2018, @07:00PM (2 children)
Stuff like this absolutely must be open source. It's basically calculating the guy's sentence. Sounds like the judges involved are clueless where software is concerned.
Everyone is somebody else's weirdo.
(Score: 2) by Bot on Sunday January 21 2018, @08:26PM (1 child)
There probably is a predetermined combination, in the answers that can be influenced by the accused guy, that makes the algorithm come up with the least pessimistic prediction.
Even if no backdoor were involved, anybody who can test the algorithm can come up with a good combination to minimize the damages.
All are equal before the law... after the law, problems arise.
Account abandoned.
(Score: 2) by frojack on Monday January 22 2018, @12:04AM
Well clearly there is a boat load of questions, mean to be asked of a defendant, and not all of them are likely to be meaningful, and might just be there so that when a critical question comes up they answer of of habit, their mind being numbed by the tiresome questioning.
No, you are mistaken. I've always had this sig.
(Score: 5, Interesting) by AthanasiusKircher on Sunday January 21 2018, @07:11PM (17 children)
There absolutely is a due process concern here. Defendants must be able to interrogate the evidence used against them. This algorithm is purporting to provide evidence justifying longer or shorter sentences, but the defendant has no ability to examine said evidence. If the prosecution called an expert witness to claim that a particular offender was more likely to re-offend and therefore should be given a longer sentence, the defense would have the opportunity to cross-examine said witness and question how the witness came to those conclusions.
The equivalent, in this case, seems to me to require disclosure of the algorithm's inner workings.
And in this case, as noted in the headline, the proprietary "predictive algorithm" is no better than random people on the internet who have access to a much more limited set of information about offenders. And, as noted in TFA:
So, in other words, if you ask random dudes on the internet and provide them only 7 pieces of info instead of a 137-item questionnaire, the dudes on the internet get an accuracy rate of 63% in predicting reoffenders; 67% if you pool the dudes' predictions. A simple algorithm with ONLY **2** pieces of data predicts at 67% accuracy.
Meanwhile, super-duper proprietary algorithm COMPAS predicts at 65% with 137 pieces of data. Seems to me that said algorithm should never be allowed to be used in court until its workings are disclosed completely -- and even then, it doesn't sound particularly useful.
(Score: 4, Interesting) by Runaway1956 on Sunday January 21 2018, @07:26PM (4 children)
But - teats on a boar hog!!
"Useful" is in the eye of the beholder. A get-tough-on-crime judge can put you in prison for 100 years for a minor crime, and justify it with, "The data and the algorithm say that this man is dangerous!" Likewise, it can be used by the leniency crowd to justify lax sentencing. It's not terribly unlike patents with "on a computer". People who little understand technology are happy to mislead you, using technology as an excuse.
And, I suspect that, as usual, this "algorithm" will be applied in different way according to a suspect's wealth, social connections, skin color, and more. I didn't notice if there was a question, "Do you, or have you ever, lived in a government housing project?"
I'm going to buy my defensive radar from Temu, just like Venezuela!
(Score: 0) by Anonymous Coward on Sunday January 21 2018, @07:36PM (1 child)
Yes your mother was quite the sow wasn't she.
(Score: 0) by Anonymous Coward on Sunday January 21 2018, @08:31PM
As was said previously, seems to have a Porcine predilection. I wonder what our Algorithm would predict of our dear Runaway? I never saw the "male pig" thing coming!
(Score: 2) by AthanasiusKircher on Sunday January 21 2018, @08:04PM
Yes, "useful" is a vague word. What I obviously was referring to was the lack of actual predictive power compared to much more primitive metrics.
But of course people could find the mysterious authority of a computer metric to be "useful" in a lot of ways for their own selfish reasons... Regardless of whether the numbers are backed up by anything.
(Score: 2) by Arik on Sunday January 21 2018, @10:11PM
Once you swallow the initial mis-step - the idea that you can apply statistical generalizations to an individual - then this would make perfect sense. I'm sure people who have lived in projects are significantly more likely to re-offend than those that have not.
If laughter is the best medicine, who are the best doctors?
(Score: 2) by wonkey_monkey on Sunday January 21 2018, @08:02PM (6 children)
This has on effect on defendants. This applies to convicts. In that sense a convict has no more insight or access to the algorithm than they do to whether the judge got out of the wrong side of bed in the morning.
systemd is Roko's Basilisk
(Score: 2) by wonkey_monkey on Sunday January 21 2018, @08:04PM (1 child)
Dammit, "no effect," not "on effect."
systemd is Roko's Basilisk
(Score: 2) by maxwell demon on Monday January 22 2018, @09:20AM
Yeah, that was a really bad typo: When reading it, I "auto-corrected" it to "an effect" which is exactly the opposite of what you wanted to say.
The Tao of math: The numbers you can count are not the real numbers.
(Score: 3, Interesting) by AthanasiusKircher on Sunday January 21 2018, @08:23PM (3 children)
Sentencing guidelines are subject to due process concerns. In many cases, courts allow testimony to aid in determining sentencing. Although rules vary and do not always guarantee the same sorts of rebuttal allowed during a trial to witnesses, expert testimony at sentencing is subject to rebuttal. For example, a probation department may prepare a presentence report for the judge, but if said report makes claims without adequate justification or evidence, defense may attempt to rebut it.
The very point is that this is something not determined by a judge (no matter what side of bed he/she got out of) but rather is effectively expert "testimony" offered to the judge, advising on sentencing. It most certainly should be subject to review.
The standards for admissibility of evidence are much lower for sentencing hearings than trial courts, but testimony is generally still subject to a due process standard of "reliability." A metric such as this one should need to prove its reliability, given little evidence that it functions in any "expert" capacity and has stats on par with random people guessing.
(Score: 3, Insightful) by frojack on Monday January 22 2018, @12:34AM (2 children)
In the far greater percentage of cases "Victim statements" are just there to quell the rage, provide a false sense of cathartic retribution, and give victims an illusion in having some say in the process.
Its not at all clear that this testimony has any effect on sentencing (other than to reduce sentences, because, after all the victims had their say).
If you believe in "correction" you probably are in favor of victim statements.
If you believe that incarceration is to provide society some brief period of less risk, you probably find them useless for sentencing purposes.
No, you are mistaken. I've always had this sig.
(Score: 2) by AthanasiusKircher on Monday January 22 2018, @01:08AM
Victim statements are not the only testimony allowed at sentencing hearings. It depends on the venue and the judge, but sometimes you also get statements from "character witnesses" for the defense and occasionally expert witnesses. (Sentencing in death penalty cases can be particularly prolonged.)
But yes, the value of victim statements is debatable.
(Score: 0) by Anonymous Coward on Monday January 22 2018, @03:39AM
I was talking with a judge once about the Victim Impact Statements and their effect on the sentences given out. According to him, in the majority of cases, they don't change a thing as they just aren't useful or reiterate stuff he already knows from other sources. In the minority of cases, they cause him to reduce the sentence as the reaction of the defendant is one of genuine remorse or the victim makes a good case for probation modifications. In one case, however, it actually caused him to increase his sentence as the victim asked him to reduce the defendant's sentence because it wasn't the Christian thing to punish him as she forgave him, didn't believe in that sort of retribution and similar things while being moved to tears; the defendant, on the other hand, was completely unmoved and even called the victim "a stupid bitch" as she walked past to her seat. Judge decided to turn the few hundred dollars in restitution (which the victim had declined) into a fine in the tens of thousands thanks to that remark.
(Score: 3, Interesting) by frojack on Monday January 22 2018, @12:27AM (2 children)
They get to examine the evidence used to convict them.
Its FAR less clear that they get to examine the evidence used to sentence them.
There is certainly no constitutional guarantees in this regard, and the courts have changed methods dramatically over time, and it still varies greatly from federal district to federal district, to say nothing of legislatively mandated minimum sentences. I'm not sure very many people have successfully litigated their sentence because of the method used in arriving at the sentence duration. In fact that variability was one of the reasons congress started imposing sentencing guidelines upon the courts.
Methods have gone from no science, to judicial whim, to legislative mandates, and now (seemingly) its come back to pseudo-science.
I suspect SOME OF IT is at least based on documented recidivism rates, hidden in a forest of pointless questions.
But Is it dramatically worse, (or better), than looking at the color of the skin and basing the judgement on that alone?
No, you are mistaken. I've always had this sig.
(Score: 3, Interesting) by AthanasiusKircher on Monday January 22 2018, @01:22AM (1 child)
Okay, let me be clear on a couple things -- in my previous posts I admittedly mixed in some facts vs. opinions.
It is a fact that sentencing guidelines and any testimony that is part of sentencing hearings is at least theoretically subject to "due process" concerns. "Due process" is a Constitutional guarantee. You are correct that how that plays out in different venues will vary substantially.
It is a fact that a common standard for evidence introduced in sentencing hearings (particularly on the federal level) is subject to a "reliability" standard (again related to "due process"), though exempt from many of the other standards of evidence for trials. I'm not a lawyer, but my understanding is that the "reliability" standard is technically only in force in some federal circuits -- not all -- but it has been cited several times.
It is my opinion that any expert testimony -- and I would consider this algorithm making claims to be a form of "expert testimony" -- should be subject to examination regarding its reliability in a sentencing hearing.
As for "Constitutional guarantees," that's all largely a matter of interpretation. Yes, the Constitution says little about sentencing per say (other than 8th Amendment guarantees against "cruel and unusual" and stuff like that), but whether and how "due process" is interpreted in various legal procedures is a matter of interpretation... and sentencing processes have not been as thoroughly litigated. Yes, judges have often been shown broad discretion, but if specific expert testimony is being used that has no reliability or basis in fact, I personally believe that should be subject to examination under any reasonable "due process" standard.
(Score: 2) by AthanasiusKircher on Monday January 22 2018, @04:06AM
(Obviously meant "per se"...)
(Score: 0) by Anonymous Coward on Monday January 22 2018, @03:41AM
I would go farther and say that the government should not be allowed to use non-free proprietary software user-subjugating at all. Why are we letting our government's computing be so dependent upon corporations? It's dangerous for everyone.
(Score: 2) by https on Monday January 22 2018, @06:14PM
I think it would be very interesting to see what its accuracy rates are when partitioned by race.
Offended and laughing about it.
(Score: 4, Insightful) by wonkey_monkey on Sunday January 21 2018, @07:57PM (7 children)
If you could say the same thing about a self-driving car algorith, it'd be a compliment (unless by "no better" you meant "actually worse", and not the colloquial sense of "only as good as").
Being "as good as random people" is actually quite a good thing for a lot of algorithms. It means you can get the job done without hiring people, random or otherwise.
That said, what they have shown in this case is that the 137-question questionnaire this particluar algorithm is based on is almost entirely superfluous. You can get as good results just looking at age and previous convictions.
systemd is Roko's Basilisk
(Score: 0) by Anonymous Coward on Sunday January 21 2018, @08:26PM
You can get as good results just looking at age and previous convictions
So is the questionnaire just hiding that very alg?
I have seen a box of matchboxes play tic-tac-toe. No electronic computer involved. Computers just run programs. Programs do not necessarily need to be electronic.
(Score: 3, Insightful) by sjames on Sunday January 21 2018, @09:35PM
That depends. You would not let a self driving semi on the road if it was no better than random people off the street. It would at least have to be not worse than the holder of a CDL.
According to TFA, COMPAS is slightly worse than a pool of random unqualified people.
(Score: 2) by c0lo on Sunday January 21 2018, @11:33PM
Context, mate, context... No, scratch that, the relevant link is formal languages [xkcd.com]
More to the point:
Mmmm... forget self-driving cars, would you like your sentence length proceed on the base of a jury chosen at random in the day of sentencing, so that they had no opportunity to hear the evidence?
Because the above analogy seems even better than the situation of the current context: an algo hears the answer to some questions which has no relation to the case, algo which has worse predictive power than random humans.
https://www.youtube.com/@ProfSteveKeen https://soylentnews.org/~MichaelDavidCrawford
(Score: 2) by AthanasiusKircher on Monday January 22 2018, @01:35AM (3 children)
The thing is -- it still depends. TFA only cites broad accuracy statistics, which are all in the ballpark of 63-67% in terms of predicting recidivism.
But that's not the only thing that matters for an algorithm like this. What about the >1/3 of cases where it predicts WRONG? There are errors that are "reasonable" and then there are errors that are bizarre or inexplicable. We've all seen examples of machine learning algorithms where it sometimes gets things SPECTACULARLY WRONG in some cases, where it's inexplicable how the algorithm could make such an error.
And that should also be a strong consideration here. It's one thing to say it got things "right" 2/3 of the time, which is on par with other metrics or algorithms or informed opinions that validate said "right answers." But let's say the algorithm also predicted things SPECTACULARLY WRONG 10% of the time, i.e., it flagged people as "high risk" for absolutely no apparent reason that any human can understand. Or, conversely, maybe it wanted to let Charles Manson out on parole after 10 days. Those would be serious concerns about whether we'd actually want to use said algorithm, even if overall it still gets ~2/3 correct that other metrics did. (By the way, it's also distinctly possible for it to get a DIFFERENT SET of 2/3 correct from human judges or the other simpler algorithms, which could also be a concern in some cases... e.g., it might be making some obvious errors.)
I don't get the sense from TFA that this is an issue in the case here, but it's important to consider not only overall "success rate" of an algorithm, but also whether it behaves in a way that makes some sense. Sure, there's the possibility that an apparently "irrational" algorithm could be finding a pattern that humans haven't discerned yet. But given how many algorithms have been known to fail in bizarre ways, we should probably exercise caution when dealing with people's lives.
(Score: 2) by mhajicek on Monday January 22 2018, @03:13AM (2 children)
What is the actual recidivism rate? If it's around 65% the algorithm could just return "Yes" every time and be right 65% of the time.
The spacelike surfaces of time foliations can have a cusp at the surface of discontinuity. - P. Hajicek
(Score: 2) by AthanasiusKircher on Monday January 22 2018, @04:05AM
Yep -- that's another obviously stat not discussed here. Though my impression from reading other stuff is that this algorithm generates a number on a scale rather than just "yes" or "no," so I'm actually not sure what the 65% or whatever accuracy means.
(Score: 3, Informative) by maxwell demon on Monday January 22 2018, @09:31AM
Indeed, the only meaningful information is to give both the false positive and the false negative rate, separately.
The Tao of math: The numbers you can count are not the real numbers.
(Score: 3, Informative) by turgid on Sunday January 21 2018, @08:57PM (6 children)
Several years ago I wrote a little program to predict the results of the National Lottery.
Back in those days, there were 49 balls (numbered 1 to 49) and seven were picked out at random by a machine (philosophical point: it may have effectively just been highly non-linear but I'm sure thermal effects would have really made it properly random). One of the balls was designated the "bonus ball." This was to provide some nuance in the statistics which my brain is now too old and clapped-out to understand.
Like a true snake oil salesman I set about designing my highly secret algorithm and I did win the lowest prize (£10) a couple of times (three balls).
I'm not sure how much money I lost over all.
But how can you predict the outcome of a completely random game? Well, I had lots of people telling me why I couldn't and not one of them could understand what my program did or why it wasn't wrong, despite not predicting the results accurately.
I refuse to engage in a battle of wits with an unarmed opponent [wikipedia.org].
(Score: 0) by Anonymous Coward on Sunday January 21 2018, @09:16PM (1 child)
You can't predict the outcome of random drawings, but you could come up with numbers that are infrequently picked.
That way if you get all 7 numbers you won't have to share your prize with all the other idiots that picked their birthdays.
(Score: 4, Funny) by turgid on Sunday January 21 2018, @10:22PM
That's why I chose Jesus's birthday instead of mine.
I refuse to engage in a battle of wits with an unarmed opponent [wikipedia.org].
(Score: 2) by c0lo on Monday January 22 2018, @12:53AM (1 child)
Well, the proposed class algorithm category... absolutely ground-breaking.
I just can't wait** for the new field of research be recognized as a stand-alone discipline.
---
** seriously, I simple cannot wait
https://www.youtube.com/@ProfSteveKeen https://soylentnews.org/~MichaelDavidCrawford
(Score: 2) by turgid on Monday January 22 2018, @07:51PM
I've been working on them for years. I figure I'll be able to take on all the major commercial software vendors. Did I tell you about my own sorting algorithm, Donkey Sort? It's nearly as bad as Gnome Sort. I figured that the Americans might be interested in my lottery prediction program for screening felons. Their justice system is crying out for modernisation and digitalisation.
I refuse to engage in a battle of wits with an unarmed opponent [wikipedia.org].
(Score: 2) by maxwell demon on Monday January 22 2018, @09:43AM (1 child)
From the description, I guess you are talking about the German lottery.
If so, the "bonus ball" (Zusatzzahl, additional number) was basically there to give additional winning categories: You made six crosses, and the highest win was getting all six "normal" numbers right. The next one is of course to have only five of them right, but if the one number which was not in the six winning numbers happened to be the extra number, you won more than if it wasn't.
If I recall correctly, a similar split was done for the lowest winning condition, three balls right.
About your philosophical point: It certainly was an ergodic (highly chaotic) system, and thus minimal differences in the initial conditions would be exponentially amplified. Not only would thermal fluctuations affect the outcome, probably even quantum uncertainties would be amplified enough to affect the results.
The Tao of math: The numbers you can count are not the real numbers.
(Score: 2) by turgid on Monday January 22 2018, @10:37AM
The UK National Lottery. In theory, a sufficiently large experiment should be able to detect any systematic bias. However, I've never had the time or brain power to attempt to work out how many observations you'd need to get a statistically significant result. It's complicated by the fact that there are several sets of balls and machines. I suspect that the number is so large that you'd need more observations than are possible during the predicted life of the universe, or at least I would hope so, or it's a flawed system.
I refuse to engage in a battle of wits with an unarmed opponent [wikipedia.org].
(Score: 2) by The Archon V2.0 on Monday January 22 2018, @08:34PM
Anyone watch Max Headroom? Where the cops had a program that linked an arrestee's personality profile to the highest match among the profiles of criminals they hadn't caught?
https://www.maxheadroom.com/index.php?title=Episode_ABC.2.1:_"Academy" [maxheadroom.com]
The damn thing even has three of the same letters in its acronym in the same order. 20 minutes into the future indeed.