SoylentNews is people
SoylentNews
Can European regulation rein in ill-behaving algorithms?
Until recently, it wasn't possible to say that AI had a hand in forcing a government to resign. But that's precisely what happened in the Netherlands in January 2021, when the incumbent cabinet resigned over the so-called kinderopvangtoeslagaffaire: the childcare benefits affair.
When a family in the Netherlands sought to claim their government childcare allowance, they needed to file a claim with the Dutch tax authority. Those claims passed through the gauntlet of a self-learning algorithm, initially deployed in 2013. In the tax authority's workflow, the algorithm would first vet claims for signs of fraud, and humans would scrutinize those claims it flagged as high risk.
In reality, the algorithm developed a pattern of falsely labeling claims as fraudulent, and harried civil servants rubber-stamped the fraud labels. So, for years, the tax authority baselessly ordered thousands of families to pay back their claims, pushing many into onerous debt and destroying lives in the process.
[...] Postmortems of the affair showed evidence of bias. Many of the victims had lower incomes, and a disproportionate number had ethnic minority or immigrant backgrounds. The model saw not being a Dutch citizen as a risk factor.
[...] As the dust settles, it's clear that the affair will do little to halt the spread of AI in governments—60 countries already have national AI initiatives. Private-sector companies no doubt see opportunity in helping the public sector. For all of them, the tale of the Dutch algorithm—deployed in an E.U. country with strong regulations, rule of law, and relatively accountable institutions—serves as a warning.
The hope is the European Parliament's AI Act, which puts public-sector AI under tighter scrutiny, will ban some applications (like law enforcement's use of facial recognition) and flag something like the Dutch Tax's algorithm as high-risk. Nathalie Smuha, a technology legal scholar at KU Leuven, in Belgium, summed it up:
"It's not just about making sure the AI system is ethical, legal, and robust; it's also about making sure that the public service in which the AI system [operates] is organized in a way that allows for critical reflection."
Originally spotted on The Eponymous Pickle.
(Score: 1, Interesting) by Anonymous Coward on Friday May 13 2022, @01:01AM
What has always bugged me about that Target story is that you're never told how many mailings went out to people who weren't pregnant. That one story got held up as the poster child for businesses to throw lots of money into data mining, but if nine other people got the same mailings and they were not expecting, then that isn't very impressive. It is easy to remember the interesting outliers, but dangerous to think of them as the mean, since they are the ones you remember.
It isn't much different than the old scam where you mail out free stock tips to 1024 houses, where in half of them you predict a stock (or I suppose more appropriate for these days, the price of bitcoin) to go up in value, and the other down. Then the next week you mail out another tip to the 512 houses where you predicted it correctly. The the next week you mail out to the 256 house, etc. By the time you get near the end, you then offer your (expensive) services to the last one (or two or four) because they're convinced how good you are because every week you've sent them tips that were correct. Not saying the Target story was a scam (though when Marketing people push stories like that, I'm usually on my guard for suspecting at least a lot of exaggeration), but that Target story is so memorable that it is the one you'd remember, and not the other angry fathers who yelled at the marketing department because the daughter wasn't pregnant.
We're many years removed from that, and I can tell you that I am not too terribly impressed with the items that get recommended to me by various algorithms. (A few years back YouTube kept showing me ads for products/services to stop smoking, where I nor anyone in my household have ever smoked a day in our lives!)