SoylentNews is people
SoylentNews
Can European regulation rein in ill-behaving algorithms?
Until recently, it wasn't possible to say that AI had a hand in forcing a government to resign. But that's precisely what happened in the Netherlands in January 2021, when the incumbent cabinet resigned over the so-called kinderopvangtoeslagaffaire: the childcare benefits affair.
When a family in the Netherlands sought to claim their government childcare allowance, they needed to file a claim with the Dutch tax authority. Those claims passed through the gauntlet of a self-learning algorithm, initially deployed in 2013. In the tax authority's workflow, the algorithm would first vet claims for signs of fraud, and humans would scrutinize those claims it flagged as high risk.
In reality, the algorithm developed a pattern of falsely labeling claims as fraudulent, and harried civil servants rubber-stamped the fraud labels. So, for years, the tax authority baselessly ordered thousands of families to pay back their claims, pushing many into onerous debt and destroying lives in the process.
[...] Postmortems of the affair showed evidence of bias. Many of the victims had lower incomes, and a disproportionate number had ethnic minority or immigrant backgrounds. The model saw not being a Dutch citizen as a risk factor.
[...] As the dust settles, it's clear that the affair will do little to halt the spread of AI in governments—60 countries already have national AI initiatives. Private-sector companies no doubt see opportunity in helping the public sector. For all of them, the tale of the Dutch algorithm—deployed in an E.U. country with strong regulations, rule of law, and relatively accountable institutions—serves as a warning.
The hope is the European Parliament's AI Act, which puts public-sector AI under tighter scrutiny, will ban some applications (like law enforcement's use of facial recognition) and flag something like the Dutch Tax's algorithm as high-risk. Nathalie Smuha, a technology legal scholar at KU Leuven, in Belgium, summed it up:
"It's not just about making sure the AI system is ethical, legal, and robust; it's also about making sure that the public service in which the AI system [operates] is organized in a way that allows for critical reflection."
Originally spotted on The Eponymous Pickle.
(Score: 2) by DeVilla on Thursday May 12 2022, @12:24AM (2 children)
Machine learning only learns if you tell it when it's wrong and/or when it's right. If you don't tell it when it's wrong, it'll assume it got it right. Just like a kid
... or a dog.
(Score: 2) by HiThere on Thursday May 12 2022, @03:16AM (1 child)
The thing is, a lot of the AI algorithms are explicitly designed to NOT learn when they're out in the field. So even if the users of the system gave it feedback this might well not have changed the results. We'd need to know a bit more about this particular example, but that might be the way to assume it was designed.
Javascript is what you use to allow unknown third parties to run software you have no idea about on your computer.
(Score: 2) by JoeMerchant on Thursday May 12 2022, @02:06PM
TFS says it was a learning algorithm... implies that the data it was fed over time influenced its later outcomes.
🌻🌻 [google.com]