SoylentNews is people
SoylentNews
Can European regulation rein in ill-behaving algorithms?
Until recently, it wasn't possible to say that AI had a hand in forcing a government to resign. But that's precisely what happened in the Netherlands in January 2021, when the incumbent cabinet resigned over the so-called kinderopvangtoeslagaffaire: the childcare benefits affair.
When a family in the Netherlands sought to claim their government childcare allowance, they needed to file a claim with the Dutch tax authority. Those claims passed through the gauntlet of a self-learning algorithm, initially deployed in 2013. In the tax authority's workflow, the algorithm would first vet claims for signs of fraud, and humans would scrutinize those claims it flagged as high risk.
In reality, the algorithm developed a pattern of falsely labeling claims as fraudulent, and harried civil servants rubber-stamped the fraud labels. So, for years, the tax authority baselessly ordered thousands of families to pay back their claims, pushing many into onerous debt and destroying lives in the process.
[...] Postmortems of the affair showed evidence of bias. Many of the victims had lower incomes, and a disproportionate number had ethnic minority or immigrant backgrounds. The model saw not being a Dutch citizen as a risk factor.
[...] As the dust settles, it's clear that the affair will do little to halt the spread of AI in governments—60 countries already have national AI initiatives. Private-sector companies no doubt see opportunity in helping the public sector. For all of them, the tale of the Dutch algorithm—deployed in an E.U. country with strong regulations, rule of law, and relatively accountable institutions—serves as a warning.
The hope is the European Parliament's AI Act, which puts public-sector AI under tighter scrutiny, will ban some applications (like law enforcement's use of facial recognition) and flag something like the Dutch Tax's algorithm as high-risk. Nathalie Smuha, a technology legal scholar at KU Leuven, in Belgium, summed it up:
"It's not just about making sure the AI system is ethical, legal, and robust; it's also about making sure that the public service in which the AI system [operates] is organized in a way that allows for critical reflection."
Originally spotted on The Eponymous Pickle.
(Score: 0) by Anonymous Coward on Thursday May 12 2022, @05:07PM (1 child)
"The government saving money may be motive not to fix the problem, but probably isn't the cause of the problem - after all saving the government money typically only benefits the politicians who get to spend it on something else, not the the people making the individual decisions, or even their organization (unless there's also a very perverse incentive structure in place)"
Guess you haven't noticed the trend of conservative politicians to use austerity as a club to destroy social services. The wealthy are happy to let the average person suffer and die than pay more taxes. The old line, Republicans say the government doesn't work then get elected and deliberatelt make things worse. DeJoy should be in jail for the destruction of the USPS.
(Score: 3, Informative) by Immerman on Thursday May 12 2022, @06:29PM
You're not really disagreeing - Republicans are among the politicians that get to spend the money on something else (in their case usually either tax cuts for their rich corporate sponsors, or domestic surveillance programs under the guise of kicking out illegal immigrants)
The person who is actually getting paid to decide if this specific person qualifies for benefits under whatever rules the current politicians have in place though? They rarely have any direct skin in the game, nor does their immediate boss. And probably not their boss either. They're all just middle-men handing out government money that they have no way to touch themselves.