Stories
Slash Boxes
Comments

SoylentNews is people

posted by hubie on Wednesday May 11 2022, @09:21PM   Printer-friendly
from the Ministry-of-Information dept.

Can European regulation rein in ill-behaving algorithms?

Until recently, it wasn't possible to say that AI had a hand in forcing a government to resign. But that's precisely what happened in the Netherlands in January 2021, when the incumbent cabinet resigned over the so-called kinderopvangtoeslagaffaire: the childcare benefits affair.

When a family in the Netherlands sought to claim their government childcare allowance, they needed to file a claim with the Dutch tax authority. Those claims passed through the gauntlet of a self-learning algorithm, initially deployed in 2013. In the tax authority's workflow, the algorithm would first vet claims for signs of fraud, and humans would scrutinize those claims it flagged as high risk.

In reality, the algorithm developed a pattern of falsely labeling claims as fraudulent, and harried civil servants rubber-stamped the fraud labels. So, for years, the tax authority baselessly ordered thousands of families to pay back their claims, pushing many into onerous debt and destroying lives in the process.

[...] Postmortems of the affair showed evidence of bias. Many of the victims had lower incomes, and a disproportionate number had ethnic minority or immigrant backgrounds. The model saw not being a Dutch citizen as a risk factor.

[...] As the dust settles, it's clear that the affair will do little to halt the spread of AI in governments—60 countries already have national AI initiatives. Private-sector companies no doubt see opportunity in helping the public sector. For all of them, the tale of the Dutch algorithm—deployed in an E.U. country with strong regulations, rule of law, and relatively accountable institutions—serves as a warning.

The hope is the European Parliament's AI Act, which puts public-sector AI under tighter scrutiny, will ban some applications (like law enforcement's use of facial recognition) and flag something like the Dutch Tax's algorithm as high-risk. Nathalie Smuha, a technology legal scholar at KU Leuven, in Belgium, summed it up:

"It's not just about making sure the AI system is ethical, legal, and robust; it's also about making sure that the public service in which the AI system [operates] is organized in a way that allows for critical reflection."

Originally spotted on The Eponymous Pickle.


Original Submission

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 0) by Anonymous Coward on Thursday May 12 2022, @01:33PM

    by Anonymous Coward on Thursday May 12 2022, @01:33PM (#1244380)

    If the human processor overturns the machine's decision, they might have to justify their decision to their boss. And if *they* get it wrong, they get reprimanded.

    Funny story. This is how Russia invaded Ukraine, except those AI were just analysts and they tended to get reprimanded if their reports said "this scenario would not be good for Russia". So, every hypothetical scenario had to be good for Russia and they had no problems. Then, "someone" assumed the report reflected reality and you have the mess today.

    AI is just computerized "yes man" if you assume it's always right ;) But the problems with training these "yes man" predates AI by millennia.