Stories
Slash Boxes
Comments

SoylentNews is people

posted by Fnord666 on Saturday July 04 2020, @10:14PM   Printer-friendly
from the garbage-in-garbage-out dept.

MIT apologizes, permanently pulls offline huge dataset that taught AI systems to use racist, misogynistic slurs:

MIT has taken offline its highly cited dataset that trained AI systems to potentially describe people using racist, misogynistic, and other problematic terms.

The database was removed this week after The Register alerted the American super-college. MIT also urged researchers and developers to stop using the training library, and to delete any copies. "We sincerely apologize," a professor told us.

The training set, built by the university, has been used to teach machine-learning models to automatically identify and list the people and objects depicted in still images. For example, if you show one of these systems a photo of a park, it might tell you about the children, adults, pets, picnic spreads, grass, and trees present in the snap. Thanks to MIT's cavalier approach when assembling its training set, though, these systems may also label women as whores or bitches, and Black and Asian people with derogatory language. The database also contained close-up pictures of female genitalia labeled with the C-word.

[...] Vinay Prabhu, chief scientist at UnifyID, a privacy startup in Silicon Valley, and Abeba Birhane, a PhD candidate at University College Dublin in Ireland, pored over the MIT database and discovered thousands of images labelled with racist slurs for Black and Asian people, and derogatory terms used to describe women. They revealed their findings in a paper [pre-print PDF] submitted to a computer-vision conference due to be held next year.

[...] The key problem is that the dataset includes, for example, pictures of Black people and monkeys labeled with the N-word; women in bikinis, or holding their children, labeled whores; parts of the anatomy labeled with crude terms; and so on – needlessly linking everyday imagery to slurs and offensive language, and baking prejudice and bias into future AI models.

Antonio Torralba, a professor of electrical engineering and computer science at CSAIL, said the lab wasn't aware these offensive images and labels were present within the dataset at all. "It is clear that we should have manually screened them," he told The Register. "For this, we sincerely apologize. Indeed, we have taken the dataset offline so that the offending images and categories can be removed."

In a statement on its website, however, CSAIL said the dataset will be permanently pulled offline because the images were too small for manual inspection and filtering by hand. The lab also admitted it automatically obtained the images from the internet without checking whether any offensive pics or language were ingested into the library, and it urged people to delete their copies of the data:

[...] Giant datasets like ImageNet and 80 Million Tiny Images are also often collected by scraping photos from Flickr or Google Images without people's explicit consent. Meanwhile, Facebook hired actors who agreed to have their faces used in a dataset designed to teach software to detect computer-generated faked images.

Prabhu and Birhane said the social network's approach was a good idea, though they noted academic studies are unlikely to have the funding to pay actors to star in training sets. "We acknowledge that there is no perfect solution to create an ideal dataset, but that doesn't mean people shouldn't try and create better ones," they said.

The duo suggested blurring people's faces in datasets focused on object recognition, carefully screening the images and labels to remove any offensive material, and even training systems using realistic synthetic data. "You don't need to include racial slurs, pornographic images, or pictures of children," they said. "Doing good science and keeping ethical standards is not mutually exclusive."


Original Submission

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 2, Insightful) by RandomFactor on Sunday July 05 2020, @12:41AM (5 children)

    by RandomFactor (3682) Subscriber Badge on Sunday July 05 2020, @12:41AM (#1016329) Journal

    It's been a hell of a six months.

    --
    В «Правде» нет известий, в «Известиях» нет правды
    Starting Score:    1  point
    Moderation   +1  
       Insightful=1, Total=1
    Extra 'Insightful' Modifier   0  

    Total Score:   2  
  • (Score: 0) by Anonymous Coward on Sunday July 05 2020, @03:51AM (4 children)

    by Anonymous Coward on Sunday July 05 2020, @03:51AM (#1016376)

    Indeed it has. But to my way of thinking the Democrats have been a relatively minor part of the problem. YMMV.

    • (Score: 0) by Anonymous Coward on Sunday July 05 2020, @09:19AM (1 child)

      by Anonymous Coward on Sunday July 05 2020, @09:19AM (#1016458)

      Are the instigators a minor part of the problem? Depends on the point of view.

      • (Score: 0) by Anonymous Coward on Sunday July 05 2020, @08:56PM

        by Anonymous Coward on Sunday July 05 2020, @08:56PM (#1016647)

        The Democrats bringing impeachment charges against Teh Rodonald? HOW DARE THEY?

    • (Score: 4, Insightful) by rleigh on Sunday July 05 2020, @10:47AM (1 child)

      by rleigh (4887) on Sunday July 05 2020, @10:47AM (#1016473) Homepage

      I'm British rather than American. But looking in the news at what's going down right now in Seattle, New York, and other Democrat-run cities, I'd have to say it looks like they are a very large part of the problem to me. They are failing to look out for the interests of the law abiding majority who pay their taxes and run businesses, work productively and live peacefully in favour of anarchists and minority interests, and are actively preventing the police from enforcing law and order in the face of rioting, looting, murder and lawlessness. It doesn't look like a recipe for success and quality of life, does it? Where are all the democrats shouting for an end to this, defending the police and calling for law and order. Deafening silence. They encouraged all of this. Actively and intentionally. Look at what the mayor of Seattle actively permitted and encouraged in the "CHOP"/"CHAZ" these past few weeks. An utter disgrace which is ending in tears, as could only be expected when you permit anarchists to rule several city blocks tooled up like warlords, shaking people down and dealing out arbitrary justice. Who was responsible for the health and safety of the residents and the businesses there? The Democrat city administrators, who let them down completely. Strange that it went from "peace and love" to "violent shithole" in just a few weeks... Well, not really, that's what anarchy gets you every time. Maintaining law and order is what we elect people for. I wonder if the individuals responsible will face legal action and justice for what they did here? As for New York, do they really want to turn once of the best cities on the planet into Detroit mk II?

      • (Score: 0) by Anonymous Coward on Sunday July 05 2020, @04:52PM

        by Anonymous Coward on Sunday July 05 2020, @04:52PM (#1016536)

        ISA/Socialist Alternative USA: Socialists Are Beating Jeff Bezos in Seattle (Again) [socialistalternative.org].

        On July 1, the Seattle City Council voted 7-2 in its Budget Committee for an Amazon Tax that would raise more than $200 million annually. The tax will apply to the top 3 percent of corporations, with Amazon paying the largest share, and will go toward a major expansion of affordable housing in a city with sky-high rents and a long history of racist gentrification.

        The victory was the result of determined class struggle by a democratically organized grassroots campaign led by [Kshama Sawant's] organization, Socialist Alternative, and a coalition of workers and progressive organizations and unions, including UAW 4121, 350 Seattle, Seattle Democratic Socialists of America, Transit Riders Union, SHARE/WHEEL, Nickelsville, Working Families Party, Seattle People’s Party, WFSE 1488, and rank-and-file Seattle Democrats.

        The Democratic Party left to its own devices gives us racialism and anarchism while the ruling class continues expropriating wealth at an astronomical rate from the working class. This of course sets the stage for the fascism that's taken root in the Republican Party. It takes socialists to show the way forward, and as it turns out, Democratic voters and even the filthy, pseudo-left DSA are receptive to it.

        I see no mention of CHOP in there. Go figure. In another article, Socialist Alternative finally expressed some frustration with the lack of interest in organizing among that so-called "organized" protest. The whole business with the police precinct handily retreating and making way for CHOP just stinks. If I put my wswsws brand tinfoil on, I would not be surprised if the intelligence agencies are involved in CHOP: the tools of regime change come home to roost.

        Anarchism is not the way forward. Anarchism has no capability of addressing the unfinished business of revolutions past. The choice is ever socialism or barbarism.