Stories
Slash Boxes
Comments

SoylentNews is people

posted by Fnord666 on Saturday July 04 2020, @10:14PM   Printer-friendly
from the garbage-in-garbage-out dept.

MIT apologizes, permanently pulls offline huge dataset that taught AI systems to use racist, misogynistic slurs:

MIT has taken offline its highly cited dataset that trained AI systems to potentially describe people using racist, misogynistic, and other problematic terms.

The database was removed this week after The Register alerted the American super-college. MIT also urged researchers and developers to stop using the training library, and to delete any copies. "We sincerely apologize," a professor told us.

The training set, built by the university, has been used to teach machine-learning models to automatically identify and list the people and objects depicted in still images. For example, if you show one of these systems a photo of a park, it might tell you about the children, adults, pets, picnic spreads, grass, and trees present in the snap. Thanks to MIT's cavalier approach when assembling its training set, though, these systems may also label women as whores or bitches, and Black and Asian people with derogatory language. The database also contained close-up pictures of female genitalia labeled with the C-word.

[...] Vinay Prabhu, chief scientist at UnifyID, a privacy startup in Silicon Valley, and Abeba Birhane, a PhD candidate at University College Dublin in Ireland, pored over the MIT database and discovered thousands of images labelled with racist slurs for Black and Asian people, and derogatory terms used to describe women. They revealed their findings in a paper [pre-print PDF] submitted to a computer-vision conference due to be held next year.

[...] The key problem is that the dataset includes, for example, pictures of Black people and monkeys labeled with the N-word; women in bikinis, or holding their children, labeled whores; parts of the anatomy labeled with crude terms; and so on – needlessly linking everyday imagery to slurs and offensive language, and baking prejudice and bias into future AI models.

Antonio Torralba, a professor of electrical engineering and computer science at CSAIL, said the lab wasn't aware these offensive images and labels were present within the dataset at all. "It is clear that we should have manually screened them," he told The Register. "For this, we sincerely apologize. Indeed, we have taken the dataset offline so that the offending images and categories can be removed."

In a statement on its website, however, CSAIL said the dataset will be permanently pulled offline because the images were too small for manual inspection and filtering by hand. The lab also admitted it automatically obtained the images from the internet without checking whether any offensive pics or language were ingested into the library, and it urged people to delete their copies of the data:

[...] Giant datasets like ImageNet and 80 Million Tiny Images are also often collected by scraping photos from Flickr or Google Images without people's explicit consent. Meanwhile, Facebook hired actors who agreed to have their faces used in a dataset designed to teach software to detect computer-generated faked images.

Prabhu and Birhane said the social network's approach was a good idea, though they noted academic studies are unlikely to have the funding to pay actors to star in training sets. "We acknowledge that there is no perfect solution to create an ideal dataset, but that doesn't mean people shouldn't try and create better ones," they said.

The duo suggested blurring people's faces in datasets focused on object recognition, carefully screening the images and labels to remove any offensive material, and even training systems using realistic synthetic data. "You don't need to include racial slurs, pornographic images, or pictures of children," they said. "Doing good science and keeping ethical standards is not mutually exclusive."


Original Submission

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 1, Offtopic) by aristarchus on Monday July 06 2020, @12:25AM (4 children)

    by aristarchus (2645) on Monday July 06 2020, @12:25AM (#1016735) Journal

    Could be worse! Could be the Portland Police, with well known Proud Boy sympathies, putting their collective knee on the throat of the judicial system with a grant of qualified immunity.

    Starting Score:    1  point
    Moderation   -1  
       Offtopic=1, Total=1
    Extra 'Offtopic' Modifier   0  
    Karma-Bonus Modifier   +1  

    Total Score:   1  
  • (Score: 0, Offtopic) by hemocyanin on Monday July 06 2020, @12:42AM (3 children)

    by hemocyanin (186) on Monday July 06 2020, @12:42AM (#1016750) Journal

    Snide snark does answer the question: what circumstance is one living through, when a violent group uses explosives to effect political change?

    • (Score: 1, Flamebait) by aristarchus on Monday July 06 2020, @12:53AM (2 children)

      by aristarchus (2645) on Monday July 06 2020, @12:53AM (#1016754) Journal

      Pampered and cowardly American! Fireworks? Not even a pressure-cooker to produce some actual concussive force? Lucky for you that you never lived in Beruit, or Northern Ireland, or Iraq or Afghanistan, or Israel, or any place that had actual use of real explosives, instead of peaceful and colorful protests. You don't have to be a Weatherman to see that you are exaggerating. Must suck soooo much that you can't say " X lives matter!" without exposing yourself to the consequences of being a morally reprehensible person. You have my sympathy. Have you tried Prozac? Thorazine is probably a bit much, but in a pinch, till the Celebrations of American Independence are over, . . .

      • (Score: 1, Insightful) by hemocyanin on Monday July 06 2020, @01:18AM (1 child)

        by hemocyanin (186) on Monday July 06 2020, @01:18AM (#1016764) Journal

        Understood. You seek to destroy America, otherwise you would not minimize how serious it is when armed groups, 100s to 1000s strong, fire deadly weapons at people and buildings.

        • (Score: 1, Offtopic) by aristarchus on Monday July 06 2020, @02:17AM

          by aristarchus (2645) on Monday July 06 2020, @02:17AM (#1016787) Journal

          Well, not really an American, so whether it is destroyed or not seem to be American's business, except insofar as it affects the rest of the world. I have been constantly pointing out these armed groups who are attempting to overthrow your constitution, the 3%ers, the Oafkebblers, the Sagebrush Rebels, the neo-nazi alt right white supremacists, and the Republican party, and that St. Louis lawyer and his Karen, but seems no one wants to open their eyes, least of all you, hemotoma.

          Now America, after WWII, had the opportunity to be a great nation. The Marshall Plan was a great thing. Of course, the reactionary approach to the recent allies, the Soviet Socialist Republics, was really a cock-up. But the US took the lead in establishing the United Nations, the International Court of Justice (after insisting, over the objections of the Brits, that a Tribunal be held to try Nazis, and make aggressive war a crime against humanity), and also demanded that former colonies of the European powers be given a path to independence, and entered into agreements with pacific nations assuring their eventual independence. America came to have the most respected system of Higher Education in the world, and the greatest intact industrial base, but also was responsible for great and significant advances in medicine, science, and technology. And after WWII, America finally undertook to deal with its own racist history, when Eisenhower ordered the US Military to integrate.

          But now, not so much. America has pursued side-deals to keep its own war criminals out of the International Criminal Court, the permanent replacement for ad-hoc war crime tribunals. America has refused to sign off on the Kyoto Accords. America has repudiated a the Iranian non-proliferation agreement. And now America is leaving the World Health Organization. American is no longer the Shining City on the Hill, so much as a cut-rate cheap real-estate development on the Lower Eastside. So, seriously, from the point of view of the rest of the world, does not America deserve to be destroyed? No one complained all that much when its doppelganger, the USSR, fell under the weight of its own structural flaws. Why should it be different with America? I, though, still have hope that America can again stand for the ideals that are true American ideals, and none of this zenophobia, racism, and tiny-handed mismagament.