Stories
Slash Boxes
Comments

SoylentNews is people

posted by Fnord666 on Monday July 17 2017, @03:27PM   Printer-friendly
from the swear-on-a-stack-of-K&Rs dept.

At The Guardian, Cathy O'Neil writes about why algorithms can be wrong. She classifies the reasons into four categories on a spectrum ranging from unintential errors to outright malfeasance. As algorithms now make a large portion of the decisions affecting our lives, scrutiny is ever more important and she provides multiple examples in each category of their impact.


Original Submission

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 2) by meustrus on Monday July 17 2017, @04:18PM (6 children)

    by meustrus (4961) on Monday July 17 2017, @04:18PM (#540367)

    Also (who RTFAs around here?) apparently the article is actually about AI, which is NOT the same thing as algorithms. The summary is shit for equating the two.

    Even so, the answer is still the same: have a suite of automated tests (but with more variance and tolerance) that verify expected results in every known edge case, and try as hard as you can to eliminate the unknowns. If you do it right, your tests will then describe invariant expectations about the outcome.

    --
    If there isn't at least one reference or primary source, it's not +1 Informative. Maybe the underused +1 Interesting?
    Starting Score:    1  point
    Karma-Bonus Modifier   +1  

    Total Score:   2  
  • (Score: 2) by The Mighty Buzzard on Monday July 17 2017, @04:27PM (5 children)

    Until the AI learns to tell the difference between actual use and testing and lie selectively...

    --
    My rights don't end where your fear begins.
    • (Score: 1, Informative) by Anonymous Coward on Monday July 17 2017, @04:44PM

      by Anonymous Coward on Monday July 17 2017, @04:44PM (#540382)

      I just want to chime in here, you are presuming the existence of a true AI and then you presume the AI would have some motivation to lie about software testing. These observations point to only one obvious conclusion: you are an idiot.

    • (Score: 0) by Anonymous Coward on Monday July 17 2017, @04:46PM

      by Anonymous Coward on Monday July 17 2017, @04:46PM (#540384)

      Just don't let VW train the AI. :-)

    • (Score: 2) by HiThere on Monday July 17 2017, @04:49PM (1 child)

      by HiThere (866) Subscriber Badge on Monday July 17 2017, @04:49PM (#540386) Journal

      You are making assumptions about its goal structure. It shouldn't *want* to tell the difference. If an AI lies, it's because it has been trained to lie, probably via the specifications. This isn't the same as being biased which is impossible to remove, but only to minimize...and then only if you realize that your training data is biased. But if, say, the specs say that you need to get a certain percentage approved, then the net can (will probably) learn to lie in order to meet that goal. This is an error in the specifications of the goal...and unfortunately it isn't uncommon.

      --
      Javascript is what you use to allow unknown third parties to run software you have no idea about on your computer.
      • (Score: 2) by maxwell demon on Monday July 17 2017, @07:07PM

        by maxwell demon (1608) on Monday July 17 2017, @07:07PM (#540487) Journal

        then the net can (will probably) learn to lie in order to meet that goal.

        No. Learning to lie would mean the net would learn what the correct response would be, but decide to give another response. While in reality the net will work out how to meet the given goal, and therefore it will give what, according to its programming, is the right answer, even though it is not the right answer according to what we actually want. Or in other words, the network is not lying, it is misinformed.

        --
        The Tao of math: The numbers you can count are not the real numbers.
    • (Score: 2) by meustrus on Monday July 17 2017, @06:23PM

      by meustrus (4961) on Monday July 17 2017, @06:23PM (#540450)

      Normally you would isolate the learning data from the testing data, and only an isolated copy of the AI will be used on the test data and them terminated. The AI will have never experienced the test data before.

      There are two situations where this breaks down:

      1. Improper isolation. Some organizations, when lacking sufficient data to come to a good conclusion, will pool all of their data together in order to train the AI the best it can and ignore verification. This is what the NSA did with its SKYNET program, resulting in ultimately unverifiable results. [qz.com]
      2. Active collusion from motivated humans, a la VW. If there is an incentive for a human to cheat, the human could easily do so. The situation you imagine is possible - teach the AI to know when it's being tested in order to hide its bias. It would be magnificently difficult to teach the AI how to lie selectively depending on a variable it isn't designed to be aware of, sort of like teaching a monkey to behave differently depending on the presence of a magnetic field. Sure, it can learn the concept, and may even gain access to a tool that can measure it, but it would be a lot easier (and probably less traceable) just to train it with the test data, as in #1 above.
      --
      If there isn't at least one reference or primary source, it's not +1 Informative. Maybe the underused +1 Interesting?