SoylentNews is people
SoylentNews
Submitted via IRC for chromas
Trust but verify: Machine learning's magic masks hidden frailties
The idea sounded good in theory: Rather than giving away full-boat scholarships, colleges could optimize their use of scholarship money to attract students willing to pay most of the tuition costs.
So instead of offering a $20,000 scholarship to one needy student, they could divide the same amount into four scholarships of $5,000 each and dangle them in front to wealthier students who might otherwise choose a different school. Luring four paying students instead of one nonpayer would create $240,000 in additional tuition revenue over four years.
The widely used practice, called "financial aid leveraging," is a perfect application of machine learning, the form of predictive analytics that has taken the business world by storm. But it turned out that the long-term unintended consequence of this leveraging is an imbalance in the student population between economic classes, with wealthier applicants gaining admission at the expense of poorer but equally qualified peers.
[...] Financial aid leveraging is one of several examples of questionable machine-learning outcomes cited by Samir Passi of Princeton University and Solon Barocas of Cornell University in a recent paper about fairness in problem formulation. Misplaced assumptions, failure to agree on desired outcomes and unintentional biases introduced by incomplete training data are just some of the factors that can cause machine learning programs to go off the rails, yielding data that’s useless at best and misleading at worst.
"People often think that bad machine learning systems are equated with bad actors, but I think the more common problem is unintended, undesirable side effects," Passi said in an interview with SiliconANGLE.
[...] Like most branches of artificial intelligence, machine learning has acquired a kind of black-box mystique that can easily mask some of its inherent frailties. Despite the impressive advances computers have made in tasks like playing chess and piloting driverless automobiles, their algorithms are only as good as the people who built them and the data they're given.
The upshot: Work on machine learning in coming years is likely to focus on cracking open that black box and devising more robust methods to make sure those algorithms do what they’re supposed to do and avoid collateral damage.
Any organization that's getting started with machine learning should be aware of the technology's limitations as well as its power.
(Score: 1, Insightful) by Anonymous Coward on Monday February 11 2019, @12:50PM (3 children)
These algorithms will just optimize for whatever you set as the target. If you choose number of paper clips produced it will gladly choose to consume all minerals in the entire solar system in order to make unused paper clips.
https://wiki.lesswrong.com/wiki/Paperclip_maximizer [lesswrong.com]
(Score: -1, Redundant) by Anonymous Coward on Monday February 11 2019, @01:34PM (2 children)
The flaw is not in the machine but as usual, the human.
A machine that only works say 99.99% of the time has a human supervisor to avoid that 0.01% of catastrophic screwups. Problem averted, right?
But when the machine doesn't work perfectly 100% of the time, the human laziness of the supervisor always wins.
Press ENTER to continue.
(Score: 0) by Anonymous Coward on Monday February 11 2019, @02:42PM
Yea, these humans should be fired though. Programming a computer to give money to rich people but not poor people and then blaming the computer for doing that is just dumb.
(Score: 0) by Anonymous Coward on Monday February 11 2019, @10:33PM
Certainly. See the Tesla "autopilot" people. They're not gonna verify anything in any significant way before trusting the machine with their/other lives.
(Score: 2, Interesting) by VLM on Monday February 11 2019, @12:57PM
You know the end of a business cycle (in general or for a specific industry) is approaching when the addicts come up with infinitely complicated schemes to squeeze those last drops of blood from a stone.
The short term story is "cool optimization strategy brah". The long term story is they're done, stick a fork in them and look for a new business model. Worship of higher ed will die with the boomers along with pro sports and TV.
(Score: 3, Insightful) by Anonymous Coward on Monday February 11 2019, @01:38PM (4 children)
I always thought the whole point of granting scholarships is to provide top-class education to poor but qualified students who might otherwise be unable to afford such education. If you wanted to maximise profits, then you'd probably be better off not bothering to offer any scholarships at all, and just provide your education at whatever price the market will bear. If a machine learning algorithm produced that solution, the problem was not with the machine learning algorithm, but with the idiot who provided the algorithm with such twisted goals to begin with.
(Score: 2) by PiMuNu on Monday February 11 2019, @02:02PM (3 children)
> just provide your education at whatever price the market will bear.
Say we have 100 potential students. 50 students will pay $10000, 50 students will pay $20000 for the same programme, because they have more money but less ability to get into a better university. Now say we have 100 places to fill, how can we get the most money out of the system?
Nb: presumably amazon will start doing this (maybe they already have).
(Score: 2) by VLM on Monday February 11 2019, @03:15PM (2 children)
If memory serves they were doing that 19 years ago and got busted for a Robinson-Patman Act violation.
The internet is a weird place; ya got centuries of law regulating drug sales and taxi services; a company that decides to ignore taxi law is praised, a company ignoring drug regulation is raided; who can guess which way it'll go until the SWAT teams roll. In that way, fully dynamic pricing might be the way of the future, or maybe it'll be smacked down, who knows.
Generally speaking with a very broad brush the US legal system for centuries has tolerated dynamic pricing for transport and hospitality and auctions, while smacking down dynamic pricing for retail and commodities.
Maybe a world where every square of toilet paper is sold individually on ebay would be tolerated, maybe not.
One of the more interesting long term societal effects of mass computerization and tracking is commoditization may literally disappear. You'll never buy "a package of ground beef" again, you'll buy "group up butt of cow named (insert something looking like ipv6 addrs here)" and god knows what the price may vary second to second. We already have this with near organic food (organic food is a USDA term and a separate issue)
(Score: 2) by PiMuNu on Monday February 11 2019, @03:17PM (1 child)
> If memory serves they were doing that 19 years ago and got busted for a Robinson-Patman Act violation.
Presumably, in the US this is true...
(Score: 2) by VLM on Monday February 11 2019, @03:32PM
Ah I found it, Sept 2000 regarding DVD sales, looks like Amazon violated Robinson-Patman regarding DVD prices (and Diamond Rio mp3 players) but they were never charged or punished, just issued an "oh shit we messed up" and a claim they'd no longer violate the act.
Its hard to search because as you'd guess the obvious search terms find many legal books for sale on Amazon, LOL.
Robinson-Patman is pretty lame because it requires proof of monopolizing the market; its OK if the same behavior merely happens to increase profit. If it puts Sears out of business as a direct and obvious result, then its a violation; if it merely makes more revenue its OK. The government has pretty much given up on enforcing it outside the very occasional political oriented prosecution.
There's better more easily to apply anti-trust laws to use against online behemoths.
(Score: 5, Insightful) by Hyperturtle on Monday February 11 2019, @01:52PM (9 children)
Someone correct me if I am mistaken.
"...instead of offering a $20,000 scholarship to one needy student, they could divide the same amount into four scholarships of $5,000 each and dangle them in front to wealthier students who might otherwise choose a different school. Luring four paying students instead of one nonpayer would create $240,000 in additional tuition revenue over four years."
The colleges clearly are targeting students that are children in wealthy families. This indicates quite directly that the strategy of not giving $20k to a "poor" student was chosen over the previous strategy of providing a scholarship to an academically talented student of lesser financial means.
It soon draws the unforeseen conclusion:
" But it turned out that the long-term unintended consequence of this leveraging is an imbalance in the student population between economic classes, with wealthier applicants gaining admission at the expense of poorer but equally qualified peers."
What was unintended? Deliberately choosing wealthy students to maximize profits somehow prevented poor students from getting an education like how the previous system worked when poor students were provided scholarship money? Do the people that run these colleges need to take some financial classes or even basic math? Giving 5k to 4 wealthy students will only offer the following lesson to the academically talented applicants not able to afford tuition: it takes money to make money.
There is absolutely no surprise here--you exclude a certain segment and of course they are excluded from the results.
Blaming machine learning or AI or whatever is just really weak blame deflection. It has nothing to do with the problem; people at the top chose to pursue profit, like most businesses would. "It's not our fault we chased rich students to maximize profits, the computer told us to! How were we supposed to know that it wasn't fairly choosing academically talented but needy students, our lack of ethics meant were blind to the fact that the computer was ethically biased!"
This is a PR stunt to try to not look so bad by blaming it on a "computer error".
The upshot: the computer did exactly what they told it to do. There's no syntax error when you choose to maximize profits and it worked. If they include public relations scenarios in their what-ifs for the revenue model, or give the poor students a handicap so they are picked more often at the expense of less profits, maybe it wouldn't look so bad when news like this comes out.
(Score: 4, Interesting) by PiMuNu on Monday February 11 2019, @02:06PM
> Blaming machine learning or AI or whatever is just really weak blame deflection
I see this behaviour all the time in graduate students (and even more senior academics).
Prof: "Why does the graph look like that?"
Student: "The computer said so"
Prof: "But Why did the computer say so?"
Student looks blank
More often than not it is a typo in the input. Occasionally it is a bad assumption in the model. Very rarely, it is some subtle physical effect.
(Score: 2) by VLM on Monday February 11 2019, @03:21PM (3 children)
A slight tangent of your arguement is that implies $240K of government backed student loans and as current trends continue, that means around $120K of taxpayer liability for defaults.
So there is justification for massive government involvement in this seemingly internal scheme.
Also under the four person solution, the government would be producing 3.5 future bartenders waitresses and baristas for 0.5 available education jobs, or if the supply is limited the government would produce only 0.5 future baristas for 0.5 education jobs. That 3.5 underemployed grads would be a lot of welfare and obamacare expense for electricians, coders, similar high paying jobs, to pay for via taxes on their high non-uni-grad wages.
(Score: 5, Interesting) by bobthecimmerian on Monday February 11 2019, @03:34PM (2 children)
I'm completely in favor of skipping college unless you have a clear career path lined up that makes the investment worthwhile. There is nothing wrong with being an electrician, plumber, or similar. But as a general thing keep in mind supply and demand: going to tech school is only the smart move when the number of people going is relatively low. If enough people become electricians and plumbers, the average electrician and plumber will be as screwed as the average auto mechanic or short order cook. Even high quality software development will be commodotized eventually - it might take a few generations, though.
Supply and demand takes focus away from the whole idea that anyone with a work ethic deserves a decent standard of living, and puts that focus towards competing with everyone else to get a bigger piece of a limited pie. And the pie is only limited because the bulk of the profits go to owners instead of the hardest and brightest workers.
(Score: 2) by VLM on Monday February 11 2019, @04:33PM (1 child)
I would agree with your observations generally, with some fine tuning, that it takes a much higher IQ to be an electrician than perhaps a school teacher, which limits supply, and you can't ship an electrican's job to India quite as easily as shipping an accountants job or a software dev. Also its interesting that the Electrician's union is about as effective as the Medical Doctor's union at gate keeping admission into their apprenticeships such that pay remains high. Depending strongly on location and exact spot in the business cycle, it might be easier to get into med school than to become an apprentice sparkie.
There are justifications based on lab gear capital costs for a nuclear engineer's education to cost $200K per person; nobody can explain why a philosophy or math degree requiring no more than paper and whiteboard should cost $200K other than corruption. I can justify mentally a couple kilobucks to get access to the microwave electronics test lab when I was a kid; hard to justify kilobucks of tuition to learn sociology 101 or Diff Eqs. DiffEqs required maybe one dollar of paper and whiteboard ink per victim, at most, aside from the foreign grad student associate prof who didn't speak English and only got about $1K for teaching the class... the rest of the money disappeared somewhere...
Its interesting that I know of several electricians who went to the local CC to learn how to run their small businesses after they became master electricians and started hiring guys to work for them. How to be a crappy wanna be accountant costs like $75 at the CC but $50K if you want to learn the same stuff and be labeled as a MBA afterwards. Of course the electricians have more experience actually running a profitable business, which makes it even funnier.
There's nothing really wrong with education as an idea, but the implementation of the system as a meal ticket and source of meal tickets is so awful it needs to be burned down can't be fixed without a reboot. Ditto health care, interestingly enough. And maybe real estate certainly properly development. Banking too. Well this list is getting depressingly long.
(Score: 2) by bobthecimmerian on Tuesday February 12 2019, @07:10PM
I'd argue that a good teacher - the kind you would actually want educating children - is almost certainly capable of becoming an electrician. And you can't export an electrician's job overseas, but if there are dozens nearby you won't pay that much anyway. As you said, like physicians they have effectively set up a guild system to protect their income ('guild' is my term for it, I don't know if you would call it that).
But to jump back to education, I think a stellar education should be a top priority for all children across the country, almost completely without regard to expense. However, I would not tie education or education plans to economic advantages. The Democratic Party promise that education opportunities will fix the shrinking middle class is a lie or at best a stupid mistake. It won't, other factors are driving wages downwards and the same supply and demand factors will prevent most of the people going into code/nursing/trades/etc... from jumping a few notches on the income ladder.
(Score: 2) by bobthecimmerian on Monday February 11 2019, @03:23PM (1 child)
Seconded. I'm sure there are countless cases where use of machine learning algorithms did have unintended consequences. But this is clearly not one of them, the school intentionally cut support for poorer students in favor of getting more tuition from wealthier ones. Maybe they should rewrite the headline as "School administration shocked when program change designed to hurt the poor results in harm to the poor."
(Score: 0) by Anonymous Coward on Monday February 11 2019, @04:33PM
Universities offering perks to those who don't need them and students who won't need the degree since they are already lined up for senior partner at daddy's firm.
Doesn't this all sounds absoutely pointless?
(Score: 2, Insightful) by Anonymous Coward on Monday February 11 2019, @03:42PM (1 child)
Yeah, the fundamental mistake was to make the university a business, which means the mandate to maximize profits. It was not the AI that made this decision.
(Score: 2) by Azuma Hazuki on Monday February 11 2019, @11:43PM
Bingo. This sort of thing is the purest and most dangerous expression of "garbage in, garbage out" it is possible to have.
I am "that girl" your mother warned you about...
(Score: 2) by ilsa on Monday February 11 2019, @09:39PM (1 child)
This has nothing to do with ML, and everything to do with a bunch of greedy jerks trying to maximize their revenue.
I mean, seriously? WTF did they think was going to happen if they target wealthy students at the expense of poor students?
(Score: 0) by Anonymous Coward on Tuesday February 12 2019, @03:23PM
Yeah, unintended consequences my ass.
It was obvious what the result would be from the first sentence.