Places like Amazon, Facebook, and Twitter are swimming in data, but their problem is that a lot of it is untrustworthy and shilled. But you don't need to use all the data. Toss big data happily, anything suspicious at all, false positives galore accidentally marking new accounts or borderline accounts as shills when deciding what to input to the recommender algorithms. Who cares if you do?
Lately I've been thinking about recommender algorithms and how they go wrong. I keep hitting examples of people arguing that we should ban the fewest accounts possible when thinking about what accounts are used by recommender systems. Why? Or why not the opposite? What's wrong with using the fewest accounts you can without degrading the perceived quality of the recommendations?
The reason this matters is that recommender systems these days are struggling with shilling. Companies are playing whack-a-mole with bad actors who just create new accounts or find new shills every time they're whacked because it's so profitable -- like free advertising -- to create fake crowds that manipulate the algorithms. Propagandists and scammers are loving it and winning. It's easy and lucrative for them.
So what's wrong with taking the opposite strategy, only using the most reliable accounts? As a thought experiment, let's say you rank order accounts by your confidence they are human, independent, not shilling, and trustworthy. Then go down the list of accounts, using their behavior data until the recommendations stop improving at a noticeable level (being careful about cold start and the long tail). Then stop. Don't use the rest. Why not do that? It'd vastly increase costs for adversaries. And it wouldn't change the perceived quality of recommendations because you've made sure it wouldn't.
Previously:
Amazon Still Hasn't Fixed Its Problem with Bait-and-Switch Reviews
Amazon's Top UK Reviewers Appear to Profit From Fake 5-Star Posts
Related Stories
Amazon's top UK reviewers appear to profit from fake 5-star posts:
Amazon is investigating the most prolific reviewers on its UK website after a Financial Times investigation found evidence that they were profiting from posting thousands of five-star ratings.
Justin Fryer, the number one-ranked reviewer on Amazon.co.uk, reviewed £15,000 worth of products in August alone, from smartphones to electric scooters to gym equipment, giving his five-star approval on average once every four hours.
[...] Overwhelmingly, those products were from little-known Chinese brands, who often offer to send reviewers products for free in return for positive posts. Mr. Fryer then appears to have sold many of the goods on eBay, making nearly £20,000 since June.
When contacted by the FT, Mr. Fryer denied posting paid-for reviews—before deleting his review history from Amazon's website. Mr. Fryer said the eBay listings, which described products as "unused" and "unopened," were for duplicates.
At least two other top 10-ranked Amazon UK reviewers removed their history after Mr. Fryer. Another prominent reviewer, outside of the top 10, removed his name and reviews and changed his profile picture to display the words "please go away."
Amazon still hasn't fixed its problem with bait-and-switch reviews:
Like thousands of other parents, I decided to get my kids a cheap drone for Christmas. I spent $24 for a plastic flying machine with rudimentary collision-avoidance capabilities. A plastic cage mostly kept small fingers away from the four propellers. The kids were delighted for the first couple of hours.
[...] The kids enjoyed the drone so much in its few brief hours of functionality that I thought I might buy them another one.... If I did more research and spent a bit more money, I hoped I could find a higher-quality model that wouldn't fall apart after a few hours.
So I went to Amazon.com, searched for "children's drone," and sorted by "average customer review," figuring the best-reviewed drones were likely to be high quality. They weren't.
[...] "Absolutely love this honey," wrote one reviewer in the UK in March 2019. "It's quite different from any supermarket-purchased honey I've tried."
(Score: 3, Funny) by ikanreed on Saturday July 09 2022, @03:42AM (2 children)
Five stars, absolute best comments on the site.
(Score: 2) by Opportunist on Saturday July 09 2022, @08:35AM
Absolute truth can only be found there!
(also, your check bounced, what gives?)
(Score: 2) by Spamalope on Sunday July 10 2022, @04:10PM
*Ikanreed is five stars... of 100! Absolutely read Ikanreed posts... for the entertainment - it's not often you see failure on that level.
* This post can be removed for $2.99.
(Why not monetize reviews this way? Not like the reviews have any sort of ethics enforcement...)
Though seriously, the suggestions to rank reviewers based on a verified address, an organic purchase history and user ranking of their reviews (from meta reviewers who themselves have an organic history). A weighting adjustment based on those factors that works like anti-spam AI using those factors feels like it could help. That's all assuming the sales portal wants it fixed.
(Score: 2) by Rosco P. Coltrane on Saturday July 09 2022, @07:57AM (3 children)
Assume everybody you don't know and trust personally is lying to you for profit.
Very simple, very effective, and it's been working since human beings started talking to each other.
(Score: 3, Funny) by Opportunist on Saturday July 09 2022, @08:37AM
My algo is "trust only people you can get close enough to to kick them in the nuts if they try to swindle you out of your money".
Worked pretty well so far. Either I got honesty from them or at least a satisfying and entertaining whining and groaning, along with a rain dance.
(Score: 1, Insightful) by Anonymous Coward on Sunday July 10 2022, @05:04AM (1 child)
Well ... What Amazon needs to do is not only have 'verified purchasers' they need to have 'verified purchasers with verified addresses' and give the user the option to filter for those.
Verifying an address is simple. Snail mail a code to someone's physical address and have them enter the code in their Amazon account to prove their address. Re-verify their address every two years or so.
The problem with 'verified purchaser' is that anyone can make up an account with a fake address and 'purchase' the items (while either shipping nothing or shipping something really cheap and using that as confirmation).
Verify addresses.
(Score: 0) by Anonymous Coward on Sunday July 10 2022, @05:12AM
and by snail mail I don't necessarily mean USPS. It can be UPS, an Amazon van, etc...
(Score: 0) by Anonymous Coward on Saturday July 09 2022, @08:02AM (1 child)
I pay a lot more credibility to negative reviews than I do positive ones... But even that is often contaminated by the competition.
You have to read the reasoning behind the review. Telling me something sucks isn't sufficient. Please tell me how the product failed to meet expectations.
(Score: 2) by Opportunist on Saturday July 09 2022, @08:40AM
Considering my first language is German, and I usually rely on reviews in that language where possible, a solid decider for the veracity of a review is grammar. If it looks like Google had a hand in the creation/translation of it, you can safely ignore whatever it says.
Seriously, don't try faking German reviews. We tend to use words you probably would not find in a dictionary, especially if the review is negative.
(Score: 4, Insightful) by Opportunist on Saturday July 09 2022, @08:50AM
You have a purchase history. Unless you're a store that only deals in a very limited selection of products, real people will probably have bought a wide variety of goods from a wide variety of vendors. Give these reviews a lot of weight, since you know that these are real people who really buy real products. Yes, there's that nonzero chance that they're "professional" shills that can get hired to buy a product from company X to give it glowing reviews, but first of all, you get your cut from that purchase, so more power to them, and second, that's one out of many.
Of course you can also review the reviews. Do they only hand out "5 stars, A++" reviews? Do they vote "against the grain", i.e. do they give glowing reviews to stuff that your other reviewers consider crap (or vice versa)? Take a page from the playbook of the page here and have your customers review the reviews. Reward them with some token discounts ("rate these 20 reviews to save a buck on your next purchase of (insert crap nobody buys here)").
You can also have "professional reviewers", like few of those stores already have (kinda). Verify their identity (c'mon, people have handed over their ID for less), then make them "trusted reviewers". Drop the hint that sellers will probably send them free crap if they offer "trusted reviews" in return. Of course, you have to dangle the damocletian sword of being stripped of the coveted title if you find them shilling.
There's plenty of ways to solve that, have your markedroids do something for their nose powder!
(Score: 2) by TheLink on Saturday July 09 2022, @11:24AM (1 child)
Because for many things there's no "best item" it's about preference AND what you prefer may be different from what someone else prefers. And there could be many groups of people with similar preferences.
So for example if you're buying gifts for a 6 year old girl you might tell the site the stuff you know the little girl likes and then it suggests a few suitable "point of views"/"preference groups" for you to use to get shopping suggestions. Then when you buy stuff for Grandma you might switch to a different view/"preference group".
Same goes for picking/suggesting a restaurant. I may like various cuisines, but that doesn't mean it's all about me all the time. But all these sites seem to be focused on this single person perspective. They should allow shoppingfor people with different preferences.
And that way Amazon etc don't do retarded stuff like keep suggesting My Little Pony stuff to you just because you bought a My Little Pony plush toy for someone else.
Is this really that hard to do? Some of them have tens of thousands of computers to do such crunching if not more right? They should have lots of geniuses working for them too.
(Score: 2) by TheLink on Saturday July 09 2022, @11:29AM
So if you want to buy stuff for a shill you can pick those groups, but if you don't their shilling doesn't affect your recommendations at all.
Top 10 restaurants for them might be different from Top 10 restaurants for you.
(Score: 4, Interesting) by anotherblackhat on Saturday July 09 2022, @03:32PM
Who do you trust?
So many people want trust to be a commodity that can be packaged, bartered and sold.
It isn't.
Trust by proxy isn't completely impossible, but it's going to involve something like the PGP web of trust.
I approve a certain number of reviewers as trustworthy, and accept the recommendations of my circle of friends.
Maybe I trust a third party to validate others, but it needs to be my choice as to who is trusted to make recommendations.
My trust in a review is going to be based on how strong my link is to the reviewer.