SoylentNews

Freeman [soylentnews.org] writes:

https://arstechnica.com/gadgets/2024/07/reddit-ceo-stands-by-change-that-blocks-most-non-google-search-engines/ [arstechnica.com]

Reddit CEO Steve Huffman is standing by Reddit’s decision to block companies from scraping the site without an AI agreement.

Last week, 404 Media noticed that search engines that weren't Google were no longer listing recent Reddit posts [arstechnica.com] in results. This was because Reddit updated its Robots Exclusion Protocol (txt file [reddit.com]) to block bots from scraping the site. The file reads: "Reddit believes in an open Internet, but not the misuse of public content." Since the news broke, OpenAI announced SearchGPT [arstechnica.com], which can show recent Reddit results.
[...]
In an interview with The Verge [theverge.com] today, Huffman stood by the changes that led to Google temporarily being the only search engine able to show recent discussions from Reddit. Reddit and Google signed an AI training deal [arstechnica.com] in February said to be worth $60 million a year. It's unclear how much Reddit's OpenAI deal [arstechnica.com] is worth.
[...]
Per The Verge, Huffman claimed that Microsoft, Anthropic [arstechnica.com], and Perplexity [arstechnica.com] haven’t been negotiating. The three companies haven't commented on Huffman's interview.

“[It’s been] a real pain in the ass to block these companies,” Huffman told The Verge.
[...]
A Microsoft spokesperson told me last week that “Microsoft respects the robots.txt standard and we honor the directions provided by websites that do not want content on their pages to be used with our generative AI models."
[...]
Huffman also reportedly made reference to a June CNBC interview [youtube.com] where Mustafa Suleyman, CEO of Microsoft AI, said: “I think that with respect to content that is already on the open web, the social contract of that content since the ’90s has been that it is fair use. Anyone can copy it, re-create with it, reproduce with it. That has been freeware, if you like. That’s been the understanding.” Suleyman added that his comment didn’t refer to certain types of web content, like news organizations.

“We’ve had Microsoft, Anthropic, and Perplexity act as though all of the content on the internet is free for them to use. That’s their real position,” Huffman said.

Related stories on SoylentNews:
Reddit Faces New Reality After Cashing in on its IPO [soylentnews.org] - 20240328
Reddit Aims for $6.4bn Valuation Ahead of Initial Public Offering [soylentnews.org] - 20240313
Reddit Sells Training Data to Unnamed AI Company Ahead of IPO [soylentnews.org] - 20240223
Reddit is Removing Ability to Opt Out of Ad Personalization Based on Your Activity on the Platform [soylentnews.org] - 20231004
Reddit Beats Film Industry, Won't Have to Identify Users Who Admitted Torrenting [soylentnews.org] - 20230803
No Apologies as Reddit Halfheartedly Tries to Repair Ties With Moderators [soylentnews.org] - 20230722
Ongoing Reddit Woes: Blackout Explained, Threatened Hacker Leak, Creative Continuing Protests [soylentnews.org] - 20230620
Reddit Rollup: IPO Dreams and Developer Discontent [soylentnews.org] - 20230612

Original Submission