While people are starting to understand the importance of privacy it is a major hurdle to get them to select a different search engine.
Search engines eat resources like crazy, so operating costs are non-negligible.
Some sites (including e.g. github) use a whitelist in robots.txt, blocking new crawlers.
The amount of spam, link-farms, referrer-linking, etc. is beyond your worst nightmare.
Returning good results takes a long time to fine-tune.
Monetizing is nearly impossible because advertising networks want to know everything about the users, going against privacy concerns.
Buying search results from other search engines is impossible until you have least x million searches/month. Getting x million searches/month is impossible unless you buy search results from other search engines (or sink a lot of cash into making it yourself).
So what do you soylentils think can be done to increase privacy for ordinary users, search-engine-wise ?
It wasn't meant as censorship. It was a pragmatic solution to dealing with the huge amount of copy-pasted, keyword-stuffing, referral-linking port sites. The number of such site is huge, and we didn't have room for it in our index.
(Score: 4, Informative) by isj on Wednesday November 21 2018, @06:59PM (1 child)
It wasn't meant as censorship. It was a pragmatic solution to dealing with the huge amount of copy-pasted, keyword-stuffing, referral-linking port sites. The number of such site is huge, and we didn't have room for it in our index.
The list of filtered-out sites is available: https://github.com/privacore/filter-lists/blob/master/adult/findx-adult.txt [github.com] (182526 entries)
You can use that for "research purposes" if you like :-)
(Score: 2, Funny) by Anonymous Coward on Wednesday November 21 2018, @10:15PM
>too big to show
You tease.