SoylentNews is people
SoylentNews
"Come on, I worked so hard on this project! And this is publicly accessible data! There's certainly a way around this, right? Or else, I did all of this for nothing... Sigh..."
Yep - this is what I said to myself, just after realizing that my ambitious data analysis project could get me into hot water. I intended to deploy a large-scale web crawler to collect data from multiple high profile websites. And then I was planning to publish the results of my analysis for the benefit of everybody. Pretty noble, right? Yes, but also pretty risky.
Interestingly, I've been seeing more and more projects like mine lately. And even more tutorials encouraging some form of web scraping or crawling. But what troubles me is the appalling widespread ignorance on the legal aspect of it.
So this is what this post is all about - understanding the possible consequences of web scraping and crawling. Hopefully, this will help you to avoid any potential problem.
Disclaimer: I'm not a lawyer. I'm simply a programmer who happens to be interested in this topic. You should seek out appropriate professional advice regarding your specific situation.
https://benbernardblog.com/web-scraping-and-crawling-are-perfectly-legal-right/
(Score: 4, Interesting) by bzipitidoo on Wednesday July 31 2019, @02:12PM (1 child)
Anyone can sue for any reason at all. The lawsuit might be thrown out of court as groundless, if it gets that far.
But court is not where most accusers want to go. They want the accused to chicken out and settle out of court. Want the accused to pay the accuser lots of money, or agree not to compete, or anything else the accuser does not like, without ever realizing that the accuser was full of it, and actually had no legal grounds for the accusation and would have lost in court. That's what a lot of the stuff in EULAs and ToS are really all about. They're trying to scare people away from exercising their rights.
In the case of the MAFIAA vs pirates, their threats are not entirely groundless, because the law is antiquated, and they lobby to keep it that way. But they've been learning that piracy is a lot, lot bigger than they are, and gradually realizing that they've taken on nothing less than the laws of nature, which is ultimately unwinnable. Unfortunately, until the law catches up with reality, they can cause a lot of misery, and accuse anyone of piracy. Those sick bastards have been known to pick on the weakest possible defendants, regardless of the credibility of a piracy accusation.
Of course costing the accused a great deal of money and time, defending themselves from baseless accusations, is the point of legal harassment, particularly of the variety that does go to court.
There are also the outright criminal frauds. Like the fake message allegedly from the IRS that you owe lots of taxes and will be in Big Trouble if you don't pay Right Now.
I'd guess it works entirely too often and too well. Lot of people are afraid of a fight, and for good reason.
It can get to the point where, yes, it seems that nothing is legal. Yet somehow, it is legal for these writers of legal bull to leave out exceptions, and just plain lie. For instance, the NFL routinely claims you can't make any use whatsoever of footage of an NFL football game without prior written permission from them. They're wrong, and they know it. They lie. Why they're allowed to overstate their rights like that, and no one smacks them down for it, I don't know. Of course, one reason is that they're powerful.
Another example is lawn care. Somehow, notices of violations that your Grass is Too High (over 12 inches in some places, and only 6 or even 4 inches in other places) leave out exceptions, such as for vegetable gardens. I continue to be astonished at the love for these Home Owners Associations and city ordinances, and their incredibly petty and fascist rules. Why do people not only put up with it, but go for it?
Lately we've been seeing a lot of news stories about racists calling the police on brown individuals who weren't doing anything wrong. Permit Patty, for instance.
In sum, don't let the likes of Permit Patty fool you.
(Score: 0) by Anonymous Coward on Thursday August 01 2019, @12:56AM
All they need is for the cost of getting them laughed out of court to be more than you can afford. If your choices are settle or bankruptcy, what do you do?