SoylentNews

Over recent weeks we have been experiencing connections from a large number of bots, spiders and scrapers. Some are the expected ones (Microsoft, Google, Amazon etc) and these tend to rate limit their requests and cause us little problem.

Others appear to be AI driven scrapers and they can result in tying up a large percentage of the site's resources. For the most part they ignore robots.txt or when we return code 429. While they are individually only an annoyance their activity can affect the speed at which the site can respond to members attempts to view a page or leave a comment. They have contributed to some of the 404 or 503 (Backend Fetch Failed) that you might have experienced recently. A small number of bots isn't a problem, but if many bots are querying the site at the same time then they can affect the speed at which the site can respond to your comment or request.

Software has been developed to block such abusive sites for a short period. In the majority of cases this will be invisible to you as users other than to hopefully improve the responsiveness of the site.

However, it is possible that sometimes there might be a false positive and you may encounter difficulties in connecting to the site. If you do experience connection problems please inform us immediately either by email or on IRC. Neither of those apply filters to connections; the short temporary blocks only apply to the site itself. We will have to contact you by email to ascertain your IP address so that we can lift any block that may have been incorrectly applied. Please do not publish an IP address in either a comment or on IRC.

If you are using a VPN or Tor it might be advisable to try another routing to circumvent any temporary block that might be affecting your connection.

From the staff, may I wish all members of our US community a very happy Independence Day and holiday weekend.

When the BBQs and beer get too much, when being sociable with other family members is getting you down, come back and join in a discussion or two.

We have some good news to share. The Stripe donation system, which some of you may have noticed has been unavailable for a while, is now fully functional again.

It took a bit of digging, but after a thorough investigation, kolie was able to isolate the problem and has successfully deployed a fix. A huge thank you is owed to him for his persistence in resolving this.

As all of you know, SoylentNews is a user-supported, community-run project. We rely entirely on the generosity of our readers to cover the server costs and other expenses that keep this site operational. Now that the donation pipeline is open again, it's time to pass around the hat.

If you find value in this community and have the means, please consider making a one-time or recurring donation. Every contribution, no matter the size, is critical in ensuring that everyone's favorite place for news and discussion can continue to operate and remain independent.

You can find the donation link here.

Feedback is always welcome. If you encounter any problems at all with the donation process, please let us know in the comments below so we can look into it. Thank you for your continued support.

Original Submission

The reason that the site was offline is that the cable to the NOC has been cut - again! It is not something that we could control.

We apologise for the problem. You could have stayed up-to-date with the cause and rectification if you had joined us on our back-up IRC - Libera.Chat, ##soylentnews (irc.libera.chat/6697)