Just to give you advance notice that the continual problem with the renewal of SSL certificates is due to occur on Monday 5 Aug.
Nobody in the new team has the necessary access nor knowledge of the current hardware configuration, and control remains with NCommander. The transfer of assets has been initiated but as one of the two members of the current Board is out of the country everything has temporarily ground to a halt. We cannot reconfigure the existing structure as legally we do not yet 'own' the database or existing hardware assets.
I have requested that NCommander assist by renewing the certificates but that depends upon his availability. He has been kind enough to help in the past. There is nothing more I can do at the moment.
I know that this is easily fixed - but until the formal exchange of the assets takes place we are on very shaky ground with regards to liabilities and responsibilities.
(Score: 5, Insightful) by JoeMerchant on Friday August 02 2024, @01:44PM
Understanding that you have a problem is the first step in correcting the problem.
Thanks for keeping up with it.
🌻🌻🌻 [google.com]
(Score: 0) by Anonymous Coward on Friday August 02 2024, @02:14PM (11 children)
It looks like SN uses Let's Encrypt...any idea why it's not set to automatically renew via cron job?
Or why it isn't running on a k8s cluster with something like ingress-nginx and cert-manager to automatically renew certificates?
I get that it's an ancient Perl application...but even in this day and age, can't you containerize it so scaling the front end is ridiculously easy, managing certs is ridiculously easy (and automatic) and just point the container at the database back-end?
No more manually installing some variant of Linux on bare metal, tweaking and configuring the Perl environment, installing modules, etc...
I have a fairly large application that does about half a million database hits per minute, and it's pretty trivial to spin up a ~$40/mo postgres database on Digital Ocean to handle it followed by a 3-node kubernetes cluster to run the front end and a few back-end processes...I haven't touched the cluster or the database in years. Upgrades are seamless, certs renew, and I occasionally tweak the code or update libraries and 'git push' to roll out a new front end.
(Score: 5, Informative) by janrinok on Friday August 02 2024, @03:13PM (4 children)
There is already a dockerised version of this site - but it is not the one that is currently carrying the load. It works and is in use today but only for our development system. Our intention is to switch to it, or a copy of it but with the live data, as soon as we have taken over responsibility for the assets, both software and hardware.
The current configuration was designed for a site with a far larger community in mind and it had significant redundancy and duplication. It is too expensive now and is unnecessary. We know how to reduce the hardware assets and improve the maintainability but we cannot do that without a major restructure and we do not have the control or authority to do such a thing. The current site is not under our own control. We cannot change contracts for example, and we cannot purchase or hire different hardware.
We are also hoping to do this with little or no down-time for the site. Hopefully you will not notice the difference. I've no doubt that there will be teething troubles but at least we have the staff who are prepared to be responsible for sorting such things out. Nobody wants to work on the current - and undocumented - system. It is thought to be too fragile.
For the last year or more we have been creating a new business structure to inherit the existing site thus relieving NCommander of his legal responsibilities and allowing us to start again with a more modest (and cheaper!) structure which is also supported by more modern technology (e.g. docker, ansible etc). NCommander wants to move on to other challenges and interests, and I can fully understand his desire to do so.
I am not interested in knowing who people are or where they live. My interest starts and stops at our servers.
(Score: 3, Interesting) by janrinok on Friday August 02 2024, @03:26PM (3 children)
A point of clarification - NCommander has reduced the assets significantly but I am not sure that we have seen the expected reduction in running costs. I am currently in discussions about this with NCommander.
I am not interested in knowing who people are or where they live. My interest starts and stops at our servers.
(Score: 2) by Whoever on Friday August 02 2024, @06:57PM (2 children)
Who is paid to provide the assets? Are you paying Linode directly or is there an intermediary?
(Score: 3, Interesting) by janrinok on Friday August 02 2024, @07:09PM (1 child)
We have no visibility of the accounts.
I am not interested in knowing who people are or where they live. My interest starts and stops at our servers.
(Score: 3, Interesting) by Whoever on Friday August 02 2024, @07:29PM
So you don't know if there is some skimming off taking place.
(Score: 3, Interesting) by gnuman on Friday August 02 2024, @04:13PM
I call BS on this, unless all you are doing is cached reads on something extremely trivial. This would be 10k requests per second. Also, on Digital Ocean, this would be 50GB storage with 2GB RAM ... with 1 vCPU.
So, is this per minute or per hour? (140/s) or per day (6/s)?
(Score: 3, Informative) by bryan on Friday August 02 2024, @04:24PM (1 child)
SN uses the wildcard cert via DNS option. This version of Lets Encrypt has historically been a bit harder to automate as you need to update a DNS record during the renewal process. These days, there are plugins to use Linode's API (SN hosts DNS there) to automatically change the record, as long as you have a recent version of certbot and someone is able to make a low-privileged account on Linode's control panel.
An even easier option would just be to use a non-wildcard cert for the main site. These are the easy certs that are trivial to automate, since they use a file in the webroot instead of DNS. I think they allow a maximum of 100 subdomains (like irc, mail, www, etc) per cert, so you could even lump them all into one. Or just register them separately - it's not like they cost anything.
(Score: 3, Informative) by janrinok on Friday August 02 2024, @04:28PM
I agree. But it is a moot point. The problem doesn't exist on the Docker configuration. We had hoped to be using it by now but there has been a delay in the transfer of assets.
I am not interested in knowing who people are or where they live. My interest starts and stops at our servers.
(Score: 1) by Brymouse on Friday August 02 2024, @06:41PM (1 child)
ok, so it's every 90 days. and you need a wildcard cert, ok can I pay for the 50 or 100 bux a 1year cert would cost and just have that installed?
By that time you should be migrated/etc from him to the new corp, no?
(Score: 2) by janrinok on Friday August 02 2024, @07:18PM
https://soylentnews.org/meta/comments.pl?noupdate=1&sid=61689&page=1&cid=1366780#commentwrap [soylentnews.org]
I do not know how much is in the current account or how it is being spent. I have asked for the latest information. The accountant (Matt Angel) has been in Germany for much of this week. He has not yet responded.
We have recommended that people do not make further contributions until we have transferred assets (including whatever funds are left) into a new account.
The problem with the certificates is NOT a financial one. We have no-one in the new team who has access to the Linode control panel, nor who has access to the servers and is prepared to start playing with the current infrastructure.
I am not interested in knowing who people are or where they live. My interest starts and stops at our servers.
(Score: 3, Interesting) by VLM on Friday August 02 2024, @09:57PM
In a very general sense I have a huge amount of experience with automation, and processes that don't work manually will just blow up even more spectacularly when a thin paste of automation is applied to the already broken overall system.
Automation works SUPER WELL when it's making an already 100% successful manual process, faster. Which, uh, not to aggravate a difficult situation, but this is likely not it at this time for various reasons, etc.
If they already had a working documented script and someone needs to run ./script.sh or whatever, then the next step is sticking it in a cron job, sure, but you're a couple steps ahead of the current position of the game, is I guess what I'm trying to say.
Or as a long term goal, I think we agree, yeah. But it's a very "crawl before you walk" situation. Surely they'll get there eventually but its not likely the next step.
(Score: 5, Insightful) by Username on Friday August 02 2024, @02:17PM (12 children)
Can you enable the site on http? If someone goes to the length of doing a mitm attack just to get my password to this site, they can have it. That would be an extreme amount of effort for no gain.
(Score: 5, Informative) by janrinok on Friday August 02 2024, @03:20PM (3 children)
Opening up a potential security hole which might allow someone to get access to the database would not be protecting your personal information. We have promised to protect the data and for the last 10 years have been successful in keeping your personally identifiable information private.
If somebody can get your password then they can also get an administrator's password. If they compromise the account of certain admins (e.g a sysadmin) then they have the keys to the castle - everything!
I am not interested in knowing who people are or where they live. My interest starts and stops at our servers.
(Score: 2) by Username on Friday August 02 2024, @05:22PM (2 children)
Chmod the admin pages 700. That should solve it.
(Score: 3, Informative) by janrinok on Friday August 02 2024, @05:33PM (1 child)
If they get inside with an Administrator's password, what good would that do?
It would, I think, also break the current software. Probably best that we don't do that. Remember that this is late 1990s software technology, originally used in in 2000's hardware, and significantly modified in 2014 for its current role.
Once someone is inside the system - as any user - they have a much better chance of getting where they shouldn't be. They can also see where other vulnerabilities might be lurking. Not that we have any of course, certainly not, safe as houses.....
I am not interested in knowing who people are or where they live. My interest starts and stops at our servers.
(Score: 2) by Username on Monday August 05 2024, @02:16PM
With 700 no one can access it remotely via http. You would have to ssh or whatever solution you use in with whatever account to access it. You will be using the os encryption, not apache or whatever you got.
(Score: 4, Insightful) by Rosco P. Coltrane on Friday August 02 2024, @03:27PM (6 children)
Just override the cert warning in your browser. From a security standpoint, it's still better than plain Jane HTTP, as long as the warning you're overriding is the cert being too old.
(Score: 2) by Reziac on Saturday August 03 2024, @02:59AM (5 children)
SeaMonkey will (usually) let me do that. No idea why on some sites there's no option to override.
Chrome/Supermium will not. Maybe I've just missed it (not everyday browser) but I don't see a setting to allow override, either.
And there is no Alkibiades to come back and save us from ourselves.
(Score: 3, Informative) by Rosco P. Coltrane on Saturday August 03 2024, @09:09AM (3 children)
I don't know about Chrom(e|ium) as I avoid the Google monoculture like the plague.
With Firefox and variants however, you can get past the un-overridable warning by clearing cookies and site data.
(Score: 4, Informative) by janrinok on Saturday August 03 2024, @09:48AM (2 children)
I had the same success just by using a private tab.
I am not interested in knowing who people are or where they live. My interest starts and stops at our servers.
(Score: 3, Informative) by Rosco P. Coltrane on Saturday August 03 2024, @12:38PM (1 child)
I believe you're seeing the same effect: private tabs start out blank, so it's like you cleared a regular tab's data and cookies.
But with a regular tab, when you open it again, the deprecated cert is still valid and you stay logged in, whereas the private tab starts blank and you have to log in again.
(Score: 2) by Reziac on Saturday August 03 2024, @01:40PM
Thanks, R and J, good information. Will try that next time I'm presented with a WTF of this sort.
And there is no Alkibiades to come back and save us from ourselves.
(Score: 2, Informative) by Anonymous Coward on Sunday August 04 2024, @02:04AM
It's because soylentnews.org sends the Strict-Transport-Security header with a long expiry. If you have previously visited the site, and the browser has saved the fact that you previously visited the site, and it has not been more than a year since the last visit, then (all?) browsers which implement HSTS will not allow the user to manually override any kind of certificate error.
(Score: 0) by Anonymous Coward on Saturday August 03 2024, @04:53PM
If they get my password here, then they can get into my luggage because I use the same password on my luggage!
(Score: 5, Insightful) by Barenflimski on Friday August 02 2024, @03:17PM
Thanks Janrinok!
(Score: 1, Funny) by Anonymous Coward on Friday August 02 2024, @07:49PM
Not the 5th! I was planning on watching paint dry on the 5th.
(Score: 5, Informative) by janrinok on Saturday August 03 2024, @07:27AM (1 child)
I am currently in discussion with Michael (NCommander) and Matt Angel regarding some contractual issues. The discussions have caused the transfer of assets to temporarily stall.
I strongly recommend that, for the moment, no further donations should be made to SoylentNews PBC until this matter is resolved. As I do not yet have all the facts I cannot attempt to answer your obvious questions. I will attempt to do so once I know the details.
I am not interested in knowing who people are or where they live. My interest starts and stops at our servers.
(Score: 4, Insightful) by Reziac on Saturday August 03 2024, @01:42PM
Thanks for keeping us all in the loop.
And for being a hard-ass about our personal security.
And there is no Alkibiades to come back and save us from ourselves.
(Score: 3, Insightful) by looorg on Saturday August 03 2024, @03:01PM
So we get a free day where we can contemplate reality and mans inhumanity to man and all that ...
(Score: 2) by VLM on Monday August 05 2024, @07:32PM
I assume we're good my browser reports the current cert was issued about when I woke up this morning (had a maintenance window to hit) and it expires Nov 3, at least in my timezone.
I was going to suggest if the site were going down in a couple hours, we could raid /g/ which I'm sure would not be controversial at all; yet is also not the worst idea I've ever had, LOL. Could have posted a "/sn/ - SN General" thread on /g/