SoylentNews is people
SoylentNews
So after an extended period of inactivity, I've finally decided to jump back into working on SoylentNews and rehash (the code that powers the site). As such, I've decided to scratch some long-standing itches. The first (and easiest) to deploy was HSTS to SoylentNews. What is HSTS you may ask?
HSTS stands for HTTP Strict Transport Security and is a special HTTP header that signifies that a site should only be connected to over HTTPS and causes the browser to automatically load encrypted versions of a website should it see a regular URL. We've forbid non-SSL connections to SN for over a year, but without HSTS in place, a man-in-the-middle downgrade attack was possible by intercepting the initial insecure page load.
One of the big views I have towards SoylentNews is we should be representative of "best practices" on the internet. To that end, we deployed IPv6 publicly last year, and went HTTPS-by-default not long after that. Deploying HSTS continues this trend, and I'm working towards implementing other good ideas that rarely seem to see the light of day.
Check past the break for more technical details.
[Continues...]
As part of prepping for HSTS deployment, I went through every site in our public DNS records, and made sure they all have valid SSL certificates, and are redirecting to HTTPS by default. Much to my embarrassment, I found that several of our public facing sites lacked SSL support at all, or had self-signed certificates and broken SSL configurations. This has been rectified.
Let this be a lesson to everyone. While protecting your "main site" is always a good idea, make sure when going through and securing your infrastructure that you check every public IP and public hostname to make sure something didn't slip through the gaps. If you're running SSLLabs against your website, I highly recommend you scan all the subjectAlternativeNames listed in your certificate. Apache and nginx can provide different SSL options for different VHosts, and its very important to make sure all of them have a sane and consistent configuration.
Right now, HSTS is deployed only on the main site, without "includeSubdomains". The reason for this is I wanted to make sure I didn't miss any non-SSL capable sites, and I'm still working on getting our CentOS 6.7 box up to best-practices (unfortunately, the version of Apache it ships with is rather dated and doesn't support OSCP stapling. I'll be fixing this, but just haven't gotten around to it yet).
Once I've fixed that, and am happy with the state of the site, SN, and her subdomains will be submitted for inclusion into browser preload lists. I'll run an article when that submission happens and when we're accepted. I hope to have another article this week on backend tinkering and proposed site updates.
Until then, happy hacking!
~ NCommander
(Score: 5, Interesting) by NCommander on Monday August 08 2016, @01:04PM
We've discussed it. Most of the boxes are Ubuntu 14.04 with the single CentOS 6 box lurking. I liked Ubuntu (and Debian) as I've had in-place updates never go castrophically wrong before; my desktop was installed with Ubuntu 12.10, and had upgraded through every intervening release since without once requiring a reinstall. CentOS is something of a joke you can't in-place upgrade.
That being said, none of the staff have been very thrilled with Linux as of late, and my recent experiences with both systemd and Ubuntu 16.04 tell me to go "fuck it" and replace it with something not brain damaged. FreeBSD seems to be the popular option, though illumos is also been considered (though its service management system has made me kinda "eh" since its very similar to systemd though it hasn't tendralled through the entire stack yet).
Still always moving
(Score: 0) by Anonymous Coward on Monday August 08 2016, @01:29PM
I'm not real thrilled with Linux lately either. Debian with systemd has worked ok for me, although I did run into some braindead behavior wear a machine wouldn't boot into a usable state due to some nonsense with mdadm due to braindead target dependencies.
I really don't like what Red Hat did with 7. The upgrade package in the repo doesn't work. Rebuild the source rpm from: https://git.centos.org/summary/rpms!redhat-upgrade-tool.git. [centos.org] The one in the repo listed on: http://dev.centos.org/centos/6/upg/x86_64/Packages/ [centos.org] is terribly out of date. The upstream packaged was fixed to work good enough for a decent admin to get the job done. Make sure that /var/run ends up symlinked to /run or systemd will lose its mind if you try to restart some services. I have not done this on CentOS, just the upstream RHEL and Oracle Linux. Oracle Linux uses a dated update tool not quite as dated as the CentOS tool that doesn't resolve package versions correctly where it sees a package with a higher version ending with el6 as being newer than el7. It is possible to manually fix all of those, but it is a huge pain. For example if RHEL6 has grep 1.2.3.el6 and RHEL 7 has grep 1.2.2.el7, the tool will think the 1.2.3.el6 is newer and stick with that. You have to reinstall ALL packages after the upgrade with the dated tool because the scripts within the packages complain about missing libraries, but still somehow successfully upgrade. Don't even get me started on firewalld (I use the old style rules files). The only reason I am using 7 in my org is young projects that require RHEL and that will be hard to upgrade or migrate, so the longer availability of security patches is the sensible thing to do. I wish they would have left grub alone, the new config file style is convoluted in my opinion. It appears that Red Hat is trying to make an OS that is for the server and a laptop, and fails at both because it is like designing a vehicle to be a gas sipping commuter that can pull a big rig's trailer.
(Score: 4, Insightful) by NCommander on Monday August 08 2016, @02:58PM
Red Hat is trying to make Red Hat be a poor knock off of Windows. It was never a good OS compared to Debian in terms of cohensively or sanity, and now it just flat out sucks. Right up until Debian jumped on the systemd train, it, and its descendants were by far one of the best experiences you could get on Linux.
Still always moving
(Score: 0) by Anonymous Coward on Monday August 08 2016, @03:07PM
All it takes is several wasted hours to seriously hate systemd
Mine turned into several days worth of time in the end
(Score: 3, Interesting) by Scruffy Beard 2 on Monday August 08 2016, @03:24PM
So far I have seen weirdness, but have not been bitten in the *ss.
I finally installed systemd on my old debian machine. The disk check caused the boot to fail (with a scary looking error) instead of simply resuming after waiting.
I should be getting my new FreeBSD install working instead of posting on the Internet.
(Score: 2) by zeigerpuppy on Monday August 08 2016, @06:31PM
I've found that pinning systemd on Debian works pretty well. All
my Debian machines run headless so there doesn't appear to be much need for systemd. I'm mostly running a wheezy/unstable hybrid on most machines which works well when you need apache 2.4, reasonably sane versions of node etc.
I also use ZFS extensively, it's awesome but I'm sure there's a performance hit in ZFSonLinux vs native on BSD.
(Score: 2) by Dr Spin on Monday August 08 2016, @01:33PM
I have also tried Mint - same results.
Why do the distro merchants think removing features is an idea with any merits at all?
And why does Linux /Gnome have millions of settings which are cunningly hidden, or just not accessible from the GUI?
and why can you view the print queue, but not delete jobs without using the command line?
Why do you need to use the command line to edit permissions of things in /dev/ to install a printer, and why does the error message you get when the new install fails to work, not mention permissions or /dev?
I believe these problems may be connected to the fact that some people are actually willing to use Windows! (I am not one of those).
Maybe it will be FreeBSD for me too.
Warning: Opening your mouth may invalidate your brain!
(Score: 2) by Scruffy Beard 2 on Monday August 08 2016, @03:28PM
The most common source of problems is user error.
I speculate that they feel that by removing settings/options few people use, they remove opportunities for error.
Of course, that theory relies on a level of attention to detail not present in the software industry.
(Score: 3, Informative) by canopic jug on Monday August 08 2016, @01:34PM
If you're comfortable with Debian then you could stay close to it with Devuan. It is basically a port of Jessie, for the time being, without systemd. It is a drop-in replacement minus the systemd crap.
Or if you really want Debian then there is Debian GNU/kFreeBSD which is pretty much all the same as Debian GNU/Linux for all the user-land activity but, again, minus the systemd crap.
Money is not free speech. Elections should not be auctions.
(Score: 5, Interesting) by NCommander on Monday August 08 2016, @02:44PM
I've looked at Devuan in the past and its a joke. They're rebuilding the entire archive out of a Jenkins instance using a based up version of git and lots of other stuff. That breaks a lot of shit like shlibs tracking, change metadata, etc. It also means they can't reasonably track any packages that are in non-git VCS (which are most of them) If you're going to build Debian, do it fucking right, don't kitbash you way to something else.
I've talked with them about it but was completely blown off. I saw a few other DDs have poked their head in and got their nose cut off for it. Debian kFreeBSD is really not a direction I'd like to go in. They shipped glibc instead of BSD libc, which causes all sorts of application compat problems so interesting build failures are common, and you get all sorts of bloat you don't want or need on FreeBSD.
Still always moving
(Score: 2) by canopic jug on Monday August 08 2016, @03:49PM
Money is not free speech. Elections should not be auctions.
(Score: 2) by NCommander on Monday August 08 2016, @03:56PM
I'm leaning far more towards FreeBSD. OpenBSD doesn't have a MAC framework. I've also found of the two, OpenBSD tends to be more difficult to get to run under hosted virtualization (aka Linode), and while its network tools (3 pf) are great, everything else falls kinda flat. Great respect for Theo, but OBSD simply lacks what we need.
We run Apache under AppArmor to prevent unexpected stupidity dating back to when we ran Apache 1.3 due to slashcode being tied to that release before I took a nuclear blowtorch to it. If you could somehow manage to get a remote shell due to a flaw in rehash, you'd be limited to a very small amount of files that Apache+rehash need to access to run, and the inability to run anything else. Not great, but much better than having unrestricted access to the filesystem as the slash user and migates many attempts to elevate to root.
Still always moving
(Score: 2) by canopic jug on Monday August 08 2016, @04:05PM
Ok. Thanks. It will be very interesting to see which way you go in the future.
The team is doing a fantastic job running the site and the infrastructure. Plus, being able to follow along somewhat with these updates is worth gold and, in my opinion, keeps with the stated nature of the site.
Money is not free speech. Elections should not be auctions.
(Score: 1) by pTamok on Monday August 08 2016, @06:07PM
That is a shame. I was hoping Devuan was going to be a way for GNU/Linux to avoid the systemd mess. A BSD is beckoning. Debian voting to embrace systemd seems to have been a watershed.
(Score: 4, Interesting) by schad on Monday August 08 2016, @02:18PM
SMF isn't as bad as systemd at all. After a brief adjustment period, I viewed it as "really just init, but slightly better in many ways."
If you actually have to create SMF service manifests, though, you may hate your life for a while. Lots of XML boilerplate. Basically all the complexity of service management is front-loaded. While advanced services become much easier to develop because SMF takes care of a lot of it for you, simple services become a lot more cumbersome. Still, it's really only a chore you have to do once. After that, you're just tweaking a setting here or there, just like you would with any other system.
SMF is a lot like IPS (the packaging system used by OpenSolaris). It's better in some ways and worse in others. I personally like it, but even I don't consider it a huge improvement over existing technologies. That's in contrast to, for example, ZFS.
But there's a certain mentality common to everything introduced in Solaris 10. If you happen to share that mentality -- which I do -- then you'll like stuff just because it "makes sense," no matter its other pros and cons. If you don't, I'm told that Solaris can be an intensely frustrating experience. So take my opinion with a grain of salt.
(Score: 2) by darkfeline on Monday August 08 2016, @03:25PM
What issues specifically have you had with systemd?
Join the SDF Public Access UNIX System today!
(Score: 5, Informative) by NCommander on Monday August 08 2016, @03:51PM
Off-hand, here's what I've run into on multiple systems:
* journald corruption on a semi-regular basis
* Recovery is non-trivial and generally requires deletion of the db files
* Can happen almost without fail on some machines on a hard restart
* bad service start announcements; i.e. I can do systemctl *unit* start, and get a false start notice for example
* Which requires a trip to journald to see what happened
* journalctl sucks if you're trying to search for something and doesn't handle rotation cleanly.
* which means if you've got a service which is non-chatty, you could see weeks or older log messages
* No ease of seperation between standard out logs, and stderr.
* No way to filter on stderr output only.
* Unit files don't fail on typo. If you somehow manage to type Userr=nobody, it will silently run your process as root and not even pop a warning
* Related. I've seen systemd race with nss_{ldap|hesiod}, where it tries to enumerate users before the network goes up, and craps itself or simply ignores the User= settings. Unaware if its fixed.
* No easy way to parse in a unit-specific config. With both upstart and sysvinit, I could put a file in /etc/defaults, and load that into the unit to successfully seperate how I want to start a daemon and how to configure it. For example, varnish's setup has to go right into the unit file. This becomes problematic when ubuntu updates varnish and I have to manually merge the units together.
* (Ubuntu specific) - service X action doesn't print success/fail messages. Plenty of scripts still depend on the old symantics
* Historically: very very buggy, and I've had deadlocks.
* At least with recent CentOS, this hasn't been a real issue, but I don't like it when the "massive daemon everything needs" goes belly up
* cgroups are required (problematic in some virtualization scenarios; not an issue for KVM/Xen. Big issue with LXC).
* Actively hostile upstream with no concern for any usecases beyond theirs.
That's off the top of my head. I can go more indepth if need be.
Still always moving
(Score: 1, Informative) by Anonymous Coward on Monday August 08 2016, @07:27PM
Gentoo is still free of systemd. I've been using it for Linux based fileservers and routers sucessfully for years.
(Score: 2) by vux984 on Tuesday August 09 2016, @02:38AM
That being said, none of the staff have been very thrilled with Linux as of late
Heh. Feels like dumping the baby out with the bathwater to make the change for change sake.
(Score: 2) by NCommander on Tuesday August 09 2016, @10:45AM
It's more the straw that broke the camels back to be honest.
From a kernel perspective, I've never really liked Linux over the BSDs. The code is complicated, bloated inplaces, and has an absolutely nasty-to-work-with upstream. Going on LKML is like asking to be shot if you do slightly out of line. I get Linus's view, but most people I know only contribute to the tree because they're paid to do so, not because they want to.
Moving further down the stack, with journald and systemd corrupting the lower bits of the userland, system stabilize seems to have gotten a lot worse. Now yes, you can run Debian without systemd (Debuvan), and a few distros, like Gentoo or SLackware haven't caved. However, I won't run a rolling release on a production server, and I have seen no indication that Slackware has no intention to add systemd. In addition, Slack is historically fragile on upgrades, and its package management system leaves a lot to be desired ...
Still always moving
(Score: 2) by Marand on Tuesday August 09 2016, @04:39AM
I don't think you can do it with Ubuntu, because they went 100% anti-sysvinit way back with upstart, but you can still stick with Debian and run it systemd-free. sysvinit is still there, along with some other alternate init systems, and you can choose which one you want just like you could before. They just changed the default to systemd, but there are still people in Debian that care about supporting other inits (like this guy [simonrichter.eu]), so hopefully the support will continue for a long time.
Rather than switch to an entirely different OS, this would probably be a better route. Like another user said, there should be little (if anything) on a server that will depend on systemd, so switching OSes over it is a bit excessive. It's more of an issue for desktop use because of how much of the desktop is getting caught up in it, even outside of GNOME.
I have a question, though: why did you even use Ubuntu for the server in the first place? I keep seeing people use it and I just don't understand that, what exactly does Ubuntu bring to the table for a server that Debian doesn't? Its release cycle is silly, it's more likely to have dist-upgrade problems (though it IS still pretty solid for it, I've found Debian to be more reliable), and most of the work Canonical does seems to be aimed at the desktop user. This is a real question; I've never understood the appeal of Ubuntu for server-use and I'm curious what the logic behind it is.
Though, on a more ranty note, if you have a problem with systemd, why the hell would you have used Ubuntu at all? Canonical was doing the exact same shit pushing upstart on people before it was production ready, and trying to replace battle-tested init systems with their own in-house NIH bullshit. Their devs also were just as involved in the anti-sysvinit push that got Debian to change its default; both the upstart and systemd camps were shouting down anybody that attempted to suggest other alternatives (including keeping sysvinit). After both camps successfully shut out everything else, then upstart lost, but Canonical and upstart is as much at fault for Debian's systemd status as anybody else. Sure, I'd have preferred upstart over systemd based on what I know of the two, but the reality of it is they both pushed this shit too soon, and they did it together.
(Score: 2) by NCommander on Tuesday August 09 2016, @10:40AM
Um, sysvinit is old to the point of needing replacement. No one has seriously argued it was getting long in the tooth, it was an argument on how to replace it. SPecifically, here are the issues
* No dependency tracking, which means you're only method of controlling startup are Sxx numbers
* Can't paralizared startup of services
* Related, one hung service can hang the boot
* No standard; Debian, Red Hat, and other OSes had subtly incompatible sysvinit semantics making writing a cross platform init script a bitch
Short list, but points 1+2 make reboot times minutes long. You can complain that "reboot times" don't matter, but end users disagree, and its nice not to take 30 minutes to reboot a server.
I actually worked at Canonical and was involved in some of the upstart discussions as well as having drinks with Keybuk multiple times. Upstart was deployed in a non-LTS branch for testing a year and a half before the next LTS (it went in Intrepid if I remember correctly). Furthermore, upstart is an init system, and all it does is load its unit files, and start processes. It doesn't hijack most of the system startup, or replace logs or whatever. It was fairly drop in replacement to the point Debian had shipped it for quite awhile. You'll notice that on my rant list, the only rant I have about startup behavior with systemd is it doesn't fail safely on malformed unit files. It's everything else I can't stand.
Still always moving
(Score: 2) by Marand on Tuesday August 09 2016, @07:56PM
Um, sysvinit is old to the point of needing replacement. No one has seriously argued it was getting long in the tooth, it was an argument on how to replace it. SPecifically, here are the issues
Not really inclined to agree that "it's old so we need to repalce it" is a good argument, especially when distros had still been managing to keep it usable. But that's just a difference of opinion.
* No dependency tracking, which means you're only method of controlling startup are Sxx numbers
This is an outdated claim. Dependency-based boot in Debian's sysvinit has been available nearly ten years, and the default for almost that long. OpenRC has been available longer than that.
* Can't paralizared startup of services
This is also an outdated claim. Parallelised boot (which I'm guessing you meant) has been available in Debian's sysvinit for 6-7 years. Not sure about OpenRC's status for it or other distros.
* Related, one hung service can hang the boot
Unless the rest of the boot process depends on it, this should not occur with the dependency-based bootup. Which means this, also, is an outdated claim.
* No standard; Debian, Red Hat, and other OSes had subtly incompatible sysvinit semantics making writing a cross platform init script a bitch
The only potentially valid complaint so far. More of an issue for package maintainers, though, so not really on-topic with regard to the rest of it. Also kind of a funny argument considering Canonical and Redhat both decided to add even more init systems instead of working with existing alternatives. Not really helping the cross-platform init problem there.
Also, that argument actually favours systemd, because now there is a cross-platform standard, whereas nobody but Ubuntu used upstart.
Short list, but points 1+2 make reboot times minutes long. You can complain that "reboot times" don't matter, but end users disagree, and its nice not to take 30 minutes to reboot a server.
No, I'd argue (and did) that the first three points you made are extremely out of date and seem to indicate you haven't really looked at anything but Ubuntu in years, if ever. Which is fine, except that you're considering switching to a BSD based on Linux complaints while apparently not really knowing the landscape outside of Ubuntu.
Furthermore, if you think sysvinit is outdated and that means it's bad, you're in for a rude awakening with FreeBSD [freebsd.org]. You may even hit some of the problems you (mistakenly) declared for sysvinit, though I'm not certain about that.
I actually worked at Canonical and was involved in some of the upstart discussions as well as having drinks with Keybuk multiple times. Upstart was deployed in a non-LTS branch for testing a year and a half before the next LTS (it went in Intrepid if I remember correctly)
You never did answer my question about what advantage there is to deploying Ubuntu servers instead of going with Debian -- which was a serious question, FYI, and I'd hoped for a real answer. Right now the lack of answer + mention of working at Canonical makes it look like a company loyalty thing more than any real decision based on merits. I'd love a real answer though. I'd guess support contracts would be a valid reason, but I doubt SN's paying for that.
Furthermore, upstart is an init system, and all it does is load its unit files, and start processes. It doesn't hijack most of the system startup, or replace logs or whatever.
Yep, like I said in the other comment, I'd have preferred to see upstart "win" rather than systemd, and that's a big part of why. I never really had any problems with upstart itself, other than Canonical completely removing alternative inits to force use of upstart. I'm kind of sad that Canonical abandoned it, because it's not like they're afraid of being incompatible...what with Mir, upstart, Unity, etc. Oh well.
Still, my big issue wasn't with upstart, it was a social problem: I didn't like how upstart+systemd proponents both forced the issue in Debian, and argued down anybody suggesting something that wasn't upstart or systemd.
It was fairly drop in replacement to the point Debian had shipped it for quite awhile.
It's still available in Debian stable. I could switch to it right now if I wanted. It does looks like it's been dropped from unstable/testing, though, probably because Canonical abandoned it. Shame, because other inits are still there. Even with Upstart gone, there's still openrc, file-rc, runit, sysv-rc, and maybe more I'm not aware of, in addition to the systemd default. There are still maintainers that care about having a choice, and they're keeping those inits working and available.
See, unlike Ubuntu, Debian actually kept maintaining packages for those init systems. In fact, Ubuntu should still have them just by virtue of being a Debian-derived distribution, but someone decided to purge non-upstart inits from Ubuntu. Just like Redhat, Canonical preferred to take away the choice and force their own in-house solution on people, which is unfortunate.
So, again, I'll suggest taking a look at other options, such as Debian, before making a knee-jerk reaction against systemd and Linux as a whole and running off to FreeBSD. You're going to have an "old" type of script-based init there, too, and there's no requirement to run systemd in Debian, so it'd probably be a better transition. You just seem to be running off old knowledge and bad assumptions about the state of non-Ubuntu Linux.
(Score: 2) by NCommander on Wednesday August 10 2016, @02:00AM
You do realize I'm a Debian Developer in addition to an Ubuntu Core Dev, right?. Anyway, my point is sysvinit has issues, and was due for either update and replacement. I thought upstart good replacement for technical reasons. The primary reason it didn't get adopted by Debian was political most over Canonical's copyright assignment policy; I remember it came up several times on d-private.
Fine. Let's get into the deep technical flaws with sysvinit.
When you refer to dependency tracking, I assume you're referring to this block at the start of the file:
Debian's dependency tracking (which is based on LSB) is a serious hack, and one I am aware of. When I was saying sysvinit has no concept of dependencies, I was referring to the daemon itself.
update-rc.d de-facto works by dynamically ordering the scripts as part of update-rc.d. Boot dependencies are statically specified, and from the original thread where this was implemented, Tollef Fog Heen [debian.org] provides an example of why this is a bad thing. The system is incredibly brittle because it can't resolve dependency loops or the case of something failing to start. The reason it all appears to work fine is considerable effort is put in by DDs to keep sysvinit chugging along.
I actually had to go look this up and it turns out you're right. It's been in unstable for years, but it didn't actually ship until Debian wheezy (2013). Debian reference manual [debian.org]. I didn't switch back to Debian as my primary OS until jessie so I completely missed this.
See previous point on "dependency" based startup on sysvinit. The fact is sysvinit has no mechanism to detach a process on boot beyond standard shell control. This generally isn't a problem for servers which can wait for a slow process to get going (and generally want to), but it is problematic on desktops for getting short startup times. Basically, in a sysvinit world, you can either wait for a init script to finish entirely, or detach and clear that boot item. Once you detach, you have no way of if a service has fully come up, or is still starting in a generic way.
A big focus is from kernel start to GDM in about 10 seconds. The system can keep booting beyond that, but you need enough up to get X11 fired up and you need a way to signal that its safe to run Xsession (which generally requires other system daemons be loaded).
Upstart handed this via a socket and dbus. systemd has an entire event hook API (https://www.freedesktop.org/software/systemd/man/sd_notify.html)
I actually missed this point. I wanted a Debian-based OS because of bad experiences with Red Hat ones. Architecturally, Debian has very strict constrains on library dependencies and preventing skew which Ubuntu inherited. (man dpkg-shlibs) combined with a very well tested in-place upgrade system. I would have been perfectly fine if we had standardized on Debian stable at the time. I was against using CentOS due to the fact that its notorious for shipping dated software combined with a very high chance of breaking everything if you upgrade in place. Ultimately, we split the difference, that the rehash machines would run ubuntu, and we'd run CentOS elsewhere. In practice, after the one person really pushing for CentOS left, everything new became Ubuntu to standardize.
In contrast, with one minor hiccup, the upgrade from Ubuntu 12.04->14.04 was a non-event. Beryllium (the CentOS box) will get wiped and reinstalled to match everything else. What we may be running on is unknown at this time.
What ultimately made the decision was that when I setup the site, I installed Ubuntu because I done the core work on my laptop which also ran Ubuntu so I didn't have to worry about library skew or anything.
I'm not sure how much I can actually say on this due to most of the Debian side being discussed on d-private. I can say the primarily reason upstart didn't go anywhere was political vs technical. What I can say, on the Ubuntu front, Mark Shuttleworth himself made the call we were going to systemd because Debian decided to do it and we couldn't justify the engineering resources.
As for the rest you mentioned, well, I'll just say I left Canonical for many reasons.
Ubuntu has sysvinit, upstart fired it up at the end of boot processing to run the legacy scripts. You are correct that Ubuntu had a hard dependency on booting with upstart as I believe we had hooked it into our udev. I generally didn't have much issue with this as upstart did one thing, and did it well, vs. systemd's kitchen sink approach. Debian's support for other init systems to my knowledge is at best spotty in the real world though ...
sysvinit had valid technical reasons to be replaced or overhauled. I just think the new 'standard' is a pile of crap. I posted my long list of technical reasons why I hate systemd elsewhere in this thread. If systemd was content to stay on desktop Linux, I probably won't care much; sysvinit is fine for servers for the most part and I've never had a serious issue with it in real life in a server-role. If systemd did one thing and did one thing well (aka, if it had limited itself to being an upstart competitor), I'd also probably not care enough to work up a damn. It's the entire thing combined with the tendrils its run through the stack that makes me throw my hands up and say enough is enough and look at getting off this ship.
I'm well aware that the boot situation on FreeBSD is extremely primitive in comparison. You have rc.conf and rc.local and that's about it. Everything is sanely logged. For a server, that's more than acceptable.
(Score: 2) by Marand on Wednesday August 10 2016, @05:37AM
You do realize I'm a Debian Developer in addition to an Ubuntu Core Dev, right?. Anyway, my point is sysvinit has issues, and was due for either update and replacement. I thought upstart good replacement for technical reasons. The primary reason it didn't get adopted by Debian was political most over Canonical's copyright assignment policy; I remember it came up several times on d-private.
Had no idea about you being a DD, since I only saw mention of Canonical work in the other comment. You're as anonymous as anybody else here to me. :p
And yeah, the copyright assignment thing sucked, I recall there being some grumpiness over it in the past, so that's not surprising at all. What I meant though was that in the open discussions about the future of Debian's default init, upstart+systemd people (including some devs on both sides) seemed keen on shutting down any attempt to suggest anything else. People were determined to make it a two-party vote, and once that happened Upstart lost to politics like you said.
As for the sysvinit thing, I'm aware it's a script-based hack and has its issues, but it still does the job mentioned, so I took your statement as a literal "it can't do this thing it does". It's not hard to imagine someone would miss a change like that in a completely different distro, so I took the remark at face value. Just a communication error there: you were being more literal in referring to init itself while I was interpreting it more broadly.
Also, thanks for the answer about the server selection. I'm with you on the general aversion to Redhat-derived distros, and preferring Debian-based distros for stability and the upgrade process. It's why I've stuck with Debian since around 2000, and why I keep ending up coming back to it after experimenting with other options occasionally. I've mentioned it here before, but my primary OS is a Debian install that started on 2.2 (potato) and has been dist-upgraded through the years, with migrations to new hardware, new disks, and even updated in-place from 32bit to 64bit. I don't know any other OS or distro that I could have done this with. I probably should just reinstall it one day, but it amuses me to keep it going instead of starting fresh at this point. :)
Vaguely related distro talk: I keep intending to spend time with NixOS. I already get a lot of use out of Nix as a secondary package manager and really like its (admittely weird as hell) filesystem layout and philosophies, so it's on my radar but I haven't gotten around to it. It starts out with systemd though, bleck. Does seem to have other options (sysv, upstart, runit) but I have no idea how much trouble it would be to switch out.
At this point, my take on systemd is I'm mostly avoiding it for now. sysvinit and the patchwork of hacks on top of it may not be the way to go long-term (I agree about it being due for update or replacement, I just think it all happened a bit prematurely, at least for Debian), but I'd still rather use it than systemd right now. Like you said, if it were just a desktop thing, it wouldn't be such a big deal. I haven't had any problems with the "higher" parts of systemd; it's not really any better or worse than dealing with, say, HAL was in that regard. (I remember HAL being a nightmare at times early on...)
It's just that I don't want systemd as my init, so I stick to systemd-shim and running anything else underneath. The init just happens to be sysv because that's what Debian used before and it just worked, but it's not like I'm not open to alternatives. Just preferably alternatives that aren't systemd's init. Hopefully by the time I don't have a choice any more, Poettering and Sievers will have gotten bored and moved on so others can clean house. It mostly worked for Pulseaudio like that, at least...
On a related note, have you ever used runit for anything? It sounds interesting but I've never gotten around to spending time with it. I've only heard of one distro (Void Linux) using it by default [voidlinux.eu], but their page on how it works is intriguing. It also appears you can even use it alongside another init in case you only want to manage certain services with it, which sounds like it could be appealing.
(Score: 2) by Justin Case on Wednesday August 10 2016, @02:26AM
No dependency tracking, which means you're only method of controlling startup are Sxx numbers
Has never caused any problems for me. Which of course I'm not arguing it hasn't been a problem for others, but that's what I love about a non-proprietary OS: choice!
Can't paralizared startup of services
Don't want to.
one hung service can hang the boot
As it should. Fix it before proceeding.
reboot times minutes long. You can complain that "reboot times" don't matter, but end users disagree, and its nice not to take 30 minutes to reboot a server.
Again, not a problem for me. I'm not an "end user" and I want to barf every time someone forces me to use software written for the ignorant and lazy, because I am neither. Oh, and if something isn't right, I don't want it to take 30 minutes to boot a server, I want the server to not boot at all. That's an incredibly effective technique for making sure the problem gets noticed, prioritized, and fixed, instead of "oh sure let's proceed down the runway even though one jet engine didn't start..."
BTW please don't take any of this as a personal attack. Love the work you're doing, and praise for doing "the right thing" with https etc.