SoylentNews is people
SoylentNews
Structural and semantic deficiencies in the systemd architecture for real-world service management
This is a in-depth architectural critique of systemd. It claims to be the first purely technical review of systemd internals, and provides a detailed analysis of several components. It criticizes on the basis of ordering related failures, a difficult to predict execution model, non-determinism in boot-order, as well as several other points.
Though many users would perceive the long processing pipeline to increase reliability and be more "correct" than the simpler case, there is little to acknowledge this. For one thing, none of jobs, transactions, unit semantics or systemd-style dependencies map to the Unix process model, but rather are necessary complications to address issues in systemd being structured as an encapsulating object system for resources and processes (as opposed to a more well-defined process supervisor) and one accommodating for massive parallelism. Reliability gains would be difficult to measure, and that more primal toolkits like those of the daemontools family have been used in large-scale deployments for years would serve as a counterexample needing overview.
(Score: 3, Funny) by PiMuNu on Friday October 16 2015, @08:07AM
I am essentially a dumb user on this front. Has anyone made a similar critique of init.d? It would be interesting to put them side-by-side and then consider "a way forward".
(Score: 1, Funny) by Ethanol-fueled on Friday October 16 2015, @08:18AM
It doesn't matter -- anti-systemd punks are just millenial punks. They won't actively revolt against anything, they'll just bitch about it on Facebook and Twitter, and the few of them who think they have the fortitude to enumerate their positions will scatter like roaches once their movement is under attack because their movement has no cohesion -- political correctness will cause it to collapse and the dumb sons of bitches have no idea how to prevent it under the attack of outside influence.
(Score: 5, Informative) by Anonymous Coward on Friday October 16 2015, @08:43AM
Millennial punks? Political correctness? Quit over-indulging your bête noir obsessions. Pottering is a millennial for fuck's sake. The anti-systemd folk are UNIX greybeards.
(Score: 1, Insightful) by Anonymous Coward on Friday October 16 2015, @09:25AM
Well... that was a fucking weird rant
(Score: 2, Informative) by Anonymous Coward on Friday October 16 2015, @03:43PM
that's Ethanol for you
(Score: 5, Insightful) by ledow on Friday October 16 2015, @08:19AM
I'm much more of the opinion that unless I've noticed a shortcoming, or one can be demonstrated that affects my usage, that we shouldn't go changing things.
I don't see what's WRONG with other systems, certainly nothing so major as to justify a drastic "every-part-of-the-system" upgrade. I'm sure there are nice new features. In fact, I'm positive of it. I'm sure that lots of things can be done.
What I don't get is what's currently so wrong that we need to change things, and need to change things so drastically that we can just fix the existing system but have to throw not just the implementation but the very idea out and start again from scratch through almost every part of a working system.
That's the bit that bugs me. People come in and mention this and that feature and I think "Well, why couldn't we do that with the 'old' way too?" Things like cgroups are KERNEL features, not systemd features. What's stopping us wrapping up the existing init systems in the same way using the same features.
Just the fact that virtually every command, every file, every way of starting a program or service has changed blows it out of the water for me. Hell, just put in a couple of legacy compatibility layers while you're pitching it at least. But, no, apparently even the COMMANDS are so bad that they have to change.
That's what I don't get. The ideal may be right, but the execution and management of it are so terribly, terribly wrong.
(Score: 2, Disagree) by bzipitidoo on Friday October 16 2015, @09:54AM
Biggest prob with init.d, or, really SysV init, and its incompatible alternate, BSD, is the shell scripting. There are many incompatible shell script dialects, though bash has emerged as the most popular. Shell script is an inferior Algol which itself is long dead, replaced by better languages. It's interpreted, it likes to run in a terminal, it depends upon these environment variables. Any library functionality it needs has to be loaded in the heaviest, slowest possible manner as a full blown executable binary that is linked to the desired library. To do some trivial text processing, might need to run grep, or sort, head or tail, awk, and cat. It just can't interface with C any more elegantly. Though, the fault for that it could be argued, is more with C. Lisp was a better choice of language. Perl was intended to replace it.
Now this system.d is trying to replace much, much more than the shell scripting. I like being able to check logs with "tail /var/log/messages", and not journalctl. I especially dislike the idiot decision to compress the current log file. There wasn't anything wrong with logrotate. Even worse, I've heard that system.d sometimes fails to log messages.
(Score: 0, Insightful) by Anonymous Coward on Friday October 16 2015, @10:27AM
And it's not just the speed. Programming something other that a | b | c using BASH is a real pain to read and write. Not to mention 1001 ways of shooting yourself in the foot with nuclear strikes in the process (hello, correct escaping, real data types…). If you want scripting, fine. Just use some real language to do that, not a bunch of sticks bound together with a duct tape.
(Score: 2) by hendrikboom on Friday October 16 2015, @03:56PM
It's not well-known, but scripting languages have come a long way. Scripting has come a long way, but not in common usage. You can now use a statically-typed, compiled language [mjambon.com] for scripting. It's statically type-checked, has a nontrivial syntax, and is compiled to a byte-code (or, if you prefer, machine code).data structures, modules, and gets compiled to byte-code (or, if you prefer, machine code) for speed.
(Score: 2) by sjames on Monday October 19 2015, @06:39AM
The nice thing about SysV init is that it doesn't care what you use for the "scripting" You can pick anything you want including compiled binaries. It really doesn't care WHAT S05myservice points to (or even where it points to as long as it's been mounted), it will run it with the argument "start".
The complex scripts are mostly just distro maintainers shooting themselves in the foot with a shotgun.
(Score: 2) by janrinok on Friday October 16 2015, @01:36PM
Of course, you can still run the traditional log as well as its new equivalent, and then you have access to grep etc. I agree, running both seems a bit of a waste of time, but I use it and I have the best of both worlds (on the 1 machine that I have running systemd)
(Score: 3, Informative) by Foobar Bazbot on Friday October 16 2015, @02:15PM
While shell script is conventional, it is not required -- you could just as easily write an init script in awk, perl, or even in a compiled language. There's some issues with ensuring that any required interpreter or library is available when needed (e.g. if it lives on /usr, and /usr isn't mounted yet), but nothing that can't be dealt with.
So write a custom interpreter that encapsulates the boilerplate logic in the interpreter, and only needs to read the essential data from its script. If you drink the systemd kool-aid, you'll surely design your interpreter around .INI-style syntax, because reasons -- hey, it could even be a subset of systemd's unit file language! But it could also take YAML, JSON, or even something bizarrely normal, like one KEY=value assignment per line (without .INI-style [section]s, and with escaping/quoting like shell assignments). For one service at a time, you replace the shell script in init.d with one using this new interpreter -- the whole system doesn't have to change at once, or include any sort of compatibility layer for unconverted shell scripts.
This doesn't directly address dependencies or default enable/disable status for each runlevel, of course, but neither does shell script; if your chosen syntax is reasonably compatible with shell script, it won't be an issue. For instance, if your distribution uses standard LSB dependency blocks, as shown below, just include one in your script, and then standard install_initd will do the right thing. If your chosen syntax doesn't treat lines starting with # as comments, maybe you can encapsulate that in a block comment... or just maybe, pick a saner syntax.
Finally, a brief example of what such a script might look like, for a very simple service:
#!/bin/inithelper
### BEGIN INIT INFO
# Provides: lsb-ourdb
# Required-Start: $local_fs $network $remote_fs
# Required-Stop: $local_fs $network $remote_fs
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: start and stop OurDB
# Description: OurDB is a very fast and reliable database
# engine used for illustrating init scripts
### END INIT INFO
daemon=/usr/bin/ourdb
startargs="--daemon --config=/etc/ourdb.conf"
(Score: 0) by Anonymous Coward on Friday October 16 2015, @02:48PM
#!/bin/inithelper
### BEGIN INIT INFO
# Provides: lsb-ourdb
# Required-Start: $local_fs $network $remote_fs
# Required-Stop: $local_fs $network $remote_fs
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: start and stop OurDB
# Description: OurDB is a very fast and reliable database
# engine used for illustrating init scripts
### END INIT INFO
daemon=/usr/bin/ourdb
startargs="--daemon --config=/etc/ourdb.conf"
----------------------------------------------------------------------
Look's a lot like /etc/rc.conf & rc.local on a FreeBSD box to me.....
Always thought it was a bit less complicated than the whole Linux run level thing myself but to each is own...
(Score: 0) by Anonymous Coward on Friday October 16 2015, @03:04PM
Why can't you just write a script that is valid and issue update-rc.d defaults ?
At least if you are just inserting a one off startup script. I don't know what the fuss about Bash is...it is pretty straight forward for most things. For everything else there is Python.
(Score: 2) by Eunuchswear on Saturday October 17 2015, @11:09AM
Yeah, everyone has heard this.
Nobody can ever point to a bug report.
Odd that.
Watch this Heartland Institute video [youtube.com]
(Score: 1) by zoefff on Tuesday October 27 2015, @03:23PM
Duh, because you have to provide logs for that. Kind of catch 22.
(Score: 2) by sjames on Monday October 19 2015, @06:42AM
Then pick something else. SysV doesn't care one way or the other. One of my problems with systemd is that it doesn't allow that in a meaningful way.
You can even throw out all of the scripting by starting something else in /etc/inittab. Once again, SysV doesn't care (even if it's not really SysV at that point).
(Score: 2) by Eunuchswear on Saturday October 17 2015, @11:07AM
Nothing. Do it. Let a million flowers bloom.
Watch this Heartland Institute video [youtube.com]
(Score: 5, Informative) by canopic jug on Friday October 16 2015, @08:29AM
Has anyone made a similar critique of init.d? It would be interesting to put them side-by-side and then consider "a way forward".
You can't compare apples and oranges. systemd is not an init system, though it has been marketed as one. It includes, far, far more. And that is one of the many complaints against it. However, if you want to look at just init systems, there is a recent overview of the history of modern init systems (1992-2015) [darknedgy.net]. That touches on them all.
Money is not free speech. Elections should not be auctions.
(Score: 2) by PiMuNu on Friday October 16 2015, @08:33AM
Thanks, that looks great!
(Score: 1) by isj on Friday October 16 2015, @09:05AM
I've written my share of init-scripts, and I have no love for traditional init.d stuff because of its shortcomings:
- race conditions on pid-files
- inability to specify fine-grained dependencies (eg. an NFS mount depends on a specific network interface as opposed "the network")
- lots of boilerplate code in init scripts
- runlevels are obsolete
- differences between distributions (although that is not exclusive a problem for init.d)
- sequential execution
For a bog-standard daemon I find it unreasonable to force me to write more than 3-4 lines describing the service. It is the responsibility of init system to keep track of pids, make sure daemons don't fork away and become untraceable, provide a unified interface for start/stop/reconfigure/reload. Over the years lots of tools and bandaids have been added to init.d to alleviate its shortcomings, but they still feel like kludges and still require me to write larger scripts.
(Score: 5, Insightful) by Thexalon on Friday October 16 2015, @12:32PM
I've written init scripts as well, and while I don't like it, I also have seen nothing that convinces me that systemd solves the problems you listed any better than those init scripts.
Which to the best of my knowledge is not a problem systemd solves. If it does, you're probably using pid files for the wrong reasons, because pid files should not be to determine whether a service is running - the right thing to check is whether it's responding as expected. The reason for that is that you aren't looking for, say, MySql, you're looking for "Is there a relational database that speaks the MySql dialect of SQL listening on port 3306"? That way, if the user switches to something that is different but compatible, you don't have to change what you're doing.
Since there's no rules about what you can't include in a shell script, what I would recommend doing is to look for that specific condition at the beginning of your shell script.
Can't you include a set of functions that eliminates that problem?
The "standard" runlevels all seem like pretty good concepts to me: shutdown, single-user recovery, no remote services, full access but no GUI, full access with a GUI, reboot. Which concept would you consider obsolete, and why?
That is entirely to be expected when packaging software. As in, those kinds of system variations are a big part of what a distribution does, unless you are now demanding that everything in the universe not only conform to one of the big boys' standards (yum/rpm, apt/dpkg), but conform to a particular placement of all files beyond the FHS [pathname.com] (e.g. what goes in /usr versus /usr/local, how /etc is organized). And what about BSDs and other non-Linux POSIX-compliant systems which do things significantly different from Linux but should still have access to all the same application software?
So? All software makes use of sequential execution. At a low-level, it's what CPU cores do for their entire existence. Unless it's causing some kind of problem, and in this case it isn't, because (a) reboots and runlevel changes are relatively rare, and (b) they're fast enough that nobody really had a problem with it for a really long time. For what it's worth, on the init.d-based systems I've run the time to go from starting to boot to fully operational was less than the time for the BIOS to finish its POST.
My understanding, from everything I've seen with systemd, is:
1. There is no such thing as a bog-standard daemon. They all tend to have their quirks. (Otherwise, nobody would need to change or replace them to make them work on a systemd-based machine.)
2. You can handle those quirks with shell scripts, or with C. Of the two, shell scripts are far less error-prone and less catastrophic when things go wrong.
Basically, what I see from systemd is taking something that is flexible and easily changed by a system administrator to something that is rigid and only can be changed in the source code of a wide variety of packages.
The only thing that stops a bad guy with a compiler is a good guy with a compiler.
(Score: 2, Interesting) by isj on Friday October 16 2015, @08:13PM
My post wasn't advocacy of systemd. It was a critique of init.d-style systems.
The problem with pid files is that they need to exist in such systems. Some daemons write their own pid-file. Some assumes you do it. Some forks away, some don't, etc. And startdaemon (or is it daemonstart?) punts and leaves that to the daemon, which requires more scripting/code.
Why should each daemon or init-script have code to write that pid file? I expect a daemon/service manager to keep track of that for me. But init.d-style systems don't.
I agree that checking if a service is functioning/available shouldn't be done with pid-files. However, the does-process-exist is used for answering the question "is service X running". Not the question "is service X functioning?".
As for the clients of the service they should of course only use the service API (sockets, files, whatever).
Since there's no rules about what you can't include in a shell script,...
More scripting...
Can't you include a set of functions that eliminates that problem?
More scripting...
Please stop asking me to write more scripting.
init.d-style systems generally assume that all daemons are special snowflakes and therefore don't provide reasonable built-in functionality for well-behaved daemons.
Let's assume that I have a well-behaved daemon, say, soylentd. In the service description file I expect that I have to specify:
- Which services/resources the daemon relies on.
- Which program/script to run to start the service.
- Whether the service forks away or not.
and then the service/daemon manager provides suitable defaults for the rest:
- internal service name: derive that from the service description filename
- how to stop: assume SIGTERM, and if that doesn't work SIGKILL
- how to reconfigure: SIGHUP is usually supported
- keep track of pid/process-group/control-group
I don't expect to include /etc/init.d/functions, write a case statement, use arcane echo options, or even have a #!/bin/sh line
I do expect the ability to override the built-in defaults for those special snowflakes/daemons.
>runlevels are obsolete
The "standard" runlevels all seem like pretty good concepts to me: shutdown, single-user recovery, no remote services, full access but no GUI, full access with a GUI, reboot. Which concept would you consider obsolete, and why?
Perhaps I was a bit hasty. I do find runlevel 1 useful when debugging boot/initialization problems. But the last time I found the distinction between runlevel 2/3/4/5 useful was back in the previous millennium. I would find 3 states more useful:
1: off
2: under service/maintenance (minimal services are runnig. More can be started manually)
3: fully running
> sequential execution
...
My point is not to use multiple CPU cores to initialize - that is usually not the bottleneck. My point is that when a service is a bit slow to start, eg. the network due to a DHCP timeout there is no reason to delay the start of X/Windows, initializing USB devices, etc. In fact, if I need to debug that DHCP problem I need X/Windows and my USB mouse. So starting the services one by one is counter-productive.
(Score: 2) by sjames on Monday October 19 2015, @06:51AM
Some of the daemons I write have NO scripting other than for convention. You can directly link the executable into rc?.d if you like. Too bad systemd can't handle that.
In other cases, my script is just a few lines, one for each of start, stop, status, and reload (sometimes not even status and reload). It doesn't get much simpler.
It's really up to you how simple or complicated you want to make it.
(Score: 2) by sjames on Monday October 19 2015, @06:54AM
There are several parallel start mechanisms already out there that will keep things like DHCP from hanging everything else up.
(Score: 2, Insightful) by Anonymous Coward on Friday October 16 2015, @01:42PM
How is this a problem?
You have access to /proc
Write an include file
For you, perhaps
It's not between distributions, it's between systems and it is the job of the system administrator to customize init scripts for their specific requirements.
This could have been fixed in initd without reinventing the wheel
(Score: 1, Insightful) by Anonymous Coward on Friday October 16 2015, @02:46PM
For you [...]
Was getting caught part of your plan?
(Score: 2, Funny) by Anonymous Coward on Friday October 16 2015, @05:05PM
I'd love to reply, unfortunately your ill-conceived snark has just crashed the discussion. Please choose an advanced option to continue.
.
HTH, HAND!
(Score: 3, Interesting) by Whoever on Friday October 16 2015, @03:30PM
Take a look at OpenRC some time. Most of your criticisms are solved by it. For example, with OpenRC under Gentoo, I could specify a dependency on "net.eth1" (ie, the eth1 interface). OpenRC supports dependencies so that the order in which services are starter is not manually specified. It doesn't have the traditional 1-6 run levels. Instead, most services are in the "default" run level. It supports parallel service startup. The dependency information is also used when re-starting a service: dependent services are also restarted.
(Score: 5, Informative) by Anonymous Coward on Friday October 16 2015, @10:04AM
init.d -> hackish, programmable
systemd -> posix thrown out the window, domain specific language, no fucks given about breaking stuff
The problem is that you don't get to pick one, you HAD one and you switched to the other, which means only one thing: powerful interests want wheel reinventing because linux was becoming too good. All the rest about systemd features/problems does not get this. Systemd is the symptom, thirst for $$$ is the problem.
(Score: 5, Insightful) by Anonymous Coward on Friday October 16 2015, @11:11AM
An (overly?) simple analogy would be manual car windows vs. powered windows operated by the car's buggy entertainment system thru the big fat finicky center panel touch console. :) Now imagine your ignition, door locks, fuel door, trunk lid, and power steering is routed through the same system.
(Score: 1) by redneckmother on Friday October 16 2015, @02:15PM
Excellent example. Have you read Normal Accidents or watched the movie Brazil?
Mas cerveza por favor.
(Score: 0) by Anonymous Coward on Friday October 16 2015, @11:32AM
systemd was the critique of init.d
(Score: 0) by Anonymous Coward on Saturday October 17 2015, @03:12PM
lold
(Score: 0) by Anonymous Coward on Friday October 16 2015, @06:23PM
This was asked and answered on Reddit [reddit.com].
(Score: 4, Funny) by NickFortune on Friday October 16 2015, @09:36AM
... oh wait.
Sorry. Never mind.
(Score: 0) by Anonymous Coward on Friday October 16 2015, @10:53AM
... the introduction of systemd in many of the leading linux distros.
The name of the kernel is "Linux". Let's show it the respect it deserves by at least getting the name right.
(Score: 2, Funny) by Anonymous Coward on Friday October 16 2015, @11:48AM
LiNuX.
(Score: 1, Touché) by Anonymous Coward on Friday October 16 2015, @12:00PM
I'd just like to interject for a moment. What you’re referring to as Linux, is in fact, GNU/systemd/Linux, or as I’ve recently taken to calling it, GNU plus systemd plus Linux. Linux is not an operating system unto itself, but rather another free component of a fully functioning GNU system made useful by the GNU corelibs, shell utilities and vital systemd components comprising a full OS as defined by Poettering.
Many computer users run a modified version of the GNU system every day, without realizing it. Through a peculiar turn of events, the version of GNU which is widely used today is often called “Linux”, and many of its users are not aware that it is basically the GNU system, developed by the GNU Project. There really is a Linux, and these people are using it, but it is just a part of the system they use.
Linux is the kernel: the program in the system that allocates the machine’s resources to the other programs that you run. The kernel is an essential part of an operating system, but useless by itself; it can only function in the context of a complete operating system. Linux is normally used in combination with the GNU operating system: the whole system is basically GNU with systemd and Linux added, or GNU/systemd/Linux. All the so-called “Linux” distributions are really distributions of GNU/systemd/Linux.
(Score: 0) by Anonymous Coward on Friday October 16 2015, @05:11PM
Wow. So you Borg guys have now expanded user-space utilities into the operating system onto which the kernel is just sitting in the corner somewhere. I suppose if I don't want to run linux, I can just unload that kernel thing which I presume is just some module hanging on to the great GNU OS? All I need to do is write my own ls, grep, etc. and I can now proclaim the mighty Anonymous Coward Operating System!!!
Man, I've lived through the GNU semantics wars of the 90s and that sounded SO out there, and I thought I'd heard it all.
GNU: Resistance is futile. You will be assimilated.
(Score: 3, Insightful) by maxwell demon on Friday October 16 2015, @06:26PM
Is it? Qiuoting froim the post you replied to (emphasis by me):
As far as I know, there's no GNU in the kernel. Not even systemd.
The Tao of math: The numbers you can count are not the real numbers.
(Score: 2) by fido_dogstoyevsky on Friday October 16 2015, @08:41PM
All the so-called “Linux” distributions are really distributions of GNU/Linux, many but not all of which include systemd.
Ftfy
It's NOT a conspiracy... it's a plot.
(Score: 0) by Anonymous Coward on Friday October 16 2015, @10:35PM
I used to always install "unix tools" on my Windows machines. I had no idea that my machine suddenly became a GNU/Windows system. I had been saying it wrong for YEARS! Thanks for setting me straight.
I have absolutely NO IDEA where GPL critics glom onto the idea that anything that gets touched by GNU/GPL becomes infected and tainted. It can't be from overreaching people like you, THAT'S for sure.
(Score: 4, Insightful) by jmorris on Saturday October 17 2015, @02:40AM
Don't worry about the naming thing, Pottering OS has a goal of a GNU Free system. Linux the kernel as device drivers with Pottering's New Technology stuff as the glue between the kernel and GNOME or server processes. Pottering is a long time UNIX Hater who dreams of a day when you can boot to a desktop without a shell, ls, grep, awk or any of that other UNIX stuff even installed. Probably won't even need a terminal emulator except to run whatever abomination PowerShell ports in as.
(Score: 3, Disagree) by janrinok on Friday October 16 2015, @01:47PM
... and the correct use to describe the distros is 'linux distros'. I was not referring specifically to the kernel, but to the various OS that use the Linux kernel.
(Score: 2) by janrinok on Friday October 16 2015, @04:48PM
Having checked with various books and publications, I find that both are in fairly common usage. I'm not sure if this makes them both correct, but at least if I am wrong then I am not alone.
(Score: 1, Insightful) by Anonymous Coward on Friday October 16 2015, @05:17PM
At least these days, particularly on the Interwebs, in these kind of things you are never wrong. You only have to condescendingly remark: "Don't you know that language evolves?"
It is a little-known fact that the Internet is run by Humpty Dumpty:
(Score: 2) by darkfeline on Friday October 16 2015, @11:27AM
>non-determinism in boot-order
How is this a bad thing, so long as dependencies are being started before their dependents? I think the "old" way of potentially relying on race conditions, where process A must be started three daemons before process B and no one knows because the init script kept working since it was written in 2001, to be significantly worse than "non-determinism in boot-order".
In fact, forced randomness would be good for fishing out these kinds of problems early so they don't cause a huge headache later on.
Join the SDF Public Access UNIX System today!
(Score: 4, Insightful) by Anonymous Coward on Friday October 16 2015, @11:35AM
>I think the "old" way of potentially relying on race conditions, where process A must be started three daemons before process B and no one knows because the init script kept working since it was written in 2001, to be significantly worse than "non-determinism in boot-order".
In theory, theory and practice are the same. In practice, they are not.
And in practice, things which are working are *always* better than things which are not.
(Score: 5, Insightful) by fritsd on Friday October 16 2015, @12:27PM
why is non-determinism bad?
Well, I guess it's simpler to debug and solve a problem of the form: "my corporation's order processing system often crashes between database startup and load-leveling front-end webserver start up", rather than a problem of the form: "my corporation's order processing system crashes 17.7% of the time and everyone is yelling at me".
Humans and computers create problems. Humans fix problems. Humans have only a limited capacity of understanding, so the init system must be structured in a "human-repairable" way otherwise it becomes byzantine, cryptic and error-prone.
Incidentally, about fast boot times: if you hear "my corporation's order processing system can restart from a crash several times per second", YOU'RE DOING IT WRONG.
(Score: 0) by Anonymous Coward on Thursday October 22 2015, @05:50PM
The pro-systemd crowd will likely spin it to say that they can spin up 1000 instances of the order processing system in second.
See for them only two things matter, desktop and containers/VMs. And in both instances fast startup and shutdown is preferred over long uptimes (or at least it seems so given that this is what gets priority by the systemd devs).
(Score: 3, Informative) by Anonymous Coward on Friday October 16 2015, @01:41PM
For desktop and some server usage non-determinism is not a bad thing per-se. However it makes things actually harder to debug. As you are not sure what order something happened in. Take for example you are debugging a bit of code. It blows up every 100th run. But it is doing so because some module you assume is 'there' just isnt. In fact your code is assumed to run after this other code. So now you have a new issue that did not exist before. How to 'wait' or 'force load'. Then what if you cant force it to load? What if you double load it? How long do you wait? Remember it is non deterministic and you want the OS to CLI to show up someday so you have to time out. It is not a good user experience when randomly your computer takes 5 mins to spin up when normally it takes 30 seconds.
For a RTOS sort of situation determinism is the name of the game. These dudes figure out how many cycles something takes to run. They make sure it takes that and no more and no less. It is deterministic. They will balk at it because they have years of exp telling them otherwise. It may be 'ok' but they will not sit down while something takes 3-50 seconds to startup depending on what phase of the moon something is in. They *need* it to startup in 3 seconds. It needs to do that every single time or something physical will literally blow up.
The *idea* of systemd is a good one. A better init system. One more like what OSX and Windows have. A nice single way to talk to the services. A nice clean way to jail them off from root. A nice clean way to handle the life cycle and dependencies between services (the network stack should be running before you spin up the proxy server sort of thing). Something all the current init systems do very well but all do differently with some holes and all of them with different commands that pretty much do the same thing. Like windows has a nice 'are you alive' ping that you do to services. You could build something like that into each app but then it is custom for every single one. OSX and Windows have a nice predefined way of doing that. systemd went off the rails thinking it should be/contain all those services. Throwing out years of work on other services with the idea 'it will be clean' when all you end up doing is creating a whole new beast with different bugs and missing features. He basically did the same thing he did with audio. Recreated the whole stack and then screwed it up *again*.
(Score: 3, Informative) by jdccdevel on Friday October 16 2015, @07:55PM
Non-determinism is ALWAYS bad in a computer's startup sequence, it should be removed whenever possible.
- In a deterministic system, I (a system administrator) can always be certain that A will start before B, because services will always start in the same order.
- In a non-deterministic system I cannot, unless there is a dependency between them. The dependencies are one way to try to bring a measure of determinism to the system.
The problem is that dependencies are not always easily defined, nor is it obvious if a given configuration change will require some change in dependencies.
How does a system administrator know if the computer starting properly was a fluke or not? The admin cannot know! It may be that 90% of the time things will start correctly, it may only be 10% of a one-off fluke, There is no way to know! I have more important things to do with my time than spend hours troubleshoot something as basic as a computer not starting some random percentage of the time, just because someone else thinks I need to save 30 seconds on my boot time once a month! (If that!)
Even once the problem is isolated, sometimes it is NOT POSSIBLE to express the dependency as a simple A before B, sometimes dependencies are dynamic.
For example, if I make a configuration change to a service, and suddenly my service depends on a library, which depends on a service that it didn't depend on before. How is that reflected in a systemd service file? In an init script, the startup script could account for that, but not in a systemd service.
Here's a real-life example I ran into the other day.
I have a arch system that loads some firewall rules, including NAT. I have some custom settings related to NAT in my sysctl configuration. One day I rebooted, and some of my sysctl settings were gone. Not all of them, just the NAT ones.
The problem was that _SOMETIMES_ the sysctl settings would be applied before the NAT module was loaded, and so my sysctl settings for NAT were not being applied.
Easy fix, you say, apply the settings afterwards. That isn't the point.
Why should I be required to hunt down issues like that? I moved my systems to Linux for stability and predictability. Specifically so that I would not have to deal with issues like that.
If my system starts correctly once, I want it to do so EVERY TIME. Not 90%, not even 99%. Nothing less than 100% of the time is good enough.
That is simply not possible with a non-deterministic boot system.
(Score: 1) by shrewdsheep on Friday October 16 2015, @08:37PM
This is a straw man argument.
systemd could (and probably should) have an option to start sequentially.
(Score: 0) by Anonymous Coward on Thursday October 22 2015, @05:53PM
Options bad, m'kay! All hail the one and only "init": systemd!
(Score: 2) by Justin Case on Saturday October 17 2015, @02:47PM
Non-determinism is ALWAYS bad in a computer's startup sequence^W^Wbehavior
FTFY
(Score: 4, Informative) by rleigh on Friday October 16 2015, @08:53PM
Former Debian sysvinit/initscripts maintainer here.
I'd just like to correct a couple of points here. Firstly, the "old" way has never been to rely on race conditions of any sort. Neither the (old) by-hand numerical ordering or the (new) LSB header dependencies ever *relied* on races of any sort. There may have been implicit ordering requirements not formally encoded in the dependencies, but that in no way means non-determinism, nor does it imply incorrect behaviour.
The second point is regarding determinism in systemd and sysvinit. With sysvinit LSB header dependencies, insserv constructs a dependency graph and serialises this as a make-style dependency list. startpar, when run serially, will effectively flatten the graph to a linear sequence. This is completely deterministic. startpar, when run with parallelisation enabled, will be able to use the make-style rules to parallelise appropriate parts of the sequence; while this is less deterministic the overall progression is clearly and explicitly defined, and the dependencies are hard requirements which are enforced, e.g. if a script depends upon three other scripts, those three are guaranteed to have run first. Note that this assumes the scripts behave properly, which is generally the case. As the article states, the dependencies are not enforced to quite the same level of strictness with systemd. With Ubuntu 15.04 on my work PC, the systemd boot is both slower than the sysvinit boot, and less reliable--most of the time it boots, but occasionally it hangs somewhere in the middle of booting and requires a hard reset--it being in a state which precludes any interactive debugging of where and why it got stuck. This is vastly less deterministic than what we had with sysvinit, and a definite step backward--a computer should be able to consistently boot.
The main wart in sysv-rc/initscripts is the hack to make network hotplug events from udev trigger init actions, but in practice while ugly and complex, I wouldn't say it was non-deterministic. You have a hotplug event leading to a defined set of actions. Could be improved for sure, but it's quite functional.
This is in no way to imply that sysvinit/sysv-rc are perfect, they clearly have some problems, but much of the criticism in favour of systemd is quite dishonest.
(Score: 2) by sjames on Monday October 19 2015, @07:01AM
Service X fails to start 33% of the time tells me nothing. Service X fails to start on boot but starts just fine manually = I missed a dependency.
An indeterminate system creates heisenbugs.
(Score: 0) by Anonymous Coward on Wednesday November 04 2015, @04:58AM
Getting away from heisenbugs and other "spooky" behavior is why people moved from Windows to Linux in the first place.
(Score: 2) by opinionated_science on Friday October 16 2015, @01:31PM
at least it gets the ball rolling...
I am a systemd user, and finally after a few years, I get its quirks.
My 2 cents, is that it is definitely an improvement over scripts as it permits *user* level service control. For things such as multi-seat display this is critical.
The one feature I would hope systemd would adopt, is the equivalent of "serialise" used for debugging. In other words a slow-and-steady-boot to deliberately.
I believe the current system it is possible to prescribe a deterministic boot, if you specify it as such.
(Score: 0) by Anonymous Coward on Friday October 16 2015, @03:00PM
Only skimmed TFA, but this may be where they hid that back-door:
(Score: 1) by khallow on Friday October 16 2015, @03:51PM
(Score: 2) by Eunuchswear on Monday October 19 2015, @11:15AM
So, where is the NSA back-door in sysvinit? That does nothing but parsing at run-time, it's all written in sh(1) for god's sake.
Watch this Heartland Institute video [youtube.com]
(Score: 5, Insightful) by PizzaRollPlinkett on Friday October 16 2015, @03:54PM
The thing about System D to me is that it's not Linux. I have used UNIX since before Linux existed. I know Linux's startup procedure. It works. Everything is cool. I'm good with it. I don't want another operating system. If I did, I'd go look for one. If System D was another operating system, it would be great, just like there's Apache and nginix or Linux and FreeBSD or iOS and Android. There is probably a need in datacenters for Docker and System D, but they're not Linux. I don't want Linux to have its normal startup ripped out and replaced, because then I have to learn System D. I don't want to do that. I want to use the same skills I've had for decades. If there's a need for a new startup system, then great, people who need it can run it and learn it. Ripping out the normal startup procedure is the same difference as ripping out the shell and putting a VMS command interpreter in its place, or ripping out Perl and putting REXX in its place. Choice is great. If someone wants to use REXX, great, use it. As long as my scripting language isn't ripped out. The problem with System D is that it's ripping out what works and what has always worked without any choice. Just goes to show that Linux isn't really as open as we thought it was. This change is being pushed by a few people who want to rip out the heart of Linux and make it something else.
BTW - this is the WORST formatted, unreadable article ever. If you zoom in to make the text big enough to see, it gets cut off on the right. Never has something so high quality been presented so poorly.
(E-mail me if you want a pizza roll!)
(Score: 2) by maxwell demon on Friday October 16 2015, @06:30PM
Simply choose "no style". While the formatting is horrible, the HTML is sane, and displays properly with "no style".
The Tao of math: The numbers you can count are not the real numbers.
(Score: 0) by Anonymous Coward on Saturday October 17 2015, @03:35PM
Agree about presentation. Fortunately Firefox has Reader Mode
http://www.betterhostreview.com/enable-firefox-reader-view.html [betterhostreview.com] (yes, this is the best link I found explaining it ....)
(Score: 0) by Anonymous Coward on Thursday October 22 2015, @03:16PM
There, it should adjust to zooming now.
(Score: 3, Informative) by present_arms on Friday October 16 2015, @04:44PM
I remaster PCLinuxOS where systemd isn't touched, we do use parallel booting and have good old fashioned logs :) I have ran systemd distros but they never last long on my boix, one bug or another raises it's head and although I know that no software is bug free, having bugs in init is just stupid. Then when I do look for the problem, I have to boot a dvd and use some crap to look at the logs... read that you can make then human readable again, tried that, most of the logs are missing.. so for now, just like I did with KDE4, Pulse Audio, I'm avoiding anything that's systemd, can't see the point when nothing was wrong in the first place. But that could be just me :)
Alie
http://trinity.mypclinuxos.com/
(Score: 1) by fritsd on Friday October 16 2015, @07:06PM
It's not just you :-)
(Score: 0) by Anonymous Coward on Saturday October 17 2015, @02:51AM
...if it were out of the hands of Lennart Poettering, freedesktop, and all of the ignorant kiddies who don't understand the nuances of running mission-critical systems.
Don't get me wrong, one of systemd's best features is its robust dependency tree processing, leading to ultra-fast start times. But systemd is still immature. The "5 years" quoted don't really count; it's only been about 1 to 3 years since EVERY distro foisted systemd on their users without a proper long-term shakedown. It makes me think of Paul Biggar of circleci, saying in his app development diatribe "It really is the future," that "We built REST and AJAX and JSON over the corpses of SOAP and CORBA, using the lessons we learned while building them."
We need to build something on the corpse of systemd, which so hastily dispatched of the venerable but agreeable init. Hopefully this will be done while keeping activist stewards Red Hat and Canonical at two swords length.
(Score: 2) by sjames on Monday October 19 2015, @07:09AM
That is one of my problems with systemd, unlike SysV init, it is not at all willing to play nice with others. You either use it or rip it out. There is no just use it for these services after booting or I'd like to replace this bit with something else. It's their way or the highway. I choose the highway.
(Score: 2) by RamiK on Saturday October 17 2015, @05:08PM
Someone hosting on plan9\werc should know better by now. But, if good and correct system design was a selling point, we wouldn't be debating *nix vs. NT in 2015. We would be debating Plan 9 vs. QNX.
Systemd has just enough planned obsoleteness in it's inherently flawed abstractions to keep system distributors in business for years to come. Seeing how that's, in effect, is the Linux development model, I wouldn't use that particular point against it's Linux adoption. And to be fair to Linux, the alternative isn't a Plan 9 or even a BSD utopia. Rather, it's a closed-source Microsoft \ Apple store.
For BSD, however, the essay raises many valid points. Keep that systemd mess away from other unices if at all feasible. It has no technical or practical benefits for non-Linux.
Here's hoping some saintly figure will write a modern browser for Plan 9 and port the OS to ARM64 when the time comes. Until that temperature drop in Lucy's house, may GNU\linux\systemd\firefox stay open and free as possible.
compiling...
(Score: 0) by Anonymous Coward on Tuesday October 20 2015, @06:09PM
Better to keep RedHat and Canonical at 1/2 sword's length. Use a real sword for measurement.