Stories
Slash Boxes
Comments

SoylentNews is people

SoylentNews is powered by your submissions, so send in your scoop. Only 19 submissions in the queue.
posted by janrinok on Sunday December 21 2014, @04:54PM   Printer-friendly
from the show-stopper-or-rare-event? dept.

Noted Linux expert Chris Siebenmann has described two catastrophic failures involving systemd.

One of the problems he encountered with systemd became apparent during a disastrous upgrade of a system from Fedora 20 to Fedora 21. It involved PID 1 segfaulting during the upgrade process. He isn't the only victim to suffer from this type of bad experience, either. The bug report for this problem is still showing a status of NEW, nearly a month after it was opened.

The second problem with systemd that he describes involves the journalctl utility. It displays log messages with long lines in a way that requires sideways scrolling, as well as displaying all messages since the beginning of time, in forward chronological order. Both of these behaviors contribute to making the tool much less usable, especially in critical situations where time and efficiency are of the essence.

Problems like these raise some serious questions about systemd, and its suitability for use by major Linux distros like Fedora and Debian. How can systemd be used if it can segfault in such a way, or if the tools that are provided to assist with the recovery exhibit such counter-intuitive, if not outright useless, behavior?

Editor's Comment: I am not a supporter of systemd, but if there are only 2 such reported occurrences of this fault, as noted in one of the links, then perhaps it is not a widespread fault but actually a very rare one. This would certainly explain - although not justify - why there has been so little apparent interest being shown by the maintainers. Nevertheless, the fault should still be fixed.

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 0) by Anonymous Coward on Sunday December 21 2014, @07:23PM

    by Anonymous Coward on Sunday December 21 2014, @07:23PM (#128088)

    A sysadmin contracted to solve a problem can be on contract to be payed hundreds of dollars per hour to fix that issue. Time spent scrolling to find a problem using this rather than the tried and tested tools like 'tail' and 'grep' can result in larger support bills.

  • (Score: 4, Insightful) by maxwell demon on Sunday December 21 2014, @07:28PM

    by maxwell demon (1608) on Sunday December 21 2014, @07:28PM (#128089) Journal

    So the tools don't allow to pipe the output to tail or grep? Then that is the real problem, not overly long log lines or too long logs.

    --
    The Tao of math: The numbers you can count are not the real numbers.
    • (Score: 5, Insightful) by Arik on Sunday December 21 2014, @07:31PM

      by Arik (4543) on Sunday December 21 2014, @07:31PM (#128090) Journal
      Yes, systemd does not produce text logs that can be read using any of the many mature and robust text tools available - it must do them in a binary format which can only be read with their crappy viewer, and yes, no longer producing log files in text is the big problem here. You can't bug-fix your way out of defective-by-design.
      --
      If laughter is the best medicine, who are the best doctors?
      • (Score: 2) by choose another one on Monday December 22 2014, @11:30AM

        by choose another one (515) Subscriber Badge on Monday December 22 2014, @11:30AM (#128284)

        Is there something about "ForwardToSyslog=yes" that doesn't work for you - does that not produce log files in text ?

        Seems to me that going from binary to text is the right way - it's trivial once you've decided on a format - going the other way (if you need the binary logs with full metadata, which I presume somebody does...) requires ensuring the text formattinging is reversible (quoting etc.) and writing a parser. Seems to me that binary-first is the right design if someone needs binary logs.

        • (Score: 1) by Arik on Monday December 22 2014, @11:49AM

          by Arik (4543) on Monday December 22 2014, @11:49AM (#128290) Journal
          The problem is that option fails when boot fails - the one time when you most need it.

          Logs should be in text and converted to binary later if someone really needs binary, not the other way around, because this allows you to read the logs after a failed boot, and binary does not. You can convert from one to another to your hearts content, after you fix the problem and get the machine back on its feet.
          --
          If laughter is the best medicine, who are the best doctors?
    • (Score: 2) by fnj on Sunday December 21 2014, @07:42PM

      by fnj (1654) on Sunday December 21 2014, @07:42PM (#128094)

      Who told you that, or did you just make it up? Tools exist, but if THE DATA ITSELF IS UNWIELDY (e.g., the long lines mentioned), then dealing with the data using any tools, including tail and grep, is going to be unwieldy.

      • (Score: 2) by maxwell demon on Sunday December 21 2014, @08:03PM

        by maxwell demon (1608) on Sunday December 21 2014, @08:03PM (#128104) Journal

        Who told you that, or did you just make it up?

        Did you actually read the post I replied to?

        --
        The Tao of math: The numbers you can count are not the real numbers.
        • (Score: 2) by fnj on Sunday December 21 2014, @08:22PM

          by fnj (1654) on Sunday December 21 2014, @08:22PM (#128112)

          Yes, actually, I did. What caused you to leap to "So the tools don't allow to pipe the output to tail or grep?" Because journalctl fully allows such piping.

          • (Score: 2) by maxwell demon on Sunday December 21 2014, @08:42PM

            by maxwell demon (1608) on Sunday December 21 2014, @08:42PM (#128115) Journal

            This:

            Time spent scrolling to find a problem using this rather than the tried and tested tools like 'tail' and 'grep' can result in larger support bills.

            --
            The Tao of math: The numbers you can count are not the real numbers.
            • (Score: 0) by Anonymous Coward on Sunday December 21 2014, @11:45PM

              by Anonymous Coward on Sunday December 21 2014, @11:45PM (#128155)

              journalctl -b -u ntpd.service | grep waking
              Dec 19 00:51:57 wheatley ntpd[1765]: new interface(s) found: waking up resolver
              Dec 19 00:51:59 wheatley ntpd[1765]: new interface(s) found: waking up resolver
              Dec 19 17:07:45 wheatley ntpd[1765]: new interface(s) found: waking up resolver
              Dec 20 00:19:19 wheatley ntpd[1765]: new interface(s) found: waking up resolver
              Dec 20 07:53:49 wheatley ntpd[1765]: new interface(s) found: waking up resolver
              Dec 20 07:53:51 wheatley ntpd[1765]: new interface(s) found: waking up resolver
              Dec 21 17:49:27 wheatley ntpd[1765]: new interface(s) found: waking up resolver

              I don't see the problem, here.

              • (Score: 0) by Anonymous Coward on Monday December 22 2014, @12:07AM

                by Anonymous Coward on Monday December 22 2014, @12:07AM (#128160)

                let me correct that for you.
                journalctl -b -u ntpd.service | grep waking
                journalctl: log corrupted.

                because they refuse to fix the automatic log corruption if you turn on log rotation.

              • (Score: 3, Insightful) by Anonymous Coward on Monday December 22 2014, @12:36AM

                by Anonymous Coward on Monday December 22 2014, @12:36AM (#128174)

                I'll highlight the problems for you:

                journalctl -b -u ntpd.service | grep waking
                Dec 19 00:51:57 wheatley ntpd[1765]: new interface(s) found: waking up resolver
                Dec 19 00:51:59 wheatley ntpd[1765]: new interface(s) found: waking up resolverC�h�S�%��͚������ER$��[����f
                Dec 20 07:53:51 wheatley ntpd[1765]: new interface(s) found: waking up resolver
                Dec 19 17:07:45 wheatley ntpd[1765]: new interface(s) found: waking up resolver
                Dec 20 07:53:49 wheatley ntpd[1765]: new interface(s) found: waking up resolver
                Dec 20 00:19:19 wheatley ntpd[1765]: new interface(s) found: waking up resolverRy}?���c]MKS�\��E�_��

                The first problem I see is that you need to use journalctl. It should never be a requirement that a log file be filtered through some program before grep can work with it.

                The second problem I see is that some of the lines are out of order. I don't know if the timestamps are wrong, or if they were logged in the wrong order, or if journalctl screwed them up, but clearly some are where they don't belong.

                The third problem I see is the random binary data at the end of some of the lines. That probably should not be there as far as I can tell.