Slash Boxes

SoylentNews is people

posted by NCommander on Monday August 08 2016, @12:00PM   Printer-friendly
from the now-with-a+-scores dept.

So after an extended period of inactivity, I've finally decided to jump back into working on SoylentNews and rehash (the code that powers the site). As such, I've decided to scratch some long-standing itches. The first (and easiest) to deploy was HSTS to SoylentNews. What is HSTS you may ask?

HSTS stands for HTTP Strict Transport Security and is a special HTTP header that signifies that a site should only be connected to over HTTPS and causes the browser to automatically load encrypted versions of a website should it see a regular URL. We've forbid non-SSL connections to SN for over a year, but without HSTS in place, a man-in-the-middle downgrade attack was possible by intercepting the initial insecure page load.

One of the big views I have towards SoylentNews is we should be representative of "best practices" on the internet. To that end, we deployed IPv6 publicly last year, and went HTTPS-by-default not long after that. Deploying HSTS continues this trend, and I'm working towards implementing other good ideas that rarely seem to see the light of day.

Check past the break for more technical details.


As part of prepping for HSTS deployment, I went through every site in our public DNS records, and made sure they all have valid SSL certificates, and are redirecting to HTTPS by default. Much to my embarrassment, I found that several of our public facing sites lacked SSL support at all, or had self-signed certificates and broken SSL configurations. This has been rectified.

Let this be a lesson to everyone. While protecting your "main site" is always a good idea, make sure when going through and securing your infrastructure that you check every public IP and public hostname to make sure something didn't slip through the gaps. If you're running SSLLabs against your website, I highly recommend you scan all the subjectAlternativeNames listed in your certificate. Apache and nginx can provide different SSL options for different VHosts, and its very important to make sure all of them have a sane and consistent configuration.

Right now, HSTS is deployed only on the main site, without "includeSubdomains". The reason for this is I wanted to make sure I didn't miss any non-SSL capable sites, and I'm still working on getting our CentOS 6.7 box up to best-practices (unfortunately, the version of Apache it ships with is rather dated and doesn't support OSCP stapling. I'll be fixing this, but just haven't gotten around to it yet).

Once I've fixed that, and am happy with the state of the site, SN, and her subdomains will be submitted for inclusion into browser preload lists. I'll run an article when that submission happens and when we're accepted. I hope to have another article this week on backend tinkering and proposed site updates.

Until then, happy hacking!
~ NCommander

This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 2) by NCommander on Wednesday August 10 2016, @02:00AM

    by NCommander (2) Subscriber Badge <> on Wednesday August 10 2016, @02:00AM (#386068) Homepage Journal

    You do realize I'm a Debian Developer in addition to an Ubuntu Core Dev, right?. Anyway, my point is sysvinit has issues, and was due for either update and replacement. I thought upstart good replacement for technical reasons. The primary reason it didn't get adopted by Debian was political most over Canonical's copyright assignment policy; I remember it came up several times on d-private.

    * Not really inclined to agree that "it's old so we need to repalce it" is a good argument, especially when distros had still been managing to keep it usable. But that's just a difference of opinion.

    Fine. Let's get into the deep technical flaws with sysvinit.

    This is an outdated claim. Dependency-based boot in Debian's sysvinit has been available nearly ten years, and the default for almost that long. OpenRC has been available longer than that.

    When you refer to dependency tracking, I assume you're referring to this block at the start of the file:

    # Provides: scriptname
    # Required-Start: $remote_fs $syslog
    # Required-Stop: $remote_fs $syslog
    # Default-Start: 2 3 4 5
    # Default-Stop: 0 1 6
    # Short-Description: Start daemon at boot time
    # Description: Enable service provided by daemon.

    Debian's dependency tracking (which is based on LSB) is a serious hack, and one I am aware of. When I was saying sysvinit has no concept of dependencies, I was referring to the daemon itself.

    update-rc.d de-facto works by dynamically ordering the scripts as part of update-rc.d. Boot dependencies are statically specified, and from the original thread where this was implemented, Tollef Fog Heen [] provides an example of why this is a bad thing. The system is incredibly brittle because it can't resolve dependency loops or the case of something failing to start. The reason it all appears to work fine is considerable effort is put in by DDs to keep sysvinit chugging along.

    This is also an outdated claim. Parallelised boot (which I'm guessing you meant) has been available in Debian's sysvinit for 6-7 years. Not sure about OpenRC's status for it or other distros.

    I actually had to go look this up and it turns out you're right. It's been in unstable for years, but it didn't actually ship until Debian wheezy (2013). Debian reference manual []. I didn't switch back to Debian as my primary OS until jessie so I completely missed this.

    Unless the rest of the boot process depends on it, this should not occur with the dependency-based bootup. Which means this, also, is an outdated claim.

    See previous point on "dependency" based startup on sysvinit. The fact is sysvinit has no mechanism to detach a process on boot beyond standard shell control. This generally isn't a problem for servers which can wait for a slow process to get going (and generally want to), but it is problematic on desktops for getting short startup times. Basically, in a sysvinit world, you can either wait for a init script to finish entirely, or detach and clear that boot item. Once you detach, you have no way of if a service has fully come up, or is still starting in a generic way.

    A big focus is from kernel start to GDM in about 10 seconds. The system can keep booting beyond that, but you need enough up to get X11 fired up and you need a way to signal that its safe to run Xsession (which generally requires other system daemons be loaded).

    Upstart handed this via a socket and dbus. systemd has an entire event hook API (

    You never did answer my question about what advantage there is to deploying Ubuntu servers instead of going with Debian -- which was a serious question, FYI, and I'd hoped for a real answer. Right now the lack of answer + mention of working at Canonical makes it look like a company loyalty thing more than any real decision based on merits. I'd love a real answer though. I'd guess support contracts would be a valid reason, but I doubt SN's paying for that.

    I actually missed this point. I wanted a Debian-based OS because of bad experiences with Red Hat ones. Architecturally, Debian has very strict constrains on library dependencies and preventing skew which Ubuntu inherited. (man dpkg-shlibs) combined with a very well tested in-place upgrade system. I would have been perfectly fine if we had standardized on Debian stable at the time. I was against using CentOS due to the fact that its notorious for shipping dated software combined with a very high chance of breaking everything if you upgrade in place. Ultimately, we split the difference, that the rehash machines would run ubuntu, and we'd run CentOS elsewhere. In practice, after the one person really pushing for CentOS left, everything new became Ubuntu to standardize.

    In contrast, with one minor hiccup, the upgrade from Ubuntu 12.04->14.04 was a non-event. Beryllium (the CentOS box) will get wiped and reinstalled to match everything else. What we may be running on is unknown at this time.

    What ultimately made the decision was that when I setup the site, I installed Ubuntu because I done the core work on my laptop which also ran Ubuntu so I didn't have to worry about library skew or anything.

    Yep, like I said in the other comment, I'd have preferred to see upstart "win" rather than systemd, and that's a big part of why. I never really had any problems with upstart itself, other than Canonical completely removing alternative inits to force use of upstart. I'm kind of sad that Canonical abandoned it, because it's not like they're afraid of being incompatible...what with Mir, upstart, Unity, etc. Oh well.
    Still, my big issue wasn't with upstart, it was a social problem: I didn't like how upstart+systemd proponents both forced the issue in Debian, and argued down anybody suggesting something that wasn't upstart or systemd.

    I'm not sure how much I can actually say on this due to most of the Debian side being discussed on d-private. I can say the primarily reason upstart didn't go anywhere was political vs technical. What I can say, on the Ubuntu front, Mark Shuttleworth himself made the call we were going to systemd because Debian decided to do it and we couldn't justify the engineering resources.

    As for the rest you mentioned, well, I'll just say I left Canonical for many reasons.

    See, unlike Ubuntu, Debian actually kept maintaining packages for those init systems. In fact, Ubuntu should still have them just by virtue of being a Debian-derived distribution, but someone decided to purge non-upstart inits from Ubuntu. Just like Redhat, Canonical preferred to take away the choice and force their own in-house solution on people, which is unfortunate.

    Ubuntu has sysvinit, upstart fired it up at the end of boot processing to run the legacy scripts. You are correct that Ubuntu had a hard dependency on booting with upstart as I believe we had hooked it into our udev. I generally didn't have much issue with this as upstart did one thing, and did it well, vs. systemd's kitchen sink approach. Debian's support for other init systems to my knowledge is at best spotty in the real world though ...

    So, again, I'll suggest taking a look at other options, such as Debian, before making a knee-jerk reaction against systemd and Linux as a whole and running off to FreeBSD. You're going to have an "old" type of script-based init there, too, and there's no requirement to run systemd in Debian, so it'd probably be a better transition. You just seem to be running off old knowledge and bad assumptions about the state of non-Ubuntu Linux.

    sysvinit had valid technical reasons to be replaced or overhauled. I just think the new 'standard' is a pile of crap. I posted my long list of technical reasons why I hate systemd elsewhere in this thread. If systemd was content to stay on desktop Linux, I probably won't care much; sysvinit is fine for servers for the most part and I've never had a serious issue with it in real life in a server-role. If systemd did one thing and did one thing well (aka, if it had limited itself to being an upstart competitor), I'd also probably not care enough to work up a damn. It's the entire thing combined with the tendrils its run through the stack that makes me throw my hands up and say enough is enough and look at getting off this ship.

    I'm well aware that the boot situation on FreeBSD is extremely primitive in comparison. You have rc.conf and rc.local and that's about it. Everything is sanely logged. For a server, that's more than acceptable.

    Still always moving
    Starting Score:    1  point
    Karma-Bonus Modifier   +1  

    Total Score:   2  
  • (Score: 2) by Marand on Wednesday August 10 2016, @05:37AM

    by Marand (1081) on Wednesday August 10 2016, @05:37AM (#386134) Journal

    You do realize I'm a Debian Developer in addition to an Ubuntu Core Dev, right?. Anyway, my point is sysvinit has issues, and was due for either update and replacement. I thought upstart good replacement for technical reasons. The primary reason it didn't get adopted by Debian was political most over Canonical's copyright assignment policy; I remember it came up several times on d-private.

    Had no idea about you being a DD, since I only saw mention of Canonical work in the other comment. You're as anonymous as anybody else here to me. :p

    And yeah, the copyright assignment thing sucked, I recall there being some grumpiness over it in the past, so that's not surprising at all. What I meant though was that in the open discussions about the future of Debian's default init, upstart+systemd people (including some devs on both sides) seemed keen on shutting down any attempt to suggest anything else. People were determined to make it a two-party vote, and once that happened Upstart lost to politics like you said.

    As for the sysvinit thing, I'm aware it's a script-based hack and has its issues, but it still does the job mentioned, so I took your statement as a literal "it can't do this thing it does". It's not hard to imagine someone would miss a change like that in a completely different distro, so I took the remark at face value. Just a communication error there: you were being more literal in referring to init itself while I was interpreting it more broadly.

    Also, thanks for the answer about the server selection. I'm with you on the general aversion to Redhat-derived distros, and preferring Debian-based distros for stability and the upgrade process. It's why I've stuck with Debian since around 2000, and why I keep ending up coming back to it after experimenting with other options occasionally. I've mentioned it here before, but my primary OS is a Debian install that started on 2.2 (potato) and has been dist-upgraded through the years, with migrations to new hardware, new disks, and even updated in-place from 32bit to 64bit. I don't know any other OS or distro that I could have done this with. I probably should just reinstall it one day, but it amuses me to keep it going instead of starting fresh at this point. :)

    Vaguely related distro talk: I keep intending to spend time with NixOS. I already get a lot of use out of Nix as a secondary package manager and really like its (admittely weird as hell) filesystem layout and philosophies, so it's on my radar but I haven't gotten around to it. It starts out with systemd though, bleck. Does seem to have other options (sysv, upstart, runit) but I have no idea how much trouble it would be to switch out.

    At this point, my take on systemd is I'm mostly avoiding it for now. sysvinit and the patchwork of hacks on top of it may not be the way to go long-term (I agree about it being due for update or replacement, I just think it all happened a bit prematurely, at least for Debian), but I'd still rather use it than systemd right now. Like you said, if it were just a desktop thing, it wouldn't be such a big deal. I haven't had any problems with the "higher" parts of systemd; it's not really any better or worse than dealing with, say, HAL was in that regard. (I remember HAL being a nightmare at times early on...)

    It's just that I don't want systemd as my init, so I stick to systemd-shim and running anything else underneath. The init just happens to be sysv because that's what Debian used before and it just worked, but it's not like I'm not open to alternatives. Just preferably alternatives that aren't systemd's init. Hopefully by the time I don't have a choice any more, Poettering and Sievers will have gotten bored and moved on so others can clean house. It mostly worked for Pulseaudio like that, at least...

    On a related note, have you ever used runit for anything? It sounds interesting but I've never gotten around to spending time with it. I've only heard of one distro (Void Linux) using it by default [], but their page on how it works is intriguing. It also appears you can even use it alongside another init in case you only want to manage certain services with it, which sounds like it could be appealing.