Stories
Slash Boxes
Comments

SoylentNews is people

SoylentNews is powered by your submissions, so send in your scoop. Only 10 submissions in the queue.
Meta
posted by NCommander on Monday August 15 2016, @07:01PM   Printer-friendly
from the fiddling-for-the-greater-good dept.

Since people seem to rather enjoy when I run articles on backend upgrades, here's another set of changes I made over the last week as I get back into the full swing working on the site.

The short list:

  • Migrated Beryllium (which hosts wiki+IRC+mail) to Apache 2.4
    • Upgraded said machine to PHP7
    • Needed to support OCSP stapling
    • Validating final checks before deploying HSTS to all public domains
    • Upgraded MediaWiki, SquirrelMail, and YOURLS to PHP7 compatible versions
  • Worked with TheMightyBuzzard and user comments to determine additional XSS protection headers we should deploy
  • Found (and removed) SSLv3 support on postfix and dovecot
  • Deployed DNSSEC on sylnt.us in preparation for signing soylentnews.org (here's the test results)

Read past the fold for more information.

Beryllium Upgrades

Beryllium is our "misc" services box. It basically hosts everything that isn't related to site infrastructure such as the wiki, our IRC server, and mail. Last week, I went through and fixed our SSL configuration on this machine to make sure that we were serving properly validated certificates, and that we had strong encryption on this box. While I succeeded on that front, for performance reasons, Apache 2.4 needed to be upgraded to support a somewhat obscure feature of TLS known as OCSP stapling.

What is OCSP stapling you ask? Well, to answer that, I need to take a moment to go into how SSL certificates work. Whenever a CA generates a certificate, they're essentially saying "this site is who it is and we're attesting to it". In a perfect world, a CA would never make a mistake, private keys would never leak, and we could always assume that a certificate is good. We don't live in that world, as such certificate authorities sometimes need to void a certificate. OCSP (which stands for the Online Certificate Status Protocol) is one of two ways to do this, and is the only method Let's Encrypt supports for certificate revocation.

OCSP is a replacement for older certificate revocation lists (CRLs) which in real-life rarely if ever worked as advertised. It's meant to allow the browser to update in real-time knowledge if a certificate is good or bad and react accordingly. OCSP however requires that the browser checks with a certificate authority's OCSP server, leaking the fact that user X is connecting to site Y. It also means that if access to the OCSP server is blocked, a user might not be aware that a certificate has been revoked. OCSP stapling solves both problems by having our servers grab the OCSP reply (which is timestamped), and sending it as part of the initial connection to our site, both increasing performance, and preventing a privacy leak.

Unfortunately, OCSP stapling requiring Apache 2.4 which required me to build it from source, and then migrate sites over from the older Apache 2.2 install. At the same time, I went through and upgraded PHP 7, and updated the other web applications we were using. For the most part, this was rather painless though I'm still tinkering with MediaWiki to make it happy on the new setup.

Beside the usual Apache pain, I went through and scanned our other major services and disabled SSLv3 support on postfix (SMTP) and dovecot. I need to go through and replace our self-signed certs with real ones here but that's a 'one step at a time thing'

XSS Mitigation

During the last site status article, an AC pointed us at this handy site showing security headers. As such, TheMightyBuzzard and I will be going through and enabling these (with the exception of public key pinning) on production sometime this week. HPKP requires quite a bit of planning to deploy and we're not ready to take that step just yet.

DNSSEC + sylnt.us

I've talked about wanted to deploy DNSSEC before, but various other things kept cropping up. That, and combined with outdated and misleading documentation kept me from actually getting around to doing this for ages. Over the weekend, I finally dug down and figured out the current best practices for DNSSEC, and with the help of audioguy, configured BIND to do automatic signing of the domain and uploaded our keys to our register.

As such, sylnt.us now has a fully validated signature chain, and a green key when checked with the DNSSEC validator. We will be signing soylentnews.org sometime in the near future, however, we ran into some DNS zone transfer issues between our nameservers and Linode which caused the RRSIG records to not properly upload. While this has been resolved for now, we're currently talking with Linode to understand why the transfer went pear-shapped and to prevent a second occurrence.

That's it for now. As always, post questions, comments below. I'll be reading!

~ NCommander

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 2) by NCommander on Tuesday August 16 2016, @07:44PM

    by NCommander (2) Subscriber Badge <michael@casadevall.pro> on Tuesday August 16 2016, @07:44PM (#388807) Homepage Journal

    None of these updates are critical. I'm realistic that much of what I do is relatively low value, and to be more blunt, there's very little of value here for someone to break into. At best someone could supposedly grab an admin's credentials in flight and delete all the stories but that's why we have backups.

    Back when Heartbleed happened (and we (and most of the internet)) were affected, I believe we had the certs changed out within 24 hours. Critical security stuff does happen timely, but this is mostly hardening and increasing the armor plating vs. replacing a hole.

    --
    Still always moving
    Starting Score:    1  point
    Karma-Bonus Modifier   +1  

    Total Score:   2  
  • (Score: 1) by shanen on Tuesday August 16 2016, @09:06PM

    by shanen (6084) on Tuesday August 16 2016, @09:06PM (#388833) Journal

    I'm glad to hear that, but I still think you deserve to be well paid for good work, and you didn't really address my original question. Putting too much reliance on the Subject: line?

    Let me clarify that I am not interested in the real cost of those improvements from a managerial perspective. You don't want to get me started on my low opinion of some managers... However, I think that Soylent News is supposed to be or wants to be a new kind of journalism, and some of that involves project management, including security-related projects.

    In this particular case, I think that security should be a high-priority ongoing-cost project. Perhaps this particular work would have justified a special implementation project, too? However, my focus is that someone ought to get paid for the skills.

    Regarding the criticality of these updates, I prefer to err on the side of "Better safe than sorry." I actually think the real threat of a breach on Soylent News would be something like a malware installer, and from that perspective SN seems to be a low priority target because it seems to be a low traffic website. If one of the goals is to increase the traffic and influence, I think that would be good, but it would also increase the value of the website as a target for attack--and in that case the greatest threat might be an attacker who is planning ahead and installing backdoors now.

    --
    #1 Freedom = (Meaningful - Coerced) Choice{5} ≠ (Beer^4 | Speech) and your negative mods prove you are a narrow prick.
    • (Score: 2) by NCommander on Tuesday August 16 2016, @10:08PM

      by NCommander (2) Subscriber Badge <michael@casadevall.pro> on Tuesday August 16 2016, @10:08PM (#388866) Homepage Journal

      Speaking candidly, personal issues really prevented me from putting the effort in to building out the site as I planned w/ more original journalism, plus a somewhat lukewarm response from the community. We'll run original articles if someone submits one but thats once in a bluemoon sorta thing.

      I've thought about wondering ways we can bring in more money for SN; maybe a hosted DNS stuff which is DNSSECed or something but to be honest, I dunno if we could really make much of anything doing that beyond what subscriptions bring in ...

      --
      Still always moving
      • (Score: 1) by shanen on Tuesday August 16 2016, @10:36PM

        by shanen (6084) on Tuesday August 16 2016, @10:36PM (#388875) Journal

        Well, I don't know if you ever saw my much ballyhooed suggestion, but I'll recap it in the context of your specific example for this thread.

        You would have begun by preparing a summary of the work as a project proposal. Since it was a security-related project, I think the links to the project proposal should have been featured pretty prominently on SN. Members who saw the proposal would see what you wanted to do, your schedule, your compensation, the testing plans (which I believe to be important in every software proposal but even more so when it comes to security, even if it's just a code walk-through with another programmer), and the success criteria. When enough members have chipped in, then the money would be released and you would do the project, and after it was finished, the results would be evaluated and reported to the donors.

        The funding mechanism I suggested could be described as a "charity share brokerage". As a supporter of SN, I could donate some money to my account, and periodically buy shares in projects that I like. If enough people agree with me, then my projects get funded, but if I pick a loser project, then it runs past its funding schedule without getting funded and I can pledge the money to some other project. (Of course the people who submit the unfunded project can try to improve it and submit it again.)

        Not sure what part of this idea caused so much umbrage. Perhaps the idea of going beyond internal projects to actually help fix problems in the real world? Hey, sorry, but that's where I live.

        --
        #1 Freedom = (Meaningful - Coerced) Choice{5} ≠ (Beer^4 | Speech) and your negative mods prove you are a narrow prick.
        • (Score: 2) by NCommander on Tuesday August 16 2016, @11:04PM

          by NCommander (2) Subscriber Badge <michael@casadevall.pro> on Tuesday August 16 2016, @11:04PM (#388887) Homepage Journal

          Hrm ... I could def. see how it could work for some things. Right now, a lot of this is just basic site maintence; updating old software and such, but this gives an idea I think I need to pitch to the community and such.

          --
          Still always moving
          • (Score: 1) by shanen on Wednesday August 17 2016, @12:21AM

            by shanen (6084) on Wednesday August 17 2016, @12:21AM (#388909) Journal

            You're welcome to any part of it you can use, but I've been thinking about variations of alternative funding models long before I ever heard of crowdfunding. There are a number of variations that might be relevant or suitable for journalistic purposes.

            Lord knows the existing models of journalism are totally broken. Primarily disaster porn and fake reality shows for eyeballs to sell to advertisers driving journalism to depths of perfidy never before seen. That's just the stage setting, but the real problems are the bad actors like terrorists and the Donald who exploit the broken systems.

            --
            #1 Freedom = (Meaningful - Coerced) Choice{5} ≠ (Beer^4 | Speech) and your negative mods prove you are a narrow prick.
            • (Score: 2) by NCommander on Wednesday August 17 2016, @12:38AM

              by NCommander (2) Subscriber Badge <michael@casadevall.pro> on Wednesday August 17 2016, @12:38AM (#388919) Homepage Journal

              There's an article in the queue set to go live at 8AM EST talking about seeing if we can get a funding model based on posts like this. If it flys with the community, it could effectively kill three birds with one stone: getting interesting original content on the site, getting those high-labor updates out the door, and getting money to the contributors.

              --
              Still always moving
              • (Score: 1) by shanen on Wednesday August 17 2016, @09:25PM

                by shanen (6084) on Wednesday August 17 2016, @09:25PM (#389306) Journal

                I thought I was watching for it, but either I missed it or it didn't go live? How about a link?

                --
                #1 Freedom = (Meaningful - Coerced) Choice{5} ≠ (Beer^4 | Speech) and your negative mods prove you are a narrow prick.