In the continuing saga of website tinkering and people's love of update posts, I'm back with some backend configuration changes. Right now, things have been relatively quiet on the backend side of things. We've got some good news, and some bad news in this update. That being said, we've made a few small updates over the weekend. Rapid fire style, let's go through them:
CAA records define which certificate authorities (CAs) are allowed to sign your domains. They essentially act as a CA whitelist, and the most recent revisions of the Certificate Authority/Browser Baseline Requirements mandates that CAs check for CAA records and respect them. In line with this policy, we've white-listed Let's Encrypt and Gandi's CAs to issue certificates for SoylentNews for the time being as these are the two CAs currently in use here.
In a fun bit of fail, this is the second time I've tried to deploy CAA, and fortunately managed to succeed this go around. The problem stems from the fact that many versions of BIND except the very latest don't recognize the "CAA" record type, and cause the zone file to not process correctly if it's present. As we're still using an older version of BIND as our master server, I had to manually create TYPE257 records as seen below:
soylentnews.org. 3586 IN TYPE257 \# 16 0005697373756567616E64692E6E6574
soylentnews.org. 3586 IN TYPE257 \# 22 000569737375656C657473656E63727970742E6F7267
soylentnews.org. 3586 IN TYPE257 \# 35 0005696F6465666D61696C746F3A61646D696E40736F796C656E746E 6577732E6F7267
soylentnews.org. 3586 IN TYPE257 \# 12 0009697373756577696C643B
Both htbridge.com and ssllabs.com show that the CAA records are properly encoded, and show an additional green bar that they're in place.
Almost two years ago, the Logjam attack on the DH key exchange was discovered and publicized. As part of our general hardening of SoylentNews, we regenerated all the DH parameters to prevent logjam from being a viable attack vector. Unfortunately, we overlooked the mail STARTTLS services on mail.soylentnews.org, and only caught it when I was checking various security things. The DH parameter files have been regenerated. Under normal circumstances, Logjam can't be exploited unless the underlying SSL cipher is relatively weak. As part of previous hardening, we kicked SSLv3 and many insecure ciphers to the curb, but unfortunately RSA_CBC_IDEA was accidentally left in place as a valid protocol for STARTTLS transport. Based on my understanding of the logjam attack, 1024-bit ciphers like RSA_CBC_IDEA are still difficult to exploit, and its likely only a nation state could successfully have breached it.
Given only SN staff have mail accounts, and that users are encouraged to change their passwords after creating an account, I think its safe to assume that we're relatively OK as far as data security and integrity go since email in general at best is opportunistically encrypted, and should always be assumed to be monitor-able (via a STRIPTLS attack). That being said, if you haven't changed your password from account creation though, it's likely a good idea to do so now.
We discovered our IMAP server has been serving a self-signed certificate during this check as well. We'll be replacing this with a properly signed certificate within the near future. I have other things on this topic that will be noted in a future post, so keep a look out for that.
Disabling HTTP Methods
A routine check of the site's security headers showed that we were accepting HTTP TRACE and other methods we don't need on production. The configuration for nginx has been modified to put a bullet in this behavior. We're still checking to make sure we got this everywhere, but we should be good on at least the production servers for now. This has bumped the site security rating up to an A on the HTBridge; we're still missing the referral security header, but we need to check to make sure there's no user impact before deploying it.
3DES Put Out To Pasture
As always in the world of encryption, various algorithms eventually become insecure and weakened as cryptanalysis gets more and more advanced. A few months ago, the SWEET32 attack against 3DES was discovered which drastically weakens the security of 3DES via the birthday paradox problem. In practice, SWEET32 requires a second exploit to even be usable as SoylentNews only allowed 3DES connections as a last resort if AES wasn't supported. As every major browser has supported AES for years, we decided to put 3DES out to pasture and have removed it from the allowed list of ciphers for SN.
Not too much to note in this round of administration games, but we're working to make overhaul changes to the stack to allow the potential for HPKP key pinning in the near future, as well as deploying TLSA/DANE support for both HTTPS and SMTP on SN. As part of this process, we'll also be enabling HSTS across subdomains, and reissuing our SSL certificates to enable OCSP Must-Staple. We'll keep you guys updated as we move towards that goal!
The "Home" link at the bottom of this article redirects to the Meta section page, and not the Home page as would be expected.
A similar problem exists on the Politics pages, in that the Home link redirects to the Politics section page.
I don't know how else to report this error.
Here you go : ) https://github.com/SoylentNews/rehash/issues/new [github.com]
Thanks, but I don't have an account on GitHub.
Here's fine. Us dev types tend to keep up with the Meta story comments. Also, there's always email@example.com [mailto].
It's nice to see that A+ score on SSL Labs.
I'm going to guess that the CAA records are what gets you from an A to an A+ ?
HSTS + TLS_FAILBACK_SCSV, and inline OCSP stapling are the baseline requirements as of right now, plus strong ciphers and no known security weaknesses. SSLLabs calls special attention for exceptional configuration if you have CAA, HSTS and/or HPKP enabled. As of right now, we have two green bars due to HSTS and CAA.
OCSP stapling requires Apache 2.4 (or a patched 2.2) or nginx, your webserver must be able to reach the CA's OCSP server and get a response (not aproblem now due to CAB BR mandating this), and it must be enabled. TLS_FALLBACK_SCSV requires a recent version of OpenSSL, and HSTS requires the header.
I recently got a commercial Apache Tomcat server application from an A- to an A by getting the right cipher suite for perfect forward secrecy. I was thinking about how to get to an A+. I would have to go up the corporate chain to get CAA to happen. But it might happen if I get the word in the right ears.
HSTS would be easy enough, but a small stand alone server gets the http requests and redirects to the https url. The application itself is a separate server and only answers https. So I hadn't given much thought to HSTS, and didn't intend to until I got past the perfect forward secrecy hurdle. Maybe time to think about HSTS now. Staging server at first with short time to live and do testing.
HPKP like CAA is an organizational matter. But a much bigger one. I would want to have high confidence that we are organizationally mature enough to implement HPKP. It's not just a technical thing. Lots of other servers, products and groups to think about.
I remember DROWN. My nice B (at the time) turned to an F. I had an SSL Labs report that showed which other servers that I shared a wildcard cert with that were the ones actually vulnerable to DROWN. That got some changes to happen fast. Amusingly, all on IIS servers.
I do security consulting and IT work on the side if you want me to assist you with getting A+ scores on your organization. See my email.
I haven't played with Tomcat in several years, but if you check the SSLLabs report, as long as you have "TLS_FALLBACK_SCSV: True" enabled, and OCSP stapling as True in the "other" section below, HSTS will get you A+. CAA isn't very risky to deploy. At worst when you screw it up, you have to change DNS recorders before replacing your SSL certificates; it has no effect on path building for trust in X509, only on issuance on the CA side of things.
Thanks for the tip to get to the A+. Something I will work towards.
I recently got my score up to an A. April 2016, a year ago, I got my score from a B up to an A-. But I had a B prior to that without ever having checked SSL Labs or trying to get that B score.
The sad story is that when I pointed out DROWN and several F scores just over a year ago, the result was that DROWN was fixed on those servers and everything I can identify was upgraded to the point that it now gets a C. Those products aren't my responsibility, and so pointing it out was all I could do. I have no doubt that IIS servers could get an A or A+ if the people who run those had a goal to do so.
I simply saw the SSL Labs score as a challenge and made it a side project or back burner project to improve from a B.
The max limit for IIS is A due to lack of support for TLS_FALLBACK_SCSV (I ran into this with a customer recently). You could get rid of the DROWN problems by rekeying your servers you contract.
Thank goodness that I never have and hopefully never will run IIS. But it is amusing to know that.
The DROWN issue was that several products, including the one I work on, share a wildcard cert. Thus share the same private key. Some servers that I don't control were vulnerable to DROWN. I discovered it immediately when DROWN became news, and sent it up the chain of command. Things happened quickly.
I had proposed that one possible way to make my product secure would be to spring for another cert with a different key.
Generally if possible, I recommend against using wildcards and just issuing SSL certificates with multiple SANs if possible (this is easy with Lets Encrypt. YMMV with other things). Not always viable depending on your setup but wildcards can be a security risk in and of themselves since even if you have multiple wildcart certs protecting your tree. Once Lets Encrypt became a thing, we lost any need to have wildcards for SoylentNews and just have a hilariously long SAN list on Beryllium services.
Without a solid argument why to buy gobs of individual certs instead of one wildcard cert I would get nowhere. I have no doubt I could set up my server for Let's Encrypt. I don't know if that is the case for all other groups that operate servers. The environment I operate in is no doubt different than what you are doing at SN. If there is a solid security argument against wildcard certs, I would be very interested in that.
The problem with wildcards is that they represent a considerably high risk factor. For example, imagine that you have www.example.com, mail.example.com, and payment.example.com, and you're in a PCI compliance setup so you've got everything that talks to the outside world living in the DMZ. You buy and deploy a wildcard certificate to all three of these domains to simply deployment and management.
Now let's say your mail server gets compromised. They now have a valid wildcard certificate that would show the lock for payments.example.com. Depending on your network situation, if the attacker can MitM or change DNS records, he can now decrypt and decode everything going to your payments server even though it was isolated from the mail server because of the wildcard. If we're in a scenario where that the payment and mail server are on the same network segment, an ARP poison attack can easily allow a MITN if you can successfully pwn that mail server. Even beyond that, because most cases when working with wildcards you have the same private key on each box, if any box gets taken down, the attacker can recover the PK, and use it to decrypt SSL traffic in-flight over the network.
There are technical reasons why SSL wildcards exist, specifically in cases where you can't know all the subdomains in general (i.e., dynamic subdomain generation is a valid case), but those should be few and far between. Wildcards are only really acceptable if they're absolutely necessary.
This is part what EV certificates disallow wildcard issuance.
That's a great argument. For anything processing payments, an individual EV certificate would be best.
I just tested one of my servers and got an A+ on it without CAA records set up. I'm not sure what exactly will bump a score down from an A+ to an A, but it looks like it's not CAA records at the moment.
If it were up to me, however, I'd make that a limiting factor for an A+ some time in the not too distant future.
Basically, you have to have enough "points" to get an "A" with no deduction for blatant insecurities and have HSTS with a six-month term.
I'm glad SN is not part of the web insecurity but rather part of the solution! Big fat kudos!
Most of these network security details are going over my head, but I have to thanks the team for re-introducing the +++ button to expand all the comments in a tree.This was the feature that made SN more friendly than the other site :)Cheers!-dbe
Ditto, I don't think the +++ button has been discussed in a Meta article since being implemented; it was my #1 issue with site layout. Working great, and making reading a pleasure. Great job!
I concur...this fixes my biggest gripe with SN.
Bring back plaintext HTTP. Where are your A6 records?
Are you kidding? No way!!!
Don't you understand that if SN used plain HTTP that just anyone, from anywhere would be able to read my comments while they are in transit?
Data just wants to be free.
if SN used plain HTTP that just anyone, from anywhere would be able to read alter my comments while they are in transit
FTFY, even though I know you are just trying to be funny.
Of course, they could read your authentication also, after which they could post additional nonsense in your name, to make you look even more uninformed.
Of course, that is the real reason for a secure connection, which I am aware of. What I wrote was intended as funny.
If there is something else you think I am uninformed about I would be happy to become informed.
This is an obvious troll, but what the hell, I'll actually answer it.
If nothing else, SoylentNews is representative that sites on the internet should have excellent configuration, and management. How many years did we piss on Slashdot for not support HTTPS, or IPv6, or other things? I'm under no belief that in the grand scheme of things, SN is a tiny fish and our data isn't that valuable to various three-letter organizations. However, we are part of a larger movement to bring encryption to the vast majority of the internet on the basis that we can do better. Furthermore, even if we wanted to, we couldn't revert to HTTP-only since we set the Strict-Transport-Security header to lock the site into HTTPS operation only.
As for A6 records, you really had to dig hard to find that one. We have the current standard AAAA records. A6 was a stupidly complex scheme, and required O(n) lookups to build a change to resolution so rapid renumbering could be done via changing the root chain. DJB has a rather good rant on the subject so feel free to Google it, but basically A6 failed because it was stupidly complex for no real gain. Renumbering in IPv6 is a lot easier than v4 for one, and a search/replace can do the trick. A6 chains also don't help in the slightest for PTR records so it wasn't a magic bullet either.
As for A6 records, you really had to dig hard to find that one.
Sigh. Don't you mean, very funny, here's yer stinkin' A6 record.
soylentnews.org. 300 IN TYPE38 \# 17 0026003C0000000000F03C91FFFE98B8FE
I had to look up the A6 specification to actually figure out if that would be valid. I'm not 100% sure the leading 00 is required. For a /128 A6 chain (which is what you defined), I think its just the pure address. That being said, that's not how you're supposed to use A6. With A6, I'd define the /64 part of the network that defines soylentnews.org like so:
(I'm not 100% sure this is right, but I'm not actually going to set up an old BIND to check my records)
soylentnews.org A6 64 2600:3c00::soylentnews.org A6 64 ::f03c:91ff:fe98:b8fedev.soylentnews.org A6 64 ::f03c:91ff:fe6e:d0a3
As such, clients can build the full A6 chain, and if I need to renumber, I change the top half of the network address to make magic happen. A6 was only supported by BIND in both client and server forms, and never adapted wildly before the RFC was listed as historical.
I use A6 records in real life, and yes, the prefix-length byte is mandatory.
The two simplest use cases are:
Prefix length 0. The A6 contains one IPv6 address. Equivalent to an AAAA record.Prefix length 128. The A6 contains one hostname. Equivalent to a CNAME record.
Anywhere between these two extremes, the A6 must contain both a suffix address and a prefix name.
Your chain has to end at a prefix like so:
prefix.soylentnews.org. A6 0 2600:3c00::soylentnews.org. A6 64 ::f03c:91ff:fe98:b8fe prefix.soylentnews.org.dev.soylentnews.org. A6 64 ::f03c:91ff:fe6e:d0a3 prefix.soylentnews.org.
What actually resolves A6 though and A6 is marked as historic. (Serious question)? Is there any actual advantages of using A6? At best, I can put A6 recorders for the full 128 address but that provides no advantage over AAAA. If I use chains as intended, then actually making a DNS lookup over IPv6 requires two round trips at minimum.
AAAA is the current standard, and as far as I can tell, Linux doesn't even check the A6 record type anymore.
There isn't any advantage. A6 just looks cool.
... for that reason alone I'm sorta tempted to add them :)
There is one loophole to identify comment posters etc. and that is to monitor the connections to your server, their IP and the time of posting. You could thwart this by letting the server delay comment posting for some random number of minutes.
Of course let users select if they want this and for how long the post is to be delayed as a maximum.
Well we don't show second-level resolution on posts. The only way you could actually pull of that attack would be to constant scrape SN as a logged in user (to bypass parts of the static cache) second after second and hope that multiple users don't post at the same time. HTTPS prevents a third-party from seeing what you're doing in-line (you can see a connection to SN on 443, and a DNS lookup, but not the URLs posted to).
You'd also have to pwn our loadbalancer in which case you've already won since you'd have to break our Kerberos tree to get in there and that means you have superuser privileges on our entire domain. If the site ever got more traffic, this type of attack would be entirely impractical.
One can also measure the amount of encrypted data sent within a certain window of time to figure out which page was loaded or comment written. A little blunt but combined with other parameters it might make a difference.
There are email servers that let you send email to them which they will resend at a random point in time. Exactly to thwart these kind of actions. I think even Bitcoin has similar facilities.