Since people seem to rather enjoy when I run articles on backend upgrades, here's another set of changes I made over the last week as I get back into the full swing working on the site.
The short list:
Read past the fold for more information.
Beryllium is our "misc" services box. It basically hosts everything that isn't related to site infrastructure such as the wiki, our IRC server, and mail. Last week, I went through and fixed our SSL configuration on this machine to make sure that we were serving properly validated certificates, and that we had strong encryption on this box. While I succeeded on that front, for performance reasons, Apache 2.4 needed to be upgraded to support a somewhat obscure feature of TLS known as OCSP stapling.
What is OCSP stapling you ask? Well, to answer that, I need to take a moment to go into how SSL certificates work. Whenever a CA generates a certificate, they're essentially saying "this site is who it is and we're attesting to it". In a perfect world, a CA would never make a mistake, private keys would never leak, and we could always assume that a certificate is good. We don't live in that world, as such certificate authorities sometimes need to void a certificate. OCSP (which stands for the Online Certificate Status Protocol) is one of two ways to do this, and is the only method Let's Encrypt supports for certificate revocation.
OCSP is a replacement for older certificate revocation lists (CRLs) which in real-life rarely if ever worked as advertised. It's meant to allow the browser to update in real-time knowledge if a certificate is good or bad and react accordingly. OCSP however requires that the browser checks with a certificate authority's OCSP server, leaking the fact that user X is connecting to site Y. It also means that if access to the OCSP server is blocked, a user might not be aware that a certificate has been revoked. OCSP stapling solves both problems by having our servers grab the OCSP reply (which is timestamped), and sending it as part of the initial connection to our site, both increasing performance, and preventing a privacy leak.
Unfortunately, OCSP stapling requiring Apache 2.4 which required me to build it from source, and then migrate sites over from the older Apache 2.2 install. At the same time, I went through and upgraded PHP 7, and updated the other web applications we were using. For the most part, this was rather painless though I'm still tinkering with MediaWiki to make it happy on the new setup.
Beside the usual Apache pain, I went through and scanned our other major services and disabled SSLv3 support on postfix (SMTP) and dovecot. I need to go through and replace our self-signed certs with real ones here but that's a 'one step at a time thing'
During the last site status article, an AC pointed us at this handy site showing security headers. As such, TheMightyBuzzard and I will be going through and enabling these (with the exception of public key pinning) on production sometime this week. HPKP requires quite a bit of planning to deploy and we're not ready to take that step just yet.
I've talked about wanted to deploy DNSSEC before, but various other things kept cropping up. That, and combined with outdated and misleading documentation kept me from actually getting around to doing this for ages. Over the weekend, I finally dug down and figured out the current best practices for DNSSEC, and with the help of audioguy, configured BIND to do automatic signing of the domain and uploaded our keys to our register.
As such, sylnt.us now has a fully validated signature chain, and a green key when checked with the DNSSEC validator. We will be signing soylentnews.org sometime in the near future, however, we ran into some DNS zone transfer issues between our nameservers and Linode which caused the RRSIG records to not properly upload. While this has been resolved for now, we're currently talking with Linode to understand why the transfer went pear-shapped and to prevent a second occurrence.
That's it for now. As always, post questions, comments below. I'll be reading!
What is your time worth? Did you get appropriately compensated for the time you spent? If not, why did you spend the time and why should we expect you to spend more time?
Perhaps more importantly, since you've already done these things, why should they matter to us?
Perhaps most importantly, how do you know that any of these things (or future improvements) actually do matter to us?
My reaction to looking over the list of improvements is that I would not chip in to support any of them, but perhaps that's just because you were not trying to sell them. I don't see anything in there about why these improvements are such good things that they will make Soylent News better from my perspective--but remember that "my perspective" does NOT matter. It's "our perspective", referring to the readers in a collective sense, that should matter.
I continue to regard Soylent News as a nice idea that is going nowhere. Not even a brisk pace in the non-movement, but I can't blame you. Too bad there is no mechanism to do better, eh? (I think there is, but it seems I'm always in the wrong time zone. My clock is several years fast again.)
Somebody doesn't keep up2date with security.
Sometimes you do things for the love of it. Oh, and it improves your general skills and knowledge that you then apply to your main profession. I guess it is professional development.
What is that worth?
You are missing the point. I think he deserves to get paid for his hard work.
Now if you have donated to support him and his work, then I do congratulate you. If you didn't then you are only proving my point. Or do you want to argue that he should take a vow of poverty?
Funny story about security. Based on a few minor pieces I wrote for IEEE Computer before I joined the Big Blue food chain, some of my coworkers (especially in the research lab) thought I was a computer security expert. I am NOT and never was, though I still enjoy reading such relevant books as the recent Data and Goliath by Bruce Schneier.
From his lingo-filled description, I didn't even know that any of the improvements were related to security, but in that case they should NOT be waiting for his convenience. If there had been a support project listing urgent security upgrades, then I certainly hope enough members would have chipped in to get them implemented ASAP. Security updates should actually be prefunded, and I might chip in on that basis because I know that remedial steps after you've been breached are much more expensive. Or aren't you keeping up with security?
Here you go. [soylentnews.org] It might not go directly in his pockets*, but it's clearly where those contributing time (I'm not one of them) would like you to throw your money.
From his lingo-filled description, I didn't even know that any of the improvements were related to security,
Really? This lingo was too dense for you (emphasis original): "XSS Mitigation During the last site status article, an AC pointed us at this handy site showing security headers."
Methinks somebody didn't read the post.
but in that case they should NOT be waiting for his convenience. If there had been a support project listing urgent security upgrades, then I certainly hope enough members would have chipped in to get them implemented ASAP. Security updates should actually be prefunded, and I might chip in on that basis because I know that remedial steps after you've been breached are much more expensive.
Here you go, again. [soylentnews.org] The more funds there are, the more on top of issues like this you can logically expect them to be.
*Or it might go to his pockets, I don't know if he's currently taking a loss or not. I think he was originally, but he might have been paid back for it.
For someone writing on a journalism-related website, my main conclusion is that you [JNCF] don't write very clearly. I'll try to deal with your "objections" as well as I can understand them.
I am not saying that essentially blind subscriptions are a terrible business model. Mostly I am saying that I want more transparency. Among other reasons, I am not so rich that I can throw money at every good cause that wants more money.
My primary suggestion is for an alternative funding mechanism that would allow small donors to see more about how their money is actually used. Perhaps the larger value of the approach is that the same mechanism would drive documentation of the features and services that the website is providing. The project proposals that get funded should become accurate descriptions of what exists (as long as the success criteria are satisfied).
If I were rich, then the shoe would be on the other foot. There are plenty of problems with big-donor financial models, but one thing you can be sure of is that big donors get to see how their money is being spent. I'm suggesting an approach to let small donors have a bit of that.
Sometimes I do donate to support journalism. For example, I am quite sure that I donated some money to the website that broke Romney's 47% comment. If I were going to subscribe, then that is probably one of the top candidates. SN in its current form would not be in the list of my top ten candidates.
By the way, your example from the original article was an example of poor writing. How do you interpret your selection to indicate a security problem on SN? Of course XSS is a hint, but even the replacement with "cross-site scripting mitigation" would be ambiguous. (And no, I did not have to look that up. I even edited some technical papers for security experts who were working on related problems.)
For someone writing on a journalism-related website, my main conclusion is that you [JNCF] don't write very clearly.
I have never been fond of most journalistic writing, but I see your point about subjects. To clarify, "he" meant NCommander. There are more reasons to choose words and phrasings than clarity alone. When my intention is strictly clarity, I am very clear. This is rarely the case on SoylentNews.
I am not so rich that I can throw money at every good cause that wants more money.
Me neither. I really like SoylentNews, even with all its nasty warts. It's certainly not for everyone, and donation is totally optional. You seem very concerned about funding, though.
Maybe this is a good suggestion. Maybe not. It doesn't seem very fleshed out. Rereading your original posts in this discussion, the idea doesn't even seem to be present. It seems like a much more general complaint that touches on a number of other issues. For what it's worth, I modded you "underrated" -- not because I agreed with anything you said, but because I didn't think you were trolling.
The emphasis was original. I felt that the heading added context to the sentence that came after it, and I wanted to clearly identify it as a heading without breaking it into a multi-line block quote as I was using for your text. If you go back and read the rest of the quote, it uses the word "security." That was the lingo you either couldn't interpret as being related to security, or didn't read.
I have to leave soon so I'm just going to focus on a few main points. The funding suggestion was much more fleshed out in some of my earlier comments on SN, after I had observed the system for a few weeks. It has also been fleshed out in various other venues, often modified for more specific purposes. The original form was called RACS (Reverse Auction Charity Shares) because of some viral marketing aspects involving discounted shares, but these days I usually start from the more generic perspective of "charity share brokerage", focusing on the key entity that is holding the money. They show up in some search engines, but I don't know of any so-called early adopters. (There's also some stuff I wrote about couch potatoes...)
The reason I focus on the funding is not because I think money is important, but because it influences behavior. People tend to respond to clear incentives, and monetary incentives tend to quite clear. If an organization is getting its funding from certain stakeholders, then it will natural evolve to focus on the needs and preferences of those stakeholders over the others. The model I am suggesting would make "wannabe problem solvers" into key stakeholders and solving problems would be linked to the organization's success, including its success in funding itself.
Getting diverted again, but I think economics sucks, to put it politely. Currently reading Seven Bad Ideas along those lines. My proposed solution is called ekronomics, perhaps a form of time-based economics.
None of these updates are critical. I'm realistic that much of what I do is relatively low value, and to be more blunt, there's very little of value here for someone to break into. At best someone could supposedly grab an admin's credentials in flight and delete all the stories but that's why we have backups.
Back when Heartbleed happened (and we (and most of the internet)) were affected, I believe we had the certs changed out within 24 hours. Critical security stuff does happen timely, but this is mostly hardening and increasing the armor plating vs. replacing a hole.
I'm glad to hear that, but I still think you deserve to be well paid for good work, and you didn't really address my original question. Putting too much reliance on the Subject: line?
Let me clarify that I am not interested in the real cost of those improvements from a managerial perspective. You don't want to get me started on my low opinion of some managers... However, I think that Soylent News is supposed to be or wants to be a new kind of journalism, and some of that involves project management, including security-related projects.
In this particular case, I think that security should be a high-priority ongoing-cost project. Perhaps this particular work would have justified a special implementation project, too? However, my focus is that someone ought to get paid for the skills.
Regarding the criticality of these updates, I prefer to err on the side of "Better safe than sorry." I actually think the real threat of a breach on Soylent News would be something like a malware installer, and from that perspective SN seems to be a low priority target because it seems to be a low traffic website. If one of the goals is to increase the traffic and influence, I think that would be good, but it would also increase the value of the website as a target for attack--and in that case the greatest threat might be an attacker who is planning ahead and installing backdoors now.
Speaking candidly, personal issues really prevented me from putting the effort in to building out the site as I planned w/ more original journalism, plus a somewhat lukewarm response from the community. We'll run original articles if someone submits one but thats once in a bluemoon sorta thing.
I've thought about wondering ways we can bring in more money for SN; maybe a hosted DNS stuff which is DNSSECed or something but to be honest, I dunno if we could really make much of anything doing that beyond what subscriptions bring in ...
Well, I don't know if you ever saw my much ballyhooed suggestion, but I'll recap it in the context of your specific example for this thread.
You would have begun by preparing a summary of the work as a project proposal. Since it was a security-related project, I think the links to the project proposal should have been featured pretty prominently on SN. Members who saw the proposal would see what you wanted to do, your schedule, your compensation, the testing plans (which I believe to be important in every software proposal but even more so when it comes to security, even if it's just a code walk-through with another programmer), and the success criteria. When enough members have chipped in, then the money would be released and you would do the project, and after it was finished, the results would be evaluated and reported to the donors.
The funding mechanism I suggested could be described as a "charity share brokerage". As a supporter of SN, I could donate some money to my account, and periodically buy shares in projects that I like. If enough people agree with me, then my projects get funded, but if I pick a loser project, then it runs past its funding schedule without getting funded and I can pledge the money to some other project. (Of course the people who submit the unfunded project can try to improve it and submit it again.)
Not sure what part of this idea caused so much umbrage. Perhaps the idea of going beyond internal projects to actually help fix problems in the real world? Hey, sorry, but that's where I live.
Hrm ... I could def. see how it could work for some things. Right now, a lot of this is just basic site maintence; updating old software and such, but this gives an idea I think I need to pitch to the community and such.
You're welcome to any part of it you can use, but I've been thinking about variations of alternative funding models long before I ever heard of crowdfunding. There are a number of variations that might be relevant or suitable for journalistic purposes.
Lord knows the existing models of journalism are totally broken. Primarily disaster porn and fake reality shows for eyeballs to sell to advertisers driving journalism to depths of perfidy never before seen. That's just the stage setting, but the real problems are the bad actors like terrorists and the Donald who exploit the broken systems.
There's an article in the queue set to go live at 8AM EST talking about seeing if we can get a funding model based on posts like this. If it flys with the community, it could effectively kill three birds with one stone: getting interesting original content on the site, getting those high-labor updates out the door, and getting money to the contributors.
I thought I was watching for it, but either I missed it or it didn't go live? How about a link?
>Soylent News as a nice idea that is going nowhere.
can i get your coat for you? call a taxi perhaps?