SoylentNews is people
SoylentNews
from the what-browser-will-you-use-to-read-the-report? dept.
Eric Rescorla has a blog post over at Mozilla about the technical details on the recent Firefox add-on outage. He covers the background of how they use certificates, how they tried to mitigate the damage from the outage, how they worked to solve the problem without breaking more things, deployment of the replacement certificate, and why it took so long to fix.
Recently, Firefox had an incident in which most add-ons stopped working. This was due to an error on our end: we let one of the certificates used to sign add-ons expire which had the effect of disabling the vast majority of add-ons. Now that we've fixed the problem for most users and most people's add-ons are restored, I wanted to walk through the details of what happened, why, and how we repaired it.
There were a lot of work arounds discussed here and elsewhere, some of them quite stupid so, lastly, remember to undo any temporary work-arounds that might have been deployed last weekend.
Earlier on SN: In Firefox All Extensions Disabled Due to Expiration of Intermediate Signing Cert
(Score: -1, Troll) by Anonymous Coward on Friday May 10 2019, @09:51PM (4 children)
This is just feature bloat with all the well known consequences. What more do you need to know?
(Score: 2, Redundant) by Snow on Friday May 10 2019, @10:05PM (1 child)
You don't want the best feature of Firefox?
(Score: -1, Redundant) by Anonymous Coward on Friday May 10 2019, @10:57PM
I've got IBS you insensitive clod!
(Score: 0) by Anonymous Coward on Saturday May 11 2019, @04:49PM
All my addons disabled but at least fucking Pocket works! Fucking Pocket. And fucking Reader. And the popup bullshit that happens whenever I type anywhere.
Thank fuck for all that.
(Score: 1, Informative) by Anonymous Coward on Saturday May 11 2019, @06:25PM
Who mods that troll? I prefer FF to most others but it is perfectly valid criticism. They have ignored user input and added "features" nobone cares about and now we have this failure that has inconvenienced millions of people.
Ah well, shitposting and shitmodding, a long standing SN tradition.
(Score: 3, Funny) by Anonymous Coward on Friday May 10 2019, @09:58PM (3 children)
All the security engineers were away for two weeks of mandatory inclusiveness training, since one had violated the CoC by using the pronoun "he" instead of "ze" in a comment, and no one was left to renew the certs.
(Score: 4, Funny) by bob_super on Friday May 10 2019, @11:03PM
I'm glad my colleagues are avoiding that problem altogether by only handing down code which has been properly stripped of any meaningful function or variable name, and all comments.
They're really saving us a lot of trouble !
(Score: 3, Flamebait) by Ethanol-fueled on Saturday May 11 2019, @12:40AM (1 child)
Isn't it a little weird how all these bad internet-related things all happen at once? Oh shit, you have to upgrade your Firefox only to undo new privacy-raping settings hidden in weird places, then you have to upgrade your Adblock only to find out that you have to do the same and it still doesn't block some obvious ads. Oh, and in the new Firefox you had to upgrade to, now you find some choices you had before taken away outright or hidden in about:config.
It's obvious as fuck that there was a motive behind this "bug." It smells as shitty as a gay man's mouth after rimjobbing an Indian.
(Score: 3, Informative) by linkdude64 on Saturday May 11 2019, @07:05AM
I refused to downgrade to Quantum, so to get (most) of my addons back I just moved laterally to FF55 Developer edition. Ported over my profile, meaning history and bookmarks, logins, etc. Was a bit of a pain, but worth it. Fuck them. I agree that the "Whoops, now you have to update!" angle is shady as fuck. If my addons were disabled without an update, then I'm assuming there's no e-fuse that's preventing them from re-enabling. There must be some technical reason, so I'll have to read TFA and see if they address it.
(Score: 2, Interesting) by RandomFactor on Friday May 10 2019, @10:01PM (2 children)
Was Waterfox.
I wasn't particularly planning to undo it.
В «Правде» нет известий, в «Известиях» нет правды
(Score: 2, Informative) by bolek_b on Saturday May 11 2019, @05:44AM
This week I had to migrate 80% of my FF installations (versions ranging between 53 and 56) to PaleMoon, where addons work as they should. So next time folks at Mozilla ponder why their market share dropped again, this is why - arrogant ignorance of legacy version users doesn't pay off.
(Score: 0) by Anonymous Coward on Saturday May 11 2019, @11:31AM
IceCat Mobile is awesome on Android phones. You can load Firefox plugins which work just the same in android as they do on the desktop. It's awesome.
F-Droid has it - https://f-droid.org/en/packages/org.gnu.icecat/ [f-droid.org]
I had to hunt down an older version as an APK to install it. Well worthwhile hunting through the older versions in the repositories.
(Score: 3, Insightful) by Anonymous Coward on Friday May 10 2019, @10:11PM (7 children)
To me the most important question is: Did Mozilla fix this issue for users who were using older versions of Firefox?
The answer, of course, is: No. The fix is only in version 66 (and later).
There are users of Firefox who have not updated for various reasons, including they are on an ESR version, a particular add-on stopped being updated after their version, or because they're using older operating systems. All of these users have been abandoned by Mozilla.
(Score: 0, Flamebait) by Anonymous Coward on Friday May 10 2019, @10:19PM (3 children)
I'm not going to donate my time to support people who figuratively use IE6. If you think the new version is shit, pick up one of the many forks.
If you think old versions should be supported, pay for the maintenance yourself rather than demanding others do so.
You aren't a customer, you are the recipient of a gift.
(Score: 0, Troll) by Anonymous Coward on Friday May 10 2019, @10:33PM
I'm pretty sure if 90% of firefox developers went away it would be better off.
(Score: 1, Funny) by Anonymous Coward on Friday May 10 2019, @11:05PM
It would be a match made in a cesspit if you really are one of FF devs.
(Score: 5, Insightful) by Bot on Friday May 10 2019, @11:20PM
1. This is not a normal bug. It is a bug that prevents people to restore the old browser from a backup and keep working. People will hate FF for this.
2. The browser is not an application, it's a virtual OS. The sites are the applications. As some applications still require some old OS, so does for example one of my home banking (hello java), and there are horror stories of other sites requiring SPECIFIC versions of java.
So 'just update your browser' is like 'just update your OS', often unfeasible.
Account abandoned.
(Score: 1, Informative) by Anonymous Coward on Friday May 10 2019, @11:17PM
I can confirm that ESR (I'm on v60.6.2esr) did receive an update related to this issue.
(Score: 1, Informative) by Anonymous Coward on Saturday May 11 2019, @05:14AM
Mozilla really doesn't advertise this, but they do maintain Extended Support Releases, the latest being version 60. I forget how long one is maintained for.
Here's the link to the 60's version bump to fix this issue: https://www.mozilla.org/en-US/firefox/60.6.3/releasenotes/ [mozilla.org]
(Score: 0) by Anonymous Coward on Saturday May 11 2019, @11:49AM
Bleh, go into the about:config and change xpinstall.signatures.required to false
(Score: 0) by Anonymous Coward on Friday May 10 2019, @10:44PM (4 children)
Which workarounds were stupid and why?
(Score: 3, Informative) by Anonymous Coward on Friday May 10 2019, @10:59PM
The ones that involved anything other than switching to Waterfox or Pale Moon, because it is stupid to continue using Firefox after this amateur-hour shitfesr.
(Score: 1, Informative) by Anonymous Coward on Friday May 10 2019, @11:18PM (2 children)
I did this one (to an older, ESR version):
In about:config, set this from true to false--
xpinstall.signatures.required;false
It let me run EFF Privacy Badger, which was all I wanted to add to FF anyway.
Does anyone have any reason why I shouldn't leave it this way, assuming I don't use any further extensions or add-ons?
(Score: 0) by Anonymous Coward on Friday May 10 2019, @11:34PM
Yes, that is also what I did. If I knew what that option did I would have switched it long ago.
(Score: 3, Interesting) by Anonymous Coward on Saturday May 11 2019, @12:02AM
This won't work after Mozilla uses Normandy to change your settings back to something they view as more secure.
(Score: 1, Interesting) by Anonymous Coward on Friday May 10 2019, @10:48PM (9 children)
Any suggestion as to how to create an independent web browser? Like SN spouted out when SD went evil?
Setting up SN was pretty damn good accompishment, but maintaining a modern browser is a gargantuan task, but there are many parties with weight we can recruit: EFF, FSF, perhaps others.
(Score: 1, Interesting) by Anonymous Coward on Friday May 10 2019, @11:04PM
The GNU folks do have a firefox fork: https://www.gnu.org/software/gnuzilla/ [gnu.org] -- I'd expect that addon signature verification is off in the fork or at least disable-able, because the modern Firefox way of requiring users to get extensions "approved" by an external third party is completely at odds with free software values.
(Score: 0) by Anonymous Coward on Friday May 10 2019, @11:38PM (4 children)
Get a very large chunk of money. That will allow us to start the project. Then get a very large income. That will allow us to continue...
(Score: -1, Troll) by Ethanol-fueled on Saturday May 11 2019, @12:44AM (3 children)
And if you want to avoid the problems that befell other mighty browsers, don't let women or minorities join your team. If anybody asks about your diversity statistics just buy some wigs (some colored pink and purple) and dresses for a photoshoot and say you're all trans.
(Score: 0) by Anonymous Coward on Saturday May 11 2019, @05:42AM
Hello
sirma'am, I find your ideas intriguing wish to subscribe to your newsletter.(Score: 0) by Anonymous Coward on Saturday May 11 2019, @05:58AM (1 child)
Or you could simply hire based on individual's ability without falling into a trap of only 2 choices.
(Score: 2) by Acabatag on Saturday May 11 2019, @08:22PM
Sadly, there are people actively working to make this a difficult choice.
(Score: 2) by Azuma Hazuki on Friday May 10 2019, @11:53PM (2 children)
I don't know off the top of my head if there's a Windows port, but Falkon works really well on Linux. I find myself using a lot more Qt than GTK apps over the last few years actually...
I am "that girl" your mother warned you about...
(Score: 0) by Anonymous Coward on Saturday May 11 2019, @02:02AM (1 child)
Just looked and there is a Windows port, but, for my Win7 SP1 (not updated), it requires this https://support.microsoft.com/en-us/help/2999226/update-for-universal-c-runtime-in-windows [microsoft.com] patch file from Microsoft.
I've got an old stable system on this ThinkPad which I'm attached to, don't think I'm going to be installing any patches at this late date.
Thanks for the recommendation, one of these days I'll be moving to Linux or something similar because I don't want to have anything to do with Windows after 7.
(Score: 2) by Azuma Hazuki on Saturday May 11 2019, @02:14AM
I actually keep Win7 around in a VM for my MIDI sequencer, Anvil Studio. Linux does everything else I want, and I can make it look like pretty much anything with some work. Maybe that setup would work for you?
I am "that girl" your mother warned you about...
(Score: 5, Interesting) by bzipitidoo on Saturday May 11 2019, @01:51AM (9 children)
I say the real problem is the practice of making certificates expire, how that's implemented, and the consequences that are imposed. The expiration is not graceful, it abruptly switches from working fine to broken. No warnings, no gradual degradation of functionality, not much of a failsafe. The users didn't do anything wrong, but they sure get punished and frightened.
The scary warning messages are way over the top, very much like the typical spam phishing email that warns your account and all your saved emails will be deleted unless you verify your password. The world is not going to end and you are not going to lose all your data just for visiting a web site with an expired cert, or because your computer's clock is off by several years. Very much like the Boy Who Cried Wolf, the false alarms undermine the credibility of the whole system. Why shouldn't the users just ignore the warnings?
Every time a mistake of this sort is made, it results in angry and frustrated users, and a very embarrassed group of engineers. And certificate mistakes happen shockingly often. This time it was Firefox's turn. Members of the group of large, tech savvy organizations that suffered an embarrassing cert expiration include IBM, Microsoft, and I think even Google. Wouldn't be at all surprising if most of the rest slipped at least once. That's not cause for laughing at them, that's evidence that the security system is messed up and ought to be changed.
(Score: 1, Insightful) by Anonymous Coward on Saturday May 11 2019, @02:15AM (5 children)
Just modded you up, interesting.
Three of us volunteer to run a website for mechanical engineering students, where the students can download some proprietary data that is useful for a specific engineering project. So far, we've stuck to http: for our small website. To get a login, the students have to prove they are from a member university that has joined our informal consortium--among other things there are arcane questions relating to this project. After manual vetting, we issue logins (a few a week, not a big deal).
We've resisted going to https: for reasons like you mention -- none of us are expert admins and the chance of us screwing up seems higher than any additional security from https:
Or maybe we are so amateur that we have missed the point altogether(grin)?
(Score: 3, Informative) by Arik on Saturday May 11 2019, @02:49AM (4 children)
If laughter is the best medicine, who are the best doctors?
(Score: 0) by Anonymous Coward on Saturday May 11 2019, @06:03AM
Even if it is they are still open to 0-day browser exploit insertions from their providers (which basically means government as well, one way or the other).
(Score: 1, Informative) by Anonymous Coward on Saturday May 11 2019, @06:24AM
if you dont care about verifying the identity of your website to the user and only want
to secure / encrypt the data exchange bit-stream then just use simple easy self-signed certs.
ofc the global players in the identity managment businesssss continue to lump both aspects together.
one could verify that one has indeed reached the genuine website both all data exchange is in the clear (so wat for some data, right?)
-or-
one is not sure one has connected to the genuine site but all data exchange to it is encrypted (self signed certs).
browser makers have to stop beating on the second because verifying a sites identity (genuine) COSTS MONEY!
a self signed cert (for encrypted data exchange) is FREE!
(Score: 0) by Anonymous Coward on Saturday May 11 2019, @04:42PM (1 child)
Seriously, what is the worst that could happen? If they know to use unique credentials (always a good idea regardless), it won't give an attacker access to anything else. "download some proprietary data", ok if this is multi-million dollar data that would be bad if it wound up in the wrong hands, then there might be an issue. But if it is run-of-the-mill crap from some manager's ass that they want to keep private just to make them feel good, then who cares?
(Score: 0) by Anonymous Coward on Saturday May 11 2019, @05:42PM
Yes, we use unique credentials -- one user (normally a university student) per login. We don't have any great way of keeping a student from sharing their login, except that we make it easy to have your own. For example, we can tell when a class has a project that requires our data--all of a sudden we get a small batch of requests for logins from one university.
The other thing we do is manually remove logins after a period of disuse, or after the student graduates. Over the last dozen years we've probably approved 4000 user names, but only 800 are active now. If anything, we err on being too aggressive when clearing out unused logins--if we delete a legit user by mistake, we apologize and tell them to apply again.
The data we distribute is measured by a specialized test lab that gives our informal group a special "student price" and, in exchange, asks that we try to restrict the data to academic use, no commercial use. After all, this test lab is in business and they expect commercial customers to pay regular rates. This is a handshake agreement, there are no legal repercussions if the data does "escape" to a company.
(Score: 0) by Anonymous Coward on Saturday May 11 2019, @11:51AM (2 children)
You can bet the next time this is about to happen the hackers will be ready and waiting to take advantage of the 5% of users using Firefox who won't be protected by plugins
(Score: 0) by Anonymous Coward on Saturday May 11 2019, @05:49PM (1 child)
> ...the hackers will be ready
What, you mean we FF users will start seeing ads again? Oh noes...
Or did you think of something more sinister that would be possible??
(Score: 0) by Anonymous Coward on Sunday May 12 2019, @08:03AM
Given this also affected torbrowser users the stakes are much higher.
Very sad this happened, again.