Hackers have breached the severs[sic] of email provider VFEmail.net and wiped the data from all its US servers, destroying all US customers' data in the process.
The attack took place yesterday, February 11, and was detected after the company's site and webmail client went down without notice.
"At this time, the attacker has formatted all the disks on every server," the company said yesterday. "Every VM is lost. Every file server is lost, every backup server is lost."
"This was more than a multi-password via SSH exploit, and there was no ransom. Just attack and destroy," VFEmail said.
[...] Back in November 2015, VFEmail was one of the many online email providers that were targeted by Armada Collective, a group of hackers who demanded ransom payments from victim companies to stop ongoing DDoS attacks.
There were servers in the US and in Europe; I think US users really means all users except the ones in the Europe server.
Hackers wipe US Servers of Email Provider VFEmail
Email Provider VFEMail’s US Servers Wiped by Hackers
VFEmail twitter account
(Score: 5, Insightful) by bob_super on Tuesday February 12 2019, @08:48PM (12 children)
Offline backups are your annoying friend.
(Score: 5, Insightful) by choose another one on Tuesday February 12 2019, @09:44PM (5 children)
It's perhaps more that anything "online" is merely replication, and replication is not backup.
(Score: 1) by warsen on Wednesday February 13 2019, @03:59AM (2 children)
Umm no, well not always. I admit I don't know much about *enterprise* backup, but at the individual level at least there are excellent tools like borgbackup that can be both online (you do need a server on which you can install borg I suppose), and encrypted, and maintains history. And that's pretty recent; there are so many others (rdiff-backup, duplicity, come to mind).
I've also heard of things like Amanda and so on, for backing up several systems. I can't believe they're all just replication.
With all this, I am pretty sure, even if that was a Windows shop (say), that there are several "enterprise class" backup products that do all the right things.
Nope; these guys were just lazy and/or penny-pinching.
(Score: 2) by Mykl on Wednesday February 13 2019, @04:38AM (1 child)
Unless they were doing all of this and the hacker just logged directly into the backup server and wiped it from there...
(Score: 2) by choose another one on Thursday February 14 2019, @09:49AM
This is the problem.
Any backup system connected to the server being backed up, can therefore be accessed from that server. Some work the other way, with an agent running on the backup side contacting the server to get it's data - but those sort of systems are more normally used for backing up lots of client (i.e. desktop, laptop) systems.
Even if the credentials for accessing the remote backup service are _not_ discoverable on the server, the config maybe and that may well be enough. Online backup stores ten previous backups? - wipe server, initiate 10 backup jobs, oops, "backups" gone...
"online" is live, connected, and accessible from the compromised server, and therefore very vulnerable to attack.
"offline" (someone his to physically go get media from the vault) cannot be attacked from the comped server, must be attacked separately, which massively increase the attack complexity (to achieve a simultaneous wipe of everything). Still not impossible (ask Mr Robot...) - but a lot harder.
(Score: 2) by driverless on Wednesday February 13 2019, @07:36AM (1 child)
Mmmm, I dunno. Tried to back myself up once and all I got were three ungrateful brats.
(Score: 2) by bob_super on Wednesday February 13 2019, @05:43PM
Maybe it was just a replication. Ask your mom.
(Score: 1, Insightful) by Anonymous Coward on Tuesday February 12 2019, @10:02PM (2 children)
Offline immutable backups. At least only mutable through human intervention.
(Score: 0) by Anonymous Coward on Wednesday February 13 2019, @12:50AM (1 child)
Well, this attack can certainly check that box.
(Score: 3, Insightful) by driverless on Wednesday February 13 2019, @07:40AM
They're going to fly to the secure storage facility, break into it, and set fire to the contents? I think you're misunderstanding what "offline" means.
(Score: 2) by opinionated_science on Tuesday February 12 2019, @11:47PM (1 child)
why doesn't the mod go to +infinity...
"There are two sorts of people in this world - those that backup and those that *will*". Douglas Adams.
(Score: 5, Insightful) by coolgopher on Wednesday February 13 2019, @04:50AM
"There are those who think they backup, and then there are those who actually test their backups" - unknown
(Score: 2) by DannyB on Wednesday February 13 2019, @03:33PM
Who needs backups? Follow the practice [techcrunch.com] of the industry leader Microsoft.
To transfer files: right-click on file, pick Copy. Unplug mouse, plug mouse into other computer. Right-click, paste.
(Score: 5, Informative) by Apparition on Tuesday February 12 2019, @08:53PM (17 children)
Bleeping Computer has a more recent article on the attack [bleepingcomputer.com] with a statement from the owner. The owner stated that he will most likely shut VFEmail down as a result of this hack.
My thoughts:
(Score: 2) by RandomFactor on Tuesday February 12 2019, @09:12PM (5 children)
Possibly some bad blood was generated in 2015 if they didn't pay the ransoms or took other actions that annoyed the attackers?
Changing the dynamics from "we will DDOS and inconvenience you" to "also if you don't pay we'll wipe your systems" could be a move to add some edge to the routine humdrum of DDOS doings.
В «Правде» нет известий, в «Известиях» нет правды
(Score: 3, Interesting) by c0lo on Wednesday February 13 2019, @01:06AM (4 children)
Insurance fraud?
Based on:
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 2) by nobu_the_bard on Wednesday February 13 2019, @01:56PM (3 children)
I doubt insurance fraud. It doesn't sound like the point of the project was ever to make money.
(Score: 3, Interesting) by Hyperturtle on Wednesday February 13 2019, @04:12PM (2 children)
It sort of looks to me like he'd been paid financially, or with continued freedom, to hand over everything to some authority and paste selected verse from a provided script.
I was thinking that if he accepted the carrot, he knew he'd be labeled a fool or similar from the various social media cloud internet experts, and couldn't ever run a similar service again. Who'd trust him with data they wanted to keep after this? He'd be made into an example of why someone else's products and services should be used. He's marketing gold right now--the example to avoid.
I mean the fact there's "no backups' since forever is pretty ludicrous. Anyone that custom wrote something special usually has the presence of mind to make a few copies and lose them at home; I have floppy disks that are ancient with stuff i did a long time ago that I never had to restore from such disks--but the disks outlasted the hardware I backed up! It's just not easy to accept that his emotional investment into the logic behind how it all worked was not met with similar enthusiasm to make sure his efforts were not destroyed via his own mistakes, let alone someone else's deliberate attempts to sabatoge it. Especially after receiving various threats over the years...
The fact there isn't even an LTO2 tape somewhere with a full copy from 2006 or something seems like disaster recovery was not funded, if that's all true... Even an old hard drive or array from a past upgrade. I usually keep a few copies of *important* stuff off the network -- and a few copies on the network of that and everything else. I expect the backups are in the hands of whoever he handed the infrastructure over to.
This destruction was so complete, for a service up for almost 20 years, that it seems more than an inside job. I'd say he handed all the stuff over to someone(s) and memorized a script he was provided, or just copied and pasted from it to twitter and wherever, perhaps his emails were coached in response to questioning--looking around, a few places indicated he didn't respond to the journalists questions in time for the posts to go live.
That could mean the guy is very busy not restoring his systems, or the questions are filtering through the people telling him what to say. Or he's found new time in life to do something else and spending time with friends and family, pleased he's still free and not financially insolvent. He seems pretty calm about his life's work (so to speak) could mean he's really a great resource under pressure, or that he's come to terms with what has happened. Or that he's still free for having cooperated and that provides some solace. Maybe he's happy to cooperate, maybe he didn't know he was hosting horrible stuff from some criminal element, Maybe he's mortified at what he had to do. But publically, he's got his act together even though everything fell apart.
The place had been up for a very long time. This may have provided some authority a wonderful database of emails to mine, going back years and years and years; I have to wonder if anything was deleted at all aside from his own ability to get into the stuff once walked away from it. Any other reason just doesn't make a lot of sense to me considering what various news articles about this have stated... my thoughts are that one can't be so smart to make a service like this, run it reliably for so long, handle all the everythings, and also forget to make useful backups knowing all of the calamaties that have struck data centers and networks and computers over the years--this service has been up forever, the guy has at least heard of it all if not seen it all. He couldn't be ignorant of the risks of having locally accessible disk to disk replication as the sole means of redundancy.
That public IP he shared as a potential source address probably was just something from a scan that could be a random hacked machine seen scanning a few times, or it's just an IP in a range that now doesnt respond anymore... doesn't matter. Maybe he was even told to use it because the IP has been seen/reported numerous times by other businesses. Never waste a good scapegoat on something that can be disproven easily.
Someone probably should check old posted policies about privacy and such and see if any signaling was done prior to the outage. It could be that nothing was changed because he never let authorities in; it may be that to make it out intact, he had to just hand it all over at once and not let some authority tap in and open him up to legal stuff if his service agreements suddenly were missing items like "and the people you are trying to avoid can read your email now."
I think what happened is bad, and I don't like to be a dramatic conspiracy theorist (except for when it comes to personalized advertisements), but my spidey sense is going off and his story just doesn't sound right. Maybe there is a lot more that we will never know because the data is truly gone--and so all we (and I) have to go from is his incomplete view of the events. I don't want to discredit him, but it still sounds like a well crafted story rather than what happened.
Hopefully what he's said is true, because such attacks can be defended against or at least mitigated via enforcement of industry standard disaster recovery best practices... I hope that his story is factual and really is the worst of it and that he's the victim of a criminal rather than something worse (like stupidity or authority). And that every other similar service providing privacy and security lives happily ever after having learned from this.
(Score: 0) by Anonymous Coward on Wednesday February 13 2019, @04:33PM (1 child)
I am pretty sure my hosting provider keeps backups of my VM in case I should remotely brick it. Has his hoster been asked about their backup measures? They may not want to answer if the government was involved, but it would be good journalism to ask.
(Score: 2) by Hyperturtle on Friday February 15 2019, @03:51PM
It didn't sound like he was quite so commoditized in that he had a number of virtuals somewhere, but even so, off-site backups sometimes contain that surprise of someone having rooted the entire topolology.
I did work for a place that, on a lark, I searched for their website after I heard the new version went live. They did all sorts of testing and said it was great. They emailed the url to tech department related people and everyone said yay nice upgrade.
I didnt read the email, didn't know they sent out a specific link, and had no idea how they tested it. but when I searched for the site on google and live or bing, instead I was greeted with casino popups, porn, and brittney spears unedited videos she didn't want released or something like that. Every time, regardless of the browser.
I checked email... saw the url in email and checked it before replying, and the url worked. no brittney. Turns out, the site detected if you were an admin or if you were accessing admin pages or URL directly and if so, displayed everything as expected. Come in as a random user and it was all casinos and porn advertisements.
They were flummoxed/shocked. And they denied it until we all got on a call and I suggested they pick any search engine from any computer or phone--and suggested they use some that both accessed the site before and some that didn't. Sure enough--they got the porn, too, as long as they searched for the page and didn't go to it directly.
I ended up capturing a bunch of traffic with wireshark to help them understand what was going on with redirects and so on, and where it was all going to and from, and also captured remote control sessions and unencrypted ftp transfers from permitted boxes that were... also compromised. turns out their entire virtuam environment, and other customers, were owned.
They had to go back 7 months before they found a backup that wasn't compromised. the very evening they set up the administrative portal, some bot found their site prior to it being locked down, and used default credentials to get in--from that compromise, everything spun up afterwards was accessible due to the remote attacker having integrated themselves in an account no one apparently looked for. All due to the developers not locking it down immediately, or checking the logs--EVER. The logs, once restored, showed very plainly the entire effort to compromise it.
the accidental discovery via a scan, the fingerprinting, and some attempts to log in with defaults. soon after one worked, a human logged in from some russian federation ip block, and cautiously poked around and made no changes. then a few days later, checked again for access and once in, made no further changes. a week after the first login, that is when they started to set up their C&C connectivity to the topology, cautiously at first, but after a month you'd think the bandwidth charts would have given something away--but no one responsible for the site development or hosting thought anything of it due to the constant movement of site assets and graphics and stuff through that environment--for my client and a bunch of others I had no business with, all on that shared rented space where the VMs were hosted. Pretty much sounded like the web development company rented out an entire rack of hardware at a mid-tier datacenter and divided it up as needed per client.
All of the useful backups they could restore required review and sterilization--fortunately, a lot of stuff can be copied to a fresh instance and not be compromised, which ultimately is what they had to do... install fresh and restore the static data, then make the customizations as required.
A lot of effort by a team of people to fix it--I am not sure what this email guy's support staff is like, but it's probably a hard fix and not much email archival data to replace if its really all gone like he says... whoever did it probably had been inside for a while, and consquently, part of the local backups that were erased, and likely on any off-site stuff made in the past few months if not longer.
(Score: 2, Insightful) by Anonymous Coward on Tuesday February 12 2019, @09:23PM (3 children)
I thought similarly. Something like this is the work of professionals. Professionals get paid one way or the other. There is no money in destroying with no warning. Therefore, there must be an incentive from somewhere else, either some bigger delayed payment, revenge, the LULz, or sabotage.
(Score: 1, Insightful) by Anonymous Coward on Tuesday February 12 2019, @10:09PM (2 children)
Maybe someone using his services needed to destroy some information and figured wholesale destruction was the best option?
(Score: 5, Insightful) by Joe Desertrat on Tuesday February 12 2019, @10:57PM (1 child)
The best way to hide what particular thing was targeted to be destroyed is to destroy everything.
(Score: 0) by Anonymous Coward on Wednesday February 13 2019, @12:52AM
But if that were the case they would have used sharks with lasers.
(Score: 2) by mrpg on Tuesday February 12 2019, @09:46PM (1 child)
No inside job, this was a one man shop:
"Yes, @VFEmail is effectively gone. It will likely not return. I never thought anyone would care about my labor of love so much that they'd want to completely and thoroughly destroy it."
https://twitter.com/Havokmon [twitter.com]
(Score: 3, Interesting) by bob_super on Wednesday February 13 2019, @05:50PM
Alternative explanation : He was tired of it, couldn't hand it to anyone, and just nuked it all, blaming "hackers" and a lack of offline backups, to avoid having people pestering him endlessly for continued access to their data. Gets to be a victim and close shop after 20 years of a thankless job.
Alternative explanation 2 : Got so drunk he nuked everything, then blames hackers.
(Score: 0) by Anonymous Coward on Tuesday February 12 2019, @09:55PM
Inside job? Could be, but there's examples of cloudy companies disappearing overnight like this without an inside man .. http://lmgtfy.com/?q=vaserv+hack+suicide [lmgtfy.com]
(Score: 5, Funny) by richtopia on Tuesday February 12 2019, @10:25PM (3 children)
It would be funny if someone came forward claiming to have sent multiple ransom notes/warnings, and all of those emails were binned by VFEmail's spam filter.
(Score: 2, Funny) by stretch611 on Wednesday February 13 2019, @10:10AM (2 children)
I'll begin with the most important.
I hacked your device and then got access to all your accounts... Including xx@xxx.net.
It is easy to check - I wrote you this email from your account.
Moreover, I know your intim secret, and I have proof of this.
You do not know me personally, and no one paid me to check you.
It is just a coincidence that I discovered your mistake.
In fact, I posted a malicious code (exploit) to an adult site, and you visited this site...
While watching a video Trojan virus has been installed on your device through an exploit.
This darknet software working as RDP (remote-controlled desktop), which has a keylogger,
which gave me access to your microphone and webcam.
Soon after, my software received all your contacts from your messenger, social network and email.
At that moment I spent much more time than I should have.
I studied your love life and created a good video series.
The first part shows the video that you watched,
and the second part shows the video clip taken from your webcam (you are doing inappropriate things).
Honestly, I want to forget all the information about you and allow you to continue your daily life.
And I will give you two suitable options. Both are easy to do.
First option: you ignore this email.
The second option: you pay me $750(USD).
Let's look at 2 options in detail.
The first option is to ignore this email.
Let me tell you what happens if you choose this path.
I will send your video to your contacts, including family members, colleagues, etc.
This does not protect you from the humiliation that you and
your family need to know when friends and family members know about your unpleasant details.
The second option is to pay me. We will call this "privacy advice."
Now let me tell you what happens if you choose this path.
Your secret is your secret. I immediately destroy the video.
You continue your life as if none of this has happened.
Now you might think: "I'll call to police!"
Undoubtedly, I have taken steps to ensure that this letter cannot be traced to me,
and it will not remain aloof from the evidence of the destruction of your daily life.
I don't want to steal all your savings.
I just want to get compensation for my efforts that I put in to investigate you.
Let us hope that you decide to create all this in full and pay me a fee for confidentiality.
You make a Bitcoin payment (if you don't know how to do it, just enter "how to buy bitcoins" in Google search)
Shipping amount: $750(USD).
Getting Bitcoin Addresses: 1GF8J1XRaiX2oHQo9VAFAtWZcRgMncg
(This is sensitive, so copy and paste it carefully)
Don't tell anyone what to use bitcoins for. The procedure for obtaining bitcoins can take several days, so do not wait.
I have a spetial code in Trojan, and now I know that you have read this letter.
You have 48 hours to pay.
If I don't get BitCoins, I'll send your video to your contacts, including close relatives, co-workers, and so on.
Start looking for the best excuse for friends and family before they all know.
But if I get paid, I immediately delete the video.
This is a one-time offer that is non-negotiable, so do not waste my and your time.
Time is running out.
Bye!
Now with 5 covid vaccine shots/boosters altering my DNA :P
(Score: 2) by stretch611 on Wednesday February 13 2019, @11:55PM (1 child)
This is an actual spam email. (I only changed my email address and borked the BitCoin account.)
Each iteration changes from time to time, but I have been getting a few of these (in my spam folder of course) everyday now for about 6-8 months. They all say something about hacking your device/router, claiming to see you on all the various pron sites, usually calling you a pervert, than threatening to reveal your perverted secrets to your friends and family unless a bitcoin transfer is made.
I only wonder how many people actually fall for this scam.
Now with 5 covid vaccine shots/boosters altering my DNA :P
(Score: 0) by Anonymous Coward on Thursday February 14 2019, @12:10AM
Fun fact, most of the reports of this particular scam at work are from women. There are a lot of inferences you could make from this factoid. But the one that feels the most truthy is that people who fear it could conceivably be true aren't reporting it, while those that report are confident that it can't possibly be true.
Makes you wonder though. I always tell people, that if they REALLY had something, they'd have sent you a sample, even a single photo of you from your webcams perspective.
(Score: 5, Funny) by SomeGuy on Tuesday February 12 2019, @09:39PM (1 child)
We can only hope they could erase Twatter and Facefuck with the same amount of efficiency.
(Score: 4, Touché) by bobthecimmerian on Wednesday February 13 2019, @12:04PM
I hate when people use labels like that. It's an insult to twats and fucking.
(Score: 4, Interesting) by NewNic on Tuesday February 12 2019, @10:53PM
TLA didn't like the privacy provided by VFEMail?
lib·er·tar·i·an·ism ˌlibərˈterēənizəm/ noun: Magical thinking that useful idiots mistake for serious political theory
(Score: 1) by ensigndna on Wednesday February 13 2019, @02:28AM (2 children)
Had they used modern storage arrays with snapshots and off site replication combined with better security measures this could have avoided the issue or at the very least undone the damage in a few hours.
(Score: 0) by Anonymous Coward on Wednesday February 13 2019, @03:19AM
did you rtfs?
(Score: 0) by Anonymous Coward on Wednesday February 13 2019, @10:33AM
How would snapshots have survived a disk formatting?
Off-site replication would have helped, however; given that the server in Europe was not affected, mirroring the data there should have been sufficient.
(Score: 2) by All Your Lawn Are Belong To Us on Wednesday February 13 2019, @03:26PM
Another instance of trusting someone else's machine that failed. But very sad for the owner.
Could my machine have been better? Maybe not, even though that would require targeting me specifically and not my mail provider. But doing local POP instead of IMAP and backing up my data locally what would I have lost aside from in-transit messages? (Especially considering my backups are on an offline rotation). The convenience of IMAP, I guess, but I could save messages to server for X days for some IMAP convenience.... or forward all popped mail to a separate IMAP account but send-to POP mailbox enabled.
In short... know where your data is and how you're handling it. But that requires actually learning things about computers other than where the power button and send button is. Now I'm off to press my key.
This sig for rent.