SoylentNews is people
SoylentNews
Hackers have breached the severs[sic] of email provider VFEmail.net and wiped the data from all its US servers, destroying all US customers' data in the process.
The attack took place yesterday, February 11, and was detected after the company's site and webmail client went down without notice.
"At this time, the attacker has formatted all the disks on every server," the company said yesterday. "Every VM is lost. Every file server is lost, every backup server is lost."
"This was more than a multi-password via SSH exploit, and there was no ransom. Just attack and destroy," VFEmail said.
[...] Back in November 2015, VFEmail was one of the many online email providers that were targeted by Armada Collective, a group of hackers who demanded ransom payments from victim companies to stop ongoing DDoS attacks.
There were servers in the US and in Europe; I think US users really means all users except the ones in the Europe server.
Hackers wipe US Servers of Email Provider VFEmail
Email Provider VFEMail’s US Servers Wiped by Hackers
VFEmail twitter account
(Score: 3, Interesting) by Hyperturtle on Wednesday February 13 2019, @04:12PM (2 children)
It sort of looks to me like he'd been paid financially, or with continued freedom, to hand over everything to some authority and paste selected verse from a provided script.
I was thinking that if he accepted the carrot, he knew he'd be labeled a fool or similar from the various social media cloud internet experts, and couldn't ever run a similar service again. Who'd trust him with data they wanted to keep after this? He'd be made into an example of why someone else's products and services should be used. He's marketing gold right now--the example to avoid.
I mean the fact there's "no backups' since forever is pretty ludicrous. Anyone that custom wrote something special usually has the presence of mind to make a few copies and lose them at home; I have floppy disks that are ancient with stuff i did a long time ago that I never had to restore from such disks--but the disks outlasted the hardware I backed up! It's just not easy to accept that his emotional investment into the logic behind how it all worked was not met with similar enthusiasm to make sure his efforts were not destroyed via his own mistakes, let alone someone else's deliberate attempts to sabatoge it. Especially after receiving various threats over the years...
The fact there isn't even an LTO2 tape somewhere with a full copy from 2006 or something seems like disaster recovery was not funded, if that's all true... Even an old hard drive or array from a past upgrade. I usually keep a few copies of *important* stuff off the network -- and a few copies on the network of that and everything else. I expect the backups are in the hands of whoever he handed the infrastructure over to.
This destruction was so complete, for a service up for almost 20 years, that it seems more than an inside job. I'd say he handed all the stuff over to someone(s) and memorized a script he was provided, or just copied and pasted from it to twitter and wherever, perhaps his emails were coached in response to questioning--looking around, a few places indicated he didn't respond to the journalists questions in time for the posts to go live.
That could mean the guy is very busy not restoring his systems, or the questions are filtering through the people telling him what to say. Or he's found new time in life to do something else and spending time with friends and family, pleased he's still free and not financially insolvent. He seems pretty calm about his life's work (so to speak) could mean he's really a great resource under pressure, or that he's come to terms with what has happened. Or that he's still free for having cooperated and that provides some solace. Maybe he's happy to cooperate, maybe he didn't know he was hosting horrible stuff from some criminal element, Maybe he's mortified at what he had to do. But publically, he's got his act together even though everything fell apart.
The place had been up for a very long time. This may have provided some authority a wonderful database of emails to mine, going back years and years and years; I have to wonder if anything was deleted at all aside from his own ability to get into the stuff once walked away from it. Any other reason just doesn't make a lot of sense to me considering what various news articles about this have stated... my thoughts are that one can't be so smart to make a service like this, run it reliably for so long, handle all the everythings, and also forget to make useful backups knowing all of the calamaties that have struck data centers and networks and computers over the years--this service has been up forever, the guy has at least heard of it all if not seen it all. He couldn't be ignorant of the risks of having locally accessible disk to disk replication as the sole means of redundancy.
That public IP he shared as a potential source address probably was just something from a scan that could be a random hacked machine seen scanning a few times, or it's just an IP in a range that now doesnt respond anymore... doesn't matter. Maybe he was even told to use it because the IP has been seen/reported numerous times by other businesses. Never waste a good scapegoat on something that can be disproven easily.
Someone probably should check old posted policies about privacy and such and see if any signaling was done prior to the outage. It could be that nothing was changed because he never let authorities in; it may be that to make it out intact, he had to just hand it all over at once and not let some authority tap in and open him up to legal stuff if his service agreements suddenly were missing items like "and the people you are trying to avoid can read your email now."
I think what happened is bad, and I don't like to be a dramatic conspiracy theorist (except for when it comes to personalized advertisements), but my spidey sense is going off and his story just doesn't sound right. Maybe there is a lot more that we will never know because the data is truly gone--and so all we (and I) have to go from is his incomplete view of the events. I don't want to discredit him, but it still sounds like a well crafted story rather than what happened.
Hopefully what he's said is true, because such attacks can be defended against or at least mitigated via enforcement of industry standard disaster recovery best practices... I hope that his story is factual and really is the worst of it and that he's the victim of a criminal rather than something worse (like stupidity or authority). And that every other similar service providing privacy and security lives happily ever after having learned from this.
(Score: 0) by Anonymous Coward on Wednesday February 13 2019, @04:33PM (1 child)
I am pretty sure my hosting provider keeps backups of my VM in case I should remotely brick it. Has his hoster been asked about their backup measures? They may not want to answer if the government was involved, but it would be good journalism to ask.
(Score: 2) by Hyperturtle on Friday February 15 2019, @03:51PM
It didn't sound like he was quite so commoditized in that he had a number of virtuals somewhere, but even so, off-site backups sometimes contain that surprise of someone having rooted the entire topolology.
I did work for a place that, on a lark, I searched for their website after I heard the new version went live. They did all sorts of testing and said it was great. They emailed the url to tech department related people and everyone said yay nice upgrade.
I didnt read the email, didn't know they sent out a specific link, and had no idea how they tested it. but when I searched for the site on google and live or bing, instead I was greeted with casino popups, porn, and brittney spears unedited videos she didn't want released or something like that. Every time, regardless of the browser.
I checked email... saw the url in email and checked it before replying, and the url worked. no brittney. Turns out, the site detected if you were an admin or if you were accessing admin pages or URL directly and if so, displayed everything as expected. Come in as a random user and it was all casinos and porn advertisements.
They were flummoxed/shocked. And they denied it until we all got on a call and I suggested they pick any search engine from any computer or phone--and suggested they use some that both accessed the site before and some that didn't. Sure enough--they got the porn, too, as long as they searched for the page and didn't go to it directly.
I ended up capturing a bunch of traffic with wireshark to help them understand what was going on with redirects and so on, and where it was all going to and from, and also captured remote control sessions and unencrypted ftp transfers from permitted boxes that were... also compromised. turns out their entire virtuam environment, and other customers, were owned.
They had to go back 7 months before they found a backup that wasn't compromised. the very evening they set up the administrative portal, some bot found their site prior to it being locked down, and used default credentials to get in--from that compromise, everything spun up afterwards was accessible due to the remote attacker having integrated themselves in an account no one apparently looked for. All due to the developers not locking it down immediately, or checking the logs--EVER. The logs, once restored, showed very plainly the entire effort to compromise it.
the accidental discovery via a scan, the fingerprinting, and some attempts to log in with defaults. soon after one worked, a human logged in from some russian federation ip block, and cautiously poked around and made no changes. then a few days later, checked again for access and once in, made no further changes. a week after the first login, that is when they started to set up their C&C connectivity to the topology, cautiously at first, but after a month you'd think the bandwidth charts would have given something away--but no one responsible for the site development or hosting thought anything of it due to the constant movement of site assets and graphics and stuff through that environment--for my client and a bunch of others I had no business with, all on that shared rented space where the VMs were hosted. Pretty much sounded like the web development company rented out an entire rack of hardware at a mid-tier datacenter and divided it up as needed per client.
All of the useful backups they could restore required review and sterilization--fortunately, a lot of stuff can be copied to a fresh instance and not be compromised, which ultimately is what they had to do... install fresh and restore the static data, then make the customizations as required.
A lot of effort by a team of people to fix it--I am not sure what this email guy's support staff is like, but it's probably a hard fix and not much email archival data to replace if its really all gone like he says... whoever did it probably had been inside for a while, and consquently, part of the local backups that were erased, and likely on any off-site stuff made in the past few months if not longer.