Major US telecom was infiltrated by backdoored Supermicro hardware, Bloomberg says
Five days after Bloomberg stunned the world with still-unconfirmed allegations that Chinese spies embedded data-sniffing chips in hardware used by Apple, Amazon, and dozens of other companies, the news organization is doubling down. Bloomberg is now reporting that a different factory-seeded manipulation from the previously described one was discovered in August inside the network of a major US telecommunications company.
Bloomberg didn't name the company, citing a non-disclosure agreement between the unnamed telecom and the security firm it hired to scan its data centers. AT&T, Sprint and T-Mobile all told Ars they weren't the telecom mentioned in the Bloomberg post. Verizon and CenturyLink also denied finding backdoored Supermicro hardware in their datacenters, Motherboard reported.
Tuesday's report cites documents, analysis, and other evidence provided by Yossi Appleboum, who is co-CEO of a hardware security firm called Sepio Systems. Bloomberg said that, while Sepio was scanning servers belonging to the unnamed telecom, the firm detected unusual communications from a server designed by Supermicro. Supermicro, according to last week's Bloomberg report, is the hardware manufacturer whose motherboards were modified in the factory to include a tiny microchip that caused attached servers to come under the control of a previously unreported division of China's People's Liberation Army. Supermicro told Bloomberg it had no knowledge of the implant, marking the second time the hardware maker has denied knowing anything about the reported manipulations.
[...] The criticism was still at full pitch on Tuesday morning when Bloomberg published its follow-up article. While it names a single source, some security experts quickly challenged the credibility of the report. "Sure this story has one named source but it technically makes even less sense than the first one," Cris Thomas, a security expert who tweets under the handle SpaceRogue, wrote. "Come on @Bloomberg get somebody who knows what they're talking about to write these stories. Calling BS on this one as well."
Previously: Chinese Spy Chips Allegedly Inserted Into Amazon, Apple, etc. Datacenters by Super Micro
Bloomberg Stands by Chinese Chip Story as Apple, Amazon Ratchet up Denials
Related: Firmware Vulnerabilities in Supermicro Systems
Supermicro Announces Suspension of Trading of Common Stock on Nasdaq and its Intention to Appeal
Related Stories
Submitted via IRC for mechanicjay
We have already seen both proof-of-concept and in-the-wild demonstrations of attacks targeting system firmware such as SMM rootkits, device firmware replacement, and even usurping firmware-based features for malware. As part of our ongoing security research efforts, we recently reviewed various Supermicro systems and discovered serious firmware vulnerabilities. Such issues affect many models and have persisted for many years, which could be problematic since these systems are commonly used as data center servers. As other researchers have shown, Supermicro is not alone. Security vulnerabilities in firmware continue to be discovered regularly. Unfortunately, malicious activity at the firmware and hardware level is invisible to most detection and response mechanisms in use today, leaving many critical systems exposed to attacks that target this area.
These vulnerabilities are easily exploitable and provide malware with the same impact as having physical access to the kind of system that is usually stored in a secure data center. A physical attacker who can open the case could simply attach a hardware programmer to bypass protections. Using the attacks we have discovered, it is possible to scale powerful malware much more effectively through malicious software instead of physical access.
Source: Firmware Vulnerabilities in Supermicro Systems
Though this happened earlier in the week, I just now found out about it. Given how well-known the company is, I thought other Soylentils would like to know about this, too.
Supermicro® Announces Suspension of Trading of Common Stock on Nasdaq and its Intention to Appeal:
Super Micro Computer, Inc. (NASDAQ:SMCI) [...] today announced that, as expected, the Company received a notification letter from The Nasdaq Stock Market Hearings Panel [...] on August 22, 2018, indicating that trading in the Company's common stock on Nasdaq's Global Select Market will be suspended effective at the open of business on August 23, 2018.
The Company previously announced on August 21, 2018 that it did not expect to regain compliance with the Nasdaq continued listing requirements by August 24, 2018, the deadline previously set by the Panel.
The Panel's letter also stated that the Panel has determined to delist the Company's shares from Nasdaq after applicable appeal periods have lapsed. The Company intends to appeal the Panel's decision to the Nasdaq Listing and Hearing Review Council. During the appeal period, trading in the Company's common stock on Nasdaq will remain suspended and Nasdaq will not delist the Company's common stock pending such appeal. Once the Company has regained compliance with its SEC filing requirements, the Company intends to promptly request that Nasdaq lift the suspension in trading of its common stock or, in the event the common stock is delisted, to promptly apply to relist its common stock on Nasdaq or another national securities exchange.
While the Company's common stock is suspended from trading on Nasdaq, the Company expects that its shares will be quoted on the OTC Markets under the trading symbol SMCI.
According to Wikipedia:
Chinese spy chips are found in hardware used by Apple, Amazon, Bloomberg says; Apple, AWS say no way
The chips, which Bloomberg said have been the subject of a top secret U.S. government investigation starting in 2015, were used for gathering intellectual property and trade secrets from American companies and may have been introduced by a Chinese server company called Super Micro that assembled machines used in the centers.
[...] China has long been suspected — but rarely directly implicated — in en masse spy campaigns based on hardware made there. The majority of electronic components used in U.S. technology are manufactured in China. Companies including component manufacturers Huawei and ZTE, as well as surveillance camera maker Hikvision, have all fallen under intense suspicion and scrutiny from the U.S. government in the past year.
I'd think that the big guys would be designing their own boards. Maybe we should only buy PCBs from South Korea.
Also at Bloomberg and The Guardian.
Following up on our story from Thursday — Chinese Spy Chips Allegedly Inserted Into Amazon, Apple, etc. Datacenters by Super Micro — there is a report from Ars Technica Bloomberg stands by Chinese chip story as Apple, Amazon ratchet up denials:
On Thursday morning, Bloomberg published a bombshell story claiming that the Chinese government had used tiny microchips to infiltrate the data centers of Apple and Amazon. Apple and Amazon, for their part, responded with unusually specific and categorical denials. It's clear that someone is making a big mistake, but 24 hours later, it's still not clear whether it's Bloomberg or the technology companies.
On Thursday afternoon, Apple laid out its case against the story in a lengthy post on its website. The post specifically disputed a number of Bloomberg's claims. For example, Bloomberg says that after discovering a mysterious chip in one of its servers, Apple "reported the incident to the FBI," leading to an investigation. Apple flatly denies that this occurred.
"No one from Apple ever reached out to the FBI about anything like this," Apple writes. "We have never heard from the FBI about an investigation of this kind."
Amazon's response has been equally emphatic and detailed. "There are so many inaccuracies in this article as it relates to Amazon that they're hard to count," Amazon wrote on Thursday. "We never found modified hardware or malicious chips in servers in any of our data centers."
Yet Bloomberg reporter Jordan Robertson, one of the article's co-authors, has stood by his story. In a Thursday afternoon appearance on Bloomberg TV, Robertson said that he talked to 17 anonymous sources—both in US intelligence agencies and at affected companies—who confirmed the story.
So what's going on? It's clear that someone isn't telling the truth, but it's hard to tell what the real story is.
A comment to that story on Ars noted:
The (alleged) chip is associated with the BMC (baseboard management controller). It has indirect access to everything that the BMC can touch, which is pretty much everything in the system.
See, also, coverage on Hackaday where a comment identifies the particular board in question as being a MicroBlade MBI-6128R-T2. A link to a tweet reveals a picture of the board in question and a followup picture showing where the extra device would be located.
Audit: No Chinese surveillance implants in Supermicro boards found
In a letter to customers issued December 11, Supermicro President and CEO Charles Liang and other top executives announced that an audit conducted by an outside investigating team had found no evidence of any malicious hardware incorporated into motherboards currently or previously manufactured by the company. The letter is the latest rebuttal to Bloomberg reports in October that claimed tiny chips that provided a backdoor for China's intelligence agencies had been integrated into boards provided to major Internet and cloud providers—a report also refuted by the companies the report claimed were targeted.
"After a thorough examination and a range of functional tests, the investigative firm found absolutely no evidence of malicious hardware on our motherboards," the letter signed by Liang, Supermicro Senior Vice President and Chief Compliance Officer David Weigland, and Senior VP and Chief Product Officer Raju Penumatcha stated.
Searching for site:soylentnews.org supermicro on Google brought up a Supermicro ad linking the CEO letter, with the link entitled "Supermicro Independent Testing | No Malicious Hardware". Do you believe them?
Previously: Chinese Spy Chips Allegedly Inserted Into Amazon, Apple, etc. Datacenters by Super Micro
Bloomberg Stands by Chinese Chip Story as Apple, Amazon Ratchet up Denials
Bloomberg Claims That a Major U.S. Telecom Operated a Server Backdoored by a Hidden Chip
Related: Apple Deleted Server Supplier After Finding Infected Firmware in Servers
Firmware Vulnerabilities in Supermicro Systems
Supermicro Announces Suspension of Trading of Common Stock on Nasdaq and its Intention to Appeal
(Score: 2) by dltaylor on Wednesday October 10 2018, @01:19PM (6 children)
Plenty of skill available to install back doors in the ME code, so it could be that.
Honestly, though, how many companies perform a thorough receiving inspection, verifying the BIOS against a third-party inventory (get an image from Intel of their bits and check them against the BIOS, for example), checking that the components on the motherboards are exactly and only what should be there (although it wouldn't be hard to fab an I/O chip to have extra "features"), and running a port scan, at least, of boards in a test environment, for example (hardly an exhaustive list)?
(Score: 2) by RS3 on Wednesday October 10 2018, @01:54PM (5 children)
I like your thinking and I'd like to think inspections are being done, but I'm skeptical. Laziness is a big factor, often justified by cost reductions. Another factor is "production variations".
To verify a BIOS image you'd have to remove the chip and read it in a chip reader. You might be able to do it in-circuit but I doubt it. Booting an MB risks a clever BIOS trojan hiding itself. If you were sure you could read hardware-level bits you could do it but I'd rather remove the chip.
(Score: 3, Interesting) by DannyB on Wednesday October 10 2018, @05:39PM (4 children)
Just an idea.
What if BIOS / UEFI were in a socketed ROM chip, or something similar. The idea being that the only way to replace it is to have physical access to the computer. Not remote access. (Yes, I get it that in a data center this might not be the best idea. But for many other PCs it might be.)
There could be a way to verify the contents of these ROM chips (or game cartridges or whatever they are). Heck, you might be able to buy a unit on Amazon that provides multiple checksums in different algorithms. That way if one hash / checksum algorithm is weak, you still can't fake all of them by manipulation of the contents.
You could obtain your ROM independently. The verification device independently. The computer independently.
If you don't like a firmware upgrade, just plug the old one back in.
The lower I set my standards the more accomplishments I have.
(Score: -1, Offtopic) by Anonymous Coward on Wednesday October 10 2018, @05:51PM
Uhhhhhh.
A socketed BIOS won't help if the motherboard has a parasitic implant.
(Score: 5, Informative) by RS3 on Wednesday October 10 2018, @07:22PM (1 child)
In fact when I hurriedly wrote my previous comment I was looking at an 8-year old ASUS MB which has, in good ASUS form, BIOS chip in an 8-pin DIP socket. That would be a big positive deciding factor for me (socketed BIOS).
Also gets you out of the "bricked" MB due to failed BIOS update.
Also allows you to program BIOS using a programmer. I get very frustrated with BIOS updates being a Windows-only .exe when I have a Linux-only server.
(Score: 2) by DannyB on Wednesday October 10 2018, @09:45PM
Pssssssst! Shhhhhhh! Don't tell anyone but that socketed ROM might even be able to hold an OS of sorts.
The lower I set my standards the more accomplishments I have.
(Score: 0) by Anonymous Coward on Thursday October 11 2018, @04:43PM
This was the entire problem with the switch to 8 pin SPI. The write protect jumper never actually write protected the hardware. And due to the design they are all soft protected only *AFTER* the system has initialized, the write protect jumper is shorted AND the write protect enable command is then sent to the bios chip... and only until power to the chip is reset or glitched at which point it is writeable again.
This is a huge flaw/'feature' in all currently produced flash chips (someone tell me if there were 2megabit or megabyte parts where this wasn't true) that renders them soft writeable in almost all situations, which is part of the excuse for 'softblocking' memory ranges using the southbridge/FCH.
Solve this and you would go a long way towards solving other security issues on modern computer mainboards, the other major one being user controlled signing keys or 'out of system' firmware verification and control units, like IPMI or ME, but user controlled.
(Score: 2, Insightful) by Anonymous Coward on Wednesday October 10 2018, @01:28PM (5 children)
Just because the boards were manufactured in China doesn't mean this is the fault of someone from there. The oddities of this story make me think apple, etc or at least the people responsible for checking this stuff at those companies were issued gag orders.
Or maybe bloomberg is just falling for fake news, or this is just propaganda as part of the ongoing transition in who you are supposed to hate/fear from russia to china.
(Score: 3, Interesting) by Anonymous Coward on Wednesday October 10 2018, @02:24PM (1 child)
The Russia/China split thing is likely due to a conflict in the ruling class. The only thing that is certain is that the focus has moved from "war on terror" to "great power conflict."
One faction of the ruling class believes that globalist capitalism will be served best by confronting Russia, I'd guess for reasons having to do with access to oil in the Middle East. That faction also blames Russia for "She Lost."
The other faction wants to drive China into submission and prevent it from usurping the USA's position in the world economy, again in the interests of furthering globalist capitalism.
Of course, neither of those factions give a shit about the "deplorables" (that's us, the working and middle classes), and both factions support the continued siphoning of wealth from the working and middle classes to the bourgeoisie.
(Score: 0) by Anonymous Coward on Thursday October 11 2018, @05:55AM
We've always been at war with EastAsia
(Score: 3, Interesting) by Anonymous Coward on Wednesday October 10 2018, @03:11PM
Good point. This equipment could have just as easily been picked by the fed in customs when it arrived and diddled in one of the NSA's hardware diddling centers. Perhaps the question isn't who got pwnd but who did the pwning.
Hell the NSA could have done it, AND been the one to leak to the press. End goal would be to pressure various companies to work with them, because they are really "the good guys", and look at the terrible threat from the Communists! It isn't like the FBI hasn't been calling everyone but Mary Poppins a Russian colaborator lately.
The idea that this is a false flag by the domestic military industrial complex, is certainly plausible. It would be nice if any part of the federal government had the integrity such that it wasn't so. But that is clearly not the case. So at this point, it really doesn't matter who did it, since the only people who really know aren't believable, and their masters have no problem with the idea of starting WWIII just for the lulz.
Best bet is to fix it, ignore the drama, and move on.
(Score: 2, Insightful) by redneckmother on Wednesday October 10 2018, @04:38PM (1 child)
"We have always been at war with Eurasia."
Mas cerveza por favor.
(Score: 3, Funny) by DannyB on Wednesday October 10 2018, @05:44PM
No it is Eastasia you ignert foal! Quick! Someone call the Ministry of Truth!
The lower I set my standards the more accomplishments I have.
(Score: 2) by crafoo on Wednesday October 10 2018, @04:34PM (5 children)
On what grounds are the critics calling BS on these two Bloomberg articles? Is it because the damning allegations come from 90% anonymous sources? That actually seems reasonable.
Not that I'm all that skeptical, in general. It likely that everything is backdoored by at least one state-sponsored spy agency.
(Score: 0) by Anonymous Coward on Wednesday October 10 2018, @04:54PM (2 children)
Anonymous sources, anonymous victim. Maybe not implausible method but sounds like a movie plot when backdoors could easily be present in firmware without the need for an obvious physical alteration to the hardware. Actually this fits the bill for folklore/urban legend. Somebody, somewhere once encountered a bogeyman. You could be next!
And on top of that, TFS claims that the story has already changed. All in all sounds like a Chinese equivalent to the Skripal poisoning, but without even a named victim. But hey, fewer details there are, fewer problems for critical thinkers and "fake news" sites like WSWS to point out?
In the Skripal case, three people were admitted to the hospital (and not allowed to talk to the public), one of whom died, but the details range from murky to implausible with conclusion jump left and right. In this case, the entire thing could be completely made up out of whole cloth.
(Score: 2) by bob_super on Wednesday October 10 2018, @09:15PM (1 child)
> an obvious physical alteration to the hardware
I dare you to notice an 0402 or 0201 chip on a modern PCB.
(Score: 0) by Anonymous Coward on Thursday October 11 2018, @07:22PM
An extra one? No problem. I'd use stuff like this:
https://en.wikipedia.org/wiki/Automated_optical_inspection [wikipedia.org]
https://en.wikipedia.org/wiki/Automated_X-ray_inspection#Use_of_AXI_in_electronics_manufacturing [wikipedia.org]
More likely to be able to sneak pictures/pwnware onto existing silicon. But extra components, even under other components, are more likely to be spotted.
Bean counters, OCD inspectors etc can make it difficult to sneak in extra components.
(Score: 3, Interesting) by DannyB on Wednesday October 10 2018, @05:45PM
Profit? Or something to do with money? Or lawyers? Or gag orders? Or payoffs?
The lower I set my standards the more accomplishments I have.
(Score: 0) by Anonymous Coward on Thursday October 11 2018, @07:12PM
1) No real evidence so far except claims - e.g. no motherboard from the Amazon nor Apple backdooring has surfaced.
2) The first article made some unlikely claims e.g. chip between fibreglass layers. If you're going to backdoor stuff you'd do it in easier ways that are harder to detect.
3) Amazon and Apple both issued denials and Apple even wrote a letter to Congress.
4) Most of the other newspapers etc seem to be keeping some distance away from the story (they're doing stuff like "Bloomberg claimed").
Each by itself isn't much or enough but combined it does make Bloomberg look like the crazy guy in the room...
Sometimes the crazy guy is right, but at this stage...
(Score: 0) by Anonymous Coward on Wednesday October 10 2018, @04:40PM
http://www.documentcloud.org/documents/4995748-Letter-20-October-208th-20version.html [documentcloud.org]
https://arstechnica.com/tech-policy/2018/10/apple-attacks-bloomberg-spy-chip-story-in-letter-to-congress/ [arstechnica.com]
If only actual backdoors are found in this Telco's servers but not Apple or Amazon then I'm going to think that this is more likely to be a false flag operation. Maybe even someone trying to manipulate stock prices...