Google has designed a low-power version of its homegrown AI math accelerator, dubbed it the Edge TPU, and promised to ship it to developers by October. Announced at Google Next 2018 today, the ASIC is a cutdown edition of its Tensor Processing Unit (TPU) family of in-house-designed coprocessors. TPUs are used internally at Google to power its machine-learning-based services, or are rentable via its public cloud. These chips are specific[ally] designed for and used to train neural networks and perform inference.
Now the web giant has developed a cut-down inference-only version suitable for running in Internet-of-Things gateways. The idea is you have a bunch of sensors and devices in your home, factory, office, hospital, etc, connected to one of these gateways, which then connects to Google's backend services in the cloud for additional processing.
Inside the gateway is the Edge TPU, plus potentially a graphics processor, and a general-purpose application processor running Linux or Android and Google's Cloud IoT Edge software stack. This stack contains lightweight Tensorflow-based libraries and models that access the Edge TPU to perform AI tasks at high speed in hardware. This work can also be performed on the application CPU and GPU cores, if necessary. You can use your own custom models if you wish.
The stack ensures connections between the gateway and the backend are secure. If you wanted, you could train a neural network model using Google's Cloud TPUs and have the Edge TPUs perform inference locally.
Google announcement. Also at TechCrunch, CNBC, and CNET.
Related: Google's New TPUs are Now Much Faster -- will be Made Available to Researchers
Google Renting Access to Tensor Processing Units (TPUs)
Nvidia V100 GPUs and Google TPUv2 Chips Benchmarked; V100 GPUs Now on Google Cloud
Related Stories
Google's machine learning oriented chips have gotten an upgrade:
At Google I/O 2017, Google announced its next-generation machine learning chip, called the "Cloud TPU." The new TPU no longer does only inference--now it can also train neural networks.
[...] In last month's paper, Google hinted that a next-generation TPU could be significantly faster if certain modifications were made. The Cloud TPU seems to have have received some of those improvements. It's now much faster, and it can also do floating-point computation, which means it's suitable for training neural networks, too.
According to Google, the chip can achieve 180 teraflops of floating-point performance, which is six times more than Nvidia's latest Tesla V100 accelerator for FP16 half-precision computation. Even when compared against Nvidia's "Tensor Core" performance, the Cloud TPU is still 50% faster.
[...] Google will also donate access to 1,000 Cloud TPUs to top researchers under the TensorFlow Research Cloud program to see what people do with them.
Previously: Google Reveals Homegrown "TPU" For Machine Learning
Google Pulls Back the Covers on Its First Machine Learning Chip
Nvidia Compares Google's TPUs to the Tesla P40
NVIDIA's Volta Architecture Unveiled: GV100 and Tesla V100
Google has announced that its custom-made tensor processing units (TPUs) have been made available for rent on the Google Cloud Platform:
A few years ago, Google created a new kind of computer chip to help power its giant artificial intelligence systems. These chips were designed to handle the complex processes that some believe will be a key to the future of the computer industry. On Monday, the internet giant said it would allow other companies to buy access to those chips through its cloud-computing service. Google hopes to build a new business around the chips, called tensor processing units, or T.P.U.s.
[...] In addition to its T.P.U. chips, which sit inside its data centers, the company has designed an A.I. chip for its smartphones.
Right now, Google's new service is focused on a way to teach computers to recognize objects, called computer vision technology. But as time goes on, the new chips will also help businesses build a wider range of services, Mr. Stone said. At the end of last year, hoping to accelerate its work on driverless cars, Lyft began testing Google's new chips. Using the chips, Lyft wanted to accelerate the development of systems that allow driverless cars to, say, identify street signs or pedestrians. "Training" these systems can take days, but with the new chips, the hope is that this will be reduced to hours. "There is huge potential here," said Anantha Kancherla, who oversees software for the Lyft driverless car project.
T.P.U. chips have helped accelerate the development of everything from the Google Assistant, the service that recognizes voice commands on Android phones, to Google Translate, the internet app that translates one language into another. They are also reducing Google's dependence on chip makers like Nvidia and Intel. In a similar move, it designed its own servers and networking hardware, reducing its dependence on hardware makers like Dell, HP and Cisco.
Also at The Next Platform, TechCrunch, and CNBC.
RiseML Benchmarks Google TPUv2 against Nvidia V100 GPU
RiseML Blog last week reported benchmarks that suggest Google's custom TPUv2 chips and Nvidia V100 GPUs offer roughly comparable performance on select deep learning tasks but that the cost for access to TPUv2 technology on Google Cloud is less than the cost of accessing V100s on AWS. Google began providing public access to TPUv2 in February via its Cloud TPU offering which includes four TPUv2 chips.
[...] Elmar Haußmann, cofounder and CTO of RiseML, wrote in the company blog, "In terms of raw performance on ResNet-50, four TPUv2 chips (one Cloud TPU) and four V100 GPUs are equally fast (within 2% of each other) in our benchmarks. We will likely see further optimizations in software (e.g., TensorFlow or CUDA) that improve performance and change this.
Google later announced that it would offer access to Nvidia Tesla V100 GPUs on its Google Cloud Platform:
The cloud giant also announced general availability of Nvidia's previous-generation P100 parts, in public beta on Google's platform since September 2017.
[...] While Google was the first of the big three public cloud providers to embrace [Nvidia Tesla (Pascal)] P100s, it was the last to adopt V100s. Amazon Web Services has offered the Volta parts since October 2017. Microsoft Azure followed with a private preview in November 2017. And IBM brought PCIe variant V100s into its cloud in January of this year.
(Score: 1, Insightful) by Anonymous Coward on Thursday July 26 2018, @04:31PM (3 children)
Wonder what the pricing will be compared to Nervana [intel.com]
Why are Google marketing the "cloud" when most serious NN models will be privately held and fiercely protected?
(Score: 2) by takyon on Thursday July 26 2018, @04:35PM (1 child)
They use TOPS, aka tera-operations per second. But then the Wikipedia lists FLOPS [wikipedia.org] anyway so whatever.
http://hexus.net/tech/news/industry/104299-google-benchmarks-tensor-processing-unit-tpu-chips/ [hexus.net]
https://arxiv.org/ftp/arxiv/papers/1704/1704.04760.pdf [arxiv.org]
https://www.nextplatform.com/2017/04/05/first-depth-look-googles-tpu-architecture/ [nextplatform.com]
[SIG] 10/28/2017: Soylent Upgrade v14 [soylentnews.org]
(Score: 1, Interesting) by Anonymous Coward on Thursday July 26 2018, @05:43PM
Aren't these for their cloud processors, not this inference only chip?
Also have a hunch it'll be YAFSDK (Yet Another Fat SDK) as opposed to the Intel offering which will be a fairly standard toolchain.
(Score: 2) by c0lo on Friday July 27 2018, @08:54AM
Their business model: you pay them for the cloud to train your NN, download the result into your local (Google made) TPU and use it close to your sensors.
The training is energy intensive, the sensing/recognition needs to be very frugal with the energy available in an IoT.
Bottom line: at no point you are allowed independence from Google.
https://www.youtube.com/@ProfSteveKeen https://soylentnews.org/~MichaelDavidCrawford
(Score: 0) by Anonymous Coward on Thursday July 26 2018, @04:34PM (3 children)
Basically, how such computing should be done? Preferably with little to none of the data reaching the central server.
Funny how computing got centralized, and now is slowly getting decentralized again. I'm happy to see tech for that developing, but I worry that data ownership will continue to be centralized.
(Score: 4, Insightful) by takyon on Thursday July 26 2018, @04:37PM (2 children)
That's the gist of the story. Google and others returning to the edge (as it should be), but trying to maintain some control/lock-in.
If I want to run a bootleg JARVIS or render DeepFake porn, I want to use my own hardware.
[SIG] 10/28/2017: Soylent Upgrade v14 [soylentnews.org]
(Score: 4, Insightful) by fyngyrz on Thursday July 26 2018, @05:05PM (1 child)
Yeah, this. I wouldn't touch this with someone else's ten foot pole. It's just another variation on third-party clouds, and just like 3rd party clouds, it's 100% a very, very bad idea.
No thanks. Local solutions much preferred.
(Score: 2) by corey on Thursday July 26 2018, @11:45PM
(Score: 2) by Snotnose on Thursday July 26 2018, @06:12PM (3 children)
When I think of IoT I think doorbells, thermostats, toasters, washing machines, etc etc etc. Not sure why my toaster needs an AI, let alone a GPU.
Then again, I'm neither rich nor live in Silly Valley.
It was a once in a lifetime experience. Which means I'll never do it again.
(Score: 2) by Freeman on Thursday July 26 2018, @06:17PM
Overpowered compared to what? The computer on Apollo One? Sure. My Phone? Doubtful. IoT stuff kind of screams more money than I know what to do with, but perhaps it can produce some useful things.
Joshua 1:9 "Be strong and of a good courage; be not afraid, neither be thou dismayed: for the Lord thy God is with thee"
(Score: 2) by crafoo on Thursday July 26 2018, @06:22PM (1 child)
Maybe they're thinking of businesses. Not necessarily improving the lives with random consumer goods, but careful, thoughtful populace instrumentation and control.
How about an automated checkout that matches faces to credit cards and watches to make sure everything is scanned. Computers that identify users by mouse movement and keyboard use. phones that flag out-of-ordinary travel and movement behaviour to the proper oversight and enforcement authorities. roadway nodes that can independently interact with self-driving cars to improve safety based on traffic flow, weather, lighting, and local emergencies. refrigerators that can estimate days-to-heart-failure based on consumption habits and report to the federal insurance oversight committee.
(Score: 0) by Anonymous Coward on Thursday July 26 2018, @08:24PM
Wow, if that's the future, I'm not interested. Luckily I'm already mid-60s, so maybe I won't live to see my fridge tattling on my eating habits, or my computer not working after I pull a muscle in my shoulder (and move the mouse differently).
Re a different thread, I'm quite happy with the simple thermostat in my house, and the one in my toaster oven, and in my oven. And the timer in the microwave is also good enough.
(Score: 2) by Pino P on Thursday July 26 2018, @06:22PM
If Google can announce the Edge TPU, will Microsoft follow up with the Chrome TPU?
(Score: 2) by leftover on Thursday July 26 2018, @11:10PM
I wonder if this is a bigger version of their AIY Vision Kit. All training data needs to pass through Google and they keep a copy. Instant world-dominating dataset on the cheap.
Bent, folded, spindled, and mutilated.