NVIDIA has detailed the full GV100 GPU [tomshardware.com] as well as the first product based on the GPU, the Tesla V100:
The Volta GV100 GPU uses the 12nm TSMC FFN process, has over 21 billion transistors, and is designed for deep learning applications. We're talking about an 815mm2 die here, which pushes the limits of TSMC's current capabilities. Nvidia said it's not possible to build a larger GPU on the current process technology. The GP100 was the largest GPU that Nvidia ever produced before the GV100. It took up a 610mm2 surface area and housed 15.3 billion transistors. The GV100 is more than 30% larger.
Volta's full GV100 GPU sports 84 SMs (each SM features four texture units, 64 FP32 cores, 64 INT32 cores, 32 FP64 cores) fed by 128KB of shared L1 cache per SM that can be configured to varying texture cache and shared memory ratios. The GP100 featured 60 SMs and a total of 3840 CUDA cores. The Volta SMs also feature a new type of core that specializes in Tensor deep learning 4x4 Matrix operations. The GV100 contains eight Tensor cores per SM and deliver a total of 120 TFLOPS for training and inference operations. To save you some math, this brings the full GV100 GPU to an impressive 5,376 FP32 and INT32 cores, 2688 FP64 cores, and 336 texture units.
[...] GV100 also features four HBM2 memory emplacements, like GP100, with each stack controlled by a pair of memory controllers. Speaking of which, there are eight 512-bit memory controllers (giving this GPU a total memory bus width of 4,096-bit). Each memory controller is attached to 768KB of L2 cache, for a total of 6MB of L2 cache (vs 4MB for Pascal).
The Tesla V100 has 16 GB of HBM2 memory with 900 GB/s of memory bandwidth. NVLink [wikipedia.org] interconnect bandwidth has been increased to 300 GB/s.
Note the "120 TFLOPS" for machine learning operations. Microsoft is "doubling down" on AI [tomshardware.com], and NVIDIA's sales to data centers have tripled in a year [tomshardware.com]. Sales of automotive-oriented GPUs (more machine learning) also increased.
IBM Unveils New AI Software, Will Support Nvidia Volta [hpcwire.com]