IBM gave the audience a deeper dive into the OpenCAPI initiative and hardware [theregister.co.uk] at the recently concluded HPC Advisory Council annual meeting in Lugano, Switzerland.
OpenCAPI is a new connection type that gives a high bandwidth, low latency, connection for memory, accelerators, network, storage, and other devices like ASICs.
One of the design goals for CAPI (Coherent Accelerator Processor Interface) was to minimise latency – shooting for "equivalent effective latency of DDR standards". They've done a good job of reaching that goal, with very low 10 nanosecond latency for devices like accelerators and even achieving 5ns latency for advanced memory.
The latency numbers are a pretty big deal – POWER8 round trip latency is about 10ns, and that's very sporty for a processor. Even more important is that the 10ns latency of OpenCAPI beats PCIe like a drum, since a typical PCIe round trip incurs a 100ns latency penalty.
When it comes to bandwidth, OpenCAPI today offers 25GBps while PCIe gen 3 currently offers 32GBps. The speaker, IBM's Jeff Stuecheli, explained that succeeding instances of OpenCAPI will run at 32GBps then 56GBps and higher.