We get it, press releases are full of hyperbole. Cerebras recently announced they’ve built the largest chip ever. The chip has 400,000 cores and contains 1.2 trillion transistors on a die over 46,000 square mm in area. That’s roughly the same as a square about 8.5 inches on each side. But honestly, the WSE — Wafer Scale Engine — is just most of a wafer not cut up. Typically a wafer will have lots of copies of a device on it and it gets split into pieces.
According to the company, the WSE is 56 times larger than the largest GPU on the market. The chip boasts 18 gigabytes of storage spread around the massive die. The problem isn’t making such a beast — although a normal wafer is allowed to have a certain number of bad spots. The real problems come through things such as interconnections and thermal management.
The white paper is detailed while still managing to be a bit fuzzy. In addition to somehow solving the interconnect, packaging and thermal problems with using a whole wafer, the architecture of the cores is supposed to be amenable to sparse matrices and the specific types of algorithms necessary for deep learning.
There’s no word as yet on cost or specific availability, but we were hoping at least for an emulator. However, it is certainly big and if it lives up to its promise could drive new deep learning applications. We’ve seen neural network coprocessors before. We even had our own deep dive into them.
Support the originator by clicking the read the rest link below.
