Why won't FPGA be viable in Machine Learning

FPGA is a custom computing architecture. FPGAs provide superior energy efficiency (Performance/Watt) compared to high-end GPUs, they are not known for offering top peak floating-point performance. Intel FPGAs offer a comprehensive software ecosystem that ranges from low level Hardware Description languages to higher level software development environments with OpenCL, C, and C++. Current trends in DNN algorithms may favor FPGAs, and that FPGAs may even offer superior performance. Even with these advantages, it's an uphill battle against the widely used GPUs.

Nvidia has penetrated both our systems for GPU learning and Scientific community code base with CUDA a lot. There would be too much inertia involved to get people move away from these (specially scientist community who have now been working on CUDA for a while).This is the main reason why any other hardware than NVIDIA GPUs with similar high bandwidth such as ATI GPUs, Intel Xeon Phi, FPGAs etc. will not be used. It is just too much effort to develop the software and because there is no large increase in performance from these hardware pieces there is just no incentive to start developing.

Also, the hardware has no major advantages over GPUs other than low power consumption. Power consumption is only relevant if you have very large clusters. However, if you have such cluster, GPUs will not be effective in the first place (and neither will be FPGAs) due to bandwidth limitations for interconnects. Most companies with will use existing CPU clusters for very large neural nets and small GPU clusters will be used for medium sized nets.

GPUs have an advantage over FPGAs and this is speed: If you look a few years ahead there is also NVIDIA's Pascal GPU which will increase processing speed for deep learning dramatically (very high bandwidth due to 3D memory; NVLink; 16bit float math). FPGAs just cannot keep up with this. You may say that you can just power more cheap FPGAs with the same wattage as a single GPU, but bandwidth problems will destroy that idea. Four FPGAs will probably be as fast as a single one due to bandwidth constraints.

So IMO, FPGA is just a fad in the field of Machine Learning. Although the advantages over GPUs might be beneficial in low Power IOT devices. But it will need a great breakthrough in FPGA technology to make it good enough to take on GPUs.