cuda-convnet2

cuda-convnet2 is Nervana's fork of Alex Krizhevsky's cuda-convnet2 containing several extensions including: a new python backend called cudanet for integration into Nervana's neon framework. They have also included several new kernels and functions to support things like multiway costs, python interface to GPU memory, support for non-texture kernels, array and scalar max/min comparisons, and local contrast normalization.

This version also features one line pip or cmake based installation and additional checking and fixes. To be able to install the framework users must ensure that they have met all required dependency packages including installing the CUDA toolkit and CUDA SDK. They must also bear in mind that a Kepler-generation GPU with shader model capability 3.5 or greater is required to run this code.

This includes the chips GK110 and GK114, which can be found on the GPUs Tesla K20, Tesla K20x, Tesla K40, GeForce Titan, and GeForce GTX 780, among others. Older GPUs, including GK104-based GPUs such as the Tesla K10 and GeForce 680, won't work. The initial cuda-convnet2 project had three major new features relative to cuda-convnet. The first was improved training times on Kepler-generation Nvidia GPUs (Geforce Titan, K20, K40). It also had Multi-GPU training support implementing data parallelism, model parallelism, and the hybrid approach described in one weird trick for parallelizing convolutional neural networks. Finally it bore less-polished code and incomplete (but improving) documentation.

The code also included contributions by Anker Guo of the Tencent BestImage team most notably 50% acceleration of batch-32 convolution kernels and 10% acceleration of batch-128 convolution kernels.