NVIDIA® cuDNN – GPU Accelerated Machine Learning

NVIDIA cuDNN is a GPU-accelerated library of primitives for deep neural networks. It emphasizes performance, ease-of-use, and low memory overhead. NVIDIA cuDNN is designed to be integrated into higher-level machine learning frameworks, such as UC Berkeley’s popular Caffe software. The simple, drop-in design allows developers to focus on designing and implementing neural net models rather than tuning for performance, while still achieving the high performance modern parallel computing hardware affords.

Visit here for more information on accelerating machine learning with GPUs.

Key Features

  • Forward and backward convolution routines designed for convolutional neural nets, tuned for NVIDIA GPUs
  • Always optimized for latest NVIDIA GPU architectures
  • Arbitrary dimension ordering, striding, and subregions for 4d tensors means easy integration into any neural net implementation
  • Forward and backward paths for many other common layer types (ReLU, Sigmoid, Tanh, pooling, softmax)
  • Context-based API allows for easy multithreading

Get Started with cuDNN Today

cuDNN provides tuned implementations of routines frequently used in deep neural network applications, including:

  • Convolution forward and backward, including cross-correlation
  • Pooling forward and backward (Maximum and Average)
  • Softmax forward and backward
  • Neuron activations forward and backward (Rectified Linear, Sigmoid, Hyperbolic Tangent)
  • Tensor transformation functions

Included convolution routines provide performance competitive with the fastest GEMM (matrix multiply) based approaches while using less memory. cuDNN features customizable data layouts, flexible dimension ordering, striding, and subregions for the 4D tensors used as inputs and outputs to all of its routines. This flexibility avoids transposition steps to or from other internal representations. cuDNN also offers a context-based API that allows for easy multithreading and optional interoperability with CUDA streams.

cuDNN is supported as part of UC Berkeley’s Caffe with integrations into other popular machine learning frameworks on the way. Please see Caffe’s documentation for enabling cuDNN with Berkeley’s framework.

cuDNN Performance Acceleration


Baseline Caffe compared to Caffe accelerated by cuDNN on K40

Baseline Caffe compared to Caffe accelerated by cuDNN on TitanZ

Baseline Caffe compared to Caffe accelerated by cuDNN

All comparisons are against a 24-core Intel E5-2679v2 CPU @ 2.4GHz running Caffe with Intel MKL 11.1.3.

Availability & Support

cuDNN is freely available to CUDA Registered Developers for both commercial and non-commercial use. As a registered developer you can download the latest version of cuDNN, access the support forum and file bug reports.

Register Download

For questions or to provide feedback, please contact cuDNN@nvidia.com.