Attached is a (new) implementation of a "spatially-sparse" convolutional network (http://arxiv.org/abs/1409.6070).

Step 1) Look at the README file.
Step 2) Define a network architecture in terms of LeNet style layers. The input layer can be much larger than 32x32, and it will still run relatively efficiently (sparsity!). Be sure to include lots of Network-In-Network layers. [See runCifar10Kaggle.cu]
Step 3) Specify some kind of affine distortion to apply to the training data. [See OpenCVPicture_AffineTransform.h]
Step 4) Hope it all compiles.
Step 5) Run it for a long time; hope it does not crash.

Requirements: CUDA (sm_20), OpenCV, Boost and a reasonably fast GPU

• sparseConvNetKaggleCifar10Edition.zip (204.05 KB)

I compile the code, then meet some error. The way I solve the errors is following:

1.If you meet 

/usr/lib/gcc/x86_64-linux-gnu/4.6/include/xmmintrin.h(1015): error: identifier "__builtin_ia32_vec_set_v4hi" is undefined/usr/lib/gcc/x86_64-linux-gnu/4.6/include/xmmintrin.h(1116): error: identifier "__builtin_ia32_pshufw" is undefined/usr/lib/gcc/x86_64-linux-gnu/4.6/include/emmintrin.h(1312): error: identifier "__builtin_ia32_vec_ext_v8hi" is undefined/usr/lib/gcc/x86_64-linux-gnu/4.6/include/emmintrin.h(1318): error: identifier "__builtin_ia32_vec_set_v8hi" is undefined4 errors detected in the compilation of "/tmp/tmpxft_00001d10_00000000-6_runCifar100.cpp1.ii".

You can use cuda >=5.5, or do not use the -O2 in compiling

2.If you meet  

/usr/local/include/boost/assert.hpp:102:47: error: ‘noinline’ was not declared in this scope

You can add -DBOOST_NOINLINE='__attribute__ ((noinline))'

3.If you meet 

/usr/bin/ld: /tmp/tmpxft_00004332_00000000-16_runCifar10Kaggle.o: undefined reference to symbol 'boost::system::system_category()'
/usr/bin/ld: note: 'boost::system::system_category()' is defined in DSO /usr/local/lib/libboost_system.so.1.55.0 so try adding it to the linker command line
/usr/local/lib/libboost_system.so.1.55.0: could not read symbols: Invalid operation

You can add -lboost_system

So the you can compile as:

/usr/local/cuda-5.5/bin/nvcc -DBOOST_NOINLINE='__attribute__ ((noinline))' runCifar10Kaggle.cu readImageToMat.o -lopencv_core -lopencv_highgui -lopencv_imgproc -lrt -lcublas -lboost_thread -lboost_system -ljpeg -arch sm_20 -O2

Thank you jiki for your suggestions and fixes.

Revised program attached.

• sparseConvNetKaggleCifar10Edition.zip (58.50 KB)

Attached is the specification for a network architecture used get a test accuracy of 0.9553.

Architecture: input=(3x126x126) -
 320C2 - 320C2 - MP2 -
 640C2 - 10% dropout - 640C2 - 10% dropout - MP2 -
 960C2 - 20% dropout - 960C2 - 20% dropout - MP2 -
 1280C2 - 30% dropout - 1280C2 - 30% dropout - MP2 -
 1600C2 - 40% dropout - 1600C2 - 40% dropout - MP2 -
 1920C2 - 50% dropout - 1920C1 - 50% dropout - 10C1 - Softmax output
 (inspired by
 Multi-column deep neural networks for image classification; Ciresan, Meier and Schmidhuber;
 Network In Network; Lin, Chen and Yan;
 Very Deep Convolutional Networks for Large-Scale Image Recognition; Simonyan and Zisserman)"""

It is a deep convolutional network trained using SparseConvNet (see http://arxiv.org/abs/1409.6070) taking advantage of
- spatial-sparsity in the 126x126 input layer,
- batchwise dropout and a form of Nesterov's accelerated gradient descent,
- (very) leaky rectified linear units, and
- spatial and color training data augmentation.

Train for 500+ iterations, with affine training data augmentation. Then run for 10 more iterations with the affine training data augmentation removed.

• Cifar10-Kaggle-Competition.cu (2.72 KB)