In main.cu, apart from the input layer l_input, there are 3 layers defined: c1, s1, f.
|
// Define layers of CNN |
|
static Layer l_input = Layer(0, 0, 28*28); |
|
static Layer l_c1 = Layer(5*5, 6, 24*24*6); |
|
static Layer l_s1 = Layer(4*4, 1, 6*6*6); |
|
static Layer l_f = Layer(6*6*6, 10, 10); |
In generalized definition of CNN, it consists of convolutional layer, pooling layer and fully connected layer. As c and f corresponds to convolutional and fully-connected, there should be a pooling layer (perhaps named p ).
According to LeNet, pooling layer is also named sub-sampling layer, which may explain the naming of s.
As the Pooling Layer chapter says:
Introduces zero parameters since it computes a fixed function of the input.
however, according to the implementation of fp_preact_s1, it calculated a weighted sum to per 4x4 slice:
|
__global__ void fp_preact_s1(float input[6][24][24], float preact[6][6][6], float weight[1][4][4]) |
|
{ |
|
const int pos = blockIdx.x * blockDim.x + threadIdx.x; |
|
const int size = blockDim.x * gridDim.x; |
|
|
|
const int N = 4*4*6*6*6; |
|
|
|
for (int n = N * pos / size; n < N * (pos+1) / size; ++n) { |
|
int idx = n; |
|
const int i1 = ((idx /= 1 ) % 4); |
|
const int i2 = ((idx /= 4 ) % 4); |
|
const int i3 = ((idx /= 4 ) % 6); |
|
const int i4 = ((idx /= 6 ) % 6); |
|
const int i5 = ((idx /= 6 ) % 6); |
|
|
|
atomicAdd(&preact[i3][i4][i5], weight[0][i1][i2] * input[i3][i4 * 4 + i1][i5 * 4 + i2]); |
|
} |
|
} |
there is still weight parameters to train, which seems violating the 'zero parameters' trait of pooling layer.
So is the s layer a pooling (sub-sampling) layer? If it is, how can it have parameters to train; or if not, is the whole network still CNN?
In
main.cu, apart from the input layerl_input, there are 3 layers defined:c1,s1,f.CUDA-CNN/main.cu
Lines 13 to 17 in 95c9f33
In generalized definition of CNN, it consists of convolutional layer, pooling layer and fully connected layer. As
candfcorresponds to convolutional and fully-connected, there should be a pooling layer (perhaps namedp).According to LeNet, pooling layer is also named sub-sampling layer, which may explain the naming of
s.As the Pooling Layer chapter says:
however, according to the implementation of
fp_preact_s1, it calculated a weighted sum to per 4x4 slice:CUDA-CNN/layer.cu
Lines 151 to 168 in 95c9f33
there is still
weightparameters to train, which seems violating the 'zero parameters' trait of pooling layer.So is the
slayer a pooling (sub-sampling) layer? If it is, how can it have parameters to train; or if not, is the whole network still CNN?