Normalization and dropout for stochastic computing-based deep convolutional neural networks

Introduction

Deep Convolutional Neural Network (DCNN) has recently achieved unprecedented success in various applications, such as image recognition, natural language processing, video recognition [3], and speech processing. As DCNN breaks several long-time records in different popular datasets, it is recognized as the dominant approach for almost all pattern detection and classification tasks. With the fast advancement and widespread deployment of Internet of Things (IoTs) and wearable devices, implementing DCNNs in embedded and portable systems is becoming increasingly attractive

DCNN Architecture Overview

A general DCNN is composed of a stack of convolutional layers, pooling layers, and fully-connected layers. A convolutional layer is followed by a pooling layer, which extracts features from raw inputs or the previous feature maps. A fully connected layer aggregates the high level features, and a softmax regression is applied to derive the final output. The basic component of DCNN is the Feature Extraction Block (FEB), which conducts inner product, pooling and activation operations.

HeadShot

Experimental Results

In this section, we present (i) performance evaluation of the SC FEBs, (ii) performance evaluation of the proposed SC-LRN design, and (iii) impact of SC-LRN and dropout on the overall DCNN performance. The FEBs and DCNNs are synthesized in Synopsys Design Compiler with the 45 nm Nangate Library using Verilog.

HeadShot