Deep Learning Notes 1 – NN (Repost)

Resource

• https://www.linkedin.com/pulse/7-steps-becoming-deep-learning-expert-ankit-agarwal
• http://iamtrask.github.io/2015/07/12/basic-python-network/
• https://d396qusza40orc.cloudfront.net/neuralnets/lecture_slides/lec3.pdf
• http://hahack.com/reading/ann2/
• https://class.coursera.org/ml-005/lecture
• https://class.coursera.org/ntumltwo-001/lecture

Introduction

Neural networks are important for deep learning. A nueral network model comprises multiple layers of logistic regression models, as shown in the figure below.

In the figure above, denotes the weight from neuron i to neuron j in layer , the index i refers to a source node and j referes to a target node. denotes the number of neurons in layer .

Forward Propagation

For a given input vector ( as a bias ), which is sent to a neural network from input layer, the information will be sent and transformed layer by layer from input layer to output layer. In output layer, it yields a output variable in case of binary classification and regression (only one neuron in output layer), it yields multiple vectors in case of multiple classification (multiple neurons in output layer).

For each neuron, the process of information…