Week 11 Multilayer Perceptron

This week i learn about multilayer perceptron, where a single layer perceptron only have an input layer and output layer, this multilayer layer perceptron have another layer inbetween input and output layer which is hidden layer.

                                              Figure 1 : Multilayer perceptron

The Multilayer Perceptron

The multilayer perceptron (MLP) is a hierarchical structure of several perceptrons, and overcomes the shortcomings of these single-layer networks.
The multilayer perceptron is an artificial neural network that learns nonlinear function mappings. The multilayer perceptron is capable of learning a rich variety of nonlineardecision surfaces.
Nonlinear functions can be represented by multilayer perceptrons with units that use nonlinear activation functions. Multiple layers of cascaded linear units still produce only linear mappings.

Differentiable Activation Functions

The training algorithm for multilayer networks requires differentiable, continuous nonlinear activation functions. Such a function is the sigmoid, or logistic function:

o = s ( s ) = 1 / ( 1 + e^-s )

where s is the sum: s=S _i=0^d w_i x_i of products from the weights w_i and the inputs x_i.
Sometimes s is called alternatively squashing function as it maps a very large input domain to a small range of outputs.
Another nonlinear function often used in practice is the hyperbolic tangent:

o = tanh( s ) = ( e^s - e^-s ) / (e^s + e^-s)

Sometimes the hyperbolic tangent is preferred as it makes the training a little easier.