VI. Feed-forward neural networks for prediction tasks

Given by Sven Laur
Brief summary: Neural networks as a toolbox for approximating complex functions. Generalised linear models and the conceptual design of a feed-forward network. Hidden layer as an adaptive and non-linear map to higher feature space. Sigmoid functions and radial-based functions as standard ways to build non-linear mapping. Backpropagation algorithm as an efficient gradient decent procedure. Higher-order methods for minimising the training error. Computer vision and invariance under shifts and rotations. Training methods for forcing this type of invariance.

Slides: PDF slides Handwritten slides

Literature:
Bishop: Pattern Recognition and Machine Learning pages 225 - 272

Complementary exercises:
Bishop: Pattern Recognition and Machine Learning pages 284 - 290

Use neural networks for the classification and prediction for various datasets listed below and compare the results obtained in the earlier exercise sessions
Iris dataset
Computer Hardware Data Set
Housing Data Set
Datasets for testing linear regression models

Build a translation invariant neural network for distinguishing numbers in Semeion Handwritten Digit Data Set
First, use random small translations to increase the data set.
Second, use tangent propagation method.
Try two-class versus multi-class classification tasks.

Free implementations:
Nnet package in R for feed-forward neural networks
Neuralnet package in R for feed-forward neural networks
A more flexible neural network package in R
PYBrain: A Python implementation of feedforwad neural networks
Shark machine-learning library for C++