In the present coursework you are required to perform a number of tasks

INSTRUCTIONS TO CANDIDATES

ANSWER ALL QUESTIONS

In the present coursework you are required to perform a number of tasks and answer a number of questions that are presented in the following. All your

 ndings (tasks results and answers to the questions) must be presented in the form of a scienti c report.  Such a report must be written in LATEX, and it must contain an abstract, introduction, methods, results and conclusions.

The description of your tasks is as follows:

Download the le data.dat from the AM41AN web page in BlackBoard. This

 le contains one thousand registers, each form by a pair of coordinates (xn, tn). The  rst coordinate xn  is a feature, or independent variable, the second tn  is the target, or dependent variable. Some level of noise has been added to the targets. A plot of t against x is presented in gure 1.

1. Your rst task is to estimate the conditional expectation:

⟨t|x⟩ = ∫ dt P(t|x) t,

where (t x) is the conditional probability of t conditioned to x, by train- ing the network presented in gure 2 through error back-propagation. This proposed network consists of three layers (input, hidden and output). The

units in the input layer correspond to z(0) = 1 and z(0) = x. The M + 10

units in the hidden layer are z(1) = 1, and z(1) = tanh1

w(1) z(0) + w(1) z(0)

 0 k k,0 0 k,1 1

for all 0  <  k  ≤  M. The output unit is linear, i.e.   z(2) =  w(2)z(1) +

∑M     w(2)z(1).  You are required to consider the cases with M = 7, 10

and 20. You may use the following error function:

1∑000 1 ( )2

2. Make a comment in your report on how this problem could have been solved using radial basis functions.

Figure 1: Plot of the points in the data set.

Figure 2: Architecture of the proposed network. The units in the input layer correspond to z(0) = 1 and z(0) = x. The M + 1 units in the hidden layer are

 0

z(1) = 1, and z(1) = tanh

 1

w(1) z(0) + w(1) z(0)

 for all 0 < k ≤ M. The output

 unit is linear, i.e.  z(2) = w(2)z(1) + ∑M

 w(2)z(1).

 3. Once you have solved the regression task set above, consider the following regularized error function:

 E˜ (w) = E0

 (w) + ν wTw. (2)

2

 Explain what are the expected e ects of the new added term ν wTw, and how these e ects depend on the value of ν. Use the spectrum of the Hessian matrix (in the following notation many subscripts and superscripts have being dropped, see Eqs. (8) to (10) bellow)

∂2E0

[H]j,k  =  ∂w  ∂w

j k

to  relate  the  weights  minimising  the  cost  with  regularization  (2),  w˜ ν,  to the cost without regularization (1), w⋆, i.e.

w˜    = (∑     λk       u  uT) w⋆, (3)

where  Huk  =  λkuk.  Make  a  plot  of  the  error  function  E0(w˜ ν)  against

ν ∈ (0, 1) for the networks with M = 7, 10 and 20.

Hints

As mentioned in the Instructions, you must present your work in the form of a report. A template for the report is provided in the le cw_report_template.pdf In the Method section you must demonstrate that the error back-propagation

 

Attachments:

Related Questions

. Introgramming & Unix Fall 2018, CRN 44882, Oakland University Homework Assignment 6 - Using Arrays and Functions in C

DescriptionIn this final assignment, the students will demonstrate their ability to apply two ma

. The standard path finding involves finding the (shortest) path from an origin to a destination, typically on a map. This is an

Path finding involves finding a path from A to B. Typically we want the path to have certain properties,such as being the shortest or to avoid going t

. Develop a program to emulate a purchase transaction at a retail store. This program will have two classes, a LineItem class and a Transaction class. The LineItem class will represent an individual

Develop a program to emulate a purchase transaction at a retail store. Thisprogram will have two classes, a LineItem class and a Transaction class. Th

. SeaPort Project series For this set of projects for the course, we wish to simulate some of the aspects of a number of Sea Ports. Here are the classes and their instance variables we wish to define:

1 Project 1 Introduction - the SeaPort Project series For this set of projects for the course, we wish to simulate some of the aspects of a number of

. Project 2 Introduction - the SeaPort Project series For this set of projects for the course, we wish to simulate some of the aspects of a number of Sea Ports. Here are the classes and their instance variables we wish to define:

1 Project 2 Introduction - the SeaPort Project series For this set of projects for the course, we wish to simulate some of the aspects of a number of