(5/5)

In the present coursework you are required to perform a number of tasks and answer a number of questions that are presented in the following. All your

ndings (tasks results and answers to the questions) must be presented in the form of a scienti c report. Such a report must be written in LATEX, and it must contain an abstract, introduction, methods, results and conclusions.

The description of your tasks is as follows:

Download the le data.dat from the AM41AN web page in BlackBoard. This

le contains one thousand registers, each form by a pair of coordinates (xn, tn). The rst coordinate xn is a feature, or independent variable, the second tn is the target, or dependent variable. Some level of noise has been added to the targets. A plot of t against x is presented in gure 1.

1. Your rst task is to estimate the conditional expectation:

⟨t|x⟩ = ∫ dt P(t|x) t,

where (t x) is the conditional probability of t conditioned to x, by train- ing the network presented in gure 2 through error back-propagation. This proposed network consists of three layers (input, hidden and output). The

units in the input layer correspond to z(0) = 1 and z(0) = x. The M + 10

units in the hidden layer are z(1) = 1, and z(1) = tanh1

w(1) z(0) + w(1) z(0)

0 k k,0 0 k,1 1

for all 0 < k ≤ M. The output unit is linear, i.e. z(2) = w(2)z(1) +

∑M w(2)z(1). You are required to consider the cases with M = 7, 10

and 20. You may use the following error function:

1∑000 1 ( )2

2. Make a comment in your report on how this problem could have been solved using radial basis functions.

Figure 1: Plot of the points in the data set.

Figure 2: Architecture of the proposed network. The units in the input layer correspond to z(0) = 1 and z(0) = x. The M + 1 units in the hidden layer are

0

z(1) = 1, and z(1) = tanh

1

w(1) z(0) + w(1) z(0)

for all 0 < k ≤ M. The output

unit is linear, i.e. z(2) = w(2)z(1) + ∑M

w(2)z(1).

3. Once you have solved the regression task set above, consider the following regularized error function:

E˜ (w) = E0

(w) + ν wTw. (2)

2

Explain what are the expected e ects of the new added term ν wTw, and how these e ects depend on the value of ν. Use the spectrum of the Hessian matrix (in the following notation many subscripts and superscripts have being dropped, see Eqs. (8) to (10) bellow)

∂2E0

[H]j,k = ∂w ∂w

j k

to relate the weights minimising the cost with regularization (2), w˜ ν, to the cost without regularization (1), w⋆, i.e.

w˜ = (∑ λk u uT) w⋆, (3)

where Huk = λkuk. Make a plot of the error function E0(w˜ ν) against

ν ∈ (0, 1) for the networks with M = 7, 10 and 20.

Hints

As mentioned in the Instructions, you must present your work in the form of a report. A template for the report is provided in the le cw_report_template.pdf In the Method section you must demonstrate that the error back-propagation

(5/5)

DescriptionIn this final assignment, the students will demonstrate their ability to apply two ma

Path finding involves finding a path from A to B. Typically we want the path to have certain properties,such as being the shortest or to avoid going t

Develop a program to emulate a purchase transaction at a retail store. Thisprogram will have two classes, a LineItem class and a Transaction class. Th

1 Project 1 Introduction - the SeaPort Project series For this set of projects for the course, we wish to simulate some of the aspects of a number of

1 Project 2 Introduction - the SeaPort Project series For this set of projects for the course, we wish to simulate some of the aspects of a number of