Show that if the λk are positive, Using the Optdigits dataset from the UCI repository, implement PCA

INSTRUCTIONS TO CANDIDATES

ANSWER ALL QUESTIONS

1. Fisher’s linear discriminant is

wˆ T Sbwˆ

wˆ    = arg max J(wˆ ) = arg max wˆ T S   wˆ ,

wˆ wˆ w

where Sb = (m1 − m2)(m1 − m2)T and Sw = Σj Σα(xα − mj)(xα − mj)T .

 (a) By writing ∂J

 = 0, show that

 S−w1Sbwˆ  = J(wˆ )wˆ .

 (b) Explain why wˆ ∗  is the eigenvector for which J(wˆ ) is the maximum eigenvalue of S−w1Sb.

 

(c) Explain why Sbwˆ

 

is always in the direction of m1 − m2 and thus show that

wˆ ∗   =   const. • S−w1  (m1 − m2).

 2. Another way to optimize Fisher’s linear discriminant (suggested by Barry Fridling):

(a) Show that for any two real vectors x and y

(xT y)2 ≤ (xT x)(yT y), (Cauchy-Schwarz).

(b) Show that if the λk are positive,

Using the Optdigits dataset from the UCI repository, implement PCA. Reconstruct the digit images  and  calculate  the  reconstruction  error  E(n) =      j  ||xˆj − x||2  for  various  values  of  n,  the number of eigenvectors. Plot E(n) versus n.

 

Attachments:

Related Questions

. Introgramming & Unix Fall 2018, CRN 44882, Oakland University Homework Assignment 6 - Using Arrays and Functions in C

DescriptionIn this final assignment, the students will demonstrate their ability to apply two ma

. The standard path finding involves finding the (shortest) path from an origin to a destination, typically on a map. This is an

Path finding involves finding a path from A to B. Typically we want the path to have certain properties,such as being the shortest or to avoid going t

. Develop a program to emulate a purchase transaction at a retail store. This program will have two classes, a LineItem class and a Transaction class. The LineItem class will represent an individual

Develop a program to emulate a purchase transaction at a retail store. Thisprogram will have two classes, a LineItem class and a Transaction class. Th

. SeaPort Project series For this set of projects for the course, we wish to simulate some of the aspects of a number of Sea Ports. Here are the classes and their instance variables we wish to define:

1 Project 1 Introduction - the SeaPort Project series For this set of projects for the course, we wish to simulate some of the aspects of a number of

. Project 2 Introduction - the SeaPort Project series For this set of projects for the course, we wish to simulate some of the aspects of a number of Sea Ports. Here are the classes and their instance variables we wish to define:

1 Project 2 Introduction - the SeaPort Project series For this set of projects for the course, we wish to simulate some of the aspects of a number of