this task, you must apply a suitable feature selection algorithm before deploying each clustering algorithm time is taken, Sum of Squares Errors (SSE), Cluster Silhouette Measure (CSM)

INSTRUCTIONS TO CANDIDATES

ANSWER ALL QUESTIONS

Task 1

For each activity in this task, you must apply a suitable feature selection algorithm before deploying each clustering algorithm.  Your clustering results should include the following measures:

Time is taken, Sum of Squares Errors (SSE), Cluster Silhouette Measure (CSM)

Submit Python code used for parts a) to c) below. You only need to submit the code for one of the 4 datasets.

1. Run the K means algorithm on each of the four datasets. Obtain the best value of K using either SSE and/or CSM. Tabulate your results in a 4 by 3 table, with each row corresponding to a dataset and each column corresponding to one of the three measures mentioned above. Display the CSM plot for the best value of the K parameter for each dataset.

1. Repeat the same activity for DBSCAN and tabulate your results once again, just as you did for part a). Display the CSM plot and the 4 by 3 table for each dataset.

1. Finally, use the Agglomerative algorithm and document your results as you did for parts a) and b). Display the CSM plot and the 4 by 3 table for each dataset.        

 

Task 2

 

1. For each dataset identify which clustering algorithm performed best. Justify your answer.

In the event that no single algorithm performs best on all three performance measures you will need to carefully consider how you will rate each of the measures and then decide how you will produce an overall measure that will enable you to rank the algorithms.                 

1. For each winner algorithm and for each dataset explain why it produced the best value for the CSM measure. This explanation must refer directly to the conceptual design details of the algorithm. There is no need to produce any further experimental evidence for this part of the question
2. Based on what you produced in a) above, which clustering algorithm would you consider to be the overall winner (i.e. after taking into consideration performance across all four datasets). Justify your answer. 

 

This task requires you to do some further research on your own. The t-sne algorithm (https://lvdmaaten.github.io/tsne/) was designed to visualize high dimensional data after reducing dimensionality.

 

1. After gaining an understanding of how it works identify one important potential advantage of t-sne over Principal Components Analysis (PCA). You may use one or more sources from the machine learning literature to support your answer.                                           
2. Select the Sales Transactions dataset that you experimented with:
   1. apply t-sne to reduce dimensionality to two components and then visualize the data using a suitable plot. Submit the Python code and your plot.                                            
   2. is the potential advantage of t-sne over PCA that you mentioned in part a) present itself in this dataset? Justify your answer with suitable experimental evidence.                                     
   3. does the 2D visualization give insights into the structure of the data? Explain your answer.
   4. if so, does it inform the choice of which clustering algorithm to use? On the other hand, if it does not narrow down the choice of clustering algorithm then explain why the visual is insufficient on its own draw a definite conclusion.               

Attachments:

Related Questions

. Introgramming & Unix Fall 2018, CRN 44882, Oakland University Homework Assignment 6 - Using Arrays and Functions in C

DescriptionIn this final assignment, the students will demonstrate their ability to apply two ma

. The standard path finding involves finding the (shortest) path from an origin to a destination, typically on a map. This is an

Path finding involves finding a path from A to B. Typically we want the path to have certain properties,such as being the shortest or to avoid going t

. Develop a program to emulate a purchase transaction at a retail store. This program will have two classes, a LineItem class and a Transaction class. The LineItem class will represent an individual

Develop a program to emulate a purchase transaction at a retail store. Thisprogram will have two classes, a LineItem class and a Transaction class. Th

. SeaPort Project series For this set of projects for the course, we wish to simulate some of the aspects of a number of Sea Ports. Here are the classes and their instance variables we wish to define:

1 Project 1 Introduction - the SeaPort Project series For this set of projects for the course, we wish to simulate some of the aspects of a number of

. Project 2 Introduction - the SeaPort Project series For this set of projects for the course, we wish to simulate some of the aspects of a number of Sea Ports. Here are the classes and their instance variables we wish to define:

1 Project 2 Introduction - the SeaPort Project series For this set of projects for the course, we wish to simulate some of the aspects of a number of