Unsupervised Learning

Sepal Length vs Sepal Width

plot(iris$Sepal.Length, iris$Sepal.Width)

Petal Length vs Petal Width

Notice that even without any information here on the species (using coloring, different symbols or other graphical tools), we can see that there are probably at least 2 “clusters” or “groupings” of the points here in this iris dataset.

plot(iris$Petal.Length, iris$Petal.Width)

Petal Length vs Petal Width BY Species (color)

So, let’s now add color so we can see how the different species of flower cluster together (or not).

plot(iris$Petal.Length, iris$Petal.Width,
     col = iris$Species)

Dendogram Cluster Analysis - Complete Linkage

plot(clusters)

Compare Clusters (Complete Linkage) to Original Species

If we want to get 3 clusters from this data, we will use the cutree() function. Then we can see how these resulting clusters line up with the original species.

clusterCut <- cutree(clusters, 3)
table(clusterCut, iris$Species)

Cluster Analysis - Average Linkage

Notice that “cluster 2” is not “clean” - it has both versicolor and virginica. Clusters 1 and 3 are pretty good.

Let’s try another linkage method; set method = “average”.

clusters <- hclust(dist(iris[, 3:4]), 
                   method = "average")

Dendogram Cluster Analysis - Average Linkage

plot(clusters)

Compare Clusters (Average Linkage) to Original Species

Notice that this “average linkage” method results in clusters that more closely align with the original species. Of course, we don’t usually know what the “correct” answer is (the true cluster designations).

clusterCut <- cutree(clusters, 3)
table(clusterCut, iris$Species)

Heatmap with updated colors

palette <- colorRampPalette(c("green","blue"))(256)
heatmap(irismatrix, Colv=F, scale='none', col=palette)

Run PCA on the iris dataset

1. Run the PCA
2. Get the summary which prints the variance accounted for by each computed component
3. Next look at the loadings which are the coefficients (or weights) of how much each original variable contributes to that component (e.g. the linear combination equation)
4. Next plot the “Scree Plot”, which plots the “eigenvalues” of each components - these are used to compute the amount of variance explained by each component. A “naive” approach to “reducing the dimensionality of the data” is to remove components with eigenvalues > 1.

fit <- princomp(iris[1:4], cor=TRUE)
summary(fit) # print variance accounted for 

## Importance of components:
##                           Comp.1    Comp.2     Comp.3      Comp.4
## Standard deviation     1.7083611 0.9560494 0.38308860 0.143926497
## Proportion of Variance 0.7296245 0.2285076 0.03668922 0.005178709
## Cumulative Proportion  0.7296245 0.9581321 0.99482129 1.000000000

loadings(fit) # pc loadings 

## 
## Loadings:
##              Comp.1 Comp.2 Comp.3 Comp.4
## Sepal.Length  0.521 -0.377  0.720  0.261
## Sepal.Width  -0.269 -0.923 -0.244 -0.124
## Petal.Length  0.580        -0.142 -0.801
## Petal.Width   0.565        -0.634  0.524
## 
##                Comp.1 Comp.2 Comp.3 Comp.4
## SS loadings      1.00   1.00   1.00   1.00
## Proportion Var   0.25   0.25   0.25   0.25
## Cumulative Var   0.25   0.50   0.75   1.00

plot(fit,type="lines") # scree plot 

Loadings Plot

Let’s see how the 4 variables “load” onto the 1st 2 principal components

plot(fit[["loadings"]],
     xlim=c(-1,1),ylim=c(-1,1))
abline(h=0, col="red")
abline(v=0, col="red")
text(fit[["loadings"]],
     c("Sepal.Length","Sepal.Width",
       "Petal.Length","Petal.Width"))

Sample PC Scores

We can also plot the updated “scores” for the first 2 principal components for all 150 samples. These “scores” are the aggregated scores based on the linear combinations of the original variables (e.g. using the weights or loadings listed above).

head(fit$scores) # top listing of the sample scores for all 4 principal components

biplot(fit) # plot samples on PC1 and PC2