Linear Discriminant Analysis (LDA) is another method that can be used for classifying objects into their respective classes. The features of an object are used to classify them into one of several classes.
For this activity, I again used the pink and red beads as the objects to be classified. The red beads are classified as class 1 and the pink beads as class 2.
In each class, four of the beads were used for the training set and the other four were used for the test set. Two features were used for LDA – the length (diameter for the red beads) of the bead and its “mean” color (the mean of its R, G, and B channels). These were stored in the matrix, x with the length in the first column and color in the second column. The “training set” features vector, x and their corresponding classes are shown below.
The matrix x was split into x1 and x2 and the mean, mu_i for each vector was obtained. The global mean vector, mu was also obtained.
The covariance matrix, c_i is then computed using the following equation:
where n_i is the number of objects in x_i; thus n1=4 and n2=4. The matrices, c1 and c2 are:
Using c1 and c2, C is obtained using the following equation:
C and its inverse, C^-1are:
For this particular data set, the prior probability vector, p is given by:
Using the said variables, it is now possible to compute for the Linear Discriminant function, f_i.
For the “training set” I used, f1 and f2 must be obtained and then compared in order to determine in which class the particular object, k belongs. If the value of f_i is greater then the object belongs in class i. The data I got for the four red and four pink beads are shown below:
The highlighted values are the larger ones (yellow for class 1 and peach for class 2). Now, I tested the remaining four red and four pink beads to determine how well these will be classified using LDA. The data for this is:
Comparing the classification made by LDA to the actual class of the beads, we can see that there is a 100 % classification.
I give myself a grade of 10 in this activity for being able to successfully implement LDA.
___________________
The code that I made for this activity is shown below:
//training set data
x1=[0.988078346 0.217334; 0.99504038 0.2057463; 0.996698007 0.2361515; 0.996963227 0.2858829];
x2=[0.137781962 0.5295532; 0.141030911 0.5210784; 0.149186437 0.5611408; 0.137715657 0.5842001];
x=[0.988078346 0.217334; 0.99504038 0.2057463; 0.996698007 0.2361515; 0.996963227 0.2858829; 0.137781962 0.5295532; 0.141030911 0.5210784; 0.149186437 0.5611408; 0.137715657 0.5842001];
//test set data
//x1t=[1.02322004 0.2186213; 0.143020064 0.5815481; 1.128777732 0.1655321; 0.150910369 0.5429531];
//x2t=[0.98290655 0.2522253; 1.092774072 0.1837175; 0.158203928 0.5429531; 0.140168945 0.5640654];
xt=[1.02322004 0.2186213; 0.143020064 0.5815481; 1.128777732 0.1655321; 0.150910369 0.5429531; 0.98290655 0.2522253; 1.092774072 0.1837175; 0.158203928 0.5429531; 0.140168945 0.5640654];
mu1=[mean(x1(:,1)) mean(x1(:,2))];
mu2=[mean(x2(:,1)) mean(x2(:,2))];
mu=[mean(x(:,1)) mean(x(:,2)); mean(x(:,1)) mean(x(:,2)); mean(x(:,1)) mean(x(:,2)); mean(x(:,1)) mean(x(:,2))];
x01=x1-mu;
x02=x2-mu;
//compute covariance matrices
n1=4;
n2=4;
c1=(x01′*x01)/n1;
c2=(x02′*x02)/n2;
for r=1:2
for s=1:2
C(r,s)=(n1*c1(r,s)+n2*c2(r,s))/(n1+n2);
end
end
p=[0.5; 0.5];
for i=1:8
f1(i)=mu1*(inv(C)*xt(i,:)’) – 0.5*mu1*(inv(C)*mu1′) + log(p(1));
f2(i)=mu2*(inv(C)*xt(i,:)’) – 0.5*mu1*(inv(C)*mu2′) + log(p(2));
end
