As of Oct 11, 2008, all activities, except for Activities 12 and 14 have been blogged. Activities 12 and 14 are currently being done and are expected to be blogged on or before 12 midnight tomorrow, Oct 12.
Update: All activities have been completed except for Activitiy 12.
Neural networks are models of neurons in the brains; they can be trained to “learn” a particular type of activity. It has many applications; for instance, this can be used as a classification tool in image processing. Instead of using rules, the neural network “learns the rules of the mapping” (M. Soriano, 2008).
For this activity, I again made use of the red and pink beads.
The first input for the neural network are the features of the training set, where x is equal to:
x=[0.988078346, 0.217334;
0.99504038, 0.2057463;
0.996698007, 0.2361515;
0.996963227, 0.2858829;
0.137781962, 0.5295532;
0.141030911, 0.5210784;
0.149186437, 0.5611408;
0.137715657, 0.5842001]‘;.
This was obtained from the data from the LDA classification employed in Activity 19:
Using the and.sce code by Jeric Tugaff, the output of the neural network is:
| 0.022191 |
| 0.021233 |
| 0.022151 |
| 0.024055 |
| 0.977368 |
| 0.976556 |
| 0.977627 |
| 0.97985 |
The classification is as follows:
| Class | Output |
| 1 | 0.022191 |
| 1 | 0.021233 |
| 1 | 0.022151 |
| 1 | 0.024055 |
| 2 | 0.977368 |
| 2 | 0.976556 |
| 2 | 0.977627 |
| 2 | 0.97985 |
The beads under class 1have outputs that are approximately 0.022407575 (mean of the 4 class 1 beads in the training set) while those under class 2 have outputs at about 0.977850025 (mean of the 4 class 2 beads in the training set). Actually, the outputs are expected to be either 0 or 1, so we round the answer to the nearest integer; thus, the output should be 0 for a class 1 bead and 1 for a class 2 bead
We now use the following data for the test set:
The output of the test set is:
| 0.023479 |
| 0.979589 |
| 0.015971 |
| 0.977019 |
| 0.029123 |
| 0.017914 |
| 0.976193 |
| 0.97911 |
To classify these, we round to the nearest integer. We get the following results:
| Output | Nearest Integer | Class |
| 0.0234794 | 0 | 1 |
| 0.9795894 | 1 | 2 |
| 0.0159705 | 0 | 1 |
| 0.9770187 | 1 | 2 |
| 0.0291225 | 0 | 1 |
| 0.0179141 | 0 | 1 |
| 0.9761925 | 1 | 2 |
| 0.9791101 | 1 | 2 |
The classification using neural networks agrees with that using LDA.
The activity was successfully done. Because of this, I give myself a grade of 10 for this activity.
Thanks to Jeric Tugaff for giving a short lecture on neural networks and for providing the and.sce code.
Linear Discriminant Analysis (LDA) is another method that can be used for classifying objects into their respective classes. The features of an object are used to classify them into one of several classes.
For this activity, I again used the pink and red beads as the objects to be classified. The red beads are classified as class 1 and the pink beads as class 2.
In each class, four of the beads were used for the training set and the other four were used for the test set. Two features were used for LDA – the length (diameter for the red beads) of the bead and its “mean” color (the mean of its R, G, and B channels). These were stored in the matrix, x with the length in the first column and color in the second column. The “training set” features vector, x and their corresponding classes are shown below.
The matrix x was split into x1 and x2 and the mean, mu_i for each vector was obtained. The global mean vector, mu was also obtained.
The covariance matrix, c_i is then computed using the following equation:
where n_i is the number of objects in x_i; thus n1=4 and n2=4. The matrices, c1 and c2 are:
Using c1 and c2, C is obtained using the following equation:
C and its inverse, C^-1are:
For this particular data set, the prior probability vector, p is given by:
Using the said variables, it is now possible to compute for the Linear Discriminant function, f_i.
For the “training set” I used, f1 and f2 must be obtained and then compared in order to determine in which class the particular object, k belongs. If the value of f_i is greater then the object belongs in class i. The data I got for the four red and four pink beads are shown below:
The highlighted values are the larger ones (yellow for class 1 and peach for class 2). Now, I tested the remaining four red and four pink beads to determine how well these will be classified using LDA. The data for this is:
Comparing the classification made by LDA to the actual class of the beads, we can see that there is a 100 % classification.
I give myself a grade of 10 in this activity for being able to successfully implement LDA.
___________________
The code that I made for this activity is shown below:
//training set data
x1=[0.988078346 0.217334; 0.99504038 0.2057463; 0.996698007 0.2361515; 0.996963227 0.2858829];
x2=[0.137781962 0.5295532; 0.141030911 0.5210784; 0.149186437 0.5611408; 0.137715657 0.5842001];
x=[0.988078346 0.217334; 0.99504038 0.2057463; 0.996698007 0.2361515; 0.996963227 0.2858829; 0.137781962 0.5295532; 0.141030911 0.5210784; 0.149186437 0.5611408; 0.137715657 0.5842001];
//test set data
//x1t=[1.02322004 0.2186213; 0.143020064 0.5815481; 1.128777732 0.1655321; 0.150910369 0.5429531];
//x2t=[0.98290655 0.2522253; 1.092774072 0.1837175; 0.158203928 0.5429531; 0.140168945 0.5640654];
xt=[1.02322004 0.2186213; 0.143020064 0.5815481; 1.128777732 0.1655321; 0.150910369 0.5429531; 0.98290655 0.2522253; 1.092774072 0.1837175; 0.158203928 0.5429531; 0.140168945 0.5640654];
mu1=[mean(x1(:,1)) mean(x1(:,2))];
mu2=[mean(x2(:,1)) mean(x2(:,2))];
mu=[mean(x(:,1)) mean(x(:,2)); mean(x(:,1)) mean(x(:,2)); mean(x(:,1)) mean(x(:,2)); mean(x(:,1)) mean(x(:,2))];
x01=x1-mu;
x02=x2-mu;
//compute covariance matrices
n1=4;
n2=4;
c1=(x01′*x01)/n1;
c2=(x02′*x02)/n2;
for r=1:2
for s=1:2
C(r,s)=(n1*c1(r,s)+n2*c2(r,s))/(n1+n2);
end
end
p=[0.5; 0.5];
for i=1:8
f1(i)=mu1*(inv(C)*xt(i,:)’) – 0.5*mu1*(inv(C)*mu1′) + log(p(1));
f2(i)=mu2*(inv(C)*xt(i,:)’) – 0.5*mu1*(inv(C)*mu2′) + log(p(2));
end
In order to determine if an object is a member of a particular set of objects using “machine vision” (instead of human vision), we use pattern recognition. Using objects that belong to the particular set we which to recognize we build a training set; we will use this to extract features that are characteristic of the objects in the training set.
I took a picture of three types of beads – large red spherical beads, medium-sized blue spherical beads, and small cylindrical pink beads.
In order to classify them, I built a training set for each class (type of bead) by getting three features – the area in pixels, the length of the longest portion of the bead (in the case of spherical beads, this was the diameter), and the “mean color” of the bead. All these features are each expressed using a single number. The area was obtained by binarizing the image, using the follow command to get the outline of the bead, and implementing Green’s theorem to get the area in terms of number of pixels. The length of the longest portion of the bead was obtained by getting the difference between the coordinates of the two ends of the longest portion of the bead. The “mean color” was obtained by averaging the color values of the three color channels for each bead.
Since the area and the length features had large values, these were normalized so that no feature would have a greater effect in the classification (this was not done anymore for the color because the values were already fractional). The euclidean distance,
from the origin (0,0,0) was then obtained. The data for the training set is shown below (please click on picture if it is not shown properly).
After this, two beads from each class (those that were not included in the training set) were subjected to pattern recognition.
They were then placed into the class whose value of the euclidean distance is closest to theirs. The data for this is:
All of the beads were properly classified into their respective classes. I think that the 100% correct classification was achieved because the features that were taken were truly different for each class.
I give myself a grade of 10 for this activity for being able to classify the beads correctly through pattern recognition.
For this activity, our group (Beth, Aiyin, April, and I) captured a video of a free-falling object and determined kinematic variables from the video. Against a dark blue background, a crumpled piece of white paper was dropped from a height, y0 of 50.0 cm and caught on video until the paper hit the ground. My goal in this activity was to determine the distance travelled by the paper (the crumpled paper only) for a particular time, t and compare it with the theoretical value.
The first step was to convert the wmv file into an avi file using Stoik Video Converter 2.0. The video was then cropped to the significant number of frames, with a frame rate of 30 frames per second; this corresponds to 0.0333333 seconds between 2 frames. The frames were then converted into separate images via VirtualDub 1.6.19. For our video, there were 9 frames, and these were labeled 1.png to 9.png. A sample frame is shown below. At the center of the blue background, 5 strips were pasted to indicate every 10th cm (10, 20, and so on). This was done so that the actual distance in centimeters could be determined by getting the ratio of pixels/cm from the picture. The topmost part of each white strip was used for measurement. 
The images were binarized.
There wasn’t much difficulty in this part since the region of interest (ROI) was distinguishable from the background. After binarizing the image, the y-coordinate of the crumpled piece of paper was taken. Since the motion of the paper was too fast for the 30 frames/sec sampling rate of the videocam, we have an “extended” object instead of the crumpled paper, so the centroid of the “extended” object was taken instead. This was done using the the locate command.
The distance traveled by the paper, in pixels is dy, the difference between the y-coordinates of the crumpled paper in the first and ninth frames. (Values are non-integral because these are already from the centroid)
dy (in pixels) = (206.66667 – 77.604167) pixels = 129.062503 pixels
The number of pixels/cm was also measured; it’s equal to 2.9166666. Dividing the distance travelled in pixels, dy by the number of pixels/cm, we have the distance travelled by the piece of paper in cm.
dy (in cm) = dy (in pixels) / (number of pixels/cm) = 129.062503 pixels / 2.9166666 pixels/cm
dy (in cm) = 44.25 cm
This is a reasonable answer since the video obtained was the falling of the paper from the 50 cm mark to a point before it hits the ground. Now, we get the theoretical value for this distance.
The equation for the motion of an object undergoing projectile motion along the y-axis is:
y = y0 + v0*t + (1/2)*g*t^2
where t is the time, y is the distance of the object from the origin at a time t, y0 is the initial distance, v0 is the initial velocity, and g is the acceleration due to gravity. Since this is a free fall, the initial velocity should be equal to zero, thus we have:
y = y0 + (1/2)*g*t^2
Since we are getting the distance travelled by the paper, we compute for y-y0 instead of y.
y-y0 = (1/2)*g*t^2
Using t= (0.0333333 sec/frame)*(9 frames) = 0.3 seconds and g=9.8 m/s^2 = 98 cm/s^2, the distance travelled by the paper (theoretical), y-y0 is:
y-y0 = (1/2)*(98 cm/s^2)*(0.3 sec)^2
y-y0 = 44.10 cm.
The percent error is:
% error = |theoretical – actual| / theoretical * 100 = | (44.10 – 44.25) cm| / 44.10 cm * 100
% error = 0.3401
This is a very small error. I give myself a grade of 10 for being able to successfully investigate this physical phenomenon (free fall of an object) using basic video/image processing techniques.
Thanks to my 3 groupmates in this activity – Aiyin Laganapan, Elizabeth Prieto, and April Teodoro – for all the help both in creating the free-fall set-up and in the processing of the video.
Color segmentation is the process of separating a region of interest (ROI) from its background based on its color. First, a subregion must be cropped from the ROI, then its histogram must be obtained. Then, we determine which parts of the image have that particular color by determining the probability that a particular pixel is a member of the ROI. There are two ways to do this: 1) Parametric and 2) Non-Parametric Segmentation.
Parametric Segmentation is carried out by assuming that the normalized chromaticity coordinates (NCC) – r and g (b is dependent on r and g, b = 1 – (r+g)) – follow a Gaussian distribution, where the probabilities for r and g are p(r) and p(g) respectively. The probability, p(r) is given by:
where mu_r is the mean and sigma_r is the standard deviation for the r values in the image. We use a similar expression for p(g), replacing the mean, standard deviation, and NCC with mu_g, sigma_g, and g, respectively. To get the joint probability that a pixel belongs to the ROI, we simply get the product of p(r) and p(g). The probabilities for all pixels is then rendered as an image.
For the following picture (a green tape dispenser), I used parametric segmentation to separate the green ROI from the background. Shown below it is the color segmented image.
Again I used parametric segmentation to separate the blue region from non-blue regions. The result is shown below.
For both regions, the result is satisfactory; it was able to segment the chosen ROI from the background.
The second method is Non-parametric Segmentation. Compared to the first method, this one is not analytical; it simply makes use of a “look-up table” for the probability that a pixel is located within the region of interest. The color histogram of the image is obtained, then using histogram backprojection the pixel value is given by its histogram value in chromaticity space. Using these new pixel values, I obtained the following color-segmented images (note: 1.0 was added as a bias to make the gray values more obvious; this is the reason why the background is white and the ROI is dark):
The images are satisfactory, maybe even better than those obtained from parametric segmentation. For the non-parametric method, even within the ROI itself, it is able to show which parts are more likely to be within the ROI (the differences in grayscale values in the color-segmented image correspond to the areas in the ROI having slightly different shades of blue or green). The color-segmented images can be further improved by morphological operations such as opening or closing in order to select the entire ROI.
I give myself a grade of 10 for being able to perform the two methods of color image segmentation successfully.
Thanks to Julie Dado and Jeric Tugaff for helping me with the color histogram code in parametric segmentation.
When taking pictures, it sometimes happens that colors are not rendered correctly. For instance, a white object came become bluish or yellowish in the picture. In order to correct render the colors in the scene we are capturing, white balance is performed. “White balancing” is the process of setting the supposedly white portion of the image as white. In this activity, we performed white balancing using two methods: by 1) the Reference White Algorithm and 2) the Gray World Algorithm.
In the Reference White Algorithm, the red (R), green (G), and blue (B) pixel values of the image are divided by the R, G, B values of the the “white” portion respectively. Values greater than 1.0 are set to 1.0. Shown below is a picture of several objects of different colors; it was taken inside the classroom, with a camera setting of “incandescent”. After it is the “white balanced” picture using the Reference White Algorithm.
Before:
After:
The white balancing was able to remove the “blue cast” in the image. (Note: The paper is now white!)
The second method is the Gray World Algorithm. In this method, the R, G, and B values of the image are each divided by the average of the R, G, and B values, respectively. Again, if the values are greater than 1.0, they are clipped to 1.0. Using the same unbalanced image, the before and after shots using the Gray World Algorithm are shown below:
Before:
After:
For Gray World, the “blue cast” was again removed! On closer inspection, I noticed that the Reference White Algorithm did a better job because the paper is whiter and the other colors less dark.
Here is another example. The same scene (same objects, still inside the classroom with the same light source) has been captured with a camera setting of “cloudy.” The Before and After shots for both algorithms are shown below:
Before:
After (using the Reference White Algorithm):
After (using the Gray World Algorithm):
The “yellow cast” has been removed from the 2 “After” images; however, using the Reference White algorithm is better. The paper is “whiter” as compared to that of the Gray World balanced image, which is a bit bluer. The “After” picture from the RW algorithm produced the closest rendition of the original colors of the objects.
For the next part of the activity, objects of the same hue were captured in a single image using the “wrong setting” and then white balanced. The image has several leaves of different colors; it was pictured outdoors so the illumination is daylight, but the setting was “incandescent.” The Before and After images are shown below.
Before:
After (using the Reference White Algorithm):
After (using the Gray World Algorithm):
Visually, the RW balanced image seems to have better green hues and the Gray World image has a bit of a reddish and bluish tinge.
Overall, the Reference White Algorithm seems to give a more balanced image, color-wise.
I give myself a grade of 10 for this activity because I was able to perform white balancing using the 2 algorithms.
Thank you, Aiyin and Beth for taking the pictures, Mark Leo for obtaining the leaves, and Rica and Benj for the tip re: clipping the values to 1.0.
————————-
The following was my code for this activity:
im=imread(‘i-cloudy.jpg’);
w=imread(‘i-cloudy-white.jpg’);
Rw=mean(w(:,:,1));
Gw=mean(w(:,:,2));
Bw=mean(w(:,:,3));
//Reference White
imw(:,:,1)=im(:,:,1)/Rw;
imw(:,:,2)=im(:,:,2)/Gw;
imw(:,:,3)=im(:,:,3)/Bw;
Mw=max(max(max(imw)));
newimw=imw./Mw;
//im(find(im>=1)) = 1
imwrite(newimw,’i-cloudy-rw.jpg’);
//Gray-world
Rwg=mean(im(:,:,1));
Gwg=mean(im(:,:,2));
Bwg=mean(im(:,:,3));
ig(:,:,1)=im(:,:,1)/Rwg;
ig(:,:,2)=im(:,:,2)/Gwg;
ig(:,:,3)=im(:,:,3)/Bwg;
Mg=max(max(max(ig)));
newimg=ig./Mg;
//im(find(im>=1)) = 1
imwrite(newimg,’i-cloudy-gw.jpg’);
The goal of this exercise in stereometry is to reconstruct a 3D object by taking multiple pictures of it, either by using 2 cameras or taking 2 images of the object. The object that I will reconstruct is a rubix cube; I obtained these two images by moving the cube by 4 cm in the 2nd image:
The first step in reconstruction is to determine the focal length, f of the camera. I obtained f from the information provided in the Summary of properties of the images, where f=6 mm.
Next, I obtained the x- and y-coordinates of 25 points in the cube for each image (x1 for image 1, x2 for image 2). From these points, I was able to get the z-coordinates, using the equation:
Having obtained z, I plotted the x, y, and z components using the splin2D function and here is what I obtained:
This resembles one side (i.e. the top portion) of the rubix cube, with a few “waves.” The waviness of the reconstruction may be due to the fact that a small number of samples were obtained.
I give myself a grade of 8 for this activity for not being able to reconstruct the entire rubix cube.
I would like to thank Elizabeth Prieto and Jeric Tugaff for helping me in this activity.
In Photometric Stereo, we reconstruct the 3D shape of an object by capturing several images of an object – using a single view but with different positions of the light source. In this activity, we aim to recreate the 3D shape of the following pictures with different light sources:
To do this, we evaluate the equation 
where I is the image matrix, V is the matrix containing the position of the light sources, and g is the matrix for the reflectance of the image. Since we know the matrix I and V, we obtain g:
from which, we obtain the unit vector of n:
We get the gradient of c = z – f(x,y) = 0:
using the following code:
for j=1:c
dfdx(j)=-nhat(1,j)/(nhat(3,j)+1.0e-15);
dfdy(j)=-nhat(2,j)/(nhat(3,j)+1.0e-15);
end
and evaluate the line integral to get the Image matrix:
using the following code:
lintfx=cumsum(dfdx2di,2);
lintfy=cumsum(dfdy2di,1);
Image=lintfx + lintfy;.
The Image matrix is plotted and the result is a hemisphere. Voila!
I give myself a grade of 10 for this activity for being able to reconstruct the sphere (3D) from the 4 images (2D).
Thanks to my collaborators: Benj Palmares and Julie Dado.
Geometric distortions are usually a result of imperfections in the imaging system (e.g. due to the camera’s lens) or improper capturing of an image (e.g. taking a picture at a wrong angle). To correct for geometric distortions, we get the new coordinates for the undistorted image and then substitute with the gray level values.
I used the following image (provided by Dr. Soriano) for this activity:
One notices that in this image, the window’s edges do not appear as straight lines. The first process in correcting this image is to
