終わりだ (Activity # 19)

Activity # 19 - Restoration of Blurred Images.. this is the last of the AP 186 activities which means this sem is finally over.. well, not quite but at least it's almost done. xD

In this activity, our objective would be to restore an image corrupted by a blurring function and some noise. This entire activity can be summarized in the form of a flowchart:

In simple terms, just as the objective states, we start by taking an image, then we give it a blur by introducing a blurring function and adding a noise in the Fourier domain, and then finally restoring it using a Minimum Square Error Filter.

Since the equation for a blurred image with T as the length of exposure is defined by the equation:

for adding a blur, the degradation function would then be expressed as (also in terms of T):

As for the noise, we make use of the Gaussian Noise that was already discussed in the previous activity. The operations are performed in the Fourier domain and is described by the following equation:

The filter used for the activity is the Weiner Filter or the minimum mean square error filter given by either this

or this

where K is a constant (alternative when Sn and Sf are unknown).

So that's practically all the physics you'll ever need for the activity--what we need to do now is apply those equations to a test image. For this activity, the I grabbed a photo of Eric Clapton with 'The Fool' guitar from Gibson's site:

The blurred image generated by the degradation function looks like this:

Now, since there are two equations for the filter, we therefore have two reconstructions for the image: one using Sn and Sf (Left) and another one using K=0.01 (Right).

We can see that we have a better image using the Sn and Sf reconstruction. However, we can observe the following trend for various K's (K=1, K=0.1, K=0.01 and K=0.001 respectively:

From the set, we can infer that K approaching integer values is biased towards the blur, while small values of K are biased towards the noise. On a side note, the optimal value is hence in between K=0.1 and K=0.001.

This activity was rather easy compared to the other activities, so I'll give myself a 10. I'd like to acknowledge Neil for his help in this activity.

AP186のアクチヴィチ１３が、完了した!! (Activity # 13)

This is probably the most difficult (if not tedious) activity in AP 186. This activity (Activity # 13) is entitled Correcting Geometric Distortions.

Since a camera uses a lens to focus an image to its sensor, lens distortions can be observed. For spherical lenses, there are two common types: Barrel Distortion and Pin Cushion Distortion. Here are the examples for each kind (c/o imagetrendsinc.com):

In order to correct the distortion, we need a reference image with a regularly occuring pattern and visuallize the distortion across the image frame. If we let f(x,y) be the correct ideal image, and g(x,y) be the distorted image, we can assume that there exists a transformation function that mapped f(x,y) to g(x',y'):

The aim would be to invert the transformation in order to obtain our ideal image. Two image properties will be corrected in the process: the pixel locations, and their respective graylevel values.

We start by expressing x'=r(x,y) and y'=s(x,y), where r and s are the transformation functions used to map the ideal image to the distorted image frame. It is good to assume that these functions take the form of bilinear functions, r(x,y) = c1*x+c2*y+c3*xy+c4 and s(x,y) = c5*x+c6*y+c7*xy+c8.

That's basically the physics of this activity as the rest of the steps would make use of matrix manipulations. First, I obtained a barrel-distorted image from imatest.com and cropped it to get this image:

It's cropped for two reasons: one, this activity aims to demonstrate the concepts used in geometric distortion correction and thus a large grid is not advisable; two, since Scilab works faster with small matrices or images. Notice that the centered square on the grid features the least amount of distortion and our ideal grid was generated from the dimensions of that square. Here are the pixel locations of the points of intersection of the distorted and ideal grids respectively:

After obtaining the pixel coordinates for the ideal image, we then solve for the coefficients c1 to c8 by setting up the following matrices for each square (thus making the procedure similar to a moving averages run):

and solving for the coefficients using the following equations:

Once the coefficients are obtained, we get the corresponding pixel locations from our distorted image using the set of transformation equations listed above (x' = r(x,y) = c1*x+c2*y+c3*xy+c4 and y' = s(x,y) = c5*x+c6*y+c7*xy+c8) and map out these pixel coordinates to the ideal image map. For the points in the ideal image plane with no corresponding point from the distorted image, we use interpolation methods (bilinear or nearest neighbor) to obtain their grayscale values. Since I practically patterned out my interpolation routine from Gilbert's source (by the way, Gilbert helped me to get this thing working after we consulted Mandy several times), please refer to his source for the interpolation part.

Here are the results I obtained from this activity:

The images above are for the original image, the bilinearly-interpolated image and the image for nearest neighbor interpolation. Although the image was not fully restored, the bilinearly-interpolated image has better quality than the nearest neighbor-interpolated image.

Once again, I'd like to acknowledge Gilbert and Mandy for helping me in getting my version of the source to work. I'm getting a 9 for this activity.. minus one for the delay but 9 for finishing it anyway.

日本人彼女募集中。（Activity # 18)

This activity is entitled 'Noise Models and Basic Image Restoration.' It introduced me to the different noise models, and the filters to use whenever they're encountered in real life (not likely). As usual, although this activity will orient the reader on the different image restoration techniques, the best advice is still to start off with a good image--one that doesn't need restoration in the first place. The second best advice would be to use Photoshop--that's why, this post and Scilab with SIPtoolbox should come as a last resort (after Matlab).

The original test image and its pdf looks like this:

Different Types of Noise

In the activity, we were given six image models, as well as the equations needed for the models to act on the probability distribution function (refer to Activity 3) of the image. These noise models are: Exponential Noise, Gamma Noise, Gaussian Noise, Salt-and-Pepper Noise, Uniform Noise and Rayleigh Noise. Note that I listed them in the order the sample images above are arranged, except for Rayleigh Noise. (Rayleigh noise is currently in the works.)

While there are six noise models, only four filters were given for us to examine. These filters are: Arithmetic Mean, Geometric Mean, Harmonic Mean and Contra-harmonic Mean.

Exponential Noise

Gamma Noise

Gaussian Noise

Rayleigh Noise

Salt and Pepper Noise

Uniform Noise

Arithmetic Mean as the name implies, takes the average of the sum of color values for each subimage (hence, a moving average; sub-image means 3x3 patch); Geometric Mean takes the average via an n-th root; Harmonic Mean divides the patch dimensions by the inverse sum; and finally, the Contra-harmonic filter takes the sum raised to an order value Q, and divides it by the sum raised to the order of Q-1.

Since this activity is pretty much about learning what kind of filter to use for a specific kind of noise, the images below can serve as a guide for the reader. Though most of the time, the arithmetic mean and geometric mean filters obtain a similar looking image, the images are different for the Gaussian and Salt-and-Pepper noise. It is also important to note that for the contra-harmonic filter, we obtain different images for different values of Q.

The images above show the contra-harmonic filter acting on Salt-and-Pepper Noise for Q=-2, Q=0 and Q=+2 respectively. From this, we can infer that the value of Q to use would depend on the image being restored--negative values for grayscale images biased towards black, while positive values for grayscale images biased towards the white. In the case of my Yuuko sketch, since it's biased towards white, it is greatly restored at Q~+2.

I thank Neil and Gilbert for their help with the noise addition parts, and myself for helping others with the filtering part. xD Anyway, since all the objectives for this activity was accomplished, and I had my own batch processing script as a bonus, I'll give myself a 10.

「ai sp@ce」 のバイトは、どうしますか? (Activity # 17)

This activity focuses on photometric stereo, a method of extracting shape and surface detail from shadows. Using this method, we are able to reconstruct an image with surface detail from a series of several images taken at different point sources.

Suppose we have the matrix V given by

where the first index of V represents the image while the second index refers to x, y, and z coordinate of the light source respectively. Assuming these sources illuminate a common object of interest, we can obtain a series of images with intensities defined by

for each point (x,y). We would then have to find g for each point using the usual matrix operation (i.e. inv(V'*V)*V'*I) and normalize the g's to obtain the value for the normal vectors. We get the surface normals using

and finally, we obtain the surface elevation at a point (u,v) using

Following the steps I listed above, here's the Scilab output for the images contained in Photos.mat

I'd like to give myself a 10 for this activity, since this is one of the RARE activities that anyone can finish in one meeting. xD

一番大切なものは? (Activity # 16)

Neural networks is the third classification method for pattern recognition explored in this course, and is the title of activity 16. Unlike the LDA, neural networks don't need heuristics and recognition rules for classification and instead makes use of learning, a function it tries to imitate from the neurons in our brain.

In any case, the activity makes use of the ANN toolbox (ANN_toolbox_0.4.2 found on Scilab's Toolboxes Center) which makes the activity more of a plug-and-play kind of activity.

So how do neural networks work? Each input connection to the artificial neuron has its own weight (synaptic strength) and is multiplied to the inputs, xi. A neuron receives weighted inputs from other neurons and lets the sum act on an activation function g. The result z is then fired as an output to other neurons.

A neural network is formed through the connection of neurons. A typical network consists of an input layer, a hidden layer and an output layer. Just like the previous activities, two modes were considered -- a training mode and a test mode. The parameters one can play around with for the functions in ANN would be the Neural Network architecture (how many neurons for each layer), the Learning Rate and the Number of Iterations.

Using the same data I used in Activity # 15

I had to do a few modifications. First, the inputs must be in a [2xN] matrix (done by transposing the bin after an fscanfMat() call), afterwhich the values must be normalized (ranging from 0 to 1). For the training set, I also had to change the classification values (1 and 2) to 1 and 0 (this is crucial to avoid having erroneous outputs). Finally, here's the output of my neural net classification after round():

For the win, I give myself a pat at the back and a well-deserved 10. xD I'd like to thank Gilbert for his help in helping me understand the ANN toolbox.

好き? (Activity # 15)

Activity # 15 is pretty much like the previous activity except for a little improvement--this time, it uses probabilisitic classification to minimize the risk or loss in classification.

Since my classes can be linearly separated (i.e. one can separate one from the other by drawing a line in the plot in the previous activity), the method I used was Linear Discriminant Analysis, the details of which can be found here.

This time, we were asked to sort out an image with patterns that are close to being similar, i.e. they should have little to no difference. I generated another image of blobs using Photoshop--the blobs are one brush size apart in terms of size and a difference of 50 ticks when it comes to G-channel color value.

Of course, the script is pretty much plug and play.. all that was needed was to obtain data using the script from the previous activity, dump them into an outfile, get the LDA script (found in the tutorial but implemented in scilab) to organize and process the files, and poof! we have an LDA plot as shown above.

Note that unlike the previous activity, it organized all the data points in a line, and that the discriminant functions were successful in classifying close patterns by increasing the difference between them (note the order of magnitude - xE+06). As expected from the image, the LDA results to a perfect classification (100% correct). I'm not sure about real life images though. I'll probably give it a shot when I find some time.

That's why, it's probably just a 9 for now.

もう。。わかんない。 (Activity # 14)

Pattern recognition is the subject of Activity 14 and will probably be the subject for the next few activities.

In the context of image processing, it can be used to decide whether a feature vector (ordered set of attributes) of a pattern (a set of features) belongs to one of several classes (set of patterns sharing common properties).

Activity 14 makes use of the Minimum Distance Classification routine, where a pattern is assigned to the class whose mean or representative to it is nearest to. This can simply be done by either taking the minimum distances of the means for each sample, or by simply plotting out the means.

My original design for this activity was actually to implement the Scilab routine on an image of coins (30 coins, three classes - 25 centavo, 1 peso and 5 peso coins, and up to four features - size, R, G and B-channel color values). However, it seems absurd to do it straightforwardly, so I decided to test my algorithm on a Photoshop-generated image (based from the original coin image) of circles with three different brush sizes and three different color values in the Red. The fifteen patterns (five for each class) used for the training set were also obtained from the image of 30. All it took was a little playing around with the bwlabel() and find() functions.

As expected, the algorithm managed to sort the patterns out. Judging from the plot, there was a clear distinction between each pattern in the image, and that the deviation of the individual patterns from the training set was small.

When applied to coins however, although there is a clear distinction when it comes to size, the color values (not only in the red) were scattered all over the place. This is probably due to the uneven lighting in the image--there were shiny coins and there were dull coins. A better result can be obtained by capturing a better photo, especially with the use of reflectors and diffusers.

For my choice of a difficult image, and succeeding in sorting them out even if it's against it's will. xD I'm getting a 10.