CS498: Computer Vision

CS498

Lab 1

This is an individual lab -- each student should submit their own lab.

When using MATLAB off-campus, you will need to use the VPN GlobalProtect. GlobalProtect can be a bit finnicky -- you may want to ask me about this if you plan to use GlobalProtect off campus. All the labs were developed off-campus, so it DOES work.

Overview

Manipulate an image as an array of numbers
Blur an image using Gaussian filtering
Explain why a filter's values should generally sum to 1 if it only has positive values.

Prelab

Install Matlab from the Software Center on your computer.

Deliverables

The lab checklist (I will give you a hard-copy of this.) Update: Deadline changed to 8am
A single Matlab file divided into cells (groups beginning with a two-percent %% comment), following this template.
The original image you worked with
The resulting images created by your program: See the checklist for details.
(optional) other images that you created with your program.

Useful Matlab commands for this lab

clear: Delete all variables
close all: Close all windows
clc: Clear the command window (but not history)
commandhistory: Opens the command history window. You can then "dock" it so it is available every time you start Matlab
imread: Read a Matlab image. e.g. I = imread('filename.png');
imshow: Show an image; open a new figure to hold the image if needed. e.g. imshow(I)
shg: "Show graph"; Bring the current figure window to the front. e.g. shg
size: Find the size of a Matlab array. e.g. sz = size(I);
zeros: Make an array of zeros with a specified size. e.g. black = zeros(sz);
ones: Make an array of ones with a specified size. e.g. white = 255*ones(sz);
ginput: Allow user to click points on a graph. e.g. [j,i] = ginput(1); will allow you to click on one (1) point on the graph. If you do not supply an argument, ginput will keep collecting points until you press the enter key.
imfilter: Filter an image using correlation. e.g. I2 = imfilter(I2, [-1 0 1]); is a simple derivative filter in the j direction.
normpdf: Create a Gaussian filter (or Probability Distribution Function -- PDF). e.g. blurFilter = normpdf(x,mu,sigma)
double: Convert an image to the double type. e.g. I = double(I)/255;
class: Find the type of an array. e.g. cls = class(I);
plot: Plot 2-d data. e.g. plot(x,y,'ro-'). If you use doc for help, be sure to check out the LineSpec link.
hold on: This command saves what's on the plot so that tne next plot command will plot over what is already there instead of clearing it. This can be useful for plotting points on images: show the image, hold the plot, and plot the point.
hold off: This command sets the plot so that the next plot command will clear the plot and start fresh.
figure: This command creates a new window or activates the given window if it is already open. figure(2)

If you would like more information about any of these functions, use help functionname to get the command-line help. doc functioname brings up the full Matlab help browser for that function.

You may have noticed in the examples above, some commands end with a semi-colon, while others do not. This is because the semicolon is not required on statement endings. Matlab uses one line for each statement. (If you want to continue a statement over multiple lines, use ... at the end of each line except the last one.) The semi-colon is used for hiding the result of a statement. So if you want to hide the results of a Matlab statement (which you generally do if it is a HUGE image), put a semicolon at the end of the line. If you want to see the results printed to standard-out, leave the semicolon off. (In the examples above, the commands without semicolons don't have any output at all.)

This Matlab Tutorial from Brown University is very good. Everything discussed in the tutorial is essential to programming in Matlab¹. Although I try not to assume familiarity with Matlab in this lab, if this is your first time using Matlab, I encourage you to work through the tutorial in depth, reading all the examples and trying some of them in Matlab to see how they work for yourself.

In particular, the tutorial has a section on working with images. Search for %(8) Working with (gray level) images. It's at the end.

1. Opening and displaying an image.

(The numbers of the sections correspond to the code cells in the required lab template)

Select any image file on your computer. The .jpg, .bmp, or .png formats work fine. Open the file with imread and display it with imshow. shg may also be helpful here.

As you work through this lab, you may want to just "try things" at the Matlab command prompt and then paste what works into your .m file. Do run clear; close all to clear all your variables and close all your figure windows, then re-run everything from your file to make sure it is complete!

2. Look at an interesting point on your image.

2.a. ginput

Use ginput to select an interesting point on your image. Try to choose a point with high contrast (both dark and bright values), perhaps a sharp corner or an edge. [j,i] = ginput(1); will assign values to variables j and i (in that order).

2.b. Create a chunk of the image

Manually create code that assigns the same values, e.g.

 i = 148;
 j = 236;

(You may be wondering why I am encouraging you to hard-code a value. The scripting techniques we use in this lab are for creating "scratch code" in matlab. Creating scratch code is an effective way of interactively creating short snippets of code, which you can then refactor into proper Matlab methods/functions.)

This makes it easy to go back to the point you selected earlier.

Select a portion of the image near that point. Your chunk should be about 10 pixels wide, 10 pixels tall, and have all three image channels. It should be an array of size roughly [10 10 3]. To select a sub-array, it is helpful to play with Matlab's range options. Consider these examples to get started:

x = [1  2  3  4;
     5  6  7  8
     9 10 11 12];
x(1,1)     % this prints 1, not 6.  Matlab is 1-based, not 0-based!
y = x(:,3); % Ask yourself: What does this do?
y = x(2,2:3); % Ditto.
              % Can you combine these techniques to take a chunk out of 
              % a three-dimensional image array?

If you would like more than these enigmatic hints, try the Matlab Tutorial from Brown University.

Some useful things to know about images in Matlab:

The first index is for the row of the pixel
The second index is for the column of the pixel
The third index is for the color channel within the pixel, in the order red, green, blue (RGB)

For example, value = I(1,5,3); will give you the blue brightness of the pixel in row 1, column 5. It will give you the blue component because 3 indicates the third channel of RGB.

It is OK if your selection is hard-coded to this particular image. In future labs, you will do this sort of thing programmatically.

Display your small image chunk. Does it look like the point you clicked?

Print your small image chunk to standard out and include it in a comment, like this: (You can comment a block of text by pressing Ctl-R, you can uncomment with Ctl-T.)

% Ismall(:,:,1) =
% 
%   221  222  215  218  216  218  221  217  226  215  218
%   221  222  218  218  210  212  219  222  234  218  214
%   217  219  226  221  219  213  217  216  223  219  214
%   219  219  220  222  229  224  221  221  221  218  214
%   217  215  197  210  220  223  216  214  225  211  216
%   217  221  217  227  197  199  197  185  207  214  218
%   214  217  217  213  194  196  195  174  155  217  222
%   216  217  215  230  202  195  222  221  184  202  220
%   213  215  221  198  176  173  192  186  170  204  217
%   215  215  202  191  204  188  164  122  156  223  215
%   214  212  207  181  185  190  186  172  180  212  215
% 
% 
% Ismall(:,:,2) =
% 
%   221  222  221  227  222  223  226  222  220  224  222
%   219  220  218  223  220  219  219  217  219  221  225
%   221  219  221  218  226  224  223  221  215  224  223
%   219  219  218  213  220  222  222  225  218  219  224
%   217  218  217  228  222  217  214  212  225  224  218
%   217  217  217  211  130  104  101   91  158  233  219
%   216  217  222  198   89   72   78   60  107  225  220
%   214  216  225  218  134   95  124  111  125  215  224
%   213  214  201  149  104   81   92   75  107  219  222
%   211  220  117   73  100   95   84   54   97  218  220
%   213  221  133   71   90   88   90   82   90  195  223
% 
% 
% Ismall(:,:,3) =
% 
%   219  220  217  222  220  219  220  215  222  223  221
%   220  221  216  226  229  227  221  213  222  214  219
%   220  217  215  209  218  218  221  214  213  220  218
%   217  217  221  218  225  225  226  234  227  213  216
%   217  211  224  230  219  201  201  200  223  217  217
%   215  216  209  196   88   60   63   57  151  231  214
%   211  215  225  191   31   12   24    0   93  228  223
%   217  214  227  220   87   43   61   22   91  223  223
%   213  220  174  117   46   34   56   29   89  226  215
%   210  223   76   25   49   52   51    9   79  222  213
%   208  216   94   22   36   40   48   30   53  188  208

Can you see the shape that you selected in the values in the image? Write a comment about what you learned from looking at these raw numbers below the numbers.

Displaying a single-color chanel

To display a single color channel of an image, you can extract that color channel using Matlab's indexing, for example, to display the red channel, you could use imshow(I(:,:,1));. However, this image will appear gray. To create an image that looks red for a red channel, you can create a full-color image that only has zeros for the second and third "image planes."

Here are two ways to do this:

% Clear the blue and green channels
Ired = I;
Ired(:,:,2:3) = 0;
imshow(Ired)
shg

% Create a new image and copy over the red channel
Ired = zeros(size(I));
Ired(:,:,1) = I(:,:,1);
imshow(Ired/255)

Notice that the first image is shown with imshow(Ired), while the second uses imshow(Ired/255). This is because in the first image, Ired has the type uint8, and in the second image, Ired has the type double. imshow interprets the image as being in the range 0-255 for uint8, and in the range 0-1 for doubles.

Matlab uses the type double by default, and it is fine if you do that throughout these labs. If you want to convert an image to the double type, use I = double(I)/255;. You can use class(I) to find the class of an image.

Another option that could be handy is the imagesc(I), which is like imshow(I), except it automatically re-scales the intensities in the image to go from full black to full white before displaying it. It seems to throw errors on me a lot, though, so I just use mx = max(I(:)); imshow(I/max) instead.

You do not need to report anything for this section. It is preparation for the next section.

3. Creating a gray-scale image

Keeping track of the three color channels can be a challenge. So we often make a gray-scale version of an image.

In this section, you will create a gray-scale version of an image, by averaging the color channels for each pixel in the image

For each pixel in row i, column j, you can compute a simple gray-scale image as gray_i = 1/3 (r_i,j + g_i,j + b_i,j). However, you do not need to create any loops to create this gray-scale image. If you add two arrays in Matlab, it will automatically perform the operation for each element. e.g. [1 2] + [30 40] will yield [31 42].

It might help to convert the image to the double type, otherwise you can have issues with rounding or overflow error.

4. Blurring an image

To blur an image, use the imfilter command. It is preferable if you create your own coefficients for the filter. If you use a built-in command to create the coefficients, you must discuss what the coefficients look like to you and how you think they work to blur the image.

As long as the image "looks" blurred, you have succeeded - there are many ways you could do this. Be sure to blur the image in both the x and y directions. Avoid a "motion blur" that only blurs along a line such as one axis or the other.

In your template, answer the questions about the filer summing to one

Fun Ideas

You can downsample an image by selecting every other pixel along the rows and columns. Can you do a simple downsampling using Matlab's range operators? If you want to store your downsampled images in a single variable, a cell array might be useful. (Not to be confused with cells in a .m file, which are completely different.) Cell arrays are like matrices, except that they can hold any matlab object in each position. Try stack{1} = I;, imshow(stack{1});, etc.
Can you create your own image by alternating black and white pixels between rows? (You can use the Matlab range operators and indexing for this, too.)
Logical indexing is a powerfull alternative to range indexing. In a logical index, the index is an array the same size as the image (or as one dimension of the indexing). You can create a logical mask with an expresssion like mask = I > 0.5;, and then only edit the selected pixels with an expression like I(mask) = 0. What creative images can you come up with using masking? (If you want to use a mask on a color image, you will need to duplicate it for each channel, e.g. [mask(:);mask(:);mask(:)])
You can also create fun images using alpha-compositing. This is where you combine images together, but multiply them by an "alpha" mask first. What fun images can you come up with using an alpha mask? (How might you compute the mask automatically from the image? Or from some simple user input?)
The gray-scale method we use in this class works well, but other weightings of the channels can give more pleasing results. Look up in the text the section on color, and try converting your image to gray-scale (or a different full 3-d color space like YCrCb). Does this make your grayscale image look better? Do you think using a color space closer to human perception would increase the performance of your computer vision algorithms? (You may want to revisit this question later in the quarter.)
What would you like to do automatically from images? Give it a try!

The fine print

¹ Except you should use (1:10) instead of [1:10]. As the squiggly orange mlint line will tell you, brackets are unnecessary around a range. They imply a call to the concatenation function, so using parenthesis will save you a few clock cycles.

Lab 1: Introduction to Image Processing in Matlab