Lab 5: Camera Geometry

Deliverables

The lab checklist will be provided in hard-copy. Refresh the PDF to get the latest. The checklist has been updated with the correct deadline: 9 am Wednesday of Week 6 (at the start of lab, with ~5 min grace period)

Please submit the lab in hard-copy or using esubmit. If using esubmit, submit exactly one PDF containing the equivalent of your lab packet, as described on the esubmit site. I will print it out. If you plan to use esubmit from off-campus, ensure that you have a working version of Global Protect by testing it off-campus a day before you need it.

Overview

In this lab, we will explore 3D camera calibration. The primary tasks will be:

• Calibrate a camera to find its intrinsic parameters using a toolbox
• Backproject objects of known size into world coordinates
• Play with live video

Determine size of image that will be used by live video capture

You can use any camera with this lab that Matlab is able to connect to and capture video from. If you are calibrating a camera with a zoom, be sure to take the pictures at a repeatable zoom setting, such as zoomed all the way out or in.

Extract demo.zip, change into the demo directory, and run the script getImageSize

The first time you run this command you will get an error message something like

You will need to install additional software to connect with the camera. Please see Appendix A.

Once getImageSize completes successfully, the answer will be the size of a single frame of the video, [i j].

Calibrate the camera

You may wish to run this demo with the Logitech webcam used in SE3910 . This would only require editing of the function getCameraInfo which selects cameras in the demo.zip file. See that file for details.

Run the command cameraCalibrator from the Matlab prompt. This will load the calibration GUI.

We will begin by capturing images. (For alternative image-capture strategies, see the Appendix B)

Print the calibration pattern, and tape it as flat as possible on a hard surface like the back of a clipboard.

In the top left corner of the camera calibration window, click on the arrow at the bottom of the Add Images button, and select "From camera"

The first time you do this, you will get this error message:

Please see Appendix A for instructions on adding the necessary software to Matlab to perform this lab. The process will require you to create a free Mathworks account, which you can use your MSOE email for.

Once you have successfully opened the camera in the camera calibrator, click on "Camera Properties" in the top-right corner. Select a resolution for your camera that matches what you got from getImageSize in the previous part. Once you have your grid ready, select "Calibrate."

Take about twenty pictures of the grid, with the grid filling as much of the image as possible, and at different angles, as in the examples below. (It is important for the grid to be at an angle, and not just flat in the image. In effect, this helps the camera measure "depth" of the camera's viewing volume.)

Once you have successfully marked all the images, click "Calibrate"

Before you exit matlab or close the calibrator, be sure to "Export Camera Parameters",, using the default name cameraParams for the exported variable. (You can save the exported variable as save cameraParams. This will save the cameraParams variable into a cameraParams.mat file, which you can load later with load cameraParams)

At this point, it can be fun to try out some of the othere features of the Calibrator GUI.

Reprojecting points

Before running the demo, be sure cameraParams is in the workspace. If you have not already done so, extract demo.zip,

Then run demo. If you end the demo early by typing Ctl-C, reset the camera capture using clear camera.

This demo should run, displaying a color video stream that will dispay a box around any red objects you put into the field of view. However, the 3D display will be unusable. (See the next section for more about that.)

The first time you run the demo, it may display this warning:

Again, you will need to install additional software. Please see Appendix A.

Editing the reprojectPoints.m method

To make the 3D display usable, edit the method project point so that it corretly projects a point (or row-vector of points) into 3D coordinates. (This file is found in demo zip, the template below is just for reference.) (THE ZIP FILE HAS BEEN UPDATED to match the corrections below

% Project point from image coordinates into camera coordinates
% Provides vector from camera center to a point along the ray in camera
% coordinates.
%
% iPix - vertical pixel index or row-vector of indices (increases downward 
%        from top left corner)
% jPix - horizontal index or row-vector of indices 
%
% zCm - depth of the point from the camera (cm) (distance parallel 
%       to optic axis from camera optic center)
%     
% fc - vector of focal lengths in j and i directions.
%      that is, fc = [fc_i fc_j] == [fc_y fc_x]
% cc - camera center vector
%      pixel location of the camera center [cc_i cc_j] == [cc_y cc_x]
%
% v - 3xn matrix of projected points in camera coordinates.

% Author: Phileas Fogg
% Term: Winter 2012-2013
% Course: CS498 - Computer Vision
% Assignment: Lab 5 - 3D Camera Calibration

function v = reprojectPoint(iPix,jPix,zCm,fc,cc)

To project a point from camera coordinates into world coordinates, use the technique we developed in class to construct an A matrix from the camera parameters. (You can ignore the radial disortion for this demo. But for the best accuracy in future projects, you will need to take it into account.) Then, use the backprojection technique we discussed in class to project the point as a 3D point. At this point you should be able to move the ball in 3-D and see it moving in 3D in the window. (You can rotate Matlab 3D figures using the rotation tool at the top of the figure.)

Fun Ideas

• It's project time! What would you like to do with computer vision? What are three small steps that will move you toward that goal?

(^1, 2, 3) Appendix A: Installing Support Software to work with camera from Matlab

While attempting to connect to the camera, you may get one of these error messages:

To resolve these problems, you will need to install the package "USB Webcams.

Click on the "Support Package Installer" link to open the Support Package Installer. Or type supportPackageInstaller at the command window.

Do the default option on the first window -- from internet

Select "USB Webcams" (or "OS Generic Video Interface") and Click the "Next" button. At this point, you may need to create a Matlab account. This is free, but requires an email address.

Now you will need to log in to proceed....

You can obtain a password from mathworks for free.

Click "Accept" if you accept the license terms, then next

Click "Install"

The install will proceed... this should be fairly quick on a good network.

... and hopefully finish successfully. After this, you should be able to use the zip file to run the demo. Hopefully, I will bring red balloons to serve as the red objects.

Appendix B: Alternative ways to capture images of the grid

For your final project, you may need to calibrate a camera that you cannot connect to Mablab. You can still calibrate the camera with matlab as long as you can deliver .jpg images to the laptop that you run Matlab on. This section outlines a few techniques for capturing images that may be helpful to you.

If you are calibrating a camera with a zoom, be sure to take the pictures at a repeatable zoom setting, such as zoomed all the way out or in.

Print the calibration pattern, and tape it as flat as possible on a hard surface like the back of a clipboard.

See the instructions for how to hold the pattern in the main text above.

Option 1: Using the matlab camera calibration toolbox.

This is the best and easiest option for this lab. For other projects, you may find the other approaches below helpful. See the instructions in the main text above.

Option 2: Matlab

Use imaqtool to capture the images. At the bottom of the screen, in the "Acquisition Parameters" section, in the "Triggering" tab, select "Manual" to see a preview video when you hit "start acquisition". Hit "Trigger" to capture a single frame.

This code may be helpful... if you save your images to the workspace as frame1, frame2, etc.

for i = 1:10
frames(:,:,:,i) = eval(['frame' num2str(i)]);
end
for i = 1:size(frames,4)
imwrite(frames(:,:,:,i),['image' num2str(i) '.png']);
end

Option 3: (If using the fancy Logitech cameras)

Go to Logitech C920 camera support, click on Downloads, and download lws251.exe for your machine.

Once this program is downloaded, run it to install the software that comes with the camera. The "Logitech Webacm Sofware" should start running as soon as the install completes. Click the "Quick Capture" button to go into caputure mode. Take about ten pictures of the grid, with the grid filling as much of the image as possible, and at different angles, as in the examples below. (It is important for the grid to be at an angle, and not just flat in the image. In effect, this helps the camera measure "depth" of the camera's viewing volume.)

Click on the "Photos" tab at the bottom. Right-click any image, and selet "Go to file." Copy the resulting images to a convenient directory, which we will call the image directory.

Option 4: Whatever technique you want

Just make sure the captured images are the same size as you will use in your application (the size returned by getImaeSize for this lab).