Outcomes

Key

• Normal (on exam)
• Added (on exam)
• (to be reviewed) To be covered in more depth later (not on exam)
• Removed from objectives (not on exam)

Global Protect

• To restart if it is continually "connecting", open services (from the control panel).
• Stop the service PanGPS
• Start the service PanGPS
• Ask me if more detailed steps desired

Matlab

• Without a loop, write matlab code to create an array containing a given range of numbers
• Without a loop, write matlab code to apply various operators and functions to every element of an array

Lab 1: Introduction to Image Processing in Matlab

• Manipulate an image as an array of numbers
• Filtering: convolution and correlation
  • Explain the advantages and disadvantages of the four standard ways of dealin with edge cases where the filter runs off of the image.
  • Filtering: Gaussian blurring
  • Sample a gaussian PDF to produce a filter, normalized to have a sum of 1
  • Explain why a filter's values should generally sum to 1 if it only has positive values.
  • Blur an image using Gaussian filtering
  • Describe the function of the two-sample derivative filter
  • Without a loop, write matlab code to create a derivative image
  • Explain why the edge filter's values should sum to 0
  • Find edges using derivative filtering
• Create and interpret edge magnitude and phase images
• Explain why all colors can be represented by just three numbers
• Give the RGB values for Red, Blue, Green, Cyan, Magenta, and Yellow
• Convert an image to grayscale by averaging RGB values
• Compute histograms of image intensities
• Interpret intensity histograms to compare the brightness of two images
• Describe how a gradient vector can be computed
• Describe the meaning of a gradient vector's angle and magnitude
• Compute a weighted histogram of edge gradient directions

Linear Algebra Review

• Select the appropriate element from a matrix based on its indices
• Multiply two small matrices by hand
• Explain the constraints on dimensions when multiplying two matrices
• Create a matrix with identical rows or columns using matrix multiplication
• Explain why a system of linear equations may have only one solution, infinitely many solutions, or no solution.
• Explain how an overconstrained set of linear equations can be solved to find an approximate solution
• Explain the role of the matrix inverse
• Explain under what conditions a matrix inverse exists
• Explain how you can find the matrix inverse
• Explain why, (e.g. when using matlab) you usually don't need to find an inverse.
• Solve a set of linear equations in Matlab using the fancy "matrix divide" operators "\" and "/".
• Compute the trace of a small matrix
• Compute the trace of a matrix, given its eigenvalues
• Compute the determinant of a 2x2 matrix
• Compute the determinant of a matrix, given its eigenvalues
• Explain why the trace and determinant are preferable to compute over computing the eigenvalues
• Define positive semi-definite matrix
• Give the key property eigenvalues of a positive semi-definite matrix

Lab 2: Homographic image warping

• (to be reviewed) Explain the sense(s) in which rotation is linear
• (to be reviewed) Explain how linear transformations can be represented with matrices
• Create a transform to rotate an image
• Create a transform to scale an image
• (to be reviewed) Create a transform to translate an image
• Apply transformations to points
• Apply transformations to images
• Explain why, to transform I1 onto I2 using homography H, you actually need to use inv(H).
• Write a matrix for an affine or a full projective transformation
• Describe the effect of affine and full projective transformations on an image
• (to be reviewed) Create a composite transform rotating an image about a desired point
• Convert between "true" and homographic representations of a point
• Explain why a full projective homography is important
• Compute a homography based on feature-point correspondences
• Define the term image interpolation
• Explain why image interpolation is important
• Apply a homographic transform to a 2D image point
• Apply a homography to an image
• Apply an arbitrary scaling and rotation to an image using image interpolation. (An important part of the SIFT algorithm.)
• Define the term robustness
• Define the term alpha-channel
• Blend images using alpha-compositing

Lab 3: Interest-point detection

• Explain why image point correspondences are useful for multi-camera geometry
• (later) Explain why image patches are useful for image interpretation
• (later) Explain the role of features in reducing image variations
• Define repeatability
• Define illumination-invariance
• Explain why illumination-invariance is important
• Describe rotation invariance
• Explain why rotation invariance is important
• Define scale-invariance
• Explain the importance of scale invariance
• Define interest-point
• Describe how interest-points can be detected
• (to be reviewed) Implement interest-point detection for SIFT
• Explain how SIFT feature-point detection achieves illumination invariance
• Explain how SIFT feature-point detection achieves rotation invariance
• Explain how a matrix can be used to represent an ellipse
• Explain how the dimensions of the ellipse can be determined from the eigenvalues of the matrix
• Define the Hessian matrix
• Explain how a surface can be modelled at a minimum/maximum by fitting a parabaloid to the ellipse
• Explain how points can be distinguished from lines by testing the ratio of the Hessian matrix's trace to its determinant.
• Define scale-space
• Explain how scale-space representations can be used to match different images of the same object
• Explain how feature-points can be selected from scale-space
• Explain why extrema in scale-space should be compared not only spatially, but also across scale
• Explain how a difference-of-gaussian pyramid stack is computed
• Describe the images that make up the scale-space representation of an image
• Explain how an image can be reconstructed from its scale-space representation
• Explain why an image pyramid can speed up processing
• Predict the location of an image feature in a scale-space pyramid
• List the two potential causes of gaps between resolution stacks in a scale-space pyramid, and explain why we need three extra layers to avoid them
• Explain how to avoid gaps when moving from one pyramid resolution to another

Lab 4: Interest-point description

• Explain the role of the gradient in achieving illumination invariance
• (later?) Explain the role of edge direction in describing an image patch
• (later?) Explain the role of coordinate transformation and sampling in achieving rotation invariance
• Implement illumination-invariance using derivative filters
• Implement illumination-invariance using histogram trimming and normalization
• Explain the two essential steps to achieving rotation invariance in the extraction of SIFT feature descriptors
• Explain the role of the histogram in achieving robustness to feature localization errors in SIFT and HoG
• Explain how a Gaussian window increases the repeatability of a SIFT feature
• Explain the role of histogram normalization in achieving illumination invariance
• Create composite transformations from rotations and translations to create acoordinate transform
• Explain why sub-pixel feature-point selection improves the performance of SIFT. (We will not implement this feature.)
• Create a transform to sample a desired rotated image region
• Map theta values back into the range -pi to pi using the modulus (mod) function

Lab 5: Image alignment using SIFT feature matching

• Explain how features are matched in SIFT
• Explain how the 2nd-nearest neighbor can be used to reduce false-positives in SIFT
• Explain the advantage of the K-D tree for SIFT feature matching
• Explain how to remove false point-matches using the RANSAC algorithm
• Explain how RANSAC detects outliers and inliers
• Estimate the number of RANSAC iterations needed to find a match
• Explain why it is usually best to use the minimum number of features possible to find the initial estimate of the transform when using RANSAC
• Use RANSAC to find correct matches in the midst of many false matches
• Explain how RANSAC determines if a match is an outlier
• Explain how the best match can be determined with RANSAC
• Explain how these metrics may fail, and how to correct for failure

Lab 6: Camera Geometry

• Intrinsically and extrinsically calibrate a camera using prebuilt tools
• Interpret the parameters of a camera's intrinsic and extrinsic calibration matrices
• Project points from a world coordinate system into a camera coordinate system
• Locate points on the unit plane a known plane using back-projection from a known camera calibration

Thresholding and Morphology

• Describe the simplest thresholding algorithm
• Threshold a very small image by hand
• Describe the effect of simple thresholding on a histogram.
• Explain the importance of adaptive thresholding
• Explain the complementary roles of thresholding and morphological operators
• Perform dilation and erosion on very small images by hand
• Explain how dilation+erosion joins pixels together
• (later?) Explain how to remove speckle noise from a binary image

Face Detection and Recognition

• Explain the difference between face detection and face recognition

Modern Computer Vision -- detection, features, machine learning, and ... detection

• Define feature in the context of machine learning
• Explain how heuristic features aid in machine learning
• Explain why features are often hand-constructed (heuristic)
• Explain how some machine learning problems can be viewed as a high-dimensional space partitioning problem
• Describe the difference between a binary classification and a regression machine learning problem
• Define training data
• Define testing data
• (later?) Describe how supervised, unsuperised, and semi-supervised learning can be distinguished based on their use of training data

Machine Learning Algorithm Performance

• Define True Positive, False Positive, True Negative, False Negative
• Explain why increasing true positives alone is not meaningful
• Explain how the tradeoffs associated with changing an acceptence threshold can be illustrated with an ROC (Receiver Operator Characteristic) diagram
• Draw points on an ROC curve
• Interpret points and lines on an ROC curve
• Explain why an ROC curve is always convex
• Explain why some ROC curves have axes ranges [0-1] on both axes
• (later) and others do not
• (later?) Explain the alternative to a distance threshold used by the SIFT algorithm
• (later?) Illustrate the differences between these algorithms with two-dimensional figures
• (later?) Explain why distances in high-dimensional space are often nearly equal

Detection Algorithms

• Describe the scanning window approach to object detection
• Given the number of windows in an image, determine the false-positive rate per window necessary to achieave a given false-positive rate per image or per sequence of images
• Describe the advantage of a cascade of classifiers
• Describe the advantage of how a Haar-like feature is computed.
• Describe how a Haar-like feature is like a blurred derivative filter
• Illustrate the strategy used by a linear Support Vector Machine to classify points in two dimensions

Cut objectives

All objectives beyond this point have been cut.

Face Recognition

• Explain the difference between face detection and face recognition
• Argue for (or against) face recognition being able to recognize anyone on the street
• Explain the role of resolution in face recognition performance
• Explain what an eigenface is

Structure from motion

• Define triangulation
• Define structure from motion
• Explain (at a high level) how to compute structure from motion for a pair of camera images
• Explain (at a high level) how bundle adjustment works

Dense motion estimation

• Explain how disparity can be computed along scan lines for a calibrated image pair
• Compute depth from disparity
• Explain the importance of smoothing parameters when computing stereo correspondences

3D Reconstruction and Structured Light

• Explain how structured light can be use to reconstruct a 3D scene
• Explain the key components of a Kinect scanner

Image-based rendering

• Define image-based rendering
• Describe a basic strategy for view interpolation
• Explain how holes can appear in a view-interpolated image, and explain a strategy for filling them