Computer Vision (CMU 16-385)
This page contains lecture slides and recommended readings for the Spring 2021 offering of 16-385.
(Overview of computer vision)
(Image transformations, point image processing, linear shift-invariant image filtering, convolution, image gradients)
Basic reading:
- Szeliski textbook, Section 3.2
(Image downsampling, aliasing, Gaussian image pyramid, Laplacian image pyramid, Fourier series, frequency domain, Fourier transform, frequency-domain filtering, sampling)
Basic reading:
- Szeliski textbook, Section 3.4, 3.5
Additional reading:
- Burt and Adelson, "The Laplacian Pyramid as a Compact Image Code", IEEE ToC 1983. (The original Laplacian pyramid paper.)
- Hubel and Wiesel, "Receptive fields, binocular interaction and functional architecture in the cat's visual cortex", The Journal of Physiology 1962. A foundational paper describing information processing in the visual system, including the different types of filtering it performs; Hubel and Wiesel won the Nobel Prize in Medicine in 1981 for the discoveries described in this paper.
(Finding boundaries, line fitting, line parameterization, Hough transform, Hough circles)
Basic reading:
- Szeliski textbook, Section 4.2, 4.3
(Visualizing quadratics, Harris corner detector, multi-scale detection)
Basic reading:
- Szeliski textbook, Section 4.1
(Designing feature descriptors, MOPS descriptor, GIST descriptor, Histogram of Textons descriptor, HOG descriptor, SIFT)
Basic reading:
- Szeliski textbook, Section 4.1.2, 14.1.2
(2D transformations, projective geometry, classification of 2D transformations, determining unknown 2D transformations)
Basic reading:
- Szeliski textbook, Section 3.6
Additional reading:
- Hartley and Zisserman, "Multiple View Geometry in Computer Vision", Cambridge University Press 2004. A comprehensive treatment of all aspects of projective geometry relating to computer vision, and also a very useful reference for the second part of the class.
- Richter-Gebert, "Perspectives on projective geometry", Springer 2011. A beautiful, thorough, and very accessible mathematics textbook on projective geometry (available online for free from CMU's library).
(Panoramas, Image homographies, Computing with homographies, direct linear transform (DLT), random sample consensus (RANSAC))
Basic reading:
- Szeliski textbook, Section 6.1
Additional reading:
- Hartley and Zisserman, "Multiple View Geometry in Computer Vision", Cambridge University Press 2004. Sections 2 and 4 in particular discuss everything about homography estimation.
(Pinhole camera, accidental pinholes, camera matrix)
Basic reading:
- Szeliski textbook, Section 2.1.5, 6.2
Additional reading:
- Hartley and Zisserman, "Multiple View Geometry in Computer Vision", Cambridge University Press 2004. Chapter 6 of this book has a very thorough treatment of camera models.
- Torralba and Freeman, "Accidental Pinhole and Pinspeck Cameras", CVPR 2012.
(Review of camera matrix, perspective, other camera models, pose estimation)
Basic reading:
- Szeliski textbook, Section 2.1.5, 6.2
- Hartley and Zisserman textbook, Chapter 6.
(Triangulation, epipolar geometry, essential matrix, fundamental matrix, 8-point algorithm)
Basic reading:
- Szeliski textbook, Section 8.1 (not 8.1.1-8.1.3), Chapter 11, Section 12.2
- Hartley and Zisserman textbook, Section 11.12.
(Revisiting triangulation, disparity, stereo rectification, stereo matching, improved stereo matching)
Basic reading:
- Szeliski textbook, Section 8.1 (not 8.1.1-8.1.3), Chapter 11, Section 12.2
- Hartley and Zisserman textbook, Section 11.12.
(Appearance phenomena, measuring light and radiometry, reflectance and BRDF)
Basic reading:
- Szeliski textbook, Section 2.2
- Steven Gortler, Foundations of Computer Graphics, Chapter 21. This book has a great introduction to radiometry, reflectance, and their use for image formation.
(Notes about radiometry, the n-dot-l model, photometric stereo, uncalibrated photometric stereo, generalized bas-relief ambiguity, shape from shading)
Basic reading:
- Szeliski textbook, Section 2.2
- Steven Gortler, Foundations of Computer Graphics, Chapter 21. This book has a great introduction to radiometry, reflectance, and their use for image formation.
(Imaging sensor primer, color sensing in cameras, in-camera image processing pipeline, radiometric calibration)
Basic reading:
- Szeliski textbook, Section 2.3
- Michael Brown, "Understanding the In-Camera Image Processing Pipeline for Computer Vision," CVPR 2016, very detailed discussion of issues relating to color photography and management, slides available here.
- Nine Degrees Below: amazing resource for color photography, reproduction, and management.
(Introduction to learning-based vision, image classification, bag-of-words, K-means clustering, classification, K-nearest neighbors, naive Bayes, support vector machines)
Basic reading:
- Szeliski textbook, Chapter 14
(Perceptron, neural networks, training perceptrons, gradient descent, backpropagation, stochastic gradient descent)
Basic reading (No standard textbooks yet!):
(Some notes on optimization, convolutional neural networks, training ConvNets)
Basic reading (No standard textbooks yet!):
(Intro to vision for video, optical flow, constant flow, Horn-Schunck flow)
Basic reading:
- Szeliski textbook, Section 8.4
(Motion magnification using optical flow, image alignment, Lucas-Kanade alignment, Baker-Matthews alignment, inverse alignment, KLT tracking, mean-shift tracking, modern trackers)
Basic reading:
- Szeliski textbook, Section 4.1.1, 5.3, 8.1
(Segmentation, image as a graph, shortest graph paths and intelligent scissors, GrabCut)
Basic reading:
- Szeliski textbook, Section 5.1.3, 5.3.1, 9.3.2, 9.3.3, 10.4.3