mint-lab/3dv_tutorial

An Invitation to 3D Vision: A Tutorial for Everyone

An Invitation to 3D Vision is an introductory tutorial on 3D computer vision (a.k.a. geometric vision or visual geometry or multiple-view geometry). It aims to help beginners understand the basic theories of 3D vision and implement practical applications using OpenCV. In addition to tutorial slides, example codes are provided for educational purposes. They include simple but interesting and practical applications. The example codes are kept as short as possible (mostly less than 100 lines) to be clear and easy to understand.

• To clone this repository (codes and slides): git clone https://github.com/mint-lab/3dv_tutorial.git
• To fork this repository to your GitHub account: Click here
• To download codes and slides as a ZIP file: Click here
• :memo: How to run example codes in Python
• :memo: How to run example codes in C++

Where does the name come from?

• The main title, An Invitation to 3D Vision, came from a legendary book by Yi Ma, Stefano Soatto, Jana Kosecka, and Shankar S. Sastry. We wish that our tutorial will be the first gentle invitation card for beginners to 3D vision and its applications.
• The subtitle, for everyone, was inspired by Prof. Kim's online lecture (in Korean). Our tutorial is also intended not only for students and researchers in academia, but also for hobbyists and developers in industry. We tried to describe important and typical problems and their solutions in OpenCV. We hope readers can understand it easily without a strong mathematical background.

Lecture Slides

• Section 1. Introduction
• Section 2. Single-view Geometry
• Section 3. Two-view Geometry
• Section 4. Solving Problems
• Section 5. Finding Correspondence
• Section 6. Multiple-view Geometry
• Special Topic) Bayesian Filtering
• Section 7. Visual SLAM and Odometry
• Section 8. 3D Representations

Example Codes

• Section 1. Introduction [slides]
• Section 2. Single-view Geometry [slides]
  • Getting Started with 2D
    • 3D rotation conversion [python]
  • Pinhole Camera Model
    • Object localization [python] [cpp]
    • Image formation [python] [cpp]
  • Geometric Distortion Models
    • Geometric distortion visualization [python]
    • Geometric distortion correction [python] [cpp] [result video]
    • Geometric distortion correction with virtual PTZ [python]
  • Camera Calibration
    • Camera calibration [python] [cpp]
  • Absolute Camera Pose Estimation (a.k.a. perspective-n-point; PnP)
    • Pose estimation (chessboard) [python] [cpp] [result video]
    • Pose estimation (book) [python] [cpp]
    • Pose estimation (book) with camera calibration [python] [cpp]
    • Pose estimation (book) with camera calibration without initial $K$ [python] [cpp] [result video]
• Section 3. Two-view Geometry [slides]
  • Planar Homography
    • Perspective distortion correction [python] [cpp]
    • Planar image stitching [python] [cpp]
    • 2D video stabilization [python] [cpp] [result video]
  • Epipolar Geometry
    • Epipolar line visualization [python]
  • Relative Camera Pose Estimation
    • Fundamental matrix estimation [python]
    • Monocular visual odometry (epipolar version) [python] [cpp] [result video]
  • Triangulation
    • Triangulation [python] [cpp]
• Section 4. Solving Problems [slides]
  • Solving Linear Equations in 3D Vision
    • Affine transformation estimation [python]
    • Planar homography estimation [python]
      • Appendix) Image warping using homography [python]
    • Fundamental matrix estimation [python]
    • Triangulation [python]
  • Solving Nonlinear Equations in 3D Vision
    • Absolute camera pose estimation [python]
    • Camera calibration [python]
    • Camera calibration (with lens distortion) [python]
• Section 5. Finding Correspondence [slides]
  • Feature Points and Descriptors
    • Harris corner (from scratch) [python]
    • SuperPoint [Github]
  • Feature Matching and Tracking
    • Feature matching comparison [python]
    • SuperGlue [Github]
    • Feature tracking with KLT tracker [python]
  • Outlier Rejection
    • Line fitting with RANSAC [python] [cpp]
    • Line fitting with M-estimator [python] [cpp]
    • Planar homography estimation with RANSAC [python]
• Section 6. Multiple-view Geometry [slides]
  • Bundle Adjustment [python] [cpp]
  • Structure-from-Motion (SfM)
    • Global SfM [python] [cpp]
    • Incremental SfM [python] [cpp]
• Section 7. Visual SLAM and Odometry [slides]
• Section 8. 3D Representations [slides]

License

• Beerware

Authors

• Sunglok Choi
• JunHyeok Choi

Acknowledgement

The authors thank the following contributors and projects.

• Jae-Yeong Lee: He motivated many examples.
• Giseop Kim: He contributed the initial version of SfM codes based on Toy-SfM and cvsba.
• Nell's Security: Their fisheye example image was used to demonstrate geometric distortion correction with virtual PTZ.
• Richard Blais: His book cover and video in the OpenCV tutorial were used to demonstrate camera pose estimation and augmented reality.
• Russell Hewett: His two hill images were used to demonstrate image stitching.
• Kang Li: His shaking CCTV video was used to demonstrate video stabilization.
• The KITTI Vision Benchmark Suite: The KITTI odometry dataset #07 was used to demonstrate visual odometry and SLAM.