A library of notes
Notes on LiDAR: how the sensor turns the flight time of laser light into a cloud of 3D points, and the algorithms that turn those points into maps, poses, objects and tracks. The crash course is a short textbook meant to be read in order, from the physics of a single laser pulse to registration, SLAM and learned 3D detectors. Alongside it, the site collects writeups of recent research papers, each linked back to the part of the course it builds on.
1 concept · 15 writeups · 1 sequence
Arrows run from what a document builds on to the document itself, so following them is a reading order. Dashed lines join documents that refer to one another.
Click a node for a preview · drag to rearrange · Ctrl/⌘ + scroll to zoom
Writeups meant to be read in order.
Everything, most recently changed first.
Concept· 8 min
Attention and transformersScaled dot-product attention, multi-head attention, the transformer block with residual connections and layer normalisation, absolute, relative and conditional positional encodings, vector attention, and local window attention with its efficient computation.
Writeup· 14 min
Point Transformer V3: Simpler, Faster, StrongerA companion to Wu et al.'s Point Transformer V3 (2024): a point-cloud transformer that serialises points along space-filling curves, attends within large patches of the sequence, and replaces relative positional encoding with a sparse convolution, trading the precision of each mechanism for the speed to scale.
Writeup· 14 min
CenterPoint: Center-based 3D Object Detection and TrackingA companion to Yin, Zhou and Krähenbühl's CenterPoint (2021): detecting LiDAR objects as heatmap peaks at their centres with every property regressed there, a cheap second stage on face-centre features, and tracking by greedy matching of velocity-compensated centres.
Writeup· 13 min
PointNet: Deep Learning on Point Sets for 3D Classification and SegmentationA companion to Qi, Su, Mo and Guibas's PointNet (2017): the first network to consume raw point sets, built from a shared per-point network and max pooling, with its alignment networks, its universal-approximation and critical-point-set theorems, and its experiments.
Writeup· 14 min
LOAM: Lidar Odometry and Mapping in Real-timeA companion to Zhang and Singh's LOAM (2014): real-time LiDAR odometry and mapping by splitting the problem into fast sweep-to-sweep odometry and slow scan-to-map mapping, both registering edge and planar feature points chosen by a smoothness measure along each scan line.
Writeup· 11 min
Deep learning on point cloudsHow networks consume point clouds (PointNet and PointNet++, voxel networks with sparse convolution, PointPillars, range-view networks), anchor- and centre-based detection heads, non-maximum suppression, the KITTI, nuScenes and Waymo metrics, LiDAR-specific training practice, and camera fusion.
Writeup· 8 min
Multi-object trackingTracking by detection: the Kalman filter and the motion models used for road users, gating by Mahalanobis distance, optimal assignment by the Hungarian algorithm, track management, LiDAR-specific difficulties, and the MOTA family of metrics.
Writeup· 7 min
Occupancy mapping and ray castingMaps of free, occupied and unknown space: the occupancy grid and its log-odds update, the inverse sensor model of a LiDAR beam, Amanatides–Woo ray traversal, OctoMap, removing moving objects from point maps, and signed distance fields.
Writeup· 10 min
LiDAR odometry and SLAMBuilding odometry from registration, LOAM and feature-based odometry, LiDAR–inertial fusion, pose-graph SLAM and loop closure with an interactive example, place recognition with Scan Context, localisation in prior maps, and the ATE and KITTI drift metrics.
Writeup· 11 min
Scan registration, from Kabsch to ICP and beyondAligning one point cloud to another: the closed-form Kabsch–Umeyama solution for known correspondences, ICP and the practical choices that make it work, point-to-plane ICP and degeneracy, Generalized-ICP, NDT, and global registration with descriptors and RANSAC.
Writeup· 8 min
Clustering and bounding boxesGrouping non-ground points into objects by Euclidean clustering, grid connected components, DBSCAN and range-image segmentation, and summarising each cluster by an oriented box found by L-shape fitting, or by a convex hull.
Writeup· 9 min
RANSAC and ground segmentationRobust model fitting with RANSAC (its algorithm, iteration count and variants) and the methods used to separate ground from obstacles in a scan: height thresholds, elevation grids, polar line fitting, range-image angles and region-wise plane fitting.
Writeup· 8 min
Filtering, normals and local featuresCropping and outlier removal, surface normals by principal component analysis and how to orient them and choose their scale, the shape features given by the covariance's eigenvalues, and keypoints and descriptors (FPFH) for matching points between clouds.
Writeup· 12 min
Coordinate frames, rigid transforms and calibrationCoordinate frames, rotations (matrices, Euler angles, axis–angle, quaternions), rigid transforms in SE(3) and the exponential map, and the operations built on them: extrinsic calibration, projection into cameras, range images and bird's-eye grids, and deskewing a moving scan.
Writeup· 8 min
Spatial data structures and neighbour searchWhy neighbour queries dominate point-cloud processing, and the structures that answer them quickly: voxel grids and spatial hashing (with voxel downsampling), k-d trees, octrees and Morton codes, and the free neighbourhoods of a range image.
Writeup· 32 min
How LiDAR works, from photons to point cloudsThe physics and electronics of a LiDAR measurement (time of flight, the LiDAR equation, emitters, detectors, FMCW and beam steering) and the structure of the data it produces: range images, point clouds, their density, occlusion, motion distortion and artefacts.