How LiDAR works, from photons to point clouds

A LiDAR (light detection and ranging) sensor measures distance by timing light. It fires a short pulse of laser light, waits for the echo, and converts the round-trip time into a range; by steering the beam across a field of view many times a second it builds up a three-dimensional sample of its surroundings, the point cloud. This writeup explains how that happens: the physics of the measurement, the ways a beam is steered, the electronics that make the measurement, the sensors on the market and how they keep time, and, most importantly for everything that follows in the course, the structure, density and defects of the data that comes out. Every algorithm in the later writeups is shaped by the facts established here: that the data is a sparse, irregular sample of surfaces, organised by beam and by time, that thins with distance and is riddled with occlusion shadows. A standard reference for the hardware is [McMa19].

Ranging by time of flight

Light travels at $c \approx 2.998 \times 10^8$ m/s. If a pulse leaves the sensor at time $0$, reflects off a surface at range $r$ and arrives back at time $\Delta t$, it has travelled $2r$, so

The numbers set the engineering problem. One nanosecond of round-trip time corresponds to $15$ cm of range, so a target at $100$ m returns its echo after about $667$ ns, and resolving range to $1$ cm requires timing to about $67$ ps. That precision is achieved not with a fast clock alone but with a time-to-digital converter (TDC), a circuit, often a chain of delay elements on a chip, that interpolates between clock edges to tens of picoseconds.

Time of flight also bounds how fast a single laser may fire. If a second pulse leaves before the echo of the first has returned, the sensor cannot tell which pulse an echo belongs to. A laser fired at a pulse repetition frequency $f$ therefore has a maximum unambiguous range

so a laser fired at $1$ MHz is unambiguous only to $150$ m. Sensors that need more points per second than one laser can give at the range they need use many lasers firing in parallel, which is one reason the multi-beam designs described below exist.

The emitter

The light source is almost always a pulsed semiconductor laser, emitting pulses a few nanoseconds long at a peak power of tens of watts, though at an average power of milliwatts, since it is dark nearly all the time. Two wavelength bands dominate.

• Around 905 nm (and nearby, such as 850–940 nm). Edge-emitting laser diodes and arrays of vertical-cavity surface-emitting lasers (VCSELs) at these wavelengths are cheap, and ordinary silicon detectors absorb them. The cost is eye safety: light in this band is focused by the eye onto the retina, so the international standard for laser safety (IEC 60825-1, whose “Class 1” every automotive LiDAR must meet) caps the pulse energy, and with it the range.
• Around 1550 nm. Light at this wavelength is absorbed in the cornea and the fluid of the eye before it reaches the retina, which allows a far higher pulse energy at the same safety class, and so a longer range. Silicon is transparent here, so the detector must be made of indium gallium arsenide (InGaAs), which is considerably more expensive, and the source is usually a fibre laser.

The beam leaves the sensor through optics that collimate it, but no beam is perfectly parallel: it spreads with a divergence of typically a few tenths of a milliradian to a few milliradians. At divergence $\gamma$ the illuminated footprint at range $r$ is about $\gamma r$ across, so a $3$ mrad beam lights a patch $30$ cm wide at $100$ m. A footprint of that size is larger than many of the details of a scene, and a footprint that falls across an edge is the origin of several of the artefacts described below.

The receiver

The echo is collected by a lens and focused onto a photodetector behind a narrow optical bandpass filter, which admits the laser’s wavelength and rejects most of the sunlight. Four kinds of detector are in use, in increasing order of sensitivity.

• A PIN photodiode produces a current proportional to the optical power falling on it, with no internal gain.
• An avalanche photodiode (APD) is biased so that each photo-generated electron is accelerated hard enough to knock loose others, giving an internal gain of roughly tens to hundreds while remaining linear: the output current is still proportional to the power received.
• A single-photon avalanche diode (SPAD) is biased beyond its breakdown voltage, in Geiger mode, so that a single photon triggers a self-sustaining avalanche, a digital click. A SPAD cannot tell one photon from many and is blind for a dead time of some nanoseconds after each click, so it is read out statistically: over many laser shots, the clicks are accumulated into a histogram of arrival times (time-correlated single photon counting) whose peak marks the echo.
• A silicon photomultiplier (SiPM) is an array of many SPADs summed into one output, which restores a rough proportionality to the number of photons arriving.

For linear detectors, the photocurrent is turned into a voltage by a transimpedance amplifier and then either compared against a threshold, with the TDC timestamping the crossing, or digitised by a fast analogue-to-digital converter so that the whole waveform can be processed. Timing a fixed threshold has a flaw: a strong echo rises through the threshold earlier than a weak echo from the same range, so the measured range depends on the target’s reflectivity. This range walk is corrected either by calibration against the echo’s amplitude or by constant-fraction discrimination, which times the moment the pulse reaches a fixed fraction of its own peak rather than a fixed level.

How much light comes back

A target far from the sensor returns a tiny fraction of the light sent to it, and the fraction decides how far a sensor can see. Suppose a pulse of peak power $P_t$ falls entirely on a matte surface at range $r$, at an angle of incidence $\alpha$ to its normal, and the surface reflects a fraction $\rho$ of it, its reflectivity. A matte surface is well modelled as Lambertian: it scatters the reflected power $\rho P_t$ over the hemisphere above it with a radiant intensity of $(\rho P_t / \pi)\cos\beta$ watts per steradian in a direction at angle $\beta$ to its normal. The receiver sits back along the incoming beam, so $\beta = \alpha$, and its aperture of area $A_r$ subtends a solid angle of $A_r / r^2$. The received power is therefore

where $\eta$ collects the efficiency of the optics and $T$ is the one-way transmission of the atmosphere, squared because the light crosses it twice. This is the LiDAR equation for an extended target, one large enough to intercept the whole beam. It has three consequences that matter throughout the course.

The return falls with the square of the range, so doubling the distance to a target quarters its signal. A pulse of $75$ W peak, reflected by a surface of $10\%$ reflectivity at $100$ m into a receiver aperture $25$ mm across ($A_r \approx 4.9 \times 10^{-4}\ \mathrm{m}^2$) with $\eta = 0.5$, returns about $60$ nW, which over a $5$ ns pulse is of the order of a thousand photons at $905$ nm. Sensors are accordingly specified by their range at a stated reflectivity, typically $10\%$, roughly that of a dark car or a tyre; a sensor that sees a white wall at $200$ m may see a black car only at $80$ m.

The return depends on reflectivity and on the angle of incidence as well as on range, so the strength of an echo, which sensors report as intensity, is not a property of the surface alone. Many sensors report a calibrated reflectivity instead, having divided out the range and their own response; even so, a surface seen obliquely returns less than the same surface seen head-on. Intensity is nevertheless very useful: lane markings and road signs are coated with retroreflective paint that returns far more light than asphalt, so they stand out sharply, and maps of intensity are used to localise vehicles.

Finally, weather enters through $T$. Fog, rain, snow, dust and exhaust both attenuate the beam and scatter some of it straight back, producing returns from the air itself.

Noise and precision

The echo competes with noise: the shot noise of the signal photons themselves, background light (above all sunlight, which the bandpass filter reduces but cannot remove), and the thermal noise of the electronics. The ratio of signal to noise governs two separate things. It decides whether an echo is detected at all, through a trade between the probability of detection and the rate of false alarms that is set by the threshold. And it decides how precisely an echo is timed. A useful rule of thumb is that the timing jitter is about the rise time of the pulse divided by the signal-to-noise ratio, so the range precision is about

which is why a weak echo from a distant, dark target is not only less likely to be detected but noisier when it is. Typical automotive sensors quote a range precision of one to three centimetres, degrading at long range.

Frequency-modulated continuous wave

Everything above concerns pulsed (or direct) time of flight, which is what the great majority of sensors use. The main alternative, frequency-modulated continuous wave (FMCW) LiDAR, does not time a pulse at all. It transmits continuously while sweeping its optical frequency linearly, a chirp of bandwidth $B$ over a duration $T_c$, and it measures range as a difference of frequencies. Understanding how requires one fact about what happens when two light waves fall on the same detector.

Mixing and beat frequencies

A photodetector does not respond to the electric field of light, which oscillates at about $193$ THz at a wavelength of $1550$ nm, far faster than any electronics can follow. It responds to the light’s power, which is proportional to the square of the field, averaged over the detector’s response time of a nanosecond or so. That squaring is what makes two waves interact.

Suppose two waves reach the detector together: a strong wave $A\cos(\omega_1 t)$, a copy of the outgoing laser light called the local oscillator, and a weak echo $a\cos(\omega_2 t + \phi)$ with its own amplitude, frequency and phase. The fields add, and the detector sees the square of the sum,

The product-to-sum identity $2\cos x\cos y = \cos(x - y) + \cos(x + y)$ splits the last term in two, and $\cos^2 x = \tfrac{1}{2}(1 + \cos 2x)$ does the same for the first two. Every resulting term oscillates either at an optical frequency ($2\omega_1$, $2\omega_2$ or $\omega_1 + \omega_2$), which the detector averages to zero, or not at all, except one. What the detector reports is

Nothing cancels. The two waves add, and since their frequencies differ slightly they drift steadily in and out of step: where their crests coincide the total power is high, where a crest meets a trough it is low. The power therefore swells and fades at the difference of the two frequencies, the beat frequency, just as two tuning forks slightly out of tune produce a sound that throbs. The echo’s amplitude and phase appear in (3), but only in how deep the throb is ($Aa$) and when it peaks ($\phi$); its rate depends on the frequencies alone. The weakness of the echo is in fact turned to advantage, since the throb’s depth is the echo’s amplitude multiplied by the strong local oscillator’s, an amplification called heterodyne gain. This mixing of a signal with a reference to measure their difference is coherent (or heterodyne) detection, the same principle by which a radio receiver tunes to a station.

The figure below lets the echo’s amplitude and phase be chosen and shows their effect. It describes each wave by a phasor, an arrow whose length is the wave’s amplitude and whose angle is its phase, so that the echo $a\cos(\omega_2 t + \phi)$ is the arrow $a e^{\rmi\phi}$ drawn against the local oscillator’s arrow $A$ along the horizontal axis. Adding two waves of the same frequency is adding their arrows. Two waves of slightly different frequencies are like two arrows turning at slightly different rates, so relative to the local oscillator’s arrow the echo’s turns once per beat, and the length of their sum, whose square is the power, swings between $A - a$ and $A + a$: this is (3) seen geometrically.

amplitude 0.45 phase 63°

beat depth Aa = 0.45 · beats in the window: 4, whatever the phasor

• local oscillator
• echo
• instantaneous power
• what the detector reports

Range from a chirp

FMCW applies this to a chirp. The outgoing light’s frequency rises at the steady rate $B / T_c$. The echo arriving now left the sensor one round-trip time $2r/c$ ago, when the frequency was lower by $(B / T_c)(2r/c)$. Both waves are rising at the same rate, so this difference stays constant through the chirp, and mixing the echo with the current outgoing light gives a beat at

which is proportional to the range. A chirp of $B = 1$ GHz over $T_c = 10\ \mu$s and a target at $150$ m give $f_b = 100$ MHz: far below the optical frequencies, and easily measured by digitising the detector’s output and taking its Fourier transform, in which each target along the beam appears as a peak. If the target moves along the beam at radial velocity $v$, the echo is also Doppler-shifted by $f_D = 2v / \lambda$, which is $1.29$ MHz for every metre per second at $\lambda = 1550$ nm. Alternating rising and falling chirps separates the two effects, since the Doppler shift adds to the beat frequency on one and subtracts on the other.

FMCW therefore measures the radial velocity of every point directly, which a pulsed sensor can only infer by comparing successive frames, and its coherent detection rejects sunlight and the pulses of other LiDARs, since light that is not at almost exactly the local oscillator’s frequency produces a beat far outside the band the receiver listens to. Its range resolution is set by the chirp bandwidth, $\Delta r = c / (2B)$, so $1$ GHz gives $15$ cm. The cost is the complexity of a narrow-linewidth laser and coherent optics, and FMCW sensors usually work at 1550 nm.

Steering the beam

A single range measurement is one point. To sample a scene, the sensor must point its measurements across a field of view, and how it does so decides the shape of the data.

• Spinning multi-beam sensors mount a vertical column of $N$ lasers and detectors, typically $16$, $32$, $64$ or $128$ channels, on a head that rotates about a vertical axis $10$ or $20$ times a second. Each channel sweeps a cone of fixed elevation, so the sensor sees $360°$ horizontally and a vertical field of view of perhaps $30°$ to $45°$. The Velodyne HDL-64E, which recorded the KITTI benchmark [Geig12], is of this kind. The elevations of the channels are often spaced unevenly, concentrated near the horizon where distant objects are.
• Mechanical scanners with mirrors keep the lasers fixed and move a mirror: a rotating polygon, an oscillating galvanometer, or a microelectromechanical (MEMS) mirror a few millimetres across. They usually cover a forward-facing field of view of about $120°$ and scan it in a raster or along a figure-of-eight pattern.
• Rotating prisms, a pair of wedges spinning at different rates, trace a rosette that never repeats, so the scan fills the field of view more densely the longer it is integrated.
• Flash LiDAR has no moving parts. It illuminates the whole field of view at once with a diffused pulse and images the echo onto a two-dimensional array of detectors, like a camera that measures time instead of brightness; arrays of SPADs make this practical. Some sensors are a hybrid, with a column of SPADs or a column of illumination swept across the scene.
• Optical phased arrays steer the beam electronically by controlling the phase of light across an array of emitters, as phased-array radar does, though they remain largely a research technology.

Whatever the mechanism, every measurement has a direction known from the geometry of the sensor (which channel fired, and the angle the head or mirror was at) and a range from the echo.

Inside the sensor

The sections above describe the physics; this one describes the electronics that put it to work. However it is built, a LiDAR has the same few jobs to do: keep time, fire the laser, turn the faint echo into an electrical signal, measure when it arrived, point the beam, and package the results. The figure shows how these jobs divide into blocks for the two architectures, and the paragraphs after it take the blocks in turn.

Pulsed time of flightFMCW

• electrical signal
• light

The reference clock

Every time measurement in the sensor is ultimately a count of the ticks of one clock. That clock begins as a quartz crystal oscillator, typically at a few tens of megahertz, often temperature-compensated (a TCXO) so that its frequency barely drifts as the sensor warms up. A phase-locked loop (PLL) multiplies its frequency to the hundreds of megahertz or more at which the digital logic runs. Two properties of this clock matter, and they matter very differently.

Its accuracy, how close its frequency is to the nominal value, matters little for ranging. A crystal off by $50$ parts per million makes every measured time, and so every range, $50$ parts per million too long or too short: $5$ mm at $100$ m, below the noise of a typical sensor, and removable by calibration anyway. Its jitter, the random wander of each tick about where it should be, matters much more, because it adds directly to the timing noise of every measurement: by (1), a jitter of $10$ ps is $1.5$ mm of range noise. Sensors therefore use low-jitter oscillators and PLLs, and the clock is distributed with the care given to a radio’s local oscillator.

The controller

At the centre sits a field-programmable gate array (FPGA), or, in sensors made in volume, a custom application-specific integrated circuit (ASIC) or system-on-chip that does the same work at lower cost and power. It schedules the firings, deciding which channel fires when, often with small random offsets so that the pulses of other LiDARs do not line up with its own; it reads the scanner’s angle; it runs the time measurement; it applies the factory calibration, correcting range walk and each channel’s angles and offsets; it computes intensity or reflectivity; and it packs the results into network packets. The time-to-digital converters described earlier are often built inside the FPGA itself, from its chains of fast carry logic used as tapped delay lines, which can resolve intervals of ten or twenty picoseconds.

The laser driver

A pulsed laser diode needs a current of tens of amperes for a few nanoseconds. The driver stores charge in a capacitor and dumps it through the laser with a fast switch, today usually a gallium nitride (GaN) transistor, which switches far faster than a silicon one. The difficulty is inductance. By $V = L\,\mathrm{d}I/\mathrm{d}t$, driving $40$ A in $2$ ns through a mere $1$ nH of wiring takes $20$ V just to overcome the wiring, so the capacitor, switch and laser are packed millimetres apart, and the layout of this small circuit decides how short and how strong the pulses can be. Arrays of VCSELs are driven in banks, a group at a time, which is how sensors with many channels fire them in sequence.

The receiver chain

The photodetector needs a bias voltage, and the kind of voltage depends on the kind of detector. An APD is biased at tens to a few hundred volts, and its gain depends steeply on temperature, so a regulated supply adjusts the bias as the sensor warms. A SPAD is biased a few volts beyond its breakdown voltage, and each one has a quenching circuit that stops the avalanche after a click and re-arms the diode.

For linear detectors (PIN photodiodes and APDs), a transimpedance amplifier (TIA) converts the microamperes of photocurrent into a voltage. It must be fast enough to preserve a nanosecond pulse, which means a bandwidth of hundreds of megahertz, yet quiet enough not to bury a weak echo, and the two conflict, since noise grows with bandwidth. After the TIA the signal takes one of two routes. In most sensors, a comparator produces a digital edge when the signal crosses the threshold, and the controller timestamps the edge, and often the falling edge too, since the time spent above threshold indicates the echo’s strength. In full-waveform sensors, a fast analogue-to-digital converter (ADC), sampling at around a gigasample per second, records the whole echo, and the controller fits the shape of each pulse, which separates overlapping returns and times them more precisely; survey scanners work this way.

SPAD-based sensors integrate the whole receiver on one CMOS chip, often two chips stacked and bonded face to face: an array of SPADs with their quenching circuits on one layer, and on the other, beneath them, the TDCs and memory that accumulate each pixel’s histogram of photon arrival times. The output of such a chip is already digital, and its manufacturing cost falls the way any chip’s does with volume, which is a large part of why these sensors have become cheap.

Steering, power and housekeeping

A spinning sensor has a brushless motor with a speed-control loop and an optical encoder that reports the head’s angle to a fraction of a degree. Power and data must cross the rotating joint, either through a slip ring or contactlessly, with power transferred inductively and data by an optical or radio link. MEMS scanners instead have a mirror driver and a sensor of the mirror’s position. Around these sit the less glamorous circuits on which safety and reliability depend: interlocks that stop the laser if the scanner stops, since a beam that is not moving concentrates its energy on one spot; heaters that clear frost from the window; detectors for dirt on the window; and temperature monitoring throughout.

The interface

Points leave the sensor as UDP packets over Ethernet, at one hundred megabits or a gigabit per second, or, in cars, over the single-pair automotive Ethernet standards. The same connector, or a separate one, carries the inputs for time synchronisation described below.

What changes for FMCW

An FMCW sensor shares the controller, clock, scanner and interface, but its optical and receiving electronics are those of a coherent radio, and considerably more demanding.

• The laser must be narrow-linewidth. Its frequency must stay coherent over the whole round trip, since the echo is compared with light emitted a round-trip time later; the laser’s random phase drift over that interval smears the beat into a broad peak. The time over which a laser of linewidth $\Delta\nu$ stays coherent is about $1/(\pi\Delta\nu)$, so a round trip to $300$ m, of $2\ \mu$s, needs a linewidth below about $160$ kHz, where an ordinary semiconductor laser’s is megahertz.
• The chirp is produced by modulating the laser’s drive current, or with a separate optical modulator, under the control of a digital-to-analogue converter driven by the controller. No laser chirps perfectly linearly, and a curved chirp spreads each target’s beat over a range of frequencies. So a small fraction of the light is sent through a reference interferometer, two paths of a known difference in length, whose own beat records the chirp actually produced, and the measured signal is resampled to undo the nonlinearity.
• The optics include an optical amplifier, an erbium-doped fibre amplifier or a semiconductor optical amplifier, to raise the transmitted power, and a circulator that sends outgoing light to the target and returning light to the receiver through the same aperture.
• The receiver mixes the echo with the local oscillator in an optical hybrid and detects the result with balanced photodiodes, pairs whose outputs are subtracted, which cancels the local oscillator’s own intensity noise and keeps the beat term of (3). Two such pairs, a quarter of a cycle apart, give the in-phase and quadrature components of the beat, which reveal its sign and so distinguish a target approaching from one receding.
• The digitisation is heavy: each channel’s beat signal is sampled at hundreds of megasamples per second, and the controller computes a Fourier transform for every measurement and finds its peaks. This processing load, far larger than timing an edge, is why FMCW sensors depend on powerful FPGAs or dedicated ASICs.

Much of this optical circuitry can be built in silicon photonics, with waveguides, splitters and detectors fabricated on a chip like an integrated circuit, which is how FMCW sensors are being made small and cheap enough for cars.

Sensors on the market

LiDARs span five orders of magnitude in price, and the differences are in range, resolution, robustness and certification rather than in principle. The examples below illustrate the spread; prices are approximate retail prices at the time of writing, and they change quickly.

Class	Example	Principle	Approximate price	Output and synchronisation
Proximity chip	ST VL53L1X	SPAD time of flight, one zone	a few dollars	I²C; none
Single-point rangefinder	Benewake TF-Luna	time of flight, 8 m	about $25	UART or I²C; none
2D scanner	Slamtec RPLIDAR A1	triangulation, 12 m, 360°	about $100	serial over USB; none
3D robotics	Livox Mid-360	time of flight, non-repetitive scan	about $1,000	Ethernet, built-in IMU; PTP, gPTP, GPS
3D spinning	Velodyne VLP-16, Ouster OS series	16 to 128 channels	thousands of dollars	Ethernet; GPS PPS, PTP, sync pulses
Automotive	Hesai AT128	128-channel hybrid solid state	hundreds of dollars in volume	Ethernet
Coherent	Aeva Aeries II	FMCW with per-point velocity	not published	Ethernet
Survey	RIEGL VUX series	full-waveform, MHz pulse rates	by quotation	Ethernet; GNSS and IMU integration, camera control

At the bottom, the proximity sensors in phones and robots are single chips holding a VCSEL, a small SPAD array and the timing electronics, reporting one distance or a coarse grid of a few zones over I²C, with a range of a few metres. Single-point rangefinders such as the TF-Luna add optics and a microcontroller to measure one distance up to about $8$ m at up to a few hundred readings a second, and are used for altitude-hold in drones and for simple obstacle detection.

Hobbyist 2D scanners such as the RPLIDAR A1 spin a single beam to map a plane, enough for a robot vacuum or a student robot to build a 2D map. Many of them, the A1 included, do not measure time of flight at all but triangulate: a small camera beside the laser sees the laser spot, and the spot’s position in the image, which shifts with distance, gives the range. Triangulation needs no picosecond electronics, but its error grows with the square of the range, which limits it to about ten metres.

3D sensors for robotics begin around a thousand dollars. The Livox Mid-360 scans a non-repetitive pattern over $360°$ horizontally and $59°$ vertically, sends points over Ethernet and includes an IMU, convenient for the LiDAR–inertial odometry described later in the course. Above it sit the spinning multi-beam sensors. The Velodyne VLP-16 “Puck”, with sixteen channels, launched in 2016 at about $8,000 and was cut to about $4,000 in 2018, and remains one of the most common sensors in research datasets. Velodyne merged with Ouster in 2023; Ouster’s OS series, built on SPAD receiver chips, offers 32 to 128 channels in short-, medium- and long-range variants, with prices generally quoted rather than listed.

Automotive sensors are a different market. A sensor built into a production car must survive a decade of heat, cold, vibration and car washes, meet functional-safety standards, and be made by the hundred thousand, and in such volumes prices have fallen remarkably: Hesai has said its next-generation automotive sensor would sell for under $200, half the price of its AT128, while the same AT128 bought singly as a development unit costs thousands. Coherent sensors such as Aeva’s Aeries II bring FMCW, with a velocity for every point, to the same market. At the top, survey scanners such as RIEGL’s VUX series, mounted on aircraft and drones to map terrain and forests, digitise full waveforms at pulse rates of over a megahertz and are sold as part of systems integrated with survey-grade GNSS receivers and IMUs.

Clocks and synchronisation

Communications radios depend on precise frequency references because their transmitter and receiver are separate devices with separate oscillators, which must agree on frequency and phase before a signal can be decoded. A LiDAR is monostatic: its transmitter and receiver sit in one box and share one clock, so every range is a comparison of the sensor with itself. Ranging therefore needs no external reference, and every sensor carries its own clock. As shown above, even an inaccurate crystal costs only millimetres; what ranging needs is low jitter, which is internal.

The same holds for coherence in an FMCW sensor. The local oscillator is a tap of the very laser that produced the outgoing light, so the measurement is self-referenced, and what matters is the laser’s coherence over one round trip, a property of the laser alone. No coherence with any other device is needed, or possible.

Where an external time source matters is in timestamps. A point is only useful once placed in the world, which requires knowing where the vehicle was when the point was measured, and that pose comes from other sensors (an IMU, wheel encoders, a GNSS receiver, cameras) with clocks of their own. A free-running crystal off by $20$ parts per million drifts by $1.7$ s a day, so two devices left to their own clocks quickly disagree. And small disagreements are costly: a timing error of one millisecond on a vehicle moving at $30$ m/s misplaces every point by $3$ cm, and on a vehicle turning at $30°$ per second it rotates the scan by $0.03°$, which is another $2.6$ cm at $50$ m. Deskewing, odometry and fusion, all described later in the course, depend on timestamps that agree across sensors to well under a millisecond.

Sensors therefore accept an external time source in one or more of three ways.

• GPS time arrives as two signals from a GNSS receiver: a pulse per second (PPS), an electrical edge marking the start of each second to within tens of nanoseconds, and a serial message (an NMEA $GPRMC sentence) saying which second it was. The VLP-16, for instance, takes both through its interface box, and stamps its packets with time since the top of the hour.
• PTP (the Precision Time Protocol, IEEE 1588) synchronises clocks over the Ethernet network itself: a grandmaster clock exchanges timestamped messages with each device, which measures and removes the network’s delay, reaching microsecond agreement or better when the network hardware timestamps the messages. Its generalised profile, gPTP (IEEE 802.1AS), is the one used on automotive Ethernet networks. Ouster’s and Livox’s sensors support PTP, and the Mid-360 accepts PTP, gPTP or GPS, preferring PTP when more than one is present.
• Sync pulses in and out tie the sensor to other hardware directly. A spinning sensor can be phase-locked to the PPS, so that it points in a chosen direction at the top of each second and every scan begins at a predictable instant; several LiDARs can then be locked so that their scans align, and a camera can be triggered at the moment the LiDAR’s beam sweeps through the camera’s field of view, so that the two observe the same instant. Ouster’s sensors can both accept a sync pulse and emit one derived from PTP.

Without any of these, a sensor stamps its data with its own time since power-on, and the computer receiving it must estimate the offset, at best by stamping packets on arrival, which adds the jitter of the network and the operating system. That is adequate for a hobby robot and inadequate for a car. In short, a LiDAR does not need an external reference to measure distance, but a system built around LiDAR needs one to know when each distance was measured.

The structure of the data

The rest of this writeup concerns what a sensor of the most common kind, a spinning multi-beam sensor, actually produces, since that data is what the rest of the course processes. Other scan patterns differ in detail but not in kind.

The raw measurement

Each time a channel fires, the sensor records a small tuple: the channel index $i$ (the ring or beam), the azimuth $\varphi$ of the rotating head read from an encoder, the range $r$ of each detected return, its intensity, and a timestamp. These are streamed over Ethernet in packets, each holding a block of consecutive firings, at rates of hundreds of thousands to a few million measurements per second. A sensor with $128$ channels taking $1024$ azimuth steps per revolution at $10$ Hz produces $128 \times 1024 \times 10 \approx 1.3$ million measurements per second, and twice that if it reports two returns per firing.

The direction of a measurement is given in spherical coordinates: the azimuth $\varphi$ about the vertical axis and the elevation $\theta_i$ of channel $i$, fixed by the sensor’s construction and listed in its calibration. The point in the sensor’s own Cartesian frame (by the usual convention $x$ forward, $y$ left and $z$ up) is

In a real sensor the channels do not all emit from the same point, so the manufacturer’s calibration also gives each channel small horizontal and vertical offsets, and an azimuth correction, that are applied in the same conversion. Computing (4) is the first step of every LiDAR driver.

The ring index

The channel index $i$ is usually stored with each point under the name ring. It is an integer from $0$ to $N - 1$ saying which of the sensor’s $N$ lasers produced the point. The name comes from what a channel draws: as the head spins, each laser sweeps a cone of fixed elevation $\theta_i$, which meets flat ground in a circle round the sensor, so the points of one channel form one ring in the bird’s-eye view of the scan, as in the figure below. The ring is not the radius. Points on one ring lie at every distance, wherever the cone happens to meet a surface: on the ground at $h / \tan|\theta_i|$, on a car much nearer, on a wall farther away.

It may seem that the ring could be recomputed from the coordinates, since by (4) the elevation of a point is $\arcsin(z / r)$, and that should identify the channel. In practice this fails, which is why sensors report the ring explicitly. Adjacent channels may be only a few tenths of a degree apart, and the per-channel offsets mean that a channel’s points do not all lie at exactly the elevation $\theta_i$ seen from the origin; a sensor mounted with a tilt, or a cloud already transformed into the vehicle’s frame, scrambles elevations altogether; and range noise blurs the computed angle of near points. The ring is a fact about how the point was measured, recorded at the moment of measurement, and much processing relies on it: it is the row of the range image, it defines the scan lines along which the LOAM odometry of a later writeup extracts its features, it lets ground segmentation compare each point with the one measured by the channel below it, and it selects the per-channel calibration applied to intensity. Drivers disagree on its direction (ROS’s Velodyne driver numbers rings from the lowest beam upwards, while Ouster numbers its beams from the top down), so the convention must be checked for each sensor.

Range images and point clouds

The measurements of one revolution, a scan or sweep or frame, can be stored in two equivalent ways, and practitioners move between them constantly.

The range image keeps the sensor’s own organisation: an $H \times W$ array, one row per channel and one column per azimuth step, each pixel holding a range (and, in further channels, intensity and anything else measured). It is exactly a depth image from a camera with a cylindrical lens. Neighbouring pixels are neighbouring measurements, so the image gives every point its neighbours for free, and image-processing operations and 2D convolutional networks apply to it directly. A firing that got no echo, because it pointed at the sky, at a target beyond range, or at a surface that absorbed or mirrored the light away, leaves a pixel empty, conventionally a range of $0$.

The point cloud is the list of the $N$ points that do exist, as an $N \times K$ array whose columns are the coordinates $x, y, z$ from (4) together with whatever else is kept for each point: intensity, ring, return number, timestamp. The Point Cloud Library [Rusu11] calls a cloud that keeps the image layout, with a width and a height and invalid points marked as NaN, an organised point cloud, and a flat list an unorganised one. Once clouds from several scans or several sensors are merged, or a cloud is filtered or downsampled, the image layout is lost, and a cloud is then just a set of points with no inherent order. Finding which points are near which then requires the spatial data structures of a later writeup.

16 beams32 beams64 beams128 beams

• ground
• car
• car
• pedestrian
• pole
• building

In practice point clouds travel in a handful of formats. In ROS, the robotics middleware most research and much industry code is built on, a cloud is a sensor_msgs/PointCloud2 message: a binary blob of fixed-size records together with a list of fields giving each one’s name, byte offset and type, plus a height and width that are $1 \times N$ for an unorganised cloud. The Point Cloud Library’s .pcd files have a short text header with the same information. The KITTI benchmark stores each scan as a raw .bin file of 32-bit floats, four per point: $x$, $y$, $z$ and reflectance. Airborne and survey LiDAR uses the ASPRS .las format (and its compressed form .laz), whose records carry GPS time, the return number and the number of returns of each shot, and a classification code such as ground, vegetation or building.

Sampling density

A LiDAR samples the world on a grid of directions, not of positions, and that one fact explains most of what makes its data awkward. Two adjacent firings of the same channel are separated by the azimuth step $\Delta\varphi$, so at range $r$ the points they produce are about $r\,\Delta\varphi$ apart, and the density of points on a surface facing the sensor falls as $1/r^2$. At an azimuth step of $0.35°$ ($1024$ steps per revolution), adjacent points are $6$ cm apart at $10$ m and $60$ cm apart at $100$ m, where a pedestrian may be covered by a few dozen points or fewer.

On the ground the effect is more extreme in the radial direction. A channel pointing below the horizon at an angle $|\theta|$, on a sensor at height $h$ above flat ground, meets the ground on a circle of radius $d = h / \tan|\theta|$. Differentiating, two channels separated by $\Delta\theta$ meet the ground at radii that differ by

growing with the square of the distance. With $h = 1.8$ m and channels $0.35°$ apart, the rings on the ground are $1.4$ m apart at $20$ m, while the points along each ring are $12$ cm apart. The ground is sampled by concentric rings, densely along each ring and sparsely across them, which the figure above makes plain; algorithms that look for neighbours within a fixed radius, or that fit planes to the ground, must allow for this anisotropy.

Occlusion

A LiDAR sees only the first surface along each beam. Everything behind an object, out to the sensor’s range, is unobserved: the object casts a shadow of empty space, as the cars and the pole do in the figure. An empty region in a point cloud therefore means either free space or unobserved space, and telling the two apart, which matters a great deal for deciding where a vehicle can drive, requires reasoning along the rays from the sensor rather than about the points alone. That reasoning is the subject of occupancy mapping.

Time and motion

A spinning sensor does not take its scan at one instant. A revolution at $10$ Hz takes $100$ ms, and a vehicle travelling at $15$ m/s moves $1.5$ m in that time, so the first and last columns of a scan are measured from positions $1.5$ m apart, and the scan is distorted, much as a rolling-shutter camera distorts a moving scene. A static wall appears bent, and where the scan’s start and end meet, a seam appears in which the same surface is recorded twice at different positions. This is why every point carries a timestamp, and why correcting the distortion, called deskewing or motion compensation, using the vehicle’s motion during the sweep, is a standard step of LiDAR processing. Objects that move during the sweep are distorted in the same way, by their own motion.

Imperfections of the data

Real point clouds contain several characteristic artefacts, and knowing their causes is part of knowing the sensor.

Mixed pixels. When a footprint straddles the edge of an object, part of the pulse returns from the object and part from whatever lies behind it. If the two echoes are far apart in time, the sensor reports two returns, as in the waveform figure above. If they are close, they merge into one distorted pulse whose detected time falls between them, and the sensor reports a point floating in the empty space between foreground and background. These edge points or veiling points trail behind the silhouettes of objects.

Multiple returns. Because one shot can yield several echoes, sensors can be configured to report the strongest return, the last, or two of them. The first return of a shot through foliage, rain or a fence comes from the obstruction and the last from the surface behind it; airborne surveys rely on last returns to map the ground beneath a forest canopy.

Retroreflectors and blooming. A retroreflective sign or number plate returns so much light that it can saturate the detector, biasing its range, and leak into neighbouring channels through stray light inside the sensor, so that a halo of spurious points, blooming, surrounds it.

Specular surfaces and multipath. Glass, polished paint, puddles and wet roads reflect light like mirrors rather than scattering it. A mirror-like surface seen obliquely returns nothing, leaving a hole, while one that bounces the beam onto a matte surface and back produces a point at the length of the whole path, which appears behind a window or beneath the road surface.

Weather and atmosphere. Rain, snow, fog, dust and exhaust produce sparse returns from the air near the sensor, at short range and low intensity, while attenuating the returns from everything beyond.

Interference. Pulses from another LiDAR can be mistaken for echoes, producing isolated false points. Sensors defend against this by randomising the timing of their pulses or encoding them, so that foreign pulses do not line up from one shot to the next.

Each of these is the reason for some filtering step that later writeups describe, and each is a reason why a LiDAR point cloud, unlike a synthetic one, is never a clean sample of a surface.

Summary

A LiDAR measures, many times a second, the range along known directions by timing laser light, with (1) relating time to range and (2) deciding how far it can see. A spinning multi-beam sensor delivers these measurements organised as a range image, one row per channel and one column per azimuth step, and the conversion (4) turns them into a point cloud of $x, y, z$ with intensity, ring and timestamp. The resulting data samples surfaces, not volumes; its density falls with the square of the range and is strongly anisotropic on the ground; it is silent behind every object; it is smeared by the sensor’s own motion; and it carries mixed pixels, spurious returns and holes. The next writeup in the course places these points in the world, through the coordinate frames and rigid transforms that relate a sensor to the vehicle carrying it and to everything else.