How is a depth map converted to a point cloud when there are non-zero distortion coefficients?

Converting a point cloud to a depth map is straight forward using the pinhole matrix and distortion coefficients. (1) First you normalize the x, y co-ordinates by the depth: x' = x / z, y' = y / z. (2) then you apply the distortion transformations:

[https:\/\/docs.opencv.org\/4.x\/d9\/d0c\/group\_\_calib3d.html](https://preview.redd.it/9i0u3olttz191.png?width=554&format=png&auto=webp&s=301ead68b91ee4a075ec98f3009db0c3eeb46f2d)

(3) Then you apply the pinhole intrinsics to get image plane co-ordinates: u = f\_x \* x'' + c\_x, v = f\_y \* y'' + c\_y.

(1) and (3) are linear, but (2) …

cloud computervision map