Kumpee Teeravech
Geoinformatics Program (GI)
Faculty of Computer Science and Information Technology (CSIT)
Rambhai Barni Rajabhat University (RBRU)
2026-08-18

Around January 2026, I had a chance to review the reComputer Industrial R2135-12 , an industrial edge-AI computer based on the Raspberry Pi Compute Module 5 and a Hailo-8 AI accelerator (Teeravech, 2026). As part of that review, I implemented a small demonstration that detected people in a video stream and estimated their locations in real-world coordinates.

In that article, I used several fundamental computer-vision concepts to convert 2D image points into 3D/world coordinates, including camera calibration, the pinhole camera model, Perspective-n-Point (PnP) pose estimation, back-projection, coordinate transformations, and ray-plane intersection. This article is a quick review of those concepts and equations so that I can revisit them later without having to reconstruct the whole derivation again.

Please note that all of the content in this article was automatically generated by AI.

A normal camera records a three-dimensional scene as a two-dimensional image. A point in the world with coordinates \[ \mathbf{X}_w = \begin{bmatrix} X_w \\ Y_w \\ Z_w \end{bmatrix} \] is projected onto an image point \[ \mathbf{x} = \begin{bmatrix} u \\ v \end{bmatrix}. \] Recovering \((X_w,Y_w,Z_w)\) from only one image point \((u,v)\) is generally impossible because infinitely many 3D points lie along the same viewing ray. Some additional geometric constraint is required.

In my reComputer demonstration, I used a simple but useful constraint: the target point was assumed to lie on the known ground plane. This turns the problem into finding where the camera ray passing through an image point intersects that plane (Teeravech, 2026).

The coordinate-recovery process can be summarized as:

1. Calibrate the camera to determine its intrinsic parameters and lens distortion.
2. Define several known reference points in a world coordinate system.
3. Identify the corresponding reference points in the image.
4. Use PnP to estimate the camera pose relative to the world coordinate system.
5. Undistort the target image point.
6. Back-project the image point into a 3D viewing ray.
7. Transform that ray from camera coordinates into world coordinates.
8. Intersect the ray with the known ground plane.

Conceptually: \[ \boxed{ \text{2D image point} \rightarrow \text{camera ray} \rightarrow \text{world ray} \rightarrow \text{ground-plane intersection} } \]

The basic geometric model used in standard photogrammetry and computer vision is the pinhole camera model (Hartley & Zisserman, 2004).

A 3D point expressed in camera coordinates, \[ \mathbf{X}_c = \begin{bmatrix} X_c \\ Y_c \\ Z_c \end{bmatrix}, \] is projected onto normalized image coordinates according to \[ x_n = \frac{X_c}{Z_c}, \qquad y_n = \frac{Y_c}{Z_c}. \] Pixel coordinates are then obtained using the camera intrinsic parameters: \[ u = f_x x_n + c_x, \] \[ v = f_y y_n + c_y. \] Here:

• \(f_x\) and \(f_y\) are focal lengths expressed in pixel units,
• \(c_x\) and \(c_y\) are the principal-point coordinates.

These values are usually written in the intrinsic matrix \[ \mathbf{K} = \begin{bmatrix} f_x & 0 & c_x \\ 0 & f_y & c_y \\ 0 & 0 & 1 \end{bmatrix}. \]

The camera does not normally share the same origin or orientation as the world coordinate system. Therefore, a rigid transformation is required: \[ \mathbf{X}_c = \mathbf{R}\mathbf{X}_w + \mathbf{t}. \] Here:

• \(\mathbf{R}\) is a \(3\times3\) rotation matrix, and
• \(\mathbf{t}\) is a \(3\times1\) translation vector.

In homogeneous form, the perspective projection can be written as \[ s \begin{bmatrix} u \\ v \\ 1 \end{bmatrix} = \mathbf{K} \begin{bmatrix} \mathbf{R} & \mathbf{t} \end{bmatrix} \begin{bmatrix} X_w \\ Y_w \\ Z_w \\ 1 \end{bmatrix}, \] where \(s\) is an unknown projective scale.

This compact equation contains the central geometry used throughout the calculation: \[ \boxed{ \text{world point} \rightarrow \text{camera coordinates} \rightarrow \text{image coordinates} } \]

Before reliable coordinate recovery is possible, the camera should be calibrated. In my original experiment, I used chessboard images and OpenCV calibration functions to estimate \(f_x,f_y,c_x,c_y\) together with the lens distortion coefficients (Teeravech, 2026).

A common distortion model contains radial parameters \(k_1,k_2,k_3\) and tangential parameters \(p_1,p_2\). OpenCV represents these approximately as: \[ (k_1,\ k_2,\ p_1,\ p_2,\ k_3). \] The calibration method commonly used in OpenCV is closely related to Zhang's flexible planar calibration approach, which estimates camera parameters from multiple views of a planar calibration target (Zhang, 2000).

In the January 2026 demonstration, the calibrated intrinsic parameters were approximately: \[ f_x = 3238.66, \qquad f_y = 3241.06, \] \[ c_x = 2042.01, \qquad c_y = 925.28. \] These numbers are specific to that camera, resolution, and calibration setup and should not be reused for another camera without recalibration.

The ideal pinhole equations assume that light rays are projected through an ideal perspective camera. Real lenses introduce distortion, so the raw pixel point \((u,v)\) is generally not exactly where the ideal pinhole model expects it.

Therefore, before constructing a viewing ray, the image point should be undistorted. OpenCV's cv2.undistortPoints() is particularly convenient because it can convert a distorted pixel coordinate into a normalized ideal image coordinate: \[ (u,v) \rightarrow (x_n,y_n). \] Once normalized, the corresponding ray in camera coordinates can be represented by \[ \mathbf{d}_c = \begin{bmatrix} x_n \\ y_n \\ 1 \end{bmatrix}. \]

Camera calibration gives the intrinsic parameters, but we still need to determine where the camera is located and how it is oriented relative to the world coordinate system.

This is a Perspective-n-Point (PnP) problem. Given:

• known 3D reference points \(\mathbf{X}_{w,i}\),
• their corresponding 2D image points \(\mathbf{x}_i\),
• the camera intrinsic matrix \(\mathbf{K}\), and
• lens distortion coefficients,

PnP estimates the rotation and translation satisfying the projection relationship \[ s_i\mathbf{x}_i \sim \mathbf{K} \left( \mathbf{R}\mathbf{X}_{w,i} + \mathbf{t} \right). \] In the original demonstration, I manually identified several reference points on the floor and then used OpenCV's cv2.solvePnP() to estimate \(\mathbf{R}\) and \(\mathbf{t}\) (Teeravech, 2026).

import cv2
import numpy as np


def estimate_camera_pose(
    pts_img,
    pts_world,
    camera_matrix,
    dist_coeffs
):
    objp = np.asarray(
        pts_world,
        dtype=np.float64
    ).reshape(-1, 1, 3)

    imgp = np.asarray(
        pts_img,
        dtype=np.float64
    ).reshape(-1, 1, 2)

    success, rvec, tvec = cv2.solvePnP(
        objp,
        imgp,
        camera_matrix,
        dist_coeffs
    )

    if not success:
        raise RuntimeError(
            "solvePnP failed"
        )

    R, _ = cv2.Rodrigues(
        rvec
    )

    return R, tvec

This point is easy to forget.

OpenCV's PnP convention gives a transformation from the object/world coordinate system into the camera coordinate system: \[ \mathbf{X}_c = \mathbf{R}\mathbf{X}_w + \mathbf{t}. \] Therefore, \(\mathbf{t}\) should not generally be interpreted directly as the camera position in world coordinates.

The camera center \(\mathbf{C}\) in world coordinates is found by setting \[ \mathbf{X}_c = \mathbf{0}. \] Thus, \[ \mathbf{0} = \mathbf{R}\mathbf{C} + \mathbf{t}, \] which gives \[ \boxed{ \mathbf{C} = -\mathbf{R}^{T}\mathbf{t} }. \] This distinction is important whenever the actual camera position must be reported or used to construct a world-coordinate viewing ray.

Once the image point has been undistorted, the normalized image coordinate gives the direction of the corresponding viewing ray in the camera coordinate system: \[ \mathbf{d}_c = \begin{bmatrix} x_n \\ y_n \\ 1 \end{bmatrix}. \] If distortion has already been removed and raw pixel coordinates are used with the intrinsic matrix, the same idea may be written as \[ \mathbf{d}_c \propto \mathbf{K}^{-1} \begin{bmatrix} u \\ v \\ 1 \end{bmatrix}. \] The symbol \(\propto\) is used because a ray direction has arbitrary scale. Multiplying \(\mathbf{d}_c\) by any positive scalar describes the same ray.

The PnP rotation maps world vectors into camera coordinates: \[ \mathbf{d}_c = \mathbf{R}\mathbf{d}_w. \] Therefore, \[ \boxed{ \mathbf{d}_w = \mathbf{R}^{T}\mathbf{d}_c }. \] It is convenient to normalize the direction: \[ \hat{\mathbf{d}}_w = \frac{\mathbf{d}_w} {\|\mathbf{d}_w\|}. \] The complete world-coordinate ray is then \[ \boxed{ \mathbf{X}(\lambda) = \mathbf{C} + \lambda\hat{\mathbf{d}}_w } \] where:

• \(\mathbf{C}\) is the camera center in world coordinates,
• \(\hat{\mathbf{d}}_w\) is the world-space ray direction, and
• \(\lambda\) is distance along the ray up to a scale determined by the normalized direction.

The single image point gives a ray but still does not give one unique 3D point. The missing information is supplied by the known ground plane.

A general plane can be written as \[ \mathbf{n}^{T}\mathbf{X} + d_0 = 0, \] where \(\mathbf{n}\) is the plane normal.

For the simple ground plane \[ Z = 0, \] we may use \[ \mathbf{n} = \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}, \qquad d_0 = 0. \]

Substitute the ray \[ \mathbf{X} = \mathbf{C} + \lambda\mathbf{d} \] into the plane equation \[ \mathbf{n}^{T}\mathbf{X} + d_0 = 0. \] This gives \[ \mathbf{n}^{T} \left( \mathbf{C} + \lambda\mathbf{d} \right) + d_0 = 0. \] Solving for \(\lambda\): \[ \boxed{ \lambda = - \frac{ \mathbf{n}^{T}\mathbf{C} + d_0 }{ \mathbf{n}^{T}\mathbf{d} } }. \] The recovered world point is then \[ \boxed{ \mathbf{X} = \mathbf{C} + \lambda\mathbf{d} }. \] This is the essential equation used in my January 2026 demonstration to recover a person's foot position from the 2D ankle location detected by the Hailo-8 pose-estimation model (Teeravech, 2026).

import cv2
import numpy as np


def create_world_ray(
    u,
    v,
    K,
    dist_coeffs,
    R,
    t
):
    # Image point
    pts = np.array(
        [[[u, v]]],
        dtype=np.float64
    )

    # Convert distorted pixel point
    # to normalized undistorted point
    undist = cv2.undistortPoints(
        pts,
        K,
        dist_coeffs
    )

    x_n, y_n = undist[0, 0]

    # Ray in camera coordinates
    d_cam = np.array(
        [x_n, y_n, 1.0],
        dtype=np.float64
    )

    # Camera center in world coordinates
    C = -R.T @ t

    # Ray direction in world coordinates
    d_world = R.T @ d_cam

    d_world /= np.linalg.norm(
        d_world
    )

    return (
        C.reshape(3),
        d_world.reshape(3)
    )

def intersect_ray_with_plane(
    C,
    d,
    plane_normal,
    plane_d
):
    n = np.asarray(
        plane_normal,
        dtype=np.float64
    )

    denom = np.dot(
        n,
        d
    )

    # Ray is nearly parallel to plane
    if abs(denom) < 1e-9:
        return None

    lam = -(
        np.dot(n, C)
        + plane_d
    ) / denom

    # Intersection is behind camera
    if lam < 0:
        return None

    X = C + lam * d

    return X

For a ground plane at \(Z=0\):

C, d = create_world_ray(
    u,
    v,
    K,
    dist_coeffs,
    R,
    tvec
)

ground_normal = np.array(
    [0.0, 0.0, 1.0]
)

ground_d = 0.0

X_world = intersect_ray_with_plane(
    C,
    d,
    ground_normal,
    ground_d
)

print(
    "World coordinate:",
    X_world
)

Assume the camera center is \[ \mathbf{C} = \begin{bmatrix} 1.0 \\ -2.0 \\ 1.5 \end{bmatrix} \mathrm{m} \] and the normalized world-ray direction is approximately \[ \mathbf{d} = \begin{bmatrix} 0.40 \\ 0.60 \\ -0.69 \end{bmatrix}. \] For a ground plane \(Z=0\), \[ \mathbf{n} = \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}, \qquad d_0=0. \] Therefore, \[ \lambda = - \frac{1.5}{-0.69} \approx 2.17. \] The world position is \[ \mathbf{X} = \mathbf{C} + 2.17\mathbf{d}. \] Approximately, \[ \mathbf{X} \approx \begin{bmatrix} 1.87 \\ -0.70 \\ 0.00 \end{bmatrix} \mathrm{m}. \] The important point is that the image provides the direction, while the known plane provides the missing depth.

In the original demonstration, I defined a floor coordinate system using several known reference points. The origin was marked on the floor, and the \(+X\) and \(+Y\) directions were defined using reference points arranged along the floor tiles. Each floor tile measured approximately \(60\times60\) cm (Teeravech, 2026).

I used those known 3D coordinates together with manually identified image coordinates to estimate the camera pose. As a simple verification, the estimated tripod-base location was reported as approximately \[ (0.3,\,-59.4,\,0.0) \] cm, close to the expected position \[ (0,\,-60,\,0). \] For person localization, the Hailo-8 pose-estimation example returned body landmarks. I extracted the ankle image coordinates, generated a viewing ray, and intersected that ray with the ground plane. One test produced an estimated foot location of approximately \[ (202.0,\ 44.2,\ 0.0) \] cm in the selected world reference frame, which was consistent with where I was standing in the scene (Teeravech, 2026).

If the ground-plane assumption is being used, the selected image point should correspond to something physically located on or very close to the ground.

A person's face, shoulder, or bounding-box center does not lie on the ground plane. Intersecting the ray from such a point with \(Z=0\) would give the ground location underneath the ray rather than the actual 3D location of that body part.

For pedestrian localization, the ankle, shoe contact point, or bottom-center of a detection box is therefore often more suitable for this simple ground-intersection model.

If every target point lies on the same plane, there is another useful formulation: a planar homography.

A homography directly maps points between the ground plane and image plane: \[ s \begin{bmatrix} u \\ v \\ 1 \end{bmatrix} = \mathbf{H} \begin{bmatrix} X \\ Y \\ 1 \end{bmatrix}. \] If \(\mathbf{H}\) is known, image-to-ground conversion can be performed by \[ \begin{bmatrix} X \\ Y \\ 1 \end{bmatrix} \propto \mathbf{H}^{-1} \begin{bmatrix} u \\ v \\ 1 \end{bmatrix}. \] This can be simpler when the application is strictly planar. However, the PnP plus ray-plane method explicitly provides the camera pose and extends naturally to other known planes and 3D geometric operations.

The mathematics is compact, but real-world accuracy depends strongly on the quality of the input. Important error sources include:

• poor camera calibration,
• incorrect lens-distortion parameters,
• inaccurate manual image-point selection,
• incorrect world coordinates for control points,
• poor geometric distribution of PnP reference points,
• an incorrect ground-plane assumption,
• camera movement after pose estimation,
• uncertainty in AI keypoint or object detection, and
• rays that intersect the ground at very shallow angles.

A small pixel error near the horizon can produce a large position error on the ground because the viewing ray is nearly parallel to the plane. This can be seen directly in the denominator \[ \mathbf{n}^{T}\mathbf{d}. \] When this value approaches zero, the estimated intersection becomes numerically unstable and very sensitive to small errors.

After estimating the camera pose, it is good practice to project the known world points back into the image and compare them with the measured image points.

If \(\hat{\mathbf{x}}_i\) is the projected point and \(\mathbf{x}_i\) is the measured point, a simple reprojection error is \[ e_i = \left\| \mathbf{x}_i - \hat{\mathbf{x}}_i \right\|_2. \] The mean reprojection error is \[ \bar{e} = \frac{1}{N} \sum_{i=1}^{N} e_i. \] A small reprojection error does not guarantee perfect world-coordinate accuracy, but a large one is a strong indication that the calibration, correspondences, or pose estimate should be checked.

Putting the main pieces together:

# ---------------------------------------------------------
# INPUTS
# ---------------------------------------------------------
# pts_world : known 3D control points
# pts_img   : matching 2D image points
# K         : intrinsic camera matrix
# dist      : distortion coefficients
# u, v      : target image point


# 1. Estimate pose
R, tvec = estimate_camera_pose(
    pts_img,
    pts_world,
    K,
    dist
)


# 2. Create a world-space ray
C, d = create_world_ray(
    u,
    v,
    K,
    dist,
    R,
    tvec
)


# 3. Define ground plane Z = 0
n = np.array(
    [0.0, 0.0, 1.0]
)

d0 = 0.0


# 4. Intersect ray with ground
X = intersect_ray_with_plane(
    C,
    d,
    n,
    d0
)

print(X)

For my own quick review, the essential equations are only a few.

World to camera: \[ \mathbf{X}_c = \mathbf{R}\mathbf{X}_w + \mathbf{t}. \] Projection: \[ s\mathbf{x} = \mathbf{K} \begin{bmatrix} \mathbf{R} & \mathbf{t} \end{bmatrix} \mathbf{X}_w. \] Camera center: \[ \mathbf{C} = -\mathbf{R}^{T}\mathbf{t}. \] Back-projected camera ray: \[ \mathbf{d}_c \propto \mathbf{K}^{-1} \begin{bmatrix} u \\ v \\ 1 \end{bmatrix}. \] World-ray direction: \[ \mathbf{d}_w = \mathbf{R}^{T}\mathbf{d}_c. \] Ray: \[ \mathbf{X}(\lambda) = \mathbf{C} + \lambda\mathbf{d}_w. \] Ray-plane intersection: \[ \lambda = - \frac{ \mathbf{n}^{T}\mathbf{C} + d_0 }{ \mathbf{n}^{T}\mathbf{d}_w }. \]

Recovering real-world coordinates from a single image point sounds like a 3D reconstruction problem, but in the simple case used in my reComputer demonstration, the geometry is quite manageable. The camera calibration and PnP solution define the camera relative to the scene; the image point defines a viewing direction; and the known ground plane supplies the missing depth.

The complete idea can therefore be summarized as: \[ \boxed{ \text{calibrate} \rightarrow \text{estimate camera pose} \rightarrow \text{back-project pixel} \rightarrow \text{intersect known plane} } \] This simple approach is useful for many applications in which objects can reasonably be assumed to touch a known floor or ground surface, including pedestrian localization, camera-based occupancy maps, robotics, industrial monitoring, and simple augmented-reality or photogrammetric measurements.

Hartley, R., & Zisserman, A. (2004). Multiple view geometry in computer vision (2nd ed.). Cambridge University Press. Cambridge University Press .

OpenCV. (n.d.). Camera calibration and 3D reconstruction. OpenCV documentation. https://docs.opencv.org/4.x/d9/d0c/group__calib3d.html

Teeravech, K. (2026, January 15). reComputer Industrial R2135-12 review – A Raspberry Pi CM5-powered fanless Edge AI PC with Hailo-8 AI accelerator. CNX Software. https://www.cnx-software.com/2026/01/15/recomputer-industrial-r2135-12-review-a-raspberry-pi-cm5-powered-fanless-edge-ai-pc-with-hailo-8-ai-accelerator/

Zhang, Z. (2000). A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11), 1330–1334. Microsoft Research .

Please note that all of the content in this article was automatically generated by AI.