Underwater Camera Calibration Based on Double Refraction

Abstract

Underwater camera calibration plays a pivotal role in underwater positioning and underwater mapping reconstruction, making it crucial for achieving precise spatial measurements in underwater environments. To elevate measurement accuracy, we have refined the calibration methodology for underwater cameras. Firstly, we conducted an in-depth investigation into the intricate challenges posed by double refraction errors arising from light passing through water–glass–air interfaces. To address this issue, we established a double refraction model based on the actual underwater light paths, laying the groundwork for our calibration efforts. Furthermore, to tackle the problem of determining multiple camera parameters, we proposed a parameter optimization method based on genetic algorithms, capable of navigating the complex parameter space. Finally, compared with other algorithms, our method enables more precise determination of underwater camera parameters.

1. Introduction

Computer vision has been widely applied across various domains, including the medical field [1], industrial sector [2], military applications, and beyond. In recent times, with advancements in underwater equipment, such as autonomous underwater vehicles (AUVs) [3], remotely operated vehicles (ROVs) [4], etc., underwater vision has become an indispensable component of underwater exploration [5], aquaculture [6], underwater archaeology [7,8], and various other applications such as underwater target recognition [9,10], underwater target localization [11], underwater object reconstruction [12], and underwater simultaneous localization and mapping (SLAM) [13]. In these applications, the accuracy of camera calibration is paramount for tasks like underwater positioning and map reconstruction.

To establish the relationship between the three-dimensional geometric position of a point on the surface of an object and its corresponding point in the image, it is necessary to build a geometric model of camera imaging. The process of solving these parameters is called camera calibration. Due to the unique characteristics of the underwater environment, underwater cameras typically consist of camera components, a closed shell, and a glass cover, causing the underwater light path to pass through three media: water, glass, and air. This inevitably leads to refraction, which introduces errors in subsequent measurement processing.

In previous studies, some researchers [14,15] directly overlooked the influence of refraction and applied calibration algorithms designed for air environments [16,17] directly in underwater settings, inevitably leading to errors. Some scholars addressed the refraction effect by approximating it as changes in focal length and distortion: Lavest [18] suggests that if calibration methods designed for air are directly applied to underwater environments, the focal length needs to be multiplied by the water index; and Qin [19] utilized advanced camera distortion models to compensate for deviations caused by light refraction through water and air media. Li [20] equivalently treats the underwater camera model as a pinhole model with changes in focal length, and provides the corresponding process for calculating camera parameters. Sun [21] proposed a rapid underwater calibration method based on two sets of mutually orthogonal parallel lines, optimizing the vanishing point to determine its focal length, which is based on an air–water equivalent focal length model. Bashar [22] proposed the theory of refraction invariance, dividing the calculation of imaging parameters into refractive and non-refractive related sets, and utilizing an approximate linear model to compute camera parameters. In such algorithms, due to the complexity of simulating refraction, there are issues with low calibration accuracy. Fortunately, recently some scholars have conducted research on underwater cameras based on real light path models: Treibitz [23] derived forward projection based on Snell’s Law, and calibrated the distance between the camera optical centre and the nearest interface, and the intrinsic parameters and distortion of the camera need to be calculated through prior calibration in air. Agrawal [24] proposed an axial camera model based on flat refraction constraints and coplanarity constraints, and presented corresponding methods to solve the high-order forward projection equations. Gu [25] formulated the refractive projection function and calibrated the interface parameters with a quaternion-based optimal solution. However, due to the complexity of parameters in the real underwater camera model, there are the following issues: 1. parameter-solving conditions rely on strict imaging conditions; and 2. constraint adjustment cannot be accurately solved. With the advancement of research, recent studies have shown the potential of intelligent optimization algorithms in solving camera parameters. Tian [26] developed a monocular camera calibration algorithm for land-based applications using an improved particle swarm optimization algorithm. Du [27] utilized an enhanced slime mold algorithm for the calibration of underwater cameras, yet the imaging model remains based on the pinhole model.

To address these challenges, we propose a correction algorithm based on the real light path’s double refraction model and demonstrate its necessity in underwater camera usage. Furthermore, due to the need for properties such as high-pressure resistance in underwater cameras, many modern underwater camera structures are relatively integrated and difficult to disassemble and reassemble. As a result, measuring the thickness of the glass becomes relatively challenging. Our model algorithm addresses the issue of glass thickness. Subsequently, we employ a genetic algorithm to solve the parameters, incorporating various strategies during the algorithm’s iterations to overcome local optima. Experimental results demonstrate the accurate calculation of camera parameters using our algorithm, making the imaging calibration process straightforward, thus presenting a feasible method for underwater camera calibration.

For the remaining sections, we first introduce the double refraction model and derive the coordinate transformation of the real light path using our method in Section 2. In Section 3, we provide the mathematical proof of the necessity of considering double refraction in underwater camera calibration. This part discusses the impact of neglecting this factor and the resulting spatial measurement errors. In Section 4, we describe our underwater calibration algorithm in detail, including the calibration process using genetic algorithms and checkerboard pattern images. Experimental validation is presented in Section 5. Finally, in Section 6, we summarize the conclusions and propose future work.

2. Underwater Camera Model

Due to the existence of water medium in the underwater environment, to prevent the camera from entering the water and short-circuiting, the underwater camera is usually placed in a watertight and sealed pressure-resistant shell, and a transparent medium such as glass is used as a light-transmitting material. This causes the light reflected to pass through the three mediums of water–glass–air before the camera can capture it.

According to Fermat’s principle, light rays travel in a straight line in the same homogeneous medium, but they are refracted when passing through a different medium. Therefore, the camera will capture the light rays of underwater objects after two refractions. As shown in Figure 1, we have to model the light path propagation model according to the real light path.

In the real light path, we propagate $A\left(x_G, y_G, z_G\right)$ in the world coordinate system to $A_2(u, \mathrm{v})$ in the pixel coordinate system. The method regards the process as two refractions: (1) coordinates derived from $A$ to $A_1$ according to the refraction relation; and (2) coordinates derived from $A_1$ to $A_2$ according to the imaging principle.

First, we convert $A$ from the world coordinate system $\left(x_G , y_G , z_G\right)$ to the camera coordinate system $\left(x_w , y_w , z_w\right)$:

$$\left[\begin{array}{c}{x}_w\\ y_w\\ z_w\\ 1\end{array}\right]=\underset{4\times 4}{\left[\begin{array}{cc}\underset{3\times 3}{R}& \underset{3\times 1}{\overrightarrow{T}}\\ \underset{1\times 3}{\overrightarrow{0}}& 1\end{array}\right]}\left[\begin{array}{c}{x}_G\\ y_G\\ z_G\\ 1\end{array}\right]$$

(1)

where $\underset{3\times 3}{R}$ represents a $3\times 3$ rotation transformation matrix, and its value is associated with the orientation of the camera coordinate system relative to the world coordinate system. Its specific definition is provided in Equation (2). $\overrightarrow{T}$ represents a 3 × 1 translation vector, indicating the displacement of the camera coordinate system origin relative to the world coordinate system. Its specific definition is given in Equation (3).

$$R=\left[\begin{array}{ccc}\cos \psi \cos \theta & \cos \psi \sin \theta \sin \varphi -\sin \psi \cos \varphi & \cos \psi \sin \theta \sin \varphi +\sin \psi \sin \varphi \\ \sin \psi \cos \theta & \sin \psi \sin \theta \sin \varphi +\cos \psi \cos \varphi & \sin \psi \sin \theta \sin \varphi -\cos \psi \sin \varphi \\ -\sin \theta & \cos \theta \sin \varphi & \cos \theta \cos \varphi \end{array}\right]$$

(2)

$$\overrightarrow{T}=\left[\begin{array}{c}{x}_{\mathrm{O}}\\ y_{\mathrm{O}}\\ z_{\mathrm{O}}\end{array}\right]$$

(3)

where $\varphi ,\psi ,\theta$ represents the rotation angle of the camera coordinate system relative to the world coordinate system; and $(x_{\mathrm{O}},y_{\mathrm{O}},z_{\mathrm{O}})$ represents the position of the camera’s optical center in the world coordinate system.

$\mathrm{A}\to {\mathrm{A}}_1$: in the camera coordinate system, let the distance between a point $\mathrm{A}$ and the optical axis be $r_{\mathrm{w}}$, and the distance between a point ${\mathrm{A}}_1$ and the optical axis be $r_a$, and meet the following requirements:

$$\left\{\begin{array}{l}{r_{\mathrm{w}}}^2={x_{\mathrm{w}}}^2+{y_{\mathrm{w}}}^2\\ {r_a}^2={x_a}^2+{y_a}^2\\ \frac{y_{\mathrm{w}}}{x_{\mathrm{w}}}=\frac{y_a}{x_a}\\ \sin {\theta }_a=\frac{r_a}{\sqrt{{r_a}^2+h^2}}\end{array}\right.$$

(4)

The relation between point ${\mathrm{A}}_1$ and point $\mathrm{A}$ can be obtained by geometric knowledge:

$$r_{\mathrm{w}}=r_{\mathrm{a}}+t\tan {\theta }_{\mathrm{g}}+(z-t-h)\tan {\theta }_{\mathrm{w}}$$

(5)

According to Snell’s law of refraction:

$$n_{\mathrm{w}}\sin {\theta }_{\mathrm{w}}=n_{\mathrm{g}}\sin {\theta }_{\mathrm{g}}=n_{\mathrm{a}}\sin {\theta }_{\mathrm{a}}$$

(6)

where $h$ is the distance between the optical center and the lower surface of the glass; $t$ is the thickness of glass; $z$ is the depth of the target point in camera coordinates; $n_{\mathrm{w}}$, $n_{\mathrm{g}}$, and $n_{\mathrm{a}}$ represent the refractive index of water, glass, and air, respectively; ${\theta }_{\mathrm{g}}$ is the refractive angle of light rays in the glass; ${\theta }_{\mathrm{w}}$ is the incident angle of the ray in water; and ${\theta }_{\mathrm{a}}$ is the refractive angle of the ray in air, and its value is $\arctan \frac{r}{f}$.

For a triangle, we can easily obtain the following equations:

$$\tan \theta =\frac{\sin \theta }{\sqrt{1-{\sin }^2\theta }}$$

(7)

At 20 °C and pressure 1 atmosphere, the refractive index of air is 1.00027, which we approximate to 1. Then, the following equation can be deduced from Equations (6) and (7):

$$\left\{\begin{array}{c}\tan {\theta }_{\mathrm{g}}=\frac{\sin {\theta }_{\mathrm{a}}}{\sqrt{{n_{\mathrm{g}}}^2-{\sin }^2{\theta }_{\mathrm{a}}}}\\ \tan {\theta }_{\mathrm{w}}=\frac{\sin {\theta }_{\mathrm{a}}}{\sqrt{{n_{\mathrm{w}}}^2-{\sin }^2{\theta }_{\mathrm{a}}}}\end{array}\right.$$

(8)

By incorporating Equations (4) and (5), we obtain:

$$\begin{array}{ll}\hfill r_{\mathrm{w}}& =r_{\mathrm{a}}+t\frac{\sin {\theta }_{\mathrm{a}}}{\sqrt{{n_{\mathrm{g}}}^2-{\sin }^2{\theta }_{\mathrm{a}}}}+(z-t-h)\frac{\sin {\theta }_{\mathrm{a}}}{\sqrt{{n_{\mathrm{w}}}^2-{\sin }^2{\theta }_{\mathrm{a}}}}\\ & =r_{\mathrm{a}}+\frac{t{r}_{\mathrm{a}}}{\sqrt{{n_{\mathrm{g}}}^2{h}^2+({n_{\mathrm{g}}}^2-1){r_{\mathrm{a}}}^2}}+\frac{(z-t-h)r_{\mathrm{a}}}{\sqrt{{n_{\mathrm{w}}}^2{h}^2+({n_{\mathrm{w}}}^2-1){r_{\mathrm{a}}}^2}}\end{array}$$

(9)

To obtain $r_a$ according to $r_{\mathrm{w}}$, direct derivation is not feasible. Therefore, we opt to employ Newton’s method for iteration:

$$g(r_{\mathrm{a}})=r_{\mathrm{a}}+\frac{t{r}_{\mathrm{a}}}{\sqrt{{n_{\mathrm{g}}}^2{h}^2+({n_{\mathrm{g}}}^2-1){r_{\mathrm{a}}}^2}}+\frac{(z-t-h)r_{\mathrm{a}}}{\sqrt{{n_{\mathrm{w}}}^2{h}^2+({n_{\mathrm{w}}}^2-1){r_{\mathrm{a}}}^2}}-r_{\mathrm{w}}$$

(10)

The zero point of function, $g(r_{\mathrm{a}})$, corresponds to the solution of Equation (9). The iterative formula of Newton’s method is as follows:

$$r_{\mathrm{a} (n+1)}=r_{\mathrm{a} (n)}-\frac{g(r_{\mathrm{a} \left(\mathrm{n}\right)})}{g^{\prime }(r_{\mathrm{a} \left(\mathrm{n}\right)})} (\mathrm{n}=0,1,2,\cdots )$$

(11)

We initiate the iterative process by substituting the projection position of the target point, under the condition of no refraction, as the initial point $r_{\mathrm{a} (0)}=\frac{r_{\mathrm{w}}h}{z}$ into Equation (11). This iterative calculation allows us to derive the solution for Equation (9).

The proof of the iterative convergence of Equation (10) is as follows:

$$g^{\prime }(r_{\mathrm{a}})=1+\frac{t{h}^2{n}_{\mathrm{g}}}{{\left[{n_{\mathrm{g}}}^2,h^2,+,\left({n_{\mathrm{g}}}^2-1\right),{r_{\mathrm{a}}}^2\right]}^{\frac{3}{2}}}-\frac{(z-t-h)h^2{n}_{\mathrm{a}}}{{\left[{n_{\mathrm{a}}}^2{h}^2+\left({n_{\mathrm{a}}}^2-1\right){r_{\mathrm{a}}}^2\right]}^{\frac{3}{2}}}$$

(12)

In the above Equation (12), both $n_a$ and $n_{\mathrm{g}}$ are constants greater than 0. It can be easily obtained from Equation (11): within the interval of $r_{\mathrm{a}}>0$, $g^{\prime }(r_{\mathrm{a}})$ is a monotonic decreasing function, and when $r_{\mathrm{a}}\to \infty$, $g^{\prime }(r_{\mathrm{a}})$ is the minimum value of 1. Therefore, when $r_{\mathrm{a}}>0$, $g^{\prime }(r_{\mathrm{a}})\ne 0$ is constantly established. In the interval of $r_{\mathrm{a}}>0$, the second derivative of $g(r_{\mathrm{a}})$ to $r_{\mathrm{a}}$ exists and is continuous, according to the local convergence theorem of Newton iteration, for the initial iteration point, $r_{\mathrm{a} (0)}$, close to the true value, $r_{\mathrm{a}}^{\ast }$, the sequence $\left\{r_{\mathrm{a} (\mathrm{k})}\right\}$ generated by Equation (11) always converges to $r_{\mathrm{a}}^{\ast }$ at a rate of no less than second order.

After obtaining $r_{\mathrm{a}}$, the coordinate of ${\mathrm{A}}_1$ in the camera coordinate system $\left(x_a, y_a, h\right)$ can be obtained according to Equation (4).

${\mathrm{A}}_1\to {\mathrm{A}}_2$: the coordinate $(x_{\mathrm{pic}},y_{\mathrm{pic}})$ in the imaging plane coordinate system can be obtained from the pinhole imaging principle:

$$h\left[\begin{array}{c}{x}_{\mathrm{pic}}\\ y_{\mathrm{pic}}\\ 1\end{array}\right]=\left[\begin{array}{cccc}f& 0& 0& 0\\ 0& f& 0& 0\\ 0& 0& 1& 0\end{array}\right] \left[\begin{array}{c}{x}_a\\ y_a\\ h\\ 1\end{array}\right]$$

(13)

In actual use, we have to consider the influence of distortion. We use Equation (14) to address the distortion effects.

$$\left\{\begin{array}{c}{x}_{\mathrm{c}}=x_{\mathrm{pic}}(1+k_1{r}^2+k_2{r}^4+k_3{r}^6)+2p_1{x}_{pic}{y}_{pic}+p_2(r^2+2x_{pic}^2)\\ y_{\mathrm{c}}=y_{\mathrm{pic}}(1+k_1{r}^2+k_2{r}^4+k_3{r}^6)+p_1(r^2+2y_{pic}^2)+2p_2{x}_{pic}{y}_{pic}\end{array}\right.$$

(14)

where $x_{\mathrm{c}}$ and $y_{\mathrm{c}}$ are real position coordinates, and $k_1,k_2,k_3,p_1,p_2$ are distortion parameters.

Due to the fact that computers typically utilize the pixel coordinate system when processing images, the corrected coordinates are converted into coordinates within the pixel coordinate system:

$$\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=\left[\begin{array}{ccc}\frac{1}{dx}& 0& u_0\\ 0& \frac{1}{dy}& v_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{c}{x}_{\mathrm{c}}\\ y_{\mathrm{c}}\\ 1\end{array}\right]$$

(15)

where dx and dy are the actual physical dimensions of a single photosensitive unit in the camera along the height and width directions, respectively; and $(u_0,v_0)$ is the position coordinate of origin ${\mathrm{O}}_{\mathrm{pic}}$ in the planar pixel coordinate system of the imaging plane.

3. Model Necessity Proof

To prove the necessity of the model, we set the “true point”, which ignores the first refraction effect, as point ${\mathrm{A}}^{\prime }$, and the mapping position of A and A′ to the image is the same as shown in Figure 2. According to the geometric knowledge, we can find the distance $r_{\mathrm{w}}$ from point A to the central optical axis.

$$r_{\mathrm{w}}=\frac{rh}{f}+t\tan {\theta }_{\mathrm{g}}+(z-t-h)\tan {\theta }_{\mathrm{w}}$$

(16)

where $r$ is the distance between the imaging point and the central optical axis in the imaging plane. Equation (17) can be obtained by substituting Equation (8) into Equation (16).

$$\begin{array}{ll}\hfill r_{\mathrm{w}}& =\frac{rh}{f}+t\frac{\sin {\theta }_{\mathrm{a}}}{\sqrt{{n_{\mathrm{g}}}^2-{\sin }^2{\theta }_{\mathrm{a}}}}+(z-t-h)\frac{\sin {\theta }_{\mathrm{a}}}{\sqrt{{n_{\mathrm{w}}}^2-{\sin }^2{\theta }_{\mathrm{a}}}}\\ & =\frac{rh}{f}+\frac{tr}{\sqrt{{n_{\mathrm{g}}}^2{f}^2+({n_{\mathrm{g}}}^2-1)r^2}}+\frac{(z-t-h)r}{\sqrt{{n_{\mathrm{w}}}^2{f}^2+({n_{\mathrm{w}}}^2-1)r^2}}\end{array}$$

(17)

According to Snell’s law of refraction, glass in the air does not change the incident angle of the external light rays, and there is only a tiny steady-state translation. This paper ignores this small quantity, and glass does not affect the transmission of light rays when used in the air. As shown in Figure 2, we can calculate the deviation, $\Delta r$, between the target position in the air and the underwater environment based on the above derivation.

$$\begin{array}{ll}\Delta r& =\frac{z}{f}r-r_{\mathrm{w}}\\ & =\frac{z-h}{f}r-\frac{tr}{\sqrt{{n_{\mathrm{g}}}^2{f}^2+({n_{\mathrm{g}}}^2-1)r^2}}-\frac{(z-t-h)r}{\sqrt{{n_{\mathrm{w}}}^2{f}^2+({n_{\mathrm{w}}}^2-1)r^2}}\end{array}$$

(18)

To see the influence of the distance of the target point from the central optical axis on $\Delta r$, we will obtain the partial derivative of Equation (18) to $r$:

$$\frac{\partial \Delta r}{\partial r}=\frac{z-h}{f}-\frac{t{n_{\mathrm{g}}}^2{f}^2}{{\left[{n_{\mathrm{g}}}^2,f^2,+,(,{n_{\mathrm{g}}}^2,-,1,),r^2\right]}^{\frac{3}{2}}}-\frac{(z-t-h){n_{\mathrm{w}}}^2{f}^2}{{\left[{n_{\mathrm{w}}}^2{f}^2+({n_{\mathrm{w}}}^2-1)r^2\right]}^{\frac{3}{2}}}$$

(19)

In the above formula, $t$ and $f$ are constants greater than 0, $n_{\mathrm{g}}$ and $n_{\mathrm{w}}$ are constants greater than 1, and $z$ is a variable greater than $t+h$. $r$ is the variable greater than 0.

From Equation (19), it is easy to see that $\frac{\partial \Delta r}{\partial r}$ is the increment function of $r$, and the minimum value of Equation (19) is obtained when $r$ is 0.

$$\frac{\partial \Delta r}{\partial r}{|}_{\min }=(1-\frac{1}{n_{\mathrm{g}}})\frac{t}{f}+(1-\frac{1}{n_{\mathrm{w}}})\frac{z-h-t}{f}$$

(20)

In the above formula, $n_{\mathrm{g}}$ and $n_{\mathrm{w}}$ are constants greater than 1, and $z$ is a value greater than $t+h$. It is easy to see that Equation (20) is greater than 0 and always holds.

From the nature of the increasing function, we can see that the value of Equation (20) is always greater than 0.

We know from the properties of derivatives that, for the $z$ given value, $\Delta r$ is an increment of $r$, and the bigger $r$ is, the bigger $\Delta r$ is. According to the principle of optical path reversible, it can be easily deduced that, when the $z$ value is constant, the greater the distance from the target point to the central optical axis is, the greater is the distortion deviation.

To see the influence of the depth of the target point on $\Delta r$, we will obtain the partial derivative of Equation (18) with respect to $z$:

$$\begin{array}{ll}\frac{\partial \Delta r}{\partial z}& =\frac{r}{f}-\frac{r}{\sqrt{{n_{\mathrm{w}}}^2{f}^2+({n_{\mathrm{w}}}^2-1)r^2}}\\ & =\frac{r}{f}\left(1-\frac{1}{\sqrt{{n_{\mathrm{w}}}^2+\frac{({n_{\mathrm{w}}}^2-1)r^2}{f^2}}}\right)\end{array}$$

(21)

In the above Equation (21), $t$ and $f$ are constants greater than 0 and $n_{\mathrm{w}}$ are constants greater than 1. It is easy to see that Equation (21) is always greater than 0. We know from the property of the derivative that, at a given value of $r$, $\Delta r$ is an increment of $z$, and the bigger $z$ is, the bigger $\Delta r$ is. According to the principle of optical path reversible, it can be easily deduced that, when the $r$ value is constant, the farther the target point is from the camera, the greater is the distortion deviation. It is necessary to study the camera imaging model in the underwater environment because the camera imaging model in the underwater environment is bound to cause the deviation of the target location.

4. Underwater Camera Calibration Based on Double Refraction

The accuracy of parameter calibration of the underwater camera-imaging model directly affects the accuracy of underwater light vision positioning. In this paper, we propose to use planar checkerboard pattern images and a genetic algorithm to calibrate the underwater camera model above and carry out calibration experiments. In the underwater camera model established above, there are parameters to be calibrated, as shown in

Table 1. Summary table of parameters to be calibrated.

Parameter’s Name	Describe
External Camera Parameters	$\psi$, $\theta$, $\varphi$ represents the rotation angle of the camera coordinate system relative to the world coordinate system; $x_{\mathrm{O}}$, $y_{\mathrm{O}}$, $z_{\mathrm{O}}$ represents the position of the camera’s optical center in the world coordinate system.
Intrinsic Camera Parameters	$(u_0,v_0)$ is the position coordinate of origin ${\mathrm{O}}_{\mathrm{pic}}$ in the planar pixel coordinate system of the imaging plane; $f_{\mathrm{x}}$, $f_y$ are the horizontal and longitudinal digital focal lengths of the camera.
Distortion Parameters	$k_1$, $k_2$, $k_3$ is the radial distortion parameter; $p_1$, $p_2$ is the tangential distortion parameter.
Refraction Parameters	$h$ is the distance from the optical centre to the lower surface of the glass; $t$ is the thickness of the glass; $n_{\mathrm{g}}$ is the refractive index of light rays in the glass; $n_{\mathrm{w}}$ is the refractive index of light rays in the water.

.

Due to the speciality of the underwater environment, our calibration parameters are compared to air more out of refraction parameters, and this is our focus of calibration. The distortion parameters and intrinsic camera parameters can be calibrated in the air by Zhang’s calibration [17]. For an image, the actual parameters that need to be calibrated in an underwater environment are only the six external parameters of the camera and four refraction parameters.

The calibration process of parameters is shown in Figure 3. Firstly, we use an underwater camera to take planar checkerboard pattern images in the underwater environment in the calibration process, and establish the world coordinate system on the checkerboard pattern. The distance between the corner points we can measure with a ruler, so the position of the corners in the world coordinate system is fixed, and we can obtain the coordinates of the corner points in the world coordinate system from the calibration plate. We use the external parameter matrix to transform the coordinate system to obtain the coordinates of the corners in the camera coordinate system, and use the double refraction model to predict the position of the corners on the image (expected position). At the same time, the position of the corresponding corners in the image (real position) is obtained through vision processing. In the initial state, the parameters in the camera model were inaccurate, so there was a deviation between the predicted position and the real position. We take the deviation as the optimization target, and the calibrated parameters are used as the variables to be optimized. The genetic algorithm is used to optimize the parameters continuously. The closer the parameter is to the accurate value, the smaller is the deviation between the predicted position and the true position, and accurate model parameters can be obtained after multiple iterations of calculations.

The position vector of the corners in the world coordinate system is $X_{\mathrm{G}}=(x_{\mathrm{G}},y_{\mathrm{G}},z_{\mathrm{G}})$; after the coordinate change in Section 2, we can obtain the position vector of the corners in the camera coordinate system: $X_{\mathrm{cam}}=(x_{\mathrm{w}},{ \mathrm{y}}_{\mathrm{w}}, \mathrm{z})$; and, finally, we can obtain the position vector of the corners in the pixel coordinate system: $X_{\mathrm{pix}}=(u, \mathrm{v})$. Equation (22) can express this process:

$$\left[\begin{array}{c}u\\ v\end{array}\right]=\left\{\begin{array}{c}{F}_1(\psi ,\theta ,\varphi ,x_{\mathrm{O}},y_{\mathrm{O}},z_{\mathrm{O}},h,t,n_{\mathrm{g}},n_{\mathrm{w}},X_{\mathrm{G}})\\ F_2(\psi ,\theta ,\varphi ,x_{\mathrm{O}},y_{\mathrm{O}},z_{\mathrm{O}},h,t,n_{\mathrm{g}},n_{\mathrm{w}},X_{\mathrm{G}})\end{array}\right.$$

(22)

We use vision processing to process images to obtain the true position of each corner:${X_{\mathrm{pix}}}^{\ast }=(u^{\ast }, {\mathrm{v}}^{\ast })$. It can be seen from Equation (21) that two equations can be established for each corner, and a total of 10 parameters need to be calibrated. Therefore, in theory, only five corners’ information is needed to determine the undetermined parameters. The actual calibration process, due to the existence of uncertain factors such as image noise using only five corners for calibration parameters, is inappropriate, and, therefore, the checkerboard pattern used in this article has dozens of corners, establishes an overdetermined equation set, calculates the sum of square errors of the corners according to the calculated value and the true value of the position of corners, and uses it as the evaluation function. As shown in Equation (23), we use the genetic algorithm to search for underwater camera parameters that minimize the sum of the squares of all corners’ errors.

$$\epsilon =\frac{1}{n}{\displaystyle \sum _{i=0}^{n-1}\left({‖u_i-{u_i}^{\ast }‖}^2+{‖v_i-{v_i}^{\ast }‖}^2\right)}$$

(23)

In the above formula, n represents the number of corners on the checkerboard pattern.

Optimization strategy. Since each picture has six external parameters, if m pictures are passed in, there are a total of 6m external parameters and 4 camera refraction parameters (camera refraction parameters are iteratively optimized in each optimization strategy), and the number of optimized parameters is 6m + 4. There will be too many parameters, and it is easy to encounter local optimization. So here, we use three strategies to carry out, and analyze them.

• The first strategy remains unchanged. We input ten calibration images into the aforementioned algorithm and optimize 6 × 10 + 4 parameters for each update.
• The second strategy takes into account the number of parameters. With each image input, we update 6 + 4 parameters and utilize the converged refraction parameters as the initial values for subsequent image updates.
• The third strategy builds upon the second one. Here, we directly utilize Zhang’s [17] calibrated external parameters (without considering refraction, treating the image as if it were in the air for calibration) as the initial values for each iteration. Additionally, the external parameter × (1 ± 0.5) serves as the upper and lower boundary values. This approach reduces the number of iterations and enhances local optimization capabilities.

5. Experiments

5.1. Experimental Data and Equipment

The experimental setup primarily comprises a custom-made underwater monocular camera (resolution: 1080 p, frame rate: 30 fps, interface: RJ45 Ethernet interface, field of view: 70°) and power supply, along with a checkerboard pattern (consisting of six rows and nine columns, totaling 54 target corners, with a 25 mm distance between corner points), and a water tank. To ensure the accuracy and reliability of the calibration results, we acquired calibration images from different distances and angles. Initially, without water in the tank, we captured images in the air to calibrate initial intrinsic parameters and distortion parameters. Subsequently, after filling the tank with water, we submerged the camera and checkerboard pattern, capturing underwater calibration images at various poses and distances.

The images captured in the air are shown in Figure 4, while the images captured in the water are shown in Figure 5. In the experiment, we utilized Python3.8.5 to handle the data and employed the OpenCV4.4.0 library for image data processing.

5.2. Different Optimization Strategies

Firstly, regarding the choice of genetic algorithm, we adopt the elitist-reservation genetic algorithm [28], with the algorithm process shown in

Table 2. Genetic algorithm flow chart.

Initialization: Define the objective function f(x) as depicted in Equation (23) Initialize a population of N individuals using various strategies. Define the maximum number of iterations Tmax and the target value F.

2. Iteration: If: the iteration count < Tmax or f(x) ≠ F Conduct statistical analysis on the current population, recording its best individual, average fitness. Independently select N − 1 parents from the current population. Independently perform crossover operations on these N − 1 parents. Independently perform mutation operations on these N − 1 individuals after crossover. Calculate the best individual of the current generation population and insert it at the first position among the N − 1 individuals after crossover to obtain the new generation population. Else: Terminate.

3. Termination: End the algorithm.

.

In the experiment, we utilized ten underwater calibration images, as depicted in Figure 1.

The first strategy: We update the ten pictures together and set the upper and lower limits of the angle to [−π, π], the upper and lower limits of the translation to [−500, 500], set the population size to 100, update 2000 times, and obtain the errors. The objective function values of the population and the best individual for Strategy 1 are depicted in Figure 6, the images after reprojection are shown in Figure 7, the errors are illustrated in Figure 8, and the calibration result is shown in

Table 3. Re-projection errors with the first strategy.

Images	Re-Projection Errors/Pixel
1	4.2749
2	6.8761
3	7.3536
4	6.5965
5	9.1818
6	9.9528
7	1.7927
8	8.0620
9	12.3986
10	20.9183
Average	8.7407

.

Based on our analysis, it can be inferred that the algorithm exhibits poor convergence, considerable re-projection errors, and encounters challenges associated with local optimization.

The second strategy: We pass each picture into the algorithm and pass the converged refraction parameters updated last time into the next image. We set the upper and lower limits of the angle to [−π, π], the upper and lower limits of the translation to [−500, 500], set the population size to 100, update 500 times, and obtain the errors. The objective function values of the population and the best individual for Strategy 2 are depicted in Figure 9, which is the result of the last image, the images after reprojection are shown in Figure 10, the errors are illustrated in Figure 11, and the calibration result is shown in

Table 4. Re-projection errors with the second strategy.

Images	Re-Projection Errors/Pixel
1	2.8960
2	1.6936
3	1.9811
4	3.3712
5	5.0023
6	2.7538
7	4.3243
8	3.9146
9	0.4683
10	0.3065
Average	2.6712

.

However, a notable issue arises in this context. For each picture, there are unique refraction parameters. Here we analyze that the upper and lower limits of the external parameters are too large to cause this. The calculated refraction parameters are not applicable to the previous pictures. It can be seen from Figure 11 that the last time the convergence is better, but the previous convergences are poor.

The third strategy: First, we directly apply Zhang’s calibration [17] to the underwater images to obtain the corresponding external parameters for use in the genetic algorithm. The external parameters are shown in

Table 5. Approximate value of external parameters.

Rotating Vector	Translation Vector
−0.25208, −0.33484, −3.06365	113.5505, 62.3148, 540.9440
−0.20195, −0.34487, −2.89052	79.9068, 110.4759, 534.3428
0.11674, 0.32511, −2.53928	3.759, 147.6164, 750.6942
0.179883, 0.13025, −2.30014	−67.7814, 192.1184, 766.6448
−0.3604, 0.77093, −2.97292	155.2326, 75.8162, 714.0783
−0.32534, 0.76271, −2.80196	128.8083, 119.4154, 736.8143
−0.07531, −0.01894, −2.61955	−26.1963, 159.760, 567.3251
−0.25969, 0.25929, −2.61589	−17.6367, 152.3984, 579.6227
0.33259, −0.55940, −2.83098	79.1891, 111.7601, 340.2037
−0.21901, 0.53206, −2.70773	103.839, 87.2968, 327.9117

.

In order to avoid the “false” phenomenon (The parameters in Strategy 2 only apply to a subset of the images) of external parameters and refraction parameters, we multiply the approximate value of the external parameter obtained above by (1 ± 0.5) as the upper and lower boundary values. We pass each picture into the algorithm and pass the converged refraction parameters updated last time into the next image. We set the population size to 100, update 500 times, and obtain the errors. The objective function values of the population and the best individual for Strategy 3 are depicted in Figure 12, which is the result of the last image, the images after reprojection are shown in Figure 13, the errors are illustrated in Figure 14, and the calibration result is shown in

Table 6. Re-Projection errors with the third strategy.

Images	Re-Projection Errors/Pixel
1	0.1776
2	0.1513
3	0.1737
4	0.1772
5	0.1939
6	0.2168
7	0.1965
8	0.2381
9	0.2184
10	0.2146
Average	0.1958

.

After obtaining the calibration result, we pass the parameters into the model and calculate the error of each corner. Since we set the target value to 0, the iteration did not terminate, but it can be observed that the iteration has converged. The error is very small and performs well in all images. Therefore, we believe that we have obtained the optimal solution.

5.3. Comparative Experiment

To validate the effectiveness of our model and algorithm, we conducted comparative experiments. We used ten checkerboard images for camera parameter calibration using Li’s [20] method based on equivalent focal length, Bashar’s [22] method based on an approximate linear model, and our proposed method. We then used the corresponding parameters for re-projection, and selected average error, root mean squared error (RMSE), and re-projection error as comparison metrics. The errors of the comparative methods are shown in Figure 15 and

Table 7. The various errors of different methods.

Method	Average Error	RMSE	Re-Projection Error
Li’s method [20]	0.1544	0.4213	0.2590
Bashar’s method [22]	0.2047	0.4166	0.3276
Ours	0.1264	0.2285	0.1958

. It can be observed that our method performs better: compared with Li’s method, our method reduces re-projection error by 24.4%, and, compared with Bashar’s method, it reduces it by 40.2%.

6. Conclusions

This paper proposes a correction algorithm based on the real light path’s double refraction model and demonstrates its necessity in underwater camera usage. During the iterations of the genetic algorithm, certain rules are applied to overcome local optima. Experimental results show that our algorithm accurately computes camera parameters, simplifying the imaging calibration process and providing a feasible method for underwater camera calibration. Comparative experiments reveal that our method reduces the re-projection error by 24.4% compared with Li’s method [17] and by 40.2% compared with Bashar’s method [19]. The model reduces errors in underwater measurements, improving measurement accuracy, and enhancing underwater positioning and SLAM. In future work, we will apply this model in real underwater environments and address some of the challenges it brings. Additionally, we will apply the model to underwater target positioning, and we also plan to develop target positioning software with this model and positioning technology. We intend to apply it in the field of underwater detection.