Prediction of wharf subsidence deformation degree based on deep learning technology

2.1. GRU algorithm in deep learning algorithm

The basic principle of predicting the degree of wharf subsidence deformation is to predict the future data based on the current and historical data [9]. Deep learning algorithms, such as the recurrent neural network (RNN), are highly appropriate for analyzing such time series data. The GRU is an enhancement of long short-term memory (LSTM) networks. It merges the forgetting gate and the input gate into an update gate, thereby reducing the computational burden of the algorithm [10], which is beneficial to the rapid early warning of changes in wharf subsidence. The forward formula is:

where $z_t$ is the update gate output [11], $r_t$ is the reset gate output, ${\omega }_z$ and ${\omega }_r$ are the weight in the update and reset gates, $x_t$ is the current input, $f\left( \right)$ is the activation function adopted by the update and reset gates, $h_{t-1}$ is the hidden state of the previous moment, $h_t^{\text{'}}$ is the temporary hidden state of the present moment, $\omega$ is the weight when calculating $h_t^{\text{'}}$, and $h_t$ is the hidden state of the current moment.

2.2. Prediction of wharf subsidence deformation

The wharf is an important distribution hub in water transportation. One of the characteristics of water transportation is its capacity for large freight volumes, which leads to the need for the wharf to bear a large amount of cargo load. Typically, the wharf usually extends from the shoreline into the water (sea) region [12]. As a hydraulic structure, a wharf will be deformed during operation due to the influence of cargo loads, natural environments, and other factors. If the deformation exceeds a certain threshold, it can result in irreversible damage, which will eventually cause damages to wharf staff and cargo safety [13]. Therefore, in order to guarantee the safety of the wharf, it is necessary to monitor the subsidence deformation and then make predictions according to the monitoring data, so as to prepare coping strategies in advance.

Fig. 1Training flow of wharf subsidence deformation prediction based on a deep learning algorithm

In this paper, a deep learning algorithm is employed to forecast the degree of subsidence deformation at the wharf, as shown in Fig. 1.

(1) A monitoring point is set at an interval in the wharf area to be monitored [14].

(2) An unmanned aerial vehicle (UAV) equipped with a camera is used to capture images of the wharf area, and the collection interval is determined by specific requirements.

(3) The DEM of the wharf area is constructed based on the monitoring points in the UAV-captured images. The construction process is as follows. Firstly, scale-invariant feature transform (SIFT) features are utilized to identify the monitoring points in the images. Then, the camera pose is estimated using these monitoring points, and the actual coordinates of the monitoring points are calculated to create a sparse point cloud. Then, the multi-view stereo (MVS) image matching algorithm is applied to generate a dense point cloud. Finally, the DEM of the pier area is derived by filtering and interpolating the dense point cloud [15].

(4) The DEM of the wharf area is organized chronologically by day, and then the samples are constructed in the form of “4+1”. The form of “4+1” means that in the continuous-time DEM, the DEM of four consecutive days is taken as the input data of the sample, and the DEM of the fifth day is taken as the actual forecast data of the sample (label).

(5) The input data from the training sample is input into a CNN, and the convolution and pooling layers of the CNN are used to extract and compress the convolutional features of the training sample.

(6) The convolutional features extracted by the CNN are fed into the GRU according to the time series for forward calculation.

(7) After the GRU forward calculation results are obtained, they are compared with the sample labels to determine whether the error converges to the preset threshold. If it does, the training is considered complete. If not [16], the parameters in the GRU and CNN are reversely adjusted according to the error, and the process returns to step (5).

3.1. Case overview

The Longtan Port area of Nanjing Port in Jiangsu Province (Fig. 2) was taken as the subject for a case analysis. Nanjing Port is located in the middle and lower reaches of the Yangtze River. Its coordinates are 118°44’ east longitude and 32°05’ north latitude. The entire Nanjing Port area is 98 km long. Longtan Port is an important part of Nanjing Port. It is a container wharf with a coastline of 910 m and an average water depth of 12.5 m at the front of the wharf. Longtan Port has a vast hinterland of cargo sources, excellent transportation conditions, and convenient access for both land and water transport.

Fig. 2Longtan Port area of Nanjing Port

3.2. Analysis and setting

The UAV used for image acquisition in the port area was a DJI Mini 4 Pro, equipped with a camera with 48 million effective pixels. It has a maximum flight time of 34 min. The navigation system employed was a combination of the global positioning system (GPS) and BeiDou.

When setting up monitoring points in the Longtan Port area, a monitoring point was positioned every 50 m along the wharf (within the red box in Fig. 2). Then, the UAV flew along the designated route, and captured images, including the monitoring points. Images were collected at 12 o'clock every day from September 2022 to September 2023. After eliminating images affected by bad weather, a total of 300 remote sensing images of Longtan Port were finally collected.

CNN and GRU algorithms were combined, and the relevant parameters of the combined prediction algorithm for wharf ground subsidence deformation degree are shown in Table 1. The CNN algorithm was used to extract image features from the DEM built by monitoring images, and then the features were input into the GRU to predict wharf subsidence. The specifications of the CNN input layer depended on the specifications of the matrix during the construction of training and testing samples. The form of “4+1” was adopted when constructing samples, i.e., the wharf DEM for four consecutive days and the wharf DEM for the 5th day. However, since the wharf DEM belongs to a point cloud map, where each point is regarded as a detection point, inputting the entire DEM into the algorithm for training at once would result in a heavy computational load. Therefore, when constructing the sample set in the “4+1” form, n detection points were selected from the DEM, the DEM data of the n detection points for four consecutive days constituted a 4×$n$ input matrix, and the DEM data on the 5th day was the output label for the sample. In order to facilitate training and testing, $n$ was set to 6, then the input layer specification of the CNN was set to 4×6.

The convolutional layers of the CNN are the key to extracting image features. The more layers there are, the more feature details will be extracted, but the amount of computation will also increase. Therefore, 1, 2, 3, 4, and 5 convolutional layers were set respectively, and the activation function in the convolutional layers was set to relu, sigmoid, and tanh, respectively, to test the performance of the algorithm under different convolutional layer numbers and activation functions.

In addition, in order to further validate the performance of the proposed algorithm, it was compared with other algorithms, LSTM and GRU, which are essentially extensions of RNNs and share a similar structural framework. The parameters for the structure settings were referred to the GRU part of the combined algorithm.

3.3. Evaluation index

The mean absolute error (MAE) and root mean square error (RMSE) were used to assess the performance of the algorithm. The formulas are:

where MAE is the mean absolute error, RMSE is the root mean square error, $n$ is the number of observation days of wharf subsidence, $y_t$ is the actual observed value of the detection point on the $t$-th day, and $y_t^{\text{'}}$ is the predicted observation value of the detection point on the $t$-th day.

3.4. Test results

The performance of the combined algorithm was evaluated using different CNN activation functions and varying numbers of convolution layers. As shown in Table 2, under the same CNN activation function, the error index of the algorithm decreased with the increase of the convolutional layers. However, after three convolutional layers, the error index showed minimal change. Under the same number of convolutional layers, the algorithm using sigmoid as the activation function exhibited the lowest error index. Considering these results and the computational amount of the algorithm, the activation function was finally set to sigmoid, and the number of convolutional layers was set to three.

Then, the performance of the three prediction algorithms was compared (Table 3). The MAE and RMSE of the LSTM algorithm for predicting the wharf ground subsidence were 0.178 and 0.184, respectively. The values for the GRU algorithm were 0.114 and 0.126, respectively. The values for the CNN+GRU algorithms were 0.079 and 0.087, respectively. It can be seen that the CNN+GRU algorithm had the best performance in predicting wharf ground subsidence and consumed the least time.

The average subsidence in the wharf area after some monitoring days and the prediction deviation of the subsidence amount by the three algorithms are shown in Table 3. The actual observed elevation value at a specific monitoring point in the wharf area over a specific time frame and the predicted elevation changes of the three algorithms are shown in Fig. 3.

As time went by, the elevation of the monitoring points gradually decreased, indicating subsidence; however, it stabilized after a certain period, indicating that the subsidence was not serious. The average subsidence changes of the entire wharf area in Table 4 further confirm that the subsidence was occurring, but it remained stable and not particularly serious. Moreover, by comparing the predicted elevation values from the three algorithms and the two evaluation indicators in Table 4, it can be seen that the predictions made by the CNN+GRU algorithm were closest to the actual observed values.

Fig. 3The actual observed value of a monitoring point in the wharf area within a period of time and the predicted value of the three algorithms