YOLOv3-MSSA based hot spot defect detection for photovoltaic power stations

3.1. Design target project information of thermal spot defect detection method for photovoltaic power plants

The paper selects a photovoltaic power plant with an installed capacity of 100 MWp in Heilongjiang Province, China as the background project. The project is located at an altitude between 140 m and 185 m, and the site is saline alkali land. The annual sunshine hours in the project area exceed 2200 hours, reaching the lowest monthly sunshine hours in December and the highest monthly sunshine hours in June. The average annual total solar radiation at the project location fluctuates between 4400 MJ/m² and 5100 MJ/m². The photovoltaic power station selects 330 Wp polycrystalline silicon modules and uses a fixed installation method to install solar cell modules. The spacing between photovoltaic arrays should be maintained at over 14 m, with a tilt angle of 38 degrees. The spacing between the front and rear photovoltaic arrays should be greater than 11 m. The specification of the centralized inverter is 2.5 MW. The 35 kV circuit breaker utilizes a breaking current of 31.5 kA. The transformer uses a three-phase integrated transformer. The project belongs to a large-scale photovoltaic power station project.

3.2. Design of an improved YOLOv3-based target detection method for PV strings

Common photovoltaic power stations use photovoltaic cells to generate electricity, using the photovoltaic effect to convert light energy into electricity. When sunlight reaches the surface of a photovoltaic cell, the contained P-N in the semiconductor is activated, and the covalent bonds between the electrons are broken [15, 16]. Fig. 1 illustrates the photovoltaic power generation process.

Fig. 1Photovoltaic power generation process

In Fig. 1, illumination of a photovoltaic cell results in the formation of hole and electron pairs. These pairs then migrate towards the N and the P region. As electrons aggregate in the P section, it gains a positive charge while holes aggregate in the N section, causing an inactive charge, ultimately generating an electric potential. The electric potential is fed into the controller and is then distributed and exchanged with the electrical energy stored in the battery bank for auxiliary power generation. Depending on the need, the electrical energy is selected for direct DC load output or for AC load output from the inverter [17, 18]. The YOLO algorithm employs a CNN exclusively to forecast the class and location of diverse targets, affording a favorable computational velocity and enhancing detection efficiency for hot spot defect detection. The YOLOv3 model is utilized accordingly for PV string target detection in PV plants. In many instances, the infra-red video of the PV string is influenced by multiple factors, leading to an overabundance of information in the image. Hence, preprocessing is applied first. OpenCV is utilized to convert the color images to greyscale, thereby eliminating redundant color information and minimizing data for future calculations. Otsufa binarisation is introduced to establish the segmentation threshold for distinguishing the target from the background. A grey-scale histogram of the infrared image is created and the probabilities of the PV group string regions and background regions are calculated as shown in Eq. (1):

where, $N$ is the sum of pixels; $i$ represents the grayscale level; $p_i$ represents the proportion of pixels with a grayscale level of $i$; $t$ represents the selected threshold value; $p_A$ represents the probability of the PV string region; $p_B$ represents the probability of the background region. The grey scale values for the PV string region and the background region are calculated as shown in Eq. (2):

where, ${\omega }_A$ and ${\omega }_B$ refer to the mean grey value of the PV string area and the background area. The average gray level of the infrared image for the PV string is determined by using the gray level values from various regions, as depicted in Eq. (3):

where, ${\omega }_0$ represents the mean grey level. The interclass variance is computed by obtaining the average grayscale of every region and the PV group string IR image. This is demonstrated in Eq. (4):

where, $\sigma$ represents the inter-class variance. The grey values are processed to obtain the Otsufa binary image once the inter-class variance reaches a significant value. Coupled Head detection head is used by the YOLOv3 model for task detection and classification, as depicted in Fig. 2.

Fig. 2Coupled head network architecture

In Fig. 2, YOLOv3's Coupled Head is Anchor-Based, where detection and classification share a single convolution. Also, data is dimensionally reduced during classification and regression, leading to fewer parameters to differentiate classification and regression features [19]. The decoupling header contains one convolutional layer with a reduced number of channels and two convolutional branches [20]. Figure 2 depicts YOLOv3's Anchor-Based Coupled Head, where detection and classification utilize a single convolution. Additionally, data is dimensionally decreased during classification and regression, resulting in fewer parameters for distinguishing classification and regression features [19]:

where, $z_c^h\left(h\right)$ represents the output of the $c$ channel with a height of $h$. $z_c^w\left(w\right)$ represents the output of the $c$ channel with a width of $w$. The location of the target area of interest is determined from the output coordinate positions and the output of the CAM is shown in Eq. (6):

where, $y_c\left(i,j\right)$ contains the attention in both vertical and horizontal directions. $g_c^h\left(i\right)$ and $g_c^w\left(j\right)$ are obtained by convolving the horizontal and vertical directions in the spatial dimension. The CAM is separated into two one-dimensional feature vectors upon feature encoding completion, integrating the spatial location information into the mapped feature map. The self-attentive module utilizes the reference of target pixel weights in the two-dimensional image. The input signal is downsampled and transposed, then dimensionally restored via convolutional operations, as demonstrated in Eq. (7):

where, $w_z$ represents the learnable weight matrix and the self-attentive module can be plug-and-play when the value of $w_z$ is 0. The structure of the obtained improved YOLOv3 algorithm is Fig. 3.

Fig. 3 shows the placement of the CAM after the 1/16 resolution feature map to extract map channel and location details, which are then fed into the next layer for further processing. The SAM, on the other hand, is set after the 1/32 resolution feature map to capture relational information regarding location and spatial data. The use of the PV string in a low-resolution feature map environment enhances target detection capabilities.

Fig. 3Improved YOLOv3 algorithm structure

3.3. Design of a hot spot defect detection method incorporating multiobjective sparrow search algorithm

The sparrow algorithm is based on sparrows' foraging and anti-predation behavior, which divides group behavior into interconnected individual behaviors and transforms local optimal solutions into global ones. It has strong optimization capabilities for complex system problems. To enhance the accuracy of detecting hot spot defects in photovoltaic power stations, the DBN model is integrated into MSSA for optimization. DBN is formed by stacking multiple Restricted Boltzmann Machines (RBMs), which are capable of feature extraction and learning using parameters [21, 22]. The RBM is comprised of two layers: the hidden layer and the visual layer. When the condition of the visual layer is certain, the neurons in the hidden layer will be independent, facilitating parallel training of neurons thereby reducing the required training time. The energy calculation between the visual and hidden layers is demonstrated in Eq. (8):

where, $s$ represents the visual layer; $h$ represents the hidden layer; $m$ represents the number of neurons; $b_i$ represents the visual layer bias; $s_i$ represents the visual layer neurons; $c_j$ represents the hidden layer bias; $h_j$ represents the hidden layer neurons; $w_{ij}$ is the neuron connection weight. The probability distribution is calculated as shown in Eq. (9):

where, $P\left(s,h\right)$ represents the probability distribution. The activation conditions of neurons have independent properties, and when the state is determined, the hidden layer activation probabilities can be calculated. The calculation of the neuron activation probability is shown in Eq. (10):

where, $P\left(h_j=1\left|s\right.\right)$ represents the activation probability of the $j$ neuron; $\sigma \left(x\right)$ is the sigmoid function. After identifying the hidden layer’s neuron states, calculating the activation probability of visual layer neurons follows. This process is demonstrated in Eq. (11):

where, $P\left(s_i=1\left|h\right.\right)$ represents the activation probability of the $i$ neuron. After determining the status of the visible layer, the status of the hidden layer can be recalculated. The result is then used to project the status of the visible layer once again. The deviation between the two results is then calculated and the value is continuously adjusted to bring the deviation as close as possible to 0. The structure of DBN is shown in Fig. 4.

Fig. 4DBN structure

In Figure 4, the DBN structure contains several RBNs and a Back Propagation Network (BP) at the top layer. The first RBM includes a first visible layer and a second hidden layer. The second RBM involves a second and a third layer. The RBM gradually passes data up after input. If the data value deviates significantly from the true value upon reaching BP, it is back propagated until it matches the expected value before being output [23]. The initial step is the pre-training phase in which the input data is propagated forward and backward to optimize parameters locally at each stage. Then comes the fine-tuning phase, involving successive supervision of the data and fine-tuning using a parameter matrix to transform the network from a local to a global optimum. After the defective data is completed and collated, the data is normalised as shown in Eq. (12) due to the poor correlation of the parameters, resulting in a difference in the proportion of weights accounted for:

where, $a^{\mathrm{*}}$ represents the data obtained after normalization; $a$ represents the original data; $\stackrel{-}{a}$ means to the mean value of the covariates; $a_{\sigma }$ is the standard deviation of the data. Among the fault data, the maximum power voltage, open circuit voltage, short circuit current, and current are the most representative characteristic values. They are used as input variables to establish five diagnostic results: normal, short circuit, aging, open circuit, and shadow. The fault diagnosis model of DBN is Fig. 5.

Fig. 5DBN fault diagnosis model

In Fig. 5, the input layer receives four types of feature data, which are processed. The model becomes self-learning upon entering the hidden layer, and all hidden layers complete the data processing, followed by outputting the data to the classifier for classification. Subsequently, the classified results are presented in the diagnosis categories. The experimental parameters need to be continuously adjusted during the model training until the best results are achieved. The sum of sparrows contained in the virtual sparrow population in MSSA is set to $m$, and the population fitness is calculated as shown in Eq. (13):

where, $G_Y$ represents the population fitness; $n$ represents the number of variable dimensions. During the process of foraging, the probability of a sparrow finding food first increases with its fitness level. Once food is found, the sparrow guides other members of the population and expands the search. To determine the location of food, refer to Eq. (14):

where, $Z_{i,j}^{d+1}$ represents the location of the food after multiple iterations. $d$ represents the iterations in the foraging process. $\alpha$ is a random number greater than 0 and less than 1. $\beta$ represents the limit on the max iterations. $Z_{i,j}^d$ represents the $i$-th sparrow location in the $j$-th dimension after $d$ iterations. $Q$ is a normally distributed random matrix. $L$ is a unit column vector. The flow of the sparrow search algorithm is Fig. 6.

In Fig. 6, the max iterations need to be set when the population is initialised, after which the population adaptation value calculation is carried out and the search for an individually and globally optimal solution begins. The speed and position are updated continuously, and the adaptation value is recalculated throughout the solution process. The search concludes when the calculation outcomes meet the optimal value condition. The probability of a sparrow finding food during foraging is shown in Eq. (15):

where, $P_j$ represents the probability of the first $j$ sparrow finding food. $Fitnes{s}_k$ represents the adaptation value of the first $k$ sparrow. The optimized MSSA hotspot defect detection method is combined with the YOLOv3 PV string target detection method to obtain a YOLOv3-MSSA-based defect detection way for PV power plants.

Fig. 6Sparrow search algorithm process

4.1. Simulation performance testing of hot spot defect detection methods

To examine the YOLOv3-MSSA based defect detection method’s effectiveness for hot spot defect detection in photovoltaic power stations, the study initially evaluated the algorithm's system performance. Table 2 displays the basic software and hardware environment settings.

The square input image has a side length of 640 mm. The batch size is 4 and the weight attenuation is 0.0005. Additionally, the hue enhancement factor is 0.015, while the saturation enhancement factor is 0.7 and the luminance enhancement factor is 0.4. For the model training, 1800 out of the 2000 images in the dataset are used, while 200 are reserved for testing. Compare with YOLOv4s algorithm, YOLOv5s algorithm, and Fast Regions with Convolutional Neural Network Features (Faster-RCNN) algorithm during testing. The loss curve is shown in Fig. 7.

The YOLOv4s model achieved an initial loss value of 0.092, followed by a decrease to 0.024 after 400 iterations, and no significant loss curve fluctuations in the first 100 iterations. The Faster-RCNN model had an initial loss value of 0.058, which decreased to 0.017 after 400 iterations, with no substantial decline fluctuations. Finally, the research method reached an initial loss value of 0.055, which dropped to 0.009 after 400 iterations, with no major fluctuations in the decline. This indicates that the research method has good convergence and performs better on training data. The recall of the research method was tested to generate a P-R curve and the results are shown in Fig. 8.

Fig. 7Model loss curve

a) YOLOv4s

b) YOLOv5s

c) Faster-RCNN

d) Research method

Fig. 8P-R curve

a) YOLOv4s

b) YOLOv5s

c) Faster-RCNN

d) Research method

In Fig. 8, the P-R curves for all models are roughly symmetrical along the line of equal horizontal and vertical coordinates, with Precision values decreasing slowly before reaching the line of equality and starting to decrease sharply after reaching the line of equality. The Precision values for all models’ P-R curves are 1 at Recall value 0 and 0 at Recall value 1. YOLOv4s, YOLOv5s, and Faster-RC’s P-R curves display a rapid decline in Precision at a Recall value of approximately 0.91, 0.93, and 0.93, respectively. The P-R curve for Faster-RCNN exhibits a substantial decline at a Recall value of around 0.95, while the P-R curve for the experimental method shows a sharp decrease at a Recall value of approximately 0.96. The research method can detect a greater number of hot spot defects in photovoltaic power plants, reduce missed detections, and provide improved accuracy in detecting hot spot defects. Additionally, this methodology can better adapt to varying sizes and shapes of hot spot defects.

4.2. Application efficiency test of hot spot defect detection method

To ensure that the research method has good efficiency in the detection of hotspot defects in real PV power stations, the application effectiveness of the research method also needs to be tested. To ensure the recall, accuracy, and F1 value of the method do not exhibit significant anomalies, a confidence threshold of 0.5 is set. The F1 value, average time consumption, and parameter quantity of the research method were tested, and the results are shown in Fig. 9.

Fig. 9F1 value, average time consumption, parameter quantity test results

a) F1 value

b) Average time consumption

b) Parameter quantity

In Fig. 9, in terms of F1 values, YOLOv4s is 0.906. YOLOv5s is 0.960. Faster-RCNN is 0.961. The research method is 0.964, which is higher than the other methods. In terms of average elapsed time, YOLOv4s is 0.055 ms. YOLOv5s is 0.023 ms. Faster-RCNN is 0.192 ms. The research method is 0.010 ms, which is significantly lower than the other methods. In terms of the number of parameters, YOLOv4s is 6.29×10⁷. YOLOv5s is 7.06×10⁶. Faster-RCNN is 2.85×10⁷. The research method is 3.46×10⁶, which is significantly less than the other methods. The presented findings suggest that the utilized research method can strike a fine balance between achieving high accuracy and recall rates while requiring a lower average detection time. Fewer parameter quantities indicate that the research method can effectively capture key information, reducing the system pressure during subsequent calculations. The detection accuracy of the research method is tested for different defect fault types and the results are shown in Fig. 10.

In Fig. 10, the detection accuracy of Deep Convolutional Neural Network (DCNN) is 89.56 % in normal condition, 82.35 % in short circuit condition, 87.1 % in open circuit condition, 90.13 % in shadow condition and 85.51 % in aging condition. The detection accuracy of DBN is 92.60 % for DBN in normal conditions, 93.52% for short circuit, 90.33 % for open circuit, 91.68 % for shaded condition and 92.36 % for aged condition. The detection accuracy of the research method is 96.26 % in the normal case, 97.57% in the short circuit case, 98.88 % in the open circuit case, 97.63 % in the shadow case and 98.2 % in the ageing case. All of these data are higher than the other methods. This indicates that the research method has good adaptability for different types of defect faults, thus ensuring a high detection accuracy during detection. To provide an in-depth analysis of the detection accuracy of the study method, a confusion matrix is created for the test set results, as shown in Fig. 11.

Fig. 10Diagnostic accuracy

a) DCNN

b) DBN

c) Research method

In Fig. 11, the five detection results of normal, short circuit, ageing, open circuit and shadowing are set to numbered tags 1-5. The misjudgment of faulty states for labels 2, 4, and 5 resulted in faulty state 3 being incorrectly identified as faulty state 4 on six occasions. The high frequency of false positives is normal due to circuit resistance becoming too high in circuits that have aged severely, resulting in the same effect as an open circuit fault. Overall, the detection accuracy of the research method reaches 98.86 %, indicating that the research method has a good judgement accuracy and is able to make good judgements for all types of hot spot defect faults. To provide a more comprehensive test of the performance of the research method, the ROC rectangular curve of the research method is plotted as shown in Fig. 12.

In Fig. 12, the ROC curve of the original DRN model is farthest from the upper left corner (U-LC) of the ROC rectangle and surrounds the smallest region. This indicates that the original DRN performs poorly in environments with changes in irradiance and temperature in photovoltaic power generation. The rectangle enclosed by AlexNet and DCNN has a larger area. The ROC curve of the research method is closest to the U-LC of the ROC rectangle and surrounds the largest region. This indicates that the research method can work smoothly in environments with changes in irradiance and temperature in photovoltaic power plants. It has good performance in hot spot defect detection. Table 3 shows the resources required for the operation of analytical research methods.

In Table 3, the research method only requires three operators to complete the operation during runtime. The minimum processor performance required at runtime is consistent with the Faster RCNN method and slightly lower than the DCNN method. When conducting the complete detection process, the DCNN method takes up to 132 seconds, the Faster RCNN method takes up to 86 seconds, and the research method only takes 21 seconds. The research method requires lower labor and material costs in actual operation.

Fig. 11Confusion matrix

Fig. 12ROC rectangular curve