As autonomous driving technology becomes more and more popular, its safety is also attracting attention. Regarding the automatic driving of vehicles, the detection of road markings is particularly important. This paper improves the lane edge detection part of the Hough transform lane line detection method. Because the traditional Canny operator edge detection method is good for image processing, but the detection time is long, this paper replaces the Canny algorithm with the Robert operator edge detection method. The sub-edge detection method can improve the detection speed of lane line extraction. In MATLAB, by using multiple edge detection operators to perform edge detection on the same image 100 times, and taking the average of the detection time, it is found that the Robert operator takes a shorter time in the detection process than the Canny operator; Then the Robert operator and Canny operator are respectively fused into the Hough transform lane line detection. After 100 times, the same image is detected, and the running time is statistically averaged for comparison and analysis. The Robert operator is better than the Canny operator. The time taken is reduced by 0.15191 s. The simulation results show that the integration of Robert operator in Hough transform lane line detection improves the real-time performance of lane detection.
- By comparing the four image edge detection operators of Robert, Sobel, Prewitt, and Canny on the detection time of the same image, the study found that the Robert operator took the shortest time.
- Integrate the Robert operator into the traditional Hough transform to detect lane lines. The study found that the integration of Robert operator makes the entire lane line detection time shorter than the traditional lane line detection time by 0.15191s.
- The similarity between the lane line detection result image with Robert operator and the traditional lane line detection result image is 0.99576.
At present, most automobiles in my country are still at the L1~L2 level . For vehicles to realize automatic driving, the detection of lane markings is the key content of automatic driving. Only when the lane markings are accurately detected can the vehicle drive safely. In the process of lane line detection, the focus is on the detection of the edges of the collected images.
Based on edge detection. MAMTA et al.  proposed a robust edge detection algorithm based on multiple thresholds, which can better deal with the edge continuity and thickness uniformity of the image. Zhou Fang et al.  proposed an edge detection operator with direction. This operator has fewer broken lines and low noise in the process of road edge extraction, and has obvious advantages in the extraction of linear landmarks. Kong et al.  improved the Canny algorithm and proposed an adaptive edge detection model to achieve better edge detection accuracy and robustness. Sun Qingfeng  used a wavelet transform edge detection method to extract image edge features, which effectively resolved the contradiction between edge location and noise. Xu et al.  proposed a sparse edge-preserving image denoising method, which improves the denoising effect of a single noise. Chen Zhuan et al.  used the advanced Sobel operator and zernike moment image to detect the route contour of the remote sensing image, and processed the edge information of the image better, but there were still some defects in the edge connectivity. Han et al.  used mathematical morphology operators and gradients to extract edge pixels of grayscale images, which improved the processing effect of Gaussian noise and salt and pepper noise. HEY et al.  proposed an image edge detection algorithm based on improved morphological gradient, which can accurately extract image edge information. Yang Donghua et al.  studied the construction of the LOG template in the Marr edge detection algorithm, and obtained a wealth of image edges, and the continuity of the edges was good. Tan Yuan et al.  carried out edge detection on remote sensing image roads based on the improved Sobel operator, and detected that the edge lines of the roads were smooth and complete, with clear contours and good continuity.
For the detection of lane line edges, the current focus is on the robustness of detection and recognition, noise resistance, and the accuracy of edge detection. There are few real-time detection studies. In this paper, based on the detection of the lane line of the Hough transform, by comparing several image edge detection methods, a Robert operator edge detection method is found to detect the edge of the lane. Comparing the Robert operator edge detection method with the traditional Canny operator edge detection method, it is found that the Robert operator edge detection method takes less time than the Canny operator edge detection method. The Robert algorithm is integrated into the Hough transform to detect the lane line, so as to shorten the lane line detection time.
2. Image edge detection method
Edge is the most basic feature of an image. Edge detection plays an important role in computer vision, image analysis and other applications. This is because the edges of the image contain useful information for identification. The purpose of image edge detection is to remove excessive interference information in the image, retain the main information of the target object, and greatly reduce the number of white spots in the image. The edge of the object in the digital image reflects the large difference in the gray value of adjacent pixels, and these differences mainly include changes in the form of steps and pulses.
2.1. Image edge detection operator
The essence of edge detection is to use algorithms to extract the boundary between the object and the background in the image. The edge is defined as the boundary of the region where the gray scale changes sharply. The classic edge detection method is to construct an edge detection operator based on a small neighborhood of pixels in the original image. The following is a theoretical analysis of several classic edge detection operators.
(1) First-order edge detection operator.
In an image, if the gray value change between adjacent points is small, the amplitude of the gradient will be small, and on the contrary, the amplitude of the gradient will also be large. Therefore, the edge pixel position of the image can be obtained by calculating the derivative. In the actual image during processing, often using a small convolution template to replace the partial derivatives of by-pixel operation. According to the structure of the template, a variety of first-order edge detection operators can be constructed.
The commonly used first-order operators are as follows:
Robert operator is also called cross differential algorithm. It is a gradient algorithm based on cross difference. It detects edge lines through local difference calculation. It is commonly used to process steep and low-noise images. When the edge of the image is close to plus or minus 45 degrees, the processing effect of this algorithm is more ideal. The disadvantage is that the edges of the extracted image are rough.
The template of Robert operator is divided into horizontal direction and vertical direction, as shown in Eq. (1). It can be seen from the template that the Robert operator can better enhance the edge of the image at plus or minus 45 degrees. Where represents the gradient in the direction and represents the gradient in the direction:
Prewitt operator is a differential operator for image edge detection. Its principle is to use the difference generated by pixel gray value in a specific area to realize edge detection. Because the Prewitt operator uses the 3×3 template to calculate the pixel values in the area, and the Robert operator uses the 2×2 template. Therefore, the edge detection result of Prewitt operator is more obvious in both horizontal and vertical directions than Robert operator. The Prewitt operator is suitable for identifying images with a lot of noise and gradual gray scale. The calculation is shown in Eq. (2): where represents the gradient in the direction and represents the gradient in the direction:
Sobel operator is a discrete differential operator for edge detection, which combines Gaussian smoothing and differential derivation. This operator is used to calculate the approximate value of the brightness of the image. According to the brightness of the edge of the image, the specific points in the area exceeding a certain number are recorded as edges. The Sobel operator adds the concept of weight on the basis of the Prewitt operator. It is believed that the distance between adjacent points has different effects on the current pixel. The closer the pixel point corresponds to the greater the impact of the current pixel, so as to achieve image sharpening and highlight the edge contour.
The Sobel operator detects the edge by reaching the extreme value at the edge according to the weighted difference of the gray scale of the upper and lower pixels and the left and right adjacent points. It has a smoothing effect on noise and provides more accurate edge direction information. Because the Sobel operator combines Gaussian smoothing and differential derivation, the result will have more noise resistance. When the accuracy requirements are not very high, Sobel operator is a more commonly used edge detection method.
Sobel edge operator more accurate positioning sub, commonly used in much noise, shades of gray image. The algorithm template is shown in Eq. (3), where represents the horizontal direction and represents the vertical direction:
(2) Second-order edge detection operator.
The second-order edge detection operator uses the feature that the second-order derivative at the edge of the object in the image is zero for edge detection. For the edge of the object in the image, the pixel value at the edge will undergo a step change, and the signs of the second derivative on both sides of the edge point are opposite.
Commonly used second-order operators are as follows:
The Canny algorithm is the best solution between suppressing noise and accurately retaining edge features [12, 13]. Canny edge detection is actually a first-order differential operator detection algorithm, which uses the step property of the image at the edge, that is, the characteristic that the image gradient obtains a maximum value at the edge for edge detection. Gradient is a vector, it has direction and modulus:
The direction of the gradient provides edge trend information, because the gradient direction is always perpendicular to the edge direction, and the magnitude of the gradient modulus provides edge strength information. In actual use, finite difference is usually used for gradient approximation. For the above formula, there are the following approximations:
2.2. Comparative analysis of four edge detection operators
Using MATLAB R2019b on the Windows 10 operating system with a CPU of 1.60 GHz and RAM of 8 GB, four operator edge detection methods are used to perform edge detection processing on the same lane image, as shown in Fig. 1. It can be clearly seen from the figure that the edges of the lane markings and the protruding edges are outlined by white contour lines, and the non-contoured parts are presented in black. In the first-order operator edge detection, the Prewitt operator detects that the image has more white points outside the white contour line, and the detection accuracy is poor. In the images detected by the Sobel operator and the Robert operator, the white points are similar, so in the first-order operator detection edge detection, the Sobel operator and the Robert operator have relatively high edge detection accuracy. In the second-order edge detection, in the Canny operator edge detection method, the image has no messy white spots, and the image to be detected has a clear outline, which is much better than the first-order edge detection method. Therefore, among the edge detection results of the four edge detection operators, the Canny operator edge detection result is more accurate.
The lane edges are detected by 4 methods. The 4 detection methods are run 100 times respectively, and the average value is taken. After statistics, the average value of the detection time of the edge detection operator is obtained, as shown in Table 1.
Table 1Average detection time of edge detection operator
First-order edge detection
Second order edge detection
Operation time / s
According to the data in the table, among the four edge detection operators, the Robert operator edge detection takes the least time, only 0.037972 s, while the traditional Canny operator edge detection method takes the longest, and the detection process uses 0.047919 s. By comparison, in terms of detection speed, Robert operator is 0.009947 s faster than Canny operator. Robert operator edge detection takes the least time in lane edge detection, and is more suitable for rapid detection of lane lines.
Through the comparative and comprehensive analysis of the four edge detection operators in image detection and detection time, Canny operator is better than Robert operator in image detection sharpness, but the white dots appearing on Robert operator are for the lane The detection of the line has less impact and does not affect the detection of the final lane line. In the comparative analysis of the time used for edge detection, it can be seen that the Robert operator takes a shorter time in image detection than the Canny operator, which only requires 0.037972 s. Based on the pursuit of detection timeliness, this paper chooses Robert operator as the edge detection method.
Fig. 1Image edge detection and processing
a) Robert algorithm
b) Sobel algorithm
c) Prewitt algorithm
d) Canny algorithm
3. Lane line detection
3.1. Lane line detection process
This article is based on the Hough transform lane line extraction. The specific detection process is shown in Fig. 2.
Fig. 2Lane detection process
1) Original image acquisition.
During the driving of the car, the camera converts the captured optical image into a digital signal, which is sent to the image capture card for digital processing. This article uses the Sony HDR-XR520E monocular high-definition camera to collect the original lane line map. Install the camera in the middle of the front windshield and adjust the shooting angle of the lens until the lane line is in the middle of the image. The original image collected is shown in Fig. 3(a).
2) Image gray-scale processing.
The images collected by the vision sensor are mostly color images. Since the three-channel model of color image contains a lot of information that has nothing to do with the detection target, it is time-consuming to process and the real-time performance will be reduced. Therefore, this article converts the collected road color image into a grayscale image, which is equivalent to subtracting two-thirds of the original data, and the lane line description information is retained in the grayscale image, which can meet the needs of lane line detection. Most color images now use the RGB color mode. When processing the image, the three components of R (red), G (green), and B (blue) must be processed separately. Herein gradation processing using the image component method:
After obtaining the original image, in order to facilitate the identification, the image gradation processing, in MATLAB, a gradation processing function pic_gray = rgb2gray (ori), after gradation processing shown in Fig. 3(b). After the image is grayscaled, the color part of the image is replaced by gray and white, which makes the outline of the image clear and easy to recognize.
Fig. 3Preprocessing of image edge detection
3) Image filtering.
Image filtering, that is, suppressing the noise of the target image while preserving the details of the image as much as possible, is an indispensable operation in image preprocessing, and its processing effect directly affects the effectiveness and reliability of subsequent image analysis. Median filtering is based on the value of the central point position of this sequence is replaced with the median value of the point field. Its advantages are simple operation, fast speed, and good denoising effect.
Therefore, this paper uses the median filter method to filter the image. In MATLAB, the filter processing function is filter1_pic=medlt2(pic_grafiy), and the image after filtering is shown in Fig. 3(c). After filtering the image, the noise of the image is reduced, the contrast of the image is enhanced, and the image is easily recognized.
4) Image binarization.
Image binarization is the process of setting the gray value of the pixels on the image to 0 or 255, that is, the entire image presents an obvious black and white effect. In digital image processing, binary image occupies a very important position, binary image of the image in order to reduce the amount of data, which can highlight the contour of an object.
In MATLAB, the image binarization function is bw_pic=im2bw(filter2 pic). The binarized image is shown in Fig. 3(d). The outline of the road line of the image is clearer and becomes white, while the non-road markings are made black.
5) Image boundary detection.
This paper uses Robert operator and Canny operator to detect the edge of the lane line respectively. In MATLAB, edge(bw pic,'Robert') means using Robert operator to extract edge information, as shown in Fig. 4(a). edge(bw pic,'Canny') means using Canny operator to extract edge information, as shown in Fig. 4(b). It can be seen from the figure that the outline of the lane line becomes clearer, the overall lane line outline is more distinct, the lane edge image has a white outline wrapped, and the other non-lane boundary lines are black areas. There is little difference between the detection of the two operators.
Fig. 4Image edge detection
6) Hoff transform.
Hough transform is a feature extraction, which is widely used in image analysis, computer vision and digital image processing. Hough transform is used to identify features in objects. The algorithm flow is roughly as follows. Through the rough outline of the image and the recognition of the graphics, the algorithm will perform voting in the parameter space to determine the shape of the object, and this is determined by the local maximum in the accumulation space.
In MATLAB, the function for Hough transform processing is [H,T,R]=hough(verge_pic), H is the X-axis data, T is the Y-axis data, and R is the initial magnification value of the image. The image through the Hough transform is shown in Fig. 5(a) and 5(b) are the Hough transform after edge detection of Roberts operator and Canny operator. It can be seen from the figure that only the coordinate value of the right point of the two pictures differs by 1, and the other values are all Unanimous.
7) Drawing and marking of lane lines.
After the Hough transform, the lane lines are drawn on the image. In MATLAB, use the function n=polyval(p,t) to draw the lane line as shown in Fig. 6(a) and 6(b) are the images after the integration of Roberts operator and Canny operator, respectively. It can be seen from the images that both images can clearly show the extracted lane lines. Based on the lane line drawing, the detected lane line is then marked on the original image, as shown in Fig. 7. The green detection line is accurately marked on the original lane line.
Fig. 5Hough transform
Fig. 6Lane line drawing
Fig. 7Lane line labeling
After comparison, it is found that the image detected by the Robert operator and the detected image processed by the Canny operator have little difference between the Hough transform, lane line extraction and lane line labeling, and both can clearly show the lane. The position of the line. Therefore, the integration of the Robert operator is the same as the traditional Canny operator, which can realize the detection of lane markings.
4. Simulation analysis
4.1. Comparison of the similarity of the test images
Based on gray histogram matching, the color image is divided into three channels (RGB), and the respective histograms are calculated, and the histogram matching is performed on the RGB three channels of the two images respectively, and finally the three matching results are averaged. The applied algorithm is as follows:
Import the two pictures a) and b) in Fig. 7 into the program in MATLAB, and the comparison results are shown in Fig. 8.
Fig. 8Comparison of similarity of test results
After comparison, it is found that the similarity of the lane line detection image after the edge processing of the Robert operator and the Canny operator is 0.99576. The results show that the Robert operator edge detection method is integrated into the Hough transform lane line detection method, and it has little effect on the final image clarity of the lane line detection.
4.2. Time comparison of test results
Using MATLAB R2019b on the Windows 10 operating system with a CPU of 1.60 GHz and RAM of 8GB, the Robert operator edge detection and Canny operator edge detection methods are integrated into the Hough transform lane line detection method to detect the lane lines. Detection 100 run with the results shown in Table 2.
From the data in the table, we can see that the time used for each test fluctuates compared with the last time. The time used for 100 test runs is averaged, and the processing is shown in Table 3.
Table 2Time used for 100 tests
Number of times
Robert operator running time/s
Canny operator running time/s
Table 3Average detection time
Operation time / s
Through the comparison of the data in the table, it can be found that after the Robert operator and Canny operator are integrated into the Hough transform to detect the lane line, the Robert operator has a greater impact on the detection of the entire lane line than the Canny operator, and the entire detection time is reduced from 0.705261 s to 0.553351 s, the entire lane detection time is shortened by 0.15191 s. The integration of the Robert operator into the Hough transform lane line greatly shortens the time of lane line detection, thereby improving the timeliness of lane line detection.
In order to pursue the timeliness of lane detection, this paper replaces the Canny operator edge detection method with the Robert operator edge detection method in the traditional Hough transform lane line detection method, and simulates in MATLAB. It is found that the Robert operator not only shortens the image edge detection time compared with the Canny operator when detecting the image edge alone, but also shortens the entire lane detection time.
Due to the shortening of the lane line detection time, the lane line can be quickly detected during the driving process of the vehicle, which improves the operating efficiency of lane line recognition and increases the safety of vehicle driving.
Robert operator still has shortcomings in edge detection. Although the similarity in the final image detection accuracy is as high as 0.99576, in the edge detection accuracy, the Robert operator is not as accurate as the Canny operator, and the result of edge extraction is relatively coarse. Therefore, this paper has used the Robert operator to extract lane lines to improve the detection speed, and the detection accuracy should be further studied. The ultimate goal is to achieve simultaneous improvement of lane line detection accuracy and detection speed.
B. Lin., “A brief analysis on the promotion of ADAS and V2X technology for intelligent connected vehicle,” (in Chinese), in The 12th China Intelligent Transportation Annual Conference, pp. 552–560, 2017.
M. Mittal et al., “An efficient edge detection approach to provide better edge connectivity for image analysis,” IEEE Access, Vol. 7, pp. 33240–33255, 2019, https://doi.org/10.1109/access.2019.2902579
Z. Fang and M. Li, “Research on the application of edge detection operator with direction in road edge extraction,” (in Chinese), Surveying and Mapping, Vol. 34, No. 4, pp. 155–158, 2011.
J. Kong, J. Hou, T. Liu, and M. Jiang, “Adaptive image edge detection model using improved Canny algorithm,” in 2018 IEEE 9th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON), pp. 539–545, Nov. 2018, https://doi.org/10.1109/iemcon.2018.8615028
S. Qingfeng., “An edge detection algorithm based on wavelet transform,” (in Chinese), Journal of Hebei North University (Natural Science Edition), Vol. 37, No. 3, pp. 10–14, 2021.
D. Xu, X. Wang, G. Sun, and H. Li, “Towards a novel image denoising method with edge-preserving sparse representation based on Laplacian of B-spline edge-detection,” Multimedia Tools and Applications, Vol. 76, No. 17, pp. 17839–17854, Sep. 2017, https://doi.org/10.1007/s11042-015-3097-0
C. Zhu’An, H. Zhifeng, and Y. Xipeng, “Remote sensing image edge detection based on improved Soble operator and Zernike moment,” (in Chinese), Journal of Hubei University for Nationalities (Natural Science Edition), Vol. 35, No. 2, pp. 229–234, 2017.
L. Han and A. Han, “An improved edge detection algorithm based on morphological operators and gradient,” Journal of Computational and Theoretical Nanoscience, Vol. 12, No. 7, pp. 1121–1125, Jul. 2015, https://doi.org/10.1166/jctn.2015.3860
Y.-B. He, Y.-J. Zeng, H.-X. Chen, S.-X. Xiao, Y.-W. Wang, and S.-Y. Huang, “Research on improved edge extraction algorithm of rectangular piece,” International Journal of Modern Physics C, Vol. 29, No. 1, p. 1850007, Jan. 2018, https://doi.org/10.1142/s0129183118500079
Y. Donghua, L. Jiuxian, and B. Zhiguo., “Research on MARR edge detection algorithm,” (in Chinese), Journal of Image and Graphics, Vol. 11, No. 6, pp. 823–826, 2006.
T. Yuan, H. Huixian, X. Jianmin, and C. Ren., “Remote sensing image road edge detection method based on improved Sobel operator,” (in Chinese), Remote Sensing of Land and Resources, Vol. 28, No. 3, pp. 7–11, 2016.
L. Xiang, L. Tao, “Classification algorithm of kernel based in Adaboost,” (in Chinese), Computer Knowledge and Technology, Vol. 7, No. 28, pp. 6970–6979, 2011.
R. Agrawal and R. Srikant, “Fast algorithms for mining association rules,” in VLDB '94: Proceedings of the 20th International Conference on Very Large Data Bases, pp. 487–499, 1994.