徐光柱，刘高飞，匡婉，等. 基于顶点与主体区域同步检测的精准车牌定位[J]. 北京航空航天大学学报，2024，50（2）：376-387 doi: 10.13700/j.bh.1001-5965.2022.0396 引用本文: 徐光柱，刘高飞，匡婉，等. 基于顶点与主体区域同步检测的精准车牌定位[J]. 北京航空航天大学学报，2024，50（2）：376-387 doi: 10.13700/j.bh.1001-5965.2022.0396 XU G Z，LIU G F，KUANG W，et al. Accurate license plate location based on synchronous vertex and body region detection[J]. Journal of Beijing University of Aeronautics and Astronautics，2024，50（2）：376-387 （in Chinese） doi: 10.13700/j.bh.1001-5965.2022.0396 Citation: XU G Z，LIU G F，KUANG W，et al. Accurate license plate location based on synchronous vertex and body region detection[J]. Journal of Beijing University of Aeronautics and Astronautics，2024，50（2）：376-387 （in Chinese） doi: 10.13700/j.bh.1001-5965.2022.0396 徐光柱，刘高飞，匡婉，等. 基于顶点与主体区域同步检测的精准车牌定位[J]. 北京航空航天大学学报，2024，50（2）：376-387 doi: 10.13700/j.bh.1001-5965.2022.0396 引用本文: 徐光柱，刘高飞，匡婉，等. 基于顶点与主体区域同步检测的精准车牌定位[J]. 北京航空航天大学学报，2024，50（2）：376-387 doi: 10.13700/j.bh.1001-5965.2022.0396 XU G Z，LIU G F，KUANG W，et al. Accurate license plate location based on synchronous vertex and body region detection[J]. Journal of Beijing University of Aeronautics and Astronautics，2024，50（2）：376-387 （in Chinese） doi: 10.13700/j.bh.1001-5965.2022.0396 Citation: XU G Z，LIU G F，KUANG W，et al. Accurate license plate location based on synchronous vertex and body region detection[J]. Journal of Beijing University of Aeronautics and Astronautics，2024，50（2）：376-387 （in Chinese） doi: 10.13700/j.bh.1001-5965.2022.0396 More Information

Corresponding author: E-mail： [email protected]

为应对非约束环境下的车牌精定位问题，提出一种基于顶点局部区域与主体区域同步检测策略的非约束性车牌定位算法。通过删减YOLOv5网络的输出结构，训练得到可同步检测车牌及顶点区域的车牌检测网络，在兼顾精度与计算速度的前提下，实现车牌顶点和主体区域的同步定位。针对一幅图中存在多个车牌区域及顶点区域存在少量漏检和误检的情况，分别设计了车牌顶点归类和单一缺失顶点预测后处理算法，借助顶点间的空间位置关系进行漏检目标预测和误检目标排查，有效改善了因场景复杂导致的个别顶点目标检测效果差的问题。所提算法在中国城市停车场数据集(CCPD)上的测试结果显示，平均精准率达99.25%，平均召回率达98.70%。所提算法不仅能够准确预测出车牌的4个顶点坐标，而且在中端GPU硬件平台上处理速度可达121帧/s，具有较好的应用价值。

深度学习 /  卷积网络 /  视觉目标检测 /  非约束车牌定位 /  车牌顶点检测 Abstract:

A novel unconstrained license plate accurate location algorithm is designed by simultaneously detecting the four local vertex regions and the body of a license plate and fusing the results to address the issue that the widely used rectangular bounding boxes in mainstream target detection methods cannot meet the license plate location accuracy requirement in many unconstrained environments where the license plate images are not commonly rectangle. At first, the four local rectangular sub-regions with centers on four vertices of a license plate were annotated as vertex-region objects according to the size of the plate’s contour-rectangle and the vertex coordinates. Then, a multi-class image dataset is built up in which the contour-rectangle region covering the whole license plate body is a class and the four kinds of vertex-region construct other four classes. In order to locate these five object classes efficiently, the output structure of the YOLOv5 network is modified by taking accuracy and efficiency into consideration and trained with the newly constructed multi-class dataset.Finally, vertex region grouping and single missing vertex forecasting are carried out as the post-processing to address the issue that there are multiple candidate license plates in an image and a few vertices region false or missing detection errors will happen in some unique instances.By exploiting the relationship among the vertexes, the post-processing can effectively recognize missing and false detection errors in some special complex scenarios and improve the whole system’s performance greatly. The proposed algorithm is evaluated on the Chinese city parking dataset (CCPD), and reaches an average positioning accuracy of 99.25% and an average recall rate of 98.70%. The performance certificates our method not only can accurately predict the coordinates of the four vertices but also can run at 121 frame/s on a moderate GPU hardware platform, which has great application potential.

Key words: deep learning /  convolutional network /  visual target detection /  unconstrained license plate location /  license plate vertex detection  曾丽华, 李超, 熊璋. 基于边缘与颜色信息的车牌精确定位算法[J]. 北京航空航天大学学报, 2007, 33(9): 1112-1116. doi: 10.3969/j.issn.1001-5965.2007.09.026

ZENG L H, LI C, XIONG Z. Precisely license plate locating algorithm based on edge and color information[J]. Journal of Beijing University of Aeronautics and Astronautics, 2007, 33(9): 1112-1116(in Chinese). doi: 10.3969/j.issn.1001-5965.2007.09.026 CHEN J I Z, ZONG J I. Automatic vehicle license plate detection using K-means clustering algorithm and CNN[J]. Journal of Electrical Engineering and Automation, 2021, 3(1): 15-23. doi: 10.36548/jeea.2021.1.002 XU Z B, YANG W, MENG A J, et al. Towards end-to-end license plate detection and recognition: A large dataset and baseline[C]//Proceedings of the European Conference on Computer Vision. Berlin: Springer, 2018: 255-271. SILVA S M, JUNG C R. License plate detection and recognition in unconstrained scenarios[C]//Proceedings of the European Conference on Computer Vision. Berlin: Springer, 2018: 580-596. SELMI Z, HALIMA M B, PAL U, et al. DELP-DAR system for license plate detection and recognition[J]. Pattern Recognition Letters, 2019, 129: 213-223. HE K M, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(2): 386-397. doi: 10.1109/TPAMI.2018.2844175 SZEGEDY C, LIU W, JIA Y Q, et al. Going deeper with convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2015: 1-9. CHEN L S, TIAN S, MA W J, et al. End-to-end trainable network for degraded license plate detection via vehicle-plate relation mining[J]. Neurocomputing, 2021, 446: 1-10. doi: 10.1016/j.neucom.2021.03.040 WANG W, YANG J, CHEN M, et al. A light CNN for end-to-end car license plates detection and recognition[J]. IEEE Access, 2019, 7: 173875-173883. doi: 10.1109/ACCESS.2019.2956357 YIN X, LIU X. Multi-task convolutional neural network for pose-invariant face recognition[J]. IEEE Transactions on Image Processing, 2017, 27(2): 964-975. WANG Y, BIAN Z P, ZHOU Y, et al. Rethinking and designing a high-performing automatic license Plate recognition approach[EB/OL]. (2021-06-17) [2022-05-20]. https://arxiv.org/abs/2011.14936. LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2017: 2117-2125. Ultralytics. YOLOv5[EB/OL]. (2022-05-12) [2022-05-20]. https://github.com/ultralytics/yolov5. REDMON J, FARHADI A. YOLOv3: An incremental improvement[EB/OL]. (2018-04-08) [2022-05-20]. https://arxiv.org/abs/1804.02767. ALEXEY B, WANG C Y, LIAO H. YOLOv4: Optimal speed and accuracy of object detection[EB/OL]. (2020-04-23) [2022-05-20]. https://arxiv.org/abs/2004.10934v1. WANG C Y, LIAO H, YEH I H, et al. CSPNet: A new backbone that can enhance learning capability of CNN[EB/OL]. (2019-11-27) [2022-05-20].https://arxiv.org/abs/1911.11929. HE K M, ZHANG X, REN S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1904-1916. LIU S, QI L, QIN H, et al. Path aggregation network for instance segmentation[EB/OL]. (2018-03-05) [2022-05-20]. https://arxiv.org/abs/1803.01534. LOSHCHILOV I, HUTTER F. SGDR: Stochastic gradient descent with warm restarts[EB/OL]. (2016-08-13) [2022-05-20]. https://arxiv.org/abs/1608.03983. 徐光柱, 匡婉, 李兴维, 等. YOLOv3与顶点偏移估计相结合的车牌定位[J]. 计算机辅助设计与图形学学报, 2021, 33(4): 569-579.

XU G Z, KUANG W, LI X W, et al. License plate location based on YOLOv3 and vertex offset estimation[J]. Journal of Computer-Aided Design & Computer Graphics, 2021, 33(4): 569-579(in Chinese). 唐倩, 贺伟, 张林江. 基于空间转换网络的端到端车牌检测与识别[J]. 光电子·激光, 2021, 32(5): 524-531. doi: 10.16136/j.joel.2021.05.0423

TANG Q, HE W, ZHANG L J. End to end license plate detection and recognition based on spatial transformer network[J]. Journal of Optoelectronics·Laser, 2021, 32(5): 524-531(in Chinese). doi: 10.16136/j.joel.2021.05.0423 QIN S S, LIU S J. Towards end-to-end car license plate location and recognition in unconstrained scenarios[EB/OL]. (2020-08-25) [2022-05-20]. https://arxiv.org/abs/2008.10916. SILVA S M, JUNG C R. A flexible approach for automatic license plate recognition in unconstrained scenarios[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 23(6): 5693-5703. REDMON J, FARHADI A. YOLO9000: Better, faster, stronger[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2017: 7263-7271. ZOU Y, ZHANG Y, YAN J, et al. License plate detection and recognition based on YOLOv3 and ILPRNET[J]. Signal Image and Video Processing, 2021, 9: 312-320. LI B, TIAN B, LI Y, et al. Component-based license plate detection using conditional random field model[J]. IEEE Transactions on Intelligent Transportation Systems, 2013, 14(4): 1690-1699. YUAN Y, ZOU W, ZHAO Y, et al. A robust and efficient approach to license plate detection[J]. IEEE Transactions on Image Processing, 2016, 26(3): 1102-1114.