地球信息科学学报 ›› 2017, Vol. 19 ›› Issue (2): 256-263.doi: 10.3724/SP.J.1047.2017.00256

• 遥感科学与应用技术 • 上一篇    下一篇

基于RCNN的无人机巡检图像电力小部件识别研究

王万国1,2(), 田兵3, 刘越1,2, 刘俍1,2, 李建祥1,2   

  1. 1. 国网山东省电力公司电力科学研究院 国家电网公司电力机器人技术实验室,济南 250002
    2. 山东鲁能智能技术有限公司,济南 250101
    3. 国网山东省电力公司,济南 250000
  • 收稿日期:2016-03-17 修回日期:2016-06-21 出版日期:2017-02-28 发布日期:2017-02-17
  • 作者简介:

    作者简介:王万国(1984-),男,硕士,中级工程师,主要从事基于数字图像的变电站和输电线路设备识别、目标跟踪、图像去雾等研究。E-mail: wangwanguo03@qq.com

  • 基金资助:
    2014年国家电网公司发展项目“无人机巡检实用化关键技术及检测体系研究”

Study on the Electrical Devices Detection in UAV Images based on RegionBased Convolutional Neural Networks

WANG Wanguo1,2,*(), TIAN Bing3, LIU Yue1,2, LIU Liang1,2, LI Jianxiang1,2   

  1. 1. Electric Power Robotics Laboratory of SGCC, Shandong Electric Power Research Institute, Jinan, 250002, China
    2. Shandong Luneng Intelligence Technology Co., Ltd. Jinan, 250101, China
    3. State Grid Shandong Electric Power Company, Jinan, 250000, China
  • Received:2016-03-17 Revised:2016-06-21 Online:2017-02-28 Published:2017-02-17
  • Contact: WANG Wanguo E-mail:wangwanguo03@qq.com

摘要:

随着无人机(UAV)在电力巡线作业中的应用推广,对无人机巡检图像的信息挖掘或目标识别需求也越来越强烈。传统的电力部件识别流程常使用经典的机器学习算法,如支持向量机(SVM)、随机森林或adaboost,结合梯度、颜色或纹理等浅层特征来对电力部件进行识别,难以充分利用无人机巡检图像的信息,并且难以达到较高的准确率。卷积神经网络(CNN)在目标识别中表现优异,在很多目标识别场景之中成为首选算法。基于区域的卷积神经网络(RCNN)通过使用CNN从图像中提取可能含有目标的区域来检测并识别目标,但是计算复杂,难以满足识别海量电力巡检图片的需求。Fast R-CNN和Faster R-CNN利用CNN网络提取图像特征,后接一个区域提议层,优化了提取可能含有目标区域的方式并改进识别目标的分类器,使目标的检测和识别几乎实时。本文详细描述了Faster R-CNN算法流程,并在无人机电力线巡检图像部件检测中使用,然后分别对DPM、SPPnet和Faster R-CNN识别方法进行了对比分析,利用实际采集的电力小部件巡检数据构建的数据集对3种方法进行测试验证,并讨论了不同参数对识别结果的影响。实验结果表明,基于深度学习的识别方法实现电力小部件的识别是可行的,而且利用Faster R-CNN进行多种类别的电力小部件识别定位可以达到每张近80 ms的识别速度和92.7%的准确率。

关键词: 深度学习, RCNN, 卷积神经网络, 无人机巡检图像, 电力部件识别

Abstract:

With the wide application of Unmanned Aerial Vehicle (UAV) in the inspection of power transmission line, the demand for objects detection and data mining from images acquired by UAV also grows significantly. Traditional detecting methods use some classical machine learning algorithms, such as support vector machine (SVM), random forest or adaboost etc. and combine the low level features such as gradient, colors or texture to detect electrical devices. These image features must be carefully designed and changed a lot from various object kinds. Thus, they are not suitable for UAV images with complex background and multiple kinds of object. On the other hand, the disadvantages of these methods are that they cannot take advantage of the high quantity and large coverage of UAV acquired images, and cannot get a satisfactory accuracy. The recent developing Deep Learning method brings light to this problem. Convolutional neural network (CNN) performs excellently in object recognition area and outstand many other methods used in the past. Without the need of extracting images’ features, CNN becomes the many state-of-the-art methods in object recognition rapidly. In object detection, Region-based convolutional neural networks (RCNN) retrieves the region that may contain the object from the images to detect and recognize the object. However, the computation is so expensive that it cannot meet the requirement of detecting massive UAV’s images and cannot be used in practical projects. Fast R-CNN and Faster R-CNN solve this problem by changing the way of object retrieval. They use features produced by CNN network layers and apply a region proposal network layer behind to locate the object. After that, fully connected layers and softmax layer follow to classify the features corresponding to object into special kinds. Using this strategy, Fast R-CNN and Faster R-CNN save lots of time to produce region proposal and can perform object detection at nearly real time. The principle and processes of Faster R-CNN and several other object detection methods are described in this paper, and they are tested for electrical devices detection from images of the power transmission line obtained by UAV. We analyzed the influence of several key parameters to the device detection results, such as the dropout ratio, non-maximum suppression (nms) and batch size. Then, we gave some constructive advice of tun ing parameters in Faster R-CNN. We also analyzed the advantages and weakness of three advanced detection algorithms, including Deformable Part Models (DPM) and two deep learning-based methods named Spatial pyramid pooling networks (SPPNet) and Faster R-CNN. Finally, we constructed image datasets of power transmission line inspection obtained by UAV and tested the three methods above. The recall ratio and accuracy ratio of them are compared and the superiority of the Faster R-CNN is validated. Testing results showed that Faster R-CNN method can detect various electrical devices of different categories in one image simultaneously within 80 milliseconds and achieve an accuracy of 92.7% on a standard test set, which is of great significance in real-time power transmission line inspection. These results also showed the advantages of the Faster R-CNN and we apply Faster R-CNN in our practical projects to detect electrical devices.

Key words: deep learning, RCNN, Region-based convolutional neural networks, UAV images, electrical devices detection