顾及视频地理映射的人群密度估计方法

doi:10.12082/dqxxkx.2022.210555

摘要/Abstract

摘要：

针对复杂场景下人群计数及地图可视化问题,提出了一种顾及视频地理映射的人群密度估计方法。首先,通过迁移学习的方式,构建了适合于复杂场景的人群语义分割模型;将视频与GIS相结合,求解摄像机和人群场景的单应矩阵;基于人群语义分割结果及单应矩阵,可将人群多边形映射至二维地图。然后,设计等距分区和格网分区2种分区方案;根据人群语义分割结果,统计不同分区方案下各子区人群密度。最后,基于训练的各子区人群密度值,计算视域内人口总数;通过人群目标点符号均匀填充人群多边形,进行地图可视化。实验结果表明：① 本文构建的人群语义分割模型,可实现大场景下人群的高精度分割,精度为94.11%;② 将视频与GIS相结合,实现了人群的地图映射和地图可视化表达,达到了人群可定位、可量测和可空间分析的目标;③ 实现了监控视频人群的精准计数,将视域划分更多子区有利于提高计数精度。与基于密度图的人群密度估计方法相比,本文方法在高空、高密集度的大场景中具有优势,能有效解决人群难以精准计数及地图可视化的问题,可用于大型活动、车站、商场、运动场馆的人群监管。

关键词: 地理视频, 视频GIS, 人群, 语义分割, 单应矩阵, 地图可视化, 人群密度, 人群计数

Abstract:

Aiming at the problem that the existing crowd counting methods cannot achieve accurate counting and map visualization of complex crowds, a crowd density estimation method considering video geographic mapping is proposed. Firstly, based on Deeplab V3+model, a crowd semantic segmentation model suitable for complex scenarios is constructed by transfer learning. Combining video with GIS, the high-precision homography matrix between video and crowd scene map is calculated according to four or more pixel coordinates between video frame and the corresponding geographic coordinates. Based on the crowd semantic segmentation model and the solved homography matrix, the crowd areas in videos are projected to the map. Secondly, to improve the accuracy of crowd number, two different partition schemes: equidistant and grid partition, are designed to divide the camera Field of View (FOV). According to the semantic segmentation result, the crowd density of each sub-region using different partition schemes is counted. Based on the crowd density and area of each sub-region, the total number in the camera FOV is calculated. Thirdly, based on the solved homography matrix, the semantic segmentation result of the crowd in the real-time video can be projected to the 2D map and the crowd number can be counted through the crowd density. In order to obtain accurate crowd density, we took a playground as the experimental area and collected multiple crowd surveillance videos at different times and under different crowd conditions. The experimental results show that: (1) the crowd semantic segmentation model constructed in this paper can achieve high-precision crowd segmentation in large scenes, with an accuracy of 94.11%; (2) Combining video with GIS, the polygon area of the crowd was filled through the point symbol of person style, the crowd mapping and visual expression were realized, and the goal of crowd localization, measurement, and spatial analysis was achieved; (3) Accurate counting of surveillance video crowd was realized, and the camera FOV was divided into many sub-areas, which is conducive to improving the crowd counting accuracy. Compared to the crowd density estimation method based on density map, the method proposed in this paper is suitable for large scenes with high altitude and high density, especially in the areas where the texture of people's head isn't clear and crowd characteristics are obscured. Our method can effectively improve the accuracy of crowd counting and map visualization and can be used for crowd supervision in large-scale events, stations, shopping malls, and sports venues.

Key words: geographic video, video GIS, crowd, semantic segmentation, homography matrix, map visualization, crowd density, crowd counting

孙银萍, 张兴国, 石新雨, 李奇泽. 顾及视频地理映射的人群密度估计方法[J]. 地球信息科学学报, 2022, 24(6): 1130-1138.DOI:10.12082/dqxxkx.2022.210555

SUN Yinping, ZHANG Xingguo, SHI Xinyu, LI Qize. Crowd Density Estimation Method Considering Video Geographic Mapping[J]. Journal of Geo-information Science, 2022, 24(6): 1130-1138.DOI:10.12082/dqxxkx.2022.210555

图/表 12

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

图11

图12

参考文献 21

[1]	方志祥. 人群动态的观测理论及其未来发展思考[J]. 地球信息科学学报, 2021, 23(9):1527-1536. doi: 10.12082/dqxxkx.2021.200787
	[ Fang Z X The theory prospect of crowd dynamics-oriented observation[J]. Journal of Geo-information Science, 2021, 23(9):1527-1536. ] DOI: 10.12082/dqxxkx.2021.200787 doi: 10.12082/dqxxkx.2021.200787
[2]	张兴国. 地理场景协同的多摄像机目标跟踪研究[D]. 南京: 南京师范大学, 2014.
	[ Zhang X G. Research on multi-camera target tracking with geographic scene collaboration[D]. Nanjing: Nanjing Normal University, 2014. ]
[3]	Xie Y J, Wang M Z, Liu X J, et al. Integration of GIS and moving objects in surveillance video[J]. ISPRS International Journal of Geo-information, 2017, 6(4):94. DOI: 10.3390/ijgi6040094 doi: 10.3390/ijgi6040094
[4]	张兴国, 刘学军, 王思宁,等. 监控视频与2D地理空间数据互映射[J]. 武汉大学学报·信息科学版, 2015, 40(8):1130-1136.
	[ Zhang X G, Liu X J, Wang S N, et al. Mutual mapping between surveillance video and 2D geospatial data[J]. Geomatics and Information Science of Wuhan University, 2015, 40(8):1130-1136. ] DOI: 10.13203/j.whugis20130817 doi: 10.13203/j.whugis20130817
[5]	王美珍, 刘学军, 孙开新,等. 最优视频子集与视频时空检索[J]. 计算机学报, 2019, 42(9):2004-2023.
	[ Wang M Z, Liu X J, Sun K X, et al. Optimum video subset and spatial-temporal video retrieval[J]. Chinese Journal of Computers, 2019, 42(9):2004-2023. ] DOI: 10.11897/SP.J.1016.2019.02004 doi: 10.11897/SP.J.1016.2019.02004
[6]	宋宏权, 刘学军, 闾国年,等. 区域人群状态的实时感知监控[J]. 地球信息科学学报, 2012, 14(6):686-692,697. doi: 10.3724/SP.J.1047.2012.00686
	[ Song H Q, Liu X J, Lv G N, et al. Real-time monitoring for the regional crowds status[J]. Journal of Geo-information Science, 2012, 14(6):686-692, 697. ] DOI: 10.3724/SP.J.1047.2012.00686 doi: 10.3724/SP.J.1047.2012.00686
[7]	蒋妮, 周海洋, 余飞鸿. 基于计算机视觉的目标计数方法综述[J]. 激光与光电子学进展, 2021, 58(14):43-59.
	[ Jiang N, Zhou H Y, Yu F H. A survey of computer vision based object counting methods[J]. Laser & Optoelectronics Progress, 2021, 58(14):43-59. ] DOI: 10.3788/LOP202158.1400002 doi: 10.3788/LOP202158.1400002
[8]	余鹰, 朱慧琳, 钱进,等. 基于深度学习的人群计数研究综述[J]. 计算机研究与发展, 2021, 58(12):2724-2747.
	[ Yu Y, Zhu H L, Qian J, et al. Survey on deep learning based crowd counting[J]. Journal of Computer Research and Development, 2021, 58(12):2724-2747. ] DOI: 10.7544/issn1000-1239.2021.20200699 doi: 10.7544/issn1000-1239.2021.20200699
[9]	屈晶晶, 辛云宏. 连续帧间差分与背景差分相融合的运动目标检测方法[J]. 光子学报, 2014, 43(7):219-226.
	[ Qu J J Xin Y H. Combined continuous frame difference with background difference method for moving object detection[J]. Acta Photonica Sinica, 2014, 43(7):219-226. ] DOI: 10.3788/gzxb20144307.0710002 doi: 10.3788/gzxb20144307.0710002
[10]	Chan A B, Vasconcelos N. Bayesian Poisson regression for crowd counting[C]. Proceedings of the IEEE International Conference on Computer Vision, 2009:545-551. DOI: 10.1109/ICCV.2009.5459191 doi: 10.1109/ICCV.2009.5459191
[11]	Gao J Y, Lin W, Zhao B, et al. C³frame-work: An open-source PyTorch code for crowd countingarXiv preprint arXiv: 1907. 02724, 2019.
[12]	Li Y H, Zhang X F, Chen D M. CSRNET: Dilated convolutional neural networks for understanding the highly congested scenes[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018:1091-1100. DOI: 10.1109/CVPR.2018.00120 doi: 10.1109/CVPR.2018.00120
[13]	Cao X K, Wang Z P, Zhao Y Y, et al. Scale aggregation network for accurate and efficient crowd counting[C]. Proceedings of the European Conference on Computer Vision, 2018:757-773. DOI: 10.1007/978-3-030-01228-1_45 doi: 10.1007/978-3-030-01228-1_45
[14]	Wang Q, Gao J Y, Lin W, et al. Learning from synthetic data for crowd counting in the wild[C]. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2019:8198-8207. DOI: 10.1109/CVPR.2019.00839 doi: 10.1109/CVPR.2019.00839
[15]	宋宏权, 刘学军, 闾国年,等. 一种可跨摄像机的人群密度估计模型[J]. 中国安全科学学报, 2013, 23(12):139-145.
	[ Song H Q, Liu X J, Lv G N, et al. A cross-camera adaptive crowd density estimation model[J]. China Safety Science Journal, 2013, 23(12):139-145. ] DOI: 10.16265/j.cnki.issn1003-3033.2013.12.024 doi: 10.16265/j.cnki.issn1003-3033.2013.12.024
[16]	邓仕虎, 张兴国, 王小勇,等. 视频和GIS协同的人群状态感知模型[J]. 信阳师范学院学报(自然科学版), 2018, 31(1):59-63.
	[ Deng S H, Zhang X G, Wang X Y, et al. Crowd status analysis based on surveillance videos and GIS[J]. Journal of Xinyang Normal University (Natural Science Edition), 2018, 31(1):59-63. ] DOI: 10.3969/j.issn.1003-0972.2018.01.011 doi: 10.3969/j.issn.1003-0972.2018.01.011
[17]	宋宏权, 王丰, 刘学军,等. 地理环境下的群体运动分析与异常行为检测[J]. 地理与地理信息科学, 2015, 31(4):1-5,11,133.
	[ Song H Q, Wang F, Liu X J, et al. Crowd movement analysis and abnormal behavior detection under geographical environment[J]. Geography and Geo-Information Science, 2015, 31(4):1-5,11,133. ] DOI: 10.3969/j.issn.1672-0504.2015.04.001 doi: 10.3969/j.issn.1672-0504.2015.04.001
[18]	Zhang X G, Shi X Y, Luo X Y, et al. Real-Time web map construction based on multiple cameras and GIS. ISPRS International Journal of Geo-Information, 2021, 10(12):803. DOI: 10.3390/ijgi10120803 doi: 10.3390/ijgi10120803
[19]	Chen, L C Zhu, Y K, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]. European Conference on Computer Vision, 2018:833-851. DOI: 10.1007/978-3-030-01234-2_49 doi: 10.1007/978-3-030-01234-2_49
[20]	Fu H, Fu B H, Shi P H. An improved segmentation method for automatic mapping of cone karst from remote sensing data based on DeepLabV3+ model[J]. Remote Sensing, 2021, 13(3):441. DOI: 10.3390/RS13030441 doi: 10.3390/RS13030441
[21]	Wang Q, Gao J Y, Lin W, et al. NWPU-Crowd: A large-scale benchmark for crowd counting and localization[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(6):2141-2149. DOI: 10.1109/TPAMI.2020.3013269 doi: 10.1109/TPAMI.2020.3013269