基于街景图像的可解释性城市感知模型研究方法

doi:10.12082/dqxxkx.2022.210712

地球信息科学学报 ›› 2022, Vol. 24 ›› Issue (10): 2045-2057.doi: 10.12082/dqxxkx.2022.210712

基于街景图像的可解释性城市感知模型研究方法

刘智谦¹(), 吕建军¹, 姚尧¹^,²^,^*(), 张嘉琪¹, 寇世浩¹, 关庆锋¹

1.中国地质大学（武汉）地理与信息工程学院,武汉 430078
2.阿里巴巴集团,杭州 311121

收稿日期:2021-11-05 修回日期:2022-01-20 出版日期:2022-10-25 发布日期:2022-12-25
通讯作者: *姚尧(1987— ),男,广东梅州人,博士,副教授,研究方向为空间大数据和城市计算。E-mail: yaoy@cug.edu.cn
作者简介:刘智谦(1997— ),男,四川达州人,硕士生,研究方向为城市计算和空间分析。E-mail: liuzhiqian@cug.edu.cn
基金资助:
国家自然科学基金项目(41671408);国家自然科学基金项目(41901332);国家重点研发计划项目(2019YFB2102903);湖北省自然科学基金项目(2017CFA041)

Research Method of Interpretable Urban Perception Model based on Street View Imagery

LIU Zhiqian¹(), LV Jianjun¹, YAO Yao¹^,²^,^*(), ZHANG Jiaqi¹, KOU Shihao¹, GUAN Qingfeng¹

1. School of Geography and Information Engineering, China University of Geosciences, Wuhan 430078, China
2. Alibaba Group, Hangzhou 311121, China

Received:2021-11-05 Revised:2022-01-20 Online:2022-10-25 Published:2022-12-25
Contact: YAO Yao
Supported by:
National Natural Science Foundation of China(41671408);National Natural Science Foundation of China(41901332);National Key Research and Development Program of China(2019YFB2102903);Natural Science Fund of Hubei Province(2017CFA041)

摘要/Abstract

摘要：

理解城市环境对人类感知的影响,对城市合理规划及布局具有重要的人文参考价值。城市环境是一个动态变化的复杂系统,具有空间异质性的特点。由于研究方法的限制,在复杂的城市环境中,以往基于街景图像的城市感知研究难以全面精细地分析环境关键要素对人类感知的影响。本研究以武汉市中心为研究区,首先利用全卷积神经网络将街景图像分割为城市地物类型,耦合感知打分数据和随机森林算法建立6类城市感知模型;然后基于沙普利值方法分解在随机森林模型中各类城市地物对人类感知的影响,并识别城市环境关键要素;最后结合分解结果,探究在非线性模型中沙普利值方法的适用性和优势。结果表明：沙普利值方法能够有效考虑环境异质性,精确地定量表示在不同场景中各类地物对人类感知的影响;城市高楼、天空、绿地空间是对人类感知影响最大的3类地物,且地物的体积和分布与其对人类感知的影响有关,图像占比大、分布连续的地物对人类感知的影响比图像占比小、分布离散的地物对人类感知的影响大;受城市环境空间异质性的影响,主要地物类型对各类感知的影响程度和形式有显著不同;高楼与人类感知为非线性关系,且具有明显的单调递增或递减的形式;绿地空间与积极感知呈非线性关系,与消极感知呈线性递减的关系。基于可解释性方法,本研究主要分析城市环境关键要素对人类感知的影响特点,探究了城市感知模型中的可解释性问题,能够为城市感知相关研究提供方法参考和理论依据,同时也可为城市规划和景观设计提供参考。

关键词: 城市环境, 街景图像, 城市感知, 全卷积神经网络, 随机森林, 沙普利值, 可解释机器学习, 城市环境空间异质性

Abstract:

Understanding the impact of urban environment on human perception has important humanistic reference value for rational urban planning and layout. The urban environment is a dynamic and complex system with spatial heterogeneity. Due to the limitations of research methods in the complex urban environment, it is difficult to analyze the impact of key elements of the environment on human perception comprehensively and precisely in previous urban perception studies based on street view images. In this study, the center of Wuhan is chosen as the research area. First, the street view images are segmented into urban ground objects by using Fully Convolutional Networks, coupled with perception scoring data and Random Forests algorithm to establish six types of urban perception models. Then, based on the Shapley value method, the impact of various urban ground objects on human perception in the forest model is decomposed, and the key elements of the urban environment are identified. Finally, combined with the decomposition results, the applicability and advantages of the Shapley value method in the nonlinear model are explored. The results show that the Shapley value method can effectively consider the spatial heterogeneity of urban physical environment and accurately and quantitatively express the impact of various ground objects on human perception in different scenarios. Urban buildings, sky, and green space are the three categories that have the greatest degree of impact on human perception. And the volume and distribution form of ground objects affects human intuitive visual experience of ground objects, which in turn affects the specific impact role of ground objects on human perception. The larger the image proportion of the ground objects or the more continuous the distribution of the ground objects, the greater the impact of ground objects on human perception. At the same time, because of the influence of the spatial heterogeneity of the urban physical environment, the impact degree and form of the main ground object types on various perceptions are significantly different. The relationship between buildings and human perception is nonlinear and has an obvious monotonically increasing or decreasing form. Green space has a nonlinear relationship with positive perception and a linear decreasing relationship with negative perception. Based on the interpretability approach, this study analyzes the impact of key elements of urban environment on human perception and explores the interpretability issues in urban perception models, which can provide methodological reference and theoretical basis for urban perception related research, as well as reference for urban planning and landscape design.

Key words: urban environment, street view imagery, urban perception, Fully Convolutional Networks, Random Forests, Shapley value, interpretable machine learning, spatial heterogeneity of urban environment

刘智谦, 吕建军, 姚尧, 张嘉琪, 寇世浩, 关庆锋. 基于街景图像的可解释性城市感知模型研究方法[J]. 地球信息科学学报, 2022, 24(10): 2045-2057.DOI:10.12082/dqxxkx.2022.210712

LIU Zhiqian, LV Jianjun, YAO Yao, ZHANG Jiaqi, KOU Shihao, GUAN Qingfeng. Research Method of Interpretable Urban Perception Model based on Street View Imagery[J]. Journal of Geo-information Science, 2022, 24(10): 2045-2057.DOI:10.12082/dqxxkx.2022.210712

图/表 9

表1

图1

表2

图2

图3

表3

图4

图5

图6

参考文献 42

[1]	Huang Y, Fei T, Kwan M P, et al. GIS-Based emotional computing: A review of quantitative approaches to measure the emotion layer of human-environment relationships[J]. ISPRS International Journal of Geo-information, 2020, 9(9):551. DOI: 10.3390/ijgi9090551 doi: 10.3390/ijgi9090551
[2]	张丽英, 裴韬, 陈宜金, 等. 基于街景图像的城市环境评价研究综述[J]. 地球信息科学学报, 2019, 21(1):46-58. doi: 10.12082/dqxxkx.2019.180311
	[ Zhang L Y, Pei T, Chen Y J, et al. A review of urban environment assessment based on street view images[J]. Journal of Geo-information Science, 2019, 21(1):46-58. ] DOI: 10. 12082/dqxxkx.2019.180311 doi: 10. 12082/dqxxkx.2019.180311
[3]	Weber R, Schnier J, Jacobsen T. Aesthetics of streetscapes: influence of fundamental properties on aesthetic judgments of urban space[J]. Perceptual and Motor Skills, 2008, 106(1):128-146. DOI: 10.2466/pms.106.1.128-146 doi: 10.2466/pms.106.1.128-146 pmid: 18459363
[4]	Zhang F, Zhou B, Liu L, et al. Measuring human perceptions of a large-scale urban region using machine learning[J]. Landscape and Urban Planning, 2018, 180:148-160. DOI: 10.1016/j.landurbplan.2018.08.020 doi: 10.1016/j.landurbplan.2018.08.020
[5]	Porzi L, Rota Bulò S, Lepri B, et al. Predicting and understanding urban perception with convolutional neural networks[C]. Proceedings of the 23rd ACM international conference on Multimedia, 2015:139-148. DOI: 10.1145/2733373.2806273 doi: 10.1145/2733373.2806273
[6]	Yao Y, Liang Z, Yuan Z, et al. A human-machine adversarial scoring framework for urban perception assessment using street-view images[J]. International Journal of Geographical Information Science, 2019, 33(12):2363-2384. DOI: 10.1080/13658816.2019.164.3024 doi: 10.1080/13658816.2019.164.3024
[7]	Kang Y, Jia Q, Gao S, et al. Extracting human emotions at different places based on facial expressions and spatial clustering analysis[J]. Transactions in GIS, 2019, 23(3):450-480. DOI: 10.1111/tgis.12552 doi: 10.1111/tgis.12552
[8]	Wang R, Liu Y, Lu Y, et al. Perceptions of built environment and health outcomes for older Chinese in Beijing: A big data approach with street view images and deep learning technique[J]. Computers, Environment and Urban Systems, 2019, 78:101386. DOI: 10.1016/j.compenvurbsy s.2019.101386 doi: 10.1016/j.compenvurbsy s.2019.101386
[9]	Dubey A, Naik N, Parikh D, et al. Deep learning the city: quantifying urban perception at a global scale[C]. European Conference on Computer Vision. Springer, Cham, 2016:196-212. DOI: 10.1007/978-3-319-46448-0_12 doi: 10.1007/978-3-319-46448-0_12
[10]	Anguelov D, Dulong C, Filip D, et al. Google street view: capturing the world at street level[J]. Computer, 2010, 43(6):32-38. DOI: 10.1109/MC.2010.170 doi: 10.1109/MC.2010.170
[11]	Zhang F, Zu J, Hu M, et al. Uncovering inconspicuous places using social media check-ins and street view images[J]. Computers, Environment and Urban Systems, 2020, 81:101478. DOI: 10.1016/j.compenvurbsys.2020.101478 doi: 10.1016/j.compenvurbsys.2020.101478
[12]	崔成, 任红艳, 赵璐, 等. 基于街景影像多特征融合的广州市越秀区街道空间品质评估[J]. 地球信息科学学报, 2020, 22(6):1330-1338. doi: 10.12082/dqxxkx.2020.200072
	[ Cui C, Ren H Y, Zhao L, et al. Street space quality evaluation in Yuexiu district of Guangzhou city based on multi-feature fusion of street view[J]. Journal of Geo-information Science, 2020, 22(6):1330-1338. ] DOI: 10.12082/dqxxkx.2020.200072 doi: 10.12082/dqxxkx.2020.200072
[13]	Wang R, Ren S, Zhang J, et al. A comparison of two deep-learning-based urban perception models: Which one is better?[J]. Computational Urban Science, 2021, 1(1):1-13. DOI: 10.6084/m9.figshare.12552233 doi: 10.6084/m9.figshare.12552233
[14]	Wang R, Yuan Y, Liu Y, et al. Using street view data and machine learning to assess how perception of neighborhood safety influences urban residents' mental health[J]. Health & Place, 2019, 59:102186. DOI: 10.1016/j.healthplace.2019.102186 doi: 10.1016/j.healthplace.2019.102186
[15]	Helbich M, Yao Y, Liu Y, et al. Using deep learning to examine street view green and blue spaces and their associations with geriatric depression in Beijing, China[J]. Environment International, 2019, 126:107-117. DOI: 10.1016/j.envint.2019.02.013 doi: S0160-4120(18)32539-X pmid: 30797100
[16]	Wang R, Helbich M, Yao Y, et al. Urban greenery and mental wellbeing in adults: Cross-sectional mediation analyses on multiple pathways across different greenery measures[J]. Environmental Research, 2019, 176:108535. DOI: 10.1016/j.envres.2019.108535 doi: 10.1016/j.envres.2019.108535
[17]	Evans G W. The built environment and mental health[J]. Journal of Urban Health, 2003, 80(4):536-555. DOI: 10. 1093/jurban/jtg063 doi: 10. 1093/jurban/jtg063
[18]	Lee A C, Maheswaran R. The health benefits of urban green spaces: a review of the evidence[J]. Journal of Public Health, 2011, 33(2):212-222. DOI: 10.1093/pubmed/fdq068 doi: 10.1093/pubmed/fdq068
[19]	Li X, Hijazi I, Koenig R, et al. Assessing essential qualities of urban space with emotional and visual data based on GIS technique[J]. ISPRS International Journal of Geo-Information, 2016, 5(11):218. DOI: 10.3390/ijgi5110218 doi: 10.3390/ijgi5110218
[20]	陈逸敏, 黎夏. 机器学习在城市空间演化模拟中的应用与新趋势[J]. 武汉大学学报·信息科学版, 2020, 45(12):1884-1889.
	[ Chen Y M, Li X. Application and new trends of machine learning in urban simulation research[J]. Geomatics and Information Science of Wuhan University, 2020, 45(12):1884-1889. ] DOI: 10.13203/j.whugis20 200423 doi: 10.13203/j.whugis20
[21]	Zhang Q, Zhu S. Visual interpretability for deep learning: a survey[J]. Frontiers of Information Technology & Electronic Engineering, 2018, 19(1):27-39. DOI: 10.1631/FITEE.1700808 doi: 10.1631/FITEE.1700808
[22]	Gilpin L H, Bau D, Yuan B Z, et al. Explaining explanations: an overview of interpretability of machine learning[C]. 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA). IEEE, 2018: 80-89. DOI: 10.1109/DSAA.2018.00018 doi: 10.1109/DSAA.2018.00018
[23]	Carvalho D V, Pereira E M, Cardoso J S. Machine learning interpretability: a survey on methods and metrics[J]. Electronics, 2019, 8(8): 832. DOI:10.3390/electronics8080 832
[24]	Murdoch W J, Singh C, Kumbier K, et al. Definitions, methods, and applications in interpretable machine learning[J]. Proceedings of the National Academy of Sciences, 2019, 116(44):22071-22080. DOI: 10.1073/pnas.1900654116 doi: 10.1073/pnas.1900654116
[25]	Du M, Liu N, Hu X. Techniques for interpretable machine learning[J]. Communications of the ACM, 2019, 63(1):68-77. DOI: 10.1145/3359786 doi: 10.1145/3359786
[26]	Cha Y K, Shin J, Go B G, et al. An interpretable machine learning method for supporting ecosystem management: application to species distribution models of freshwater macroinvertebrates[J]. Journal of Environmental Management, 2021, 291:112719. DOI: 10.1016/j.jenvman.2021.1 12719 doi: 10.1016/j.jenvman.2021.1 12719
[27]	Lundberg S M, Lee S. A unified approach to interpreting model predictions[C]. Proceedings of the 31^st International Conference on Neural Information Processing Systems, 2017:4768-4777.
[28]	Chen L, Yao X, Liu Y, et al. Measuring impacts of urban environmental elements on housing prices based on multisource data-a case study of shanghai, china[J]. ISPRS International Journal of Geo-information, 2020, 9(2):106. DOI: 10.3390/ijgi9020106 doi: 10.3390/ijgi9020106
[29]	Yu S, Chen Z, Yu B, et al. Exploring the relationship between 2D/3D landscape pattern and land surface temperature based on explainable eXtreme Gradient Boosting tree: A case study of Shanghai, China[J]. Science of the Total Environment, 2020, 725:138229. DOI: 10.1016/j.scitotenv.2020.138229 doi: 10.1016/j.scitotenv.2020.138229
[30]	Parsa A B, Movahedi A, Taghipour H, et al. Toward safer highways, application of XGBoost and SHAP for real-time accident detection and feature analysis[J]. Accident Analysis & Prevention, 2020, 136:105405. DOI: 10.1016/j.aap.2019.105405 doi: 10.1016/j.aap.2019.105405
[31]	Li L, Li J, Jiang Z, et al. Methods of population spatialization based on the classification information of buildings from China's first national geoinformation survey in urban area: a case study of Wuchang district, Wuhan city, China[J]. Sensors, 2018, 18(8): 2558. DOI:10.3390/s1808 2558 doi: 10.3390/s18061808
[32]	Breiman L. Random forests[J]. Machine learning, 2001, 45(1):5-32. DOI: 10.1023/A:1010933404324 doi: 10.1023/A:1010933404324
[33]	Lundberg S M, Erion G, Chen H, et al. From local explanations to global understanding with explainable AI for trees[J]. Nature Machine Intelligence, 2020, 2(1):56-67. DOI: 10.1038/s42256-019-0138-9 doi: 10.1038/s42256-019-0138-9 pmid: 32607472
[34]	Tyrväinen L, Ojala A, Korpela K, et al. The influence of urban green environments on stress relief measures: Afield experiment[J]. Journal of Environmental Psychology, 2014, 38:1-9. DOI: 10.1016/j.jenvp.2013.12.005 doi: 10.1016/j.jenvp.2013.12.005
[35]	Twohig-Bennett C, Jones A. The health benefits of the great outdoors: A systematic review and meta-analysis of greenspace exposure and health outcomes[J]. Environmental Research, 2018, 166:628-637. DOI: 10.1016/j.env res.2018.06.030 doi: S0013-9351(18)30332-3 pmid: 29982151
[36]	Roberts H, Sadler J, Chapman L. The value of Twitter data for determining the emotional responses of people to urban green spaces: A case study and critical evaluation[J]. Urban Studies, 2019, 56(4):818-835. DOI:10.117/004 2098017748544 doi: 10.1177/0042098017748544
[37]	Kuo F E, Bacaicoa M, Sullivan W C. Transforming inner-city landscapes: Trees, sense of safety, and preference[J]. Environment and Behavior, 1998, 30(1):28-59. DOI: 10. 1177/0013916598301002 doi: 10. 1177/0013916598301002
[38]	He L, Páez A, Liu D. Built environment and violent crime: an environmental audit approach using google street view[J]. Computers, Environment and Urban Systems, 2017, 66:83-95. DOI: 10.1016/j.compenvurbsys.2017. 08.001 doi: 10.1016/j.compenvurbsys.2017. 08.001
[39]	Ballew M T, Omoto A M. Absorption: how nature experiences promote awe and other positive emotions[J]. Ecopsychology, 2018, 10(1):26-35. DOI: 10.1089/eco.2017.0044 doi: 10.1089/eco.2017.0044
[40]	Ma B, Zhou T, Lei S, et al. Effects of urban green spaces on residents' well-being[J]. Environment, Development and Sustainability, 2019, 21(6):2793-2809. DOI: 10.1007/s10668-018-0161-8 doi: 10.1007/s10668-018-0161-8
[41]	McEwan K, Ferguson F J, Richardson M, et al. The good things in urban nature: A thematic framework for optimising urban planning for nature connectedness[J]. Landscape and Urban Planning, 2020, 194:103687. DOI: 10.1 016/j.landurbplan.2019.103687 doi: 10.1 016/j.landurbplan.2019.103687
[42]	Pouso S, Borja Á, Fleming L E, et al. Contact with blue-green spaces during the COVID-19 pandemic lockdown beneficial for mental health[J]. Science of the Total Environment, 2021, 756:143984. DOI: 10.1016/j.scitotenv.2020. 143984 doi: 10.1016/j.scitotenv.2020. 143984

名称	类型	来源
街景图像数据集	图像数据	通过腾讯地图API获取
感知打分数据集	表格型	由志愿者人工评判街景图像所得
城市地物类型数据集	表格型	通过FCN-8s方法对街景图像进行语义分割所得

地物类别	地物中文名称	出现次数/%	占比均值/%
自然地物	草	77.714	1.428
	植物	80.101	1.547
	天空	99.936	29.180
	树	98.865	14.207
人工地物	围栏	90.553	1.294
	地面	77.883	1.624
	指示物	56.249	0.060
	人行道	92.127	2.614
	广告牌	84.955	0.337
	路灯	69.736	0.046
	道路	99.545	18.430
	机动汽车	90.223	2.553
	墙	91.250	2.042
	高楼建筑	98.929	14.951

感知类型	R²	RMSE	MAE
美丽	0.925	2.807	2.073
无聊	0.855	2.310	1.746
沮丧	0.974	1.680	1.258
活泼	0.935	3.320	2.491
安全	0.947	2.518	1.900
富裕	0.953	2.490	1.852

基于街景图像的可解释性城市感知模型研究方法

Research Method of Interpretable Urban Perception Model based on Street View Imagery

RichHTML

PDF (PC)

赞

可视化

摘要/Abstract

引用本文

使用本文

图/表 9

参考文献 42

相关文章 15

Metrics

本文评价

推荐阅读 0

[1]	李玉洁, 江洪, 刘宣广. 基于CatBoost的植被总初级生产力遥感模拟方法及在福建省的应用[J]. 地球信息科学学报, 2023, 25(9): 1908-1922.
[2]	曹煜, 方秀琴, 杨露露, 蒋心远, 廖美玉, 任立良. 基于随机森林的西辽河流域CCI土壤湿度降尺度研究[J]. 地球信息科学学报, 2023, 25(8): 1669-1681.
[3]	林炫歆, 肖桂荣, 周侯伯. 顾及土地利用动态变化的滑坡易发性评估方法[J]. 地球信息科学学报, 2023, 25(5): 953-966.
[4]	李心雨, 闫浩文, 王卓, 王炳瑄. 街景图像与机器学习相结合的道路环境安全感知评价与影响因素分析[J]. 地球信息科学学报, 2023, 25(4): 852-865.
[5]	万江琴, 费腾. 基于街景图像的“三生空间”识别方法研究[J]. 地球信息科学学报, 2023, 25(4): 838-851.
[6]	衡雪彪, 许捍卫, 唐璐, 汤恒, 许怡蕾. 基于改进全卷积神经网络模型的土地覆盖分类方法研究[J]. 地球信息科学学报, 2023, 25(3): 495-509.
[7]	秦肖伟, 程博, 杨志平, 李林, 董文, 张新, 杨树文, 靳宗义, 薛庆. 基于时序遥感影像的西南山区地块尺度作物类型识别[J]. 地球信息科学学报, 2023, 25(3): 654-668.
[8]	黄帅元, 董有福, 李海鹏. 黄土高原区SRTM1 DEM高程误差校正模型构建及对比分析[J]. 地球信息科学学报, 2023, 25(3): 669-681.
[9]	杜树坤, 张晶, 韩志军, 公茂玉. 基于随机森林模型的“网格-月”尺度武装冲突风险预测及影响因素分析——以中南半岛为例[J]. 地球信息科学学报, 2023, 25(10): 2026-2038.
[10]	潘雨飘, 赵翔, 王静, 张亦清, 刘耀林. 基于SMOTE-RF算法的村庄发展类型识别方法研究[J]. 地球信息科学学报, 2023, 25(1): 163-176.
[11]	杨洋, 邵哲平, 赵强, 潘家财, 胡雨, 梅强. 基于厦门港的海上交通事故地理空间分布及风险预测研究[J]. 地球信息科学学报, 2022, 24(9): 1676-1687.
[12]	吴亚楠, 郭长恩, 于东平, 段爱民, 刘玉, 董士伟, 单东方, 吴耐明, 李西灿. 基于不确定性分析的遥感分类空间分层及评估方法[J]. 地球信息科学学报, 2022, 24(9): 1803-1816.
[13]	聂祥琴, 陈瀚阅, 牛铮, 张黎明, 刘炜, 邢世和, 范协裕, 李家国. 基于时序影像的农业活动因子提取与闽西耕地SOC数字制图[J]. 地球信息科学学报, 2022, 24(9): 1835-1852.
[14]	张旭, 柳林, 周翰林, 岳瀚, 孙秋远. 基于贝叶斯逻辑回归模型研究百度街景图像微观建成环境因素对街面犯罪的影响[J]. 地球信息科学学报, 2022, 24(8): 1488-1501.
[15]	高位, 涂伟, 李明晓, 方碧宸, 陈栋胜, 黄正东, 贺彪. 基于影响因子动态更新的都市圈国土-人口协同模拟[J]. 地球信息科学学报, 2022, 24(8): 1502-1511.