地球信息科学学报 ›› 2011, Vol. 13 ›› Issue (4): 492-499.doi: 10.3724/SP.J.1047.2011.00492

• 网络地理信息分析方法与应用 • 上一篇    下一篇

基于网络爬虫的地名数据库维护方法

张春菊, 张雪英, 朱少楠, 徐希涛   

  1. 南京师范大学虚拟地理环境教育部重点实验室,南京 210046
  • 收稿日期:2011-04-07 修回日期:2011-06-22 出版日期:2011-08-25 发布日期:2011-08-23
  • 通讯作者: 张雪英(1970-),博士,教授,主要从事地理信息系统、地理信息智能处理和服务等方面的研究。 E-mail: zhangsnowy@163.com. E-mail:zhangsnowy@163.com.
  • 基金资助:

    国家自然科学基金项目(40971231)。

Method of Toponym Database Updating Based on Web Crawler

ZHANG Chunju, ZHANG Xueying, ZHU Shaonan, XU Xitao   

  1. Key Laboratory of Virtual Geographical Environment, Ministry of Education, Nanjing Normal University, Nanjing 210046, China
  • Received:2011-04-07 Revised:2011-06-22 Online:2011-08-25 Published:2011-08-23

摘要: 目前,我国地名数据库建设存在大、中颗粒度地名集中,小颗粒度地名较为缺乏,地名资料陈旧、时效性较低,简称、别名等非标准地名信息和地名的相对位置信息缺失等问题。而地名数据库的更新维护工作主要通过人工测绘手段完成,存在周期长、成本高、效率低等缺点。针对这一问题,本文以现有地名数据库和空间关系词汇为基础,基于Google搜索引擎服务,提出一种以网页资源为数据源,利用网络爬虫技术和地名识别技术,进行地名数据库更新维护的方法。首先,设计以地名为主题的网络爬虫,实现非结构化的网页数据中海量空间敏感网页文本的主动获取;然后,采用HTML DOM技术解析空间敏感网页并应用CRF地名识别模型自动识别网页文本中地名;最后,设计相关算法进行网页文本中地名信息的自动解析,实现新地名和地名空间位置信息的获取,进行地名数据库的更新维护。以"南京师范大学仙林宾馆+西北"为空间检索实例,验证了此方法的可行性。

关键词: 网络爬虫, 地名识别, 主题相关性, 地名数据库

Abstract: Generally, toponym database provides description information on place names and its spatial location and feature type. It provides basic information for national administration, economic development, domestic and foreign exchanges, etc. It is a basis for public place name services, particularly for Location-Based-Service (LBS) with a growing demand. Therefore, a toponym database with complete and timely place name information is a premise and guarantee for efficient LBS services. However, currently, there are some problems about place names in our national toponym database. Most of the place names are with a big particle size, and small particle sized and non-standard place names are in shortage, and there are no relative position descriptions of place names in toponym database. Moreover, toponym database updating is based on manual surveying with disadvantages of long cycle, high cost, low efficiency and time consuming. In this paper, a new method for toponym database updating is explored on the technology combination of search engine, web crawler and place name recognition. Firstly, a mass of space-sensitive web pages are obtained by a web crawler which is based on Google search engine and a spatial search subject of "place name" or "place name + spatial relation terms". Secondly, after analysis of web pages with a DOM tree method, place name recognition is completed based on Conditional Random Fields (CRF) recognition model. Finally, automatic spatial location interpretation of place names is completed from candidate web texts which include new place names and spatial location information of place names. This paper also presents a case study with a spatial search subject of "Nanjing Normal University, Xianlin hotel + northwest". The experiment result shows that this method is feasible and effective. However, timely and accurately locating of place names in web pages are in challenge, because publishing time of web pages and change time of place names driven by events in web pages are not considered in this paper. This may result in potential lag of place name information and can't ensure the completeness and consistency of toponym database. In recent years, public participation internet maps can provide accurate and real-time place name source, especially coordinate information, such as GoogleMap, GoogleEarth, OpenStreetMap, etc. Our future work will focus on time attribute interpretation of place names from web pages and obtaining of place names as well as their coordinates from internet maps. Moreover, an integration of place names from different data sources will provide a more effective toponym database updating.

Key words: toponym database, web crawler, place name recognition, theme correlation