ARTICLES

Method of Toponym Database Updating Based on Web Crawler

Expand
  • Key Laboratory of Virtual Geographical Environment, Ministry of Education, Nanjing Normal University, Nanjing 210046, China

Received date: 2011-04-07

  Revised date: 2011-06-22

  Online published: 2011-08-23

Abstract

Generally, toponym database provides description information on place names and its spatial location and feature type. It provides basic information for national administration, economic development, domestic and foreign exchanges, etc. It is a basis for public place name services, particularly for Location-Based-Service (LBS) with a growing demand. Therefore, a toponym database with complete and timely place name information is a premise and guarantee for efficient LBS services. However, currently, there are some problems about place names in our national toponym database. Most of the place names are with a big particle size, and small particle sized and non-standard place names are in shortage, and there are no relative position descriptions of place names in toponym database. Moreover, toponym database updating is based on manual surveying with disadvantages of long cycle, high cost, low efficiency and time consuming. In this paper, a new method for toponym database updating is explored on the technology combination of search engine, web crawler and place name recognition. Firstly, a mass of space-sensitive web pages are obtained by a web crawler which is based on Google search engine and a spatial search subject of "place name" or "place name + spatial relation terms". Secondly, after analysis of web pages with a DOM tree method, place name recognition is completed based on Conditional Random Fields (CRF) recognition model. Finally, automatic spatial location interpretation of place names is completed from candidate web texts which include new place names and spatial location information of place names. This paper also presents a case study with a spatial search subject of "Nanjing Normal University, Xianlin hotel + northwest". The experiment result shows that this method is feasible and effective. However, timely and accurately locating of place names in web pages are in challenge, because publishing time of web pages and change time of place names driven by events in web pages are not considered in this paper. This may result in potential lag of place name information and can't ensure the completeness and consistency of toponym database. In recent years, public participation internet maps can provide accurate and real-time place name source, especially coordinate information, such as GoogleMap, GoogleEarth, OpenStreetMap, etc. Our future work will focus on time attribute interpretation of place names from web pages and obtaining of place names as well as their coordinates from internet maps. Moreover, an integration of place names from different data sources will provide a more effective toponym database updating.

Cite this article

ZHANG Chunju, ZHANG Xueying, ZHU Shaonan, XU Xitao . Method of Toponym Database Updating Based on Web Crawler[J]. Journal of Geo-information Science, 2011 , 13(4) : 492 -499 . DOI: 10.3724/SP.J.1047.2011.00492

References

[1] Goodchild M F, Hill L L. Introduction to Digital Gazetteer Research[J]. Geographical Information Science, 2008, 22(10):1039-1044.

[2] 张雪英,张春菊,闾国年. 地理命名实体分类体系的设计与应用分析[J].地球信息科学学报,2010,12(2):220-227.

[3] 陈钻,万庆,吴杰.基于XML的无线位置服务地理信息服务器的实现[J].地球信息科学,2004,6(4):100-104.

[4] http://www.alexandria.ucsb.edu/adl/.

[5] http://nhd.usgs.gov/gnis.html.

[6] http://www.ga.gov.au/place-name/.

[7] 狄琳,欧阳宏斌.全国1:25万地名数据库的设计与建立[J]. 测绘通报, 2010, 10:32-33.

[8] 陈春华.1∶5万地名数据库到1:1万地名数据库转换的研究与开发[J]. 测绘通报,2006, 5:71-72.

[9] 张保钢,杨伯钢,孔俊元. 北京市地名数据库的维护更新[J]. 北京测绘, 2010, 3:28-30.

[10] Palkowsky B and MetaCarta I. A New Approach to Information Discovery-Geography Really Does Matter. In Proceedings of the SPE Annual Technical Conference and Exhibition, 2005.

[11] Hill L L. Core Elements of Digital Gazetteers: Place Names, Categories, and Footprints. Research and Advanced Technology for Digital Libraries.Berlin, Germany: Springer, 2000,280-290.

[12] 李金良,张雪英,樊晓春. 汉语地名时空信息一体化表达[J]. 地理与地理信息科学,2010,26(6):6-10.

[13] 陈丛丛. 主题爬虫搜索策略研究. 山东大学, 2009.

[14] 李勇,韩亮.主题搜索引擎中网络爬虫的搜索策略研究[J].计算机工程与科学, 2008,30(3): 4-6.

[15] 陈财森,王韬,郑伟. 基于搜索引擎调用的主题搜索设计与实现[J]. 计算机工程与设计,2008, 29(21): 5627-5629.

[16] Diligenti M, Coetze M, Lawrence S, et al. Focused Crawling Using Context Graphs.In Proceedings of the 26th International Conference on Very Large DataBases, Cairo, 2000,527-534.

[17] 刘秉权,王喻红,葛冬梅,等. 基于结构树解析的网页正文抽取方法. 黑龙江省计算机学会2007年学术交流年会, 2007, 14-17.

[18] 周俊生,戴新宇.自然语言信息抽取中的机器学习方法研究[J]. 计算机科学, 2005, 32(3): 186-199.

[19] 张小衡,王玲玲. 中文机构名称的识别与分析[J]. 中文信息学报, 1997, 11(4): 21-32.

[20] 王志强. 基于条件随机域的中文命名实体识别研究. 南京理工大学, 2006.

[21] Bowerman M. Learning How to Structure Space for Language: A Cross linguistic Perspective[M]. // Bloom P,et al.(Eds.). Language and Space, Cambridge, MA, USA: MIT Press. 1996,385-436.
Outlines

/