基于链接分析的网页文本核心地名提取方法

doi:10.3724/SP.J.1047.2016.00435

摘要/Abstract

摘要：

本文围绕互联网中网页文本蕴含的丰富地理空间信息,抽取网页文本中蕴含的地名实体,提出了一种地名共现网络模型,该模型综合考虑网页中地名的频次信息,表达网页文本中地名的共现及联系传递特征。在此基础上,提出一种基于链接分析的网页文本核心地名的提取方法,通过PageRank算法计算每个地名在共现网络中的链接权重,对网页文本构建的共现地名网络进行核心地名的提取,从而在庞大的网络资源中发现具有显著的焦点特征或导航枢纽特征的重要地名。最后,采用人民日报与新浪新闻体育版2份语料进行实验验证,证明了该方法的有效性。

关键词: 地名, 地名共现, 链接分析, 复杂网络, 地理信息检索

Abstract:

Geographical information explodes with the emergence of Internet, which also adopts brand new ideas to obtain geospatial data with traditional GIS methods. With the abundant geospatial information on the web, we proposed a toponym co-occurrences network model by extracting the toponym entities from web page texts using nature language process methods, as well as uniforming the toponyms, in order to conduct a comprehensive analysis of the web pages. The network set up in this paper is a weighted directed graph, of which every vertex represents a distinct toponym, and the co-occurrence of each two toponyms is displayed as one edge of this network. The frequency of geographic names is taken into consideration synthetically, which shows the weight of each network edge, as well as explains the co-occurrence relationship and transformation occurrence characteristics of those toponyms. On this basis, a method of toponym extraction from web page texts based on link analysis is carried out, taking advantage of the PageRank algorithm to calculate the link weight of every toponym in the co-occurrence network and rank each geographical name with a PageRank score. In this way, the importance of the toponym is calculated and the core geographic names with remarkable features or navigation features in all huge network resources can be found. A case study based on the actual data extracted from People’s Daily and Sina News Sport web pages is carried out to verify the technical solution, which shows that the proposed solution is both feasible and practically effective, which can also be applied to geographical information retrieval. Results show that the core toponym of co-occurrence network differs in different themes of web pages, and when the time sequence factor is taken into account, the core toponym results may also be different within a single theme of web pages.

Key words: toponym, toponym co-occurrence, link analysis, complex network, geographical information retrieval

钟翔, 高勇, 邬伦. 基于链接分析的网页文本核心地名提取方法[J]. 地球信息科学学报, 2016, 18(4): 435-442.DOI:10.3724/SP.J.1047.2016.00435

ZHONG Xiang,GAO Yong,WU Lun. Extract Core Toponyms from Web Page Text Based on Link Analysis[J]. Journal of Geo-information Science, 2016, 18(4): 435-442.DOI:10.3724/SP.J.1047.2016.00435

图/表 10

图1

图2

图3

图4

表1

表2

图5

表3

表4

表5

参考文献 19

[1]	余丽,陆锋,张恒才.网络文本蕴涵地理信息抽取:研究进展与展望[J].地球信息科学学报,2015,17(2):127-134. doi: 10.3724/SP.J.1047.2015.00127
	[ Yu L, Lu F, Zhang H C.Extracting geographic information from Web texts: status and development[J]. Journal of Geo-Information Science, 2015,17(2):127-134. ] doi: 10.3724/SP.J.1047.2015.00127
[2]	张雪英,张春菊,闾国年.地理命名实体分类体系的设计与应用分析[J].地球信息科学学报,2010,12(2):220-227.
	[ Zhang X Y, Zhang C J, Lv G N.Design and analysis of a classification scheme of geographical named entities[J]. Journal of Geo-Information Science, 2010,12(2):220-227. ]
[3]	陈玉萍,张秀.地名地址普查与建库研究[J].测绘通报,2015(6):103-107. doi: 10.13474/j.cnki.11-2246.2015.0191
	[ Chen Y P, Zhang X.Research on the address names census and database building[J]. Bulletin of Surveying and Mapping, 2015,6:103-107. ] doi: 10.13474/j.cnki.11-2246.2015.0191
[4]	刘瑜,张毅,田原,等.广义地名及其本体研究[J].地理与地理信息科学,2007,23(6):1-7. doi: 10.3969/j.issn.1672-0504.2007.06.001
	[ Liu Y, Zhang Y, Tian Y, et al.On general place names and the associated ontology[J]. Geography and Geo-Information Science, 2007,23(6):1-7. ] doi: 10.3969/j.issn.1672-0504.2007.06.001
[5]	Liu J.Chinese named entity recognition algorithm based on the improved hidden Markov model[J]. Journal of Chemical & Pharmaceutical Research, 2014,6(7):1474-1478
[6]	Xia H, Rao R N.The method of medical named entity recognition based on semantic model and improved SVM-KNN algorithm[C]. IEEE International Conference on Semantics, Knowledge and Grid. 2011:21-27.
[7]	Chieu H L, Ng W T.Named entity recognition with a maximum entropy approach[C]. Proceedings of the seventh conference on Natural language learning at HLT-NAACL, 2003,4:160-163.
[8]	Lin Y, SunC J, Li S F, et al. CRF-based active learning for Chinese named entity recognition[C]. IEEE International Conference on Systems, Man and Cybernetics, 2009:1557-1561.
[9]	李照航,郭风华,李仁杰,等.大量网络游记文本中热度地名提取方法与实证研究[J].地理与地理信息科学,2015,31(1):68-73. doi: 10.3969/j.issn.1672-0504.2015.01.015
	[ Li Z H, Guo F H, Li R J, et al.Method and case study of hot-toponym extraction from mass amount of internet travel blog text[J]. Geography and Geo-Information, 2015,31(1):68-73. ] doi: 10.3969/j.issn.1672-0504.2015.01.015
[10]	Wu C, Shen L, Wang X.A new method of using contextual information to infer the semantic orientations of context dependent opinions[C]. IEEE International Conference on Artificial Intelligence and Computational Intelligence, 2009,4:274-278.
[11]	Grube N.Toponyms, emblem glyphs, and the political geography of southern Campeche[J]. Anthropological Notebooks, 2005,11(1):87-100.
[12]	朱少楠,张雪英,李明,等.基于行政隶属关系树状图的地名消歧方法[J].地理与地理信息科学,2013,29(3):39-42. doi: 10.7702/dlydlxxkx20130309
	[ Zhu S N, Zhang X Y, Li M, et al.Toponym disambiguation based on administrative district relation tree[J]. Geography and Geo-Information, 2013,29(3):39-42. ] doi: 10.7702/dlydlxxkx20130309
[13]	王宇. 基于网络文本的地名空间模糊建模[D].南京:南京师范大学,2012:36-53.
	[ Wang Y.Vague modeling method of place names based on web pages[D]. Nanjing:Nanjing Normal University, 2012:36-53. ]
[14]	Liu Y, Wang F, Kang C, et al.Analyzing relatedness by toponym co-occurrences on web pages[J]. Transactions in Gis, 2014,18(1):89-107. doi: 10.1111/tgis.12023
[15]	Henzinger M.Link analysis in web information retrieval[C]. IEEE Data Engineering Bulletin, 2000:3-8.
[16]	Brin S, Page L.The anatomy of a large-scale hypertextual Web search engine[J]. Computer Networks, 1998,30:107-117.
[17]	Boldi P, Santini M, Vigna S.PageRank as a function of the damping factor[C]. Proceedings of the 14th international conference on World Wide Web, ACM, 2005:557-566.
[18]	刘挺,车万翔,李正华.语言技术平台[J].中文信息学报,2011,25(6):53-62.
	[ Liu T, Che W X, Li Z H.Language technology platform[J]. Journal of Chinese Information Processing, 2011,25(6):53-62. ]
[19]	Bastian M, Heymann S, Jacomy M.Gephi: An open source software for exploring and manipulating networks[J]. ICWSM, 2009,8:361-362. doi: 10.13140/2.1.1341.1520

序号	地名	出现文本数	出现词频总数/个
1	中国	366	2259
2	阿尔及利亚	6	11
3	阿富汗	26	106
4	伊朗	24	159
5	英国	30	57
6	阿拉伯	12	25
7	埃及	10	62
8	巴勒斯坦	8	63
9	华盛顿	12	14
10	加沙	8	29

序号	地名	出现文本数	出现词频总数/个
1	澳大利亚	439	1388
2	墨尔本	70	110
3	坎贝尔镇	14	16
4	悉尼	87	136
5	中国	675	2782
6	新加坡	21	27
7	沙特	228	639
8	韩国	198	577
9	布里班斯	156	280
10	朝鲜	126	267

序号		ATF-PDF模型			PageRank模型
序号	地名	ATF-PDF		排名	地名	PR	排名
1	中国	0.490583	1		中国	0.0232	1
2	北京	0.10099	2		北京	0.0131	2
3	美国	0.055318	3		美国	0.0099	3
4	上海	0.039507	4		上海	0.0086	4
5	海地	0.022525	5		新疆	0.0061	5
6	小岗村	0.020617	6		山东	0.0055	6
7	新疆	0.019691	7		广东	0.0054	7
8	人民大会堂	0.016037	8		亚洲	0.0053	8
9	伊朗	0.015747	9		河北	0.0052	9
10	四川	0.013802	10		俄罗斯	0.0051	10

序号		ATF-PDF模型			PageRank模型
序号	地名	ATF-PDF		排名	地名	PR	排名
1	北京	0.3099	1		中国	0.1078	1
2	中国	0.2285	2		澳大利亚	0.0525	2
3	澳大利亚	0.1139	3		北京	0.0417	3
4	沙特	0.0457	4		韩国	0.0284	4
5	美国	0.0450	5		亚洲	0.0269	5
6	乌兹别克斯坦	0.0404	6		沙特	0.0263	6
7	亚洲	0.0383	7		乌兹别克斯坦	0.0263	7
8	韩国	0.0374	8		日本	0.0262	8
9	洛杉矶	0.0363	9		巴西	0.0131	9
10	布里班斯	0.0357	10		美国	0.0124	10

排名	1月			2月			3月
排名	地名		PR		地名	PR	地名	PR
1	中国	0.0232		中国	0.0462		中国	0.1461
2	北京	0.0131		北京	0.0240		北京	0.0591
3	美国	0.0099		美国	0.0217		俄罗斯	0.0323
4	上海	0.0086		上海	0.0164		美国	0.0276
5	新疆	0.0061		四川	0.0140		西藏	0.0237
6	山东	0.0055		西藏	0.0130		上海	0.0171
7	广东	0.0054		俄罗斯	0.0122		日本	0.0162
8	亚洲	0.0053		广东	0.0101		智利	0.0124
9	河北	0.0052		天津	0.0097		新疆	0.0120
10	俄罗斯	0.0051		日本	0.0082		台湾	0.0112