地学数据共享网用户Web行为预测及数据推荐方法

doi:10.3724/SP.J.1047.2017.00595

Abstract

Abstract:

Efficient and precise discovery of geoscience data on data sharing websites has been a challenge for years. This study applied Web mining techniques for National Earth Science Data Sharing Platform to derive user searching and visit behaviors using clustering algorithm. We proposed cluster-based approaches for search recommendation and visit recommendation. At data preprocessing stage, data cleaning, user identification, session identification and search terms extraction were performed. At user behavior mining stage, DBSCAN algorithm was employed for session clustering with Jaccard distance metric, considering the binary nature of session vectors. To mine user search patterns, we regard the collection of search term in each cluster as a document of text, and the collection of the whole historical search terms as corpus. Thereby, TF-IDF value of each search term in each cluster was then generated. In the scenario of online search recommendation, the real-time search term is taken to index the TF-IDF values in the clusters, and return the cluster with highest TF-IDF value. The items with top frequency is generated as recommendation list. As in the scenario of online visit recommendation, real-time visit vector is taken to query the clusters by the distance between the visit vector and cluster centroids. The nearest cluster is selected to generate most frequent items in the cluster as recommendation. Results of the experiment revealed the hot research topics of geoscience in recent years. The proposed search recommendation has a fair precision and recall, and visit recommendation was considerably improved compared to frequency-based approach. It can be concluded that: (1) web users of geoscience data sharing are more professional and predictable compared with normal web users; (2) DBSCAN is density-based clustering algorithm. It is vital to specifically define user behavior and chose a proper distance metric; (3) TF-IDF-based approach to predict users' search needs is feasible. The resulted search recommendation could be complementation to keyword-based searching. The outcome of this study would potentially contribute to the development of National Earth Science Data Sharing Platform, and even other science data sharing platform.

Key words: Web Usage Mining, spatial data mining, user behavior mining, science data sharing, Earth System Science data

WANG Mo,WANG Juanle,HE Yuntao. An Approach for Prediction of Web User Behavior and Data Recommendation for Geoscience Data Sharing Portals[J].Journal of Geo-information Science, 2017, 19(5): 595-604.DOI:10.3724/SP.J.1047.2017.00595

Figures/Tables 10

Fig. 1

Fig. 2

Tab. 1

Tab. 2

Fig. 3

Tab. 3

Fig. 4

Fig. 5

Tab. 4

Fig. 6

References 29

[1]	李娟,刘德洪,江洪. 国际科学数据共享现状研究[J].图书馆建设,2009(2):19-22.
	[Li J, Liu D H, Jiang H.Research on international scientific data sharing[J]. Library Development, 2009,2:19-22. ]
[2]	王卷乐,诸云强,谢传节.地球系统科学数据共享网络平台的设计和开发[J].地学前缘,2006,13(3):54-59.
	[Wang J L, Zhu Y Q, Xie C J.Network platform design and development for Earth System Science data sharing[J]. Earth Science Frontiers, 2006,13(3):54-59. ]
[3]	Liu B.Web usage mining[J]. Web Data Mining: Exploring Hyperlinks, Contents, and Usage Data, 2007:449-483.
[4]	Liu B.Web data mining: exploring hyperlinks, contents, and usage data[M]. New York: Springer Science & Business Media, 2007.
[5]	Mobasher B, Dai H, Luo T, et al.Discovery and evaluation of aggregate usage profiles for web personalization[J]. Data Mining and Knowledge Discovery, 2002,6(1):61-82. doi: 10.1023/A:1013232803866
[6]	Zhang X, Edwards J, Harding J A.Personalised online sales using web usage data mining[J]. Computers in Industry, 2007,58:772-782. doi: 10.1016/j.compind.2007.02.004
[7]	邓爱林,左子叶,朱扬勇.基于项目聚类的协同过滤推荐算法[J].小型微型计算机系统, 2004,25(9):1665-1670. doi: 10.3969/j.issn.1000-1220.2004.09.023
	[ Deng A L, Zuo Z Y, Zhu Y Y.Colaborative filtering recommendation algorithm based on item clustering[J]. Mini-micro Systems, 2004,25(9):1665-1670. ] doi: 10.3969/j.issn.1000-1220.2004.09.023
[8]	王国霞,刘贺平.个性化推荐系统综述[J].计算机工程与应用,2012,48(7):66-76.
	[Wang G X, Liu H P.A survey on personalised recommender systems[J]. Computer Engineering and Applications, 2012,48(7):66-76. ]
[9]	Van Meteren R, Van Someren M.Using content-based filtering for recommendation[C]. Proceedings of the Machine Learning in the New Information Age: MLnet/ECML2000 Workshop, F, 2000.
[10]	Herlocker J L, Konstan J A, Terveen L G, et al.Evaluating collaborative filtering recommender systems[J]. ACM Transactions on Information Systems (TOIS), 2004,22(1):5-53.
[11]	Vaz P C, Martins de Matos D, Martins B, et al. Improving a hybrid literary book recommendation system through author ranking[C]. Proceedings of the 12th ACM/IEEE-CS joint conference on Digital Libraries, F, 2012.
[12]	Azaria A, Hassidim A, Kraus S, et al.Movie recommender system for profit maximization[C]. Proceedings of the 7th ACM conference on Recommender systems, 2013.
[13]	Wei S, Zheng X, Chen D, et al.A hybrid approach for movie recommendation via tags and ratings[J]. Electronic Commerce Research and Applications, 2016,18:83-94. doi: 10.1016/j.elerap.2016.01.003
[14]	Domingues M A, Gouyon F, Jorge A M, et al.Combining usage and content in an online recommendation system for music in the long tail[J]. International Journal of Multimedia Information Retrieval, 2013,2(1):3-13. doi: 10.1145/2187980.2188224
[15]	Wang X, Wang Y.Improving content-based and hybrid music recommendation using deep learning[C]. Proceedings of the 22nd ACM international conference on Multimedia, 2014.
[16]	Guerbas A, Addam O, Zaarour O, et al.Effective web log mining and online navigational pattern prediction[J]. Knowledge-Based Systems, 2013,49:50-62. doi: 10.1016/j.knosys.2013.04.014
[17]	王卷乐,孙九林.地球系统科学数据共享标准规范体系研究与应用[J].地理科学进展,2010,28(6):839-847. doi: 10.11820/dlkxjz.2009.06.002
	[Wang J L, Sun J L.Study on scientific data sharing standards and specifications systems for Earth System Science and its application[J]. Progress in Geography, 2010,28(6): 839-847. ] doi: 10.11820/dlkxjz.2009.06.002
[18]	诸云强,孙九林,廖顺宝,等.地球系统科学数据共享研究与实践[J].地球信息科学学报,2010,12(1):1-8.
	[Zhu Y Q, Sun J L, Liao S B, et al.Earth system scientific data sharing research and practice[J] Journal of Geo-information Science, 2010,12(1):1-8. ]
[19]	Chitraa V, Davamani D, Selvdoss A.A survey on preprocessing methods for web usage data[J]. arXiv preprint arXiv:10041257, 2010,7(3):78-83.
[20]	Wang M, Wang J.A data preprocessing framework of geoscience data sharing portal for user behavior mining[C]. Proceedings of the Geoinformatics, 2015 23rd International Conference on, F 19-21 June, 2015.
[21]	王末,王卷乐. Web 环境下地学数据共享用户行为模式分析[J].地球信息科学学报,2016,18(9):1174-1183. doi: 10.3724/SP.J.1047.2016.01174
	[Wang M, Wang J L.A study on user behavior of geoscience data sharing based on web usage mining[J]. Journal of Geo-informaiton Science, 2016,18(9):1174-1183. ] doi: 10.3724/SP.J.1047.2016.01174
[22]	Berendt B, Mobasher B, Nakagawa M, et al.The impact of site structure and user environment on session reconstruction in web usage analysis[C]. Proceedings of the International Workshop on Mining Web Data for Discovering Usage Patterns and Profiles, 2002.
[23]	Ester M, Kriegel H P, Sander J, et al.A density-based algorithm for discovering clusters in large spatial databases with noise[C]. Proceedings of the Kdd, 1996.
[24]	Tan P N.Introduction to data mining[M]. Pearson Education India, 2006.
[25]	Jaccard P.The distribution of the flora in the alpine zone[J]. New phytologist, 1912,11(2):37-50. doi: 10.1111/j.1469-8137.1912.tb05611.x
[26]	Choi S S, Cha S H, Tappert C C.A survey of binary similarity and distance measures[J]. Journal of Systemics, Cybernetics and Informatics, 2010,8(1):43-48.
[27]	Sparck Jones K.A statistical interpretation of term specificity and its application in retrieval[J]. Journal of documentation, 1972,28(1):11-21.
[28]	Konstan J A.Introduction to recommender systems: Algorithms and evaluation[J]. ACM Transactions on Information Systems (TOIS), 2004,22(1):1-4. doi: 10.1145/963770.963771
[29]	Pedregosa F, Varoquaux G, Gramfort A, et al.Scikit-learn: Machine learning in Python[J]. Journal of Machine Learning Research, 2011,12(10):2825-2830. doi: 10.1524/auto.2011.0951

类别	详情
主机IP	128.227.49.92
时间	05/Aug/2014:10:26:42 +0800
方法	GET
URL	/extra/res/libs/kendo/extensions/kendo.extension.ui.js
协议	HTTP/1.1
状态	200
文件大小	15 072 Byte
访问来源	http://www.geodata.cn/extra/TopicsWin2/pro3.jsp
客户端	Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Firefox/31.0

类别	详情
URL	131.111.250.153,200,HTTP/1.1,-,-,"""-""", 2014.7.1, /Portal/mdsearch/regionresult.jsp?sw=%E5%8C%97%E4%BA%AC&sm=1&ps=10,…
搜索词编码	%E5%8C%97%E4%BA%AC
编码类型	utf-8
解码结果	北京

年份	原始日志记录/条	清洗后记录/条	用户数/个	会话数/个	活跃会话/个	搜索次数/次	搜索词数量/次
2011	10 062 608	2 664 473	62 557	219 918	54 121	76 793	4589
2012	9 546 068	2 394 507	76 098	234 585	55 726	82 914	3883
2013	10 584 125	2 708 978	82 302	264 906	58 237	110 056	5426
2014	11 062 608	2 845 150	78 111	348 495	68 562	111 913	6243
2015	12 236 056	2 914 507	89 937	365 752	70 969	122 868	6761

聚类编号	聚类会话数量	数据主题	主题占比/%
01	8355	土地覆被及土地利用	83.6
02	6712	土壤数据	78.4
03	6248	环境与灾害	84.3
04	5471	地形地貌	74.5
05	4676	冰川冻土	69.8

An Approach for Prediction of Web User Behavior and Data Recommendation for Geoscience Data Sharing Portals

RichHTML

PDF (PC)

Like

Knowledge

Cited

Abstract

Cite this article

share this article

Figures/Tables 10

References 29

Related Articles 9

Metrics

Comments

Recommended 0

[1]	YANG Yaping, JIANG Hou, SUN Jiulin. Practice of Scientific Data Sharing: A Case Study of the National Earth System Science Data Center [J]. Journal of Geo-information Science, 2020, 22(6): 1358-1369.
[2]	XU Zhen,JING Yaodong,BI Rutian,GAO Yang,WANG Peng. The Discovery of Spatial Association Patterns of Resource and Environment Information Based on Grid Data [J]. Journal of Geo-information Science, 2018, 20(1): 28-36.
[3]	WANG Mo,WANG Juanle. A Study on the User Behavior of Geoscience Data Sharing Based on Web Usage Mining [J]. Journal of Geo-information Science, 2016, 18(9): 1174-1183.
[4]	LIN Hui,ZHENG Xinqi. Data Mining and GIS Modeling of Urban Spatial Distribution Characteristics Based on Voronoi Diagram: Taking Shanxi Province as an Example [J]. Journal of Geo-information Science, 2015, 17(1): 62-68.
[5]	CHEN Jiangping, HUANG Bingjian. Application and Effects of Data Spatial Autocorrelation on Association Rule Mining [J]. , 2011, 13(1): 109-117.
[6]	QIN Kun, LI Zhenyu, DU Yi. Review of Spatial Data Mining Methods Based on Concept Analysis [J]. , 2009, 11(1): 10-17.
[7]	WANG Zhangang, ZHUANG Dafang, QIU Dongsheng, MING Tao. Application Analysis of Data Mining and Visualization in Forestry [J]. , 2007, 9(4): 19-22,141.
[8]	MEI Xin, NIU Zhenguo, WU Mengquan, YANG Xiaodong. Design of the Regional Circular Economy Decision Support System Based on Spatial Data Mining [J]. , 2007, 9(1): 88-92.
[9]	CHEN Chongcheng, TU Jiandong, HUANG Hongyu. The Study and Application on Visual Spatial Clustering [J]. , 2005, 7(2): 89-93.