面向遥感影像场景分类的类中心知识蒸馏方法

doi:10.12082/dqxxkx.2023.220781

Abstract

Abstract:

Convolutional neural networks have been widely used in the task of Remote Sensing Image Scene Classification (RSISC) and have achieved extraordinary performance. However, these excellent models have large volume and high computational cost, which cannot be deployed to resource-constrained edge devices. Moreover, in the RSISC task, the existing knowledge distillation method is directly applied to the compression model, ignoring the intra-class diversity and inter-class similarity of scene data. To this end, we propose a novel class-centric knowledge distillation method, which aims to obtain a compact, efficient, and accurate network model for RSISC. The proposed class-centric knowledge distillation framework for remote sensing image scene classification consists of two streams, teacher network flow and student network flow. Firstly, the remote sensing image scene classification dataset is sent into the teacher network pre-trained on a large-scale dataset to fine-tune the parameters. Then, the class-centric knowledge of the hidden layer is extracted from the adjusted teacher network and transferred to the student network based on the designed class center distillation loss, which is realized by constraining the distance of the distribution center of similar features extracted by the teacher and student network, so that the student network can learn the powerful feature extraction ability of the teacher network. The distillation process is combined with the truth tag supervision. Finally, the trained student network is used for scene prediction from remote sensing images alone. To evaluate the proposed method, we design a comparison experiment with eight advanced distillation methods on classical remote sensing image scene classification with different training ratios and different teacher-student architectures. Our results show that: compared to the best performance of other distillation methods, in the case of the teacher-student network belonging to the same series, the overall classification accuracy of our proposed method is increased by 1.429% and 2.74%, respectively, with a given training ratio of 80% and 60%; and in the case of teacher-student networks belonging to different series, the classification accuracy is increased by 0.238% and 0.476%, respectively, with the two given ratios. Additionally, supplementary experiments are also carried out on a small data set of RSC11 with few classes and few samples, a multi-scale data set of RSSCN7 with few classes and multiple books, and a large complex data set of AID with many classes of heterogeneous samples. The results show that the proposed method has good generalization ability. Trough the comparison experiments with similar techniques, it is found that the proposed method can maintain excellent performance in challenging categories through confusion matrix, and the proposed distillation loss function can better deal with noise through testing error curve. And visualization analysis also shows that the proposed method can effectively deal with the problems of intra-class diversity and inter-class similarity in remote sensing image scenes.

Key words: scene classification, model compression, knowledge distillation, class center, Reproducing Kernel Hilbert Space, remote sensing, deep learning, convolutional neural network

LIU Xiao, LIU Zhi, LIN Yuzhun, WANG Shuxiang, ZUO Xibing. Class-centric Knowledge Distillation for RSI Scene Classification[J].Journal of Geo-information Science, 2023, 25(5): 1050-1063.DOI:10.12082/dqxxkx.2023.220781

Figures/Tables 13

Fig. 1

算法1 类中心知识蒸馏算法
第一阶段：微调教师网络
输入：教师网络模型 $T$ ，预训练参数，训练样本 $X = x i, y i i = 1 N$
计算标准交叉熵损失函数 $H y t, y t r u e$
反向传播更新 $T$ 的参数，直到损失函数收敛
输出：教师网络模型 $T$
第二阶段:通过类中心蒸馏训练学生网络
输入：教师网络模型 $T$ ，学生网络模型 $S$ ，训练样本 $X = x i, y i i = 1 N$
初始化：学生网络参数 $θ$ 和超参数
按标签 $y$ 整理训练集 $X = x 1 i, · · ·, x n i, y i i = 1 M$ 每批次从随机类中抽取随机的样本
根据式（7）计算总体损失函数 $L t o t a l$
反向传播更新学生网络的参数 $θ$ ，直到损失函数收敛
输出：学生网络模型 $S$ ，参数 $θ$
第三阶段：测试学生网络
输入：学生网络模型 $S$ ，测试样本
输出：预测结果
按标签 $y$ 整理训练集 $X = x 1 i, · · ·, x n i, y i i = 1 M$ 每批次从随机类中抽取随机的样本

Tab. 1

Fig. 2

Tab. 2

Network structure of T/S models and Information about the features of the output layer

网络名称	ResNet50	ResNet18	MobileNetV2
卷积池化层	$7 × 7,64, s t r i d e 2 + 3 × 3 m a x p o o l, s t r i d e 2$	$3 × 3,32, s t r i d e 2$
卷积层1	$1 × 1,64 3 × 3,64 1 × 1,256 × 3$	$3 × 3,64 3 × 3,64 × 3$	$1 × 1,64 3 × 3,64 1 × 1,256 × 3$
输出层特征1		$F ∈ R 56 × 56 × 64$	$F ∈ R 56 × 56 × 256$
卷积层2	$1 × 1,128 3 × 3,128, s 2 1 × 1,512 + 1 × 1,128 3 × 3,128 1 × 1,512 × 3$	$3 × 3,128, s 2 3 × 3,128 + 3 × 3,128 3 × 3,128$	$1 × 1,128 3 × 3,128, s 2 1 × 1,512 + 1 × 1,128 3 × 3,128 1 × 1,512 × 3$
输出层特征2	$F ∈ R 28 × 28 × 512$	$F ∈ R 28 × 28 × 128$	$F ∈ R 28 × 28 × 512$
卷积层3	$1 × 1,256 3 × 3,256, s 2 1 × 1,1024 + 1 × 1,256 3 × 3,256 1 × 1,1024 × 3$	$3 × 3,256, s 2 3 × 3,256 + 3 × 3,256 3 × 3,256$	$1 × 1,256 3 × 3,256, s 2 1 × 1,1024 + 1 × 1,256 3 × 3,256 1 × 1,1024 × 3$
输出层特征3	$F ∈ R 14 × 14 × 1024$	$F ∈ R 14 × 14 × 256$	$F ∈ R 14 × 14 × 1024$
卷积层4	$1 × 1,, 512 3 × 3,512, s 2 1 × 1,2048 + 1 × 1,, 512 3 × 3,512 1 × 1,2048 × 3$	$3 × 3,512, s 2 3 × 3,512 + 3 × 3,512 3 × 3,512$	$1 × 1,, 512 3 × 3,512, s 2 1 × 1,2048 + 1 × 1,, 512 3 × 3,512 1 × 1,2048 × 3$
输出层特征4	$F ∈ R 7 × 7 × 2048$	$F ∈ R 7 × 7 × 512$	$F ∈ R 7 × 7 × 2048$
全局池化层		$7 × 7 a v e r a g e p o o l, f c, s o f t m a x$

Tab. 2

Tab. 3

Tab. 4

Tab. 5

Tab.6

Tab. 7

Accuracy confusion matrix of comparison experiment on RSC11 Dataset (%)

单独训练的学生网络												微调的教师网络
	密林	草地	港口	高建筑	低建筑	立交	铁路	居民区	公路	疏林	储存罐		密林	草地	港口	高建筑	低建筑	立交	铁路	居民区	公路	疏林	储存罐
密林	98.21										1.79	密林	100
草地		100										草地		100
港口			97.22	2.78								港口			100
高建筑				91.11		2.22		2.22			4.44	高建筑				97.62				2.38
低建筑					85.42	6.25					8.33	地建筑					91.67	4.17	2.08		2.08
立交					2.33	74.42	2.33		18.6		2.33	立交				2.27		68.18	9.09		20.45
铁路				3.12		3.12	87.5		6.25			铁路						8.00	88.00		4.00
居民区						1.61	1.61	87.1	4.84		4.84	居民区						1.79		98.21
公路			1.89	1.89		9.43	3.77		83.02			公路				3.28		13.11	8.20		75.41
疏林	2.17									97.83		疏林										100
储存罐					5.71			2.86			91.43	储存罐											100
基于改进NST方法训练的学生网络												基于本文方法训练的学生网络
	密林	草地	港口	高建筑	低建筑	立交	铁路	居民区	公路	疏林	储存罐		密林	草地	港口	高建筑	低建筑	立交	铁路	居民区	公路	疏林	储存罐
密林	100											密林	100
草地		100										草地		100
港口			92.31	2.56					5.13			港口			100
高建筑				90.91	4.55			2.27			2.27	高建筑				97.73	2.27
地建筑				2.50	95.00	2.5						低建筑					95.24	2.38					2.38
立交						89.19	5.41		5.41			立交						86.11	5.56	2.78	5.56
铁路				3.12	3.12		87.5		3.12		3.12	铁路				3.33	6.67		90.00
居民区				1.67				91.67	3.33		3.33	居民区			1.69			1.69	1.69	93.22	1.69
公路						14.75	3.28		81.97			公路						15.15	3.03		81.82
疏林	2.17									97.83		疏林										100
储存罐					7.14						92.86	储存罐					2.33						97.67

Tab. 7

Fig. 3

Fig. 4

Fig. 5

References 48

[1]	Ghazouani F, Farah I R, Solaiman B. A multi-level semantic scene interpretation strategy for change interpretation in remote sensing imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(11):8775-8795. DOI:10.1109/TGRS.2019.2922908 doi: 10.1109/TGRS.2019.2922908
[2]	Hu F, Xia G S, Hu J W, et al. Transferring deep convolutional neural networks for the scene classification of high-resolution remote sensing imagery[J]. Remote Sensing, 2015, 7(11):14680-14707. DOI:10.3390/rs71114680 doi: 10.3390/rs71114680
[3]	Gu Y T, Wang Y T, Li Y S. A survey on deep learning-driven remote sensing image scene understanding: Scene classification, scene retrieval and scene-guided object detection[J]. Applied Sciences, 2019, 9(10):2110. DOI:10.3390/app9102110 doi: 10.3390/app9102110
[4]	Ojala T, Pietikainen M, Maenpaa T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(7):971-987. DOI: 10.1109/TPAMI.2002.1017623 doi: 10.1109/TPAMI.2002.1017623
[5]	Zhu Q Q, Zhong Y F, Zhao B, et al. Bag-of-visual-words scene classifier with local and global features for high spatial resolution remote sensing imagery[J]. IEEE Geoscience and Remote Sensing Letters, 2016, 13(6):747-751. DOI:10.1109/LGRS.2015.2513443 doi: 10.1109/LGRS.2015.2513443
[6]	Romero A, Gatta C, Camps-Valls G. Unsupervised deep feature extraction for remote sensing image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(3):1349-1362. DOI:10.1109/TGRS.2015.2478379 doi: 10.1109/TGRS.2015.2478379
[7]	Cheng G, Xie X X, Han J W, et al. Remote sensing image scene classification meets deep learning: Challenges, methods, benchmarks, and opportunities[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13:3735-3756. DOI:10.1109/JSTARS.2020.3005403 doi: 10.1109/JSTARS.2020.3005403
[8]	郭子慧, 刘伟. 深度学习和遥感影像支持的矢量图斑地类解译真实性检查方法[J]. 地球信息科学学报, 2020, 22(10):2051-2061. doi: 10.12082/dqxxkx.2020.200001
	[ Guo Z H, Liu W. Land type interpretation authenticity check of vector patch supported by deep learning and remote sensing image[J]. Journal of Geo-Information Science, 2020, 22(10):2051-2061. ] DOI:10.12082/dqxxkx.2020.200001 doi: 10.12082/dqxxkx.2020.200001
[9]	余东行, 张保明, 赵传, 等. 联合卷积神经网络与集成学习的遥感影像场景分类[J]. 遥感学报, 2020, 24(6):717-727.
	[ Yu D H, Zhang B M, Zhao C, et al. Scene classification of remote sensing image using ensemble convolutional neural network[J]. Journal of Remote Sensing, 2020, 24(6):717-727. ] DOI:10.11834/jrs.20208273 doi: 10.11834/jrs.20208273
[10]	Sun H M, Lin Y W, Zou Q, et al. Convolutional neural networks based remote sensing scene classification under clear and cloudy environments[C]// 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW). IEEE, 2021:713-720. DOI:10.1109/ICCVW54120.2021.00085 doi: 10.1109/ICCVW54120.2021.00085
[11]	Chen W T, Ouyang S B, Tong W, et al. GCSANet: A global context spatial attention deep learning network for remote sensing scene classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2022, 15:1150-1162. DOI:10.1109/JSTARS.2022.3141826 doi: 10.1109/JSTARS.2022.3141826
[12]	Wang Q, Huang W, Xiong Z T, et al. Looking closer at the scene: Multiscale representation learning for remote sensing image scene classification[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33(4):1414-1428. DOI:10.1109/TNNLS.2020.3042276 doi: 10.1109/TNNLS.2020.3042276
[13]	彭瑞, 赵文智, 张立强, 等. 基于多尺度对比学习的弱监督遥感场景分类[J]. 地球信息科学学报, 2022, 24(7):1375-1390. doi: 10.12082/dqxxkx.2022.210809
	[ Peng R, Zhao W Z, Zhang L Q, et al. Multi-scale contrastive learning based weakly supervised learning for remote sensing scene classification[J]. Journal of Geo-Information Science, 2022, 24(7):1375-1390. ] DOI:10.12082/dqxxkx.2022.210809 doi: 10.12082/dqxxkx.2022.210809
[14]	Zhang R, Chen Z H, Zhang S X, et al. Remote sensing image scene classification with noisy label distillation[J]. Remote Sensing, 2020, 12(15): 2376. DOI:10.3390/rs12 152376 doi: 10.3390/rs12010012
[15]	Zhang B, Zhang Y J, Wang S G. A lightweight and discriminative model for remote sensing scene classification with multidilation pooling module[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019, 12(8):2636-2653. DOI:10.1109/JSTARS.2019.2919317 doi: 10.1109/JSTARS.2019.2919317
[16]	Chen G Z, Zhang X D, Tan X L, et al. Training small networks for scene classification of remote sensing images via knowledge distillation[J]. Remote Sensing, 2018, 10(5):719. DOI:10.3390/rs10050719 doi: 10.3390/rs10050719
[17]	Zhao H R, Sun X, Gao F, et al. Pair-wise similarity knowledge distillation for RSI scene classification[J]. Remote Sensing, 2022, 14(10):2483. DOI: 10.3390/rs14102483 doi: 10.3390/rs14102483
[18]	Li H, Kadav A, Durdanovic I, et al. Pruning filters for efficient ConvNets[EB/OL]. 2016: arXiv: 1608.08710. https://arxiv.org/abs/1608.08710
[19]	Yang T J, Chen Y H, Sze V. Designing energy-efficient convolutional neural networks using energy-aware pruning[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2017:6071-6079. DOI:10.1109/CVPR.2017.643 doi: 10.1109/CVPR.2017.643
[20]	Gou J P, Yu B S, Maybank S J, et al. Knowledge distillation: A survey[J]. International Journal of Computer Vision, 2021, 129(6):1789-1819. DOI:10.1007/s11263-021-01453-z doi: 10.1007/s11263-021-01453-z
[21]	Buciluǎ C, Caruana R, Niculescu-Mizil A. Model compression[C]// Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining. New York: ACM, 2006:535-541. DOI:10.1145/1150402.1150464 doi: 10.1145/1150402.1150464
[22]	Ba J and Caruana R. Do deep nets really need to be deep?[C]. Advances in Neural Information Processing Systems, 2014, 27. DOI:10.48550/arXiv.1312.6184 doi: 10.48550/arXiv.1312.6184
[23]	Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network[EB/OL]. 2015: arXiv:1503.02531. https://arxiv.org/abs/1503.02531
[24]	Romero A, Ballas N, Kahou S E, et al. FitNets: Hints for thin deep nets[EB/OL]. 2014: arXiv:1412.6550. https://arxiv.org/abs/1412.6550
[25]	Zagoruyko S, Komodakis N. Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer[EB/OL]. 2016: arXiv: 1612.03928. https://arxiv.org/abs/1612.03928
[26]	Huang Z H, Wang N Y. Like what You like: Knowledge distill via neuron selectivity transfer[EB/OL]. 2017:arXiv: 1707.01219. https://arxiv.org/abs/1707.01219
[27]	Kim J, Park S, Kwak N. Paraphrasing complex network: Network compression via factor transfer[EB/OL]. 2018: arXiv: 1802.04977. https://arxiv.org/abs/1802.04977
[28]	Heo B, Kim J, Yun S, et al. A comprehensive overhaul of feature distillation[C]// 2019 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2020:1921- 1930. DOI:10.1109/ICCV.2019.00201 doi: 10.1109/ICCV.2019.00201
[29]	Ahn S, Hu S X, Damianou A, et al. Variational information distillation for knowledge transfer[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2020:9155-9163. DOI:10.1109/CVPR.2019.00938 doi: 10.1109/CVPR.2019.00938
[30]	Yim J, Joo D, Bae J, et al. A gift from knowledge distillation: Fast optimization, network minimization and transfer learning[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2017:7130-7138. DOI:10.1109/CVPR.2017.754 doi: 10.1109/CVPR.2017.754
[31]	Lee S H, Kim D H, Song B C. Self-supervised knowledge distillation using singular value decomposition[M]//Computer Vision - ECCV 2018. Cham: Springer International Publishing, 2018:339-354. DOI:10.1007/978-3-030-01231-1_21 doi: 10.1007/978-3-030-01231-1_21
[32]	Chen P G, Liu S, Zhao H S, et al. Distilling knowledge via knowledge review[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2021:5006-5015. DOI: 10.1109/CVPR46437.2021.00497 doi: 10.1109/CVPR46437.2021.00497
[33]	Passalis N, Tefas A. Learning deep representations with probabilistic knowledge transfer[M]//Computer Vision - ECCV 2018. Cham: Springer International Publishing, 2018:283-299. DOI:10.1007/978-3-030-01252-6_17 doi: 10.1007/978-3-030-01252-6_17
[34]	Tung F, Mori G. Similarity-preserving knowledge distillation[C]// 2019 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2020:1365-1374. DOI: 10.1109/ICCV.2019.00145 doi: 10.1109/ICCV.2019.00145
[35]	Park W, Kim D, Lu Y, et al. Relational knowledge distillation[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2020:3962-3971. DOI: 10.1109/CVPR.2019.00409 doi: 10.1109/CVPR.2019.00409
[36]	Zhao B R, Cui Q, Song R J, et al. Decoupled knowledge distillation[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022:11943-11952. DOI: 10.1109/CVPR52688.2022.01165 doi: 10.1109/CVPR52688.2022.01165
[37]	Wang L, Yoon K J. Knowledge distillation and student-teacher learning for visual intelligence: A review and new outlooks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(6):3048-3068. DOI:10.1109/TPAMI.2021.3055564 doi: 10.1109/TPAMI.2021.3055564
[38]	杨宏炳, 迟勇欣, 王金光. 基于剪枝网络的知识蒸馏对遥感卫星图像分类方法[J]. 计算机应用研究, 2021, 38(8):2469-2473.
	[ Yang H B, Chi Y X, Wang J G. Knowledge distillation method for remote sensing satellite image classification based on pruning network[J]. Application Research of Computers, 2021, 38(8):2469-2473. ] DOI: 10.19734/j.issn.1001-3695.2020.07.0387 doi: 10.19734/j.issn.1001-3695.2020.07.0387
[39]	Gretton A, Borgwardt K M, Rasch M J, et al. A kernel two-sample test[J]. Journal of Machine Learning Research, 2012, 13(25):723-773.
[40]	Sejdinovic D, Gretton A. What is an RKHS? Lecture Notes, 2012
[41]	Zhao L J, Tang P, Huo L Z. Feature significance-based multibag-of-visual-words model for remote sensing image scene classification[J]. Journal of Applied Remote Sensing, 2016, 10(3):035004. DOI:10.1117/1.JRS.10.035004 doi: 10.1117/1.JRS.10.035004
[42]	Zou Q, Ni L H, Zhang T, et al. Deep learning based feature selection for remote sensing scene classification[J]. IEEE Geoscience and Remote Sensing Letters, 2015, 12(11):2321-2325. DOI:10.1109/LGRS.2015.2475299 doi: 10.1109/LGRS.2015.2475299
[43]	Liu B D, Xie W Y, Meng J, et al. Hybrid collaborative representation for remote-sensing image scene classification[J]. Remote Sensing, 2018, 10(12): 1934. DOI:10.3390/rs10121934 doi: 10.3390/rs10121934
[44]	Xia G S, Hu J W, Hu F, et al. AID: A benchmark data set for performance evaluation of aerial scene classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(7):3965-3981. DOI:10.1109/TGRS.2017.2685945 doi: 10.1109/TGRS.2017.2685945
[45]	Xie S N, Girshick R, Dollár P, et al. Aggregated residual transformations for deep neural networks[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2017:5987-5995. DOI:10.1109/CVPR.2017.634 doi: 10.1109/CVPR.2017.634
[46]	Sandler M, Howard A, Zhu M L, et al. MobileNetV2: inverted residuals and linear bottlenecks[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 2018:4510-4520. DOI:10.1109/CVPR.2018.00474 doi: 10.1109/CVPR.2018.00474
[47]	Maaten L van der and Hinton G. Visualizing data using t-sne[J]. Journal of Machine Learning Research, 2008, 9(86):2579-2605.
[48]	Selvaraju R R, Cogswell M, Das A, et al. Grad-CAM: Visual explanations from deep networks via gradient-based localization[C]// 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, 2017:618-626. DOI: 10.1109/ICCV.2017.74 doi: 10.1109/ICCV.2017.74

数据集	分辨率/m	类别数/个	尺寸/mm	每类样本数/个	样本总数/个	特点
RSC11	0.2	11	512 × 512	约100	1232	小型的遥感影像场景分类数据集
UCM	0.3	21	256 × 256	100	2100	经典的高分辨率土地利用数据集
RSSCN7	-	7	400 × 400	400	2800	涵盖4个采样尺度的，类内多样性大的场景分类数据集
AID	0.5~0.8	30	600 × 600	200~400	10000	复杂的多源、多分辨率、类间相似性高、样本不均衡的航空图像数据集

蒸馏方法	师生架构（Model T/S）	同系列（ResNet-50/ ResNet-18）		不同系列（ResNet-50/MobileNet-V2）
	训练比率	80%	60%	80%	60%
	Baseline	92.14	90.00	91.43	90.48
响应	KD	95.48	92.38	93.33	90.24
响应	DKD	95.71	91.91	94.52	92.14
特征	NST	94.05	91.67	92.62	90.48
特征	VID	93.57	89.41	92.38	89.05
网络层间关系	KDSVD	92.62	90.36	92.62	89.88
网络层间关系	ReviewKD	94.29	92.62	94.29	91.07
实例关系	RKD	94.52	92.02	92.38	87.38
实例关系	SP	93.33	91.43	80.48	59.05
类中心	本文方法	97.14	95.36	94.76	92.62

模型	FLOPs (G)	Parameters (M)	压缩率/%
ResNet50	32.88	23.55
ResNet18	14.55	11.19	47.52
MobileNetV2	2.50	2.25	9.55

数据集	教师网络	学生网络	响应		特征		网络层间关系		实例关系		类中心
数据集	教师网络	学生网络	KD	DKD	NST	VID	KDSVD	ReviewKD	RKD	SP	本文方法
RS_C11	92.35	90.14	91.95	90.95	89.74	88.13	87.73	90.34	89.74	87.73	94.37
RSSCN7	91.07	88.75	88.30	87.32	87.59	87.41	87.05	88.21	88.75	86.52	91.70
AID	95.68	88.98	92.73	93.03	91.43	88.96	89.20	92.58	91.45	91.65	94.10

Class-centric Knowledge Distillation for RSI Scene Classification

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 13

References 48

Related Articles 15

Metrics

Comments

Recommended 0

[1]	FAN Lanxin, WU Yanhong, CHI Haojing, ZHENG Siqi, YAN Jiaheng, REN Yongkang, SUN Zhonghua. Detecting Spatiotemporal Changes of Freshwater in Northwest China under a Warm-Wetting Climate using Remote Sensing [J]. Journal of Geo-information Science, 2023, 25(9): 1842-1854.
[2]	XING Ziyao, DONG Xinrui, ZAN Xuli, YANG Shuai, HUANG Zihan, LIU Zhe, ZHANG Xiaodong. Flood Inundation Mapping and Estimation using VGI, Remote Sensing Images and Other Multi-source Data [J]. Journal of Geo-information Science, 2023, 25(9): 1869-1881.
[3]	ZHANG Junyao, YANG Xiaomei, WANG Zhihua, YANG Haikun, ZHANG Bochun, WAN Qing, LEI Mei. Analysis of Pattern Evolution of Coal Mine Areas in Inner Mongolia from the Perspective of the Green Development Concept [J]. Journal of Geo-information Science, 2023, 25(8): 1655-1668.
[4]	CAO Yu, FANG Xiuqin, YANG Lulu, JIANG Xinyuan, LIAO Meiyu, REN Liliang. Downscaling of CCI Soil Moisture in the Xiliaohe River Basin based on Random Forest [J]. Journal of Geo-information Science, 2023, 25(8): 1669-1681.
[5]	ZHANG Hanbo, LI Tong, LI Xiaofang, DENG Ying, DENG Yingbin, JING Wenlong, HU Yiqiang, LI Yong, YANG Ji. A Short-term Prediction Method of Water Bloom based on UAV Remote Sensing and GWR [J]. Journal of Geo-information Science, 2023, 25(8): 1682-1698.
[6]	ZUO Xibing, LIU Zhi, JIN Fei, LIN Yuzhun, WANG Shuxiang, LIU Xiao, LI Meilin. Global-local Feature Adaptive Fusion Method for Small Sample Classification of Hyperspectral Images [J]. Journal of Geo-information Science, 2023, 25(8): 1699-1716.
[7]	LIN Na, HE Jing, WANG Bin, TANG Feifei, ZHOU Junyu, GUO Jiang. Intelligent Extraction of Urban Vegetation Information based on Vegetation Spectral Signature and Sep-UNet [J]. Journal of Geo-information Science, 2023, 25(8): 1717-1729.
[8]	ZHANG Chunju, LIU Wencong, ZHANG Xueying, YE Peng, WANG Chen, ZHU Shaonan, ZHANG Dayu. Knowledge Graph Construction Method of Gold Mine based on Ontology [J]. Journal of Geo-information Science, 2023, 25(7): 1269-1281.
[9]	ZHANG Tong, LIU Renyu, WANG Peixiao, GAO Chulin, LIU Jie, WANG Wangshu. Physics-informed Machine Learning and Its Research Prospects in GeoAI [J]. Journal of Geo-information Science, 2023, 25(7): 1297-1311.
[10]	WU Min, ZHANG Mingda, LI Panpan, ZHANG Yongjian. Research on Provenance Model for Multi-source Remote Sensing Images [J]. Journal of Geo-information Science, 2023, 25(7): 1325-1335.
[11]	LING Zhenfei, LIU Tao, DU Ping, ZHAO Dan, CHEN Puyi, MA Tianen. A Variational Graph Convolutional Auto-encoder Model Supporting Similarity Retrieval of Building Groups Patterns [J]. Journal of Geo-information Science, 2023, 25(7): 1405-1417.
[12]	ZHAO Dan, DU Ping, LIU Tao, LING Zhenfei. Spatio-temporal Distribution Prediction Model of Urban Theft by Fusing Graph Autoencoder and GRU [J]. Journal of Geo-information Science, 2023, 25(7): 1448-1463.
[13]	JIN Guangyin, SHA Hengyu, ZHANG Jinlei, HUANG Jincai. Travel Time Estimation Method based on Dual Graph Convolutional Networks via Joint Modeling of Road Segments and Intersections [J]. Journal of Geo-information Science, 2023, 25(7): 1500-1513.
[14]	YANG Yingpin, WU Zhifeng, HUANG Qiting, LUO Jiancheng, WU Tianjun, DONG Wen, HU Xiaodong, XIAO Wenju. Extraction of Sugarcane Plantation Distribution in Western Guangdong by Combining Remote Sensing and Statistic Data and Its Spatiotemporal Analysis [J]. Journal of Geo-information Science, 2023, 25(5): 1012-1026.
[15]	HOU Huitai, LAN Chaozhen, XU Qing. UAV Absolute Positioning Method based on Global and Local Deep Learning Feature Retrieval from Satellite Images [J]. Journal of Geo-information Science, 2023, 25(5): 1064-1074.