面向遥感影像场景分类的类中心知识蒸馏方法

doi:10.12082/dqxxkx.2023.220781

摘要/Abstract

摘要：

卷积神经网络已广泛应用于遥感影像场景分类任务，然而优秀的模型体量大，无法部署到资源受限的边缘设备中，直接应用现有的知识蒸馏方法压缩模型，忽略了场景数据的类内多样性和类间相似性。为此，本文提出一种类中心知识蒸馏方法，旨在获得一个紧凑高效且精度高的遥感影像场景分类网络。首先对预训练的教师网络进行微调，然后基于设计的类中心蒸馏损失将教师网络强大的特征提取能力迁移到学生网络，通过约束师生网络提取的同类特征分布中心的距离完成知识的转移，同时在蒸馏过程中结合真值标签训练，最后学生网络单独用于预测。实验在4个数据集上与8种先进的蒸馏方法在不同训练比率、不同师生架构下进行了比较，本文方法均达到最高分类精度。其中，在训练比率为60%的RSC11、UCM、RSSCN7及AID数据集中，相比于性能最好的其他蒸馏方法，师生网络属同系列时分类总体精度分别提升了2.42%、2.74%、2.95%和1.07%。相似技术对比实验及可视化分析进一步证明了本文方法优异的性能。本文所提出的类中心知识蒸馏方法更好地传递了复杂网络所提取的类内紧凑、类间离散的特征知识，提高了轻量网络分类的性能。

关键词: 场景分类, 模型压缩, 知识蒸馏, 类中心, 再生核希尔伯特空间, 遥感, 深度学习, 卷积神经网络

Abstract:

Convolutional neural networks have been widely used in the task of Remote Sensing Image Scene Classification (RSISC) and have achieved extraordinary performance. However, these excellent models have large volume and high computational cost, which cannot be deployed to resource-constrained edge devices. Moreover, in the RSISC task, the existing knowledge distillation method is directly applied to the compression model, ignoring the intra-class diversity and inter-class similarity of scene data. To this end, we propose a novel class-centric knowledge distillation method, which aims to obtain a compact, efficient, and accurate network model for RSISC. The proposed class-centric knowledge distillation framework for remote sensing image scene classification consists of two streams, teacher network flow and student network flow. Firstly, the remote sensing image scene classification dataset is sent into the teacher network pre-trained on a large-scale dataset to fine-tune the parameters. Then, the class-centric knowledge of the hidden layer is extracted from the adjusted teacher network and transferred to the student network based on the designed class center distillation loss, which is realized by constraining the distance of the distribution center of similar features extracted by the teacher and student network, so that the student network can learn the powerful feature extraction ability of the teacher network. The distillation process is combined with the truth tag supervision. Finally, the trained student network is used for scene prediction from remote sensing images alone. To evaluate the proposed method, we design a comparison experiment with eight advanced distillation methods on classical remote sensing image scene classification with different training ratios and different teacher-student architectures. Our results show that: compared to the best performance of other distillation methods, in the case of the teacher-student network belonging to the same series, the overall classification accuracy of our proposed method is increased by 1.429% and 2.74%, respectively, with a given training ratio of 80% and 60%; and in the case of teacher-student networks belonging to different series, the classification accuracy is increased by 0.238% and 0.476%, respectively, with the two given ratios. Additionally, supplementary experiments are also carried out on a small data set of RSC11 with few classes and few samples, a multi-scale data set of RSSCN7 with few classes and multiple books, and a large complex data set of AID with many classes of heterogeneous samples. The results show that the proposed method has good generalization ability. Trough the comparison experiments with similar techniques, it is found that the proposed method can maintain excellent performance in challenging categories through confusion matrix, and the proposed distillation loss function can better deal with noise through testing error curve. And visualization analysis also shows that the proposed method can effectively deal with the problems of intra-class diversity and inter-class similarity in remote sensing image scenes.

Key words: scene classification, model compression, knowledge distillation, class center, Reproducing Kernel Hilbert Space, remote sensing, deep learning, convolutional neural network

刘潇, 刘智, 林雨准, 王淑香, 左溪冰. 面向遥感影像场景分类的类中心知识蒸馏方法[J]. 地球信息科学学报, 2023, 25(5): 1050-1063.DOI:10.12082/dqxxkx.2023.220781

LIU Xiao, LIU Zhi, LIN Yuzhun, WANG Shuxiang, ZUO Xibing. Class-centric Knowledge Distillation for RSI Scene Classification[J]. Journal of Geo-information Science, 2023, 25(5): 1050-1063.DOI:10.12082/dqxxkx.2023.220781

图/表 13

图1

算法1 类中心知识蒸馏算法
第一阶段：微调教师网络
输入：教师网络模型 $T$ ，预训练参数，训练样本 $X = x i, y i i = 1 N$
计算标准交叉熵损失函数 $H y t, y t r u e$
反向传播更新 $T$ 的参数，直到损失函数收敛
输出：教师网络模型 $T$
第二阶段:通过类中心蒸馏训练学生网络
输入：教师网络模型 $T$ ，学生网络模型 $S$ ，训练样本 $X = x i, y i i = 1 N$
初始化：学生网络参数 $θ$ 和超参数
按标签 $y$ 整理训练集 $X = x 1 i, · · ·, x n i, y i i = 1 M$ 每批次从随机类中抽取随机的样本
根据式（7）计算总体损失函数 $L t o t a l$
反向传播更新学生网络的参数 $θ$ ，直到损失函数收敛
输出：学生网络模型 $S$ ，参数 $θ$
第三阶段：测试学生网络
输入：学生网络模型 $S$ ，测试样本
输出：预测结果
按标签 $y$ 整理训练集 $X = x 1 i, · · ·, x n i, y i i = 1 M$ 每批次从随机类中抽取随机的样本

表1

图2

表2

师生网络结构及输出层特征信息

网络名称	ResNet50	ResNet18	MobileNetV2
卷积池化层	$7 × 7,64, s t r i d e 2 + 3 × 3 m a x p o o l, s t r i d e 2$	$3 × 3,32, s t r i d e 2$
卷积层1	$1 × 1,64 3 × 3,64 1 × 1,256 × 3$	$3 × 3,64 3 × 3,64 × 3$	$1 × 1,64 3 × 3,64 1 × 1,256 × 3$
输出层特征1		$F ∈ R 56 × 56 × 64$	$F ∈ R 56 × 56 × 256$
卷积层2	$1 × 1,128 3 × 3,128, s 2 1 × 1,512 + 1 × 1,128 3 × 3,128 1 × 1,512 × 3$	$3 × 3,128, s 2 3 × 3,128 + 3 × 3,128 3 × 3,128$	$1 × 1,128 3 × 3,128, s 2 1 × 1,512 + 1 × 1,128 3 × 3,128 1 × 1,512 × 3$
输出层特征2	$F ∈ R 28 × 28 × 512$	$F ∈ R 28 × 28 × 128$	$F ∈ R 28 × 28 × 512$
卷积层3	$1 × 1,256 3 × 3,256, s 2 1 × 1,1024 + 1 × 1,256 3 × 3,256 1 × 1,1024 × 3$	$3 × 3,256, s 2 3 × 3,256 + 3 × 3,256 3 × 3,256$	$1 × 1,256 3 × 3,256, s 2 1 × 1,1024 + 1 × 1,256 3 × 3,256 1 × 1,1024 × 3$
输出层特征3	$F ∈ R 14 × 14 × 1024$	$F ∈ R 14 × 14 × 256$	$F ∈ R 14 × 14 × 1024$
卷积层4	$1 × 1,, 512 3 × 3,512, s 2 1 × 1,2048 + 1 × 1,, 512 3 × 3,512 1 × 1,2048 × 3$	$3 × 3,512, s 2 3 × 3,512 + 3 × 3,512 3 × 3,512$	$1 × 1,, 512 3 × 3,512, s 2 1 × 1,2048 + 1 × 1,, 512 3 × 3,512 1 × 1,2048 × 3$
输出层特征4	$F ∈ R 7 × 7 × 2048$	$F ∈ R 7 × 7 × 512$	$F ∈ R 7 × 7 × 2048$
全局池化层		$7 × 7 a v e r a g e p o o l, f c, s o f t m a x$

表2

表3

表4

表5

表6

表7

图3

图4

图5

参考文献 48

[1]	Ghazouani F, Farah I R, Solaiman B. A multi-level semantic scene interpretation strategy for change interpretation in remote sensing imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(11):8775-8795. DOI:10.1109/TGRS.2019.2922908 doi: 10.1109/TGRS.2019.2922908
[2]	Hu F, Xia G S, Hu J W, et al. Transferring deep convolutional neural networks for the scene classification of high-resolution remote sensing imagery[J]. Remote Sensing, 2015, 7(11):14680-14707. DOI:10.3390/rs71114680 doi: 10.3390/rs71114680
[3]	Gu Y T, Wang Y T, Li Y S. A survey on deep learning-driven remote sensing image scene understanding: Scene classification, scene retrieval and scene-guided object detection[J]. Applied Sciences, 2019, 9(10):2110. DOI:10.3390/app9102110 doi: 10.3390/app9102110
[4]	Ojala T, Pietikainen M, Maenpaa T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(7):971-987. DOI: 10.1109/TPAMI.2002.1017623 doi: 10.1109/TPAMI.2002.1017623
[5]	Zhu Q Q, Zhong Y F, Zhao B, et al. Bag-of-visual-words scene classifier with local and global features for high spatial resolution remote sensing imagery[J]. IEEE Geoscience and Remote Sensing Letters, 2016, 13(6):747-751. DOI:10.1109/LGRS.2015.2513443 doi: 10.1109/LGRS.2015.2513443
[6]	Romero A, Gatta C, Camps-Valls G. Unsupervised deep feature extraction for remote sensing image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(3):1349-1362. DOI:10.1109/TGRS.2015.2478379 doi: 10.1109/TGRS.2015.2478379
[7]	Cheng G, Xie X X, Han J W, et al. Remote sensing image scene classification meets deep learning: Challenges, methods, benchmarks, and opportunities[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13:3735-3756. DOI:10.1109/JSTARS.2020.3005403 doi: 10.1109/JSTARS.2020.3005403
[8]	郭子慧, 刘伟. 深度学习和遥感影像支持的矢量图斑地类解译真实性检查方法[J]. 地球信息科学学报, 2020, 22(10):2051-2061. doi: 10.12082/dqxxkx.2020.200001
	[ Guo Z H, Liu W. Land type interpretation authenticity check of vector patch supported by deep learning and remote sensing image[J]. Journal of Geo-Information Science, 2020, 22(10):2051-2061. ] DOI:10.12082/dqxxkx.2020.200001 doi: 10.12082/dqxxkx.2020.200001
[9]	余东行, 张保明, 赵传, 等. 联合卷积神经网络与集成学习的遥感影像场景分类[J]. 遥感学报, 2020, 24(6):717-727.
	[ Yu D H, Zhang B M, Zhao C, et al. Scene classification of remote sensing image using ensemble convolutional neural network[J]. Journal of Remote Sensing, 2020, 24(6):717-727. ] DOI:10.11834/jrs.20208273 doi: 10.11834/jrs.20208273
[10]	Sun H M, Lin Y W, Zou Q, et al. Convolutional neural networks based remote sensing scene classification under clear and cloudy environments[C]// 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW). IEEE, 2021:713-720. DOI:10.1109/ICCVW54120.2021.00085 doi: 10.1109/ICCVW54120.2021.00085
[11]	Chen W T, Ouyang S B, Tong W, et al. GCSANet: A global context spatial attention deep learning network for remote sensing scene classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2022, 15:1150-1162. DOI:10.1109/JSTARS.2022.3141826 doi: 10.1109/JSTARS.2022.3141826
[12]	Wang Q, Huang W, Xiong Z T, et al. Looking closer at the scene: Multiscale representation learning for remote sensing image scene classification[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33(4):1414-1428. DOI:10.1109/TNNLS.2020.3042276 doi: 10.1109/TNNLS.2020.3042276
[13]	彭瑞, 赵文智, 张立强, 等. 基于多尺度对比学习的弱监督遥感场景分类[J]. 地球信息科学学报, 2022, 24(7):1375-1390. doi: 10.12082/dqxxkx.2022.210809
	[ Peng R, Zhao W Z, Zhang L Q, et al. Multi-scale contrastive learning based weakly supervised learning for remote sensing scene classification[J]. Journal of Geo-Information Science, 2022, 24(7):1375-1390. ] DOI:10.12082/dqxxkx.2022.210809 doi: 10.12082/dqxxkx.2022.210809
[14]	Zhang R, Chen Z H, Zhang S X, et al. Remote sensing image scene classification with noisy label distillation[J]. Remote Sensing, 2020, 12(15): 2376. DOI:10.3390/rs12 152376 doi: 10.3390/rs12010012
[15]	Zhang B, Zhang Y J, Wang S G. A lightweight and discriminative model for remote sensing scene classification with multidilation pooling module[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019, 12(8):2636-2653. DOI:10.1109/JSTARS.2019.2919317 doi: 10.1109/JSTARS.2019.2919317
[16]	Chen G Z, Zhang X D, Tan X L, et al. Training small networks for scene classification of remote sensing images via knowledge distillation[J]. Remote Sensing, 2018, 10(5):719. DOI:10.3390/rs10050719 doi: 10.3390/rs10050719
[17]	Zhao H R, Sun X, Gao F, et al. Pair-wise similarity knowledge distillation for RSI scene classification[J]. Remote Sensing, 2022, 14(10):2483. DOI: 10.3390/rs14102483 doi: 10.3390/rs14102483
[18]	Li H, Kadav A, Durdanovic I, et al. Pruning filters for efficient ConvNets[EB/OL]. 2016: arXiv: 1608.08710. https://arxiv.org/abs/1608.08710
[19]	Yang T J, Chen Y H, Sze V. Designing energy-efficient convolutional neural networks using energy-aware pruning[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2017:6071-6079. DOI:10.1109/CVPR.2017.643 doi: 10.1109/CVPR.2017.643
[20]	Gou J P, Yu B S, Maybank S J, et al. Knowledge distillation: A survey[J]. International Journal of Computer Vision, 2021, 129(6):1789-1819. DOI:10.1007/s11263-021-01453-z doi: 10.1007/s11263-021-01453-z
[21]	Buciluǎ C, Caruana R, Niculescu-Mizil A. Model compression[C]// Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining. New York: ACM, 2006:535-541. DOI:10.1145/1150402.1150464 doi: 10.1145/1150402.1150464
[22]	Ba J and Caruana R. Do deep nets really need to be deep?[C]. Advances in Neural Information Processing Systems, 2014, 27. DOI:10.48550/arXiv.1312.6184 doi: 10.48550/arXiv.1312.6184
[23]	Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network[EB/OL]. 2015: arXiv:1503.02531. https://arxiv.org/abs/1503.02531
[24]	Romero A, Ballas N, Kahou S E, et al. FitNets: Hints for thin deep nets[EB/OL]. 2014: arXiv:1412.6550. https://arxiv.org/abs/1412.6550
[25]	Zagoruyko S, Komodakis N. Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer[EB/OL]. 2016: arXiv: 1612.03928. https://arxiv.org/abs/1612.03928
[26]	Huang Z H, Wang N Y. Like what You like: Knowledge distill via neuron selectivity transfer[EB/OL]. 2017:arXiv: 1707.01219. https://arxiv.org/abs/1707.01219
[27]	Kim J, Park S, Kwak N. Paraphrasing complex network: Network compression via factor transfer[EB/OL]. 2018: arXiv: 1802.04977. https://arxiv.org/abs/1802.04977
[28]	Heo B, Kim J, Yun S, et al. A comprehensive overhaul of feature distillation[C]// 2019 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2020:1921- 1930. DOI:10.1109/ICCV.2019.00201 doi: 10.1109/ICCV.2019.00201
[29]	Ahn S, Hu S X, Damianou A, et al. Variational information distillation for knowledge transfer[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2020:9155-9163. DOI:10.1109/CVPR.2019.00938 doi: 10.1109/CVPR.2019.00938
[30]	Yim J, Joo D, Bae J, et al. A gift from knowledge distillation: Fast optimization, network minimization and transfer learning[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2017:7130-7138. DOI:10.1109/CVPR.2017.754 doi: 10.1109/CVPR.2017.754
[31]	Lee S H, Kim D H, Song B C. Self-supervised knowledge distillation using singular value decomposition[M]//Computer Vision - ECCV 2018. Cham: Springer International Publishing, 2018:339-354. DOI:10.1007/978-3-030-01231-1_21 doi: 10.1007/978-3-030-01231-1_21
[32]	Chen P G, Liu S, Zhao H S, et al. Distilling knowledge via knowledge review[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2021:5006-5015. DOI: 10.1109/CVPR46437.2021.00497 doi: 10.1109/CVPR46437.2021.00497
[33]	Passalis N, Tefas A. Learning deep representations with probabilistic knowledge transfer[M]//Computer Vision - ECCV 2018. Cham: Springer International Publishing, 2018:283-299. DOI:10.1007/978-3-030-01252-6_17 doi: 10.1007/978-3-030-01252-6_17
[34]	Tung F, Mori G. Similarity-preserving knowledge distillation[C]// 2019 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2020:1365-1374. DOI: 10.1109/ICCV.2019.00145 doi: 10.1109/ICCV.2019.00145
[35]	Park W, Kim D, Lu Y, et al. Relational knowledge distillation[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2020:3962-3971. DOI: 10.1109/CVPR.2019.00409 doi: 10.1109/CVPR.2019.00409
[36]	Zhao B R, Cui Q, Song R J, et al. Decoupled knowledge distillation[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022:11943-11952. DOI: 10.1109/CVPR52688.2022.01165 doi: 10.1109/CVPR52688.2022.01165
[37]	Wang L, Yoon K J. Knowledge distillation and student-teacher learning for visual intelligence: A review and new outlooks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(6):3048-3068. DOI:10.1109/TPAMI.2021.3055564 doi: 10.1109/TPAMI.2021.3055564
[38]	杨宏炳, 迟勇欣, 王金光. 基于剪枝网络的知识蒸馏对遥感卫星图像分类方法[J]. 计算机应用研究, 2021, 38(8):2469-2473.
	[ Yang H B, Chi Y X, Wang J G. Knowledge distillation method for remote sensing satellite image classification based on pruning network[J]. Application Research of Computers, 2021, 38(8):2469-2473. ] DOI: 10.19734/j.issn.1001-3695.2020.07.0387 doi: 10.19734/j.issn.1001-3695.2020.07.0387
[39]	Gretton A, Borgwardt K M, Rasch M J, et al. A kernel two-sample test[J]. Journal of Machine Learning Research, 2012, 13(25):723-773.
[40]	Sejdinovic D, Gretton A. What is an RKHS? Lecture Notes, 2012
[41]	Zhao L J, Tang P, Huo L Z. Feature significance-based multibag-of-visual-words model for remote sensing image scene classification[J]. Journal of Applied Remote Sensing, 2016, 10(3):035004. DOI:10.1117/1.JRS.10.035004 doi: 10.1117/1.JRS.10.035004
[42]	Zou Q, Ni L H, Zhang T, et al. Deep learning based feature selection for remote sensing scene classification[J]. IEEE Geoscience and Remote Sensing Letters, 2015, 12(11):2321-2325. DOI:10.1109/LGRS.2015.2475299 doi: 10.1109/LGRS.2015.2475299
[43]	Liu B D, Xie W Y, Meng J, et al. Hybrid collaborative representation for remote-sensing image scene classification[J]. Remote Sensing, 2018, 10(12): 1934. DOI:10.3390/rs10121934 doi: 10.3390/rs10121934
[44]	Xia G S, Hu J W, Hu F, et al. AID: A benchmark data set for performance evaluation of aerial scene classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(7):3965-3981. DOI:10.1109/TGRS.2017.2685945 doi: 10.1109/TGRS.2017.2685945
[45]	Xie S N, Girshick R, Dollár P, et al. Aggregated residual transformations for deep neural networks[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2017:5987-5995. DOI:10.1109/CVPR.2017.634 doi: 10.1109/CVPR.2017.634
[46]	Sandler M, Howard A, Zhu M L, et al. MobileNetV2: inverted residuals and linear bottlenecks[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 2018:4510-4520. DOI:10.1109/CVPR.2018.00474 doi: 10.1109/CVPR.2018.00474
[47]	Maaten L van der and Hinton G. Visualizing data using t-sne[J]. Journal of Machine Learning Research, 2008, 9(86):2579-2605.
[48]	Selvaraju R R, Cogswell M, Das A, et al. Grad-CAM: Visual explanations from deep networks via gradient-based localization[C]// 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, 2017:618-626. DOI: 10.1109/ICCV.2017.74 doi: 10.1109/ICCV.2017.74

数据集	分辨率/m	类别数/个	尺寸/mm	每类样本数/个	样本总数/个	特点
RSC11	0.2	11	512 × 512	约100	1232	小型的遥感影像场景分类数据集
UCM	0.3	21	256 × 256	100	2100	经典的高分辨率土地利用数据集
RSSCN7	-	7	400 × 400	400	2800	涵盖4个采样尺度的，类内多样性大的场景分类数据集
AID	0.5~0.8	30	600 × 600	200~400	10000	复杂的多源、多分辨率、类间相似性高、样本不均衡的航空图像数据集

蒸馏方法	师生架构（Model T/S）	同系列（ResNet-50/ ResNet-18）		不同系列（ResNet-50/MobileNet-V2）
	训练比率	80%	60%	80%	60%
	Baseline	92.14	90.00	91.43	90.48
响应	KD	95.48	92.38	93.33	90.24
响应	DKD	95.71	91.91	94.52	92.14
特征	NST	94.05	91.67	92.62	90.48
特征	VID	93.57	89.41	92.38	89.05
网络层间关系	KDSVD	92.62	90.36	92.62	89.88
网络层间关系	ReviewKD	94.29	92.62	94.29	91.07
实例关系	RKD	94.52	92.02	92.38	87.38
实例关系	SP	93.33	91.43	80.48	59.05
类中心	本文方法	97.14	95.36	94.76	92.62

模型	FLOPs (G)	Parameters (M)	压缩率/%
ResNet50	32.88	23.55
ResNet18	14.55	11.19	47.52
MobileNetV2	2.50	2.25	9.55

数据集	教师网络	学生网络	响应		特征		网络层间关系		实例关系		类中心
数据集	教师网络	学生网络	KD	DKD	NST	VID	KDSVD	ReviewKD	RKD	SP	本文方法
RS_C11	92.35	90.14	91.95	90.95	89.74	88.13	87.73	90.34	89.74	87.73	94.37
RSSCN7	91.07	88.75	88.30	87.32	87.59	87.41	87.05	88.21	88.75	86.52	91.70
AID	95.68	88.98	92.73	93.03	91.43	88.96	89.20	92.58	91.45	91.65	94.10

单独训练的学生网络												微调的教师网络
	密林	草地	港口	高建筑	低建筑	立交	铁路	居民区	公路	疏林	储存罐		密林	草地	港口	高建筑	低建筑	立交	铁路	居民区	公路	疏林	储存罐
密林	98.21										1.79	密林	100
草地		100										草地		100
港口			97.22	2.78								港口			100
高建筑				91.11		2.22		2.22			4.44	高建筑				97.62				2.38
低建筑					85.42	6.25					8.33	地建筑					91.67	4.17	2.08		2.08
立交					2.33	74.42	2.33		18.6		2.33	立交				2.27		68.18	9.09		20.45
铁路				3.12		3.12	87.5		6.25			铁路						8.00	88.00		4.00
居民区						1.61	1.61	87.1	4.84		4.84	居民区						1.79		98.21
公路			1.89	1.89		9.43	3.77		83.02			公路				3.28		13.11	8.20		75.41
疏林	2.17									97.83		疏林										100
储存罐					5.71			2.86			91.43	储存罐											100
基于改进NST方法训练的学生网络												基于本文方法训练的学生网络
	密林	草地	港口	高建筑	低建筑	立交	铁路	居民区	公路	疏林	储存罐		密林	草地	港口	高建筑	低建筑	立交	铁路	居民区	公路	疏林	储存罐
密林	100											密林	100
草地		100										草地		100
港口			92.31	2.56					5.13			港口			100
高建筑				90.91	4.55			2.27			2.27	高建筑				97.73	2.27
地建筑				2.50	95.00	2.5						低建筑					95.24	2.38					2.38
立交						89.19	5.41		5.41			立交						86.11	5.56	2.78	5.56
铁路				3.12	3.12		87.5		3.12		3.12	铁路				3.33	6.67		90.00
居民区				1.67				91.67	3.33		3.33	居民区			1.69			1.69	1.69	93.22	1.69
公路						14.75	3.28		81.97			公路						15.15	3.03		81.82
疏林	2.17									97.83		疏林										100
储存罐					7.14						92.86	储存罐					2.33						97.67