引入特征交互的红外与可见光图像自适应融合
作者:
作者单位:

作者简介:

通讯作者:

中图分类号:

基金项目:

江苏省产业前瞻与关键核心技术-碳达峰碳中和科技创新专项资金项目(No.BE2022044)资助。


Adaptive fusion of infrared and visible light images withintroduction of feature interaction
Author:
Affiliation:

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    在图像融合领域,现有的基于卷积神经网络(CNN)或Transformer架构的方法存在两个局限性:首先,浅层纹理特征与深层语义特征之间无法有效聚合;其次,红外与可见光特征的权重比例无法自适应变化。本文提出一种引入特征交互的红外与可见光图像自适应融合方法。首先,构建一种基于Transformer的特征交互模块,聚合跨尺度特征信息,增强特征表达能力。其次,设计一种融合模块,自适应地调整特征权重比例。所提出的融合方法通过两阶段训练策略完成。第一个阶段,应用创新的特征交互概念训练编码器,增强特征表达,重建特征图像。第二个阶段,基于设计的权重自适应调整模块训练红外与可见光特征融合任务。公开数据集的实验结果表明,与现有方法相比,本方法在主观和客观的评价方面均优于其他典型方法。

    Abstract:

    In the field of image fusion,existing methods based on Convolution Neural Network(CNN)or Transformer architectures have two limitations:first,effective aggregation between shallow texture features and deep semantic features is not possible;second,the weight ratio of infrared and visible light features cannot be adaptively changed.In this paper,an adaptive fusion method of infrared and visible images that introduce feature interaction is proposed.Firstly,a feature interaction module based on Transformer is constructed to aggregate cross scale feature information and enhance feature representation.Secondly,a fusion module is designed to adaptively adjust the feature weight ratio.The proposed fusion method is completed by a two stage training strategy.In the first stage,the encoder is trained using innovative feature interaction concepts to enhance feature representation and reconstruct feature images.In the second stage,the weight adaptive adjustment module based on the design trains the infrared and visible feature fusion task.The experimental results on publicly available datasets show that this method is superior to other typical methods in terms of subjective and objective evaluation compared with the existing methods.

    参考文献
    相似文献
    引证文献
引用本文

陈从平,闫焕章,郁春明,江高勇,凌阳,戴国洪.引入特征交互的红外与可见光图像自适应融合[J].激光与红外,2023,53(7):1052~1059
CHEN Cong-ping, YAN Huan-zhang, YU Chun-ming, JIANG Gao-yong, LING Yang, DAI Guo-hong. Adaptive fusion of infrared and visible light images withintroduction of feature interaction[J]. LASER & INFRARED,2023,53(7):1052~1059

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:
  • 最后修改日期:2022-11-01
  • 录用日期:
  • 在线发布日期: 2023-07-14
  • 出版日期: