To efficiently improve the fusion effect of infrared images and visible images as well as fusion speed,an algorithm based on feature extraction is proposed. Firstly the mathematical morphological top-hat operation is improved to extract the feature and background from the source images;then the fusion rules are designed to fuse the feature and background images. Finally the fused image is obtained by recomposing. Besides,parameter selection of the proposed method is analyzed,and a self-adaptive weighting fusion rule for background image fusion is designed. The experiment results indicate that the proposed method can efficiently extract feature information from source image,provide abundant background information and improve the processing velocity for hardware achieving.