Aiming at edge blur and target unsharpness in infrared and visible image fusion algorithm, an infrared and visible image fusion algorithm based on the direction of the edge detection is proposed. Based on the traditional edge detection algorithm, a direction factor is introduced. And according to the different characteristics with the high frequency sub-band and low frequency sub-band wavelet coefficients, different fusion rules are adopted. Experimental results show that the algorithm can effectively extract the edges information and the target signal in the image. It has the advantages of high reliability and high definition.