Incorrect segmentation always occurs because the target is much smaller than its background. Common threshold selection methods only rely on the probability information from the image histogram without directly considering the uniformity of inter-class gray distribution. Aiming at these problems a threshold selection algorithm based on the area difference between target and background and modified gray entropy is proposed. Firstly，an adaptive median filter and a mean filter are used for image preprocessing so as to reduce the noise. Then，a formula of a modified gray entropy is presented，which can well solve the problem of undefined value in the entropy calculation. The final formula of threshold selection is constructed which exploits the characteristics of large area difference between target and its background. Finally，in order to further reduce the computation complexity of algorithm，an optimal search strategy of threshold in histogram is put forward. The experimental results show that the presented algorithm can effectively remove image noise and segment infrared small targets. Compared with Otsu algorithm and maximum entropy algorithm，and the operation time is reduced by about 80%.