A new method of infrared image segmentation is presented. Firstlymarginal pixels are chosen that has the gray levelwhich lie in the optimal threshold as the seed of growing process. Secondly the gray level decision criterion is used to perform the growing process and accomplish the segmentation for the target. Experiments show that big and small infrared targets can be segmented out of complex background quickly, completely and rapidly by the method.