A point matching algorithm based on self-similarities is proposed to solve the difficulty of IR and visible images matching.Firstly,sums of square in small neighborhoods are calculated.Secondly,by introducing Gaussian scale space,feature points are extracted by FAST-9 corner detector which has scale-invariance.And the main orientation for each point is assigned according to the neighborhood information.Thirdly,correlation surfaces with corresponding scale are chosen for region.Extreme value of each correlation surface is extracted to construct a normalized descriptor with 100 dimensions.Finally,the nearest neighbor algorithm is used to match control points after eliminating non-informative descriptors.Experimental results indicate that the proposed method is robust to changes in rotation change,affine change and scale change.Meanwhile,it gets a higher correct ratio than SIFT.