As the infrared image of rail crack has low contrast,low signal-noise ratio,fuzzy texture detail and is difficult to enhance the target region,an infrared image enhancement optimization algorithm of rail crack based on the multi-scale top-hat and bottom-hat transformation is proposed. First,an improved top-hat transform is used to extract multi-scale bright image regions,and multi-scale dim image regions is extracted using an improved bottom-hat transform;then the bright and dark areas of the image are established through the maximum value extracted from the multi-scale bright and dim image regions;finally,image enhancement is realized through weighted processing. Experimental results indicate that the proposed algorithm can suppress the noise and highlight image edge of the target,meanwhile effectively improve the image contrast,which lays a necessary foundation for the further image processing.