In order to complete the task of detecting and tracking infrared dim objects much faster,the paper introduces an effective hardware design.Image pre-processing is completed by implementing Top-hat arithmetic in FPGA,which mitigates the DSPs′ task greatly,so,it is able to do more operations in the limited time,and the system′s real-time requirement can be fulfilled better.When the image processing system is used to detect and track the infrared dim objects，it comes good result.