A real-time moving object detection and tracking system aimed at fixed monitoring scene was designed and achieved. Improved three frames difference can detect moving object accurately and rapidly in a complex background. Tracking object will lose easily when using the Lucas-Kanade optical flow of pyramid image. The traditional template matching method can track the object much accurately because of high utilization rate of the image,but its amount of computation is too large. The algorithm in this paper can avoid above problems by integrating the two methods. The experiment shows that this algorithm can achieve tracking object better and gain the trajectory of moving object,which has good real-time performance and robustness.