A new method for detection and tracking of real-time moving object in video image was purposed.This method uses background difference based background modeling and developed timed motion history image to detect the moving object.The algorithm of motion gradients based timed motion history image is applied to track the moving object after getting the silhouette of moving object.The result of test shows that this method can detect and track the moving object in the given region effectively.The system can remedy the deficiency of fail detection when the moving object stops temporarily.