With normal lens for moving target tracking，the FOV is small and it is easy to lose the target under complicated background.To solve the problems，an improved mean-shift algorithm based on the fish-eye lens is put forward.Firstly the moving target’s initial frame is extracted by using SIFT algorithm，then the next frame with target’s location is forecasted by using the Kalman filtering algorithm，finally the object is tracked by using improved Mean-shift algorithm.The result proves that this improved mean-shift algorithm based on fish-eye lens has good tracking performance，compared with the traditional tracking method.It has the characteristics of wide FOV，real-time，robustness and good accuracy etc.