To deal with the robustness of object tracking in the time-variant scene,an algorithm combined Kalman filter with Mean-Shift algorithm is proposed in this paper.We employ YCbCr features to describe the object,and Kalman filter is used to predict the position and velocity of the target.According to different disturbance circumstances,the two algorithms tracking results are done with liner weight method by using different scale factors to get the final position of the target.Furthermore,a model update strategy is utilized to alleviate the model drift.Experimental results show the good performances of the proposed algorithm.