Abstract：In video tracking,in order to achieve large numbers of operating processing in a finite time,some measures for improving operating speed are needed. A hardware system design based on TMS320C6416 high-speed DSP and FPGA is introduced. The experiments demonstrate that the system is much easier than the former DSP only video processing system for fulfilling the real-time requirement in carrying out complicated algorithm and multi-target extraction.