The research on vehicle occupant number detection is of great significance to promote the construction of HOV lanes.Based on the Faster RCNN network model,this paper combines the multi-spectral infrared system to obtain the image of the car cab.In the existing data,due to overexposure and underexposure problems,the target feature differences in the image are large,and the accuracy of network detection is not high.For this,the convolution calculation of the deformed structure is used to improve the ability to express the receptive field of the feature unit and the target edge information.Use deformed ROI-Pooling to enhance the feature expression after ROI feature mapping and enhance the generalization ability of the network.In the case of multiple occupants,the KL loss is introduced due to misdetection and missed detection caused by occlusion among the occupants.At the same time,the combination of Soft-NMS and variance voting is used to improve the rationality of the NMS filtering process of repeated target frames and improve in addition to the rationality of position regression and the prediction ability of overlapping targets,the overall detection accuracy has been improved.The experimental results show that the accuracy of the algorithm detection of the network under different numbers in this paper has been improved,and can basically meet the requirements of the industry regulations greater than 80 %.