The decision of speed and direction is the key to the study of night-time unmanned vehicle driving.Based on infrared image and radar information,a multi-tasking classification network of infrared images containing depth information is proposed for the decision of the speed and direction of unmanned vehicles at night.The depth network is trained by infrared cameras and data collected by radars.The depth-infrared image is estimated using a convolution-deconvolution neural network to obtain depth information.The depth information is used to make classification network training labels.The AlexNet classification network is used to obtain speed decision suggestions.Then according to the infrared image road information training direction classification network,the driving decision problem of the unmanned vehicle is transformed into a classification model,and the classification model is combined with the depth estimation network.Experimental results show that the angle accuracy and speed accuracy of the network are 87.43% and 85.89% respectively,and the time taken for decision making of the acquired image using the trained model is 0.04 s/frame,which can meet the requirements of real-time performance.