Human behavior recognition is widely applied in safety monitoring,security monitoring,smart home,etc.and proves to be of great research significance in these areas.The recognition method of human behavior based on infrared information has attracted the attention of scholars at home and abroad because infrared information is barely influenced by illumination and able to protect users′ privacy.In this paper an infrared image data set is constructed by sequential frame stitching of infrared information with seven behavior categories.The traditional ResNet 18 network which works perfectly on visible image recognition has poor performance on infrared image recognition.In this paper,corresponding improvements are made according to the characteristics of infrared images.First,a multi branch isomorphic structure is established to replace 7×7 convolution,which increases the expressive ability of the network.Second,maximum pooling is combined with average pooling to avoid useful information lost.Finally,the multiple residual structures are constructed by introducing asymmetric convolution block and combining with the improved CBAM module to optimize the residual block,then increase the network diversity and enhance the feature extraction ability of the network.The experimental results show that the recognition rate of the improved ResNet 18 network is 99.96%,which is higher than that of the original ResNet 18 network as well as other networks based on infrared images.