Infrared focal plane array detector has a multilayer configuration which is mainly composed of electrical lead board, Si-ROIC, indium bumps and detector chip. Substantial thermal stress is generated in the IRFPA packaging assembly at cryogenic temperature due to the difference in coefficient of thermal expansion(CTE) of various materials in the assembly, which affects the thermal cycling reliability significantly. A finite element analysis model for investigation of thermal stress distribution in detector at cryogenic temperature is established. This model is utilized to analyze the influence of CTE, Young′s modulus and thickness of electrical lead board on thermal stress and warpage for Si and CdZnTe substrate detectors;Some improved methods are presented respectively according to the analytical results for two kinds of detectors, and these methods are verified by calculation.