Based on complex amplitude modulation function of optical components,transfer matrix of complex-shaped cavity is established in the paper.The eigen transverse mode distribution and corresponding loss coefficient of UR90 are calculated.The result is consistent with the calculating results by FFT method.The method of numerical Matrix is analyzed,and the relationship between convergence degree with discrete element is explained.