Based on the analysis of the discontinuity in the switch lines in image mosaic,we propose a novel optimization algorithm of switch line adjustment based on the continuity restriction of line structures.The processing includes:automatic extraction of linear structures at the border of the switched images;calculate the correspondence between the linear structures in adjacent images;adjust switch lines based on the continuity restriction of linear structures.The experiments demonstrated that the proposed algorithm can improve the switch line of image mosaic.