In order to test the hardness of metal materials,using the provisions of the shape of the pressure head in the polished surface squeezing an irregular square,measuring the length of the diagonal square can be drawn from the measured hardness index.This paper presents a use of image processing technology to achieve accurate measurement of the length of a square diagonal to achieve new method for hardness testing.In this paper,Harris operator to seek the corners of a square edge,and then after the image is divided into four parts for each half-edge of least squares linear fit,after obtained vertex angle,pixel distance of square diagonal can be calculated easily.Through the mapping process of venires caliper pictures obtained the basis for calculating the actual length of the final draw diagonal square.Experiments show that the algorithm can detect the diagonal automatically.Algorithm is a direct impact on the speed and accuracy of test results.Accuracy of this algorithm can reach micron-level.Result of hardness reached a 2% margin of error requirements than traditional approaches can improve the accuracy of 2 to 3 percentage points.And that the algorithm is small,fast,can be used for real-time automatic detection of the hardness of materials project.