The rate of “Not Good” products due to the presence of abnormal objects during the quality control inspection of mobile phone’s panels is as high as 38 percent without supervision and 19 percent with naked eye’s inspection. To reduce the loss of small abnormal objects and improve the recognition rate of object detection of automatic panel quality monitoring, the paper proposes abnormal object recognition based on a template matching and subtract background technique. Firstly, the object recognition network based on convolutional neural networks (CNNs) is introduced initially. Then, the Canny Edge Detection method improves the image input quality. Next, template matching is utilized to discover things that depart from a standard pattern or a predetermined model. Models for background reduction will be implemented instantly into smartphone panel quality checking systems. Continually, a vision-based quality control (QC) strategy will be developed. Lastly, experimental results from a practical vision-based automatic anomalous object identification system demonstrate the viability and enhancement of the suggested strategy.