Real-Time Text Extraction from Malaysian Road Direction Signs Using Dashboard Camera

Abstract

The road direction sign, which serves as a navigation tool, aids the driver in travelling on the road. A real-time text extraction from road direction signs is investigated, using input video images acquired from a dashboard camera while driving on Malaysian roads. This paper proposed a three-stage approach for real-time text extraction: (i) traffic direction signboard detection; (ii) text detection; and (iii) text recognition. In the direction signboard detection, the YOLOv5s model was applied and achieved a precision of 97%and a recall of 96%. In the text detection stage, three text detection models, EAST, DB and PAN, were investigated to select the best text detection model. The DB model is chosen with the highest IoU of 0.8238 and the shortest inference time of 0.1938s. In the text recognition stage, four models (Pytesseract, CRNN, RobustScanner and ABINet) are employed to perform evaluation and pick the best model. The CRNN is selected with the lowest character error rate of 5.85%, the highest accuracy of 97.84%, and a fast inference time of 0.324s. The proposed models, YOLOv5s, DB, and CRNN are combined to run sequentially in the real-time system during daytime conditions. It is performed by using a laptop that is equipped with a GPU and programmed in Python. The real-time text extraction system can run at an adaptive frame rate of 6.5 frames per second with an accuracy of 97%.