We propose an end-to-end vehicle type and licence plate localisation and recognition system, based on a combination of existing methods, modifications to neural network architecture and improvements in the training process, resulting in high accuracy licence plate localization and recognition with minimal post-processing.

The proposed system was trained and tested on a custom data set of 17000 manually labeled images, containing in total 19000 license plates, 18300 motorcars, 1414 trucks, 722 ambulances, 492 motorbikes, 45 trailers, 26 police motorcars, 15 firetrucks and 6 other operative cars. Additional transformations such as added noise, zoom and rotation
were used to generate more images.

The methods and tools used include FCN, transfer learning, Feature Extraction Maxout CNN, Recurrent LSTM networks, Tensorflow etc.

The resulting system is capable of localizing multiple types of vehicles (including motorcycles) as well as their licence plates. The achieved precision of the localisation is 99.5%. The whole number recognition accuracy is 96.7% and character level recognition accuracy is 98.8%. An end-to-end test for a single image took 0.2 seconds on a GPU
and 8 seconds on a single core CPU.

N. Dorbe, R. Kadikis, K. Nesenbergs. “Vehicle type and licence plate localisation and segmentation using FCN and LSTM”, Proceedings of New Challenges of Economic and Business Development 2017, Riga, Latvia, May 18-20, 2017, pp. 143-151

pdf