Manual data entry of vehicle information is time-consuming and prone to errors, as the automotive and logistics industry experiences daily. Together with MOSOLF, the ETECTURE team developed a tool to automate the reading of data contained in vehicle registrations, to simplify and speed up the process.
About the client
The MOSOLF Group is one of the leading system service providers for the automobile industry in Europe. The MOSOLF Group’s services cover the complete value-added chain for automobile logistics: from the end of the conveyor belt to recycling. In addition to transporting vehicles (cars, light commercial vehicles, “high & heavy”), its business operations include workshop services, special vehicle construction, industrial coatings, mobility services, releasing agent services, electromobility, and vehicle recycling. MOSOLF provides all-around, customized solutions within this context for the automobile industry, fleet operators, and car dealers from one source and also handles the associated data flow by means of modern software solutions.
The challenge
Manual data collection in the automotive and logistics industry is a daily task, with much of the information able to be obtained from the vehicle registration document. However, in practice, this information is manually typed, as the document does not contain a machine-readable zone like other official documents, such as a German ID card. This process is not only time-consuming but also prone to errors. Our challenge was to accelerate the process through optical character recognition (OCR) technology and reduce the risk of errors.
Besides, the challenge was that the German vehicle registration format (appearance and composition) had been unchanged since October 2005 and the document’s appearance in different federal territories often varied in font type and size. Additionally, the anti-counterfeiting features of the official document also pose particular challenges, such as the “multicolor guilloche security underprint (processed in two stages) with iris flow and integrated microscripts on both sides.
Furthermore, while the dark green grid lines help the human reader to associate the individual fields, they make machine text recognition more difficult. In particular, the so-called print offset, or text protruding beyond the lines, causes significant difficulties in automated recognition. OCR systems that rely on the isolation of individual characters through edge detection are unable to process these documents, as the grid lines connect all characters. Binarizing the image based on color values, i.e. separating black text from green grid lines, is also difficult, as the exact shade of green depends heavily on the recording quality and the recording device.
Example of the German vehicle registration document
The solution
The ETECTURE team has developed a custom-tailored OCR system for vehicle registration certificates that was initially based on a template-matching algorithm for text recognition. To achieve this, they created prototypes of each character that could appear on a vehicle registration certificate. The algorithm calculated the correlation between the image and the prototype at each position, and a character was recognized if the correlation was high.
However, due to variations in the appearance of the vehicle registration certificates in practice, we reached the limits of text recognition with the template matching algorithm. As a result, we had to use neural networks, which were not a challenge for different font styles.
First, the vehicle registration certificate is recognized and located on the image. Then, the individual text fields can be cut out, resulting in small image sections with the associated text. Now, the neural networks come into play, which provides the recognized text from the small image section.
In addition, we have defined rules for selecting the allowed characters for each field. For example, a date can only contain numbers and dots but no letters. The chassis number (vehicle identification number, VIN) can only contain numbers and uppercase letters. The recognized chassis number can also be validated by a checksum printed on the vehicle registration certificate. This additional validation helps to minimize errors in reading the chassis number.
For some other fields, there are lists of possible values that can be entered into the field. For example, vehicle classes and construction types are specified in the directory for classifying motor vehicles and their trailers.
OCR solution for Mosolf developed by the ETECTURE team
The recognized text is presented clearly next to the corresponding field cuts, making it easy to compare with the input image and correct any errors. The check digit validation for the chassis number is highlighted in color.
The user not only has the ability to check the results of the machine text recognition but can also make necessary corrections if needed. Finally, the user can download the data in CSV or JSON format. Of course, exports in other file formats or direct integration with third-party systems used in the company are also possible.
Technologies
The development team implemented the text recognition algorithm in Python (using TensorFlow and OpenCV). They packaged it into a Flask web server that provides the necessary interface. It takes in an image and returns a JSON object that contains the mapping of field labels to the text recognized in each field. The front end was built using HTML, CSS, and Vanilla Javascript. The user can upload a previously taken image via file input. The image is then sent to the backend, processed, and the result is played back to the front.
What’s next?
This approach is not only suitable for vehicle registration certificates but can also be used with other documents as long as their structure also follows a fixed, recurring pattern. Possible applications could be identity documents, delivery notes, bills of lading, invoices, etc.
The presented algorithm showcases its strength by incorporating prior knowledge and training using artificially generated data. This allowed us to achieve significantly better results in our experiments and test runs with this approach compared to generic tools like Tesseract or Google Mobile Vision. Both tools are theoretically capable of reading more or less any document, but in the case of the vehicle registration certificate, they failed to deliver good results with the supplied poor results
ABOUT ETECTURE
