The FARI Conference returns on 17 and 18 November in Brussels, find out more.
An Initiative of
Supported by
FEB 2025
Early evidence of how LLMs outperform traditional systems on OCR/HTR tasks for historical records
Authors: Hugues Bersini, Julien Baudru, Seorin Kim, Vincent Ginis, Wouter Ryckbosch.
We explore the ability of two LLMs — GPT-4o and Claude Sonnet 3.5 — to transcribe historical handwritten documents in a tabular format and compare their performance to traditional OCR/HTR systems: EasyOCR, Keras, Pytesseract, and TrOCR. Considering the tabular form of the data, two types of experiments are executed: one where the images are split line by line and the other where the entire scan is used as input. Based on CER and BLEU, we demonstrate that LLMs outperform the conventional OCR/HTR methods. Moreover, we also compare the evaluated CER and BLEU scores to human evaluations to better judge the outputs of whole-scan experiments and understand influential factors for CER and BLEU. Combining judgments from all the evaluation metrics, we conclude that two-shot GPT-4o for line-by-line images and two-shot Claude Sonnet 3.5 for whole-scan images yield the transcriptions of the historical records most similar to the ground truth.
Contributors
Share
Other publications
Date
JUN 2025
Researchers
Report
Automating the Achievement of SDGs: Robotics Enabling & Inhibiting the Accomplishment of the SDGs
Date
MAY 2022
Researchers
Date
OCT 2024
Researchers
Report
Legal Requirements for Automated Coordination Mechanisms for the Sharing of Energy Through Proxies
Date
FEB 2025
Researchers