Semantic analysis of handwritten document images offers a wide range<\/p>\n
of practical application scenarios. A sequential combination of handwritten text<\/p>\n
recognition (HTR) and a task-specific natural language processing system offers an<\/p>\n
intuitive solution in this domain. However, this HTR-based approach suffers from the<\/p>\n
problem of error propagation. An HTR-free model, which avoids explicit text recognition<\/p>\n
and solves the task end-to-end, tackles this problem, but often produces poor results.<\/p>\n
A possible reason for this is that it does not incorporate largely pre-trained semantic<\/p>\n
word embeddings, which turn out to be one of the most powerful advantages in the<\/p>\n
textual domain. In this work, we propose an HTR-based and an HTR-free model and compare<\/p>\n
them on a variety of segmentation-based handwritten document image benchmarks including<\/p>\n
semantic word spotting, named entity recognition, and question answering. Furthermore,<\/p>\n
we propose a cross-modal knowledge distillation approach to integrate semantic knowledge<\/p>\n
from textually pre-trained word embeddings into HTR-free models. In a series of<\/p>\n
experiments, we investigate optimization strategies for robust semantic word image<\/p>\n
representation. We show that the incorporation of semantic knowledge is beneficial for<\/p>\n
HTR-free approaches in achieving state-of-the-art results on a variety of benchmarks.<\/p>\n","protected":false},"excerpt":{"rendered":"
Semantic analysis of handwritten document images offers a wide range of practical application scenarios. A sequential combination of handwritten text recognition (HTR) and a task-specific natural language processing system offers an intuitive solution in this domain. However, this HTR-based approach suffers from the problem of error propagation. An HTR-free model, which avoids explicit text recognition and solves the task end-to-end, tackles this problem, but often produces poor results. A possible […]<\/p>\n","protected":false},"author":12,"featured_media":0,"template":"","meta":{"_acf_changed":false,"footnotes":""},"publication-type":[30],"class_list":["post-32477","publication","type-publication","status-publish","hentry","publication-type-article"],"acf":[],"publishpress_future_workflow_manual_trigger":{"enabledWorkflows":[]},"_links":{"self":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication\/32477","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication"}],"about":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/types\/publication"}],"author":[{"embeddable":true,"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/users\/12"}],"version-history":[{"count":0,"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication\/32477\/revisions"}],"wp:attachment":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/media?parent=32477"}],"wp:term":[{"taxonomy":"publication-type","embeddable":true,"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication-type?post=32477"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}