Annotated text data is essential for structuring biomedical knowledge. State-of-the-art machine learning models achieve F1 scores >90\\% for named entity recognition (NER), but relation extraction (RE) remains challenging, with F1 scores of 60\\% (up to 75\u201380\\% on binary datasets using additional strategies). These models require fine-tuning on costly manually annotated datasets.<\/p>\n
This paper investigates structured multi-extraction prompting as a technique for enabling Large Language Models (LLMs) such as Mistral and OpenAI models to perform biomedical relation extraction (RE) without fine-tuning, reducing manual annotation burdens. We evaluate this approach across zero-shot and few-shot settings and compare extracting relations individually versus in batches, assessing whether multi-extraction with structured generation can serve as a viable, cost-effective alternative to fine-tuned models.<\/p>\n
Our results show that multi-extraction prompting with structured generation significantly boosts LLM performance on biomedical relation extraction (RE) tasks, roughly matching fine-tuned models on binary classification in zero-shot settings and approaching their performance on multi-class tasks with few-shot examples but the gap remains larger in those cases. By providing the model with a list of entities and leveraging structured, dynamic generation, we harness LLMs\u2019 in-context learning to enhance both efficiency and accuracy. This method improves performance by up to 19\\% while reducing token usage by up to 550\\%, lowering computational costs comparing it to a single extraction baseline.<\/p>\n
In summary, structured multi-extraction prompting enables modern LLMs to perform binary RE tasks without fine-tuning, matching or approaching the performance of small specialised models, while substantially reducing token usage and computational cost\u2014making them viable annotators for low-resource scenarios where little to no training data exists.<\/p>\n","protected":false},"excerpt":{"rendered":"
Annotated text data is essential for structuring biomedical knowledge. State-of-the-art machine learning models achieve F1 scores >90\\% for named entity recognition (NER), but relation extraction (RE) remains challenging, with F1 scores of 60\\% (up to 75\u201380\\% on binary datasets using additional strategies). These models require fine-tuning on costly manually annotated datasets. This paper investigates structured multi-extraction prompting as a technique for enabling Large Language Models (LLMs) such as Mistral and […]<\/p>\n","protected":false},"author":14,"featured_media":0,"template":"","meta":{"_acf_changed":false,"footnotes":""},"publication-type":[32],"class_list":["post-39130","publication","type-publication","status-publish","hentry","publication-type-inproceedings"],"acf":[],"publishpress_future_workflow_manual_trigger":{"enabledWorkflows":[]},"_links":{"self":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication\/39130","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication"}],"about":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/types\/publication"}],"author":[{"embeddable":true,"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/users\/14"}],"version-history":[{"count":0,"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication\/39130\/revisions"}],"wp:attachment":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/media?parent=39130"}],"wp:term":[{"taxonomy":"publication-type","embeddable":true,"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication-type?post=39130"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}