{"id":32375,"date":"2026-01-21T17:01:51","date_gmt":"2026-01-21T17:01:51","guid":{"rendered":"https:\/\/lamarr-institute.org\/publication\/constructing-ccee-an-llm-evaluation-dataset-forcomplex-context-aware-event-extraction-for-generegulatory-networks\/"},"modified":"2026-09-17T11:23:54","modified_gmt":"2026-09-17T11:23:54","slug":"constructing-ccee-an-llm-evaluation-dataset-forcomplex-context-aware-event-extraction-for-generegulatory-networks","status":"publish","type":"publication","link":"https:\/\/lamarr-institute.org\/de\/publication\/constructing-ccee-an-llm-evaluation-dataset-forcomplex-context-aware-event-extraction-for-generegulatory-networks\/","title":{"rendered":"Constructing CCEE an LLM evaluation dataset forComplex Context-aware Event Extraction for generegulatory networks"},"content":{"rendered":"<p>This paper presents a first look at CCEE (Complex Context-aware Event Extraction), a currently in the works novel<\/p>\n<p>evaluation dataset for context-rich gene regulatory network extraction from scientific literature. We propose an<\/p>\n<p>annotation scheme for cancer research papers, capturing both core gene interactions and extensive contextual<\/p>\n<p>information across 10-14 categories per event, addressing limitations in existing datasets to test construction of<\/p>\n<p>disease specific biomedical knowledge graphs. Unlike previous datasets that focus primarily on entity connections<\/p>\n<p>of isolated triplets, CCEE links contextual attributes directly to gene regulatory events, providing a more integrated<\/p>\n<p>representation of scientific knowledge. We illustrate the annotation on 9 papers manually labeled by multiple<\/p>\n<p>experts, and give a first impression of challenges and ways to address them. Additionally we show first evaluations<\/p>\n<p>of LLMs as an annotation system. While it under performs human experts in interaction type labeling, it matches<\/p>\n<p>human performance on attributing entities as context to interactions.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This paper presents a first look at CCEE (Complex Context-aware Event Extraction), a currently in the works novel evaluation dataset for context-rich gene regulatory network extraction from scientific literature. We propose an annotation scheme for cancer research papers, capturing both core gene interactions and extensive contextual information across 10-14 categories per event, addressing limitations in existing datasets to test construction of disease specific biomedical knowledge graphs. Unlike previous datasets that [&hellip;]<\/p>\n","protected":false},"author":14,"featured_media":0,"template":"","meta":{"_acf_changed":false,"footnotes":""},"publication-type":[32],"class_list":["post-32375","publication","type-publication","status-publish","hentry","publication-type-inproceedings"],"acf":[],"publishpress_future_workflow_manual_trigger":{"enabledWorkflows":[]},"_links":{"self":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication\/32375","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication"}],"about":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/types\/publication"}],"author":[{"embeddable":true,"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/users\/14"}],"version-history":[{"count":0,"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication\/32375\/revisions"}],"wp:attachment":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/media?parent=32375"}],"wp:term":[{"taxonomy":"publication-type","embeddable":true,"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication-type?post=32375"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}