A Perspectivist Corpus of Numbers in Social Judgements

With growing interest in the use of large language models, it is becoming increasingly important to understand whose views they express. These models tend to generate output that conforms to majority opinion and are not representative of diverse views. As a step toward building models that can take differing views into consideration, we build a novel corpus of social judgements. We crowdsourced annotations of a subset of the Commonsense Norm Bank that contained numbers in the situation descriptions and asked annotators to replace the number with a range defined by a start and end value that, in their view, correspond to the given verdict. Our corpus contains unaggregated annotations and annotator demographics. We describe our annotation process for social judgements and will release our dataset to support future work on numerical reasoning and perspectivist approaches to natural language processing.

  • Published in:
    Proceedings of the 3rd Workshop on Perspectivist Approaches to NLP at LREC-COLING 2024
  • Type:
    Inproceedings
  • Authors:
    May, Marlon; Flek, Lucie; Welch, Charles
  • Year:
    2024

Citation information

May, Marlon; Flek, Lucie; Welch, Charles: A Perspectivist Corpus of Numbers in Social Judgements, Proceedings of the 3rd Workshop on Perspectivist Approaches to NLP at LREC-COLING 2024, 2024, 42--48, May.etal.2024a,

Associated Lamarr Researchers

Prof. Dr. Lucie Flek

Prof. Dr. Lucie Flek

Area Chair NLP to the profile