Discrete vs. continuous rating scales for language evaluation in NLP

BELZ, ANJA and Kow, Eric (2011) Discrete vs. continuous rating scales for language evaluation in NLP In: The 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, 19-24 June, 2011, Portland, Oregon, USA, 19-24 June, 2011.

Full text not available from this repository.

Abstract

Studies assessing rating scales are very common in psychology and related fields, but are rare in NLP. In this paper we assess discrete and continuous scales used for measuring quality assessments of computer-generated language. We conducted six separate experiments designed to investigate the validity, reliability, stability, interchangeability and sensitivity of discrete vs. continuous scales. We show that continuous scales are viable for use in language evaluation, and offer distinct advantages over discrete scales.

Item Type: Contribution to conference proceedings in the public domain ( Full Paper)
Subjects: Q000 Languages and Literature - Linguistics and related subjects > Q100 Linguistics
Faculties: Faculty of Science and Engineering > School of Computing, Engineering and Mathematics > Natural Language Technology
Depositing User: Converis
Date Deposited: 16 Feb 2012 13:01
Last Modified: 07 Feb 2013 12:26
URI: http://eprints.brighton.ac.uk/id/eprint/9901

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year