Discrete vs. continuous rating scales for language evaluation in NLP


As of July 2018 University of Brighton Repository is no longer updated. Please see our new repository at http://research.brighton.ac.uk.

Belz, Anja and Kow, Eric (2011) Discrete vs. continuous rating scales for language evaluation in NLP In: The 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, 19-24 June, 2011, Portland, Oregon, USA, 19-24 June, 2011.

Full text not available from this repository.


Studies assessing rating scales are very common in psychology and related fields, but are rare in NLP. In this paper we assess discrete and continuous scales used for measuring quality assessments of computer-generated language. We conducted six separate experiments designed to investigate the validity, reliability, stability, interchangeability and sensitivity of discrete vs. continuous scales. We show that continuous scales are viable for use in language evaluation, and offer distinct advantages over discrete scales.

Item Type: Contribution to conference proceedings in the public domain ( Full Paper)
Subjects: Q000 Languages and Literature - Linguistics and related subjects > Q100 Linguistics
Faculties: Faculty of Science and Engineering > School of Computing, Engineering and Mathematics > Natural Language Technology
Depositing User: Converis
Date Deposited: 16 Feb 2012 13:01
Last Modified: 25 Mar 2015 12:08
URI: http://eprints.brighton.ac.uk/id/eprint/9901

Actions (login required)

View Item View Item


Downloads per month over past year