Discrete vs. continuous rating scales for language evaluation in NLP
BELZ, ANJA and Kow, Eric (2011) Discrete vs. continuous rating scales for language evaluation in NLP In: The 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, 19-24 June, 2011, Portland, Oregon, USA, 19-24 June, 2011.Full text not available from this repository.
Official URL: http://www.aclweb.org/anthology-new/P/P11/P11-1000...
Studies assessing rating scales are very common in psychology and related fields, but are rare in NLP. In this paper we assess discrete and continuous scales used for measuring quality assessments of computer-generated language. We conducted six separate experiments designed to investigate the validity, reliability, stability, interchangeability and sensitivity of discrete vs. continuous scales. We show that continuous scales are viable for use in language evaluation, and offer distinct advantages over discrete scales.
Actions (login required)
Downloads per month over past year