System building cost vs. output quality in data-to-text generation

Belz, Anja and Kow, Eric (2009) System building cost vs. output quality in data-to-text generation In: Proceedings of the 12th European workshop on natural language generation (ENLG 2009), Athens, Greece, 30-31 March, 2009.

Full text not available from this repository.

Abstract

Data-to-text generation systems tend tobe knowledge-based and manually built,which limits their reusability and makes them time and cost-intensive to createand maintain. Methods for automating(part of) the system building process exist,but do such methods risk a loss inoutput quality? In this paper, we investigatethe cost/quality trade-off in generation system building. We comparefour new data-to-text systems which were created by predominantly automatic techniques against six existing systems for the same domain which were created by predominantlymanual techniques. We evaluate the ten systems using intrinsic automatic metrics and human quality ratings.We find that increasing the degree towhich system building is automated doesnot necessarily result in a reduction in outputquality. We find furthermore that standardautomatic evaluation metrics underestimate the quality of handcrafted systems and over-estimate the quality of automatically created systems.

Item Type: Contribution to conference proceedings in the public domain ( Full Paper)
Subjects: G000 Computing and Mathematical Sciences > G400 Computing
Q000 Languages and Literature - Linguistics and related subjects > Q100 Linguistics
Depositing User: Converis
Date Deposited: 04 Apr 2013 03:01
Last Modified: 25 Mar 2015 12:12
URI: http://eprints.brighton.ac.uk/id/eprint/11411

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year