Supporting text mining for e-Science: the challenges for Grid-enabled natural language processing

Abstract

Over the last few years, language technology has moved rapidly from ‘applied research’ to ‘engineering’, and from small-scale to large-scale engineering. Applications such as advanced text mining systems are feasible, but very resource-intensive, while research seeking to address the underlying language processing questions faces very real practical and methodological limitations. The e-Science vision, and the creation of the e-Science Grid, promises the level of integrated large- scale technological support required to sustain this important and successful new technology area. In this paper, we discuss the foundations for the deployment of text mining and other language technology on the Grid — the protocols and tools required to build distributed large-scale language technology systems, meeting the needs of users, application builders and researchers.