Title: Coref-QA: Towards More Applicable Coreference Resolvers
Speaker: Haixia Chai (HITS)
Abstract
The majority of the developments in coreference resolution are focused on improving the performance on the standard coreference annotated datasets, e.g., CoNLL-2012. However, it is not clear how these improvements translate into downstream task performances. In this paper, we introduce Coref-QA, i.e., the first evaluation framework that assesses the impact of coreference resolution on a downstream task, namely answer selection. To measure that the resulting improvements in coreference resolution are not specific to a single dataset, domain, or QA model, Coref-QA covers multiple text domains and encompasses several answer selection methods. Our results confirm that intrinsic evaluation does not always correctly approximate the utility of the coreference resolvers for the end tasks and intrinsic evaluations should be accompanied by extrinsic evaluations.
