Tilburg center for Cognition and Communication (TiCC)

We study how people communicate with each other and how computer systems can be taught to communicate with us.

vignetweb

TiCC Colloquium: Antske Fokkens

What: Beyond the f-score: Evaluation in NLP
Where: AZ 414
When: Wednesday, 30 November 2016, 12:30 - 13:30 hours



Abstract

Shared tasks and (shared) corpora have proven themselves highly valuable for NLP. They have allowed us to evaluate our methods and compare them to others helping us, our readers and reviewers to assess the quality of our methods. A downside of the wide-spread approach of comparing results on a gold dataset is that it is relative common practice to draw conclusions based on the highest numbers without looking into what is behind this. However, what goes wrong and why can be highly relevant for end-applications and, specially given the well-known difficulties with reproducing results, looking into the details of how and why results improve (or not) is highly relevant. In this talk, I will propose two ways of looking 'beyond the f-score'; intrinsic evaluation shown through a use case investigating error propagation in parsing and extrinsic evaluation in digital humanities studies.

When: 30 November 2016 12:30

End date: 30 November 2016 13:30