Why only Micro-$F_1$? Class Weighting of Measures for Relation Classification

Abstract

Relation classification models are conventionally evaluated using only a single measure, e.g., micro-F1, macro-F1 or AUC. In this work, we analyze weighting schemes, such as micro and macro, for imbalanced datasets. We introduce a framework for weighting schemes, where existing schemes are extremes, and two new intermediate schemes. We show that reporting results of different weighting schemes better highlights strengths and weaknesses of a model.

David Harbecke

PhD Student

Leonhard Hennig

Senior Researcher

Christoph Alt

Senior Researcher

My research interests include transfer-learning, multi-task learning, few- and zero-shot learning.