Summary Evaluation with and without References

Authors: Juan-Manuel Torres-Moreno, Horacio Saggion, Iria da Cunha, Eric SanJuan, Patricia Velázquez-Morales

 Polibits, 42, pp. 13-19, 2010.

Abstract: We study a new content-based method for the evaluation of text summarization systems without human models which is used to produce system rankings. The research is carried out using a new content-based evaluation framework called FRESA to compute a variety of divergences among probability distributions. We apply our comparison framework to various well-established content-based evaluation measures in text summarization such as COVERAGE, RESPONSIVENESS, PYRAMIDS and ROUGE studying their associations in various text summarization tasks including generic multi-document summarization in English and French, focus-based multi-document summarization in English and generic single-document summarization in French and Spanish

 Keywords: Text summarization evaluation; content-based evaluation measures; divergences

PDF: Summary Evaluation with and without References, Alternative link