nltk.ribes()¶

nltk.ribes(references, hypothesis, alpha=0.25, beta=0.1)¶

The RIBES (Rank-based Intuitive Bilingual Evaluation Score) from Hideki Isozaki, Tsutomu Hirao, Kevin Duh, Katsuhito Sudoh and Hajime Tsukada. 2010. “Automatic Evaluation of Translation Quality for Distant Language Pairs”. In Proceedings of EMNLP. http://www.aclweb.org/anthology/D/D10/D10-1092.pdf

The generic RIBES scores used in shared task, e.g. Workshop for Asian Translation (WAT) uses the following RIBES calculations:

RIBES = kendall_tau * (alpha**p1) * (beta**bp)

Please note that this re-implementation differs from the official RIBES implementation and though it emulates the results as describe in the original paper, there are further optimization implemented in the official RIBES script.

Users are encouraged to use the official RIBES script instead of this implementation when evaluating your machine translation system. Refer to http://www.kecl.ntt.co.jp/icl/lirg/ribes/ for the official script.

Parameters:	references – a list of reference sentences hypothesis (list(str)) – a hypothesis sentence alpha (float) – hyperparameter used as a prior for the unigram precision. beta (float) – hyperparameter used as a prior for the brevity penalty.
Returns:	The best ribes score from one of the references.
Return type:	float