Balanced-COPA README.md

This archive contains materials related to the Balanced Choice Of Plausible Alternatives (B-COPA) evaluation of open-domain commonsense causal reasoning.

balancedcopa-dev-all.xml : 1000 questions of the development set. Original dev set questions have ids 1 -- 500. Mirrored instances, that make the dev set balanced-COPA, have ids 1001 -- 1500.

For more information, please check out the following webpage: http://balanced-copa.github.io

Also, please see the following publications:

1. Roemmele, M., Bejan, C., and Gordon, A. (2011) Choice of Plausible Alternatives: An Evaluation of Commonsense Causal Reasoning. AAAI Spring Symposium on Logical Formalizations of Commonsense Reasoning, Stanford University, March 21-23, 2011.
2. Pride Kavumba, Naoya Inoue, Benjamin Heinzerling, Keshav Singh, Paul Reisert and Kentaro Inui. When Choosing Plausible Alternatives, Clever Hans can be Clever. To appear in Proceedings of COIN: COmmonsense INference in Natural Language Processing, pp. -, November 2019.

– September 2019