Understanding the peer review process could help research and shed light on the mechanisms that underlie crowdsourcing. In this paper, we present an agent-based model of peer review built on three entities - the paper, the scientist and the conference. The system is implemented on a BDI platform (Jason) that allows to define a rich model of scoring, evaluating and selecting papers for conferences. Then, we propose a programme committee update mechanism based on disagreement control that is able to remove reviewers applying a strategy aimed to prevent papers better than their own to be accepted (rational cheating). We analyze a homogeneous scenario, where all conferences aim to the same level of quality, and a heterogeneous scenario, in which conferences request different qualities, showing how this affects the update mechanism proposed. We also present a first step toward an empirical validation of our model that compares the amount of disagreements found in real conferences with that obtained in our simulations. © 2013 World Scientific Publishing Company.
A simulation of disagreement for control of rational cheating in peer review
Tipo Pubblicazione:
Articolo
Publisher:
World Scientific Publishing, Singapore, Singapore
Source:
Advances in Complex Systems 16 (2013). doi:10.1142/S0219525913500045
info:cnr-pdr/source/autori:Grimaldo, Francisco; Paolucci, Mario/titolo:A simulation of disagreement for control of rational cheating in peer review/doi:10.1142/S0219525913500045/rivista:Advances in Complex Systems/anno:2013/pagina_da:/pagina_a:/intervallo
Date:
2013
Resource Identifier:
http://www.cnr.it/prodotto/i/281437
https://dx.doi.org/10.1142/S0219525913500045
info:doi:10.1142/S0219525913500045
http://www.scopus.com/record/display.url?eid=2-s2.0-84879650706&origin=inward
Language:
Eng