The following Jupyter notebook demonstrates how PyFLAGR can be used to compare various rank aggregation methods with the Comparator class. The notebook itself can be downloaded from GitHub. The input files can be downloaded from the following links:
testdata.csv: This example file contains the preference lists that were submitted by 50 voters for 20 queries. Each input list contains 30 elements.testdata_qrels.csv: This example file contains the relevance judgments for the elements of all preference lists for all queries oftestdata.csv.