The following Jupyter notebook demonstrates how PyFLAGR can be used to aggregate ranked lists of elements. The notebook itself can be downloaded from GitHub. The input files can be downloaded from the following links:

  • testdata.csv: This example file contains the preference lists that were submitted by 50 voters for 20 queries. Each input list contains 30 elements.
  • testdata_qrels.csv: This example file contains the relevance judgments for the elements of all preference lists for all queries of testdata.csv.