FLAGR - Fuse, Learn, Aggregate, Rerank

4.7. Weighted module

The Weighted module provides access to the implementations of the weighted methods of FLAGR. Most rank aggregation approaches treat all input preference lists equally; in contrast, the weighted methods employ exploratory analysis techniques to automatically identify the expert voters in an unsupervised fashion. In the sequel, they assign higher weights to those who were identified as experts, thus boosting the scores of their submitted elements. The module in question includes three classes: DIBRA, Agglomerative and PreferenceRelationsGraph.

Implementation file
pyflagr/pyflagr/Weighted.py

Distance-Based Iterative Rank Aggregation: The DIBRA Python class
This class links to the FLAGR implementation of the weighted method of Akritidis et al., 2022. It derives from RAM, a base class defined in the RAM module. It inherits the flagr_lib connector from RAM and through it, it obtains access to the FLAGR shared library. Its constructor determines the data types of the input arguments and the return type of the DIBRA exposed function. Observe the similarity between the members of self.flagr_lib.DIBRA.argtypes and the input arguments of the DIBRA exposed function.

DIBRA initially employs a standard non-weighted method to generate a starting consensus list. Then, it iteratively assigns converging weights to the voters according to the distances of their submitted lists with this consensus list. Therefore, the DIBRA constructor takes as arguments all the possible hyper-parameters of the supported non-weighted methods. Specifically:

Parameter Type Default Description
eval_pts Integer, Optional. Considered only if rels_file or rels_df is set. 10 Determines the elements in the aggregate list on which the evaluation measures (i.e. Precision, and nDCG) will be computed. For example, for eval_pts=10 FLAGR will compute Average Precision, \(P@1, P@2, ... P@10\) and \(N@1, N@2, ... N@10\).
aggregator Hyperparameter, String, Optional. combsum:borda The selected non-weighted method that produces the initial aggregate list. Possible values are:
  • combsum:borda: CombSUM with Borda Normalization.
  • combsum:rank: CombSUM with Rank Normalization.
  • combsum:score: CombSUM with minmax Score Normalization.
  • combsum:z-score: CombSUM with Z-Score Normalization.
  • combsum:simple-borda: CombSUM with simple Borda Normalization.
  • combmnz:borda: CombMNZ with Borda Normalization.
  • combmnz:rank: CombMNZ with Rank Normalization.
  • combmnz:score: CombMNZ with minmax Score Normalization.
  • combmnz:z-score: CombMNZ with Z-Score Normalization.
  • combmnz:simple-borda: CombMNZ with simple Borda Normalization.
  • condorcet: Condorcet Winners.
  • copeland: Copeland Winners.
  • outrank: Outranking Approach.
w_norm Hyperparameter, String, Optional. minmax The voter weights normalization method. Possible values include:
  • none: no weight normalization takes place.
  • minmax: minmax weight normalization.
  • z: z weight normalization.
dist Hyperparameter, String, Optional. cosine The correlation/distance metric that measures the distance between an input list and the temporary aggregate list. Possible values include:
  • rho: Spearman's \(\rho\).
  • cosine: a metric based on cosine similarity (see Akritidis et al., 2022).
  • footrule: Spearman's Footrule distance metric.
  • tau: Kendall's \(\tau\)
prune Hyperparameter, Boolean, Optional. False Triggers a weight-dependant list pruning mechanism.
gamma Hyperparameter, Float, Optional. 1.5 The \(\gamma\) hyper-parameter that determines the steplength of weight learning.
d1 Hyperparameter, Float, Optional. 0.4 The \(\delta_1\) hyper-parameter. Applies only if prune=True.
d2 Hyperparameter, Float, Optional. 0.1 The \(\delta_2\) hyper-parameter. Applies only if prune=True.
tol Hyperparameter, Float, Optional. 0.01 Controls the convergence precision. This tolerance threshold represents the minimum precision of the difference between the voter weight in an iteration and the voter weight of the previous iteration.
max_iter Hyperparameter, Integer, Optional. 50 Controls the maximum number of iterations before the voter weights converge.
pref Hyperparameter, Float, Optional. 0.0 The preference threshold. Applies only if aggregator=outrank.
veto Hyperparameter, Float, Optional. 0.75 The veto threshold. Applies only if aggregator=outrank.
conc Hyperparameter, Float, Optional. 0.0 The concordance threshold. Applies only if aggregator=outrank.
disc Hyperparameter, Float, Optional. 0.25 The discordance threshold. Applies only if aggregator=outrank.


DIBRA also includes an aggregate() function which receives the user-defined parameters and passes them to the DIBRA exposed function, that subsequently performs the aggregation of the ranked input preference lists. The arguments of aggregate() are:

Parameter Type Default Description
input_file String - Required, unless input_df is set. Empty String A CSV file that contains the input lists to be aggregated.
input_df Pandas DataFrame - Required, unless input_file is set. None A Pandas DataFrame that contains the input lists to be aggregated.
Note: If both input_file and input_df are set, only the former is used; the latter is ignored.
rels_file String, Optional. Empty String A CSV file that contains the relevance judgements of the involved list elements. If such a file is passed, FLAGR will evaluate the generated aggregate list/s by computing several retrieval effectiveness evaluation measures. The results of the evaluation will be stored in the eval_df DataFrame. Otherwise, no evaluation will take place and eval_df will be empty.
rels_df Pandas DataFrame, Optional. None A Pandas DataFrame that contains the relevance judgements of the involved list elements. If such a dataframe is passed, FLAGR will evaluate the generated aggregate list/s by computing several retrieval effectiveness evaluation measures. The results of the evaluation will be stored in the eval_df DataFrame. Otherwise, no evaluation will take place and eval_df will be empty.
Note: If both rels_file and rels_df are set, only the former is used; the latter is ignored.
output_dir String, Optional. Temporary directory (OS-specific) The directory where the output files (aggregate lists and evaluation) will be stored. If it is not set, the default location will be used.

 

Agglomerative weighted aggregation: The Agglomerative Python class
This class employs the implementation of the Agglomerative weighted method of Chatterjee et al., 2018. Similarly to all weighted methods, it derives from RAM, a base class defined in the RAM module. It inherits the flagr_lib connector from RAM, and through it, they obtain access to the FLAGR shared library. Its constructor determines the data types of the input arguments and the return type of the Agglomerative exposed function. Observe the similarity between the members of self.flagr_lib.Agglomerative.argtypes and the input arguments of the Agglomerative exposed function.

The constructor's arguments include:

Parameter Type Default Description
eval_pts Integer, Optional. Considered only if rels_file or rels_df is set. 10 Determines the elements in the aggregate list on which the evaluation measures (i.e. Precision, and nDCG) will be computed. For example, for eval_pts=10 FLAGR will compute Average Precision, \(P@1, P@2, ... P@10\) and \(N@1, N@2, ... N@10\).
c1 Hyperparameter, Float, Optional. 0.1 The \(c_1\) hyper-parameter of the algorithm.
c2 Hyperparameter, Float, Optional. 0.5 The \(c_2\) hyper-parameter of the algorithm.

Agglomerative also includes an aggregate() function which receives the user-defined parameters and passes them to the Agglomerative exposed function, that subsequently performs the aggregation of the ranked input preference lists. The arguments of aggregate() are identical to those of DIBRA.

 

Preference relations weighted method: The PreferenceRelationsGraph Python class
This class links to the FLAGR implementation of the weighted method of Desarkar et al., 2016. Similarly to all weighted methods, it derives from RAM, a base class defined in the RAM module. It inherits the flagr_lib connector from RAM, and through it, they obtain access to the FLAGR shared library. Its constructor determines the data types of the input arguments and the return type of the PrefRel exposed function. Observe the similarity between the members of self.flagr_lib.PrefRel.argtypes and the input arguments of the PrefRel exposed function.

The constructor's arguments include:

Parameter Type Default Description
eval_pts Integer, Optional. Considered only if rels_file or rels_df is set. 10 Determines the elements in the aggregate list on which the evaluation measures (i.e. Precision, and nDCG) will be computed. For example, for eval_pts=10 FLAGR will compute Average Precision, \(P@1, P@2, ... P@10\) and \(N@1, N@2, ... N@10\).
alpha Hyperparameter, Float, Optional. 0.1 The \(\alpha\) hyper-parameter of the algorithm.
beta Hyperparameter, Float, Optional. 0.5 The \(\beta\) hyper-parameter of the algorithm.


PreferenceRelationsGraph also includes an aggregate() function which receives the user-defined parameters and passes them to the PrefRel exposed function, that subsequently performs the aggregation of the ranked input preference lists. The arguments of aggregate() are identical to those of DIBRA.

The following interactive block diagram depicts the architecture of PyFLAGR and its linkage to the FLAGR shared library.

PyFLAGR architecture