Model Evaluation

rankfm model tuning and evaluation functions

rankfm.evaluation.discounted_cumulative_gain(model, test_interactions, k=10, filter_previous=False)[source]

evaluate discounted cumulative gain wrt out-of-sample observed interactions

Parameters:
  • model – trained RankFM model instance
  • test_interactions – pandas dataframe of out-of-sample observed user/item interactions
  • k – number of recommendations to generate for each user
  • filter_previous – remove observed training items from generated recommendations
Returns:

mean discounted cumulative gain wrt the test users

rankfm.evaluation.diversity(model, test_interactions, k=10, filter_previous=False)[source]

evaluate the diversity of the model recommendations

Parameters:
  • model – trained RankFM model instance
  • test_interactions – pandas dataframe of out-of-sample observed user/item interactions
  • k – number of recommendations to generate for each user
  • filter_previous – remove observed training items from generated recommendations
Returns:

dataframe of cnt/pct of users recommended for each item

rankfm.evaluation.hit_rate(model, test_interactions, k=10, filter_previous=False)[source]

evaluate hit-rate (any match) wrt out-of-sample observed interactions

Parameters:
  • model – trained RankFM model instance
  • test_interactions – pandas dataframe of out-of-sample observed user/item interactions
  • k – number of recommendations to generate for each user
  • filter_previous – remove observed training items from generated recommendations
Returns:

the hit rate or proportion of test users with any matching items

rankfm.evaluation.precision(model, test_interactions, k=10, filter_previous=False)[source]

evaluate precision wrt out-of-sample observed interactions

Parameters:
  • model – trained RankFM model instance
  • test_interactions – pandas dataframe of out-of-sample observed user/item interactions
  • k – number of recommendations to generate for each user
  • filter_previous – remove observed training items from generated recommendations
Returns:

mean precision wrt the test users

rankfm.evaluation.recall(model, test_interactions, k=10, filter_previous=False)[source]

evaluate recall wrt out-of-sample observed interactions

Parameters:
  • model – trained RankFM model instance
  • test_interactions – pandas dataframe of out-of-sample observed user/item interactions
  • k – number of recommendations to generate for each user
  • filter_previous – remove observed training items from generated recommendations
Returns:

mean recall wrt the test users

rankfm.evaluation.reciprocal_rank(model, test_interactions, k=10, filter_previous=False)[source]

evaluate reciprocal rank wrt out-of-sample observed interactions

Parameters:
  • model – trained RankFM model instance
  • test_interactions – pandas dataframe of out-of-sample observed user/item interactions
  • k – number of recommendations to generate for each user
  • filter_previous – remove observed training items from generated recommendations
Returns:

mean reciprocal rank wrt the test users