Random graphs for performance evaluation of recommender systems

The purpose of this article is to introduce a new analytical framework dedicated to measuring performance of recommender systems. A standard approach is to assess the quality of a system by means of accuracy related statistics. However, the specificity of the environments in which recommender systems are deployed requires paying much attention to speed and memory requirements of the algorithms. Unfortunately, it is implausible to assess accurately the complexity of various algorithms with formal tools. This can be attributed to the fact that such analyses are usually based on an assumption of dense representation of underlying data structures. In real life, though, the algorithms operate on sparse data and are implemented with collections dedicated for them. Therefore, we propose to measure the complexity of recommender systems with artificial datasets that posses real-life properties. We utilize a recently developed bipartite graph generator to evaluate how the state-of-art recommender system behavior is determined and diversified by topological properties of the generated datasets.