Developing a Scorecard to Rank Reviews

Share on linkedin
Share on facebook
Share on twitter

The client had created an app where users review products and upload their rating in the form of a video. The scope of the project was to come up with a set of rules and conditions to ensure that users who use the app regularly and give quality reviews of products get priority over those who produce poor quality reviews.

The app was still in testing and there was no data to actually drive the algorithm build.

The client had two different requests as part of this project.

  • A ranking algorithm based on the quality of the reviewer as well as the review.
  • Proposal for improvements in the data capturing methodology for a future empirical build.

Incline Solution

To overcome the lack of data a scorecard model approach was used. This allowed industry best practices as well as the data structure presented on the data supplied to be used. The client insisted that the model developed should be able to tell between a user who is trusted as a reviewer as well as a user who is popular in the app.

Essentially this meant coming up with a model that classifies the interactions on a users’ reviews as well as a model that looks at the reviewers’ basic use of the app. A popularity score was developed which is based off the interactions with the video and an authenticity score which is based off the reviewers’ profiles as well as the interactions with reviews.

The aforementioned models were conceived in order to determine users who are regarded as good reviewers as well being able to distinguish between users who use their public influence in order to get more viewings on their reviews.

The below figure gives a brief overview of how the process works:

The current data set was missing key variables needed to distinguish trusted reviewers. Thus, Incline suggested new variables be added to the app’s database that would improve metadata yield and user interaction. With these new variables a future algorithm build can use a more empirical build approach.


Incline used its model building experience to build a prototype algorithm. This took into account industry best practices, such as recency, as well as the clients request to distinguish between popular and trusted reviews. A number of different variables were offered to the developer team in order to develop a better data trail, more definitive variables in order to see the user’s preferences. With this approach a future build can provide a more empirical build approach where population segmentations can be applied and using variables that are proven to be statistically significant to the model build.

For more information on how Incline can help your company’s model build see our website.