Using Rayyan's 5-star Relevance Ranking

Rayyan's AI-powered 5 star relevance ranking system helps reviewers accelerate their systematic reviews by learning patterns in decision criteria as reviewers make decisions and using that learning to rate the probability that an article will be included.

To use the Rayyan 5-star relevance rating, follow the steps below:

1. Make at least 50 inclusion and/or exclusion decisions in your systematic review, with at least 5 decisions of both types (included and excluded). 

2. Click Compute Ratings

You will see the message: New ratings are being calculated while Rayyan's Artificial Intelligence engine calculates the likelihood of each article to be included in your literature review (5 stars is most likely).

Once it is finished, you will see the message: New ratings available. You'll also see star ratings, in the Ratings column at the right, ranging from zero stars - near zero likelihood of inclusion to 5 stars - very high likelihood of inclusion. Please note that all decided references by any collaborator at the time of computation won't have ratings.

3. You can now sort your review by least or most likely to be included by clicking on the sorting arrow besides the column name. When sorting on the 5 star rating system, the order is precise - in other words, the exact calculation dictates the order while the visible stars show an approximation.

4. As you make more decisions, you can compute ratings again at any time. The more decisions you make, the more accurate the ratings will be - as Rayyan learns more from each decision.