Detecting duplicate references/articles in a review (Rayyan Beta) – Rayyan Help Center

Rayyan automatically detects and resolves 100% duplicate articles from searches. Rayyan then offers duplicate resolution for statistically likely duplicates by comparing the title, author, journal and year. Rayyan was ranked number one for its combination of accuracy and sensitivity for deduplication in an independent third-party study.

To Detect duplicates in Rayyan, follow the steps below:

1. Detect Duplicates: After you have imported all your references collected from various sources into Rayyan, you will want to initiate the deduplication process by clicking the Detect duplicates button. Detect Duplicates.png

2. Click Proceed. Detect Duplicates 2.png

3. You will see the message: Duplicate detection is running, please wait. Detect Duplicates notification.png

Note Detecting duplicates can be performed once on each unique dataset in a review. If you wish to run it again, you will have to add or delete references/articles from your review first. If you see the message: A search must be added or deleted before running duplicate detection again, this means that you have already run duplicate detection on your dataset. Please continue to the next step to resolve the duplicates.

Once duplicate detection is finished, you will see the message: x duplicates found. You will also see the Possible Duplicates facet Under the Imported References section.

4. Under Possible Duplicates, click Unresolved to filter by unresolved duplicates. Detect Duplicates 4.png

5. Click on the record you wish to resolve.

6. Click Resolve Duplicates.

7. The detected duplicates will be listed side by side, each with a confidence percentage. Compare these duplicates and choose to keep the left, right, or both articles. Selecting "Keep Left Article" will result in the deletion of the right article, and vice versa.

Detect Duplicates 6.png

You can see that the counts will be updated on the Possible Duplicates facet. You might need to refresh the page to get the latest update. Detet Duplicates 7.png

Note If Rayyan identifies 2 duplicates, you'll have the option to make a decision and keep just one of them. The system will automatically move the other duplicate to the "Resolved" status. This is to ensure that you do not delete them all by mistake.

In addition, Rayyan offers subscribers the Auto-Resolver feature for a large dataset of articles to De-duplicate. The Auto-Resolver offers the flexibility for users to choose their desired similarity percentage for automatic resolution. By visually inspecting all the possible duplicates with a high similarity percentage, you can confirm that they are indeed duplicates and run the Auto-Resolver to resolve all the possible duplicates at the same time. If, however, you detect any high similarity possible duplicates that are not duplicates, then you may resolve those manually first or you may select the next higher percentage that only contains confirmed duplicates, before running the Auto-Resolver.

8. Run the Auto-Resolver

9. Select the Duplicate Percentage to Auto-resolve: Based on your confirmation that the remaining high similarity articles contain only duplicates, select the highest duplicate percentage that you have confirmed to auto-resolve. For example, if you confirmed that all the remaining possible duplicate articles listed as 90% or higher are in fact duplicates, then select 90%.

Resolving Duplicates and Uploading Data on Rayyan_ Step-by-Step Guide - Step 2.png

10. Select Cancel or Resolve Anyway to run the Auto-Resolver: Please be advised that you won't be able to undo this action once you confirm. All the duplicates of the percentage you selected and higher will be resolved all at once, which means one copy of the article will be moved to Resolved and the remaining copies will be Deleted. Again, please be advised that you won't be able to undo this action once you confirm.

Resolving Duplicates and Uploading Data on Rayyan_ Step-by-Step Guide - Step 3.png

11. Pay attention to the notifications: Rayyan will start processing all the references and will notify you of the start and end of the process and report the results.

12. View the results: The Auto-Resolver will automatically populate the Resolved articles filter under the Possible Duplicates facet with the articles it kept, and move the duplicate copies to the Deleted filter.

The icon appearing beside the percentages suggests that the respective references are Auto-Resolved duplicates, while the references without it are those that have been resolved manually.

Auto-Resolver Icon.png

Please note that some features are available only with certain memberships. Please always check the memberships page to determine if your current membership includes a specific feature. If you would like to change your membership, please visit change my membership.

Related articles