Detecting duplicate references/articles in a review – Rayyan Help Center

To Detect duplicates in Rayyan, follow the steps below:

1. Visit the Review Data Screen.

2. Click Detect Duplicates.

After you have imported all your references collected from various sources into Rayyan, you will want to initiate the deduplication process by clicking the Detect duplicates button.

Detect Duplicates.png

2. Click Proceed. Detect Duplicates 2.png

3. You will see the Detecting Duplicates status message appear.

Note Detecting duplicates can be performed once on each unique dataset in a review. If you wish to run it again, you will have to add or delete references/articles from your review first. If you see the message: A search must be added or deleted before running duplicate detection again, this means that you have already run duplicate detection on your dataset. Please continue to the next step to resolve the duplicates.

Once duplicate detection has completed, you will see the message: n duplicates found. You will also see the Possible Duplicates facet Under the Imported References section.

4. Under Possible Duplicates, click Unresolved to filter by unresolved duplicates.

Possible duplicates.png

5. Click on the record you wish to resolve.

6. Click Resolve Duplicates.

7. The detected duplicates will be listed side by side, each with a confidence percentage. Compare these duplicates and choose to keep the left, right, or both articles. Selecting "Keep Right Article" will result in the deletion of the left article, and vice versa.

Additionally, you can notice the number of articles that are considered duplicates and combined under one group. When resolving, keep an eye on which articles the system is comparing to ensure accurate and efficient duplicate management.

You can see that the counts will be updated on the Possible Duplicates facet. You might need to refresh the page to get the latest update.

Note If Rayyan identifies 2 duplicates, you'll have the option to make a decision and keep just one of them. The system will automatically move the other duplicate to the "Resolved" status. This is to ensure that you do not delete them all by mistake.

In addition, Rayyan offers subscribers the Systematic Auto-Resolver feature. The Systematic Auto-Resolver enables you to select and combine your own deduplication criteria. To use it, follow these instructions:

8. Run the Systematic Auto-Resolver.

Please note, if your membership does not grant you access to this feature, you will be prompted to update your membership to obtain access.

9. Select the Duplicate Resolving Criteria and Percentage to Auto-resolve.

The duplicate resolving criteria consist of the following:

Title
Author
Journal
Year
Pages
DOI number

By selecting any of these criteria, Rayyan will compare references for exact matches and automatically resolve them as duplicates.

Additional Criteria:

Overall Similarity Percentage: Compare the overall similarity percentage of the references. For example, selecting an overall similarity percentage of 80% will deduplicate all references that are at least 80% similar.
Imported References: Include the search methods as a criterion for deduplication. This allows you to select which imported reference file to default to when duplicates that match your criteria are found from different searches, preserving the duplicate reference from your selected search.

Advanced Auto-Resolver Settings:

Text Normalization: Converts all letters to lowercase and ignores punctuation, ensuring consistent comparisons. This setting is off by default.
Articles Customization Protection: Protects any article that has been assigned a decision, ensuring that you don't mistakenly delete an already screened article. This setting is on by default.

If you combine the Overall Similarity Percentage with the exact match criteria, it will provide you with additional safeguards to ensure the most precise deduplication possible.

10. If you use the Overall Similarity Percentage criteria, you will receive a message to confirm you would like to proceed. Select Cancel or Resolve Anyway to run the Auto-Resolver.

Please be advised that you won't be able to undo this action once you confirm. All the duplicates of the percentage you selected and higher will be resolved all at once, which means one copy of the article will be moved to Resolved and the remaining copies will be Deleted. Again, please be advised that you won't be able to undo this action once you confirm.

11. Pay attention to the notifications.

Rayyan will start processing all the references and will notify you of the start and end of the process and report the results.

12. View the results.

The Auto-Resolver will automatically populate the Resolved articles filter under the Possible Duplicates facet with the articles it kept, and move the duplicate copies to the Deleted filter.

The icon appearing beside the percentages indicate that the respective references are Auto-Resolved duplicates, while the references without it are those that have been resolved manually.

Please note that some features are available only with certain memberships. Please always check the memberships page to determine if your current membership includes a specific feature. If you would like to change your membership, please visit change my membership.

Related articles