Rayyan Screening Conflicts: Frequently Asked Questions

Conflicts among review members can arise during the screening process in Rayyan, especially when there are disagreements over whether an article should be included or excluded. These conflicts must be resolved to ensure the accuracy of your systematic review. Below are answers to some frequently asked questions regarding conflict resolution.

1. What are conflicts, and why do they occur?

Conflicts in Rayyan occur when two or more reviewers make opposite or conflicting decisions about the same article. For example:

• Reviewer 1: Included, Reviewer 2: Excluded – Conflict

• Reviewer 1: Included, Reviewer 2: Excluded, Reviewer 3: Maybe – Conflict (The article will appear in both the Conflicts and Maybe list)

• Reviewer 1: Excluded, Reviewer 2: Maybe – No Conflict (The article will only appear in the Maybe list)

• Reviewer 1: Included, Reviewer 2: Maybe – No Conflict (The article will only appear in the Maybe list)

Note: A Maybe decision is a tentative or undecided decision and is not considered final. It will not be treated the same as "Include" or "Exclude." If an article has a "Maybe" decision from one reviewer, it will not be considered a conflict unless there are opposing final decisions (e.g., "Included" vs. "Excluded").

2. How can I view articles with conflicts?

To view the list of articles with conflicts:

1. Turn off Blind Mode:

   • Navigate to the review where you want to resolve conflicts.

   • Go to the Screening stage and locate the Blind Mode feature in the toolbar.

   • Toggle Blind Mode off (Note: Only the owner of the review can turn Blind Mode on/off).

2. Access the Conflict List:

   • Once Blind Mode is off, go to the "Filter by inclusion" facet in the Screening stage.

   • From the dropdown menu, select the Conflicts filter. This will display all articles where there is a disagreement between reviewers.

3. What should I do when I find conflicts?

Once you’ve accessed the list of conflicts, follow these steps:

1. Review and Discuss:

   • Examine the conflicting articles and discuss them with your team.

   • Discuss the reasons behind the different decisions, such as varying interpretations of inclusion criteria or different preferences for article relevance.

2. Reach a Consensus:

   • Collaborate with your team members to reach a consensus on whether to Include or Exclude each article.

   • Once an agreement is reached, all team members need to update their decision:

     • If the article should be included, any member who previously excluded it should update their decision to "Include."

     • If the article should be excluded, any member who previously included it should update their decision to "Exclude."

4. How can I resolve conflicts in the Full-text Screening stage?

The process for resolving conflicts is the same in the Full-text Screening stage. However, keep in mind:

• Conflicts may arise again in Full-text Screening, even if they were resolved during the initial screening.

• Follow the same steps as above to discuss and update decisions.

• It is recommended to resolve conflicts in the Screening stage before moving to Full-text Screening for a smoother workflow.

5. Does Rayyan resolve conflicts automatically based on the majority vote?

No, Rayyan does not resolve conflicts based on the majority vote. For a conflict to be resolved, all reviewers must align on the same decision. Even if the majority of reviewers agree on an inclusion or exclusion, the conflict remains until every reviewer updates their decision.

6. Can I override a conflicting decision?

Currently, Rayyan does not offer an automatic override feature. All conflicts must be resolved through discussion and consensus among reviewers.

7. How can I ensure that conflicts are fully resolved?

After updating the decisions of all reviewers:

• Return to the Conflicts filter to check if the article still appears in the conflict list.

• Once all decisions are aligned, the article should no longer show under Conflicts.

8. Is there a recommended order for resolving conflicts?

Yes, we recommend resolving conflicts during the Screening stage before proceeding to Full-text Screening. This helps ensure a cleaner and more organized workflow, minimizing confusion in the later stages.

💬 Still Need Help?

We’re here for you. Submit a support ticket and we’ll assist you personally.

And don’t forget to follow us on Twitter @rayyanapp for updates, tips, and tricks!