How to Resolve Screening Conflicts in Rayyan – Rayyan Help Center

Before you begin Conflicts can only be viewed and resolved when Blind Mode is off. Blind Mode can be turned off by the review owner, or on Enterprise plans by an Enterprise Owner or Enterprise Admin. Make sure all reviewers have completed their screening before you start.

A conflict is a reference where your team's decisions don't settle the outcome. How Rayyan decides that depends on the conflict resolution method your review uses: Consensus or Majority. This article covers how each method works, how to choose one, and how to resolve the conflicts that remain.

Choose a conflict resolution method

The method is set per review and applies to both Screening and Full-text screening.

Open your review and click Review settings in the top bar.
Select the Screening tab.
Under Conflict resolution method, choose Consensus or Majority.
Click Save changes.

Every review uses Consensus unless you change it.

Method	How a reference is decided	When it becomes a conflict
Consensus (default)	All reviewers must agree before the reference is resolved.	Any disagreement between reviewers.
Majority	A clear majority determines the final decision automatically.	Only when there is no clear majority, for example a 1 to 1 or 2 to 2 split.

Note Changing the method applies right away to every reference in the review, including ones already decided. For example, under Majority a reference with two Include decisions and one Exclude is included automatically. If you switch to Consensus, the same reference moves to the Conflict list, because not all reviewers agree. Switching from Consensus to Majority works the other way: any conflict with a clear majority is resolved automatically.

Best practice Choose your method before screening starts and record it in your protocol, so every decision in your review follows the same rule.

How Maybe decisions are treated

Maybe is treated as undecided under both methods. It does not trigger a conflict on its own and is not counted toward a majority. A conflict can only occur between final decisions (Include and Exclude).

Scenario	Result
Reviewer 1: Include / Reviewer 2: Exclude	Conflict
Reviewer 1: Include / Reviewer 2: Exclude / Reviewer 3: Maybe	Conflict (also appears in the Maybe list)
Reviewer 1: Exclude / Reviewer 2: Maybe	No conflict, appears in the Maybe list only
Reviewer 1: Include / Reviewer 2: Maybe	No conflict, appears in the Maybe list only

How to view conflicts

Go to the stage where you want to review conflicts: Screening or Full-text screening.
Make sure Blind Mode is off.
Click the Filter by inclusion dropdown next to the reference list.
Select Conflict. The list updates to show only references whose decisions don't settle the outcome under your review's method.

Common issue Not seeing any conflicts even though you know disagreements exist? Check that Blind Mode is off, since conflicts are hidden while it's on. Under Majority, also remember that a disagreement with a clear majority counts toward the majority decision and never appears as a conflict.

Resolve conflicts under Consensus

With the Conflict filter active, review each conflicting reference.
Discuss the disagreement with your team. Look at the reasons behind the different decisions, such as varying interpretations of your inclusion criteria.
Once you reach agreement, each reviewer updates their decision. If the team agrees to include, anyone who excluded the reference changes their decision to Include. If the team agrees to exclude, anyone who included it changes to Exclude.
When all reviewers have the same decision, the reference leaves the Conflict list automatically.
Repeat until the Conflict filter shows no remaining references.

Important Under Consensus, Rayyan does not resolve conflicts automatically. All reviewers must reach the same decision. Even if only one person disagrees, the reference stays in the Conflict list.

Resolve conflicts under Majority

Under Majority, Rayyan doesn't change any reviewer's decision. A reference with a clear majority counts toward that decision and appears under the Included or Excluded filter like any other decided reference, while each reviewer's own decision stays visible as they made it. The only conflicts left are ties, and a tie is broken by adding one more decision. If AI Reviewer is active in your review, its decisions count toward the majority the same way as any other reviewer's.

With the Conflict filter active, review the tied references.
Have another reviewer screen them. If you need someone new, invite them to the review first.
Once their decision gives one option a clear majority, the reference counts toward that decision and leaves the Conflict list.
If the decisions are still tied, add another reviewer and repeat.

Best practice To limit ties, plan for an odd number of reviewers on each reference when you use Majority.

Resolve conflicts in Full-text screening

The process in Full-text screening is the same: turn Blind Mode off, filter by Conflict, and resolve according to your review's method. Conflicts may come up again in Full-text screening even if they were resolved during title and abstract screening. This is expected, since Full-text screening is an independent stage with its own set of decisions.

Best practice Resolve all conflicts at each stage before moving to the next. Don't move to Full-text screening until all Screening conflicts are resolved, and don't move to Data Extraction until all Full-text screening conflicts are resolved. Carrying unresolved conflicts forward creates inconsistencies in your results.

What to do when a reviewer is no longer available

Under Majority, the simplest option is to invite another reviewer to break the tie. Under Consensus, if a conflict can't be resolved because one of the reviewers involved has left or isn't responding, use this workaround:

Turn Blind Mode off to view the conflicting references.
Filter by Conflict to isolate them.
Use Data Sampling to create a set from those conflicts. Select Filtered as the data source when creating the sample.
Turn Blind Mode back on so the designated conflict resolver isn't influenced by the existing decisions, including those made by the unavailable reviewer.
As the conflict resolver goes through the conflicting references, have them apply a label to flag their decision, for example Conflict - Include or Conflict - Exclude.
Once all conflicts are resolved, turn Blind Mode off again.
Move the eligible references to the next stage using the existing decision together with the label, for example Included + Label (Conflict - Include).

This approach keeps the original disagreement in your review history for transparency, since the unavailable reviewer's decision is never removed or changed.

The conflict history is not kept as a separate record after resolution. The customizations_log in your export records all decisions in sequence. If documenting your conflict resolution process matters for your protocol, keep an external record of the disagreements and how they were resolved.

Stage complete

Category 3: Collaborating with Your Team

Next: Screening Your References →