Risk of Bias assessment helps you evaluate how much confidence to place in the results of each study you include in your review. Rayyan lets you run this assessment directly against your included articles, using a tool suited to the study design. This article covers how to add articles, choose a tool, complete an assessment, and work with your team. Each tool has its own article with full guidance on completing that assessment.
Add articles to your assessment
- Open your review and click the Risk of bias tab.
- Click Add articles. In the Add Data window, choose a data source from the dropdown: Screening, Full text screening, or Data extraction. Then choose Included Articles, Maybe Articles, or Labeled Data. If you choose Labeled Data, a label list appears. Pick the label or labels to pull from, then click Add.
- The selected articles appear in the assessment table. The Full text column shows whether a PDF is attached to each one.
Choose a tool for each article
Rayyan supports three tools: RoB 2, MMAT (Mixed Methods Appraisal Tool), and NOS (Newcastle-Ottawa Scale). Each is suited to different study designs, and availability depends on your plan.
| Tool | Study design | Plan availability |
|---|---|---|
| RoB 2 | Randomized controlled trials (RCTs) | All plans |
| MMAT | Mixed methods, qualitative, quantitative RCT, quantitative non-randomized, and quantitative descriptive studies | Advanced plan and above |
| NOS | Cohort and case-control studies | Advanced plan and above |
- Open the dropdown in the Tool column for an article.
- Choose RoB 2 (randomized trials), or hover over MMAT or NOS to open its submenu.
- For MMAT, choose the category that matches the study: Qualitative, Quantitative RCT, Quantitative Non-Randomized, Quantitative Descriptive, or Mixed Methods. For NOS, choose Case-control or Cohort.
If the tool you need isn't listed, click Vote for the next tool at the bottom of the same dropdown. If your plan doesn't include a tool, selecting it shows an upgrade prompt.
Complete an assessment
Once a tool is selected for an article, click Start assessment or double-click the article to open it. The article's full text opens with the assessment panel for your selected tool alongside it. Steps and scoring differ by tool, so refer to the article for the one you're using:
- RoB 2: How to Complete Your Assessment
- MMAT: How to Complete Your Assessment
- NOS: How to Complete Your Assessment
Working with a team
Risk of Bias supports the same collaborative workflow as Data Extraction.
- When Blind Mode is on, each reviewer or collaborator assesses articles independently without seeing the others' answers.
- When Blind Mode is off, Rayyan flags any differences between reviewers or collaborators at the domain or overall judgment level as unmatched answers.
- The team reviews the flagged articles together, discusses, and updates their answers to reach alignment. There is no automatic resolution.
Save progress and mark as completed
- You can save at any point, even if the assessment isn't finished. Your answers are there when you come back.
- The status changes to In progress after the first save.
- Mark as completed becomes available once the required questions for the selected tool are answered. Requirements vary by tool, so check the article for the one you're using.
- Once completed, the status changes to Complete.
Continue to: RoB 2: How to Complete Your Assessment