Risk‐of‐bias VISualization (robvis): An R package and Shiny web app for visualizing risk‐of‐bias assessments

There is currently no generic tool for producing figures to display and explore the risk‐of‐bias assessments that routinely take place as part of systematic review.
In this article, the authors, therefore, present a new tool, robvis (Risk‐Of‐Bias VISualization), available as an R package and web app, which facilitates rapid production of publication‐quality risk‐of‐bias assessment figures. A timeline of the tool’s development and its key functionality is also presented.


R script BestDose


## sample size
n1 <- 8
n2 <- 16
n3 <- 32
n4 <- 64

## iterations m <- 500
yfig <- vector()
xfig <- vector()
facetfig <- data.frame()

bda1 = data.frame(matrix(rnorm(400), nrow=100))

for (j in c(n1, n2, n3, n4)) {
for (i in 1:m) {
bda2 = bda1[sample(nrow(bda1), j), ]
pa <- oneway.test(values ~ ind, stack(bda2))$p.value
xfig[i] <- pa
bda3 <- subset(bda2, select = – X1)
bda3$colMax <- apply(bda3, 1, function(x) max(x))
bda3$X1 <- bda2$X1
pt <- t.test(bda3$X1, bda3$colMax, paired=TRUE)$p.value
yfig[i] <- pt
facetfig <- rbind(facetfig,cbind.data.frame(N=rep(j, m), xfig, yfig))
p <- ggplot (facetfig, aes(x=xfig, y=yfig)) + geom_point(shape=16, size=4)
p + facet_grid(. ~ N, labeller = label_both) +
theme(strip.text = element_text(face = “bold”, size = 24), strip.background = element_rect(fill = “lightblue”)) +
annotate(“rect”, ymin = 0, ymax = 0.05, xmin = 0.05, xmax = 1, alpha=.1, fill=”blue”) +
scale_x_log10(labels=trans_format(“log10”, math_format(10^.x))) +
scale_y_log10(labels=trans_format(“log10”, math_format(10^.x))) +
geom_hline(yintercept = .05) + geom_vline(xintercept = .05) +
labs(y=”p value for best-dose t-test”, x=”p value for all-dose ANOVA”) +
theme(axis.title = element_text(size = 24),
axis.line = element_line(colour = “black”), panel.grid.minor = element_blank(),
panel.grid.major = element_blank(), axis.text = element_text(size = 20))

Signing digitally with Microsoft

A. Background & Definitions

Microsoft Office (incl. Word, Excel and PowerPoint) offers the possibility to create a digital signature (digital ID) as an electronic stamp of authentication on digital documents.

​B. Guidance & Expectations

To create a digital signature, a signing certificate is required:

  • you can create your own digital certificate for free
  • to allow verification of the authenticity of your digital signature, you can purchase a digital certificate from a reputable third-party certificate authority for a fee – link​

To create a digital signature:

  • ​in the document or worksheet, place your pointer where you want to create a signature line
  • on the Insert tab, in the Text group, click the Signature Line list, and then click Microsoft Office Signature Line – link​       

To add another layer of security, the record can be signed not only by one person (e.g. a scientist who performed the experiment) but by several people (e.g. by the author and the Principal investigator):

  • if two (or more) people are to sign the document, it is required to create a signature line for every person who is supposed to sign. This is necessary as the document cannot be modified any longer once a first person has signed. The next person to sign the document can then select the respective signature line and add his/her signature (as this is not considered a document modification)​


  • ​​​​Once a document is signed, the document can not be modified anymore. Even minor changes undone/reverted before saving, are considered as a modification

C. Resources

Information on digital signature certificate – link

Microsoft support information about adding signature lines to Office documents – link

Octopus: The primary research record

Created in partnership with the UK Reproducibility Network, the Octopus platform is free to use and publishes all kinds of scientific work, whether it is a hypothesis, a method, data, an analysis or a peer review.

The principle behind Octopus is to break the ‘unit of publication’ down from being a ‘paper’ to its constituent parts.

The Octopus beta version is now available and can be accessed at https://demo.science-octopus.org/.
On the site, there is a feedback link in the footer of each page which you can use to report bugs or thoughts and if time allows, there’s a full feedback form to provide more detailed information HERE.

Please be aware that anything uploaded to the site will be publicly visible (please be careful not to accidentally upload anything copyright or inappropriate) and all content will be deleted after this round of testing.

Using Microsoft OneNote as an ELN

Electronic research data documentation can be achieved with either comprehensive software applications or “do-it-yourself” solutions. Whereas the first is often quite cost intensive, the latter one is usually labor intensive to set it up properly. An interesting intermediary solution is provided by Microsoft OneNote:
Guerrero and colleagues compared different electronic lab notebook (ELN) applications and found that Microsoft OneNote is very competitive in its capabilities compared to dedicated solutions (LINK). A survey also revealed that its users preferred OneNote compared to other solutions. Last year, the same group described the adaptation of OneNote to the lab environment (LINK) and touched on the following aspects:

  • Structure and labeling: Here, the research unit needs to agree on the organization of generated research data and on having a convention for naming and classifying different experiments
  • Data acquisition: OneNote allows saving of raw data within the program and hyperlinking of larger files
  • Data presentation: Tools are described for data presentation and connecting with other Microsoft applications, e.g. integration of Microsoft Excel
  • Sharing: OneNote provides comprehensive sharing features which makes collaborations easy
  • Storing, securing and legalizing: With specific settings and usage of Microsoft SharePoint it is even possible to be FDA Code Title 21 Part11 compliant and files can be backed up in a OneNote file format – only the implementation of legally binding time stamps does not seem to be currently possible.

Overall, this article provides many practical tips on establishing OneNote as an alternative to a conventional lab notebook.
An interesting resource for additional reading material is the blog by Dr Martin Engel. In his posts he describes the transition from paper-based lab notebooks to OneNote (LINK) or the use of OneNote (LINK) in more detail.

Sample Size Matters – An App Exploring Sample Size and Power

Problems with statistical analysis and data visualization in published papers (REF1REF2REF3REF4) have led to calls to improve training for biomedical scientists. Although statistics are essential for biomedical research, statistics training is not always required to complete a PhD.
A new set of free online simulators allow readers visually to explore statistical concepts. Each simulator includes a series of questions. Users can adjust sample sizes and other parameters to explore concepts and find answers. Sample questions include:

  • Distributions and summary statistics: How many data points do you need to determine whether the data are normally distributed?
  • Power and p-values: How likely is it that you will get a significant p-value if your hypothesis is true, and the power for your study is 50%? How likely is it that you will get a significant p-value if your hypothesis is false and there is no effect?
  • Effect sizes: Why doesn’t a smaller p-value mean that you’ve found a larger effect? Why do underpowered studies that find significant differences tend to overestimate the size of the effect?
  • Publication bias: Why is publication bias a greater problem for small effects, compared to large effects? Why is publication bias more likely to occur in fields where researchers conduct small, underpowered studies?

Scientists can use the simulators independently, or integrate them into courses or laboratory training. The simulators are also included in an online course: “Sample Size Matters: Misconceptions about Graphs and Statistical Analyses in Lab and Clinical Research”. Each unit examines misconceptions about data visualization and statistical analyses that are common in fields that regularly perform small sample size studies. Examples of misconceptions include “bar graphs can be used to present continuous data”, “a smaller p-value means you’ve found a larger effect”, or “being underpowered only matters if you don’t find a significant difference”. The course is self-paced, takes approximately ten hours to complete and costs $50. Scientists can find more information or register HERE.