Enabling reproducibility in Secure Data Facilities

In the context of the constantly evolving controlled/confidential data landscape reproducibility has become a growing concern for journals, researchers and data service providers. Personal/confidential and sensitive data can only be accessed via a multi-stage application process, hence it has long been recognised by journals that peer reviewers cannot directly reproduce scientific research based on these data due to access constraints and needed resources.

As a commonly accepted workaround, the code can be submitted to the journal along with the paper. Researchers can use the standard output request channels to ask for the code files to be released from the Secure Data Facility/Trusted Research Environment (TRE). However, well-established Secure Data Facilities are increasingly receiving inquiries on better alternatives to facilitate and assist more robust and transparent reproducibility for peer reviewers before journal article publication.

Our talk will examine possible alternative solutions for how Secure Data Facilities could handle the new, more transparent, reproducibility requirements for personal/confidential data, including the very practical implications of proposed processes. Theoretically, options could range from certified reproducibility provided by a tailor-made service (with-)in the Secure Data Facility to allowing access for peer reviewers in the Secure Data Facility. We will also discuss these options in terms of consequences and potential challenges for non-blind versus blind peer review (single- and double-blinded). We will outline the considerations each option would require as well as its very practical implications. The main aim of the presentation is to help pave the way for enabling the reproducibility of scientific research based on controlled/confidential data in future, and on how Secure Data Facilities can better support the peer review process.