Definitions

Last updated on 2024-11-19 | Edit this page

Overview

Define the concepts of Open science, Reproducible research, and Sustainable software.
Define related concepts like Reliability and Usability.
Define related features for each concept.

Three introductory concepts informed our approach to this material.

Definition:

Make data inputs, software, and data outputs freely available by publishing all of them with open licences (Irving et al. 2021), to facilitate project reuse.
Also make their dissemination available to any member of an inquiring society, from professionals to citizens (ORION Open Science, 2020), to improve its transparency and public ownership.

Related feature:

Open Licence: An open licence that permits reuse using MIT or GPL for software (Choose a License, 2023) and CC BY or CC0 for data, prose and other creative products (Creative Commons, 2023, Irving et al. 2021).
DEI: Diversity, Equity, and Inclusion. There are four CHAOSS’s metrics for projects: Project Access, Communication Transparency, Newcomer Experience, Inclusive Leadership. (The GitHub Blog, 2023)

Definition:

Ensure that anyone with access to data inputs and software can feasibly generate the data outputs, both to check or build on them. (Irving et al. 2021)
Practice of describing and documenting the research process in such a way that another researcher can re-run the software on the same data input to get the same data outputs.

Related features:

Documentation strings: in one or two lines using active verbs to describe how inputs turn into outputs (Irving et al. 2021).
Literate programming is the practice of mixing code and descriptive writing in order to execute and explain a data analysis simultaneously in the same document (Eli Lilly and Company, 2022).
Software descriptions structured in four types with complementing purposes: tutorials, how-to guides, technical references, and explanations. (Documentation System, 2023).

Related concepts:

Reliability: Result consistency across many repetitions of the same experiment. (Dymocks Tutoring, 2022)
Usability: Capacity to provide conditions to perform the tasks safely, effectively, and efficiently. (Wikipedia, 2023)

Definition:

The ease with which to maintain and extend rather than replace. (Irving et al. 2021) It depends on the quality of the software, the skills of the potential maintainers, and if users can afford to keep up to date (how much the community is willing to invest).

Related features:

Modular code: Build programs out of short, single-purpose functions with clearly-defined inputs and outputs (Wilson et al, 2017)
Unit testing: Small test of one particular feature of a piece of software. (Wilson et al, 2017)
Version control: Keeping track of changes that you or your collaborators make to data and software. (Wilson et al, 2017)
Community around software: Users and collaborators that can communicate effectively with maintainers given the software documentation and by public or private platforms like chat channels, video conferencing, and more. (Wilson et al, 2017)

Often used interchangeably but use them differently can help to differentiate the characteristics of a project (Irving et al. 2021):

We can have open science projects without documentation, thus not reproducible.
We can have an automated and documented project not open to the public, thus not open science.
We can have open and reproducible software but lack incentives for maintainers, thus not sustainable.

The definitions of Open science, Reproducible research, and Sustainable software help us identify their specific software features.
Differentiating these concepts helps us to differentiate the characteristics of a project.