skip to content

Faculty of Mathematics and Natural SciencesIntelligent Methods for Earth System Sciences

Evaluating Software Quality of Research Software

Within this project, interviews were conducted and analysed with scientists from our institutes associated with the KPA. The focus is on the quality requirements for software that is created and used by scientists themselves as part of their scientific work. Since this research software (RSE) is often accompanied by complex issues (e.g. model simulations and data analysis), it is created by scientists who are experts in the respective field, while the actual processes of software development tend to run in the background. Since modern science is increasingly dependent on the use of software and the results are based on this software, the question of the quality of this research software must also be given higher priority.

 

1 / 4

The main questions in the interview are:

  • RQ1: How do researchers elicit, document, and analyze requirements for research software?
  • RQ2: What processes do researchers follow and which parts relate to requirements?
  • RQ3: What are requirements-related challenges that researchers face?

The interview analysis shows that the primary quality feature of the research software named by the researchers is the functional correctness of the research software, followed by performance. Other quality characteristics are consistency, portability and installability. Reproducibility was named as a decisive quality feature across almost all subject areas.

Furthermore, it turns out that the requirements for the software to be created are often not defined at all. As a result, the process of validation and verification based on previously created requirements hardly exists. The management from the requirements to the test is not clearly defined. One problem is that the processes of structured software development are simply not known in most cases because they are not learnt or taught. Given the often small size of the teams or working groups, this is seen as too much time spent for too little benefit. There is no reward for high-quality software.

Andreas Vogelsang and his colleagues conducted and analysed these interviews and documented the results in detail in a publication together with the future challenges and visions for research software.

Bajraktari, M. Binder, A. Vogelsang: "Requirements Engineering for Research Software: A Vision“, In: 32nd IEEE International Requirements Engineering (RE) 2024 conference, https://arxiv.org/abs/2405.07781