Software Development in Scientific Research
A case study of a familiar process in a nontypical context — Part I
- How to engineer useful software for a research project?
- How do the familiar software development practices differ in this context?
- What can research labs learn from the software engineering community?
This article presents our initial answers to these questions, derived after a year of dedicated research and development on our Clean Code and Design Educational Tool. Through this open process, our goal is to share our insights, gather advice for improvement, and contribute to enhancing the field’s maturity.
This article is the first part of a two-part series. Here we examine the inception of a research idea that entails some software development, discuss how we typically plan the execution of the research and development efforts, and review the requirements engineering practice for the resulting software.
Inception
Typical solution development starts with a specific problem, often one that brings monetary gains when solved. The solution providers then utilize their know-how, connections, and other strengths to solve the problem to the best of their abilities.
Scientific research starts with a question. The initial question is refined, analyzed from multiple aspects, and assessed for relevance and ambition. For this analysis, it helps to have a developed research taste [1].
An example of an initial research question might be:
Are teenagers angrier when communicating over Twitter?
After some refinement, the question might become:
How prevalent is cyberbullying among teenagers on social media?
The refined question is broader in scope (Twitter vs. Social media), considers a different property (angrier vs. cyberbullying), and defines a different outcome (yes/no vs. statistical data concerning prevalence). Such a question is often the first step of a research project.
The topic of cyberbullying on social media offers many research questions, and many of them do not require much software development. “How do teens perceive the prevalence of cyberbullying?” is a question that can be examined through a survey, adequately designed and distributed to many college students. “How often are comments reported for cyberbullying?” is a question that could be answered with the aid of a simple tool for scraping or querying such events from social media platforms. Finally, “How to design a system that efficiently detects cyberbullying?” is a question that requires much research, development, and empirical experimentation.
Planning
A young research group, like ours, suffers from the chicken and egg problem. To develop competency, we require funding for our ideas. To get funding, we need to develop both the group’s capacity and the idea sufficiently. Without the funds and related responsibilities, ideas are developed solely on enthusiasm, which is chaotic, challenging to manage, and with few opportunities for synergy.
We have seen that the very act of preparing a research proposal and applying for funding is an exercise that increases the group’s capacity and the idea’s maturity. Such proposals require the idea to be sufficiently analyzed and decomposed, calling for additional research and brainstorming to define the goals and vision for the research clearly. Furthermore, these proposals often require designing a project timeline (e.g., Gantt chart) highlighting key activities, related risks, budget requirements, and expected deliverables and milestones.
When a research proposal includes significant software development, any planning related to the development entails a challenging balancing act. On the one hand, the research group needs to be specific enough to convince the proposal reviewers of the validity of their idea. On the other hand, the group needs to leave enough room to adapt the feature set as they learn more about the domain.
In traditional software development, the stakeholders usually have a general idea of the requirements and milestones. While the details are often unclear (hence the need for agile [2]), epics and high-level features are known in advance. In scientific research, the details are usually unknown (hence the need for research). Furthermore, as we have discovered, a whole research direction may turn out to be invalid [3]. For comparison with typical software engineering, imagine that the development team found that a department of stakeholders they have been talking to for the last quarter has no connections to the organization ordering the software.
Because of this volatility in research, it is important to define goals and deliverables at an appropriate level of abstraction while avoiding too many details. Building on the previous example, we can define a sarcasm detection module as a deliverable for our cyberbullying detection solution. Such a name is better than a deep learning-based sarcasm detection module as it is less constraining and enables us to adapt as we learn more from the related literature.
Notably, a high-level generic name is not a substitute for well-developed ideas, so the project proposal should include promising avenues of research that we wish to explore and expect to give good results. The initial plan might be centered on deep learning-based sarcasm detection. However, it should recognize the risk that such an approach might not provide satisfactory results, and a hybrid solution might be better.
Related Work
Regarding requirements engineering, some empirical research heavily aligns with traditional software engineering and involves closely working with stakeholders or research subjects. However, most research requiring software development relies on scientific papers to define (at least the initial) set of requirements.
Another difference is that traditional software development seeks to provide turn-key solutions, enabling easy end-to-end integration into the context of an organization and comprehensive support for its users. This approach includes requirements for “little things” like user input validation, other security controls, full management support of the data processed by the system, error handling, and failure recovery.
Most, if not all, of the aforementioned “little things” are never part of the research group’s backlog. This is understandable as the group is burdened with research tasks and requirements targeting a novel segment of the problem domain. With limited time, they need to research the algorithms for the sarcasm detection module, develop a tool to collect and process the required data, run the algorithms, and act based on their results. Ensuring the tool works when a request timeouts or is deployed on another operating system is not in scope.
By setting aside the “little things,” much of the software developed as part of scientific efforts is difficult to reuse, limiting reproducibility studies and work that builds on the previous research [4]. While many of the “little things” truly have little priority in the context of proof-of-concept or lab-setting solutions, not all such features are valueless.
We should strive to improve certain aspects of the development process to improve the maturity of the field.
The question is, “What can be done with the limited resources of a research group?” Part of the problem could be addressed top-down, where scientific journals can require a higher level of quality for the developed solutions. However, this should be supplemented with a bottom-up approach, where researchers utilize the rich resources and practices nurtured by the software development community.
In the second part, we will explore the simple techniques that integrate into the design, implementation, and testing of our solutions that significantly improve the usefulness and visibility of the results from our research efforts.
[1] Feamster, N., 2013. Cultivating Your Research Taste. https://greatresearch.org/2013/09/13/cultivating-your-research-taste/
[2] State of Agile, https://stateofagile.com/
[3] Veritasium, The Biggest Myth in Education, https://www.youtube.com/watch?v=rhgwIhB58PA
[4] Crick, T., Hall, B. and Ishtiaq, S., 2017. Reproducibility in Research: Systems, Infrastructure, Culture. Journal of Open Research Software, 5(1).
