Requirements Engineering Approaches for Big Data Project Development

Abstract

Belachew Regane, Sarah Beecham, Norah Power and Dagmawi Lemma

Context: Today’s digital world with millions of users results in vast amounts of data. This ‘big data’, characterized according to its volume, variety, velocity, and veracity, is impacting the lives of data users worldwide in many ways and has become important for day-to-day decision-making.

Problem: Requirements engineering (RE) - approaches used to engineer user requirements - plays an important role in software development in general. However, when it comes to big data applications, it is unclear which requirements engineering approaches apply. There is therefore a need to investigate this further.

Objective: This study aims to answer the following research questions: (1) How are requirements engineering (RE) activities performed to address the needs of stakeholders in the context of big data? (2) How are users' perspectives addressed in the RE activities in a big data context? (3) What are the requirements engineering approaches that have been proposed for big data project development?.

Method: To address our three research questions, we conducted a systematic mapping study focusing on requirements engineering and the existing requirements engineering approaches in software engineering in the context of big data projects.

Findings: A total of 787 papers were examined, with 720 papers found through string-based search and a further 67 through snowball search. From the total search results, 17 relevant papers were identified and reviewed by applying inclusion-exclusion criteria. Findings show that in the realm of Requirements Engineering (RE) activities, there is a notable lack of emphasis on requirements negotiation, validation, and prioritization. Additionally, there is a scarcity of knowledge, methods, techniques, and tools tailored for conducting requirements engineering within the realm of big data. The user's role and perspective in RE are insufficiently considered. Although the goal-oriented RE approach is somewhat acknowledged among proposed methods, it has drawbacks such as neglecting the user's viewpoint, being relatively static and general in requirement representation, struggling to adapt to changing requirements, and having its effectiveness as the primary RE approach questioned. This approach primarily focuses on addressing the 'why' aspect of the system rather than the 'how' which aids in decision-making.

Conclusion: Based on the findings, it is clear that there is a need for more research to be conducted to find a better way to have a suitable RE approach for big data application development.

PDF