Requirements Gathering is the Most Undertaught Area in Software Development
Studies show a shocking 70% of digital transformation projects fail (Sources: Boston Consulting Group and McKinsey[i]).
Another study conducted by Info-Tech Research Group[ii] shows that 70% of those failures are due to issues with requirements. If we take these two statistics together, we can infer that 49% of digital transformation projects will fail due to requirements. That’s almost half! Moreover, another study (Tecnova[iii]) indicates that 80% of rework can be traced to poor requirements gathering.
The NHS pulled the plug on a £11 billion system[iv]. The plan was to create the world’s largest single civilian system linking all parts of the NHS. However, it was scrapped as were not able to deliver. This is a public sector example of a project which has completely failed due to poor requirements at the early stage of project.
Despite these statistics, 70% of organisations still do not take effective action to adequately improve the quality of their requirements (Technova). Considering requirements gathering is the greatest pitfall in software development and costs the industry millions a year, we still do not pay enough attention to them.
Software requirements seem to be the ‘poor relation’ of the software development process. It seems to be under taught and under appreciated by the development community. The statistics above seem to prove that there is, indeed, an issue with the quality of requirements in software development.
The use and often misuse of the agile methodology also seems to suggest that some have given up trying to define detailed requirements for systems upfront. In many organisations, using an agile approach, the requirements are ‘fleshed out’ as the project progresses. Whilst this is certainly an approach it makes it very difficult to plan a fixed budget for a fixed set of functionality and indeed if using an agile approach with a fixed budget then the features generally will be reduced if the project begins to overrun.
We have conducted our own research and present below some evidence that the important area of requirements is given less credibility than other parts of software development. Much education and coverage is given to technology and getting things done but less to what we are building in the first place.
The lack of attention given to requirements gathering can be traced back to the teachings of software development at a university level.
Within our company many of the developers have admitted that requirements gathering was given no emphasis during their time at university. The input of various developers across various universities and timeframes of their degrees all agreed that teaching of requirements gathering was under taught. Some mentioned they did learn about some of the tools used to design systems such as UML, ERD Diagrams and User Stories. However, these do not cover all the areas of requirements required to develop software in the commercial world. It was highlighted that there was more focus on OOP concepts, databases, languages like Java, C#. Although these are all important, they become useless if you can’t feed in something of use, i.e, fully defined requirements.
We looked at a major top 10 University in the UK for Computer Science degrees. Their Software Engineering degree’s curriculum has hardly any mention of requirements gathering across the four years. Within the average twenty classes that will be taken by software engineer students only two of which explicitly cover requirements gathering. From that analysis, it can be concluded that possibly as low as a mere 2.5% of this university’s Software Engineering degree covers requirements gathering, as they only make up a quarter of those two classes. This example seems to chime with our conversations with those who have studied computer science or software engineering.
Similar results can be seen across all the Scottish Universities we researched online as well as in alternative routes to education such as more commercial software engineering conversion courses, whose curriculum also does not emphasise the importance of requirements gathering.
Considering requirements gathering is necessary for all digital projects and the evidence it is linked to failure then it is a concern this area is not given much consideration across education. Unlike coding languages which may vary project to project, having effective requirements gathering techniques is relevant every time.
Lack of Literature
Assessing the situation with a broader lens than just the teaching of requirements gathering in education you will see that there is significantly less literature in this area too when compared with other parts of the software development process. This graph illustrates the number of books on Amazon for various search terms of topics within software engineering. As you can see again, software requirements is significantly less covered than the other areas. A single coding language, of which there are many, has ten times more books on it than software requirements. Not every project uses the same language but every project should involve requirements gathering. It would make sense that there is less literature for a subject area which is ignored at university level and further conveys the issue at hand.
Google Search Results
Moreover, similar results can be seen within Google Search Results.
This graph conveys that there is significantly less content on Google surrounding software requirements than there is for other software development related subject areas. This further suggests that there is a lack of information out there on requirements . Can this can be attributed to the lack of education in this field and misunderstanding of its importance in general?
The points highlighted above illustrate how requirements gathering would appear to be under serviced across the board. Whether that is in the literature and online content or the teaching of software development. Considering it is proven to be the most troublesome area in software development, the fact that it is under taught in learning could be the reason requirements gathering is done so poorly.
Software requirements gathering is an often-overlooked skill and its contribution to project success is undervalued. Project success starts with correctly understanding what the system should do in the first place so start the project as you mean to go on and undertake a full, detailed and structured requirements gathering process at the very beginning of the project. If you don’t know what you are building in the first place is it any wonder that so many projects fail?
Our new product Requiment, guides you through prepared questions to determine a full and detailed requirement specification and project scope based on outcomes . This application acts as a virtual business analyst, guiding users through the requirements process, whilst providing insight and recommendations to the user based on the type of application being designed as well as trends from other users. Ultimately, our mission with Requiment is to make the process of software requirements capture more accurate, agile and efficient leading to more successful projects.