Design and development methodologies for regulated software

While software development regulation is now emerging in the medical, energy, and security fields, the regulatory process tends to be slow and behind the technological curve. Indeed, unlike the regulatory environment, software development methodologies are quick to evolve. How can we ensure that we are using the best software development methodologies while creating a product that meets regulatory requirements? How can we create an approach that addresses the slow pace of regulation yet capitalizes on the quick evolution of software development methodology?

The interplay between software development and regulating agencies

The goal of software development regulation is to ensure the highest possible quality of the final product while protecting the user. Guidelines are established for the entire product development process — requirement, planning, design, testing/verification, and maintenance. The regulations applied to the requirement and design stages help regulatory agencies determine how the product will be used, and how it can best be tested. These stages of development create a requirement for detailed specifications about the product; based on these documents, regulatory agencies are able to determine what standards the system must meet. This usually results in a back-and-forth process between the developing party and the regulatory agency. The testing and verification stages involve strict guidelines. All of this design and testing allows regulatory agencies to ensure that the system operates as described and meets specific requirements. The last stage of regulation is the maintenance and distribution of the code — including responding to issues and ensuring ongoing quality. Whereas the overall code and libraries get reviewed, there are fewer guidelines on how code should be written. This is where developers need to ensure that their code is robust, secure, and maintainable.

Defining and comparing two methodologies for software development

We will discuss two widely accepted methodologies: Waterfall and Agile. Each has its strength and weakness for regulatory-compliant software development.

Waterfall is a methodical approach. Its linear format allows for each step to be easily understood. When each task is completed, the next one begins. The first stage involves the gathering of requirements, then design, implementation, verification, and finally, maintenance. For very well-defined projects with stable requirements, Waterfall is a great software development methodology, but its rigid format increases the cost, and often the time, spent in development — especially at the later stages of the cycle.

As is the case for other methodologies, this development methodology has certain trade-offs when used to develop regulated products. Waterfall meets the strict documentation requirements to comply with the regulatory standard. In a pure Waterfall methodology, all the documentation needs to be created and confirmed before moving on to the next stage. While this rigidity aligns with many regulatory agencies, it lacks the flexibility to quickly adapt to changes. Just ask yourself how many times a product changes from inception to implementation; yet this linear approach to development makes it difficult to add a feature or change the scope of a product or project. Deviations from the original plan cost time and money, requiring developers to either go back to the requirement stage or to execute hotfixes and “hacks” to achieve the desired performance, adding to the technical debt of the project. Besides, as experienced software developers know, this approach can lead to “spaghetti code”, where a program flow is tangled and twisted. The extent of the drawbacks is beyond the scope of this blog post; suffice it to say that Waterfall is to be avoided.

Agile focuses on developing the software in iterations that represent small increments of new functionalities. One of the benefits of this type of development is that it segments the code in separate portions, creating a neater code base. Another benefit of small, incremental additions is that each feature can be tested independently. But Agile doesn’t usually work for larger teams, or teams that are accustomed to the waterfall methodology. It is particularly useful in later stages of development, as well as for User-Interface-focused projects.

Unlike Waterfall, Agile provides considerable flexibility, and it segments projects into sprints. What makes Agile unique is its flexibility, which allows for a more adaptable, efficient methodology that segments tasks and the underlying code. The design is performed separately for each feature, and can be expanded at almost any time, since this methodology separates each feature into self-contained units. This method can potentially reduce the technical debt of a specific project: because of the quick iteration of Agile development, some documentation can be left for the end, since reviewing documentation and ensuring that every document is updated before every sprint can be time-consuming.

Mixed method development

Waterfall supports teams in that they do not have to continuously verify whether their product meets requirements and standards, because they are able to assess the compliance of their design from the onset of development. The trade-off is the rigidity of the process that makes it difficult to adapt and change in order to meet the demands of consumers, markets, and technological changes.

Agile, on the other hand, gives teams the flexibility needed to make changes and modifications on the go, but requires teams to actively assess whether each incremental change meets regulatory standards.

How can we address the requirement that development be properly documented, while also using a development strategy that is responsive and flexible? Well, we can combine the two methodologies to minimize overhead while ensuring best coding practices.

Hybrid methodology

The best approach is to begin by defining business goals and engineering requirements using the Waterfall method. Once goals and requirements are broadly defined and meet regulatory requirements, they can be broken down into higher-level system design components. Then we can use the Agile methodology to develop the specific features of the project. At the completion of the project, we return to the Waterfall method to verify the whole system.

At the high-level requirement gathering and design portion of the development process, we determine the business goal to be achieved, and the high-level components that will be needed to meet it. Because individual functions are not important for regulatory purposes at this point, they need not be perfectly specified. The regulatory agency might want to ensure that you are using proper bolts, libraries, and coding practices, but it doesn’t concern itself with how to write a loop, for example, or the ins and outs of how the system updates itself. This is where we should be using Agile development, as it will ensure the highest quality of code. The output of the previous stage should give us a backlog of tasks that will need to be implemented using Agile methodology. Different features can be added later, but must be kept within the scope of the project. For example, if you are designing an autonomous aircraft, you can’t easily change the foundation to develop an autonomous boat. That is to say, even the Agile method’s flexibility must be controlled to ensure successful and timely development. Once the product is developed, we can return to a more linear model to complete the verification stage.

By combining two methodologies, we can create an approach to developing software in highly regulated markets while limiting specific risks: time and technical debt. This combined methodology can also be used for less regulated fields. Every development team should focus on mitigating risks, developing robust systems, and ensuring future development. This is especially true when we consider the interconnectedness of technology innovation. Proper practices will allow us to feel safer and develop more robust systems.

Importance of software methodology

Recently, we released another blog post that highlighted the impact of data in our daily lives, and whether data should be shared with the ubiquitous Google and Facebook, among others. To reiterate, “Strong security is the responsibility of every software and product developer, and should be a priority.” This applies not just to the regulated markets but also to many other types of projects.