After my last post, I thought it may be worthwhile to just expand on what we mean in project management by Waterfall, or specifically the Waterfall sequential design process as used in software development.
The idea of this sequential design has been around for over 40 years for software, but originally owes its roots to manufacturing and construction industries. Although the Royce paper brought it to the fore, it was really a case of the great and the good, with the top-down command and control mentality interpreted it in their own mindset and so the concrete steps were set.
In its simplest form the Waterfall can be seen as 6 steps:
- Requirements analysis: This is the first and most critical step; the organization works to detail exactly what the requirement is and to detail it exactly. Everything it must do, how it will be used, how it should play with others, the performance. This all gets summarized and condensed into a formal requirements specification.
- Design: Specifically “systems design” this step consists of “the process or art of defining the hardware and software architecture, components, modules, interfaces, and data for a computer system to satisfy specified requirements.” This explains why the first step is key; the formal specification now drives everything. To cut to the chase, assuming all goes well this stage will create a number of design specifications which are used in the next step, the actual implementation.
- Implementation: This step is the formal build of the product as per the systems design specification. This is the world of the development team made up of the requisite skill set (coders, UX designers, QA, etc.) The work here should be built to standard practices and standards before being debugged, tested and plumbed into the architecture.
- Testing: Both individual components and full systems are now methodically put through the wringer. The QA team (not the QA from the Implementation) will run through multiple test cases to check the code works as expected. Any defects, are noted and reported back to the dev team to correct. Similarly, the documentation is reviewed to make sure what has been delivered meets the agreed specifications.
- Installation: So with the tests successfully completed and signed off, the product is polished and made ready to roll out to the eager customer. This may be a physical team onsite, or remotely, or a hosted service in the cloud, but a version is badged – say 1.0 – and rolled out read for showtime.
- Maintenance: As tempting as it is to take the money and run, there is one more step in the Waterfall. After go-live there may be updates made to the system to either fix bugs, add new features, or improve on what was rolled out. These changes should be made by numbered and tracked maintenance releases, or a hotfix for a quick bug fix.
What are the advantages?
The Waterfall has survived for this long, and is still going strong, because it works.
By breaking the cycle into distinct steps there are clear entry and exit points to allow clear tracking of progress against development goals, business case, and budget. By insisting on clear specifications (or, more specifically, by insisting the client state clearly their requirements up front and agreeing the design) there is a clear benchmark of schedule and expectations to hit.
Another advantage is that clear and unambiguous requirements up-front quality as the developer has a clear target and the testing is the same as will be in the formal QA.Finally, assuming that the same team is involved in requirements gathering, design, and then the production of the formal specifications, the process can facilitate the transfer of knowledge between the team members even if located in multiple locations.
Or should that be The Dark Side? Probably the greatest criticism of the Waterfall process is that, for modern software development, it is almost impossible to get the necessary clarity up front for specifications to work. Ken Schwaber illustrates this in his book, Agile Project Management with Scrum (2009, Microsoft Press.)
To quote Ken, “the vertical axis traces requirements complexity, and the horizontal axis traces technology complexity. The intersection of these two kinds of complexity defines the total level of complexity of the project. Almost all of today’s software development projects are complex. Those that are chaotic are unworkable, and some of their complexities must be resolved before work can progress. The third dimension of complexity is the people developing the software. They all have different skills, intelligence levels, experience, viewpoints, attitudes, and prejudices. Every morning, each wakes up in a different mood than the day before, depending on his or her sleep, health, weather, neighbors, and families. These people then start to work together, and the complexity level goes through the roof. Taking into account this last dimension—people—in addition to technology and requirements, I believe that the last “simple” project occurred in 1969, when one person from order processing at Sears Roebuck asked me to sort some cards and generate a report on an IBM 360/20.“
It gets worse; in another book by Schwaber and Beedle (Agile Software Development with Scrum) the following is recounted when they asked a world-class team of highly respected theorists in industrial process control led by Babatunde Ogannaike why the Waterfall was failing them.
This world-ranking team “were amazed and appalled that my industry [software], was trying to do its work using a completely inappropriate process control model. They said systems development had so much complexity and unpredictability that it had to be managed by a process control model they referred to as “empirical.” They said this was nothing new, and all complex processes that weren’t completely understood [or had changing inputs] required the empirical model [and not the defined process model]. … [Ogannaike] said my business was an intellectually intensive business that required too much thinking and creativity to be a good candidate for the defined approach. … He was particularly amused that the tasks were linked together with dependencies [in a PERT chart], as though they could predictably start and finish just like a well-defined industrial process.”
Add to this that clients quite often have an IKIWISI (I’ll know it when I see it) approach to design. Then add to that the chance the markets will move, or a competitor will bring out a new product that you need to riposte, or the Government decides it knows best and unloads a truck-load of oversight and regulations that you need in ASAP, and the idea of getting the design and spec right up front becomes sheer folly.
This was a key driver behind the Agile movement; rather than try and nail it all down up front, run a few iterations past the client and get real-world feedback.
So what next? There remains the case where a company is pretty much repeating their work; for example, deploying a standard software product. In this case, unless the client is seeking a significant level of customization, there should be little worry about using the Waterfall. However, should the client require a heavy level of bespoke code or features, the Waterfall could drown you.
That said, even if a company is rolling out an almost cookie-cutter product, they themselves may need to be able to add new features or fix bugs on a rapid basis – in which case, an Agile incremental development path would be better than Waterfall.
Know your poison.