What is an Agile Methodology? How does it beat Waterfall?

Using an agile methodology for project management can help CEOs, organizations, managers, teams and individuals rapidly adapt to change, beat slower competitors and win profitable markets. Agile methodologies were created to prevent the frequent and expensive manufacturing and development failures that arose in “waterfall” or “ad hoc” projects.

Most people tackle large projects using an intuitively obvious approach called “the waterfall method”: plan a sequence of activities upfront (for example: design, prototype, build, test, deploy), then focus on one type of activity after another until they have completed the whole thing. Only in the end do they have something of value. From software development to car manufacturing, the modular sequencing in waterfall has proven extremely risky, resulting in multi-million dollar project cancellations and corporate bankruptcies. The problem arises from the enormous costs that precede real-world testing. There’s a lot of risk riding on the final stage.

Different industries found different frameworks to cut the risk, and each framework is an “agile methodology.” Toyota developed the Toyota Production System (TPS) to manufacture cars “just in time”. The US Air Force developed the OODA Loop concept (Observe Orient Decide Act) for rapid tactical decisions. Software developers created Extreme Programming (XP) to create higher quality code. Managers created Scrum to focus teams on the most profitable work. Product managers created Lean Startup to discover markets. William Shewhart created and Edwards Deming popularized the Plan-Do-Study-Adjust cycle (alias PDCA plan-do-check-act) cycle to improve industrial quality. David Allen created Getting Things Done (GTD) to help individuals realize their goals.

These agile methodologies share a common approach: structure work to exploit change by running a never-ending sequence of controlled, time-limited, adaptive experiments. At the conclusion of each experiment, the organization assesses the outcome, re-orients its processes (and sometimes its goals), establishes a hypothesis and starts another experiment. An agile methodology allows organizations to operate profitably amidst upheaval and surprise from markets, suppliers, competitors, employees, laws, finance and tools.

An agile methodology is not an obvious choice. An agile methodology requires more frequent testing, packaging and revision, adding labor and material costs. An agile methodology requires more discipline, adding training and coaching costs, as well. One might think waterfall could cost less than agile in stable situations, because its overhead seems lower, but it turns out such situations have become increasingly rare. Environments that once seemed stable now have disruptive, creative competitors that threaten market leaders.

The results are in: agile methodologies have obviously beaten waterfall in most competitive settings. Organizations that embraced an agile methodology have become shining examples of productivity, quality and innovation. Toyota became the most profitable car manufacturer in the world using TPS. The highly maneuverable F-16 and F-18 fighter jets were designed to support the OODA Loop in the 1970s, and militaries still buy them almost 40 years later. Sabre increased productivity by 42% and reduced defects by 80% using XP. The PDSA cycle is widely used to improve health care quality. The US government saved its online insurance exchange, healthcare.gov, from a looming public disaster by adopting Scrum. Individuals all over the world reduce stress and reliably follow-through using GTD. Researchers acknowledge that users must exert significant effort to maintain the discipline these agile methodologies demand.

Next week: Top-Down Agile Beats Bottom-Up