Senex Rex
  • Team
  • Blog
  • Resources
  • Contact

Religion vs Science in the Battle for Agile

Posted on May 13, 2015 by Dan Greening Posted in Advocacy

While agile has zealots, it is not a religion. Agile is a scientific method that converts economic chaos to profit.


Critics tell me “agile is just a religion” at least weekly. My friend Stephen Tryon likes to say that religions observe, construct a hypothesis, then defend the hypothesis at all costs. Religions do not change with new scientific discoveries, their rigor can be short-lived, and their rituals benefit us indirectly at best. The religion premise produces absurd results. Some companies have “converted” from Scrum to Lean Startup, a laughable idea since they affect different groups and used together they magnify profit. (How about “converting” to both?) Some companies claim to use SAFe (Scaled Agile Framework), but not its underlying team-methodology, Scrum. The result cannot adapt rapidly to market and production change, and thus fails to fulfill the agile goal. People reinvent hybrid agile-waterfall chimeras or allow teams to choose waterfall or agile, saying “we’re agnostic,” failing to realize the company’s time-to-market and profitability depend on their choice (agile may not be the right choice for an activity, but thoughtful analysis will reveal a right choice). A colleague argues people love Scrum because “engineers like to build stuff” and “engineers don’t like to be yelled at” [deha2013].

In fairness, agile advocates can behave religiously. The popular Agile Manifesto for Software Development, used by many to define “agile”, leaves much to interpretation. The Manifesto does not explicitly discuss measurement, experimentation, or limiting work-in-process, all essential characteristics of all agile methodologies (and science, by the way). The Manifesto is not testable. The Manifesto focuses on software, yet when people outside software ask us for an agile definition, we still offer the Manifesto (and their interest immediately declines). We advocate agile methodologies as a panacea for production and market problems. We don’t proactively discuss agile’s costs, tradeoffs and sweet spots; worse, most of us don’t know them. We defend the hypothesis at all cost. Religion.

Creativity Science

Though critics and friends alike spiritualize agile, agile methodologies are scientific frameworks that optimize creativity for chaotic economies. We can study these things. To be recognized as a scientific field, rather than a religion, agilists must describe their activities as science. The scientific method teaches us to observe, hypothesize, test the hypothesis and repeat. Guess what? In well-managed Scrum, Retrospective meeting participants observe and hypothesize the effects of process specifics (such as Definition of Done), Sprints test the hypothesis, and then we repeat! Every agile methodology has similar practices.

Agile methodologies adapt incrementally to improve chaotic economies. In analyzing this field, I am taking a “big tent” perspective on agility: if a methodology “feels agile,” I try to incorporate it. If conflicts arise, they help mark the boundaries of agile. Scrum and Lean/Kanban optimize creative production economies in general; XP (Extreme Programming) optimizes software economies in specific; Lean Startup optimizes customer marketing economies; GTD (Getting Things Done) and Pomodoro optimize personal production; Toyota Production System and XM (Extreme Manufacturing) optimize complex manufacturing; Hoshin Kanri optimizes executive strategy; the Rockefeller Habits optimize executive leadership.

Chaos theorists study emergent properties. Applying simple rules to entities in a chaotic system (such as individuals in a Scrum team) can create a complex adaptive system (such as higher productivity teams). We can’t predict the exact form of the resulting system, but we can seek improvements in key metrics, such as production rate. Imposing strict rules can create an ordered system, but ordered systems rarely produce unexpected outcomes. A system can lose creativity when it strictly controls individuals [Six Sigma is Draining Employees’ Creativity]. Since agile methodologies always address creative processes, we must not overly control our systems.

Emergent Profit

What simple rules generate emergent profit from chaotic economies? (Here, I use “profit” to represent value returned divided by cost; the units need not be currency. See Measure Economic Progress for more.)

We can take two ideas and turn them into rules to help generate agility: 1) Unless we can measure “profit”, we can’t identify whether the emergent outcomes are better or worse (many businesses suffer from unmeasured ad hoc decisions). 2) Given that emergent outcomes cannot be inferred from the simple rules we impose, we must experiment. Their corresponding patterns look like this:

  1. Economic actors sometimes focus so intently on activity, they can’t tell whether they are improving …
    … therefore measure economic progress, as often as feasible.
  2. Creative actors produce new things with new tools in chaotic economies. Since it was never produced this way before, no one can predict long-term outcomes …
    … therefore proactively experiment to improve.

In chaotic systems, outcome predictions worsen exponentially over time. We can predict how a simple rule change will affect the outcome in the short-term, but the long-term it is impossible. If we can keep the rule changes simple and the time short, we can perhaps improve the short-term outcome slightly. If we keep making incremental changes, rapidly observe and adapt, we improve the long-term economic outcome, otherwise we won’t. This observation forms the third pattern in our agile pattern language.

  1. Creative activity exploits current resources to serve current markets, but chaos will soon erode any advantage …
    … therefore limit work-in-process to rapidly test and adapt the work.

These three agile base patterns, present in every agile methodology, transform chaotic economies into complex adaptive profit systems.

Religion or Science?

How can we tell whether agile is a religion or science? If we believe that our economy is chaotic, these three agile base patterns produce a complex adaptive system. Many economists believe economies exhibit chaos [good1990]. The likelihood of this system improving over time depends on two questions:

  • Do the experiments we construct in pattern #2 explore the space productively?
  • Does the work-in-process limit in pattern #3 generate a complex adaptive system?

It turns out these two questions also relate very strongly to my PhD dissertation, which explored a complex adaptive system called simulated annealing. I found that if simulated annealing experiments are fractal (they explore a wide-ranging space in early “sprints”, then narrow the range with time) and if random errors have the same properties (they get smaller over time)—the system will converge to a good result [gree1995].

This gives us guidance about agile Retrospectives. Early retrospectives for a team should explore ideas much more freely—pair programming or not, co-working or not, multi-lingual programming or not, longer or shorter sprints, etc.—and then reduce the “radicalness” of experiments over time (analogous to fractal exploration). Early sprints can survive a looser Definition of Done (big difference between sprint completion and “truly shippable”), but later sprints should ship to customers (analogous to fractal errors). [I must admit, I didn’t realize this surprising parallel until I worked on this article.]

If we don’t assume our economy is chaotic, we have defined agile sufficiently that we can construct experiments to compare it to our previous management approach. Since agile uses economic metrics to guide its progress, but “waterfall methods” typically do not, we can often conclude from looking at an organization, “Anything that measures and adapts is better than this!” Otherwise, we can make an experimental comparison.

If a non-agile approach ever wins, I’ll be surprised. Systematic Corporation, in Aarhus, Denmark, made numerous comparisons in its highly controlled CMMI Level 5 software development environment, and concluded that Scrum-based management reduced defect repair cost by 40% [jako2009] and increased delivery rate by 100% [jako2015], compared to waterfall under CMMI 5. Similar results are likely from the other agile methodologies.

I hope I have convinced you agile is science, not religion. Many enthusiastic colleagues do feel spiritually attuned to agile. Agile’s transparency, experimental approach and collective responsibility seem to produce greater trust and happiness. But at the core, agile methodologies are testable management approaches to maximize the economic benefits of creativity. They are science.

References

[deha2013] Brian de Haaff, “We are in a Holy Scrum War: Scrum zealots are like vegans and barefoot runners,” https://medium.com/i-m-h-o/we-are-in-a-holy-scrum-war-d3511064dd71 (September 13, 2013).

[good1990] Richard M. Goodwin, Chaotic Economic Dynamics, Oxford University Press (1990).

[gree1995] Daniel R Greening, Simulated Annealing with Errors, PhD Dissertation (1995), UCLA Computer Science Department.

[jako2009] Carsten Ruseng Jakobsen and Jeff Sutherland, “Scrum and CMMI – Going from Good to Great: Are you ready-ready to be done-done?,” Agile Conference 2009, IEEE.

[jako2015] Carsten Ruseng Jakobsen, personal communication (May 4, 2015).

Reviewers

Many thanks to Hala Saleh, Charles Eliot, Erik Gibson, Stephen Tryon, Laureatte Loy and Bianca Sias for early draft reviews.

Share this:

  • Click to print (Opens in new window)
  • Click to share on Twitter (Opens in new window)
  • Click to share on Facebook (Opens in new window)
  • Click to share on LinkedIn (Opens in new window)
  • Click to share on Reddit (Opens in new window)
  • Click to email a link to a friend (Opens in new window)
  • More
  • Click to share on Tumblr (Opens in new window)
  • Click to share on Pinterest (Opens in new window)
  • Click to share on Pocket (Opens in new window)
agile base patterns agile methodology pattern language Scrum
« Product Owner
Agile Leadership Patterns—Santa Barbara and Webinar »

Leave a comment Cancel reply

You must be logged in to post a comment.

Continue with Facebook
Continue with LinkedIn

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Pages

  • 5 Points
  • Account
  • Agile Capitalization
  • Agile Capitalization Video: Greening and Rudd
  • Agile Training
  • Agility Language
  • Call for Papers: Agile/Lean at HICSS (Kauai, January 5-8, 2016)
  • Call for Papers: Agile/Lean at HICSS
    (due June 15, 2016)
  • Certified Enterprise Coaching
  • Clients
  • Contact Us
  • Courses: Agile Capitalization Workshop
  • Courses: Agile Development
  • Courses: Agile Product Management
  • Courses: Executive Introduction to Agile
  • Courses: [email protected] Practitioner
  • Dan R. Greening
  • Enterprise Scrum: Scaling Scrum to the Executive Level
  • Glossary
  • Home
  • Jeff McKenna
  • John Horton
  • Kay Lynn Gabaldon
  • Login
  • Logout
  • Members
  • Password Reset
  • Premium Content
  • Premium Content 2
  • Privacy Policy
  • Register
  • Release Duration and Enterprise Agility
  • Resources
  • Rob Myers
  • Senex Rex Team
  • Short Course: Agile Manager
  • Sign up for Premium Content
  • Software Moneyball
  • Subscribe
  • The First of Five Challenges to Large Organizations that Force Agility
  • Troy Magennis
  • User
  • Vincent T. Mills
  • Senex Rex Blog Posts
  • Rapid Agile Forecasting

Archives

  • August 2018
  • February 2018
  • February 2017
  • March 2016
  • August 2015
  • July 2015
  • June 2015
  • May 2015
  • April 2015
  • February 2015
  • August 2014
  • May 2014
  • April 2014
  • March 2014
  • February 2014
  • January 2014
  • August 2013
  • July 2013
  • June 2013
  • May 2013
  • April 2013
  • February 2013
  • January 2013
  • December 2012
  • November 2012
  • October 2012
  • April 2012
  • March 2012
  • December 2011
  • July 2011
  • March 2011
  • February 2011
  • September 2010
  • August 2010
  • May 2010
  • April 2010
  • March 2010
  • January 2010
  • May 2009
  • March 2009

Categories

  • Advocacy (11)
  • Agility (10)
    • Agile Base Patterns (7)
  • Calls for Papers (4)
  • Enterprise (23)
  • Events (5)
  • Job Search (1)
  • Marketing (2)
  • Metrics (12)
  • Personal (7)
  • Personal Improvement (2)
  • Portfolio Management (11)
  • Product Management (9)
  • Quality (6)
  • Scrum (31)
  • Software (4)
  • Training (2)
  • Uncategorized (5)

WordPress

  • Log in
  • WordPress

Subscribe

  • Entries (RSS)
  • Comments (RSS)

Pages

  • 5 Points
  • Account
  • Agile Capitalization
  • Agile Capitalization Video: Greening and Rudd
  • Agile Training
  • Agility Language
  • Call for Papers: Agile/Lean at HICSS (Kauai, January 5-8, 2016)
  • Call for Papers: Agile/Lean at HICSS
    (due June 15, 2016)
  • Certified Enterprise Coaching
  • Clients
  • Contact Us
  • Courses: Agile Capitalization Workshop
  • Courses: Agile Development
  • Courses: Agile Product Management
  • Courses: Executive Introduction to Agile
  • Courses: [email protected] Practitioner
  • Dan R. Greening
  • Enterprise Scrum: Scaling Scrum to the Executive Level
  • Glossary
  • Home
  • Jeff McKenna
  • John Horton
  • Kay Lynn Gabaldon
  • Login
  • Logout
  • Members
  • Password Reset
  • Premium Content
  • Premium Content 2
  • Privacy Policy
  • Register
  • Release Duration and Enterprise Agility
  • Resources
  • Rob Myers
  • Senex Rex Team
  • Short Course: Agile Manager
  • Sign up for Premium Content
  • Software Moneyball
  • Subscribe
  • The First of Five Challenges to Large Organizations that Force Agility
  • Troy Magennis
  • User
  • Vincent T. Mills
  • Senex Rex Blog Posts
  • Rapid Agile Forecasting

Archives

  • August 2018
  • February 2018
  • February 2017
  • March 2016
  • August 2015
  • July 2015
  • June 2015
  • May 2015
  • April 2015
  • February 2015
  • August 2014
  • May 2014
  • April 2014
  • March 2014
  • February 2014
  • January 2014
  • August 2013
  • July 2013
  • June 2013
  • May 2013
  • April 2013
  • February 2013
  • January 2013
  • December 2012
  • November 2012
  • October 2012
  • April 2012
  • March 2012
  • December 2011
  • July 2011
  • March 2011
  • February 2011
  • September 2010
  • August 2010
  • May 2010
  • April 2010
  • March 2010
  • January 2010
  • May 2009
  • March 2009

Categories

  • Advocacy (11)
  • Agility (10)
    • Agile Base Patterns (7)
  • Calls for Papers (4)
  • Enterprise (23)
  • Events (5)
  • Job Search (1)
  • Marketing (2)
  • Metrics (12)
  • Personal (7)
  • Personal Improvement (2)
  • Portfolio Management (11)
  • Product Management (9)
  • Quality (6)
  • Scrum (31)
  • Software (4)
  • Training (2)
  • Uncategorized (5)

WordPress

  • Log in
  • WordPress
Copyright ©2013-2015 Senex Rex LLC. All Rights Reserved.
We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits. By clicking “Accept”, you consent to the use of ALL the cookies.
Do not sell my personal information.
Cookie SettingsAccept
Manage consent

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. These cookies ensure basic functionalities and security features of the website, anonymously.
CookieDurationDescription
cookielawinfo-checkbox-analytics11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checkbox-functional11 monthsThe cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checkbox-necessary11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-others11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-performance11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
viewed_cookie_policy11 monthsThe cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.
Functional
Functional cookies help to perform certain functionalities like sharing the content of the website on social media platforms, collect feedbacks, and other third-party features.
Performance
Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.
Analytics
Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics the number of visitors, bounce rate, traffic source, etc.
Advertisement
Advertisement cookies are used to provide visitors with relevant ads and marketing campaigns. These cookies track visitors across websites and collect information to provide customized ads.
Others
Other uncategorized cookies are those that are being analyzed and have not been classified into a category as yet.
SAVE & ACCEPT