Agile Pattern: Chunk Before Choosing

Agile Pattern: Chunk Before Choosing

Creative people with limited resources, such as product managers, developers, CEOs, investors and artists, must choose which items to assess, staff or fund. They compare value, cost, flexibility and risk to make a decision.

Faced with too many options, we choose badly …

Continue reading

Agile Supports Software Success

Dan_Head_ShotAgile posits this trade off: that creative projects, such as software development, have such huge market, technical and budget uncertainty, that we should pay the high expense of repeated regression testing, packaging, deployment, and rework, to enable us to test our market and technical theories early and often, adapting our approach as we learn more.

Here is my elevator description of Scrum: it is rhythmic experimentation to improve production.

You don’t need agile/Scrum methods if you are certain of market, process and technical perfection or near-perfection. We have nothing to learn with such certainty, so experimentation is useless.

However, the billions of dollars wasted in failed software projects (see IEEE Spectrum 2005, “Why Software Projects Fail”) at abject failure rates exceeding 50% indicate that confident waterfall engineers are dangerously arrogant. We have much to learn about making more successful software projects. It is true that there are charlatans and religious zealots in the agile crowd, and I apologize for them, but there is growing evidence that agile practices are highly correlated with successful, low-cost projects, and enormously successful startups.

Senex Rex at Work: April 2014

Senex Rex is an agile and lean product consulting, coaching and training company. We tend to focus on metrics. We teach teams and managers how to measure, experiment, learn, improve and win. We help clients become highly profitable long term. When our clients make more money, they have greater freedom to innovate and their employees and shareholders have more freedom to enjoy life. We think agility helps in many cases, so we often teach and coach agile theory and practice. Few contractors teach clients how to sustainably retain and improve agility; we specialize in that. We have many other tools in our tool box. Here’s a snapshot of the work Senex Rex did in April of 2014.

sr favicon 512

Two-Hour Scrum, Lean Startup Overview

We often offer a free 2-hour overview of Agile/Scrum, Lean Startup and Catalytic Leadership to company leaders in active client locations (currently San Francisco Bay Area, Seattle, Santa Barbara and Salt Lake City). In exchange, we ask an executive to write a LinkedIn review (positive or negative). This April, we spoke with a well-known logging and operational intelligence company. The attending vice-president wrote furiously during the session and followed up strongly with his teams. We evidently made an impression. Our highly empirical approach to Scrum and Lean Startup inspires executives, especially when they see how these practices radically reduce market, quality and delivery risk. Would your company benefit from our overview? Contact us. Continue reading

Forecasting without Historical Data

We can forecast even when no historical data exists, if we use our experience and judgment. In Part 1 of our probabilistic forecasting series we looked at how uncertainty is presented; in Part 2 we looked at how uncertainty is calculated. Both of those parts presumed historical data was available.

TOTALLY MISSED [Converted].epsAlthough estimating without historical data makes many people uncomfortable, acting responsibly often requires us to do it. Fear of being wrong may cause us to avoid making any forecast at all, leaving someone else to make uniformed decisions. Forecasting helps us make better decisions by reducing uncertainty, even when there is little information. Probabilistic forecasting may involve experts expressing their guesses as a range. Wider value ranges in their “guesses” may indicate more uncertain inputs.

We recommend adopting these practices to get good estimates from experts:

  1. Estimate as a group to uncover risks that may expand the range of uncertainty (use Planning Poker or other anchor-bias reducing mechanisms to help expose differences).
  2. Estimate using a range, not a single value
  3. Coach experts to estimate using ranges to combat their particular biases towards optimistic and pessimistic

Range estimates must be wide enough that everyone in the group feels that the real value is within the range, as in “95 times out of 100 this task should take between 5 and 35 days.”

People can learn to be good estimators. Most people perform estimation poorly when faced with uncertainty (see “Risk Intelligence: How to Live with Uncertainty” by Dylan Evans and “How to Measure Anything” by Douglas Hubbard.) They found that practicing picking a range that most likely contains the actual value of known problems (wingspan of a Boeing 747, miles between New Your and London for example), then giving experts feedback on the actual answer, increased estimation accuracy. Practice helps resolve personal pessimistic and optimistic biases.

When estimating how long IT work will take, teams should provide a lower and upper-bound. When a project of sequential stories needs forecasting, it’s simple: the project forecast range is between the sum of the lower-bounds and the sum of the upper-bounds. However, few large projects involve completing stories strictly in sequence. If you have multiple teams, people working in parallel or complex dependencies, a simple sum doesn’t work (not to mention the unlikely luck of every pice of work being at the lower bound or the higher bound). Most projects need a more powerful technique for accurate forecasting.

Monte Carlo simulation can responsibly forecast complex projects, even if the only data you have is expert opinion. When Monte Carlo simulation is performed properly, we can propagate uncertainty accuracies from different components to create a responsible project forecast. For example, a statement like “We have an 85% chance of finishing on or before 7th August 2014” is mathematically supportable.

In next part of our probabilistic forecasting series, we will look at the likelihood of values within a range, how that can help narrow our forecast risk, and why work estimate ranges follow predictable patterns that help us be more certain.


Probabilistic Forecasting

In Part 1 of this series we discussed how probabilistic forecasting retains each estimate’s uncertainty throughout the forecast. We looked at how weather forecaster’s present uncertainty in their predictions and how people seem comfortable that the future cannot be predicted perfectly and life still continues. We need this realization in IT forecasts!

In Part 2 we look at the approach taken in the field of probabilistic forecasting, continuing our weather prediction analogy.

We can observe the present with certainty. Meteorologists have been recording various input measure for years, and evidence suggests ancient cultures have understood the seasons to the extent they knew what food items to plant and when. These observations and how they played out over time form the basis for tomorrow’s weather forecast. Modern forecasters combine today’s actual weather conditions with historical observations and trends, using computer models. Continue reading

Forecasting Defined

This is the first article in a series that will introduce alternative ways to forecast date, cost and staff needs for software projects. It is not a religious journey; we plan to discuss estimation and forecasting like adults and understand how and when different techniques are appropriate given context.

Stakeholders often ask engineers to estimate the work for a project or feature. Engineers then arrive at a number of story points or a date and  present the result as a single number. They rarely share uncertainty or risk with those estimates. Stakeholders, happy to get “one number”, then characterize engineer estimates as commitments, and make confident plans that depend on achieving the estimate. Problems arise when uncertainty and risks start unfolding and dates shift. Failure to communicate engineering uncertainty is a key difference between estimation and forecasting. Continue reading

Senex Rex at Work: February 2014

You may be interested in what Senex Rex does. Our mission is to help clients become highly profitable long term. When our clients make more money, they have greater freedom to innovate and their employees and shareholders have more freedom to enjoy life. We happen to think agility helps in many cases, so we often teach and coach agile theory and practice. Few contractors teach clients how to sustainably retain and improve agility; we specialize in that. We have many other tools in our tool box. Here’s a snapshot of the work Senex Rex did in February 2014. Continue reading

The Goal Revisited

The Goal by Eliyahu Goldratt is a business novel that recounts how a factory manager shakes off complacency and isolation to save his factory and its employees. Many MBAs, system scientists and agilists have read it.

I read The Goal 7 years ago. I was so excited I sent our CEO an email. “Have you read it? It has so many messages for us!” I said, breathlessly. “Yep. I’ve read it. Great book,” he said.


My coach friends and I have lately been trying to inspire executives and managers to sustain agile practices, to become competent agile coaches themselves. To help managers understand organizational agility, we have distributed copies of The Goal, foisting it on managers and begging them to read it. I felt I had to refresh my memory of it. On this, my second reading, I am inspired again, but for different reasons. Continue reading

Agile 2010 Impressions

Agile 2010 was held in Orlando near the Disney Epcot resort August 9 to 13. I focus on agile enterprises and attended many enterprise-focused talks. If you are interested in the developer focused view, Martin Fowler provides his thoughts here. My impressions follow.

Portfolio management is being implemented in conjunction with quarterly “sprints” in other places beside Citrix Online, including Motley Fool (Max Keeler), Tektronix Communications (Brian Miller) and Progressive Medical (Ben Blanquera). The four of us are starting to share ideas and experiences. Continue reading

Andy Warhol explains Incremental and Iterative Development

In any project, you should complete your your work in stages. This lets you show customers (those who pay) and users (who might not pay) your work in progress, to ensure you satisfy their initial expectations and adapt to new ones.

You will likely stage your project incrementally or iteratively. Most people don’t know the difference, but if they stage a project incrementally, it is easy to waste a lot of time. Iterative development is usually better.

I wrote this, hoping to help. Continue reading