Context: Creative people—such as engineers, designers, managers, researchers, lawyers, architects and investors—increasingly work on critical projects in teams. But…
Sustained creative progress isn’t predictable
Creative people working on a large project often brush off customers with, “It will be done when it’s done!” If creative professionals had done it before, we could expect them to estimate and deliver on time, but copying isn’t creative, and real creativity naturally involves uncertainty. Many have tried to impose detailed planning to creative efforts, but overplanning has produced extraordinary failures costing billions of dollars [char2005]. Nevertheless, the value of creative work often depends on timely delivery, and patrons can become desperate.
Forces
Creative people can shy away from economic matters, and yet economics drive their own ability to create and thrive. As a young researcher, I used paychecks as bookmarks, discovered them months later and deposited them; I’m not sure I got them all. Expecting creative people to be highly conscious of their own cost, or their work’s value, is often too much to ask.
Creative people love their work. When creative people engage, they forget about time and become more productive. Creative people can be perfectionists, and so may prefer perfecting components over delivering a usable whole. At the same time, assembling components to produce a usable whole takes time. Assembling components can seem like wasted effort to some creative people, even though their non-working efforts are useless to others.
Each of us has a limited capacity to make thoughtful decisions [danz2011]. Because creative people make critical decisions frequently in their work, adding other decisions to their load can limit their creative output.
Interruptions degrade creativity and quality [foro2014]. Related results show that multitasking can improve productivity within limits, but quality always suffers with increased multitasking [adle2012].
The Pareto principle theorizes that 20% of an investment is responsible for 80% of its return [pare1971]. If the most valuable 20% of the work were completed first, the return might pay for the remaining 80%.
… therefore, have them deliver increments rhythmically
One complete component rarely provides value to a customer, but a working assembly of partial components, called an Increment, often can. For example, a perfectly-built wall alone provides no shelter, but a house Increment—composed of a frame (four wall skeletons), a few roof trusses, and a board— can allow a customer to check out the view and provide feedback, and shelter workers and tools from the rain. An Increment must satisfy a Definition of Done, which usually includes assembly, packaging and possibly delivery requirements.
In Scrum, the time available to create an Increment is called a Sprint. The Sprint must have sufficient time to add new work to the Increment, to satisfy the Definition of Done, and to meet.
Roles
A team that performs Sprints is called a Scrum Team. A Product Owner assembles a Product Backlog of work the team might develop. A ScrumMaster organizes meetings, orchestrates consensus, enforces team agreements and fields external requests. Members of the Development Team do the creative work.
Events
Every Sprint includes a set of meetings in a prescribed order. They are:
- The Scrum Team attends a Retrospective meeting reviews previous work requests and deliveries, establishes development and process rules to try in the next sprint, and predict how those rules will improve its Increment. The Definition of Done is established or revised at the Retrospective meeting.
- The Scrum Team attends a Sprint Planning meeting to review and adjust the Product Backlog, select high ranking work items that can comprise the next Increment, and add them to a Sprint Backlog. The Scrum Team also decides on a Sprint Goal, a unified focus that directs collaboration on the Scrum Team.
- While working, the Development Team attends a Daily Scrum meeting every 24 hours to coordinate and collaborate.
- The Scrum Team attends a Sprint Review meeting to inspect the Increment and revise the Backlog based on feedback.
- The Scrum Team may attend a Backlog Refining meeting to reduce the burden of a long Sprint Planning meeting.
Artifacts
The Product Backlog is a strictly ordered list of the work expected of the Scrum Team. It is controlled by the Product Owner, though many Product Owners permit anyone to add work at the end of the Backlog.
The Sprint Backlog is a strictly ordered list of the work accepted by the Scrum Team in the Sprint Planning meeting.
The Increment is a working product that demonstrates completed work from the Sprint Backlog.
The Sprint Goal is the main purpose of a Sprint. It can be a feature or theme in the Sprint Backlog, a process goal, or any other coherent focus around which the Scrum Team can collaborate.
Example
In 2007, a team I managed was asked to develop a cloud-based screencasting software application, where a person could record a computer session. One team member suggested we use Scrum, a technique I’d never encountered. After reading everything I could find about it, we decided to use it. Initially, I took the role of Scrum Master; we recruited a product manager to serve as our Product Owner; and everyone else was a Development Team member. We later recruited a project manager trained in Scrum to replace me as Scrum Master.
Our Scrum Team rapidly produced working Increments, surprising many in our company. In five months, the team of just five engineers developed client applications that ran on Windows and Mac, and a clustered Java-based server application—we considered the three applications together as a system as our Increment, so if any of them didn’t work, the Increment was broken. Early iterations tried to develop the clients with a common non-native infrastructure; they worked but were hard to revise and maintain. Because Scrum had us performing early Increments, we could cheaply decide to switch to native infrastructures for the clients. In early iterations, we tested our user interface, and we discovered that some interfaces were not intuitive. Because it was early in development, we could switched them at relatively low cost and did.
The system we built was highly resilient; it would continue to run even if a hosting server crashed. We didn’t expect to need this resiliency initially, but scalability was a huge risk identified early and we wanted to test whether we could build a resilient server farm. That decision saved our butts. Our first public demonstration was at a famous new-products conference, streamed live. We didn’t know about the streaming part. Under unexpected load of 1700 users, a host server crashed. The screencasting application quietly switched to a different server, and continued working. Gradually, one after another of our six servers crashed. Our team went rapidly to work identifying bugs, fixing them, and bringing servers back online, all while the conference was taking place. No user was interrupted, and no one but the developers knew this was happening. But behind the scenes there was a lot of human and machine activity. Our team got a reputation for bulletproof software.
This experience transformed me: I became a strong advocate for agile methods and ultimately became a Scrum Alliance Certified Enterprise Coach (a management consultant for agile organizations). I have a PhD in computer science and I have been programming since I was 15; now I was programming humans instead of machines.
Resulting Context
Scrum frees creative people to focus on development. In Scrum, the Product Owner performs economic analysis and prioritization. The Scrum Master fields interruptions that might occur in uncontrolled environments.
Scrum forces creative people to deliver a working Increment that can obtain customer feedback. Because the Sprint duration can be selected by the Scrum Team to include a balance of new development and packaging, the Development Team is not overly burdened by packaging, but at the same time can’t ignore it and build up technical debt indefinitely and create high economic risk. Scrum allows Development Team members to decide for themselves when, during the Sprint, they should start packaging the Increment. Customer feedback ultimately benefits developers by fostering enthusiastic customers.
Scrum limits administrative activities to ritualized, regular meetings with time-limits. As developers gain comfort with these routines, they became part of their “muscle memory” and they can focus more on development at other times.
Scrum reduces unnecessary decision-making for creative people by designating a single “sacrificial lamb for interruptions,” the Scrum Master. They then have greater capacity for decision making to use in developing a better Increment.
Scrum produce high value early and makes it usable by customers. The Product Owner can exploit the Pareto Principle to put the 20% of work items that have 80% of the value at the top of the Backlog, to ensure that the Development Team focuses on those items first.
Related Patterns
Scrum specializes the Agility pattern language for creative teams. The figure above shows how the top four patterns of Agility—Limit work in progress, Measure economic progress, Adaptively experiment, and Embrace collective responsibility—are realized in Scrum. By selecting a Sprint Backlog and demanding that the Development Team produce an Increment at the end of a Sprint, Scrum limits work in progress. Scrum mandates that the Scrum Team reflect on past work in the Retrospective, and it that meeting many Scrum Team’s review metrics related to economic progress (such as completed points, reported bugs, happiness metric, etc.). Finally, by inducing Scrum Team members to jointly commit to completion of an Increment and a Sprint Goal, individual team members become responsible.
Each pattern of Scrum contributes directly to at least one base pattern of Agility, as shown in the figure above. The graph shows how Scrum has no strong patterns contributing to External Advocacy, and this may be related to difficulties in sustaining high-levels of agility, when Scrum emerges bottom-up in an organization.
References
[adle2012] Rachel F. Adlera and Raquel Benbunan-Fichb, “Juggling on a high wire: Multitasking effects on performance,” International Journal of Human-Computer Studies 70:2 (February 2012), pp. 156–168.
[char2005] Robert N. Charette, “Why Software Fails,” IEEE Spectrum (September 2005).
[danz2011] Shai Danzigera, Jonathan Levavb, and Liora Avnaim-Pesso, “Extraneous factors in judicial decisions,” Proceedings of the National Academy of Sciences 108–17 (April 26, 2011), pp. 6889–6892.
[foro2014] Cyrus K. Foroughi, Nicole E. Werner, Erik T. Nelson and Deborah A. Boehm-Davis, “Do Interruptions Affect Quality of Work?” Human Factors: The Journal of the Human Factors and Ergonomics Society 56:7 pp 1262-1271 (November 2014 ).
[pare1971] Pareto, Vilfredo and Page, Alfred N., Translation of Manuale di economia politica (“Manual of political economy”), 1971, A.M. Kelley, ISBN 978-0-678-00881-2.
[suth2015] Jeff Sutherland, personal communication, 2015.
[suth2017] Jeff Sutherland and Ken Schwaber, The Scrum Guide (2017), https://www.scrumguides.org/docs/scrumguide/v2017/2017-Scrum-Guide-US.pdf
Acknowledgements
[Please provide feedback. I’m happy to provide LinkedIn or other links in this acknowledgement section to those who help out.]
This pattern conforms to the 2017 definition of Scrum [suth2017].
Author
Dan Greening
