Evolutionary Design

Posted August 12, 2015 by Joshua Kerievsky

What is agile's most valuable practice?

It's called Evolutionary Design.

No other practice does a better job of managing risks in software development.

Evolutionary Design influences how we organize people and teams, plan what to build, collaborate, integrate, develop and release.

So why aren't more people using it?

Perhaps they don't understand Evolutionary Design and how valuable it is?

It's time to change that.

Beginning With A Primitive Whole

In order to practice Evolutionary Design, you must understand how to plan and develop a primitive whole.

A primitive whole is a fully assembled yet extremely under-developed version of a finished whole.

The main parts are all there, but they are primitive.

The primitive whole doesn't do everything that the finished whole will eventually do.

While you may envision a fantastic finished product or feature, you begin with an immature implementation.

Consider how the guitar below evolves from primitive whole to finished guitar:

IterativeIncrementalDevelopment

Wait a minute, is that first guitar even a guitar?!?

It only has two strings, no tuning pegs, no frets, saddle or bridge and even lacks a sound hole!

And it's precisely what we wanted: a primitive starting place, an under-developed whole (not a bunch of unassembled guitar parts) that will evolve based on needs, priorities, risks and feedback.

Given a description of a fully-functioning feature or product, it takes skill to see it as a primitive whole.

Just Say No

Producing a primitive whole requires saying "No" when beginning a feature or product.

In the earliest iterations of our new payroll system there will be no accounting for sick leave, no support for vacations, no handling of federal or state taxes or leaves-of-absence.

In the first shipping version of our plugin, we'll use our older technology stack and no new programming languages, no new event streaming model and no upgrade to the latest WebSockets (which will necessitate upgrading and testing a lot of production software).

Saying no early on doesn't mean you ship crap.

What you create in early iterations isn't the final, shipping version.

You ship when you are ready to ship.

Collaboration & Integration Risk

Producing a primitive whole early on helps us manage collaboration and integration risk.

Why is that important?

Because collaboration and integration risks are rife in software development.

I've seen veteran software people produce sophisticated, well designed parts without integrating them into a working solution, only later experiencing serious integration problems that delay delivery by weeks.

I've seen role-specific teams for product management, front end development, database, back end development, user experience, quality assurance, etc., face serious collaboration and integration issues simply because they manage their own backlogs and aren't remotely organized to evolve anything together.

Meanwhile, the biggest risk these people face early on is not that they can each finish independent pieces of work and increase a project percentage done metric, but that they can successfully collaborate and integrate even basic functionality.

A cross-functional team or collection of teams, consisting of all the key people needed to plan and build a product, is far better prepared to handle collaboration and integration risk.

Integrating Early & Often

Producing a primitive whole requires that people collaborate and integrate early and often.

We prefer to do this as continuously as possible, hence the agile practice called Continuous Integration.

Imagine you have five teams working together to produce one product.

All five teams must have a shared understanding of what they will build, as well as what the primitive whole will and won't do.

And all five teams must continuously integrate to produce the working, primitive whole.

Delaying integration with other teams only makes integration pain worse.

Maintaining separate backlogs, burn down charts and source code repositories per team actively interferes with continuous integration across teams.

Ten years ago, my colleagues and I helped a group of 500 people, assembled across 30 teams, working on a suite of time-management applications, integrate their work after just one month of development.

While that may not sound very amazing, consider this: they normally waited 6 months to integrate and they had fallen into a cycle of 6 months of work, followed by 6 months of hardening time (fixing bugs and getting the software to work).

After introducing Evolutionary Design, this very large group discovered that they could indeed evolve functionality and integrate all of their work early and often.

They eventually got to the point of integrating across all 500 people once a week.

A Doorway to Discovery

Evolutionary Design helps us discover what we most need to deliver.

Imagine opening up several brand new buildings on a college campus but not providing any clear pathways to those buildings.

Christopher Alexander and team did that on the University of Oregon campus in the early 1970s.

Students and faculty began using the new buildings and as they walked to and from the buildings, natural pathways began to emerge.

Once the architects could see where the pathways emerged, they could confidently construct paved pathways.

This is the opposite of upfront planning.

In software, getting early evolutions of a product or feature into the hands of customers allows you to rapidly discover their needs.

We recently released an early beta of a new plugin to a few clients.

We had internal discussions about what cool new features to build next, none of which mentioned anything as unsexy as adding proxy support for corporate firewalls.

Yet after quickly releasing the basic plugin, we learned that users could not use it without proxy support.

Evolutionary Design drove us to release early, which allowed us to gather feedback faster and discover our users essential needs.

Discovering what you need to deliver is far different from managing a backlog of predetermined work.

The Most Valuable Practice?

Evolutionary Design is both a management and technical practice: it influences how we organize people, plan work, develop and release.

I said at the beginning of this blog that it is agile's most valuable practice.

It helps us manage collaboration and integration risk by guiding us to produce a primitive whole.

It tests that we are collaborating successfully by asking us to integrate early and often.

It helps us discover what our users need.

So why is this practice missing from most of the agile teams and organizations I visit around the globe?

Is it because popular forms of agility (e.g. Scrum) don't explicitly mention it?

Or because the Agile Manifesto is too implicit in advocating it?

Or because it's hard to implement, requiring too many organizational changes?

I'm curious to hear your thoughts.

Is Evolutionary Design agile's most valuable practice?

Why isn't it practiced widely around the industry?

Please add a comment to this blog and share your ideas.

And if you'd like to see more people benefiting from Evolutionary Design, please forward them this blog.

Thanks to Alex Freire, Tim Ottinger and Woody Zuill for reviews and suggestions for this blog.
  • Pingback: Evolutionary Design - Industrial Logic | Purpos...()

  • Philip Skogsberg

    I would say that most people in agile development pay lip service to some form of evolutionary design, they understand why it’s important and that i makes sense. But on a day to day basis when product owners, designers or developers make decisions about what features go into a new product, or what features to add to an existing product, we tend think we know more about what the users want than what we can honestly know. In addition, building a minimimum viable version of the final product is kind of scary, we don’t want to deliver anything less than what we think should be included in the final version.

    So in the end, I think we end up wanting to do evolutionary design, but fall pray to some version of feature or scope creep because one or some features “just have to be included”.

    • Joshua Kerievsky

      Agreed Philip, too few are analyzing risks and managing them. Evolutionary Design is one of the best tools for managing implementation risks.

  • Pierre Rasmussen

    I agree with Philip’s comments. The cost to take a chance is to high.

    I also believe that lot of people “get stuck” in Agile/Scrum/WhatEver. When they have transformed their organisation they believe they are done. The target they are aiming for is fixed, which I believe is one mistake lot of organisations do.

  • Pingback: Agile Digest 33 - L'accompagnement agile()

  • Randy A MacDonald

    I ask an analogous question in my own domain: APL is very powerful, why isn’t it practiced widely around the industry?

    • Joshua Kerievsky

      Great point Randy!

  • http://www.oplaunch.com/ Mark A Hart, OpLaunch

    Joshua,
    You missed an opportunity with the illustration. Instead of four versions of a guitar from primitive whole to finished guitar, consider the following…

    Buddy Guy, legendary blues guitarist was born in 1936. As a boy, he made a living picking cotton in Lettsworth, Louisiana. He “made his first guitar out of old paint cans and wire from the front screen door. When his father got tired of all the mosquitoes that came into the house, he bought his son an old acoustic guitar with only two strings on it. Soon Guy was able to pick out a passable version of Hooker’s ‘Boogie Chillen.'” http://www.encyclopedia.com/topic/Buddy_Guy.aspx

    It is interesting that in the first pane of your illustration had two strings. Buddy Guy’s first acoustic guitar had two strings. The number of strings is not the most important item.

    The primary goal of the project should not have been to build a guitar. The clue to the missed opportunity is in the story I quoted.

    A more powerful illustration would contain a representation of a musician and eventually an audience. A musician’s goal to to create music and share with others. A guitar is one potential piece of hardware.

    The evolutionary development is a success when there is a smile on the musician’s face using your piece of hardware. This success requires “role-specific teams for product management, front end development, database, back end development, user experience, quality assurance, etc., face serious collaboration and integration.”

    • Joshua Kerievsky

      Hi Mark – thanks for your great comment. What you are describing is something I call Hypothesis to Happiness, an idea inspired by Lean Startup. It has overlap with Evolutionary Design, though I’m not sure it is the same thing. Usually by the time I’m doing Evolutionary Design, I’ve done experiments, I may have pivoted and I know roughly what to build. Now I’m ready to build it and I use Evolutionary Design to help me manage risk and continue to discover what the customers actually need. This is different from the “purpose” of the project, something that is critical and perhaps higher level than Evolutionary Design. Does this make any sense?

  • Pingback: Over-Starting and Under-Finishing - Industrial Logic()

  • Oscar Centeno

    Thanks Joshua for starting this conversation here. I think one reason people (us) resist using evolutionary design is that software is coded on a way that it’s hard to change. If there are no technical practices like good oo and modular design and unit testing, we wouldn’t be able to change it easily, so won’t be able to evolve it. If we had easy to change software, we could do a whole draft, learn and adjust accordingly. Software must be maintainable for a team to be agile. Do you concur?