I consult, write, and speak on running better technology businesses (tech firms and IT captives) and the things that make it possible: good governance behaviors (activist investing in IT), what matters most (results, not effort), how we organize (restructure from the technologically abstract to the business concrete), how we execute and manage (replacing industrial with professional), how we plan (debunking the myth of control), and how we pay the bills (capital-intensive financing and budgeting in an agile world). I am increasingly interested in robustness over optimization.

I work for ThoughtWorks, the global leader in software delivery and consulting.

Friday, November 30, 2018

The Fallacy of Composition

That six month experiment with Agile practices yielded off-the-charts results in time-to-market with quality, as well as sponsor and user satisfaction. It cost a lot, but everybody is smitten, and word has gone round the company. Everyone wants to know: what was different? Well, everything: automated build pipelines, stories, stand-ups, showcases, spikes, desk checks, you name it. Ok, great. Let's make sure everybody in the organization is doing those things everywhere, all the time.

That "Agile pilot" was a deliberate investment in the formation of behaviors designed to yield local (that is, team-level) optimization. Sustained, intense concentration on collaboration among consumers, designers, builders, testers and managers resulted in more experiments, more builds, more deployments, and more frequent feedback. This resulted in not only a better software asset, but an outcome that everybody is emotionally invested in.

The immersive nature in which these new muscle memories were developed came at a cost. You took the flack for using unapproved tools. You paired up your staff, effectively doubling the size of the team and more than doubling the run rate. You had to deal with the daily drama of your employees coping with the muscle confusion of learning new ways of working as well as having to deal with strong consultant personalities. A long-time employee quit - and with him, years of tribal knowledge walked out the door - because he was demoralized by the experience. No one seems to remember that your business partner all but demanded that you to pull the plug on the entire exercise just a few weeks in. With success comes selective amnesia.

Still, it was a success, if a success on a small scale. The experience of success in the small makes us prone to gross misunderstanding of what success looks like in the large. It is easy to fall victim to the fallacy of composition: the inference of the properties of the whole from the properties of its parts. There are two obvious ways this manifests itself: in not being aware of characteristics of success at scale, and in not being aware of the characteristics of what it takes to scale it up.

A small product team with no external dependencies is a closed ecosystem, like a terrarium. A terrarium ecosystem is not particularly diverse, and any deficiencies are easily corrected through minor adjustments a manager can directly make through direct intervention. An enterprise-scale ecosystem has a geometrically larger number of dependencies and interactions; these are not very easily compensated for by management-level employees through direct intervention.

Success at scale requires understanding an organization as a whole, through different lenses. "How do we spread these practices throughout the organization" looks at mechanical activities that yield local (team-level) optimization in a fixed span of time. They make no provision for what happens at an organizational level over a long span of time: different parts of the organization will move at different speeds, held back by legacy constraints that are not so easily undone; inter-team and inter-organizational dependencies are not simply solved through wistful promises of "alignment"; people with scarce skill sets cannot be co-located with every delivery team that needs them; how hiring is done and who is hired will determine whether change is sustained and evolves; procurement practices that buy for unit cost optimization rather than outcomes reward vendors for the wrong behaviors; operating company norms that prioritize efficiency at the expense of learning will render feedback loops inert.

In failing to see the characteristics of the organization that are not present in the parts we are looking at, we are blind to the organizational phenomenon that will impair and ultimately reverse the benefits of change.

It is also important to recognize that the characteristics of change at large scale are not the same as the characteristics of change at small scale.

When answering "what is different", the rigor, discipline and commitment necessary to raise the bar tend not to be part of the answer. This separation of "ends" and "means" invites dilution of skill transfer and subsequently dilution of the skills themselves: coaching replaces immersion, decks and self-service wikis replace coaching. In addition, the implicit reduction of risk - having been "proven out" in a pilot, the practices must no longer require a heavy investment - recasts Agile as an operational problem rather than one of learning and growth. It's just a different process from what we follow today, right?

"What is feasible, or beneficial, in the small may be infeasible, or harmful, in the aggregate."1 Heavy investment in immersive activities at scale - in effect, making an investment to "push" change - is prohibitively expensive at enterprise scale. The sustained period of immersive skill development that was highly beneficial in a single team is infeasible at scale. Diluting change makes it outright harmful, as it can yield false positives of everything from collaboration to quality. That whole "we're going Agile" thing will become a thin veneer over the chronic problems that plague your organization, and once that happens something that could be of genuine benefit is corrupted to a point of uselessness.

The failure to recognize that an enterprise is a geometric rather than a linear degree of difference from a single team dooms a lot of well intentioned change programs. We also see the effect in maturity models and enterprise frameworks that are precise at atomic levels of change but vague or misleading at organizational levels of change. We'll look at the latter in a future post.

1 Kay, John. Other People's Money: The Real Business of Finance.