I consult, write, and speak on running better technology businesses (tech firms and IT captives) and the things that make it possible: good governance behaviors (activist investing in IT), what matters most (results, not effort), how we organize (restructure from the technologically abstract to the business concrete), how we execute and manage (replacing industrial with professional), how we plan (debunking the myth of control), and how we pay the bills (capital-intensive financing and budgeting in an agile world). I am increasingly interested in robustness over optimization.

I work for ThoughtWorks, the global leader in software delivery and consulting.

Thursday, May 31, 2018

Organizing for Innovation, Part III: A New Metaphor

Before looking at autonomy at scale, we need a different understanding of what an organization is. This matters because the way we perceive an organization will determine our interpretation of what "good" and "bad" look like.

Some years ago, I wrote that management thinking is still dominated by the "Freds": Frederick the Great, who's Prussian military structure became the model for the modern organization; and Frederick Taylor, a pioneer of scientific management. Frederick the Great organized his military to function like a machine. Frederick Taylor optimized performance down to the task level. The organization-as-machine metaphor has dominated management thinking for well over a century.

A machine is optimized to require the least amount of labor to produce the highest volume of throughput possible with minimal waste. Machines are designed such that each of its component parts performs specific tasks in a consistent and repetitive fashion. When organized into an orchestrated flow of execution, a machine will yield a high volume of consistent output.

Machines are made for optimal performance within a limited range of environmental variation; they are not adaptive to their environments. If something changes in the internal or external environments, the machine will perform at a less-than-optimal state. An exceptional condition to those the machine was designed to operate in will cause an error in execution. Exceptions are subsequently a source of inefficiency to a machine: the machine's purpose is the consistent and repetitive execution of tasks; exceptions inhibit that execution. An exception must therefore be contained so that the machine returns to efficient execution as quickly as possible.

If a lot of things change in the environment, the machine will completely break down. The machine is not designed to intelligently adapt, it is designed to single-mindedly execute.

The organization managed as a machine may achieve optimal performance, but at the cost of adaptability. We need a different metaphor for the organization: without one, we're doomed to end up with what we've already got. If we see an organization as a machine, we will define it in terms of efficiency: what gets measured is what gets managed, and when we apply old-style thinking we come to rely on old-style managing. If the organizational goal is innovation, we need the organization to have the characteristics described in the previous post in this series: it must be attuned to its environment, and it must be highly adaptable to it.

An more appropriate metaphor for the adaptive organization is to think about the organization as a brain. The brain is a highly adaptable organ:

  • While different regions specialize in different activities, control and execution are not localized - regions are closely independent and capable of acting on behalf of each other when necessary
  • Memory is distributed, not localized
  • Robust connectivity allows for simultaneous processing and awareness of what is going on elsewhere
  • Cross-connectivity creates redundancy that allows the brain to operate in a probabilistic rather than a deterministic manner, allows room to accommodate random error, and creates excess capacity that allows new functions to develop1

The brain is designed to facilitate the process of self-organization where internal structure and function can evolve along with changing circumstances; machines do not do this.

The manager in the organization-as-brain is concerned with redundancy of capability at atomic and coarse levels; recording, curation and accessibility of institutional memory; and unencumbered communications (no strict hierarchy) that are highly contextualized (explain why you want what you want). These are anathema to the manager who's goal is efficiency of execution. Redundancy is paying for something twice. Memory only matters to the most senior people as it affects decisions with long-term ramifications; in the organization-as-machine, those decisions are exclusively their purview. Open communications creates a lot of noise that interferes with orders from management. Having to explain why somebody needs something wastes time.

Traditional organizational thinking inhibits the conditions that create innovation. To have any reasonable expectation of innovation, we need to have the right expectations of how the organization functions if we are to manage in a way that fosters innovation.

Or course, a business is not managed by metaphor. But the way an organization is understood determines the way it is managed. A mindset rooted in learning rather than efficiency provides a set of "first principles" against which measurements and management decisions can be reconciled. It is therefore important to have the right mind-set about the nature of organization to understand how the organization of autonomous teams can exist at scale.

With these goals in mind, the characteristics of an organization of autonomous teams at scale become easier to understand. We'll look at those in the next post.

1 Morgan, Gareth. Images of Organization. Sage Publications, 1986.