I consult, write, and speak on running better technology businesses (tech firms and IT captives) and the things that make it possible: good governance behaviors (activist investing in IT), what matters most (results, not effort), how we organize (restructure from the technologically abstract to the business concrete), how we execute and manage (replacing industrial with professional), how we plan (debunking the myth of control), and how we pay the bills (capital-intensive financing and budgeting in an agile world). I am increasingly interested in robustness over optimization.

I work for ThoughtWorks, the global leader in software delivery and consulting.

Tuesday, July 31, 2018

Organizing for Innovation, Part V: The Leadership Challenge

Organizations of autonomous teams require a different set of behaviors than organizations that are run like a machine. People in a self-directed team form their own appreciations for what should be done, prioritize what will be done, and self-determine how it will be done. They are unencumbered by hierarchy, expected to communicate with anybody in the organization they need. They are unencumbered by role definition, as people simply do whatever work is required even when that means acquiring new skills or knowledge. They are unencumbered by organization, as teams form task-forces to solve for problems that are beyond the scope of any single defined team.

This all sounds great. Who wouldn't want to work this way?

The majority of people working in enterprise IT today, that's who.

Enterprise tech labor is highly codified (bounded responsibilities) and stratified (seniority). Aside from the fact that this serves the interests of a multitude of non-tech corporate functions such as human resources, vendor management and finance, it also provides a great deal of comfort to the individual. The employee knows very precisely what is expected of them to earn a salary increase or advance their career, while the contractor knows what they are obliged to do to satisfy the terms of their contract. Over time, jobs become working annuities that require little servicing (such as skill acquisition or excessively long working hours) for comfortable levels of compensation with job security (by e.g., being the only ones familiar with a technology, service or function).

The autonomous team environment is anathema to this. For starters, the autonomous team operates with a lot of ambiguity. New appreciations change the team's priority constantly, meaning there is no deterministic plan. Plus, people adapt themselves to the work that needs to be done as opposed to working in strict swim-lanes. An autonomous team environment implicitly subordinates the traditional goals of the individual: the success of the team is success for the individual. An autonomous team breaks down when its members put individual achievement over team accomplishment.

Of course, working this way is a question of both skill and will. Whether labor can re-orient itself from machine to autonomy is a debate well beyond the scope of any blog post, but suffice to say that some - few, several or most - will not be able to make the transition from doing what they are told to do by management, to figuring out for themselves what needs to be done and doing it. In addition, whether labor willingly chooses to re-orient itself is another matter entirely. Over-exposure to enterprise change programs - or more likely, over-exposure to failed enterprise change programs - won't inspire enthusiasm for yet another one. There is also the suspension of disbelief people need to make to give devolved authority and autonomous team structures a try.

This brings us to the leadership challenge that creating an organization of autonomous teams poses. Obviously, there are institutional barriers that have to be overcome. HR has to be comfortable with ambiguous roles and titles. Vendor contracts for supply of specialist labor have to be replaced with contracts tied to business outcomes. Finance needs an alternative to predictive planning and financial budgeting.

However, the challenge to leadership goes beyond mechanical processes and structural changes. In an organization of autonomous teams the nature of leadership changes from giving commands to guiding actions. The leader does not direct people's performance to achieve a goal (organization-as-machine), but instead projects a goal that people direct themselves toward achieving (organization-as-brain). The leader in the autonomous organization makes use of:

  • Concrete statements of expectations: leaders have to make expectations crystal clear. Saying "all enterprise software is deployed into production every other week" is a clear expectation. Saying "we will be a continuous deployment organization" is a woolly statement: "continuous" will be interpreted relative to the current skills and learned helplessness of the people responsible for enterprise software today. The latter statement also misses the point: how the organization functions (continuous deployment) is less important than describing what it achieves (new software released across the board every other week.) The prescriptive implications of that statement deny the people in the organization the opportunity to figure out for themselves how best to achieve the goal. The leader communicates the expectation; it is up to the people in the organization to figure out how to achieve it.
  • Rewards and recognition: obviously, what gets measured is what gets managed. If the goal is to increase frequency of deployments, reward the teams that find ways to make reliable production releases weekly, then twice weekly, then daily, then multiple times per day. Story-telling is important as well, especially as an organization adopts new behavioral norms and its business partners develop new expectations of it. For example, in its early days, FedEx executives were fond of repeating the story of the junior employee who rented a helicopter on his personal American Express card to fly him to a mountain location to repair snow-damaged phone lines so that they could service remote customers. When you're building a reputation for absolutely, positively getting packages delivered overnight, stories like this go a very long way.
  • Acknowledging and solving constraints: every team, not to mention the organization as a whole, will be short of the capacity, capability and capital to achieve every organizational goal. While constraints force institutional responses such as prioritization and innovation, they are also opportunities for a systemic change. It is the leader's obligation to work with constrained teams to identify the actions they can take today so that they will be less constrained in the future. For example, if there is a capability constraint, what can the team do now to develop skills and knowledge so that it can do more for itself? If a financial constraint, how can the team build a stronger business case or define a set of experiments to explore potential value? In the face of constraints, the leader's responsibility is to develop an organization's ability to learn how to learn.

At the foundation of the organization of autonomous teams is Theory Y. The aspirational leader of an organization of autonomous teams has to believe that people are motivated, responsible, and do not need close supervision. If you do not fundamentally believe this, do not bother pursuing an organization of autonomous teams. Once you revert to command-and-control (Theory X), you have lost the mantle of leadership in a devolved organization.

In the next installment in this series, we will put all of these concepts together to visualize what the autonomous organization looks like at scale.

Thursday, June 28, 2018

Organizing for Innovation Part IV: Autonomy at Scale

It is easy to understand how a small organization of autonomous teams can function. When there are only a few teams, there is a small community, and it is simple for people to communicate with one another in both formal and informal ways.

It is not difficult to see how a large organization of largely independent teams can scale. For nearly 60 years, Gore-Tex has shown that devolved authority can work just fine at scale.

Autonomy scales at Gore-Tex because there is very little overlap between outerwear and dental floss, and subsequently less coordination. What happens when the business becomes complex, with lots of interdependencies among lots of teams?

Dependencies by themselves do not make it difficult to understand how devolved authority can work on a small scale. If there are 4 teams, there are a maximum of 6 communication pathways (n x (n - 1) / 2). Even if there are transitive dependencies, the small size of the community - 4 tech leads, 4 product owners, etc. - makes cross-functional communications relatively easy. But a technology organization of tens of thousands of people will have hundreds of teams - and therefore an extraordinarily large number of communication pathways. Translating a small number of enterprise goals into hundreds of millions of synapses sounds like opacity at best, chaos at worst.

There are four things that an organization of autonomous teams needs if it is to scale.

The first is an implicit hierarchy, but one of purpose rather than control.1 Traditionally, hierarchy is meant to control activity througyh supervisory responsibility and assigned decision rights: the subordinates in one division take direction from superiors in the same division. If, per the initial blog post in this series, decision rights are devolved, "span of control" does not exist in the organization of autonomous teams.

Hierarchy also influences communications. If those "higher up" in the hierarchy use the things produced by those "further down", there is an obvious pattern of communications between producers and consumers. In manufacturing, there might be different teams independently assembling subsystems such as brakes or drivetrains from individual component parts. Each of their subsystems might then be consumed by teams on the line installing them into more comprehensive systems (such as the powertrain), and, ultimately, the finished vehicle itself.

This is called a "hierarchy of purpose." The hierarchy is constructed largely around degrees of granularity. Brakes and drivetrains are smaller assemblies that form larger subsystems that contribute to the finished product. A provider of cloud-based infrastructure such as Amazon can organize in the same way: the "finished product" of a cloud instance consists of more "primitive" components of virtual storage, server and network. Each of those subsystems in turn consists of more finely grained primitive components of network protocol communication, CPU, load balancing, and so forth.

Creating complex higher-order offerings as composites of lower-level capabilities is the “platform effect” of innovation.

It's worth mentioning that enterprise program management has long tried the same structure. It chokes on itself when transitive dependencies that extend several layers deep expose the difference between a boundary in work (a team has exclusive responsibility for producing something used by many other teams) and a boundary in fact (the output is high-touch service, support and maintenance, making those boundaries porous). A primitive component must be consumable in a friction-free manner.

This is a logical transition to the second thing the organization of autonomous teams requires, which is practical patterns of communications. The platform effect scales effectively because consumption patterns are the de-facto communication patterns within the organization. In a hierarchy of purpose, the organization does not have hundreds of millions of communication pathways, because the number of consumers is limited. A cloud instance may consume a load-balancing primitive, but it does so through a more coarsely grained "network" intermediary.

The decision classes introduced in the initial post in this series act as a control system for each individual team. The wider the divergence of type of consumer, the more difficult it is to create patterns of demand and prioritize to form a product strategy. The nested team organization limits the divergence of consumer demand, which creates a narrower range of appreciations, which make cohesiveness of strategy and execution at an individual product level far easier to perform.2

The autonomous-team collective maintains enterprise cohesiveness by virtue of its communication patterns. To understand how, we have to revisit the devolved decision classes. Appreciations (what should be done) act as the organizational system for managerial decisions (what can be done); managerial decisions function as the instrumentation system for appreciations. Managerial decisions, in turn, act as the organizational system for technical decisions (how will it be done); technical decisions, in turn, act as the instrumentation system for managerial decisions.

Scope Nature Organizes Instruments
Appreciations   What should be done?   Managerial    
Managerial What can be done? Technical Appreciations
Technical How will it be done?   Managerial

How this works effectively in practice becomes clear when we look at the characteristics of a single autonomous team that we saw in the last post. The transmission of minimum critical specifications through a hierarchy of purpose limits the noise and confusion. The reception of minimum critical specifications from multiple consumers are interpreted as appreciations through double-loop learning. The relationship of control systems and instrumentation systems of the different decision classes categorizes the information appropriately.

The third thing required for an organization of autonomous teams to function at scale is the ability to handle extraordinary patterns of communication. As there is no hierarchy of control, the organization needs protocols that allow it to adapt itself to a changing problem space, as well as to resolve inter-team conflicts.

A dynamic problem space is addressed through task-forces, which may be short- or long-lived, depending on the nature of the need or opportunity. Task forces are formed organically from members of affected teams to solve for problems that are existential to any one team. For example, suppose three teams conclude that they need a new class of capability that is outside the boundaries of all of them. They could elect to form a "task force" in the form of a long-lived team, staffed by reassigning members of their respective teams to this new one on a full-time basis. Not all task-forces are long-lived and full-time, of course: a task force to address a simple challenge might require each person commit only 1 day each week for a month, allowing them to otherwise remain focused on their line responsibilities.

Another other exceptional form of communication are inter-team conflicts. For example, team A elects not to prioritize something really important to team B, and team B lacks the resources or capability to do it for itself. Without patterns for extraordinary circumstances, it would end with a very grumpy team B. Inter-team conflict is handled through mediation and adjudication protocols. At the first stage of mediation, an acceptable 3rd party - that is, someone who is not a stakeholder in the conflict - is asked to mediate a decision. If the conflict is still not resolved following the initial mediation, a committee of 3rd parties are formed to arbitrate a decision. This provides a fair hearing among peers without the need to resort to authority given through a hierarchy of control.

Finally, the organization of autonomous teams needs mechanisms for aligning the strategic goals of the organization with team and individual execution. Appreciations provide connective tissue among strategic, team and individual goals and objectives, but they still need to be reinforced by executive management. Intermediary (layered) objectives have a tendency to change the interpretation of organizational goals, and therefore the goals of each individual.

Alignment is achieved by telegraphing executive intent throughout the organization. There are techniques popularized by a number of firms - OKRs, V2MOM, and the Big Bets Spreadsheet - and probably many others. Whatever the mechanism, their purpose is to communicate unambiguous goals to give each individual a clear means of reconciling a decision - why, what and how - with an outcome that advances the organizational goals. This creates direct line-of-sight between effort and result - and therefore tactical action with strategic outcome - and allows the organization of autonomous teams to function without an excessive number of people in low-value supervisory roles.

With the right set of characteristics, then, an organization of autonomous teams can reach scale, even in complex environments. But it clearly has a vastly different operating model to the traditional control style imposed over the machine-like organization. Is the labor force equipped for this? Is leadership? We will look at these questions in the next post.

1 Susman, Gerald. Autonomy at Work: A Sociotechnical Analysis of Participative Management Prager Publishers, 1976.

2 There is no guarantee of control, of course. An individual team can still thrash or produce unwanted product.

Thursday, May 31, 2018

Organizing for Innovation, Part III: A New Metaphor

Before looking at autonomy at scale, we need a different understanding of what an organization is. This matters because the way we perceive an organization will determine our interpretation of what "good" and "bad" look like.

Some years ago, I wrote that management thinking is still dominated by the "Freds": Frederick the Great, who's Prussian military structure became the model for the modern organization; and Frederick Taylor, a pioneer of scientific management. Frederick the Great organized his military to function like a machine. Frederick Taylor optimized performance down to the task level. The organization-as-machine metaphor has dominated management thinking for well over a century.

A machine is optimized to require the least amount of labor to produce the highest volume of throughput possible with minimal waste. Machines are designed such that each of its component parts performs specific tasks in a consistent and repetitive fashion. When organized into an orchestrated flow of execution, a machine will yield a high volume of consistent output.

Machines are made for optimal performance within a limited range of environmental variation; they are not adaptive to their environments. If something changes in the internal or external environments, the machine will perform at a less-than-optimal state. An exceptional condition to those the machine was designed to operate in will cause an error in execution. Exceptions are subsequently a source of inefficiency to a machine: the machine's purpose is the consistent and repetitive execution of tasks; exceptions inhibit that execution. An exception must therefore be contained so that the machine returns to efficient execution as quickly as possible.

If a lot of things change in the environment, the machine will completely break down. The machine is not designed to intelligently adapt, it is designed to single-mindedly execute.

The organization managed as a machine may achieve optimal performance, but at the cost of adaptability. We need a different metaphor for the organization: without one, we're doomed to end up with what we've already got. If we see an organization as a machine, we will define it in terms of efficiency: what gets measured is what gets managed, and when we apply old-style thinking we come to rely on old-style managing. If the organizational goal is innovation, we need the organization to have the characteristics described in the previous post in this series: it must be attuned to its environment, and it must be highly adaptable to it.

An more appropriate metaphor for the adaptive organization is to think about the organization as a brain. The brain is a highly adaptable organ:

  • While different regions specialize in different activities, control and execution are not localized - regions are closely independent and capable of acting on behalf of each other when necessary
  • Memory is distributed, not localized
  • Robust connectivity allows for simultaneous processing and awareness of what is going on elsewhere
  • Cross-connectivity creates redundancy that allows the brain to operate in a probabilistic rather than a deterministic manner, allows room to accommodate random error, and creates excess capacity that allows new functions to develop1

The brain is designed to facilitate the process of self-organization where internal structure and function can evolve along with changing circumstances; machines do not do this.

The manager in the organization-as-brain is concerned with redundancy of capability at atomic and coarse levels; recording, curation and accessibility of institutional memory; and unencumbered communications (no strict hierarchy) that are highly contextualized (explain why you want what you want). These are anathema to the manager who's goal is efficiency of execution. Redundancy is paying for something twice. Memory only matters to the most senior people as it affects decisions with long-term ramifications; in the organization-as-machine, those decisions are exclusively their purview. Open communications creates a lot of noise that interferes with orders from management. Having to explain why somebody needs something wastes time.

Traditional organizational thinking inhibits the conditions that create innovation. To have any reasonable expectation of innovation, we need to have the right expectations of how the organization functions if we are to manage in a way that fosters innovation.

Or course, a business is not managed by metaphor. But the way an organization is understood determines the way it is managed. A mindset rooted in learning rather than efficiency provides a set of "first principles" against which measurements and management decisions can be reconciled. It is therefore important to have the right mind-set about the nature of organization to understand how the organization of autonomous teams can exist at scale.

With these goals in mind, the characteristics of an organization of autonomous teams at scale become easier to understand. We'll look at those in the next post.

1 Morgan, Gareth. Images of Organization. Sage Publications, 1986.

Sunday, April 29, 2018

Organizing for Innovation, Part II

Last month we defined autonomy by the classes of decisions that are devolved to the team level, specifically that the smallest organizational unit - a team - has the ability to decide what it should do, can do, and will do. Looking at it this way makes clear the sharp differences between autocratic and autonomous management philosophies. It also helps us to understand that there need to be very special conditions for autonomy to succeed, even on a small scale.

It seems plausible that autonomy can work among a small group of people having a natural predisposition to collaborate and low asymmetry in their depth of skills and knowledge. But there has to be more to it than just a handful of similarly talented and like-minded people working together. If there isn't, than successful autonomous teams are largely an accident of hiring, and not a replicable phenomenon.

According to Morgan, there are four things that characterize an autonomous team.

One is redundancy of functions. Team members have the skills to be able to perform each other's jobs and substitute for one another when necessary. They are called "redundant" functions because each team member has skills they are not using for the work they are doing at any point in time (e.g., coding a new feature doesn't require a change to the build script). A team of poly-skilled people is itself an organization that is flexible enough to reorganize down to its most atomic level - the individual contributor. It adapts naturally because "[t]he nature of one's job is set by the changing pattern of demands with which one is dealing."1

By comparison, a team of specialists can be an autonomous unit when the external environment is stable, but it cannot sustain autonomy in the face of changing conditions because specialists lack the ability to adapt. When a specialized skill becomes unnecessary, the specialist becomes redundant along with it. A revolving door of members destroys the cohesiveness of a team.

The lack of individual adaptability also creates apathy within each member of the team. Problems such as poor quality or long time-to-market are seen as "someone else's problem" to solve because specialists working on the line don't know, or don't care, or don't have the authority to solve them. As a result, "[a] degree of passivity and neglect is thus built into the system."2

The team of specialists therefore lacks the capacity to self-organize because its members cannot change their job to reflect the changing patterns of demand, and because each member is invested in their skillset more than the team itself. Fixing problems within a team of specialists must be initiated and controlled by higher authority that exists outside the team. In dynamic external conditions, a team of specialists is doomed because the whole will always be less than the sum of its parts, while a team of generalists will acquire the skills and knowledge it needs to solve whatever the problem at hand may be.

Another characteristic of autonomous teams is requisite variety. A team's internal capabilities must mirror the breadth and depth of the environment within which it functions if it is to deal with challenges and opportunities posed by the environment. That skill variety must exist within the team itself so that it can be directly applied where and when it is needed.

A team lacking diversity of function must depend on others so that it can respond to environmental challenges. That dependency impairs a team's ability to self-organize and act, and therefore erodes its autonomy. For example, a team that develops an appreciation for something it should do will be inhibited from doing it if it has to negotiate with other teams for skills it does not have itself.3

Satisfying requisite variety is where technology platforms enable autonomous teams. While it is true that it is people and not assets who innovate, the assets can enable or prohibit such innovation. Teams that can consume components produced by others in a self-service manner do not suffer a dependency. The more comprehensive the components available for consumption, the greater the requisite variety a consuming team can possess, the larger and more complex the environment a single team can engage.

There is more to requisite variety than just skills and capabilities. It also makes a case for human diversity within a team. The appreciations a team develops are richer and more nuanced when they are recognized and crystalized through the diversity of its participants. Another way to look at it is, a homogeneous team will develop homogeneous solutions, and through a lack of human diversity will be structurally blinded to both opportunity and threat. By way of example, I once worked with a bank that was slow to realize that the average age of their employee matched the average age of their customer, that the average had been steadily rising for many years and was now well above the national population average. Year-on-year growth of assets under management looked spectacularly good, primarily because wealth distribution overwhelmingly favored the baby boomer generation. Unfortunately, it completely masked a dearth of new customer acquisition. Along the way, they had become generationally tone deaf, failing to develop experiences and products that appealed to younger generations and subsequently grow their customer base.

The next characteristic of autonomous teams is minimum critical specification. Vague charters and ambiguous boundaries create the capacity for self-organization because they build-in the expectation that teams are responsible for self-definition. A team cannot rely on management edicts that tell them what to do and how to do it. A team must instead define itself through practice and inquiry. General guidelines give a team an abstraction that they must constantly solve for, bringing them face-to-face with the appreciations, or "why" they do or do not do something.

Telling a team precisely what to do robs it of the capacity for self-determination and self-organization because it locks them into a swim lane. A team that is precisely chartered is institutionally specialized. We saw earlier that a team loses adaptability when its individual members lack redundancy of function. In a similar fashion, an organization loses adaptability when individual teams lack minimum critical specification, because teams themselves are stripped of their capacity to adapt based on what they see on the line.

Finally, a team must be capable of learning how to learn. This is also known as double-loop learning. Single-loop learning is the ability to detect and correct deviations from the norm, responding to threats to contain and minimize the impact of exceptions. In double-loop learning, a team is able to analyze a situation in its totality and question the relevance of the things that it does as well as the need to do things it is not doing. Single-loop learning is concerned with staying on-course. Double-loop learning is concerned with determining whether a team is doing what it should be doing in the first place. A well-functioning Agile retrospective is an example of double-loop learning.

Both minimum critical specification and learning how to learn point to the need for abstract thinkers, people who can understand a situation and adjust accordingly. Large ex-growth enterprises are operating companies, not developing companies. Operating companies need efficient execution, so they are populated with concrete thinkers, people who are conditioned through incentives and rewards and professional certifications to keep the ship sailing "steady as she goes". Abstract thinkers are constantly questioning why and looking for the right course of action based on all available information. To the concrete thinker, an exception is a problem to be contained. To the abstract thinker an exception is an opportunity to learn.

These four characteristics - redundancy of function, requisite variety, minimum critical specification and learning how to learn - make it possible for a team to self-organize, self-direct and self-regulate in response to changing external conditions. It is not difficult to see how these form the core characteristics of autonomy. It is also not difficult to see how their respective antitheses - specialization, dependency, precise chartering and single-loop learning - are the defining characteristics of the sclerotic organization.

Combined, these characteristics create what Susman4 calls "learning cells" within an enterprise. While they form the basis of the autonomous team, a cell does not simply multiply to form a more complex organization. We’ll next look at what it takes to scale the learning organization.

1 Morgan, Gareth. Images of Organization Sage Publications, 1986.

2 Ibid

3 If "does not have" is institutionalized as "can not have" through shared services - for example, because of an acute shortage of supply, or because those functions are used as control mechanisms to "protect production" - then the pretense of "autonomy" is a veneer over the management anti-pattern of "responsibility without authority".

4 Susman, Gerald. Autonomy at Work: A Sociotechnical Analysis of Participative Management Prager Publishers, 1976.

Saturday, March 31, 2018

Organizing for Innovation, Part I

Innovation happens through people, not assets. Assets can be an impediment to innovation: software that is brittle, monolithic, poorly encapsulated, or high-maintenance inhibits creative new uses of it. But assets don't innovate by themselves. Innovation happens through the people you have.

We saw last month that innovation is stifled where management's prevailing goal is control. If we want innovation borne of individual creativity, the reasonable thing to do is to look at organizational structures of autonomy and devolved decision-making. Unfortunately, as we saw two months ago, there are no formulas for devolving decision rights. We also saw there are few reference implementations, and no objective measures that show autonomous structures outperform command-and-control styles. Deciding to devolve requires unflinching conviction that it is the right thing to do, and the intestinal fortitude to muddle through what doesn't work to figure out what does. Because there are no half-measures of devolution, the stakes are high: by choosing to do this, you are betting your career and possibly the entire business on its success.

To better understand devolved decision making, it helps to understand the classes of decisions that define autonomy. According to Susman, there are three:

Scope Nature Environment Artifact Hierarchy
Institutional What should be done? Accommodate or defend against what it cannot understand or cope with Appreciations Board
Managerial What can be done? Decisions are uncertain and highly reactive Strategic plans Senior management
Technical How will it be done? "Supervisors of risk": decision making is fluid and creative Implementation plans Middle management

Source: Susman, Gerald. Autonomy at Work: A Sociotechnical Analysis of Participative Management

It is conceptually easy to understand how devolution works in small companies because the distance between decision makers and decision executors isn't very great. Start-ups don’t have large boards and employees take direction directly from the founder, who is less concerned with precision execution than finding things that drive usage and growth. Senior technology leaders who decide on the “how” are also the people who implement the “how”. There isn't much distance between the Chief Executive and the Chief Cook and Bottle Washer.

The larger the organization, the more polarized the control over each decision class. Appreciations - why should we do something - are the provenance of the board, who are few in number and very far removed from the insides of the company and the ecosystem in which it functions day-to-day. Questions of “what” are held tightly by management, providing a means of co-opting the board in assessing how well management executed, not necessarily on the success it achieved in exploiting the appreciations the board set forth. Held to performance targets from management, and saddled with lowest-common-denominator rented labor (thank you procurement departments everywhere for dehumanizing the secondary labor force for nearly two decades now), questions of “how” are similarly held tightly by technical managers.

The more disenfranchised the line - as in, the greater the extent to which individual employees are only permitted to do exactly what they're told to do - the harder it is for anyone to fathom a devolved model, let alone function within one.

The gulf between "stay in your lane" and "chart your own course" makes clear that there is much more to devolving authority than investing small teams with the responsibility of figuring out what they should do, can do, and will do. In part II, we'll look at the organizational characteristics of a self-directed team, one that functions in a genuinely autonomous manner. After that, we'll look at autonomy at scale: what needs to be in place for autonomous teams to function cohesively in a complex corporate ecosystem.

Wednesday, February 28, 2018

Innovation Versus Control

Firms in industries ranging from financial services to retail pharmacy to fast food aspire to be "platform companies." In the minds of their chief executives, the emergence of Amazon and the evident superiority of platform economics make this necessary for their continued survival. It is also a good story to tell Wall Street as it allows a firm to create the aura of being the technology leader in their space while trafficking in the success of companies like Amazon.

"Platform" is conceptually conveyed as a technological phenomenon. But it stands to reason that the defining characteristic of the platform organization is neither the technology assets that they produce (e.g., friction-free consumable primitives), nor how they produce them (e.g., lean and agile process). The benefits of a platform are only yielded if the creativity and imagination of the rank and file can be unleashed through those assets, to experiment, learn, and implement quickly. This means devolving decision rights far down into the organization, a.k.a. autonomous teams.

I've been brushing up on organizational behavior theory, and during my research I came across this paragraph. There is a lot of wisdom condensed into these two sentences:

In general, the longer the time period required for the consequences of strategic decisions to be realized and evaluated, the less flexible are resources for commitment to alternative objectives. Furthermore, (1) the longer the time period in which strategic decisions operate as constraints on the decisions made by technical-level personnel and (2) the lower the complexity of the tasks required to carry out operational plans, the more likely that operational planning will take place at a higher level.
-- Gerald I. Susman, Autonomy at Work

The first sentence neatly captures why things like Agile and Continuous Delivery and Lean Startup are so appealing. We reach critical mass of feedback on a strategic imperative - and therefore judgment on the wisdom of that imperative - more quickly with lots of frequent deliveries of small but business-valuable things than we do with infrequent, large deliveries of comprehensive business solutions. The sooner we reach the inflection point where a body of feedback confirms or contradicts a strategic decision, the more quickly we can move on to the next phase of our strategy, or change course. This separates the sclerotic laggards from the adaptive innovators. In addition, the presence of continuous market intelligence serves to separate the agile and adaptive from the strategic flailers.

This is intuitively obvious, but seeing it in black and white serves as a means test for the fulfillment of business strategy: is a firm asserting, confirming, or just guessing at what the market will buy?

The second part of the paragraph helps us to better understand the organizational dynamics within a small and innovative company versus those within a large integrated program team or an enterprise.

The first part is simple enough: the longer it takes to realize a strategic imperative... Longer is bad, check; already established in the first sentence. The second part is where it becomes interesting: and, the simpler the tasks required to deliver that strategic imperative... This statement is an indictment of the labor carrying out those tasks and the management defining them.

All together, the second sentence tells us that a long-lived initiative expected to be fulfilled through simple tasks relegates executives to the role of supervisor.

This is a damning statement in a number of ways.

The moniker "executive" is highly relative, potentially to a point of meaninglessness. The greater the degree to which technical execution is decomposed into simple tasks, the higher up the responsibility for operational planning. The higher up the responsibility for operational planning, the less meaningful the title of the person doing that planning. C-levels engaged in day-to-day prioritization and resource allocation are not executives. They are mid-level managers who have benefited from title inflation. It also means that the scope of executive decision-making - strategy - is concentrated in just a few hands. This renders quite a few people executives in title only, and deprives a company of its next generation of leadership by stifling their formation.

Anyone touting the potential for innovation from a delivery team engaged in task execution is living in a world of make-believe. Innovation stems from the combination of autonomy and complexity: give a team the freedom to solve a complex problem any way they see fit, and they are likely to come up with something novel. A system based on completion of simple tasks deprives a team of any complexity to sink their teeth into. Additionally, a system of rudimentary task completion is inherently a control system, which has zero tolerance for independent thought or action that is off-plan. Innovation is scarce where control is the priority.

Enterprise-y Agile processes function as systems of control, not innovation. Any system that adjusts the work to suit the labor instead of adjusting the labor to suit the work will require a high degree of centralized control. Enterprise Agile processes tolerate, and even advocate, decomposing work into tasks and assigning them to specialist labor. This values the control of labor over the creativity of labor. Per the previous point, technical-level employees are systemically disenfranchised. A system based on control through tasks offers no leeway for devolved decision rights; the only right an individual has is to complete the tasks they've been told to complete. This makes enterprise Agile processes more prone to suppressing than unleashing innovation.

The dynamics of small teams in small companies are not directly transferable to small teams in large enterprises. Small teams in small companies have high degrees of overlapping responsibility, little tolerance for specialization, light processes, and engage in high-bandwidth, omni-directional communication. Large organizations codify things such as roles and responsibilities, career development, processes, and work (e.g., technology) guidelines, and engage in low-bandwidth, hierarchical communications. In small companies, trust is largely based on the expectation that everybody will do whatever it takes to achieve a common outcome; in large companies, trust is largely based on the expectation that specialized people respond to precise requests with precise responses. Team dynamics are functions of HR structures, organizational values and systems, communication patterns, and ingrained behavior patterns, all of which are highly resistant and even subersive to change when they have decades to develop within a company. The executive in a legacy enterprise who says they want to transform the company into a "start-up" betrays their naïveté of the magnitude - and unlikeliness - of that task.

The paragraph at the beginning of this post captures what many in the tech biz have experienced for decades. Since the 1990s, enterprise IT has been a story of scale. As it scaled, it became more prominent on the income statement, and was forced to place a premium on control. Occasionally, it basks in the reflected glory of innovative consumer technology firms, or gets elevated by a CEO as a source of untapped potential. Unfortunately, enterprise IT has never been able to reconcile an expectation for innovation with the fact an over-emphasis on control gives everybody in management a demotion, suppresses innovation, and stifles attempts at organizational renewal, all while holding a company back from fulfilling its strategic potential because it takes such a long time to get anything done.

The most interesting thing about that paragraph? It was first published in 1976. Industrial, not tech firms, were the prominent companies of the time. The lessons remain the same.

Wednesday, January 31, 2018

You say you want a devolution...

"This isn't to say that alternative approaches to management are dead, or that they have no future. It is to say that in the absence of serious upheaval - the destabilization / disruption of established organizations, or the formation of countervailing power to the trends above - the alternatives to the Freds will thrive only on the margins (in pockets within organizations) and in the emerging (e.g., equity-funded tech start-up firms)."

-- Me, September 2013

I wrote that nearly 5 years ago. That previous summer I cracked the spine on some management books I had last read a quarter of a century earlier. When I first read those books in the 1980s, there certainly did seem to be a management revolution afoot. In the late 1970s, large industrial firms in the US were plagued with quality and performance problems, a rank-and-file that was fully aware of but apathetic to them, and management that was clueless about what to do. The epitome of the industrialized era in western nations turned out to be a company that would systemically disappoint both customer and investor alike. The long dominant organization-as-machine model was commonly perceived to have matriculated to a state of intellectual bankruptcy. Out went command-and-control, in came employee empowerment and team autonomy. Meet the new boss!

Yet when I read these management books anew in the early 2010s, it was clear that the revolution had been stopped dead in its tracks somewhere along the way. Same as the old boss!

I have had reason to re-visit this recently, this time in the context of enterprise technology platforms. A company that develops recomposable, atomic components that can be consumed in a self-service manner by other developers can help to yield more coarsely-grained solutions more quickly. Making those coarsely grained solutions recomposable components as well should enable an organization to create with both greater ambition and speed.

The objective of a platform is not to build both big and small things more quickly or to build more efficiently, but to create more effectively. A platform should allow for a greater number of experiments and more comprehensive feedback. Employees closest to an opportunity - current and potential consumers, technology, competitors, people and capital - are the ones best positioned to pursue that opportunity through exploring, learning, and adjusting. In an emerging area of business or tech, a local team muddling through stands a better chance of success than a distant management imposing its will over a market. In practice, muddling through experiments and feedback requires some degree of authority devolved to the team level, so that a team can decide and act for themselves.

The notion of authority devolved to the team level brings up the question of the autonomous organization yet again. Plus ça change...

The same old idea comes with the same old questions. What does an organization of autonomous teams look like? Can it work? How does it scale?

Before we ask, "can an organization of autonomous teams work?", we have to ask, "what does autonomy at team level mean?" Does it mean the authority and responsibility for what they do and when they get it done? Does it include design and architecture? Can they act on things that are nominally the responsibility of other teams? Do they get to pick and choose the people on their team and the providers they source people from? Do they have to secure their own funding? Who do they answer to? How are they measured?

It may mean all of these things, or it may mean just a few. Autonomy is in the eye of the beholder. To some, just having operational autonomy - authority over what, when and how a team fulfills delivery goals - is sufficient. To others, operational autonomy without owning the P&L and balance sheet - everything from capital to compensation levels - is merely responsibility without authority under the guise of self-direction.

Every firm that has gone down this path has come face to face with the same questions and challenges. Every firm of any scale that has achieved any degree of success has ended up with some hybrid implementation: some things are decentralized, some things centralized; some for a short period of time, others for a longer period of time, and some permanently. For example, we want teams to be responsible for the production operations of their creations, but we must first incubate an ops capability; once we are comfortable that ops has completed its gestation period it will be broken up and absorbed into the line teams. However, to alleviate administrative burden and to avoid violating labor laws we will have a centralized HR function, but we do want ideas to compete for funding, so we will have utility and risk capital allocation processes.

One question, many different answers, and answers that change at different points in time as circumstances require or allow.

When there are many different answers to a single question, it is the wrong question to ask. Looking for specificity where there is none will only sow seeds of confusion and ultimately doubt. And, while there is plenty to be learned from the experiences of others, self-reported testimony must be taken with a grain of salt, and the success of others comes with no guarantee of portability.

A better question to ask is, how convinced are you that team autonomy is a solution to whatever challenges you face? You need to be overwhelmingly convinced that it is, because you need a high tolerance for the ambiguity, uncertainty, and constant adjustments and experiments you will have to run to find and maintain the right balance - that is, construct the right hybrid - for your set of circumstances. You also have to be comfortable without a lot of hard evidence that it solves whatever you had hoped that it would. Even had you not devolved a greater degree of decision-making to the team level, that product might have been a success, that innovation might have emerged, those employees might still have joined your firm. Can't prove the counterfactual.

If you are convinced, and decide to add your name to the list of those that have elected to crack this nut, the operationalizing questions are much different. The one that you will ask again and again and again is the obvious: how do we strike the balance: what do we think that hybrid should be today? what do we think it could possibly be? how do we go about figuring that out?

In addition, given the cyclical love-hate relationship with devolved authority, you must also ask: what makes it more likely, and what makes it less likely, that it will have staying power in your organization?