I consult, write, and speak on running better technology businesses (tech firms and IT captives) and the things that make it possible: good governance behaviors (activist investing in IT), what matters most (results, not effort), how we organize (restructure from the technologically abstract to the business concrete), how we execute and manage (replacing industrial with professional), how we plan (debunking the myth of control), and how we pay the bills (capital-intensive financing and budgeting in an agile world). I am increasingly interested in robustness over optimization.

I work for ThoughtWorks, the global leader in software delivery and consulting.

Wednesday, February 28, 2018

Innovation Versus Control

Firms in industries ranging from financial services to retail pharmacy to fast food aspire to be "platform companies." In the minds of their chief executives, the emergence of Amazon and the evident superiority of platform economics make this necessary for their continued survival. It is also a good story to tell Wall Street as it allows a firm to create the aura of being the technology leader in their space while trafficking in the success of companies like Amazon.

"Platform" is conceptually conveyed as a technological phenomenon. But it stands to reason that the defining characteristic of the platform organization is neither the technology assets that they produce (e.g., friction-free consumable primitives), nor how they produce them (e.g., lean and agile process). The benefits of a platform are only yielded if the creativity and imagination of the rank and file can be unleashed through those assets, to experiment, learn, and implement quickly. This means devolving decision rights far down into the organization, a.k.a. autonomous teams.

I've been brushing up on organizational behavior theory, and during my research I came across this paragraph. There is a lot of wisdom condensed into these two sentences:

In general, the longer the time period required for the consequences of strategic decisions to be realized and evaluated, the less flexible are resources for commitment to alternative objectives. Furthermore, (1) the longer the time period in which strategic decisions operate as constraints on the decisions made by technical-level personnel and (2) the lower the complexity of the tasks required to carry out operational plans, the more likely that operational planning will take place at a higher level.
-- Gerald I. Susman, Autonomy at Work

The first sentence neatly captures why things like Agile and Continuous Delivery and Lean Startup are so appealing. We reach critical mass of feedback on a strategic imperative - and therefore judgment on the wisdom of that imperative - more quickly with lots of frequent deliveries of small but business-valuable things than we do with infrequent, large deliveries of comprehensive business solutions. The sooner we reach the inflection point where a body of feedback confirms or contradicts a strategic decision, the more quickly we can move on to the next phase of our strategy, or change course. This separates the sclerotic laggards from the adaptive innovators. In addition, the presence of continuous market intelligence serves to separate the agile and adaptive from the strategic flailers.

This is intuitively obvious, but seeing it in black and white serves as a means test for the fulfillment of business strategy: is a firm asserting, confirming, or just guessing at what the market will buy?

The second part of the paragraph helps us to better understand the organizational dynamics within a small and innovative company versus those within a large integrated program team or an enterprise.

The first part is simple enough: the longer it takes to realize a strategic imperative... Longer is bad, check; already established in the first sentence. The second part is where it becomes interesting: and, the simpler the tasks required to deliver that strategic imperative... This statement is an indictment of the labor carrying out those tasks and the management defining them.

All together, the second sentence tells us that a long-lived initiative expected to be fulfilled through simple tasks relegates executives to the role of supervisor.

This is a damning statement in a number of ways.

The moniker "executive" is highly relative, potentially to a point of meaninglessness. The greater the degree to which technical execution is decomposed into simple tasks, the higher up the responsibility for operational planning. The higher up the responsibility for operational planning, the less meaningful the title of the person doing that planning. C-levels engaged in day-to-day prioritization and resource allocation are not executives. They are mid-level managers who have benefited from title inflation. It also means that the scope of executive decision-making - strategy - is concentrated in just a few hands. This renders quite a few people executives in title only, and deprives a company of its next generation of leadership by stifling their formation.

Anyone touting the potential for innovation from a delivery team engaged in task execution is living in a world of make-believe. Innovation stems from the combination of autonomy and complexity: give a team the freedom to solve a complex problem any way they see fit, and they are likely to come up with something novel. A system based on completion of simple tasks deprives a team of any complexity to sink their teeth into. Additionally, a system of rudimentary task completion is inherently a control system, which has zero tolerance for independent thought or action that is off-plan. Innovation is scarce where control is the priority.

Enterprise-y Agile processes function as systems of control, not innovation. Any system that adjusts the work to suit the labor instead of adjusting the labor to suit the work will require a high degree of centralized control. Enterprise Agile processes tolerate, and even advocate, decomposing work into tasks and assigning them to specialist labor. This values the control of labor over the creativity of labor. Per the previous point, technical-level employees are systemically disenfranchised. A system based on control through tasks offers no leeway for devolved decision rights; the only right an individual has is to complete the tasks they've been told to complete. This makes enterprise Agile processes more prone to suppressing than unleashing innovation.

The dynamics of small teams in small companies are not directly transferable to small teams in large enterprises. Small teams in small companies have high degrees of overlapping responsibility, little tolerance for specialization, light processes, and engage in high-bandwidth, omni-directional communication. Large organizations codify things such as roles and responsibilities, career development, processes, and work (e.g., technology) guidelines, and engage in low-bandwidth, hierarchical communications. In small companies, trust is largely based on the expectation that everybody will do whatever it takes to achieve a common outcome; in large companies, trust is largely based on the expectation that specialized people respond to precise requests with precise responses. Team dynamics are functions of HR structures, organizational values and systems, communication patterns, and ingrained behavior patterns, all of which are highly resistant and even subersive to change when they have decades to develop within a company. The executive in a legacy enterprise who says they want to transform the company into a "start-up" betrays their naïveté of the magnitude - and unlikeliness - of that task.

The paragraph at the beginning of this post captures what many in the tech biz have experienced for decades. Since the 1990s, enterprise IT has been a story of scale. As it scaled, it became more prominent on the income statement, and was forced to place a premium on control. Occasionally, it basks in the reflected glory of innovative consumer technology firms, or gets elevated by a CEO as a source of untapped potential. Unfortunately, enterprise IT has never been able to reconcile an expectation for innovation with the fact an over-emphasis on control gives everybody in management a demotion, suppresses innovation, and stifles attempts at organizational renewal, all while holding a company back from fulfilling its strategic potential because it takes such a long time to get anything done.

The most interesting thing about that paragraph? It was first published in 1976. Industrial, not tech firms, were the prominent companies of the time. The lessons remain the same.