I consult, write, and speak on running better technology businesses (tech firms and IT captives) and the things that make it possible: good governance behaviors (activist investing in IT), what matters most (results, not effort), how we organize (restructure from the technologically abstract to the business concrete), how we execute and manage (replacing industrial with professional), how we plan (debunking the myth of control), and how we pay the bills (capital-intensive financing and budgeting in an agile world). I am increasingly interested in robustness over optimization.

I work for ThoughtWorks, the global leader in software delivery and consulting.

Saturday, December 31, 2016

Myths of Replatforming

Replatforming is all the rage these days.

Platforms are conceptually popular with investors: in theory, a platform makes the mundane portions of a business efficient, scalable and adaptable, allowing a company to release the creative talents of its people to pursue growth and innovation. Replatforming makes for a convincing story following an acquisition because it explains to investors how deal synergies will be achieved, sets a tone of equivalency among employees of both the acquired and acquiring firm, describes a vehicle through which the company will reinvent itself with modern technology and practices, and conveys pragmatism in the need to clean house of dilapidated infrastructure. The replatformed business promises innovation at scale and larger cash flows from operations that, combined, position it to grow both organically and through acquisition. Destiny awaits.

In practice, replatforming is messy. Core systems are a complex web of integrated rules and functions spun over generations. They're difficult to disentangle because we have all the wrong people: most of those with the business understanding that went into building those systems have long since left, while the architects and senior engineers who created the outer layers - the web and mobile components - are still on the payroll. Replatforming initiatives become a black hole almost from the start because people don't know how to "eat the elephant": their ways of working are out-dated, they struggle to come to terms with the totality of what needs to change, and they can't envision a future that is much different from the present.

I've had a front row seat to a number of replatforming initiatives over the years, and I've seen several myths that plague them from the inside.

We'll build the platform first, then change the organization once it's live. Employees will see replatforming as an exercise in re-creating software. The existing systems have their shortcomings, but we run a multi-billion dollar business on that code, so we must be pretty good at software. Let's stick to what we know to create the software - because that's what the business needs, right? - and then we'll change process and organization once it's up and running. The fatal flaw in this line of thinking is Conway's Law: software mirrors the communication patterns of the organization that develops it. When the goal of replatforming is to rapidly innovate at scale (as it is usually alleged to be) you have to start with cross-functional, integrated teams with devolved authority that can autonomously deliver end-to-end solutions. Coming from traditional organizational structures of hierarchy, shared services and specialists, that's a lot of change. It creates confusion and disorder that just about everybody is uncomfortable with. It also makes for a slow start on developing the software that makes middle managers nervous; the more nervous they get, the more inclined they'll be to abandon organizational change. An over-managed, hierarchical, silo'd organization will develop over-engineered, tightly-coupled, brittle software; no after-the-fact restructure will compensate for that because you're bound by Conway's Law. As the saying goes, 'tis always best to begin as one intends to proceed. Replatforming doesn't succeed unless organization change precedes.

We need a product organization. As business operations become more encoded in algorithm and less executed through manual labor, we need long-term stewardship of our software from both our business and engineering communities. The popular way of doing this is by creating a product organization. Don't bother doing that if you can only define their responsibilities in a self-referential manner (e.g., "the product owner owns the product"), if you disproportionately define the scope of their responsibilities as user experience, if they have no direct accountability for how their outcomes impact the line of the business, and if you're substituting systems knowledge (how things work) for business knowledge (why things work). Not only is this not value-generative, it adds a layer of decision making intermediation and creates ambiguity in responsibilities for requirements, analysis, design, and prioritization. It's made much worse when product managers are alleged to have authority, but have their decisions reversed by higher-ups or invalidated by people with stronger business knowledge. Per the prior paragraph, we need to create the organization that will both create and perpetuate the platform. Creating a mature product organization is hard. Better to encapsulate product responsibility into the line of the business (where it arguably should be in the first place, and not adjunct to it, but that's another blog for another day) than to create a product organization stuck in perpetual adolescence. The latter will result in systemic distortions in the code, courtesy Conway's Law.

We need to put all business requests on hold for the next [long period of time] while we replatform. The older a company is and the more acquisitions it has been involved with, the more far reaching and complex its core systems will be. The more depleted it is of business knowledge - the why, not the how - the more mysterious those systems will be. Employees will be predisposed to define a new platform through the lens of feature parity with the old. The lower the caliber of business knowledge (that is, the more that system knowledge substitutes for business knowledge), the higher the degree of feature parity that employees will insist defines the minimum viable product - a position reinforced by traditional software development methods that released software once every few months, not days or hours. Additionally, ambitious replatforming efforts lay bare deficiencies in organization, skills, capability, knowledge, process, and infrastructure. Changing those takes time. These two points are conjoined to form the mistaken belief that business needs will have to take a back seat while people figure all this stuff out, but those business needs would be deferred regardless because there's no way to go live with a partial solution, and we can't very well pursue a moving target. To destroy this myth, start with that "long period of time" the business is being asked to wait. It always seems to be on the order of a year to a year and a half. At the very least double it, because nobody's estimate for something they've never done before is going to be particularly accurate. Think your customers will wait that long - two to three years - for you to get your act together? At best, a fork-lift replacement will get you tomorrow where you needed to be yesterday. From an engineering perspective, progressive strangulation of incumbent systems is almost always a question of will, not possibility. From a business perspective, progressive strangulation is a question of personas and user journeys, not features and functionality.

We'll build it the way we should have built it if we knew then what we know now. It's tempting to think our best bet is to re-build something with the wisdom our people gained from the experience of building it the first time round. That's a risky bet. It assumes they've drawn the right conclusions from their experiences. That's a lot to hope for, since that requires a lot of introspection, critical analysis, and personal acknowledgement of error and mistake, something most people aren't predisposed to do. It also assumes they've kept current with technology and process. Microservices, containerization, elastic cloud, and continuous delivery are significant departures from where software development was just a decade ago. The people who got the company into the mess it's in aren't the people who will get the company out of it. In their hands, you'll get what you've already got, only worse, like multi-generational in-bred aristocracy. Replatforming requires forward thinking technology, ideas, and execution. Changing culture, capability and mind-set requires a major transfusion; be prepared to lose a lot of blood.

Aging infrastructure, cheap capital and a dearth of innovation have fueled consolidation in a variety of industries. Replatforming will be with us for as long as those conditions exist. The myths don't have to be.