I consult, write, and speak on running better technology businesses (tech firms and IT captives) and the things that make it possible: good governance behaviors (activist investing in IT), what matters most (results, not effort), how we organize (restructure from the technologically abstract to the business concrete), how we execute and manage (replacing industrial with professional), how we plan (debunking the myth of control), and how we pay the bills (capital-intensive financing and budgeting in an agile world). I am increasingly interested in robustness over optimization.

I work for ThoughtWorks, the global leader in software delivery and consulting.

Wednesday, August 31, 2016

Method, Part II

Last month, we looked at method as a codification of experience borne of values and expressed through rules, guidelines, practices, policies, and so forth. This month, we'll take a look at the relationship between method and the things that influence it, and that it influences.

The principal framework is an article by Cliff Jacobson describing the change in method that impacted camping and outdoor activity starting in the 1950s, drawing comparisons to changes in method in software development. When we think of method in software, we generally think big: "Agile versus Waterfall". But there are more subtle changes that happen in method, specifically through the codification of skill into tools.

Plus ça change...

* * *

Method...

... and Values

"Environmental concerns? In those days, there were none. Not that we didn’t care, you understand. We just didn’t see anything wrong with cutting trees and restructuring the soil to suit our needs. Given the primitive equipment of the day, reshaping the land was the most logical way to make outdoor life bearable.
"In 1958 Calvin Rutstrum brought out his first book, The Way of the Wilderness. Suddenly, there was new philosophy afield. Calvin knew the days of trenched tents and bough beds were numbered. His writings challenged readers to think before they cut, to use an air mattress instead of a spruce bed. Wilderness camping and canoeing were in transition."
-- Cliff Jacobson

As values regarding nature changed from "tame the land" to "conservation", the method of camping had to change. Of course, it took a long time for the new values to settle in. And even once it did, it took a long time for practitioners to change what they did. Resistance to change is a powerful thing, and both practitioner and gear lag would have kept practitioners executing to an old value set in the field for a long, long time.

Values changed in software, too. When users of software were largely internal to a company, before software became such a high cost item for non-tech companies, and before software was weaponized, development moved at a much slower and more deliberate pace. Once the values changed, the method of software delivery was also pressured to change. The change in relationship between humanity and the outdoors is similar to the change in relationship between companies and their software.

... and Skills

But this narrative applies to both a wholesale change in method as well as to the transition from skills-centric to tool-centric method.

"I discovered the joys of camping at the age of 12 in a rustic Scout camp set deep in the Michigan woods. It was 1952, just before the dawn of nylon tents and synthetic clothes. Aluminum canoes were hot off the Grumman forms, though I’d never seen one. Deep down, I believed they’d never replace the glorious wood-ribbed Old Towns and Thompsons."

Early backpackers had to make do with bulky tarps, fashioning poles and tent pegs from branches - sometimes, even sapling trees - in their campsites. The emphasis among the early outdoorsmen was on the skill of adapting the environment to human survival, and achieving Spartan levels of comfort was a symbol of mastery. Being able to fashion poles and pegs from tree limbs was important in the 1940s as tents didn't necessarily come with them. This was not only destructive, it became unnecessary with the evolution of lightweight and portable aluminum poles and stakes. In a relatively short period of time, being good at pioneering became, at best, only useful in an emergency (you need to fashion a tent peg because you discover you've lost some aluminum ones).

Building Agile trackers in spreadsheets and crafting them anew with each project was somewhat akin to fashioning new tent pegs every time you go camping. Creating a new tracker with each project was a waste of money, and having 5 different teams with 5 different trackers was confusing. The advent of cheap commercial trackers made this unnecessary. Still a good skill to have in an emergency - a project tracker so badly polluted with low priority Stories and tasks is an impediment when you want to make a clean and quick start - but fashioning a tracker is no longer itself a core skill.

... and Tools

"The emphasis had shifted from skills to things."

Early tools supporting a method are crude, often hand made and narrow in their usefulness, and several tend to spring up at the same time. The emphasis is on skills.

But with ever increasing popularity of the activity (trips to the Boundary Waters, or Agile software development) comes the tools. Skills take time to learn and master. Tools make the activity at hand easier to perform, and subsequently more accessible and more enjoyable to more people because they're more successful at it. Canoes are made of strong yet lightweight materials so they're more tolerant to misuse while simultaneously easier to portage. Sleeping bags are made of synthetic materials that are water resistant (unlike down) so that somebody who does a sloppy job at packing a canoe pack won't suffer if the bilge water soaks the contents of the bag.

Of course, tools can be a source of efficiency or a source of trouble. A hatchet makes it easy to build safe, small fires out of short cut sticks. But a hatchet can cause grave injury to somebody if they don't know the proper method for chopping wood with it. Nobody is likely to suffer bodily harm from an over-engineered build script (no matter how many felonious thoughts cross the mind of other people in the team) but an overloaded, single-stage build that reduces build frequency and that fails frequently will cause more harm than good.

"Today, high-tech gear and high-powered salesmanship have become a substitute for rock-solid outdoor skills."

As the complexities of a method get codified into the gear, it becomes difficult to separate one from the other. The tools become a proxy for the method because the state-of-practice matures in conjunction with improvements in science (materials or software) and affordability. Today, we create sophisticated, multi-stage pipelines that instantiate their deployment environments and deploy on every commit. 15 years ago, it was amazing to have a build run every few minutes that would both compile source code and run tests. We can't imagine forging our own crude tools to do basic tasks, or even why we'd want to do it.

Newer tools don't lend themselves to older practices. Tightly rolling a modern (down) sleeping bag won't get it into its stuffsack. Managing cloud instances like rack-mounted servers in a physical data center will run up the bills really fast.

This can be a serious point of confusion for middle managers tasked with making their organization "Agile". If we use Jira, Jenkins and jUnit we must be Agile.

... and People

"I felt quite inadequate, like a peasant in Camelot."

Tools can render entire skills sets irrelevant. The right brain creativity to fashion a tracker for some specific project was no longer needed when the commercial tracking tools arrived. It became a left brain activity of making sure all the requirements were entered into the tracker and configuring canned status reports. Suddenly the thing somebody did that was an act of value has been rendered obsolete by the gear.

The information modeler who was capable of telling the right story based on the nature of the task and team is shoved aside by the efficient administrator who's primary job is to maintain team hygiene. It's entirely possible that the hygienist doesn't really understand why they perform the tasks they perform, but they've been told to hustle people to stand-up and make sure people update status on their (virtual) cards. They're also much cheaper than the craftsman they replaced.

This is pretty destabilizing to people. Where Cliff Jacobson felt inadequate by the gear (and the associated cost), the individual can be stripped of their own sense of self-worth by a change in the method. This can happen when the method changes owing to the values (we need to deploy daily and Waterfall won't let us do that). You might have fancied yourself pretty good at software within your organization, but now the boss is telling you that your worldview is out of touch, you're not up to scratch and you're not only told that you're going to do it differently, but how you're going to do it. That's not likely to elicit warm and welcoming feelings. Just the opposite.

But it can also happen when the change in method is a shift from skills to things. Suddenly anybody can appear to be good at project tracking. That can stir resentment that encourages resistance to the tools and pine for the spreadsheets.

The reverse - a sudden shift from tools to skills - has no less an impact. There are development stacks that are entirely tool driven. When the boss comes in and announces that all vendor dependencies in the code and process gotta go, the tool dependency no longer compensates for weak skills. The person accustomed to going glamping may not much care for back country backpacking.

... and Basics

"Chemical fire-starters take the place of correct fire making; indestructible canoes are the solution to hitting rocks; bizzard-proof tents become the answer to ones inability to stormproof conventional designs; GPS positioning has replaced a map and a compass. And the what-if-you-get-your-down-bag-wet attitude attracts new converts every year. In the end, only the manufacturers win."

Cliff Jacobson argues that tools are a poor substitute for skills. Where they support the value system - Leave No Trace camping - they're welcome. But where they are simply gadgets for convenience or separate the individual from the experience, they're not. They're also predatory, exploiting a person's laziness, or fear of being unable to master a skill, or feeling of inadequacy in dealing with challenging situations that might arise.

To same extent, the impulses that spurred the software craftmanship movement are likely similar to those of Messers Rustrum and Jacobson:

"'I’ve canoed and camped for nigh on seventy years and have never got my down bag wet,' he bellered. 'People who get things wet on trips don’t need new gear. They need to learn how to camp and canoe!'"

Pack correctly and paddle competently and you'll never sleep in a soggy bag. We don't need armies of people mindlessly executing test scripts if we build in quality in the first place.

... and the future of method.

Method is a mirror, not a driver. It reflects values and experience, it doesn't create them. Values shift as our priorities change. Experience changes as we learn what new technologies allow, and sometimes re-learn discipline long lost. Method reflects this; it doesn't inform or define this. The values and experience are there, or they are not.

Method is never a destination. It's an echo.