I consult, write, and speak on running better technology businesses (tech firms and IT captives) and the things that make it possible: good governance behaviors (activist investing in IT), what matters most (results, not effort), how we organize (restructure from the technologically abstract to the business concrete), how we execute and manage (replacing industrial with professional), how we plan (debunking the myth of control), and how we pay the bills (capital-intensive financing and budgeting in an agile world). I am increasingly interested in robustness over optimization.

I work for ThoughtWorks, the global leader in software delivery and consulting.

Monday, September 18, 2006

Management Trends in Financial Services Application Development

IT in Financial Services is maturing into a business-value measured and driven capability. To meet demands for both rapid business innovation and high quality, it is moving toward highly disciplined, low ceremony processes that give responsibility and ownership as well as enable maximum performance, from highly capable people.

Application development in the more volatile parts of Financial Services is typically characterized as being "on fire" more than it is considered a "centre of excellence." This is accepted as m.o. largely because in these parts of the business, IT has simply ceeded control to the business operations, such as the trading desks of securities firms. Traders, being belligerant, noisy and aggressive, typically call the shots with IT. No doubt, they bring business urgency: millions of dollars (if not more) are on the line with each passing second. However, having ceeded control, IT only manages to the moment, not to an overall capability to manage business urgency. This means a lot of mistakes and rework, subsequently inefficiency and cost, precicely in the parts of the business that would benefit the most from disciplined responsiveness.

Financial Services IT increasingly recognizes that a lot of it's inability to respond is self-inflicted. The drive toward Continuous Integration is an example: how much time is wasted and how much delay is introduced by doing a build by hand, something that's done once a week (if not more frequently) in support of a production or QA release? Build can be transparent, and just about as close to something happening in zero time as you can get in IT. Because it brings a benefit of tremendous quality improvement for relatively little cost of implementing the practices and the tools (CruiseControl being free), there has recently been a signifcant groundswell of adoption in Financial Services development to automated build.

There's three specific management-oriented trends that suggest the ad-hoc, purely reactive activities are giving way to a more controlled, consistent way of working.

An emphasis on structuring, capturing and managing requirements is the first trend. Teams writing trading applications typically have little discipline in requirements definition and management. Priority decisions are off the cuff, meaning a lot of information isn't captured and expectations are simply not met. There is a growing trend toward structurally simple, highly accessible, easily managable and ubiquitously understandable expression of software requirements. This provides a durable communication channel between the business and IT as to what needs to be done, it survives business shocks of priority change and staff turnover, and it does so in a lightweight, non-ceremonial, non-cumbersome way. To that end, a common language and structure for requirements expression - the Story - is beginning to take root, specifically because it effectively communicates the essentials: the role, need, justification and acceptance criteria driving a requirement from a business, not a technical, perspective. That Stories can be managed with open-source repositories such as XPlanner, and that these scale from the team to a global level, make professionalizing requirements capture a low-disruption and high-value-added change.

The pursuit of uniform operational transparency is the second trend. It's increasingly less acceptable to have management by "I feel good about this" and increasingly more important to express the course, speed and direction of a project in a fact-based manner. Short delivery windows (e.g., iterative delivery) coupled with clear business requirements (the prior point) provide a controlled responsiveness to change. Together, what's in, what's not, what's priority and what's bumped are unambiguously communicated; in addition, team performance - of targets exceeded or missed, of capacity in excess or short supply - are factual, exposed and nearly indisputable. Particularly in the most volatile of business environments they provide accuracy of what's actually happening on the ground, both professionalizing delivery management and preventing a team from devolving into a free-for-all. Ultimately, the combination provides a great deal more information as to what everybody is doing for the business. They also uniquely and uniformly scale-up to a programme or a department, giving global status reporting that exposes hotspots and happystates well in advance of delivery dates. This approach to management provides unparalleled transparency: there is true uniformity across and within a large programme of work because it doesn't allow substitution of opinion ("we're about xx% done") for fact ("xx% of requirements are QA complete.") While it takes longer to onboard, the trend toward iterative delivery and status reporting at both project and programme levels is becoming established in Financial Services IT.

Greater rigor in identifying, recruiting and retaining best-and-brightest is the third trend. Financial services companies including investment banks, insurance companies and funds managers are all posting record numbers. Simultaneously, venture capital and angel money are back and bigger than ever, and the overall IT economy is healthier and more structurally sound than it was in the late unpleasantness (2001-2003). This is creating tremendous pressure on IT organizations to recruit and retain the high-performance people who thrive in dense-and-dynamic environments. To this end, financial services companies are increasingly scrutinizing the ways in which they hire, looking to ways to identify not just good executors, but people who (a) understand how to satisfy the requirements of the business doman and, (b) know how to maintain operational discipline so that they're not over-run by the business domain. This requires more than just good coders, analysts and managers; it requires people with high degrees of situational awareness and professionalism. Finding the right people is more than just technical and personality questions, a company must be seen to be a destination employer to attract them, build the screening processes to identify them, and maintain the environment to retain them. The feedback from IT to HR is already affecting recruiting practices, and it will continue to drive it in this direction as demand increases.

Of late, there has been a metrics trend that swept through IT organizations. This has turned out to be not so much a current trend as much as an echo of the future, a foreshadowing of things to come. There's still demand for performance, output and compliance metrics in IT, but the realization is that the call for metrics can't be answered until items 1 and 2 come to fruition. Once portable, consistent and durable requirements capture and fact-based project management have a firm footing, IT will be ready to address metrics again. Reliable metrics will still require integrity of completion (i.e., that every piece of code is certified to functional and technical tests prior to being called complete), but improved analytics and project management will lead the introduction of business-oriented metrics to IT.

The bottom line: Financial Services IT is moving toward low-ceremony, disciplined processes that give autonomy to enable maximum performance from highly capable people. Strength in fundamentals will ultimately allow IT to both measure performance and drive results in business impact terms.