Friday, August 19, 2016

Refactoring the Enterprise

Why Refactoring – Is this yet another Agile Method

First, this is not a call for another Agile Method.  I won’t try to write a book and repackage a bunch of stuff you already know and then offer to certify you.  Nothing wrong with that but not my thing.  Second, I am using Refactoring because I have been recently re-inspired by doing refactoring katas thanks to Llewellyn Falco and Woody Zuill and reading some great books The Nature of Software Development by Ron Jeffries and Building Microservices to name two. What I describe below is similar in some ways to refactoring legacy code.

I have been talking about this idea of carving out a bit of business capability that a single team can deliver and run without the need to coordinate with other teams. The fancy name for this is Domain Driven design with Bounded Context.

Most businesses these days, regardless of the product or service they deliver, have become software development companies. Many of these companies are running a fragile, organically grown, or acquired, collection of code and infrastructure that is difficult and risky to change. At the same time, market demands, competition, and a changing technology landscape make business agility more important than ever. 

There are already some enterprise frameworks for agile but they feel heavy to me and they seem to put the focus on detailed upfront planning and ongoing processes which seems inconsistent with the Agile Manifesto.

I think a refactoring approach can be used to transform organizations in an iterative, emergent way that limits risk and maximizes long term ROI.  Here’s what that might look like.

Meet with executives at any level in the organization to understand their strategic objectives and known barriers to achieving them.

·      A strategic objective might be to provide device specific, yet consistent access to product information.
·      A barrier to achieving that might be that the business rules are embedded in the webpage code and backend systems without a clear delineation. Creating a mobile app that provides the same selection and pricing as the existing website could be very expensive and require a huge amount of testing with a long time line.

Enterprise and Solution Architects are in the best position to identify challenges and opportunities in the IT space that align to those objectives and remove barriers. The architects will also have a key role in integrating new services into the existing architecture.

Because we must be able to deploy at any time – even multiple times per day - organizational leaders must clear a path for a parallel delivery pipeline that allows on demand deployment to production environments. Versioning and feature flags allow partial feature upgrades and multiple versions of software to coexist in the production environment.  Creating a parallel delivery path allows the lights to stay on while we iterate to the fully workable solution.

One way to start the journey would be to select a specific business capability that needs to change, or scale, but is difficult or risky in the current process. After creating the bounded context; design a product that provides that business capability to customers – who may be internal or external. 

For our example above we might create a service that encapsulates the business rules and the product data to determine which products should be offered to a customer. The service could be called via a restful interface that includes specific selection criteria.  The service would return a list of products (prices might be included or might be provided by a different service.) The development team for each device platform could decide how best to display the offerings.

To get started on the new service we would form a product team (Product Owner, Developers) that will develop and support the product starting with the smallest useful feature set, MVP, in parallel with the current delivery scheme. The Developers are responsible for testing and supporting their product – even in production.  The Product Owner is responsible for maximizing value and being available to the team at all times – What Woody calls Decision Queue Time – can easily multiply the time to value for a development effort.

To ensure the business agility we promise, it’s not enough to just write the software. The product team must be able to deliver running tested features into production on their first iteration.  This means they need to have version control, build automation, acceptance test automation, non-functional test automation, and automated deployment from the very beginning. Assign a Coach to help with these things and the ongoing challenges they will face.  The first running tested feature may be as simple as “Hello World” but it will be in version control, have been automatically built, passed its acceptance tests, passed non-functional requirements testing, been automatically deployed into the production environment, and validated as available and responsive. This is the proof that all the parts work together and it’s a great way for the team to experience the definition of done.

It is important to make clear the expectation of excellent technical practices.  Release to production every iteration using automated tests and automated build/deploy.  Version APIs and use feature flags as required to enable trunk based development.  Design and build in monitoring, high availability, recovery, graceful degradation, and scaling from the beginning.

Also, every developer must be familiar with and responsible for the quality of every piece of code. This is a key risk mitigation strategy; nobody wants to worry about crucial code that only one person can work on or complex code that fights modification.

This approach provides learning early and often but still requires patience. The first effort will be slow – people will need training and experience.  There will be missteps and disappointment along the way.  Understand and plan for this – you have already reduced risk by undertaking this as a parallel effort to whatever process is already in place and by automating tests and deployment.

We started off talking about an enterprise transformation but we’ve only talked about a single team so far. In the spirit of the agile manifesto we want to iterate and learn. When the first effort starts to show success, add another product and team.  Use this pattern of adoption until you run out of products and markets that need attention.  Your organizational structure will change over time to reflect the new architecture.  Many teams will be able to support more than one related product – that will emerge as you go.

Changing the focus from the activities of Agile/DevOps adoption to the achievement of strategic business objectives through refactoring allows us to start learning faster and to stop at any time with minimal waste. Moving to a product focus allows us to leverage Conway’s law in reverse to transform the organizational structure around the business outcomes we are trying to create.

I am interested in your feedback especially if you have specific examples and ideas on why this could never work in the complex domains of the real world.

Thanks for reading --Mike