Shamrock Mapping - A simple tool for improvement

I am a big fan of Wardley Maps and draw simple maps on a white board or napkin during every discussion of strategy.

Simon Wardley says they are imperfect as all models are and that's true - they are also the best way I know to facilitate a strategic conversation.

Chris McDermott has created a nice variation - Maturity Mapping - that combines Wardley Maps and Cynefin to help teams understand and address their maturity in different areas.

I have recently been using yet another variation that I call Shamrock Maps. A Shamrock Map is super simple and can be used for any part of the value chain. I usually draw a regular Wardley map using Visibility and Evolution as the axes and then zero in on the area(s) of strategic interest.

For those areas, I change the axes to Autonomy and Mastery - autonomy is our ability to move freely without interference from external factors; mastery is our ability to move freely without interference from ourselves.

I have been starting with a guess about where a particular activity/outcome lives. From there we observe the external WEIGHT. What policies, external dependencies, governance etc. impede us from releasing our work product into the real world and benefitting from feedback?

Having identified the top of mind factors weighing us down, we move to LIFT. What can we do to overcome or remove weight? Can we negotiate with an external approver to show our compliance as part of the process? Can we remove our dependence on other teams? Do we need to lobby for a policy update? There might be a lot of actions or only a few - this is a good time to think creatively.

Having discussed the autonomy axis it's time to move to mastery. Again we start with the impediments DRAG. Are we one deep in a critical skill or knowledge of the domain? Is crucial tooling missing? Is the codebase scary and dangerous to work in?

Finally we consider THRUST to overcome that drag. Would adding characterization tests help? How about spending a small amount of time doing some structural re-factoring to learn about the codebase while making it easier to reason about? Would making or buying some tool help? Could we use pairing or mobbing to overcome specialization?

One big advantage I've noticed over the last couple of weeks using this is that we end up talking about impediments and ways to overcome them that had not been identified as addressable. When we're in an environment for a long time, the constraints of the environment become normal to us and almost invisible. Firewall changes take 3 weeks - make sure to account for it in the plan. No! Let's figure out how to make that a non-event.

Anyway, I have found this approach useful and I'm writing this post because people I have shown this to are starting to use it and then tell me they can't find any information about Shamrock Mapping - now maybe they can :-)

Thanks for reading and hit me on twitter @mikeonitstuff with any feedback

Metrics that matter - or might at least

I've been thinking a lot about how to help align management behaviors to support organizational Agility. I think - still trying this in the real world - that our best chance might be to use the existing structures of incentives with some tweaks to the content.

We have Job descriptions, success factors, and personnel reviews at our disposal. I might think that those things are impediments to becoming our best selves and yet they prove super durable, so I've been working on how to use them to create a more supportive environment.

I'll spare you my apple seed planted on a table top analogy...

For this post I want to talk about organization level metrics that might lead to a better environment and appear hard to game via bad behavior.

1. How many layers of communication are between the user and the team implementing a solution for that user? The ideal number is zero as shown below, each additional communication point reduces the fidelity of the message.



2. Average number of projects each person in the organization is working on. One is ideal. When we assign people to multiple projects we essentially abdicate our responsibility to prioritize the goals. It's ineffective, inefficient, and unfair to team members.

3. Number of projects in flight organization wide. Just like teams and individuals, organizations benefit from limiting work in progress. When we focus on a small number of items we can increase the flow rate - projects get done - and maximize return while having less of our budget at risk.

4. Number of production deployments over time. The point of tracking this against management is to encourage smaller more independent releases reducing execution risk and realizing value earlier.

5. Equipment delivery lead time. Almost everything we do requires some sort of equipment. It could be a laptop for a developer, a VM in a private cloud, or the ability to drop a container into a cloud service. Whatever it is, wait is waste. We want to encourage reducing that wait time through the most appropriate means available. It might be automation, negotiation with vendors, or moves from owning to renting.

 

6. Percentage of capital budget allocated. In my heart of hearts I am a cash accounting kind of guy AND the finance folks don't care about my heart of hearts so we are stuck with capital budgets. The more money we have tied up in work in progress the more risk we carry. Every week a project goes without a production release accumulates risk; every production release resolves that risk. We might not like what we have but we know what it is and how much we've spent. We want to encourage frequent risk resolution events (production releases).

7. Percentage of time set aside for learning and improving current work. People need time set aside to learn and to improve their own work processes. We want to make that explicit as part of any transformation.

8. Hours of manual work. By manual work, we mean repetitive manual tasks. This is the kind of work that can eat a lot of time and also drains people of their creativity and energy. Eliminate it, automate it, just don't ignore it.

9. Number of inter-team dependencies. This is a count of the teams you must coordinate with before deploying your code to production. In a perfect world you would have total freedom to deploy non-breaking changes whenever they’re ready.

10. Number of after hours calls and emails. People need time to refresh and sending after hours work emails sends the wrong message. If we need 24/7 coverage, we must staff for it.

I am hopeful that giving transformation managers something to measure that makes the environment more supportive of agility will be a useful and effective lever in the work we do. Please share your thoughts with me on twitter at @mikeonitstuff

PLEASE NOTE - These graphics are from a stock service for the most part and the one about deployments is problematic. If you created any of them, let me know and I will happily credit.

Thanks for reading -- Mike

Hierarchy of Software Development needs

In reading Dave Snowden's article on a sense of direction, I was reminded of my experience several years ago working as a Patient Care Tech at UAB. I had started volunteering there and then took a part time job to gain some clinical exposure and decide if I wanted to make a career change into nursing - turned out I did not.

Anyway, I worked in the Emergency dept. which was a level one trauma unit. A trauma call is a complex thing. Injuries are often hidden and severity can vary wildly even with patients that present with similar injuries and histories.

There are some basics of course. Airway, Breathing, Circulation - if you don't get those right it does not matter how well you set the bones or if you stitched up the cuts in a way the won't leave a scar.

There are some corollaries in the world of Agile. They could be Build, Test, Deploy - if you can't build, without breaking stuff, and deploy it for use - the color selection and button placement don't matter. Like, at all.

After a patient is breathing reasonably and holding their blood in, a team can worry about things like blood chemistry, organ and brain injuries, broken bones and abrasions.

Similarly, when a team can routinely and safely deliver software for production use, they can start worrying about what features to deliver, in what order and how to manage that work.

Scaling the management framework might be kinda like taking up a new hobby after a full recovery.

My guideposts

When I talk about Organizational Transformations or Agile Transformations, I am talking about the engagements as I usually see them. Some company - often a market share leader in their industry - wants to be more like what they imagine Google, or Facebook to be. They talk to their consulting supplier of choice or maybe several and decide on a framework to become Agile and awesome.

By the time I come along, there's usually a project plan - sometimes disguised as a backlog - that includes milestones and payment triggers etc. There will be planning meetings and a sorting of the workers to become agile teams of one type or another, decisions about whether a particular function should use scrum or kanban, whether we should use Jira or Rally, how many columns to represent todo, doing, done etc.

What's often missing is the universal understanding of why we're doing this and what we hope to accomplish. This often leads to dark scrum and a general sense that the people in charge and the consultants they've hired are full of shit.

I assume the point of these engagements is to become capable, high performing, learning organizations that delight customers and take all the market share from their lesser competitors. The goal of the exercise is not to become good at performing the ceremonies and practices of any framework or method - methods and practices are probably necessary, but are not sufficient.

Here are some things that I think are important:

• Increase capacity/capture value everyday - any day might be the last day for the project.
• We should never be one-deep in a skill or position - people should be able to go on vacation or take a promotion without leaving a gaping hole.
• We must be able to make a simple software change to production in one day - shorter is better.
• Teams must be capable of independent action - if three teams are required to make a change to your three tier app, you gotta fix that.
• Teams have discretion and authority for their work - keep friction and approval processes down
• No out-of-band reporting - the natural by-products of the activity must tell the story. 
• You must always know what your apps are doing - I never want to learn from a customer call or twitter that our website acting weird - same for internal apps.
• Every component or activity must fight for it's life - extra is extra if it's dumb get rid of it.
• We should always minimize complexity where we can - Keep your stack short.
• No resume driven development - we'll make time for learning and playing.
• Keep up with the tech - play with the cool stuff enough to know if/when to adopt it.
• Working here should make you more marketable - learn and grow, I'll help.
• Behave - nobody wants to work with a jerk and they don't have to.
• It's not OK to suck - if what we do sucks, we have to fix that.

There are other things of course but these are the big ones for me.

Thanks for reading - Mike

More thoughts on Organizational Transformation

My post yesterday generated some good criticism and discussion on Twitter. One of the common misunderstandings was the idea that I advocate the cascading of explicit goals down the organization.

This is exactly what I was trying to avoid. I am questioning now my proficiency in the language...

Anyway, when I said goals at each level inform the goals at the next next level, I created confusion. I'll try to clear things up a bit with some examples below.

I might have a goal (desired outcome) of reducing the company's travel spend by 50%. If I think I have all the answers, I could ban all travel for a period of time or create an onerous approval process. Both options that reveal my lack of regard for others in the organization and my arrogance.

Alternatively I could broadcast the goal - we need to spend half as much on travel - and some visibility into current spend vs. the baseline. After that I would leave it to those spending the money to figure out how to reach the target. They might prioritize travel, book flights earlier, use AirBnB instead of the W - lot's of options for most companies to spend less especially in the short term.

I have a goal that I want a simple software change to go from identified by the business to safely in production in half the time it takes today. I could specify Scrum, require unit test coverage, and demand a CI/CD pipeline - or - I could broadcast the goal and let the people closest to the work decide how best to reach it.

There might still be problems with this approach and it may be useful only in an ordered system - but I want to at least dispel the idea that I advocate passing detailed requirements or squishy objectives down through the layers of an organization.

What I want to do is increase the amount of information about organizational direction while also increasing self-organization and autonomy.

Goals or vector signalling will be vague and often frustrating for the people doing the work - that lack of detail coupled with a measurable outcome leaves room for creativity and discovery.

Thanks for reading - Mike

Thoughts on Organizational Transformation

My current thoughts on Organizational Transformation wherein I propose a simple repeating pattern of self-organized, autonomous, independent teams at every level of the enterprise under change.
 
We’ll start with some definitions to aid in subsequent discussion:

• Team - any group of people working together toward a goal or set of goals
• Goal - a desired, measurable outcome resulting from the work of a team or teams
• Product - a tangible artifact produced by a team to be used by consumers
• Service - a process carried out by a team at the request of a consumer
• Consumer - a person or team that uses products and/or services
• Outcome - Movement in a business metric or situation

Before starting any transformation it’s very important to understand and articulate the desired outcomes of the effort.

Because so many of these engagements are IT driven I’ll use a IT example:

• Enterprise X wants to compete and win in our industry.
• IT strives to eliminate friction for product teams using IT so they can quickly, safely and economically design, develop and support business capabilities through changes to existing systems and creation of new systems.
• Marketing will improve on our our ability to...
• HR must make it easy for...

With a clear goal in mind we can bring together the team to determine:

• What services and capabilities to provide?
• What activities to continue or start?
• What activities to stop doing?
• Who are our customers?
• How will we measure the outcome?

The services and capabilities listed at each level in the organization are usually decomposed at the next level down until they are actually implemented.

This provides alignment from the top level transformation goal, through the intermediate goals of each department all the way to the individual contributor executing a task.

Team members at one level are likely to support a team at the next level. So a VP on the CTO’s team at the IT department level has a team of directors and managers that decompose an IT function such as security or cloud infrastructure into smaller pieces for their teams to implement.

Each team, regardless of level, is self organizing, autonomous, and as independent as possible. They are also intensely customer focused whether their customer is another team using their product, a business function requesting new or changed software, or a Director trying to understand and align with executive guidance.

Thanks for reading - Mike

Give your stories an expiration date

My wife will tell you that I am a hoarder. There's room between my fence and the forest behind my house where I 'store' things we might want to have later.

I say that to let you know - I get why people don't want to throw good stuff away.

Still, I think there's a really good case for adding a Time To Live value to our backlog items. The problem with a backlog - especially those in Jira and similar products is that it seems easy to keep a large number of items. It seems practically free or at least very cheap.

There are these hidden costs though.

Management overhead: When I have a lot of stories I feel compelled to look through them all to make sure I'm not missing something important that should be moved up in priority. If there is high priority work lurking in the depths of my backlog, I must not being paying attention.

Undue Attatchment: I really love some stories. I had the idea, wrote it down, maybe thought up some acceptance criteria and then - nobody cared. I keep the story around because it just seems like a great idea to me. Not only that but I put work into it and don't want to lose that work. Here's the thing, if it's really a good idea it will come back, maybe more than once, in a better form that somebody besides me cares about.

Distraction: Sometimes teams feel obligated to get through the list and will take on stories, not because they seem useful but because they are the right size to fill out a sprint. There's almost undoubtably a better use of that capacity.

I prefer a small backlog - maybe 2-3 iterations worth - of stories that I and the team feel urgency about. Iteration planning should be about the highest priority work not about filling capacity.

If your story goes more than a couple of iterations without making the cut - kill it. The truly good ideas will percolate and come back better when the time is right.

Thanks for reading - Mike 

Refactoring the Enterprise

Why Refactoring – Is this yet another Agile Method

First, this is not a call for another Agile Method.  I won’t try to write a book and repackage a bunch of stuff you already know and then offer to certify you.  Nothing wrong with that but not my thing.  Second, I am using Refactoring because I have been recently re-inspired by doing refactoring katas thanks to Llewellyn Falco and Woody Zuill and reading some great books The Nature of Software Development by Ron Jeffries and Building Microservices to name two. What I describe below is similar in some ways to refactoring legacy code.

I have been talking about this idea of carving out a bit of business capability that a single team can deliver and run without the need to coordinate with other teams. The fancy name for this is Domain Driven design with Bounded Context.

Most businesses these days, regardless of the product or service they deliver, have become software development companies. Many of these companies are running a fragile, organically grown, or acquired, collection of code and infrastructure that is difficult and risky to change. At the same time, market demands, competition, and a changing technology landscape make business agility more important than ever. 

There are already some enterprise frameworks for agile but they feel heavy to me and they seem to put the focus on detailed upfront planning and ongoing processes which seems inconsistent with the Agile Manifesto.

I think a refactoring approach can be used to transform organizations in an iterative, emergent way that limits risk and maximizes long term ROI.  Here’s what that might look like.

Meet with executives at any level in the organization to understand their strategic objectives and known barriers to achieving them.

·      A strategic objective might be to provide device specific, yet consistent access to product information.

·      A barrier to achieving that might be that the business rules are embedded in the webpage code and backend systems without a clear delineation. Creating a mobile app that provides the same selection and pricing as the existing website could be very expensive and require a huge amount of testing with a long time line.

Enterprise and Solution Architects are in the best position to identify challenges and opportunities in the IT space that align to those objectives and remove barriers. The architects will also have a key role in integrating new services into the existing architecture.

Because we must be able to deploy at any time – even multiple times per day - organizational leaders must clear a path for a parallel delivery pipeline that allows on demand deployment to production environments. Versioning and feature flags allow partial feature upgrades and multiple versions of software to coexist in the production environment.  Creating a parallel delivery path allows the lights to stay on while we iterate to the fully workable solution.

One way to start the journey would be to select a specific business capability that needs to change, or scale, but is difficult or risky in the current process. After creating the bounded context; design a product that provides that business capability to customers – who may be internal or external. 

For our example above we might create a service that encapsulates the business rules and the product data to determine which products should be offered to a customer. The service could be called via a restful interface that includes specific selection criteria.  The service would return a list of products (prices might be included or might be provided by a different service.) The development team for each device platform could decide how best to display the offerings.

To get started on the new service we would form a product team (Product Owner, Developers) that will develop and support the product starting with the smallest useful feature set, MVP, in parallel with the current delivery scheme. The Developers are responsible for testing and supporting their product – even in production.  The Product Owner is responsible for maximizing value and being available to the team at all times – What Woody calls Decision Queue Time – can easily multiply the time to value for a development effort.

To ensure the business agility we promise, it’s not enough to just write the software. The product team must be able to deliver running tested features into production on their first iteration.  This means they need to have version control, build automation, acceptance test automation, non-functional test automation, and automated deployment from the very beginning. Assign a Coach to help with these things and the ongoing challenges they will face.  The first running tested feature may be as simple as “Hello World” but it will be in version control, have been automatically built, passed its acceptance tests, passed non-functional requirements testing, been automatically deployed into the production environment, and validated as available and responsive. This is the proof that all the parts work together and it’s a great way for the team to experience the definition of done.

It is important to make clear the expectation of excellent technical practices.  Release to production every iteration using automated tests and automated build/deploy.  Version APIs and use feature flags as required to enable trunk based development.  Design and build in monitoring, high availability, recovery, graceful degradation, and scaling from the beginning.

Also, every developer must be familiar with and responsible for the quality of every piece of code. This is a key risk mitigation strategy; nobody wants to worry about crucial code that only one person can work on or complex code that fights modification.

This approach provides learning early and often but still requires patience. The first effort will be slow – people will need training and experience.  There will be missteps and disappointment along the way.  Understand and plan for this – you have already reduced risk by undertaking this as a parallel effort to whatever process is already in place and by automating tests and deployment.

We started off talking about an enterprise transformation but we’ve only talked about a single team so far. In the spirit of the agile manifesto we want to iterate and learn. When the first effort starts to show success, add another product and team.  Use this pattern of adoption until you run out of products and markets that need attention.  Your organizational structure will change over time to reflect the new architecture.  Many teams will be able to support more than one related product – that will emerge as you go.

Changing the focus from the activities of Agile/DevOps adoption to the achievement of strategic business objectives through refactoring allows us to start learning faster and to stop at any time with minimal waste. Moving to a product focus allows us to leverage Conway’s law in reverse to transform the organizational structure around the business outcomes we are trying to create.

I am interested in your feedback especially if you have specific examples and ideas on why this could never work in the complex domains of the real world.

Thanks for reading --Mike

Strong Pairing, Mobbing, and Woody's Workshop

First - some background.

I don't program computers for a living and haven't in a long time.  I am interested in programming techniques, languages, and approaches though; I could code my way out of paper bag if my life depended on it.

For a long time I have been a proponent of XP practices like pair programming, TDD, small stories, frequent delivery, etc. 

At my last company I had started to sit with developers while they worked and talk with them about some of the issues they faced.  General problem solving, simplifying, and helping to focus on one thing at a time.  This would be very weak pairing as I would be conversant but not fluent in the tools.

At the same company, two of the teams had started to do a lot of group sessions where they would work through problems and solutions together in an informal way that allowed people to join and leave as they needed to while the group continued to make progress.  We didn't have a name for it but it seemed to work well.  This had grown out of our hiring process where all available members would participate in interviews and technical challenges that required collaboration, resourcefulness, and the courage to ask questions and sometimes say "I don't know."

A few months ago on the AgileMentoring.com Slack board, somebody mentioned Mob Programming which I had not heard of.  I was very interested though based on the previous good experience with group sessions, and intrigued by some of the details such as frequent driver changes.

I watched the video, bought the Mob Programming book and reached out to Woody on Twitter.  We had a phone call one Sunday afternoon and I decided I needed to learn as much as possible about this way of working.

Cut to #Agile2016 and an available Mob Programming workshop in Atlanta after the conference. While walking around and talking to a large number of interesting people, I noticed there was a Mob in session working on a Refactoring Kata for a simple Tennis Scoring program.

I walked over and sat down to observe.  In the space of less than 10 minutes, several approaches to refactoring the ugly If block were coded.  All of them worked but left one or another team member dissatisfied because they were clever, or confusing, or ugly, finally - still in the 10 minutes or so - somebody suggested a clean, simple, and elegant solution.  The tests passed and it was over.

Llewellyn Falco mentioned that they would be doing another Kata as a Mob the next morning.  Well the next morning the projector had disappeared and there had been a party the previous night so only Llewellyn and I showed up.  He offered to do a Strong Style Pairing session with me.  Strong Pairing is where any coding idea must go through somebody else's hand on the keyboard.  I was a bit nervous, we would be coding in C# - a language I have not used or even looked at - and with just the two of us there would be no cover for my ignorance.  It turned out that Llewellyn was very patient, we came to a nice refactoring solution in a about 40 minutes with some good conversation and advice thrown in.

Shortly after that, I realized I was a train ride and car drive away from Woody's Mob Programming workshop.  I arrived a few minutes late and Woody was still covering stuff that's in his book - whew!

Woody walked 4 volunteers through the rotation process while the rest of us tried not to interject and learned to feel what that is like.

In no time, we broke into groups of 4-5 - each with our own computer setup - and strict directions that only the current navigator could suggest what to code and only the current driver could type stuff in.  

This was hard.  As the navigator, I really wanted to consult with the team.  As an observer, I really wanted to help the navigator.  As the driver - all my recently learned keyboard shortcuts were useless because we were using a different IDE from what Llewellyn and I had used.

After satisfying a couple of requirements, we were allowed to call for a consult at any time and talk as group but with hands off the keyboard.  This was a lot better, but we found it easy to spend too much time evaluating different ideas without putting them to code.  A couple of times when we wanted to refactor because the code was getting ugly, we couldn't agree on an approach right away so we just made the test go green and put refactoring on the back burner.  I did mention that we were doing TDD - right?

A quick word about TDD.  With the group it was easy to write the test for each new requirement first. We quickly grew comfortable with this approach and it made refactoring attempts really easy and stress free.  One time we made too many changes and turned about half our tests red.  Woody had us revert until all the tests were green and try again.  His view - which I now share - is that you should only start a refactoring when all your tests are green, and that you should never be more than a couple of edits between passing all tests.

Finally, as we continued to knock out requirements more and more quickly, and Woody - our product owner - was busy with other teams, we started to use the gaps to review our approach and identify refactoring opportunities that would make us happier with our code.  Better names, cleaner tests, etc.

The last stage as a team was to be in full Mob mode - anybody could make a suggestion at any time while remaining Kind, Considerate, and Respectful.  This was a lot of fun and we already felt like a team after something like 1 hour of working together.

When we started only one person on the team was an actual C# programmer.  This was scary but turned out to not matter. Almost any problem is going to require only a small set of language features and you learn the ones you need pretty fast.  A nice side effect is that everybody learns them.

I also thought that the frequent Driver Rotation would be disruptive - we were using 4 minutes.  It was not a problem and seemed to have the effect of time-boxing our work.  We would think about how we could go green in the space of a rotation - whether it was a new requirement or a refactoring.

I want to do a lot more of this and find a really good way to Mob Remotely.  I highly recommend that you get Woody to do a Workshop for your teams and give this a solid try.  The concepts are simple but I think your chances of success and adoption go up with the workshop.

The Mouse is dead - long live something else

My wife came home with a new issue the other day.  Kindergartners will take a pass on computer time rather than learn how to use a mouse.  You might say - they're five, what are you talking about and who cares anyway.

Well the thing is five year olds have, for the past several years, loved their computer time.  So what's different this year?  Tablets.  Well Tablets and the Smart Board which is like a wall sized tablet.  The kids are completely unimpressed with with the abstraction of movement and selection required to use a mouse.

So in the immediate future I think it's going to be touch, the guys at leap like hand waving and that might catch on, or it might have the same problem as the mouse for these kids - not direct enough.

I can see a near future where something like Google Glass is all I need.  I'll tell Putty what window to open and the *nix aware voice recognition will take all the vowels out to make good command lines.

Data Sharing Among Applications

Too Much Complexity - It's a big Pain In My Ass

I have this theory that most of the shit we do in IT is more complex than it needs to be  There are entire industries built around creating new vocabularies for old concepts and packaging fundamental data processing concepts into new buzzwords - almost always making simple things less simple.

This is not new, it's been common practice for at least 15 years.  I think prior to about 15-20 years ago there were too few IT pros to get away with it - we would call BS on your "new idea" if it was already commonly understood or too obvious.

Recently, I've been thinking of how to de-couple applications that have grown up organically as close siblings.  You've seen this where each application has to have its own copy of the data from other apps.  We have an institutionalized scheme for noticing changes and pushing them to all interested parties.  Or where one application is doing complex multi-table queries against the physical tables in another application's database.  Or even worse where my app consumes some of your data and changes the status of the records directly in your DB.

Many people don't seem to think this is a problem or even particularly ugly.  I might just be a whiny old guy to think something more elegant would be better - maybe so but I'm gonna whine.

Problems with close sibling applications

You're too much in my business.  I mean where's the privacy?  Where's the respect?  And now I have to get your permission to re-arrange my furniture?  I feel constrained.  I feel put upon.  We need to re-evaluate our relationship.

All this everybody's in everybody else's business mean it's really hard to make simple changes.  I might break your app.  You might break mine.  Who uses what?  What apps are at risk if I make this change?  It's all very scary and leads to paralysis, delays, and really frustrated business stakeholders that just want to get shit done.

Popular Solution Approaches

Generally you see a few approaches.  One is to decide that the entire enterprise is going to implement some sort of Enterprise Service/Data bus and become SOA everywhere.  Just a few million bucks on consulting, hardware, software, projects to create web services for all our apps, and we'll be good to go.  Three years - tops.

Another approach is to undertake a CMDB project.  We'll document all our dependencies, design a data store to put everything in and then we'll be able to see exactly what changes will affect what processes and applications.  Again it'll only cost a few million bucks and we should be able to do this in no time - two or three years.  Let's ignore for a moment that we have a poor understanding of our dependencies to somehow overcome.

For both  of these you also have to ignore that the business still wants to get shit done and they compete for the resources you're using on these "nice to have" IT initiatives.  They compete using business cases with revenue, presented by professional sales people.

In the old days we had the idea that there should be an enterprise database - a single, shared, easily accessible database that everybody would use.  We would define every data element in the data dictionary and if your app needed to have last names, you would go to the data dictionary to find out that last names are always varchar(25) or whatever.  Of course this was a complete failure, with data dictionaries having at least as many definitions for a standard last name as there were existing applications that use last name.  The idea is still attractive - but it just does not work in the real world.

My opinion is that these approaches are too complex to undertake as a project and are doomed to failure - I would love to hear some success stories though.

What we can start doing now

I'm also a big fan of Fundamentals - setting a pick, getting down on the ball, looking where you want to go - before advanced techniques like driving to the basket, turning a double play, and getting a knee down.

So I suggest starting with something simple.  One or two apps at a time, find out what data they share and how. 

If they're doing queries directly against your tables - move that to a view that also does whatever joins are required to give them the data set they need.  Think about things like - my app looks for a record in one table and if it's not found, queries a history table for the same info - could you create a view that allows me to look just once?

What if I need to change values in your data?  First explore whether that's really needed, if it is, create a stored procedure that you control and I just call.  Then you are free to do whatever you need in your app as long as my view and update procedure stay the same - you don't even need to talk to me about it...

After this first step of creating Views and Procedures we can start thinking about creating endpoints.  Web Services, an intermediate Database server that Federates access without replicating data (use a data warehouse to Federate big data), or whatever other endpoint you can think of.

Views?  That's so 1990 - we are SOA all the way XML, HTML...

Views are an enabler that support whatever you want to do next while providing immediate benefits - not the least of which is figuring out what data you actually share.

Views might be 1990 but just about everybody can understand a select statement.  And think about the benefits to testing.  Rather than needing a full blown environment with all the services up and running.  Or mocking all those services.  I can have test data in a Spreadsheet or Access or MySQL or whatever I want and simply point the app to the datasource.  I can also peruse the data that my app sees without having to know all about the schema in the source DB.

Conclusion

It's our job to resist gratuitous complexity.  Go to the fundamentals that enable whatever is next and only add moving pieces that can carry their weight and more.

Thanks for reading - Mike

Turnaround Toolbox - Set Scope and Accountability for the team

Having explored what everybody thinks we should do - Expectations - and getting control of how folks engage us - Inputs - it's time to set the scope for the team.  

Scope includes all the tasks we do, products we deliver, and also what we're accountable for.

Tasks and Deliverables:  

For each task your team does and deliverable you produce, you need to know the Who, What, Where, When, Why, and How.  

Who on the team does the work?  If the task is one-deep, who would provide secondary, and tertiary coverage?  A lot of times you find these little one off items that one person knows about and knows how to do.  If it's going to stay in your box, you must be able to support it through vacations, resignations, etc.  Counting on a single person is an unacceptable risk.

What is the task?  Is it an operational procedure, document creation, external communication?  Categorizing the What of items makes it easier to organize them for documentation and assignment.

Where does the activity occur?  What you want to understand here is whether it's location specific.  Manning a teller window in a bank is location specific.  Creating a spreadsheet from a report can probably be done anywhere.

When and with what frequency does the activity happen - or need to happen?  You might find that you have tasks at a frequency that you're not staffed for.  For instance 24/7 operational monitoring without 24/7 staffing.  It might still stay in your scope, but with a different How component.     

Why do you do it?  Everything must be supported by a compelling reason - everything you do comes at a price.  Arriving at the cost can be difficult when nobody has to write a discrete check for the item, but it's important to get some idea.  Is it a good value?  Does it mitigate a risk?  Does it meet a contractual or regulatory requirement?

How is it done?  If we don't have documentation we'll need it - either to keep the task, or to transition it to a more appropriate home.   

A few words about accountability.  If it's in your box, you are accountable for it.  Given that, you want to make sure that you have control over everything in your box.  This means you must push out stuff where you're just the middle man.  Being the middle man is fine if you add some value and you limit your accountability to your performance as the middle man.  

The most important thing is that you do not want to end up accountable for things you cannot control.  So if your job is to predict or report on the sunrise - that's fine - if somebody expects you to control the sunrise - well that's a recipe for disappointment.

So, figure out what your team should do, make sure you can perform, and limit your accountability to things that you can control.

Thanks for reading - Mike

Turnaround Toolbox - Define, Limit, and Enforce your inputs

In the last post I talked about Expectations.  Today I want to talk about getting control of the inputs.  It's really hard to provide good customer service if you take requests via multiple, uncontrolled channels.  Think about trying to grocery shop to a budget with random people throwing stuff in your buggy at random times - you could probably keep track of most of it, but that ipod touch from the electronics department would be a painful miss when you got to the register.

We've all had experience with a lot of input channels - email, IM, telephone, one or more ticketing system, scheduled tasks, etc.  Any or even all of these together can work if you have a process in place to manage and track them.  Some organizations even take tweets now.  The key is getting control so that nothing falls through the cracks.

Define your inputs:

If you take email does it go to individual in-boxes, an organizational in-box, or a distribution list?  Who is responsible for keeping track of the requests?  How are items assigned?  Do responses come from individuals or from an organizational sending address?  What happens when somebody goes on vacation?  I am not a fan of taking requests via email because there are just so many cracks for things to fall into.  I get hundreds of emails a day and the signal to noise ratio is not very good.  But it might be a different story for you so if email works use it; you still must define how you are going to handle email requests.

Instant messaging  would seem to be an even harder nut to crack than email, in my experience IM provides a natural indicator for immediate requests - if I don't respond to your IM you can be pretty sure I am not working on your issue.  IMs don't work as well for problems that require more time or coordination.  Much like a phone call, once we quit talking, I am on to something else and your issue is a fast fading memory.

Phone calls - see IMs.  Good for immediate stuff but once we hang up it is over.

I like some sort of ticketing system something like Fogbugz that is easy to use like email but keeps track of ownership, history and related artifacts is nice.  Salesforce works as well as a lot of others.  It does not matter so much which system you use as long as you use one.  Ticketing systems give the issue a unique handle that people can refer back to, use as a license to work, and do reporting on.  Try to avoid multiple disconnected systems though.

Limit your inputs:


Only take requests via inputs you have defined and can manage.  This should be obvious but I can tell you from experience that it is not.  People that want stuff from you will use any path they can, but they will crucify you when their request falls into the Grand Canyon sized crack in the "interupt an ongoing hallway conversation to mention this little item" method of request input.  If you have a process for managing requests via Tweets, Email, Tickets, and Phone Calls then go ahead and takes requests through those channels - just don't take requests through Facebook Wall Posts until you figure out how to manage them to completion and get that defined for your customers and staff.

Enforce your inputs:


This is both the most difficult and most important, especially if the team has historically taken requests however they come in.  When you first start enforcing your inputs with customers it can seem like they're getting the run-around.  You want to avoid this by gradually moving from taking the requests as you always have to only accepting requests through the defined channels.  It's a journey.

At first I will start with some canned language that tells the requester how to get the best results along with a personal note that I have opened the request for them and here's the number for tracking purposes.  Say I get a request from a Sales Rep asking why Customer x was charged for Product y.

I'll send back something like:

Mary,

I have opened request xxxxx for you.  You can track it's status here [link to the ticket].  In the future please open a request by going to [some URL] and filling out the request form.  This will allow us to assign your request to the best resource and track it to completion.

Thanks,

On the next two or three requests from Mary I would send:

Mary,

For best results please open a request by going to [some URL] and filling out the request form.  This will allow us to assign your request to the best resource and track it to completion.

Thanks,

Finally, I would move to:

Mary,

What's the ticket number?

Thanks,

This is not really a science and I might also have a lot of phone calls and chat sessions extolling the virtues of the defined system.  It does work over time.

That's one half of the battle - the other half is getting your staff on board.  It going to feel like they are being unhelpful and wasting time because it's easier, quicker, and all around better to just dispatch the request then and there.

They're right of course for the ones they know about - the problem is the ones they don't notice and come back to you two weeks later as a "your whole team sucks" escalation.

I expend a lot of effort in the beginning trying to see what people are working on and how it came to them.  When I see somebody working on a proper request, I praise them.  When I see somebody working from a non-sanctioned request, I reiterate the need for using the defined inputs, and make them put the request in process.  It's a huge pain in the ass for both of us, but I know it will pay off quicker than expected.

A few words about ticketing systems.  Once you have a request in process it's fine to do some out-of-band communication to move the request along.  IM the requester for a clarification, pick up the phone, host a conference call.  Whatever works to help get the job done.  What you should definitely not do or tolerate is the deal where your customer puts in a request and then your team updates the request with something like "Need more information" and does not reach out to the requester.

Most people do not live in the ticketing system and will not continue to check status.  They will check on the day of the deadline and be pissed off that the work is not done - leading to a well deserved "your whole team sucks" escalation.

Defining, Limiting, and Enforcing your inputs will go a long way toward helping you meet the organizations expectations for your team.  It takes work, time, and discipline but the payoff is a smoother flow of work and many fewer complaints.

Thanks for Reading - Mike