Why agile software engineering transformations fail...and sometimes succeed

All of my fellow Agile and Software Craft practitioners have experienced failure.  It happens far more often than success, and usually for obvious reasons.  Obvious or not, there are some clear precursors to a successful transformation.

To support high-performing teams, you MUST have the following pillars:

• Solid DevOps infrastructure and practices
• Skill in key software engineering practices
• Teams who are engaged, motivated, and empowered
• Manageable (as in, not-too-much) technical debt in your codebase

If you don't have these things, you need to create them before pushing too hard to be "agile", and certainly before measuring success or failure of your efforts.

Fortunately, there are clear paths to creating all of these.  Unfortunately, there are very few organizational cultures that have the ability to follow these paths.

DevOps Infrastructure and Practices

This is the easiest one to see and remedy (though as with all of these, it does require effort).  If you don't have a cloud-based DevOps infrastructure, there are plenty of experts in the marketplace who can create one for you.  If your teams don't follow good practices (code checkin discipline, a continuous integration (CI) pipeline based on infrastructure-as-code, clear PR rules and automated deployment), these things are all trainable.  They are also the easiest way to improve the developer experience, which has major upsides around retention and productivity.

In other words, if you're lacking in this area, consider tackling your DevOps first. Or at least in parallel with any other pillars.

Software Craft Practices

The "Software Craftsmanship" movement of the previous decade spelled all of this out, and it is still true today.  It is usually what engineers think of when they say "Agile".  Most software engineers who learn these practices wouldn't want to work without them.  Unless they are missing the other pillars.

Cross-Functional, Self-Organizing Teams

Another way of saying this is, "empowered teams."

This is the easiest pillar to create, but many organizations struggle the most with it.  It requires that you drop command-and-control-based management practices, and instead empower teams to do their best work.  Leaders need to actively seek out areas to trust their people.  Leaders also need to raise their tolerance for mistakes and setbacks, and instead appreciate how their teams learn from these things, rather than punishing them.

Trust is difficult in many organizations.  It is a daunting challenge to find a balance between giving people room to learn, and risking damage to the organization's reputation and assets.  Nonetheless, you simply cannot have adaptable, high-performance teams without it.  

However, it can be done, even in less risk-tolerant industries like finance.

A Flexible-Enough Codebase

Another way of saying this is, "Don't carry too much technical debt."  If you aren't familiar with the term, "technical debt" is a way of describing how difficult code is to change.  From a code perspective, it means long, tightly-coupled modules, opaque method and variable names, and a lot of duplication.

If you have too much tech debt, every change requires deep knowledge of the code, and this forces specialization.  If your team members must be hyper-specialized, then only Person X can work on Product Y - it just takes too much time to cross-train everyone unless they're constantly in the code.

Fortunately, there is a simple (but time-consuming) way around this.  "Legacy Rescue" techniques offer a path to clean up the parts of the code that are touched the most.  The payoff for doing this starts slowly, but ramps up quickly, and the end-result are faster production deployments with far fewer surprises.

What If We Have All Of These?

Odds are that you don't have all of them, but if so, congratulations!  You need only start practicing a flexible agile methodology, with the key cadence meetings, and your engineering teams will no longer be a bottleneck.

Of course, that's when you'll find out how agile your product development process is...

The Learning Organization: Improving on the code randori

For most of the past year, I have been pushing to make my workplace more of a learning organisation.  I'm doing this for purely selfish reasons - I simply enjoy working in a place filled with smart, motivated people.    A focus on learning and continuous improvement tends to attract people who enjoy that, and chase out people who don't.  (Senior management expresses it differently, but I think my reasons are more honest.)

A big part of this effort has been to run a regular "craft day".  "Craft days" are  in-house software conferences with a heavy emphasis on hands-on practice of programming skills.  (This has been disappointing for people who wanted to make beaded purses and wallets, or whittle figurines from a block of wood.  You just can't please everyone.)

It really is awesome that I can do this during working hours, unlike what Chad Fowler describes and which most of us have had to do - learn almost all of our new skills outside of work hours.

But...

The down-side to craft-days is that we can only do them every other month.
While it's great that we can do this sort of thing at all, it's only going to give us a limited return.  We need to do hands-on practice regularly to really take the lessons to heart.  It also servers to stoke the fires of geekiness in our pocket-protector-covered hearts, motivating us to stay on top of the technology curve.

It becomes much easier to engage people and figure out what they really need if you spend more time with them.  Bob Martin has a pretty good (and pretty honest) explanation of why occasional exposure to learning doesn't really help much here.

Accordingly, I volunteered to to a twice-weekly Code Randori sessions.  The first session wasn't very valuable, so with the help of the group (there were 25 or so people in attendance), we're going to try the following tweaks for the next one:

• A simpler problem (but still a real one that we encounter at work), with no code written for it.  (TDD the whole thing.)
• The problem visible to all even while code is being written. (Whiteboard)
• The skills we're focusing on visible to all.  (Whiteboard)

For anyone who is interested, I'll put the retro conclusions in the session notes.

Photo by some guy named zimpenfish on Flickr

Why are we having a "Post-Agile" debate?

I just read a post by Kurt Häusler to the Software Craftsmanship Google group this evening.

The background for this is a long-term debate about what exactly "Software Craftsmanship" means, and what it's major focus should be.  I think that Kurt did a great job spelling out two of the major points of view:

It seems a lot of people suffering from "agile-fatige" and looking for something post-agile are in two camps. One camp seems to be competent leaders working with poor developers. For them things like Scrum have bought some improvements in the organization and they despair that after all this "agility" the code is still crap. These people crave something like craftsmanship to light a fire amongst their developers. The other camp seems to be competent developers suffering under poor leadership. They see the XP practices and clean code as obvious to the point of being trivial, and wonder why people like Bob Martin are calling for more focus on code, when it is clear, to them, that future improvements lie in better management practices, better understanding of value streams and improved relations between customers, management and developers. These people are currently looking at things like lean and kanban, and finding a lot of valuable stuff there. 


My own perspective is that we don't really need a "post-agile" movement or focus.  Agile (with it's multiple meanings) is still not the norm in most organisations, large or small.  Getting organizations - particularly large organizations - to do more than pay lip service to agile methodologies should still be our main focus.


I wonder if the drive to come up with a "post-agile" methodology or movement is really being pushed so a new crop of developers can have their names associated with it.  Is it more of an exercise in branding than engineering or craftsmanship?
  
Photo by Lovefusion Photography