Friday, January 27, 2023

Software quality vs rapid growth: are they incompatible goals?

Note: I'm specifically talking about software development organizations, since that's where my experience is.
 
With that out of the way, the short answer is "yes".  Once a company grows beyond a certain size (somewhere between 300 and 1000 people), it simply can't sustain shared values across all of the teams.  Here's why:
 

quality

"Software quality" describes how easy software is to understand, change, and run.  High quality software, in my experience, is only created when teams have:

  • High safety - team members feel it is safe to raise issues, acknowledge failures, and admit when they don't know how to do something.
  • Comprehensive automated tests - the software is safe to change because an automated test suite exists and is run every time a change is made.
  • Sound infrastructure - a CI/CD pipeline and modern code versioning tools ensure that it is easy to see what changed when, and build + test + deployment is simple to execute whenever a change is promoted to production.
  • Good practices - to me, this is usually XP practices.  Things like teamwork (pairing / mobbing or just good code reviews) and test practices (TDD and a balanced test pyramid) are what create the top 50% of code quality.  Everything else really just supports the practices.

There is more to this such as good design and data engineering practices, as well as good agile backlog management and product development.  The core of my experience is in software development, though, so I see things through that lens.

growth

When I refer to "growth", I'm talking about growing a business.  [Edit: more specifically, a consulting business, where "growth" = "more people", since people are the source of revenue.] 

I have spent a lot of time in consulting over the past 15 years, and most consulting businesses aim to grow by 50% or more whenever they can.  I have worked in one location where the software engineering team (and the revenue) doubled in size of the course of a year.  (And it wasn't a small team to start with.)

Growth is good, right?

Well, not always.

the conflict

Very high growth comes at a high cost.  When you increase the size of your team by more than around 25% in a year, it becomes very hard to sustain good practices.  There simply isn't enough time to show the new hires "how we do things here", so they fall back on the way they did things before you hired them.  If you are very stringent about hiring (i.e., only people who already profess to follow the same practices), this can still work, but such people are pretty rare, so it's an unlikely scenario.

Instead, what usually happens is that there is confusion and some level of conflict on how things should be done.  Particularly on teams with more new hires than old hands.

A focus on growth also means that there is considerable pressure to take on work that isn't well-suited to a high-performance team.  If a client doesn't want to update infrastructure and insists that your team works with their non-agile process, the response is often "let's start that way, and we'll reshape the project over time to be better suited to our team."  

But that last part doesn't happen.  There just isn't the time or energy to fight the battles you'd need to fight to make that a reality.  Not when the team is busy trying to create a working product in a sub-optimal environment.

a middle ground

Here's a possible answer: Grow at a sustainable pace.  

Target 10%, with 15% as a stretch.  Say "no" to work that is a poor fit.  

Build a reputation and market the hell out of it.  

If demand gets really high, don't just raise your rates across the board, or you'll filter out most of the work that your people are best suited to do.  Tell your clients that they need to change in order to get value out of what you do, and offer to help them change.

why we don't see this in practice

In large companies, the people setting strategic goals are too disconnected from the day-to-day operations to realize that they are creating problems (and attrition) from the people delivering the product.  They are also humans, and as a human, I know that I'm really good at fooling myself.  

It's easy to tell yourself that you need to grow whenever you can in order to pay bonuses and keep people employed.  The truth is that when you grow too fast, you'll have to lay people off later because of economic boom & bust cycles.  

 My % growth numbers may be a bit naive.  Maybe they should be twice that.  But one thing I am sure of: Growth targets in big-box consulting are far too high to be sustainable for everyone except the executives.