Agile and the Stories We Tell Ourselves

As coaches, when we teach Agile to companies, we normally start with Stuff To Do. Create Scrum teams. Build a backlog. Track your work-in-process on a Kanban board. Hold bi-weekly Scrum-of-Scrums. Get yourself some SAFe training to scale up. And so on and so on.

For a lot of companies, Agile stops there. They do the stuff, their metrics improve, and they feel pretty good about themselves because "Hey, we’re Agile now!" But are they? I’ve seen any number of projects use Scrum ceremonies in a non-Agile way. I’ve personally participated in daily stand-ups and retrospectives for a project that did all their scoping and high-level designs for the next two years up front. The classic Big Design Up Front mistake. I’ve seen attempted Agile transformations derailed when senior managers came into a planning event telling the teams “Here are the set of features you need to complete within the next PI.”

I have seen over and over again corporations violating the heart of Agile while pretending to adopt it. And I have watched their failures — the blown budgets, the missing of promised deadlines, the angry finger-pointing meetings — all because the managers and their teams misunderstood Agile.

Why is Agile — the big-A variety — so vanishingly rare? And why is there so much bad Agile out there?

The Basics

I was puzzling over this question when I read something that helped me put things into context. Agile is not Scrum. It is not SAFe. It is not a framework. It is not a set of ceremonies. Go back to the Agile manifesto, and what you’ll find is a set of values and twelve supporting principles.

I personally keep the manifesto taped on my wall. It starts with a simple statement:

We are uncovering better ways of developing software by doing it and helping others do it. Through this work, we have come to value:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan.

That first sentence requires a leap of faith. How do I know that the seventeen signatories are developing software better than I am? I’ve never worked with Beck, Fowler, Cockburn, Martin, or anyone else who originally signed on to the manifesto, so how can I be sure?

What I’m getting at here is that Agile is, at its heart, a set of values. It’s a belief system. And, like many belief systems, believing that the system can work is the first step to making it work for you. If you can grasp this, you will understand why Agile has more in common with Alcoholics Anonymous than with ITIL or Six Sigma. In AA, the participant has to believe that the twelve steps will lead to sobriety or the program makes no sense whatsoever. In Agile, the corporations have to believe that self-organizing, empowered teams can get the work done better than micromanaged teams and set-in-stone deadlines. Without this belief — or even belief in the possibility for this to work — your Agile transformation will fail. You may follow the Scrum guide to the letter, but your results will be half-assed, and you will have no idea why.

Here we start to get at why Agile works for some and fails miserably for others. There are a core set of beliefs we all rely on about what people are like. A lot of literature in AA talks about admitting your own helplessness and turning your life over to God (or as they put it, to a higher power). I’ve never participated in AA, but it seems pretty clear that taking this steps requires you to believe that the higher power exists, actually loves you, and wants to help. Otherwise, why bother? If you believe you are handing your life over to an evil deity who wants to smite you and send you to hell for your sins no matter what you do, you might think twice about this step. In other words, the alcoholic tells herself a story about who God is because she’s stuck, she’s desperate, and people around her are telling her that this will help. And in the beginning, she doesn't need to believe any of this. She just needs to accept that adopting the belief might help

The power of core beliefs is in the stories we tell ourselves. Every religion contains a narrative about how the universe came about and why we are such a mess. In the same way, every executive has his or her own story about what type of people show up to work each morning and how much they can be trusted. It is deeply, perhaps excruciatingly, difficult for technology executives to think in quasi-religious ways about how they manage their employees. But if your Agile transformation is going to be in any way genuine, you need to face this difficulty, or you're going to be tripped up by it.

Hobbes and Locke

To take a less fraught example, consider the early modern philosophers Thomas Hobbes and John Locke. Both had a very different concept of what people are like, and arrived at very different conclusions as to how they should be governed. To Hobbes, people were awful and would destroy each other if given half a chance. The natural state of human life was, in Hobbes’ own words, “solitary, poor, nasty, brutish, and short.”

Locke, on the other hand, was a lot more optimistic about the human condition. To Locke, humans were not born awful. Instead, they were born with a blank slate — a tabula rasa — and could be influenced for good or ill. Surround a newborn with a bunch of people who believe that people are craven and disgusting, and that baby will grow up to have some bad characteristics. Put the same child among people who believe that people are innately good, and the child will develop a much sunnier outlook on life.

These are two mutually-exclusive theories about who people are and what they’re like. Both naturally give rise to ideas about how people should be lead. Hobbes believed that you need a strong, centralized, absolutist authority to curb all those self-destructive tendencies. For an Hobbesian, it would make no sense to build a core software competency out of empowered teams, as those teams would destroy each other and the company they work for at the first opportunity.

Locke, on the other hand, believed that people had innate rights dictated by natural laws. Government to Locke is a social contract with the governed, and individuals have certain rights (like property) that the government cannot and must not trample on. Because people start out as blank slates, it is in the government’s best interest to influence and educate people in desirable ways. And the government should check with the governed at regular intervals to make sure that it is fulfilling its own part of the social contract. But for the most part, where it cannot be helpful, the government should stay out of the people's way.

So who’s right? Are people irredeemably awful, or do we have sparks of good in us that can, given the right environment, grow and take root?

Well, in light of our modern understanding, they’re both pretty clearly wrong. Hobbes had no basis to understand why we modern folk bother with elections. Locke’s tabula rasa left no room for our modern understanding of both the effects of trauma and how much our brain chemistry contributes to our temperament.

On the other hand, Locke’s ideas gave us Constitutional Democracy and the modern nation state. Hobbes gave us a form of governance currently favored by drug cartels. We can let history judge that one.

Outside of rightness and wrongness, it’s clear that one set of beliefs has proven, at least in retrospect, more useful. But more than that, between the Hobbesians and the Lockeans are a conflicting set of stories that dictate how leaders ought to behave.

Taking the Leap

What does any of this have to do with Agile transformation? In my experience, many business leaders are closet Hobbesians. They are, to put it mildly, very uncomfortable with the idea that their teams and their customers might actually want to help them rather than rob them. The Hobbesian manager will naturally gravitate toward a system of command-and-control, because to do otherwise literally makes no sense. Agile requires a more Lockean mind — or the ability to imagine a more Lockean sensibility working out for everyone. There’s a very real leap of faith involved, one that most business leaders are either unable or unwilling to make.

Let me be very clear: the empirical evidence is overwhelming. Agile works. Leveraging the collective intelligence of the entire organization works much better than trying to run the show yourself. But until that initial leap of faith happens, Agile will fail. As coaches, we can cite study after study showing the beneficial effects of Agility. We can show our clients that their competition is close on their heels. We can demonstrate that they need to make a move now if they want to survive. But without that faith, the un-Agile organization will remain on the floor, bottle in hand, looking up at the window at all the sunshine that falls on everyone else but them.

Agile will not transform your organization until you are ready to trust it. This trust requires a heroic amount of humility.

So, as you go about your work today, ask yourself, What’s my core belief? Do you believe in the possibility of Agile? Are you ready to take that leap?