Is it Time to Ditch the User Story?

My colleague and friend Matt Philip has written a blog post with an intriguing premise: namely, that the user story is dead, and that we should, in the name of all that is good and holy, ditch them altogether.

Sounds ridiculous? Well, not so fast. One of the things I have come to appreciate about Matt is there is very little in the world of work he won't question. With him, there are no sacred cows. He will put anything and everything under the microscope, even going so far as asking clients to stop using the word Agile, because doing so smacks of appeal to authority.

So when something like the canonical User Story grows like kudzu over every part of our software development, it makes sense to take stock and reevaluate. Are we doing the right thing? Or, at the very least, is there is no better thing right now we ought to be doing?

What I'd like to do here is winnow through Matt's argument and discover, for myself and for whomever else stumbles across this blog, what purpose user stories still serve. Can this patient be saved?

What's the Story, Morning Glory?

The term "User Story" predates the Agile Manifesto. Alistair Cockburn first referred to user stories in his blog in 1998, saying that a story is "a promise for a conversation." Kent Beck recommended user stories when he introduced Extreme Programming (XP) in 1999. Various bits of lore adhered to the concept until, in 2004, Mike Cohn published a comprehensive book that is a must-read if you want to understand how to work with them.

In the twenty-two years since they were introduced, we have welcomed into the profession an entirely new generation of developers who have only ever known user stories. For these folks, I'd like to introduce you to what you have been spared. Behold the Rational Unified Process Requirements Template. This document, which could easily bulk up to be between twelve and twenty-five pages, was intended to spell out at great length everything your program would do. If you went to university, you probably have not-so fond memories of late nights spent cranking out term papers. Now imagine cranking out thirty of these suckers in a row like your job depended on it, because your job actually does depend on it.

When the Agile Manifesto says "Working software over comprehensive documentation," the Rational Unified Process, which took over-specification to new heights of insanity, while not explicitly named, was heavily implied. It is genuinely difficult to imagine from our perspective just how big a burden documentation was. Every identified step of the RUP had dozens of document artifacts. These documents required a phalanx of scribbling gremlins to have any hope of satisfying one additional requirement of the process itself: namely, that the sponsor of the system would keep every page perfectly up to date over the life of the software. That's Comprehensive with a capital freakin' "C."

Back in the 90s, I landed more than one developer job because, as a Russian Literature major, I had no qualms about cranking out page upon page of this sort of thing. More than one manager back then confided to me that my communication skills helped me win jobs over people who were better coders. One even went so far as to say, "I love your req specs. They read like a book." Now and then I even managed to write some code.

How well did this work? Pretty good for me personally. The software? A disaster. Thanks for asking.

In light of my personal history, let's consider the assertion that Matt's blog post leads with:

But in a vast and sad irony, user stories have become the heavyweight documentation and process that they were meant to replace.

Let me not mince words. That is unmitigated bullshit. This writer has clearly never experienced the Stygian Horror that predated User Stories. Or he has the memory of a goldfish, which I know for a fact he does not. We can—indeed, we must—consider whether there's a better alternative to the status quo, but please don't expect me to entertain for a moment that we are anywhere near what the manifesto authors were reacting against.

What's the Story Here, Officer?

So right at the outset, Matt's argument suffers from excessive naiveté. That in and of itself does not make him wrong. Nor does it suggest that there isn't something smelly and off-kilter in User Story Land. I'd like to take a moment to dig into his argument, because he is on to something. So let's look at the strongest part of his argument: the diagnosis.

His assessment and his recommendations on what to replace user stories with comes from an anecdote.

I’ve been working on a side project building a web app with a friend, and I’ve been able to experience how refreshing and dynamic software creation can be when we free ourselves from the methodological fetters that we inherit and reinforce, of which stories are one (and one that I’ve been guilty of perpetuating).

That gets a big "hell yes!" from me. Anyone who works in software will appreciate the power of the side project -- the piece of work you have carved out just for you. My new, still-in-progress, TDD/JS/framework-free blog scratches that itch. So does this version of my blog. So are the hours I spend playing piano. Nobody pays me to do this. This is work I do for me. Doing it keeps me energized. It helps me stay connected with what I love.

I don't track user stories when I do a side-project. I don't measure velocity or estimate in points. Nor am I particularly concerned with completing what I start. In my senior year of high school, I started learning a D Minor Capriccio by Brahms (Op. 116, No. 7). It's a truly bizarre piece, full of strange rhythms and haunting harmonies. Decades later, I still have not completely learned it. And that's fine, because it's not like I have a customer at the other end waiting to pay me for it.

If I did, however, have a paying customer—if, say, a Philharmonic Orchestra suddenly found huge subscriber demand for showcasing high-intermediate mediocrities like myself—then you bet I would learn it. In fact, this dynamic would change my approach to practicing and performing altogether, as it would thrust me into a new role as a high-intermediate professional mediocrity.

How much change? Hard to say. But, having met more than my fair share of professional artists over the years, I can say with some confidence that joy and freedom are likely casualties once a paycheck appears on the horizon.

Can you, however, really fault someone for wanting more joy in development? If Matt is happy doing what he is doing, I applaud that he wants to spread that happiness around. That is a great and generous impulse, but is it enough?

Consider the scope of his project. Matt wants to take his side project, which is giving joy and happiness to a sample size of one—I would say two, but I have no insight into how much sunshine is dappling the heart of the other guy— and use lessons derived from it to overturn one of the bedrock practices of teams the world over.

Hand me the popcorn. This should be good.

In the Heart of the Heart of the Story

So what does he propose to use in lieu of user stories? Let's go hunting for it, shall we?

It’s actually easier than you might expect, though when you think about it, it shouldn’t surprise you. Software makers (and I use that term broadly to mean programmers, designers, quality advocates — anyone who enjoys creating software) are usually creative problem solvers. Give them the problem, and they’ll not only discover the solution, they’ll enjoy the experience of freedom, knowing the goal they’re aiming for. And they’ll create even better solutions that what we might’ve originally conceived (and codified on a user story).

Thank you! That clears up... literally nothing. I'm searching this paragraph for whatever the hell he's actually talking about, and I'm coming up empty. The best I can do, as a replacement for user stories, is "Give them the problem." I'm sitting here waiting for the punchline. Give them the problem? There are 36,000 developers writing code at Amazon. Who gives them all "the problem"? And who tracks that the problems are solved? And how do the problem solvers make sure their solutions aren't giving equal and opposite problems to other problem solvers? The solution is so nebulous, so devoid of basic mechanics, that criticizing it feels like trying to crack fog with a pickaxe. Swing away, but there's nothing there to swing at.

This is but one of the problems you may encounter when you try to scale up the fun time you just had on your side project to a solution an entire industry might want to adopt. You actually have to give them something to adopt. As in, you fly overseas, sign some papers, and fly back with a baby. Something that simple and actionable. The last time I checked, you can't change a diaper on cheery sentiment.

Lower down is another swing-and-a-miss:

I’ll propose what I think teams should do. All you need are some enabling constraints:

• clear product vision and goals
• trust and psychological safety
• low level of work in progress
• tight feedback loops

Excuse me while I rev up a rhetorical technique that Matt has used on me more than once: I don't see how the mere existence of the user story, in its current form, conflicts with any of those enabling constraints. You can easily do everything he suggests with or without user stories. Since I can discern no clear alternative to user stories anywhere in this post, I vote keep them around for now.

So here's my verdict on Matt's argument: no actual meat, bones, or moisture reposes upon the miasma of sentiment here presented. To paraphrase Gertrude Stein, there's no there here.

The Real Story

So as far as providing a workable alternative to the user story, the post is a dud. Where it shines, however, is in its suggestion that our current practice could use an overhaul. We should at least cast a critical eye on the overstuffed mess that is the current Agile-industrial complex. I recently watched a keynote address by Luke Hohmann where he pointed out just how simple the Agile process really is. Teams at all levels of a business get together; they discuss what worked and what didn't; they do more of what worked and less of what didn't. Lather, rinse, and repeat.

So why is this so damn hard for companies to do? Why are they so eager to open their wallets to allow chuckleheads like Matt and myself to wander in to check their compliance with what has to be the world's tiniest process diagram?

The problem is not the user story per se. The problem is how user stories have come to be used. That is, they are written up by dour product owners and business analysts, reviewed by teams who, staring at their palms, mutter, "Eh, it's a five, I guess." And voila! The story is now pointed! It goes into a very, very long queue of other stories with their associated points to be worked on eventually, which we have generally accepted to mean never.

We are very good at creating long lists of things to work on. We are less good at creating things that work. Our inability to hew to the original intent of the user story—indeed, the user story's current status as the project management prop du jour—contributes a lot to this problem.

Ditching user stories will not, in and of itself, correct the dysfunction. But is there some way to use the current state of the art to help alleviate the situation? Is there some Minimum Viable User Story that could help get us out of the rut we're in? Let's consider for a moment how we might pick up where Matt left off.

What Does a Story Do?

As far as I can tell, there are three functional elements to a user story. Any well-designed story should do at least the following:

• It should document requirements in the most efficient manner possible.
• It should provide a bounded unit of work which can be quickly understood and implemented by a developer.
• It should specify a set of criteria to test whether the implementation is complete or not.

There are other criteria, but I'm willing to stand pat on these three as the bare bones of what we need if we're going to produce things. If you have ever been paid to write code, you will understand why the last two are so important. I still get a twitch every time I remember a manager I worked for. His mantra was "How hard can it be?" as he piled on yet another arbitrary must-have requirement onto my plate. I lasted a few weeks with him before I left. If you can't put a boundary around the work to be done, you have just guaranteed that its completion date will regress infinitely to the future—as too many companies still fail to realize.

Let me go ahead and suggest another requirement that might help keep us focused:

• You cannot create any story if a team cannot complete it within four months.

Now we're getting somewhere. Go ahead and create your epics and features for, say, the next year. But any story you write that has not been picked up and completed after a short amount of time shall disappear as if dropped into the Memory Hole at the Ministry of Truth. Feel free to experiment with even shorter time horizons.

This last stipulation, I feel, could have the same overall effect on product planning that Kent Beck's test && commit || revert workflow has had on unit testing. It forces your planning into the short term, the only place where those meaningful conversations have any hope of happening. Also, it forces you to make shrewd choices instead of bulking up unsustainable queues.

Outside of these four criteria, teams should feel free to write their stories however they'd like to. As long as the description is understandable, and as long as there are specified tests to ensure we know how to declare the work done, and as long as the definition of the work and its guaranteed completion keeps to a tight boundary, then you're good.

In fact, you don't even have to call them stories. Call them Items. Or Penumbra. Or Fred. Call them Ishmael.

There. I killed the user story. Now let's talk about your team's Ishmael velocity...

Postscript

Apropos of nothing, if you want to write a blog post to introduce a simple, useful replacement for the status quo, allow me to recommend Matt Wynne's amazing article on Scenario Mapping. It's easy to follow, it's practical, and I've seen first-hand how it can revolutionize a team's approach to testing.