When Scrum fails

David Andersen used his Meetup talk i Oslo last week to describe a start-up’s journey from Scrum to a more Kanban based way of doing things. I seldom work with start-ups, but David’s talk resonated with my experiences in contexts that have little do do with start-ups. I summarized the insight in a tweet:
“The Meetup with @agilemanager yesterday reinforced my belief that Scrum works best when you are in a state that you should avoid being in”

I freely admit that as tweets go, this one is pretty cryptic. A lot of people did not understand what I meant. Others stumbled into false logic and thought I was implying that Scrum does not work in most contexts/states. No! That is not the point. Scrum works for a lot of people and in a lot of different contexts.

The point I was making is this: One of the core ideas with lean start-ups is that you have to get your product “out there” as soon as possible. The real feedback you get when you have real customers is invaluable. At the the other extreme, my experiences with large corporate replacement projects has taught me the same lesson. Get the new system in parallel production as soon as possible. A few years ago this led me to create a wiki that collected patterns and insights for how to do this. But once you have succeeded in getting real users for your product, things tend to get complicated. These users will demand that you fix problems quickly and address missing features as soon as possible. All of this can be handled in Scrum, but it gets progressively harder the bigger this pressure is. It is also easy to fall into one of the traps that can ensnare a well meaning scrum master:

The valiant gate keeper

A couple of years ago, I talked with an experienced Scrum master in a large project. He was very satisfied with the Scrum had helped him shield the team so that they were able to complete their sprints. The team was very satisfied with his work and they managed to deliver lots of features at the end of each sprint.

I recently met a woman that had been on the business side of the same project. She told me that this same team had not worked very well in terms of the project as a whole. The Scrum master’s insistence on shielding the team had lead to considerable problems elsewhere in the project.

I honestly believe that both the Scrum master and the businesswoman were truthful in their descriptions. The Scrum master had succeeded in helping his team deliver value but at the same time this had come at the cost of being too inflexible when unforeseen problems occurred.

Sprint fiction

Would things have been better if the Scrum master had been more flexible? I’m not sure. I recently coached a team that was having trouble adopting Scrum. After observing them for a while I realized that up to 50% of the work they were doing in a sprint was not foreseeable at the start of the sprint. When you also take into account that they worked in a highly specialized environment where tasks could not freely be reallocated to other team members you get a situation where sprints are bound to fail. So in this context, flexibility was helping the business as a whole, but it was making a mockery of the whole idea with sprints.

Now, a lot of people will read these descriptions and their heads will be bursting with ideas for how to have done things better. That is not the point! Of course you can do better! The fact that you can do better does not invalidate the observation that Scrum gets harder to implement the more chaotic and interdependent your environment is. I think Scrum can be great in some contexts and that you can handle these types of problems… up to a point. But once you pass that point you should seriously consider looking at a more Kanban based approach. (I am not going to tell you where that point is. That you must find out for yourself.)

Niklas Björnerstedt

Posted on 2012.02.06 at 20:48

Comments are off for this post.

One Response to When Scrum fails

  1. smalltalk80 says:

    Dagfinn Reiersøl on February 7, 2012 at 7:21 pm said:

    Ken Schwaber says (I’m paraphrasing): Scrum doesn’t solve your problems, it only exposes them. So my question is, when you abandon Scrum because it you can’t get it to work properly, are you doing the only right thing, or are you just avoiding dealing with the impediment that Scrum has exposed?

    If the environment is chaotic and interdependent, do you really need to use a Kanban based approach, or would you be better off changing the environment, if possible. Is chaos and interdependence caused by poor code quality, insufficient training and other variables that should addressed? And could Kanban be just what you need to stay mediocre in those areas? I think the answer is yes — sometimes. But my experience is insufficiently varied to generalize about it, so I’m just pointing out the possibility.

    Niklas Bjørnerstedt on February 7, 2012 at 9:19 pm said:

    Hi Dagfinn,
    I am aware of that possibility, it is just that it was not relevant in the case I was involved with. It was an organization that has hundreds of developers and scores of ongoing projects. These projects often involve large parts of the organization to some degree. Add to that the fact that my mandate was to help a few of these teams, not the whole organization. It would have been quixotic for me to set the goal of changing the environment. Most scrum “remove the impediments” stories are really about big projects, not big organizations where the actions of one project often have unintended consequences for other projects. A scrum of scrums is not a good solution in this kind of environment.
    The case David Anderson talked about in the Meetup is also a situation where you just have to accept your environment. A startup is a startup.

    Geir Amsjø on February 21, 2012 at 4:46 pm said:

    Hi Nicolas,
    David Anderson like to tell “Scrum stinks, Kanban is the way” stories. That is fine, but it is interesting that the examples he uses describes a bad Scrum implementation. The same goes for your story with the valiant Scrum Master. In the story there is no Product Owner, and that is probably the reason why the Scrum Master was allowed to continue with his eager team shielding practice. The Scrum Master is supposed to shield the development team if necessary. A good product owner would never allowed this bad, sub-optimal behavior to continue without bringing it up as a major problem in a retrospective.

    Niklas Bjørnerstedt on February 21, 2012 at 9:10 pm said:

    Geir,

    Actually, there were multiple product owners on the valiant Scrum master project. Since I was not part of the project I cannot say whether this particular product owner did a good job. I find it interesting that you are so certain that the cause of the problems was a poor scrum implementation. It is dangerously close to the “if you get sick your faith is not strong enough” argument. Suppose the product owner had taken this problem up in a retrospective. The result might then have been a team that was forced to abandon most sprints. It is not always possible to eliminate the root causes of disturbances.

    Some of the remedies to this kind of problem that I’ve heard in Scrum circles are based on ever more planning. That is not a road I am comfortable with.

    PS Spell my name right next time… :)
    Geir Amsjø on February 23, 2012 at 11:03 am said:

    Sorry about the name spelling, Niklas. I know i have done that before also, probably because I used to communicate a lot with a Nicolas before..

    In a good Scrum implementation this situation might go on for some Sprints, but eventually (probably in a retrospective) it will be detected and labelled as an impediment. And dealt with as such.
    I totally agree that if 50% of your work cannot be planned for a Sprint, you have a serious problem. You can do one of two things:
    1. Accept that this is how the the organization works, and it is nothing we can do about it: Go for Kanban instead of Scrum to free yourself from the iterations and the commitment.
    2. Threat it as an impediment and try to solve the (organizational) problem.

    In most cases (I have seen and heard about) the root cause of all these “not plannable tasks” are support cases i.e. failure demand. This should be attacked and not accepted, don´t you agree? I know you as a Lean thinker and even a Systems thinker…

    Scrum is designed to challenge the whole organisation and surface problems. It is explicitly empirical. It is NOT a “project management model”. The good Scrum implementations acknowledge that and they will not allow impediments like the ones you (and others) describe live for long.

    Niklas Bjørnerstedt on February 24, 2012 at 10:13 am said:

    Geir,

    Yes, impediments should be identified and removed… when that is possible. My point is that in certain contexts there are impediments that can not be removed. These impediments do not have to be failure demand. Some of the examples i cite are about customer demands that are so urgent that they can not be ignored. They can also arise from other parts of the organization that are outside the project’s control.

    Geir Amsjø on February 24, 2012 at 12:54 pm said:

    OK, now we seem to be closing in to the core issue – and I didn´t expect us to have very different positions. Impediments should be removed, the question is however, when to give in. It is a question of attitude and the urge to improve the end-to-end process. It is obviously harder to attack end-to-end problems or dysfunctions when there is another organization (a customer) owning significant parts of that process. In my mind people tend to give in too easy!

    I think the heading of this article is provocative and in a way wrong. If you use Scrum you should have problems – otherwise you are doing it wrong(!) Of course it is tempting to turn away from these problems. They may be painful to fight. That is why Courage is one of the Scrum values. The framework is designed to surface dysfunctions – so that they can be dealt with.

    So do Scrum fail? I don´t think so. This is a simple, empirical framework that is very adaptable. If you run into problems that seems unsolvable short term, it will most likely be possible to address more long term. There might be good reasons for not taking the fight right now, but please do not blame the framework for that.

    Niklas Bjørnerstedt on February 24, 2012 at 1:41 pm said:

    Geir,
    I think the title of the article is correct if you consider sprints to be an integral part of Scrum. The point I am making is that sprints (and the supporting ceremonies) do not work well in some environments. In the trend towards lean start-ups there are more and more efforts that encounter these problems. At the other extreme there are organizations that are so complex that efforts to make sprints work risk pushing the organization into a planning fallacy that is very close to the thinking behind waterfall processes.

    Geir Amsjø on February 24, 2012 at 3:12 pm said:

    OK, I agree that in some environments I would recommend Kanban and not Scrum. That is because the nature of work is “always urgent”. No need to estimate or to plan, just do it. Use the WIP limit to improve. I recommend it because I can not see any urge to change this.

    When it comes to the very complex organizations I would really encourage to move away from accepting complexity towards fighting complexity. And I would have used Scrum, even if it will be painful in the beginning.

    So, yes you might say that Scrum fails in some special cases. The Sprint fiction chapter is OK. But I think it is kind of sad that far too many don´t like the pain of seeing their own problems and start blaming this framework rather than solving problems.

    The valiant gate keeper chapter on the other hand is just a bad implementation or poor understanding of Scrum.

About Niklas

Born in Sweden, grew up in New York, lives in Norway. Yes, I have problems with identity, but I do think that my background helps me see things from a different perspective.

Read more