In my previous post, I discussed the origins of Kanban to ensure we understood the needs it was designed to fill, and to emphasize some points often ignored by modern implementors. In this post, I will dive directly into the “Scrum versus Kanban” debate by summarizing my search for objective evidence. I looked for any project-related studies, but only found ones addressing software.
Unfortunately, it turns out most of what you hear about Kanban for software is situation-specific and not objective. Two researchers noted, “the reported benefits of flow (Kanban) are largely based on anecdotal evidence that is driven by consultants who may be biased in favor of these results.”[1] A 2018 review of all the published journal articles on Kanban over 17 years found only 23 related to projects (instead of manufacturing or supply chain). Almost all of those were case studies, so again the results were “context-specific and may not be applicable to other environments.”[2]
Indeed, a literature review of 32 variants of Kanban found many concerns related to using it in different contexts, because it was invented by a specific company for its specific industry and needs. The literature from 1982 until 2002 concluded Kanban was “not adequate in situations with unstable demand, processing time instability, non-standardized operations, long setup time, great variety of items, and raw material supply uncertainty.”[3] Except maybe for the raw materials, the rest of those describe every project I’ve ever led. Combined with use of the word “flow” as a synonym, this raises the question of whether Kanban is appropriate for managing projects as opposed to repetitive, flowing work.
First, I will expose my biases. I have trained teams to use Kanban for flow, most recently a hardware testing team. But to me, there are three major weaknesses of Kanban for project work:
- The best type of tool for a given job is always the type designed for that job. As detailed in my previous post, Kanban was designed for a flow of a limited selection of routine work products requiring little cooperation to build.
- Repeated studies have found that deadlines are necessary to maximize team performance. Agile rejects the Waterfall Myth about long-term deadlines, but Scrum asserts that power by giving you a deadline every sprint.
- Virtually every study I’ve read about teams over 20 years lists the value of multiple perspectives as a key reason teams outperform manager-directed work groups when teams are appropriate. Kanban has no prescribed meetings for leveraging those perspectives, and I could argue my career was built on the evidence that team meetings without formal structure add little value.
Nonetheless, those case studies on using Kanban for software generally found improvements over whatever came before. The best-designed study compared teams within a large Indian company that had adopted at least one lean practice to those that had not.[4] (Kanban is considered a Lean method, but these practices were not limited to Kanban.) The study’s context within one company raises the concerns about applicability quoted above. But this case provided the advantage of comparing lean-er versus non-lean projects matched by business unit, contract type, size, and in one set, complexity. Obviously, too, the teams operated within the same organizational culture, policies, upper management, etc., so it was an apples-to-apples comparison. The study found that relative to non-lean projects, those using a lean practice had:
- Less schedule deviation, overall and in one set of matched projects.
- Less deviation from labor hour estimates, overall and in both matched sets.
- Lower standard deviations in both statistics, which you could interpret as another sign the lean projects were more predictable.
Quality was no better, however. The researchers suggest that may have happened because lean was often adopted after the project was in trouble. On average they were already 42% complete. Imagine how much better these results would have been for projects adopting more than one practice, from the start of the project.
That review of 23 software studies concluded, “flow tools such as Kanban and WIP limits have (1) improved control and management of workflow; (2) resulted in a more continuous, smooth flow of value; and (3) reduced the average lead time needed to complete customer requests.”[5] However, it does not indicate whether that was compared to other Agile methods or to something else.
A Slovenian professor performed a case study of four teams making the transition from Scrum to Kanban. Of the seven challenges he identified to adopting flow for software, only one was unique to Kanban (“Establishing WIP limits”). With minor wording changes, the other six have been problems with adopting Scrum as well, such as, “Over-control versus the need to monitor,” changing the culture to support the mindset, and disciplined use of the work tracking tool.
I found similar crossover in the reasons those Scrum teams gave for switching to Kanban: “There was a lack of visibility of the work being conducted, as well as a perceived ad hoc approach to the estimation, development, and release of software generally. There was a belief that flow could (1) provide greater visibility into what the development teams were doing, (2) promote performance improvement, (3) improve feedback loops, and (4) expose resourcing constraints and capacity utilization.”[6]
To my thinking, three of these indicate poor implementation of Scrum. Disciplined use of an Agile tracker eliminates #1, and interview quotes suggest they were not using one. Proper pre-grooming, Sprint Reviews, and Retrospectives fix #3. Number 4 comes from a lack of individual capacity planning as done in my Full Scale agile™ and other methods. Fixing those would absolutely have improved performance (#2). I’ll add that one benefit listed for Kanban was better “communication between development and testing teams,” suggesting the teams were not cross-functional as Scrum teams are supposed to be. Furthermore, they continued to use sprints after the transition, so they really switched to what some people call “Scrumban.”
As I wrote on the FuSca™ page about Kanban, the author of two books and a speaker at global conferences on Kanban provided an unexpected confirmation. I read his book subtitled, Managing Large-Scale Projects with Kanban. Despite the title, his teams ended up doing retrospectives every two weeks, followed by planning meetings for the “Next Ten Features,” and were considering a high-level Demo on the same cadence. They also added separate backlog grooming sessions a few days prior to planning. “This evolution toward a more Scrum-like model was not really intentional,” he wrote. “It was just the result of a series of process improvements triggered by real-life problems.”[7]
In a sidebar he added:
“This seems to be a general pattern: I see many Kanban teams that gradually discover (or sometimes rediscover) the value of many of the Scrum practices. In fact, sometimes Kanban teams start doing Kanban because they didn’t like Scrum and then later discover that Scrum was actually pretty good and their problems had been exposed by Scrum, not caused by it.”
In another example, a company was struggling with many software development problems despite introduction of Scrum. Every one of the 11 problems had nothing to do with Scrum, but rather poor implementation. Examples included:
- “Confusion upon the definition of ‘done.'”
- “Low prioritization of the technical backlog items.”
- “Non-conformance of large features to one sprint.”[8]
The first two could have been fixed within the standard Scrum Alliance framework. And the study said the last was still a problem after the transition to Kanban, in that teams had difficulty breaking down large features. In truth, all they really changed was to add visual boards and eliminate Sprint Planning. Demos and Retros were still done on a set schedule. And the lack of a sprint schedule was listed as a new problem! That prevented stakeholders from being able to plan for physical releases.
Finally, the study’s data had a major problem in that it did not compare similar time periods relative to adoption of Scrum and Kanban. The Kanban gains were similar to those after Scrum was first introduced, suggesting a Hawthorne effect (wherein performance improves just because someone paid attention to it!).
Another Scrum-to-Kanban study with a similar data problem admitted, “much of Kanban’s indicated advantage might have simply been due to the fact that Kanban was used after Scrum. (The firm) was familiar with agile methods for more than three years before Kanban was introduced…”[9]
My reading of the literature has both hardened and softened my position on Kanban. I am even more convinced that between the two of them, disciplined Scrum is the appropriate tool for project work, and Kanban is for flowing work. However, you can use Kanban for project work, and if you are using waterfall or no formal project management, it’s likely to improve performance. The flip side is, much of the Scrum overhead that adds value for projects is wasted for flowing work.
But “Scrumban” proponents have the direction wrong. That hybrid isn’t an improvement of Scrum by adding Kanban; it’s an improvement of Kanban for projects by adding Scrum.
Please share this post at the bottom of the page.
[1] Dennehy, Denis, and Kieran Conboy, ‘Identifying Challenges and a Research Agenda for Flow in Software Project Management’, Project Management Journal, 49.6 (2018), 103–18 <https://doi.org/10.1177/8756972818800559>
[2] Ahmad, Muhammad Ovais, Denis Dennehy, Kieran Conboy, and Markku Oivo, ‘Kanban in Software Engineering: A Systematic Mapping Study’, Journal of Systems and Software, 137 (2018), 96–113 <https://doi.org/10.1016/j.jss.2017.11.045>
[3] Muris Lage, Jr. and Moacir Godinho Filho, “Variations of the Kanban System: Literature Review and Classification,” International Journal of Production Economics 125, no. 1 (May 2010): 13–21, https://doi.org/10.1016/j.ijpe.2010.01.009.
[4] Staats, Bradley R., David James Brunner, and David M. Upton, ‘Lean Principles, Learning, and Knowledge Work: Evidence from a Software Services Provider’, Journal of Operations Management, 29.5 (2011), 376–90 <https://doi.org/10.1016/j.jom.2010.11.005>
[5] Ahmad et al. 2018.
[6] Viljan Mahnič, “From Scrum to Kanban: Introducing Lean Principles to a Software Engineering Capstone Course,” n.d.
[7] Kniberg, H., Lean from the Trenches: Managing Large-Scale Projects with Kanban (Raleigh, N.C: Pragmatic Bookshelf, 2011).
[8] Natalja Nikitina and Mira Kajko-Mattsson, “Developer-Driven Big-Bang Process Transition from Scrum to Kanban,” in Proceeding of the 2nd Workshop on Software Engineering for Sensor Network Applications – SESENA ’11 (Proceeding of the 2nd workshop, Waikiki, Honolulu, HI, USA: ACM Press, 2011), 159, https://doi.org/10.1145/1987875.1987901.
[9] D. I. K. Sjøberg, A. Johnsen, and J. Solberg, “Quantifying the Effect of Using Kanban versus Scrum: A Case Study,” IEEE Software; Los Alamitos 29, no. 5 (October 2012): 47.