Top 10 Challenges to Scrum Adoption

Organizational change is hard, right? When adopting Scrum in your team, or in your organization, you will often encounter many challenges moving to the new way of thinking. Knowing these challenges helps you improvise, overcome, and adapt to solve these challenges. We’ve been working with Scrum teams since 2008, so our view of the top challenges is certainly colored by our own experiences. We chose not to directly address the “scaled Scrum” (i.e. multiple teams closely coordinating on the same product/system/value stream) challenges. That might be the subject of a future article.

1. Leadership that Refuses to get Agile Coaching and Training specifically for Leadership
   • In the “2017 State of Scrum” survey, the #1 most important consideration when adopting Scrum was “Active senior management and support”. #3 was Participation of experienced Scrum trainers and Scrum coaches. The TOP 3 items causing tension between Scrum teams and the wider organization were: Top Down Command and Control management, Resistance to change (by the org), and lack of understanding or support (from the org). The #1 motivator for undertaking an Agile adoption was “Active senior management sponsorship and support.” In this article from Harvard Business Review, co-creator of Scrum Jeff Sutherland drives home that Executives should get in depth training on Agile/Scrum (i.e. not the “Executive Summary”): https://hbr.org/2016/05/embracing-agile

“When we ask executives what they know about agile, the response is usually an uneasy smile and a quip such as ‘Just enough to be dangerous.’ …because they haven’t gone through training, they don’t really understand the approach. Consequently, they unwittingly continue to manage in ways that run counter to agile principles and practices, undermining the effectiveness of agile teams in units that report to them…With the best of intentions, they erode the benefits that agile innovation can deliver.”

1. • This challenge is best described with a true story we heard from a well respected Agile thought leader.
     • Company: “We want you to come teach your Agile Leadership class, but instead of 2 days, we’d like you to shorten it to a half day.”
     • Very well respected Agile thought leader/trainer: “Why only a half day?”
     • Company: “Our company leadership doesn’t have enough time to sit through a two day class on Agile.”
     • Agile Thought Leader: “Then we don’t have enough time to get involved with your failed Agile transformation, because that’s what will happen. Let us know when your leadership is ready to succeed instead of fail.”
2. Waterfall Thinking
   1. In the “2017 State of Scrum” survey, the #2 reported Challenge to implementing Scrum was the difficulty of transitioning from Waterfall to Scrum.
      1. Because this anti-pattern is so prevalent, there are many names used to describe this phenomenon: Scrum in Name Only, WaterScrum, Scrummerfall, flaccid Scrum, the Methodology Facade Pattern, Cargo Cult Scrum, Half Baked Scrum, Fake Agile, Faux Agile, FrAgile, Agilefall, Agile Hybrid, waterfall backsliding, waterfall inertia, etc.
         1. The “Agile Hybrid” is one where your organization is sort of thinking halfway in Agile, halfway in waterfall. Some orgs try to blend Agile/waterfall, but that in and of itself is Waterfall Thinking. Agile approaches like Scrum form an ecosystem of interlocking parts. Remove even one of the more important interlocking parts, and the ecosystem will degrade over a period of time and then fail altogether, and it will be an absolute morale killer. Because the failure sometimes takes a while to occur, it is sometimes hard for the Agile beginners to know why it failed.
      2. Note that there are many other spinoffs of waterfall: RUP, Spiral, Iterative, traditional Project Management, and some people even consider Kanban a spinoff of waterfall. So, when we talk waterfall here, we are also speaking of waterfall’s “children” too.
3. Assuming Agile is quick or easy to Adopt
   1. Many orgs, and indeed some Scrum Coaches, act like a team or an org can adopt Scrum or Agile techniques within a few months, AND reap all of the benefits of Agile in the same time frame. It doesn’t work that way. First you have to learn to walk, then jog, then run, then fly. While a few benefits can accrue early, it often takes months or years to get good enough at these techniques to reap the /3-10X more successful benefits.
4. Waterfall Organizational Design by Application (WODA) (aka “Too many Component Teams”, creates unmanageable “External Dependencies”)
   1. In the “2017 State of Scrum” survey, the #1 reported Challenge to implementing Scrum was organizational design.
   2. WODA is present in an org where there is a team, dev manager or project manager who “owns” one or more applications (systems/products, etc). in the org. or where applications live in silos and have a ton of upstream and downstream connections to other applications. Features often cross multiple apps, apps owning a particular set of biz logic or functionality. This creates a traffic gridlock pattern in the organization where no one is able to make much progress on anything. While people and teams appear to busy, overall value delivery slows to a crawl and software deliveries are constantly late and impacted by each other. Simply starting to use Agile terminology and practices won’t solve this problem, though it will make the problem much more visible.
   3. The other big problem with WODA is that it often involves dev managers who “own” an application. These app owner dev managers often interfere with the Scrum roles as such:
      1. Interfere with Product Owner: Assigning work to the Scrum Team without going through the Product Owner.
      2. Interfere with the Scrum Master: Allowing Dev Team members to escalate impediments directly to the dev manager rather than to the Dev Team first, then Scrum Master, then the dev manager if needed.
      3. Interfere with Dev Team: These well meaning managers often try to manage day to day tasking or by misinterpreting Scrum artifacts and techniques. Another way to interfere is to use performance appraisals of Dev Team members has a way to maintain command and control authority over the team.
      4. It’s important to note that many of these managers have good intentions. In many ways, these managers are still being judged on their ability to “own” an application or function, so they think they are doing the right thing for the company, but they are not. The root cause of this problem is usually a lack of Agile/Scrum knowledge on the part of the managers and the leadership/people those managers report to (See #1 above). Managers need to learn how to be effective leaders in a Scrum organization.
      5. WODA is a larger version of a strong preference for Component Teams in an organization, again, a Waterfall holdover(hangover?). The org needs to move to Feature Teams instead.
5. Waterfall Organizational Design by Function (WODF)
   1. In the “2017 State of Scrum” survey, the #1 reported Challenge to implementing Scrum was organizational design.
   2. WODF (just as bad or worse than WODA) is when your organization is trying to do Scrum or be Agile, but the structure of the organization is still designed for waterfall. The worst examples of this are functional silos, where the people in different functions (architecture, programming, testing, UX design, business analysis, project management, release management, DBA) are all in different departments and/or report up to different functional hierarchies. This dysfunction can also present itself as:
      1. Individual functions sit together, rather than cross functional teams sitting together.
      2. Teams that are highly dispersed/remote in multiple locations and timezones rather than being co-located in the same room or Agile Team Workspace. Truth be told, dispersed teams don’t work well in waterfall either.
      3. Misaligned guidance coming down to teams from differing functional hierarchies.
      4. Annual budgeting processes that involve wishful thinking predictions about schedule, scope, and cost.
      5. Too many layers of functional management
      6. Too much authority being projected onto individuals by their functional managers who write their annual performance reviews.
      7. A separate department or team for any of the above mentioned functions. Ideally, in Agile organizations, all of those functions for a particular team would report to the same management regardless of their function.
      8. The other big problem with WODF is that it often involves functional managers who “own” a particular function (Programming, Testing, Arch, etc). These functional dev managers often interfere with the Scrum roles as such:
         1. Interfere with Product Owner: Assigning work to the Scrum Team without going through the Product Owner.
         2. Interfere with the Scrum Master: Allowing Dev Team members to escalate impediments directly to the dev manager rather than to the Dev Team first, then Scrum Master, then the dev manager if needed.
         3. Interfere with Dev Team: These well meaning managers often try to manage day to day tasking or by misinterpreting Scrum artifacts and techniques. Another way to interfere is to use performance appraisals of Dev Team members has a way to maintain command and control authority over the team.
         4. It’s important to note that many of these managers have good intentions. In many ways, these managers are still being judged on their ability to “own” an application or function, so they think they are doing the right thing for the company, but they are not. The root cause of this problem is usually a lack of Agile/Scrum knowledge on the part of the managers and the leadership/people those managers report to (See #1 above). Managers need to learn how to be effective leaders in a Scrum organization.
6. Waterfall Organizational Design by Matrix (WODM)
   1. In the “2017 State of Scrum” survey, the #1 reported Challenge to implementing Scrum was organizational design.
   2. WODM is essentially a special case of WODA, where a brand new “team” of “silo skill specialists” is assembled for each new “important” project, where the new “team” is formed by cannibalizing members from other existing teams, or worse, by cannibalizing people from other teams by saying things like “you are now dedicated 30% to this new project A, while maintaining your 40% on project B, and 30% on project C). The rest of the problems suffered by this design are described in the WODA section elsewhere in this article.
7. Big Bang Release Strategy(BBRS)
   1. This dysfunction is characterized by orgs attempting to change to Scrum but keeping the waterfall habits of big bang releases. While some improvement can come to Scrum teams in these scenarios, often the big bang waterfall bias will make these teams operate more like waterfall teams. (aka FrAgile, WaterScrum, etc — see “Waterfall Thinking” above) If your org is planning releases for a single product that span more than ~2 months, you are still doing big bang releases.
   2. Also prevalent in this challenge is the failure to understand vertical slicing of PBI’s/User Stories that have INVEST qualities, Minimum Viable Products, and the inability to recognize the value of incremental software delivery and short feedback loops from end users.
8. The Snowflake Objection
   1. This is the “Scrum won’t work for us” mindset that usually sounds like “Our team/org/business domain is ‘different’ than everyone else doing Scrum, so Scrum won’t work here without serious modifications. This also known as “ScrumBut,”, “Incorrect Scrum”, etc
   2. Scrum has been proven to work in ALL software development efforts in all industries. Vanilla software development, System Integration, Data Warehousing, Business Intelligence, Analytics/Reporting, small orgs, large orgs, external software, SaaS, internal systems — you name it, Scrum works great. Just do a google search and you can find people who have done it and written about it. You can also check out the Scrum Case Studies web site.
      1. A project mindset instead of a Product mindset, and the failure to recognize how early and often value delivery to end users is a huge business advantage of Agile practices
9. Adherence to Failing/Legacy Project Management techniques
   1. The Iron Triangle. Another key sign of Waterfall Thinking is any kind of preference for fixed date/fixed scope deliverables(aka the “Iron Triangle”). In the Agile space we call this “wishful forecasting” or “wishful thinking”. Software development is an inherently complex activity. Only 14% of Waterfall projects are able to meet the schedule/scope/cost iron triangle. Agile Scrum approaches are 3X as likely to meet the iron triangle, but this still is not a very accurate way of judging software success.
   2. Projects instead of Products. Communicating in terms of “Projects” (Waterfall) rather than Products (Agile) or Product Value Streams (Agile). This is often tied to Waterfall funding mechanisms like Annual Budgeting, and funding projects(Waterfall) instead of teams(Agile) or sets of Teams (Agile).
   3. Treating the Product Backlog like a Project Plan. Many orgs assume that the Product Backlog is a project plan, thus including every single activity on a team that takes time. One huge indicator of this is product backlog items or stories that do not represent software features (Technical stories, Analysis stories, etc) This is a huge misunderstanding. The Product Backlog is a Feature Breakdown Structure (Agile), not a Work Breakdown Structure(Waterfall). It’s focus is on feature delivery times, not tracking everything a team does.
10. Lack of User/Business Engagement
    1. Especially damaging are these mindsets from business side stakeholders and end users
       1. “You’re doing a rewrite/upgrade — just make it work how the old system worked with a few tweaks, and call us when it’s all done.”
       2. “We’re going to need all of the functionality before it’s useful to us or before we’ll give you feedback. Call us when it’s done.”
       3. “We’re too busy doing our day jobs to collaborate and give feedback to you every Sprint”
       4. “We are on board with Agile, but we don’t have time to participate in that ourselves”

What Now?

You may be asking yourself, or asking us, “Ok, so great, but how do we solve these challenges?”. The best way to solve these challenges is to find people inside or outside of your organization who are heavily experienced and/or heavily certified in Scrum scaling techniques (preferably they have both heavy experience and heavy certification). Obviously we provide those services (see “About” link at top of page for how to contact us), but we don’t want this article to be a sales pitch about us. First, listen to the Scrum Experts in your organization, then partner with them and accept coaching from them. They will lead you down the path that will enable your best understanding of these large challenges to Scrum Adoption.

Also, see our list of Top 10 Challenges to Scrum Teams for some team level challenges.

Some other Challenges

Here are some topics that didn’t make the top 10, but are likely in the next 10 challenges to Scrum Adoption: Dev Manager Mindset Change, Component Team Preference, Failure to Embrace Scrum (CoP, dedicated SM’s, New Career Pathing, etc)Waterfall Annual Budgeting Systems, Managing via Waterfall Metrics, Status Report thinking, Lack of Test Automation Skills, Slow/Dysfunctional Value Streams, Manufacturing Mindset, System Re-Write Death Sprials, Death March Scrum, Technical Debt Buildup, “I” shaped teams/people, Technical Stories, Software that is not Releasable.