Top 10 Challenges That Scrum Teams Face

Below are the top 10 challenges we see Scrum Teams facing in terms of trying to execute and reap business value utilizing Scrum. These problems are faced by all software Scrum teams, whether the teams are doing more straightforward full stack software development, data warehousing, business intelligence, web sites, system integration, etc. All of these types of Scrum teams see these challenges.

1. Scrum Adoption Challenges At the Org Level: Almost all Scrum Teams struggle due to organizational level Scrum adoption challenges, and many of the below are just specific instance of those challenges. As such, it’s probably a good idea if you go ahead and familiarize yourself with the Top 10 Challenges to Scrum Adoption.
2. Leadership Refuses to support Scrum and/or Refuses to get Proper Training/Coaching for Scrum Teams: Scrum is night and day different from what you used to do, and because orgs “don’t know what they don’t know”, they often refuse to invest in proper training and coaching. Learning about Scrum is easy. Doing Scrum is incredibly difficult, but it yields a 3-10X success factor, so it’s worth the effort to do it. A training class or coach can accelerate the learning curve by months or even years.
3. Upstream/Downstream Systems (External Dependencies):
   1. Some software teams have a large presence of upstream and downstream systems and/or data sources. Most software teams have more control of their own data, but the presence of many upstream/downstream systems means the control of the data can be scattered far and wide across the organization. This is often present in companies/orgs that are late adopters to Scrum. See “Waterfall Organizational Design by Application (WODA) in Top 10 Challenges to Scrum Adoption.
      1. In particular, “vertical slicing” in software development, with respect to User Stories, doesn’t seem to be the best metaphor for stories that have a lot of upstream and downstream system dependencies — maybe “end to end” or “expected inputs / expected outputs of the system” or “I/O channels” are better metaphors for those kinds of stories. See this alternative to the typical User Story Slicing model.
   2. While Upstream/downstream systems tend to be the most damaging External Dependencies, there are many other kinds of external dependencies that can be damaging as well. Delays in signoffs from other departments (legal, finance, hardware/software acquisition, marketing, release management, software operations), as well as delays or errors from dependence on skill specialists (architects, dba’s, other component teams, etc) can also damage flow and value delivery.
4. Creating Releasable Software Every Sprint: Often times a Scrum team will not be able to create a releasable bit of software every Sprint, even though that is strictly required by Scrum. The impediments to creating releasable software are often due to the above and below challenges or due to more organizational level Scrum adoption challenges. Regardless of the source, teams and Scrum Masters across the org need to band together to attack these impediments to creating releasable software every Sprint.
5. Poor Acceptance Criteria: Many software teams have trouble learning how to create and communicate well structured Product Baklog Items or User Stories that have good acceptance criteria (aka acceptance tests).
   1. Sometimes stories:
      1. Do not describe system boundaries, but instead intermediate steps (aka acceptance criteria that are really technical tasks, so called “technical stories”, or design instead of requirements, etc) or
      2. Have acceptance criteria that is not described in end the user/domain language — but in technical language instead
      3. Do not respect the INVEST criteria
   2. The above challenges can be mitigated by having a Scrum Master or Agile Coach who is strong in User Story practices and technical enough to know the difference between system boundaries and intermediate technical steps. (As always, there are trade offs: having a Scrum Master who is technical can invite technical micromanagement from that technical Scrum Master)
   3. We put this challenge close to the top because it’s usually the first one you want to tackle with a team new to Scrum(after getting some Scrum basics right, of course). The reason this one is so damaging is not about adherence to Scrum practice, but because the failure to adhere to this particular practice means you lose a TON of transparency onto what is working well and what is not working well in your team’s software delivery efforts. Get this one right early on, and you will have maximum transparency. Get it wrong… well…. get it wrong, and you will have just about every challenge on this page, and several of the challenges that didn’t even make this list. (the decrease in transparency makes it much harder to see those challenges — so they will last much much longer too)
6. PO Unavailability: Many software teams struggle with getting a business side Product Owner who can engage enough with the team. This creates a huge chokepoint in the most important stream of work — the Product Backlog. This spawns off many other anti-patterns. There are several ways to skin this cat, but a good start is making sure the team has a good grasp on the Product Owner role basics, as well as the /modern definition of the Product owner role. There are lots of ways to solve this challenge, but they depend heavily on your organizational context and the root cause of your challenge.
7. Little Access to Users and Key Stakeholders: This is an age old problem in software development, exacerbated by Waterfall habits. The Scrum development team members should have direct access to the users (if they are internal to their org) or at least direct access to user analytics (if users are external to their org). Sometimes built in telemetry and A/B testing can help with understanding the user viewpoint as well. When choosing which Scrum team members to talk to the end users, you definitely want to pick ones with good comm skills as well as coach them to never make forecasts or commitments on when particular functionality will be delivered. The goal is to get super early, super direct feedback from users as early as possible, preferably as early as during the release planning or Product Backlog Refinement process and definitely in the Sprint Review. It’s also worth noting that this challenge, along with “PO Unavailability”, create another large challenge: the Development Team has no idea what business value their Product is delivering. This is very very damaging to the overall effort and wastes a massive amount of product development money.
8. Scrum Outsiders: People who are outside the Scrum team often attempt to apply Waterfall management practices onto the Scrum team, without recognizing they are almost certainly harming the Scrum Team by doing so. These are usually front line management or project management who has never been to Scrum training and/or never practiced Scrum. These people need to compensate for this lack of experience by doing some deep learning on Scrum, and accept coaching from their Scrum Masters and Scrum Coaches. It’s worth noting that the Scrum Master has a specific duty in Scrum to coach those outside the Scrum team on how best to reap the benefits of Scrum. A good, cheap, entry level learning experience would be for all involved management to get the PSM I certification, which only costs $150 to attempt the cert exam.
9. Massive Unknowns: Some software teams, during certain development periods in the product’s lifecycle, will go through a period of having a massive amount of unknowns (people, requirements, infrastructure, technologies, green field development, etc). Truth be told, all software development is complex and has significantly large unknowns, so the issue we’re describing here is when there seem to be a massive amount of unknowns. There is often a discomfort on Scrum Teams around not having “stories” that account for the time spent in refinement and discovery understanding these unknowns (discovery, requirements, analysis, and just enough of the design) to create well structured User Stories with the INVEST attributes. Teams will be tempted to create “Analysis Stories”, “Design Stories”, and “Spike Stories” to account for this time, thought this is really just a waterfall habit. (“Technical Stories” is sort of the “broader term” to describe this waterfall habit of feeling the need to create stories for any team activity that takes significant time)
   1. This one is especially damaging because many orgs decide to try Scrum for the first time with a “new project” or major new product development effort. Due to the organizational pressure exerted around such efforts, often times management and teams are very quick to backslide into waterfall thinking and feel the need to create a story to describe every significant activity of the team. This is really just a sign of thinking the Product Backlog in Scrum is the equivalent of a traditional Project Management project plan. That is not what a Product Backlog is at all, and because the Product Backlog is the centerpiece of work in Scrum, this is a particularly harmful anti-pattern that spawns many other anti-patterns. See the discussion on “Treating the Product Backlog like a Project Plan” in the Top 10 Challenges to Scrum Adoption.
   2. 3rd Party Component Analysis/Design: Many software teams choose to use 3rd party proprietary software components (libraries, frameworks, web services, API’s created by another organization or from outside the organization), or even open source components that are not well documented/supported.
      1. These components often behave like black boxes with proprietary bits that are not visible to the component integrators, which creates challenges around how to use the component wisely, This often requires more analysis/design up front about how best to learn and use the component. This work can be folded into Product Backlog Refinement.
10. Extensive Discovery/Market Research: Some teams have a challenge where more refinement and analysis might be needed than other teams in order to understand the target market for the product. This is especially challenging for product companies that sell software or products to the public.This can be mitigated by having more personnel committed to market research or by having market experts working closely with or on the Scrum Teams. This can also be mitigated by simply having market experts take the time to teach the Scrum Team about the market.

 

What Now?

You may be asking yourself, or asking us, “Ok, so great, but how do we solve these challenges?”. The best way to solve these challenges is to find people inside or outside of your organization who are heavily experienced and/or heavily certified in Scrum techniques (preferably they have both heavy experience and heavy certification). Obviously we provide those services (see the “About” link above for how to contact us), but we don’t want this article to be a sales pitch about us. First, listen to the Scrum Experts in your organization, then partner with them and accept coaching from them. They will lead you down the path that will enable your best understanding of these large challenges to Scrum Teams.

A Few More Challenges

Below are a few more challenges to Scrum Teams that we found to be of interest.

1. See our list of the Top 10 Challenges to Scrum Adoption
2. 3rd Party Component Testing: There is a question as to how much testing should be done against the component since in it’s already been tested by it’s creators.
   1. Most teams use a “spot check” mentality when testing includes 3rd party component behavior, with a focus on
      1. More testing on the extensions/glue code that your custom code brings to the table.
      2. Lots of end to end testing that also does “spot checks”
   2. Speaking in terms of the Test Automation Pyramid, this would call for a heavier focus on “Integration” and “UI” testing that is more end to end focused. Both of those higher levels of testing end up using more tools and scripting languages rather than programming languages (programming languages are generally more focused on unit testing ). Compared to Unit testing, “Integration” and “UI” tests are more expensive to create and maintain (but the excellent ROI of automated testing is almost always present once learning curves are dealt with)
3. Complex Business Domains: Some software teams operate in automating business domains that are extremely complex. Examples are: aerospace, medical, telecom, embedded devices, or domains where compliance rules are complex — banking, security, etc. Examples that might not be as complex are: social media, retail, web sites centered around content, education, etc.
4. Lopsided User Stories: Most software teams experience situations where a particular User Story seems extra heavy in either the coding or testing effort. A good rule of thumb for the number of acceptance criteria(aka acceptance tests) for a single User Story is somewhere around 3-5, which usually reflects a “little bit of coding”, and a “a few acceptance tests”. However, sometimes one line of code can spawn dozens of acceptance tests, and sometimes a ton of code results in only two acceptance tests. This “lopsided-ness” will also probably heavily influence how you split the story.