Dealing with Hard to Find Bugs (Sprint Killers) in Scrum

This question was asked in an online forum(I’m paraphrasing):

> How do people here handle the impact of difficult errors/bugs (but not legacy bugs) on sprint progress?  Like ones that take weeks to solve?

In my professional opinion, the answer is: we make them transparent and try to improve upon them — at Daily Scrums, Sprint Reviews, and Sprint Retrospectives.

I tend to coach teams to handle bugs in Scrum using the Bradley Bug Chart.

One of the aspects of the Bradley Bug Chart is that bugs like the one mentioned (i.e. non legacy bugs) end up on the Sprint Backlog.  Because they end up on the sprint backlog, if one is using Story points and velocity, no story points are assigned and no velocity is gained from fixing bugs.  This, once again, helps provide transparency on to the lack of progress that the team might be making due to bug fixing.  The truth can be a hard pill to swallow, but the truth will also help set you free from these mistakes in the future.

The transparency should help all involved understand that there is something that needs improving, that is dragging down the team’s ability to produce new features and working software.  I would argue that this is not a sprint killer.  It is simply a fact of complex software development.

The real issue comes down to this:  Scrum transparency is trying to tell your team something.  What is it trying to tell your team?  What is your team going to do about it?

Related Articles

• The Tao of Scrum – Transparency

• One way to Handle Bugs and Production Support in Scrum

• One way to Handle mid-Sprint Scope Changes in Scrum

ScrumCrazy.com update:

• Looking for Agile/Scrum/Kanban Coaching or Training?  Contact us for more info.  We have some good specials going on right now, but they won’t last long!
• Finally, a Scrum certification course aimed at ALL members of the Scrum team! Developers, Testers, Business Analysts, Scrum Masters, Product Owners, etc.  Feb 28th in the Denver Tech Center.  More info and sign up here!

Kanban vs. Scrum: Kanban is NOT for Software Development, but Scrum is!

Last week I was a panelist at the Agile Denver meeting, where the title of the panel was Kanban vs. Scrum! The Big Smackdown!

If you click through the link, you’ll notice that the title was misleading, on purpose, as is the title of this article(“Kanban vs. Scrum…”).  However, I want to catch the attention of those who think that this is even a valid question or consideration.  The title also speaks to a common problem in the industry that has been around for several years, and won’t seem to go away.

There are a number of software teams and organizations that think they should choose between Kanban and Scrum as their software development process.  This is a GIANT and RISKY mistake, in my professional opinion.

It’s not an either/or proposition.  Scrum is about software development.  Kanban is about change management.

There are several reasons why choosing Kanban as your team’s software development process is a mistake.

1.  You are applying Kanban to the incorrect context.

Would you use a hammer to insert a screw in a wall?  You can, but you’ll probably damage your wall in the process, and the same is true of Kanban as a software development approach.  David Anderson, the creator of The Kanban Method, has apparently said this over and over again since 2005, but no one seems to listen.

Don’t take my word for it, listen to David:

“Kanban is NOT a software development life cycle or project management methodology! It is not a way of making software or running projects that make software!” — David Anderson

“There is no kanban process for software development. At least I am not aware of one. I have never published one.”  — David Anderson

“It is actually not possible to develop with only Kanban.  The Kanban Method by itself does not contain practices sufficient to do product development.” — David Anderson*

(*The first two came from the “over and over..” link above.  The last quote was sent to me via email from someone at David’s company.  I think they just pasted in something David had already written)

I should also mention that others have mentioned to me that David talks out of both sides of his mouth about Kanban, Agile, and software development, perhaps trying to capitalize on the fame and success of Agile software development.  That may be true, but it may also be true that David has been saying all of these things for years and no one is paying attention to what he says, which is unfortunate.

2.  Kanban is modeled more after the assembly line and manufacturing.  Scrum is modeled more after creative product design.

Which do you think more closely resembles software development?  Laverne and Shirley on the assembly line at the Shotz Brewery? Or the group of NASA engineers on the ground who saved the lives of the Apollo 13 astronauts by coming up with a creative solution to a problem within a time-box?  If you think software rolls off of an assembly line, then I think that it is unfortunate that you’ve never worked in a creative software development environment — it’s AWESOME!

Maybe my Laverne and Shirley reference is oversimplified.  The reason to use Scrum instead of Kanban for software development delves down into process control theory, and the difference between a “defined process” and an “empirical process.”  In short, a defined process works better when the inputs and outputs to the process are well known and repeatable (like a manufacturing line).  An empirical process works better when the inputs and outputs to the process are less known and very difficult to repeat.  No two software features are alike.  This is why it’s darned near impossible to measure software productivity directly, even though some naive “bean counters” still try to.  Like the stock market, no one metric will predict it accurately, but a range of indicators can help predict it more accurately.  So, in summary, Scrum is based on empirical processes like product design.

One of the very key parts of empirical processes is the characteristic of inspecting and adapting the product.  Think of yourself making a pot of soup from scratch, without a recipe.  Think about all of the “taste-tweak ingredients-taste” experiments(feedback loops) you would need to get a pot of soup that tastes good.

Scrum has the frequent feedback loops built in, for a variety of audiences(Dev Team, Product Owner, Stakeholders, Users) , and for a variety of topics(process-Sprint Retro, product-Ordered Product Backlog, product-Sprint Review, product-Valuable/Releasable Increments).  Kanban has no such built in loops, but again, that’s because it wasn’t designed for software development!

3.  From a Complexity Science view, Kanban is for ‘complicated’ work while Scrum is for “complex” work.

I know the Kanban folks don’t like hearing this, but I think Ken Schwaber was right when he said this, and I think history will prove him right about Kanban as it was described in David Anderson’s book.  In short, the Cynefin model defines 5 domains, of which 2 of them are “complicated” and “complex” work.

‘Complicated’ work is best solved via ‘good practice’ and ‘experts’ who can find ’cause and effect’ fairly easily. When I think of ‘complicated’ work, I think of an the IT support person who sets up your computer or trouble shoots it.  Yes, you need an expert to solve these problems, and the vast majority of the time, the steps to solve these kinds of problems are fairly consistent and repeatable.  They are not exactly repeatable, just mostly repeatable.   If the steps were exactly repeatable then they would fall into the ‘Simple’ domain of Cynefin.

‘Complex’ work is best solved via ‘safe to fail experiments’ and one can only ascertain cause and effect after the fact.  Each Sprint in Scrum is a ‘safe to fail’ experiment because, while the Sprint increment is always releasable, the business side of the house makes the decision on whether it is safe/valuable to release it or not.  In the case of an increment that is un-safe, the team course corrects and comes back with an increment the next sprint that is hopefully safe or more-safe.  These safe to fail experiments can be repeated over and over again until it’s time to release the increment.

Applying Kanban Correctly

Having said all of the above, there IS a time and place for Kanban — a correct context, if you will.  If you’ve been reading closely, that context is as a change management process, which is ‘complicated’ work, and requires that there be already existing processes in place.  So, if your software team is doing XP, Scrum, Crystal, Waterfall, RUP, DSDM, FDD, etc, then you can layer Kanban on top of it to help find bottlenecks and waste.  Also, for all of those teams out there that don’t use a software development process(framework, approach, etc) that is named in the industry, you’re probably doing cowboy coding, ad-hoc, or command and control project management — none of which is a software development process either.  So, layering Kanban on top of crap will still yield crap.

For those that want to apply Kanban at the enterprise level to monitor the flow of work through their Scrum teams (Or XP, Crystal, etc), or want to use it for IT support or Dev Ops, I say have at it and I hope it helps you.  I imagine just visualizing your workflow alone will help in those contexts.  I myself have recommended and coached Kanban for a couple of teams — but only because those teams exhibited the right context for Kanban to be successful.

Bottom Line

Having said all of this, just visualizing your workflow and the other Kanban principles is not enough for software development.  Software development has things like business value, technical complexity, and user experience/acceptance/adoption — all of which are not addressed directly by Kanban.  Scrum does address these areas, as I have shown above.  But hey, let’s not forget, the Kanban Method is “not a way of making software or running projects that make software.”  Would you criticize a hammer for not doing a good job of being able to insert a screw into a wall?

Related Articles

• The Tao of Scrum – Transparency

• Scrum For Lay People

• One way to handle Bugs and Production Support in Scrum

ScrumCrazy.com update:

• Looking for Agile/Scrum/Kanban Coaching or Training?  Contact us for more info.  We have some good specials going on right now, but they won’t last long!
• Finally, a Scrum certification course aimed at ALL members of the Scrum team! Developers, Testers, Business Analysts, Scrum Masters, Product Owners, etc.  Feb 28th in the Denver Tech Center.  More info and sign up here!

Agile and Scrum Hiring: Recruiters Mistakenly Looking for 12 Years of Agile Experience

We used to have the same exact challenge in the Java world.  Java came out somewhere in the neighborhood of 1995, and by 1998, recruiters were looking for people with 5 or 10 years of Java experience.  It happened to me often as a young budding Java programmer.

I’ve seen this happen with Agile as well.  I work with a lot of recruiters and partners on a lot of initiatives, and one of them is doing Agile and Scrum “phone screens.”  Before I do a phone screen for a client/recruiter, I ask for the job description so I know what they are looking for.  When I see job requirements that ask for something like “12 years of Agile”  or “7 years Scrum Master experience,” I have a talk with the recruiter.

I usually just politely say something like this to the recruiter:

“Agile really didn’t exist until about 2001, and didn’t really start taking off until about 2008 or later.  Even today, most organizations are relatively new to Agile.  The folks that were involved before 2008 are generally early pioneers that now charge about $200-1000/hr for their services.  So, if you’re looking to hire one of the pioneers, I can certainly refer you to them.  On the other hand, if your client is new to Agile and didn’t realize what they were asking for, I’d be happy to collaborate with you to tweak the job description to what it is they really *are* looking for.”

(Agilistas may want to to quibble with some of the facts or dates in that statement, but it’s fairly accurate, and it gets the point across.  The intended audience is recruiters, not Agilists.  Having said that, I welcome feedback and corrections!)

One other point of note is that there are a lot of people who claim they were doing “Agile” or “Agile Project Management” who are essentially “glossing over” or attempting to deceive about their “X years of experience in Agile.”  I have several ways of figuring this out in my phone screens, but I won’t mention my tactics here as they are trade secrets of ScrumCrazy.com .    Scrum Master and other Agile job descriptions that require numerous years of experience tend to draw out these fakers even more — and I’m sure that’s not what the clients really want.

So, if you as a job seeker run into this issue, feel free to refer to this article to help explain realistic expectations when looking for good Agile or Scrum people.

For you recruiters out there, feel free to contact me about my phone screen services, but also understand that the person who referred you here is trying very much to help you in your quest to make your clients happy, just as I am.

In the interest of transparency,  I (very mildly) financially benefit from the phone screens that I mentioned, but I don’t want this blog post to come off as a pitch for services.  Having said that, I feel like I’m providing a fairly low cost, high ROI service that helps the entire Agile community as a whole.

I’m Giving a Free Global Webinar this Wednesday on “Acceptance and Story Testing Patterns”

I just wanted to send a quick note to my followers to let you know that I’m giving a free global webinar this Wednesday on “Acceptance and Story Testing Patterns.”

Here is the abstract for the presentation:

Acceptance Testing, also known as Story Testing, is vital to achieve the Agile vision of “working software over comprehensive documentation.” It’s very important that acceptance tests are easily automated, resulting in a phenomenon you may have heard of, called the “Agile Specification.” In this presentation, we’ll discuss eight different patterns of expressing acceptance tests so that they are easy to execute and automate. We’ll talk about popular patterns like Given/When/Then and Specification by Example, as well some other patterns you’ve probably never seen. Attendees will participate in an interactive exercise that will allow them to apply the most frequently used Acceptance Testing patterns.

You can sign up for the free webinar here:

http://tinyurl.com/d4d6ehw

Towards a Catalog of Scrum Patterns

I’ve been thinking a lot lately about Scrum patterns.  To that end, I’ve drafted a conceptual model, which is depicted in the below image.  (You may want to click the image twice to see a bigger view of it)

“Sprint Progress Monitor” is my term for what used to be called the “Sprint Burndown.”  In the 2011 Scrum Guide, Sprint Burndowns were no longer required but some way of monitoring sprint progress, via summing the remaining work, was still required.  In the Scrum Guide, it is made clear that these practices can be implemented using several techniques, and “techniques may vary” from team to team.  So, this is what inspired me to use the term “Scrum Technique Pattern.”  A Scrum Technique Pattern(STP for short) implements the intentional gaps left by the Scrum Guide.  Teams can inspect and adapt their technique patterns, while still fulfilling the Scrum framework.  Said another way, there are intentional “variation points” in Scrum, and the STP’s are different ways of implementing those variation points.

“Optional Scrum Patterns” are patterns that can be used in conjunction with Scrum, but are not specific implementations of Scrum techniques as specified or required in the Scrum Guide.  These can be just about anything, but they must follow and/or support Scrum principles in order to be considered as an Optional Scrum Pattern.

I also see the need for the ability to create a “mashup” of different Scrum patterns to create a set of practices and techniques for a particular team or context.  For instance, we might start out with a Scrum Starter Kit that includes things like:

• Holding a Release Planning meeting every 2-3 months (OSP)
• 2 Week Sprints (STP)
• Using Story Points to estimate Product Backlog Items(STP)
• Using Ideal Hours to estimate Sprint Backlog plan items(STP)
• Doing a typical burndown chart to sum the remaining work for the sprint(STP)
• Representing Undone Work(OSP)
• Doing the typical “round robin” style of the Daily Scrum(STP)
• Doing a fairly typical “Plus-Delta” retrospective analysis(STP)

Any of the above patterns could be interchanged with other patterns, and the Scrum implementation would still conform to doing correct Scrum.  Presumably the change to different patterns would be an attempt to improve a team’s ability to delight their customers and users.

Who writes these patterns?  Where do they come from?

In a word, anywhere.  I’ve seen dozens of them on the internet and in books.  I’ve documented a few original ones myself(though many of mine are adapted from others, and I would make every reasonable attempt to give credit where credit is due).  I would not be attempting to create all of these patterns, but to assemble them and label them so Scrum practitioners can easily understand which of them are optional, and which are fulfilling required practices of Scrum.  Of course, it will also become helpful to understand where these patterns fit into the grand scheme of Scrum, and I have some ideas along those lines, too.

I will keep iterating on this concept, and I hope to get back to you with more on this topic in the future.

I am Now a Professional Scrum Trainer with Scrum.org

Hello ScrumCrazy followers,

I want to let you know that I am now a Professional Scrum Trainer with Scrum.org.  I won’t be an employee of Scrum.org, but I am a trainer who is licensed to offer official Scrum.org courses, which lead to Scrum.org Scrum certifications.

What this means:

• I am now one of only about 100 trainers in the U.S. who is certified to train on Scrum!
  • About 25 of those are Scrum.org trainers, and the other 75 are with that other Scrum organization. 
• I was personally trained, interviewed, and approved by Ken Schwaber, the co-creator of Scrum, as well as the current Scrum.org trainer community.
• I will still continue Scrum Coaching as my main engagement, where I plan to spend 90+% of my time.
• I plan to teach a few 2-day courses in 2013
  • I am available for public and private courses

As more details emerge, I will pass them along, but I will work very hard to keep the “sales talk” to a minimum in my blog posts.  While we’re on the topic, I will be available for coaching and consulting again starting in January 2013.  Of course, I am always available to talk about providing training courses.  (See, that wasn’t so painful!)

I am very much looking forward to collaborating with the Scrum community in general, as well as with my fellow trainers, where I’m sure I will learn a lot.

As I said in my post yesterday, this also means that I will begin publishing blog posts regularly again.  I have quite a few queued up that just needing polishing and publishing.  I look forward to blogging to you soon!

Charles Bradley

Founder and Scrum Coach-in-Chief, ScrumCrazy.com

Hello Again! Big Announcement Coming Tomorrow! (Scrum related)

Hello ScrumCrazy followers,

I know.  It’s been quite a while since I’ve posted a blog post.  I have been working very diligently behind the scenes on something quite exciting.  I apologize that I haven’t been posting much lately.

Not to worry, as I will begin posting much more regularly starting just after my big announcement tomorrow.

Tune in tomorrow to see the big news!

Charles Bradley

Founder and Scrum Coach-in-Chief, ScrumCrazy.com

Agile2012 Q&A – Ron Jeffries and Chet Hendrickson on Simplicity

I had the good fortune of being in a Q&A Session at Agile2012 with Ron Jeffries and Chet Hendrickson, early pioneers of Extreme Programming(XP), and both now Certified Scrum Trainers. Ron and Chet were sitting on the “expert couch” taking questions from the audience. I decided to break the ice by going on stage (questioners had to sit in the “therapy” chair next to the couch, and were time-boxed to 10 minutes) and asking them the first question of the Q&A session.

Keep in mind that this is just my recollection of the event. I’ll encourage anyone present to correct or add to what was said from their recollection. I lobbed them a softball of sorts. It wasn’t a hard question, but I wanted to hear their answer as Agile Manifesto authors. A friend in the biz had recently asked this question of me:

How do you interpret this Agile Manifesto principle?

• “Simplicity–the art of maximizing the amount of work not done–is essential.”

I asked Ron and Chet to give some background on what that principle was meant to convey. Ron spoke from a programming perspective mostly. He said that simplicity refers to Kent Beck’s 4 rules of simple design. Kent’s definition stresses:

“…In priority order, the code must:

1. Run all the tests
2. Contain no duplicate code
3. Express all the ideas the author wants to express
4. Minimize classes and methods…”

The above is copied from Ron’s article on Emergent Design. Ron added that simpler is often simpler than you think, and simpler is often smaller than you think. He mentioned the phrase “the simplest thing that could possibly work” as a reminder to keep design and code simple.

Chet then chimed in on the topic of slicing stories smaller. By slicing them smaller, we can de-prioritize smaller stories in a more surgical way, thus avoiding work that is not needed right now, and may be never needed. There are also other huge benefits to smaller stories in that they are easily digestible, and thus less work to understand and implement. Chet stressed that smaller is probably smaller than you think. I then chimed in and mentioned a concept I had read in one of the XP books. If all of the details of your user story won’t fit on a card, then you should use a *smaller* card so that your stories are sized smaller because you’re clearly making stories too large and probably need practice at making them smaller. Chet finished by mentioning that, while simple is simpler and smaller than you think, you should be constantly inspecting and adapting your “simplicity” optimum.

Handling Non Functional Requirements in User Stories and Scrum

Handling non-functional requirements in User Stories can at first seem difficult, but as it turns out, there’s a pretty easy way to handle them.

For performance requirements and many other non functional requirements(NFR’s), one can use constraints and stories. What I usually coach is to create a story to document the NFR and define story tests for it. Then, I suggest adding the story tests as a “constraint.” A constraint is something that all implemented stories(features and functionality) must comply with. If you’re using Scrum, then you’ll want to add something like this to your Definition of Done(DoD): “All stories must comply with all of the story constraints.”

Example

Step 1: Identify and quantify the constraint and put it in terms that your users and business stakeholders will understand.

Story Title: System response time

• Story Test #1: Test that the system responds to all non search requests within 1 second of receiving the request
• Story Test #2: Test that the system responds to all search requests within 10 seconds of receiving the request

Some things to keep in mind:

• If you cannot quantify the story in concrete terms, this should be a bad smell that usually indicates a requirement that is too vague to be implemented. Vague NRF’s have the same problems that vague functional requirements do: It is hard to answer the question “How will I know when this story is correctly done?”
• Be sure not to specify a technical solution or implementation in the story, because stories are about “The What”(“What the user wants”) and they are not about “The How” (“How this is implemented”).
• Plan, estimate, split(if necessary), and implement this story like all other user stories, as part of the Product Backlog(Scrum).

Once this story is complete, the entire system will be in compliance with this particular constraint.

If your constraint is not system-wide or far reaching:

• Just add it as a story test for that story. But again, specify the requirement, not the implementation, in terms the business stakeholders will understand.

The decision to create a constraint or not will rest on whether the constraint should be taken into account in at least several future stories(or system wide). If it will apply to several future stories, then create a constraint. If it won’t apply to several future stories, then just add the NFR as a story test to the stories that it applies to, or create a separate story to comply with the NFR in the small part of the system that requires it.

Step 2: Add the Story Tests to your list of constraints (and to your Definition of Done if you’re doing Scrum)

Publish your list of constraints(and/or DoD) somewhere that is highly visible. Even if you keep your constraints electronically, print them out in large print and post them somewhere on your Scrum board or in your team area.

Constraints

Test that the system responds to all non search requests within 1 second of receiving the request.

Test that the system responds to all search requests within 10 seconds of receiving the request.

Test that the system logs a user out after 10 seconds of inactivity and redirects their browser to the home page.

Test that any update to a person’s payment information(anywhere in the system) is logged to the payment_preferences log, along with the following information: IP Address of logged in person Old preference value, new preference value Date/time of change

Test that any time a person’s credit card number is shown in the application, that only the last 4 digits display.

A note about Story size estimating:
Once a new constraint is added to the system, any stories in the product backlog that will have to comply with this constraint may need re-sizing if there is material time required to comply with the constraint. Said another way, all estimates for future stories will need to take into account the fact that the constraint must be complied with in order to call the story “done.”

If you’re doing Scrum, then add the constraints to your Definition of Done.

Definition of Done

All stories must comply with all of the story constraints<link to constraints page on wiki>.

All code must be peer reviewed within 4 hours of checkin.

If a change is made to the web services interface, the change must be documented on the official web services api wiki page<link to api on wiki>.

All code must have automated testing that is consistent with the “Automated Testing Guidelines”<link to guidelines on wiki>

Any change in of functionality that is visible in the GUI must be at least tested manually(automated tests also acceptable) against the integration environment before making the functionality available for a QA review.

Another note about Story size estimating:
Like I said above for the constraints, the Definition of Done should always be taken into account when sizing user stories. It might help to bring a copy of your DoD to your grooming and planning meetings to remind developers what all is included in their estimates.

Related Articles

• A User Story Checklist
• User Story Basics – The Longer Story
• To see what User Story Traps cause a lot of pain on teams, see User Story Traps
• To see how to make sure your stories are “ready” for implementation, check out Roman Pichler’s article, The Definition of Ready
• To see more info on creating Acceptance Tests, see my Presentation – Story Testing Patterns
• To see how well your team is implementing the User Story practice, see the The Bradley User Story Maturity Model
• To see more information on grooming user stories, see:
  • What does Product Backlog Grooming Look Like?
  • Tips for Effective Backlog Grooming
• Ron Jeffries’ article: Essential XP: Card, Conversation, Confirmation
• The best book on the subject of User Stories, IMO, is Mike Cohn’s User Stories Applied.
• I wrote a skit for a presentationI did at MileHighAgile 2011 that demonstrates a team doing Product Backlog Grooming for a User Story. In the skit, backlog grooming is covered pretty well, with the one exception that estimation was not covered in the skit. You can read download the script of the skit here.

A Visual Diagram of the User Story Life Cycle

This is a diagram I have created to illustrate a potential life cycle for a User Story, as well as how the Agile planning levels relate to the clarity of a User Story at different stages.

The italicized text attempts to answer the question “When does this phase typically happen?”

As in all things Agile, there is no one size fits all solution, so you’d need to tailor this to your organization and your team. My contention is that most User Stories go through all of the phases/levels whether you consciously realize it or not.

You can also download a PDF version of the diagram.