A Response to Mike Cohns Comments on 64% of Software Features Rarely or Never Used

I have a saying.  “Scrum Trainers usually agree on 99% of Scrum, but they spend a lot of time debating the other 1%.”

Let me say this first.  I’m a huge fan of Mike Cohn.  I teach Scrum and Agile classes all over the country at Fortune 50 companies, and it is very rare for a class to go by that I won’t mention at least one of his awesome books on Scrum.  I also recommend him on my list of favorite Agile resources on one of our web sites.  In addition to all of this, I’ve had numerous personal interactions with Mike one on one, and he’s always been extremely nice to me, traded professional practice opinions/advice, and he even offered to let me attend one of his classes at a “trainer courtesy” discount one time.  Great guy!  In summary, I like the guy a lot personally, and I highly respect him professionally.  He’s done a ton for the software and Agile industry, and no one should forget that.

So, with that said, let’s get back to that 1% debate.  🙂

In his recent blog post, Mike reveals some little known details about the oft cited 64% of features that are rarely or never used in software systems.  His information is factual and likely true.  I’m ok with all of that.

What I don’t understand is, why bother broadcasting this?

This is one of the most credible studies available on the subject.  If you think hard about this data for a minute, you’ll realize why it is incredibly difficult to obtain… No company wants to admit that there is a TON of bloat in their software!  But, what percentage of Microsoft Excel/PowerPoint/Word features do you use and benefit from?  What percentage of Rally features do you actually use and benefit from?  Bloat bloat bloat, negative value, negative value, negative value.   In my recent articles on the New New Product Owner, I’ve talked about the need for the New New Product Owner to be a marketplace expert, so that they can maximize the value and profits from software development for their company.

Now, the value equation is way more complicated than “rarely or never used”, but still, I think we all know that there is a TON of negative ROI functionality in any non trivially sized application, and there is a TON of software teams with far too little focus on value and profits.  Anyone who has worked on the front lines of software development knows that.  The oft cited study just helps confirm some of our suspicions.  One of our Agile Metrics consulting services at AgileSoftwareTraining.com is helping to give company leaders even more transparency into how to extract more profits and cost savings out of all of their software development efforts, whether they be internal or external systems.  Give us a call if you’re interested.

What makes that limited study useful as a teaching tool is it gets people to think about value, and think about low value, low ROI features, and realize that value delivery is important, far too important to ignore.

Newer Study confirms the same idea!

Further, Standish again re-iterated this point in 2014, ostensibly from wider data: https://www.standishgroup.com/sample_research_files/Exceeding%20Value_Layout.pdf — “Standish Group research shows that 80% of features and functions have low to no value.”

There are other “studies” cited in our industry that are totally bogus, software leprechauns if you will, and I’m totally against relying on those.  Things like the “Cone of Uncertainty” and the so-called “Weinberg study” on task switching have shown to be totally made up.  However, the Standish Group study is real, with real data, and it is highly credible, even if somewhat limited in its scope.

So, Mike wants us to stop citing the study, or for us to caveat it with “in the weeds” details.  Of course, that will just confuse those new to Scrum and the teaching value would be lost.  And people would focus less on software value and profits.  I don’t think that’s good.  I’m totally open to hearing about a more credible public study, but I’m unaware of one. 

With all due respect to a friendly colleague, and one of the best Scrum trainers on the planet, I think ignoring or caveating the 64% study is bad for the industry.  Let’s just put this in the 1% bucket that we as Scrum trainers will agree to disagree on.  🙂

If you’d like to disagree with my contrarian view, feel free to sound off in the comments below!

Focus on Telling User Stories, NOT Writing User Stories

Ebin Poovathany has written a wonderful article on how we should focus more on the verbal conversation aspects of User Stories rather than focusing too much attention on “writing” User Stories. I myself have written an article about this as well (See Trap #’s 1, 8, 10,and 13). It’s great to see that this topic is starting to get more attention in the industry.

As Ebin points out, using so called “User Story Templates” (“As a user, I want..”, “In order to…I want…”, etc) causes people to backslide into older waterfall habits, and creating the same old kinds of documents that we used to create in waterfall, along with the same old problems. He said this is sad, and as a User Story proponent, I agree. It’s a horrible misunderstanding, but it’s rampant in our industry. The User Story practice was always intended as a very close, verbal collaboration between the Dev Team and the PO/Customer. In modern times, you can achieve this very easily with good Product Backlog Refinement practices.

Anyway, it’s totally worth your five minutes to read Ebin’s article, and be sure to share it with your teams and organizations too!

To learn how to avoid User Story Traps and maximize your User Stories practice, see more info about our User Stories Class.

Scrum and Agile Adoption: Backsliding into Waterfall Habits

It’s very common, when an organization is in the Shu level of learning Agile or Scrum, for it to fall into old, bad, waterfall habits.  Today I’d like to talk about two bad waterfall habits:  tracking so called ‘individual velocity,’ and tracking actual effort expended on a task or story.

First, tracking some sort of metric like “individual velocity” is probably an excellent way to completely sabotage your project/product and it’s also a great way to kill your Agile adoption.  A key concept in the Agile Manifesto principles, as well as in Scrum, is team work and “self-organizing” teams.  Self organization is generally the ability for a team to create and execute their own plan of work(Sprint Backlog), as well as decide “How” to do their work.  Whenever there is a single entity (individual on or off the team, department, etc) who strongly influences or makes unilateral decisions for how a team works, there is, by definition, no self organization.  Tracking individual velocity, or any similar “individual incentive” (this can include raises, performance reviews, awards, etc) does not encourage team work at all.  In his book, Agile Estimating and Planning, Mike Cohn says it this way:

“If I am forced to choose between finishing a story on my own and helping someone else, what incentive does measuring individual velocity give me?  Individuals should be given every incentive possible to work as a team. If the team’s throughput is increased by my helping someone else, that’s what I should do. Team velocity matters; individual velocity doesn’t. It’s not even a metric of passing interest.”

Tracking actual effort expended is another really bad waterfall habit.  Tracking estimates and actuals is just another example of where Common Project Management Metrics Doom IT Departments to Failure.  Mike Cohn, an Agile and Scrum thought leader, and a former project manager himself, says this about tracking estimates and actuals:

“On a project, it is far more useful to know how much remains to be done rather than how much has been done. Further, tracking effort expended and comparing it with estimated effort can lead to ‘evaluation apprehension’ (Sanders 1984). When estimators are apprehensive about providing an estimate, the familiar ‘fight or flight’ instinct kicks in, and estimators rely more on instinct than on analytical thought (Jørgensen 2004).

Tracking effort expended in an effort to improve estimate accuracy is a very fine line. It can work (Lederer and Prasad 1998; Weinberg and Schulman 1974). However, the project manager or whoever is doing the tracking must be very careful to avoid putting significant evaluation pressure on the estimators, as doing so could result in estimates that are worse rather than better.

Additionally, keep in mind that variability is a part of every estimate. No matter how much effort is put into improving estimates, a team will never be able to estimate perfectly. Evidence of this is no further away than your morning commute to work. There is an inherent amount of variability in your commute regardless of how you travel, how far you must go, and where you live. If you drive to work, no amount of driving skill will eliminate this variability.”

In a later post on the ScrumDevelopment Yahoo Group, Mike summarizes it this way:

” I’d say you shouldn’t do it because it doesn’t add value commensurate with its cost. Don’t argue with your bosses that it ‘adds no value’ because
comparing what you originally thought a task would take with what it did take can help make you a better estimator.

But, it can be time-consuming to track actuals, especially for a full team where some on the team are probably already decent estimators.

Because Scrum already has solid mechanisms for monitoring whether all the work gets done in a sprint (high team commitment, daily burndown charts, daily scrum, and so on), Scrum does not have the same reliance on early and accurating estimating that a predictive or waterfall approach does.

So–the cost to gather actuals is the same in Scrum or waterfall. The benefit in Scrum is greatly reduced.”

So, be very careful when backsliding into old waterfall habits.  It usually happens in small doses, which is why many Shu level adoptions don’t notice it, especially if they don’t have a skilled Scrum Coach or highly experienced Scrum Master around.  The other thing to keep in mind is that, even if you do have someone skilled around, this old adage still applies:  “You can lead a horse to water, but you can’t make ’em drink.”

What kind of backsliding into old waterfall habits have you seen?  What do you suggest be done about them?  Sound off in the comments below!

“Kanban vs. Scrum: The Big Smackdown” — Video now on YouTube

A few weeks ago, I participated on a panel discussion at the January meeting of AgileDenver.  Be sure you click through to the link with the meeting description before you view the videos.

Part 1
http://youtu.be/nhfhU0hqyQ4

Part 2
http://youtu.be/sfBILgeUS2w

I hope no one will take this as me having a personal jihad on Kanban, because that is not true.  I do, however, hope to inspire people to stop, and think twice, before incorrectly applying Kanban as a software development approach.  In my earlier post, I talk about why I think that is a mistake.

I also hope no one will take this as me thinking Scrum is always better than Kanban or vice versa.  Context is everything, and I simply suggest that those approaches should be applied to the context for which they were designed.  I think people should also give great deference to the creators of the respective approaches in this regard, so as to avoid what Martin Fowler calls semantic diffusion.  While Fowler calls it semantic diffusion, I have always known it more simply as the telephone game.  Luckily, we have good shepherds around like Anderson, Schwaber, and Sutherland, to remind us of the original ideas.

I also want to reassure my blog followers that I don’t intend to spend a lot more time writing about Kanban vs. Scrum.  I have some new articles that are very Scrum specific coming soon.

Related Articles:

• Kanban vs. Scrum: Kanban is NOT for Software Development, but Scrum is!

• One way to handle Bugs and Production Support in Scrum

• One Way to Handle Mid Sprint Scope Changes in Scrum

Dealing with Hard to Find Bugs (Sprint Killers) in Scrum

This question was asked in an online forum(I’m paraphrasing):

> How do people here handle the impact of difficult errors/bugs (but not legacy bugs) on sprint progress?  Like ones that take weeks to solve?

In my professional opinion, the answer is: we make them transparent and try to improve upon them — at Daily Scrums, Sprint Reviews, and Sprint Retrospectives.

I tend to coach teams to handle bugs in Scrum using the Bradley Bug Chart.

One of the aspects of the Bradley Bug Chart is that bugs like the one mentioned (i.e. non legacy bugs) end up on the Sprint Backlog.  Because they end up on the sprint backlog, if one is using Story points and velocity, no story points are assigned and no velocity is gained from fixing bugs.  This, once again, helps provide transparency on to the lack of progress that the team might be making due to bug fixing.  The truth can be a hard pill to swallow, but the truth will also help set you free from these mistakes in the future.

The transparency should help all involved understand that there is something that needs improving, that is dragging down the team’s ability to produce new features and working software.  I would argue that this is not a sprint killer.  It is simply a fact of complex software development.

The real issue comes down to this:  Scrum transparency is trying to tell your team something.  What is it trying to tell your team?  What is your team going to do about it?

Related Articles

• The Tao of Scrum – Transparency

• One way to Handle Bugs and Production Support in Scrum

• One way to Handle mid-Sprint Scope Changes in Scrum

ScrumCrazy.com update:

• Looking for Agile/Scrum/Kanban Coaching or Training?  Contact us for more info.  We have some good specials going on right now, but they won’t last long!
• Finally, a Scrum certification course aimed at ALL members of the Scrum team! Developers, Testers, Business Analysts, Scrum Masters, Product Owners, etc.  Feb 28th in the Denver Tech Center.  More info and sign up here!

Kanban vs. Scrum: Kanban is NOT for Software Development, but Scrum is!

Last week I was a panelist at the Agile Denver meeting, where the title of the panel was Kanban vs. Scrum! The Big Smackdown!

If you click through the link, you’ll notice that the title was misleading, on purpose, as is the title of this article(“Kanban vs. Scrum…”).  However, I want to catch the attention of those who think that this is even a valid question or consideration.  The title also speaks to a common problem in the industry that has been around for several years, and won’t seem to go away.

There are a number of software teams and organizations that think they should choose between Kanban and Scrum as their software development process.  This is a GIANT and RISKY mistake, in my professional opinion.

It’s not an either/or proposition.  Scrum is about software development.  Kanban is about change management.

There are several reasons why choosing Kanban as your team’s software development process is a mistake.

1.  You are applying Kanban to the incorrect context.

Would you use a hammer to insert a screw in a wall?  You can, but you’ll probably damage your wall in the process, and the same is true of Kanban as a software development approach.  David Anderson, the creator of The Kanban Method, has apparently said this over and over again since 2005, but no one seems to listen.

Don’t take my word for it, listen to David:

“Kanban is NOT a software development life cycle or project management methodology! It is not a way of making software or running projects that make software!” — David Anderson

“There is no kanban process for software development. At least I am not aware of one. I have never published one.”  — David Anderson

“It is actually not possible to develop with only Kanban.  The Kanban Method by itself does not contain practices sufficient to do product development.” — David Anderson*

(*The first two came from the “over and over..” link above.  The last quote was sent to me via email from someone at David’s company.  I think they just pasted in something David had already written)

I should also mention that others have mentioned to me that David talks out of both sides of his mouth about Kanban, Agile, and software development, perhaps trying to capitalize on the fame and success of Agile software development.  That may be true, but it may also be true that David has been saying all of these things for years and no one is paying attention to what he says, which is unfortunate.

2.  Kanban is modeled more after the assembly line and manufacturing.  Scrum is modeled more after creative product design.

Which do you think more closely resembles software development?  Laverne and Shirley on the assembly line at the Shotz Brewery? Or the group of NASA engineers on the ground who saved the lives of the Apollo 13 astronauts by coming up with a creative solution to a problem within a time-box?  If you think software rolls off of an assembly line, then I think that it is unfortunate that you’ve never worked in a creative software development environment — it’s AWESOME!

Maybe my Laverne and Shirley reference is oversimplified.  The reason to use Scrum instead of Kanban for software development delves down into process control theory, and the difference between a “defined process” and an “empirical process.”  In short, a defined process works better when the inputs and outputs to the process are well known and repeatable (like a manufacturing line).  An empirical process works better when the inputs and outputs to the process are less known and very difficult to repeat.  No two software features are alike.  This is why it’s darned near impossible to measure software productivity directly, even though some naive “bean counters” still try to.  Like the stock market, no one metric will predict it accurately, but a range of indicators can help predict it more accurately.  So, in summary, Scrum is based on empirical processes like product design.

One of the very key parts of empirical processes is the characteristic of inspecting and adapting the product.  Think of yourself making a pot of soup from scratch, without a recipe.  Think about all of the “taste-tweak ingredients-taste” experiments(feedback loops) you would need to get a pot of soup that tastes good.

Scrum has the frequent feedback loops built in, for a variety of audiences(Dev Team, Product Owner, Stakeholders, Users) , and for a variety of topics(process-Sprint Retro, product-Ordered Product Backlog, product-Sprint Review, product-Valuable/Releasable Increments).  Kanban has no such built in loops, but again, that’s because it wasn’t designed for software development!

3.  From a Complexity Science view, Kanban is for ‘complicated’ work while Scrum is for “complex” work.

I know the Kanban folks don’t like hearing this, but I think Ken Schwaber was right when he said this, and I think history will prove him right about Kanban as it was described in David Anderson’s book.  In short, the Cynefin model defines 5 domains, of which 2 of them are “complicated” and “complex” work.

‘Complicated’ work is best solved via ‘good practice’ and ‘experts’ who can find ’cause and effect’ fairly easily. When I think of ‘complicated’ work, I think of an the IT support person who sets up your computer or trouble shoots it.  Yes, you need an expert to solve these problems, and the vast majority of the time, the steps to solve these kinds of problems are fairly consistent and repeatable.  They are not exactly repeatable, just mostly repeatable.   If the steps were exactly repeatable then they would fall into the ‘Simple’ domain of Cynefin.

‘Complex’ work is best solved via ‘safe to fail experiments’ and one can only ascertain cause and effect after the fact.  Each Sprint in Scrum is a ‘safe to fail’ experiment because, while the Sprint increment is always releasable, the business side of the house makes the decision on whether it is safe/valuable to release it or not.  In the case of an increment that is un-safe, the team course corrects and comes back with an increment the next sprint that is hopefully safe or more-safe.  These safe to fail experiments can be repeated over and over again until it’s time to release the increment.

Applying Kanban Correctly

Having said all of the above, there IS a time and place for Kanban — a correct context, if you will.  If you’ve been reading closely, that context is as a change management process, which is ‘complicated’ work, and requires that there be already existing processes in place.  So, if your software team is doing XP, Scrum, Crystal, Waterfall, RUP, DSDM, FDD, etc, then you can layer Kanban on top of it to help find bottlenecks and waste.  Also, for all of those teams out there that don’t use a software development process(framework, approach, etc) that is named in the industry, you’re probably doing cowboy coding, ad-hoc, or command and control project management — none of which is a software development process either.  So, layering Kanban on top of crap will still yield crap.

For those that want to apply Kanban at the enterprise level to monitor the flow of work through their Scrum teams (Or XP, Crystal, etc), or want to use it for IT support or Dev Ops, I say have at it and I hope it helps you.  I imagine just visualizing your workflow alone will help in those contexts.  I myself have recommended and coached Kanban for a couple of teams — but only because those teams exhibited the right context for Kanban to be successful.

Bottom Line

Having said all of this, just visualizing your workflow and the other Kanban principles is not enough for software development.  Software development has things like business value, technical complexity, and user experience/acceptance/adoption — all of which are not addressed directly by Kanban.  Scrum does address these areas, as I have shown above.  But hey, let’s not forget, the Kanban Method is “not a way of making software or running projects that make software.”  Would you criticize a hammer for not doing a good job of being able to insert a screw into a wall?

Related Articles

• The Tao of Scrum – Transparency

• Scrum For Lay People

• One way to handle Bugs and Production Support in Scrum

ScrumCrazy.com update:

• Looking for Agile/Scrum/Kanban Coaching or Training?  Contact us for more info.  We have some good specials going on right now, but they won’t last long!
• Finally, a Scrum certification course aimed at ALL members of the Scrum team! Developers, Testers, Business Analysts, Scrum Masters, Product Owners, etc.   Contact us for more info

Agile and Scrum Hiring: Recruiters Mistakenly Looking for 12 Years of Agile Experience

We used to have the same exact challenge in the Java world.  Java came out somewhere in the neighborhood of 1995, and by 1998, recruiters were looking for people with 5 or 10 years of Java experience.  It happened to me often as a young budding Java programmer.

I’ve seen this happen with Agile as well.  I work with a lot of recruiters and partners on a lot of initiatives, and one of them is doing Agile and Scrum “phone screens.”  Before I do a phone screen for a client/recruiter, I ask for the job description so I know what they are looking for.  When I see job requirements that ask for something like “12 years of Agile”  or “7 years Scrum Master experience,” I have a talk with the recruiter.

I usually just politely say something like this to the recruiter:

“Agile really didn’t exist until about 2001, and didn’t really start taking off until about 2008 or later.  Even today, most organizations are relatively new to Agile.  The folks that were involved before 2008 are generally early pioneers that now charge about $200-1000/hr for their services.  So, if you’re looking to hire one of the pioneers, I can certainly refer you to them.  On the other hand, if your client is new to Agile and didn’t realize what they were asking for, I’d be happy to collaborate with you to tweak the job description to what it is they really *are* looking for.”

(Agilistas may want to to quibble with some of the facts or dates in that statement, but it’s fairly accurate, and it gets the point across.  The intended audience is recruiters, not Agilists.  Having said that, I welcome feedback and corrections!)

One other point of note is that there are a lot of people who claim they were doing “Agile” or “Agile Project Management” who are essentially “glossing over” or attempting to deceive about their “X years of experience in Agile.”  I have several ways of figuring this out in my phone screens, but I won’t mention my tactics here as they are trade secrets of ScrumCrazy.com .  🙂  Scrum Master and other Agile job descriptions that require numerous years of experience tend to draw out these fakers even more — and I’m sure that’s not what the clients really want.

So, if you as a job seeker run into this issue, feel free to refer to this article to help explain realistic expectations when looking for good Agile or Scrum people.

For you recruiters out there, feel free to contact me about my phone screen services, but also understand that the person who referred you here is trying very much to help you in your quest to make your clients happy, just as I am.

In the interest of transparency,  I (very mildly) financially benefit from the phone screens that I mentioned, but I don’t want this blog post to come off as a pitch for services.  Having said that, I feel like I’m providing a fairly low cost, high ROI service that helps the entire Agile community as a whole.

A Great Link for Patterns on How to Hold a Great Agile Standup Meeting

I never cease to be amazed by the great work that Martin Fowler has on his web site.  (In all fairness, though, this post seems to be mostly from Jason Yip).

Anyway, GREAT LINK!

“It’s Not Just Standing Up: Patterns for Daily Standup Meetings”

http://martinfowler.com/articles/itsNotJustStandingUp.html

Also, while you’re here, you might want to take a look at one of my older blog posts:

“Bad Smells of the Daily Scrum”

https://scrumcrazy.wordpress.com/2010/09/18/bad-smells-of-the-daily-scrum/

As usual, all feedback welcome.

P.S.  Have you noticed that I’ve been posting a lot more lately?  One reason is that I’ve had more time lately to write, and another reason is that I figured out that WordPress now has a feature that allows me to schedule when the posts are to be delivered, so I can just queue them up! Sweet!

.

One Way to Handle Bugs and Production Support in Scrum

This article has been updated and republished here.

Steve Denning – Rattling the Cages of Command and Control Managers on Forbes.com

Are any of you following Denning’s recent articles on Forbes?  He is “killing it” for Agile.

I especially like the bottom link…

http://www.forbes.com/sites/stevedenning/2011/04/29/scrum-is-a-major-management-discovery/

http://www.forbes.com/sites/stevedenning/2012/04/09/the-best-kept-management-secret-on-the-planet-agile/

http://www.forbes.com/sites/stevedenning/2012/04/11/why-cant-the-c-suite-grasp-agile-management/

http://www.forbes.com/sites/stevedenning/2012/04/17/the-case-against-agile-ten-perennial-management-objections/

Anyone have opinions on Steve’s work?  I’m not familiar with it other than passing references to his book.