Geeks With Blogs

L. Charles Andersen


Is it true that Agile doesn’t work?

Before anybody posts my address and a photo of my house on Facebook along with death threats, I don’t believe that Agile doesn’t work. Personally, I believe that it is the current best (much of the time) approach to software development when long-term development speed, responding to changing requirements, and quality are important.

Agile is not really a methodology, but a categorization of several approaches to software development that previously had taken the label, lightweight. Agile practices have been around for at least a couple of decades now, but have only been called Agile since 2001 [1]. We have proponents such as Robert C. “Uncle Bob” Martin asserting that, “The jury is in, the case is closed. TDD works, and works well.” [2] Kent Beck, Martin, and numerous other agile practitioners have demonstrated remarkable success using Agile methodologies.

Even with that, we see many screeds against Agile [3] [4], even to the extent that TDD [5] and Agile [6] are dead, and that most unit tests are waste [7]. Note that TDD is just one part of agile but some use it to condemn the methodology. We hear the same story with pair programming. On top of that, someone has written an Anti-Agile Manifesto [8].

My intent here is identify a commonality in the arguments given by some that Agile does not work. My hope is that it will induce discussion that will perhaps lead to viable solutions. We have to remember that our common goal as professionals is producing quality software that adds value to the business within budget constraints. We also prefer to accomplish this without destroying our personal lives or health. I believe that the problems people attribute to Agile’s “failure” are not attributable to Agile, but to the people themselves.

Analyzing Agile’s purported failures

The course objective of a particular discipline which I am certified to teach is, “To develop the knowledge, skills, and attitude necessary to [perform the discipline].” Knowledge encompasses what to do and why to do it, or the theory; skill covers how and the ability to do it; attitude is the maturity to apply the other two parts consistently and appropriately. Looking at that objective statement we see a three-legged stool; all three parts are necessary, none alone is sufficient to indicate competence in the discipline.

I believe this applies to many, if not all, disciplines including software engineering and Agile practices.

From casual observation of teams with which I have worked and/or managed, and from comments I have heard and read, I have identified some shortcomings with the anti-agile crowd and its complaints that fall into one or more of those three categories:

They lack the knowledge, the theoretical foundation of:

  • their craft (i.e. the fundamentals of software engineering)
  • the history that led us to Agile
  • the problems that Agile attempts to address

They lack the skills to:

  • write code following the fundamentals of software engineering
  • apply appropriate practices for the problem on which they are working

They lack the attitude to:

  • overcome their predisposition to make agile fail
  • embrace change
  • trust the theory and fundamentals of software engineering and Agile
  • recognize (or admit) what they don’t know, then study to improve

Before we get into specific examples, I will describe my perspective on the problem. Agile is not a process and is not a silver bullet (and has never been described as such by its original proponents).  Various Agile proponents have described sets of practices such as XP, Scrum, Crystal (Clear), etc. that have become known as Agile methodologies. When somebody says that Agile is dead or that it doesn’t work, they make a blanket statement against numerous different, albeit similar, methodologies.

Agile describes a set of values that when followed by competent professionals can lead to successful software projects but is also very open to changing requirements. This means that a team does not “do” Agile, a team strives to “be” Agile. This means that an Agile team can respond to changing requirements without sacrificing quality or productivity.

So, then, why do Agile teams fail?

I do not believe that Agile teams fail. I believe that some teams fail and some of those teams purport to be (or are attempting to be) Agile. I hear teams ask, “Are we” (or say, “We are”) “doing agile” (Scrum, XP, etc.). But agile is a set of principles, not a goal. Agile is not something we do, it is something we are. Those questions/statements are looking for some process or practice to identify them as Agile, so by the Agile Manifesto, those teams are not Agile. This is at least a failure of knowledge.

One thing that none of the Agile methodologies has is a way to make a person a software engineer. They have an unspoken, unwritten assumption that the team has competent developers. That is not a shortcoming of Agile; every methodology that I have seen makes the same assumption.  Perhaps Agile makes that incompetence more apparent. This is a failure of the developers' skills.

Others don’t like Agile, perhaps for some emotional reason, so set out to prove it doesn’t work by making it fail. I have seen team members take a passive-aggressive approach and cause a project to fail, then blame it on Agile. Still others don’t understand how to apply the tools so they fail, but blame the tools or practices so by extension, Agile. This is a failure of attitude.

Again, none of these reasons constitutes a failure of Agile. They are all failures of the people.

I understand the human nature that makes some people resistant to change. This is a significant problem in software development that Alistair Cockburn wrote about in 1999 in, “Characterizing people as non-linear, first-order components in software development” [9]. Although the programmers had good tools at their disposal, they refused to use them even though they had sufficient training. That is a problem in attitude.

A colleague once was working on contract for a company that used software as its main avenue to provide its services. Since software drove their business and they needed fast and frequent releases, they were “doing Agile.” One day leading up to a release, he noticed that many of his unit tests had been disabled. He questioned why and was told that his tests were breaking the build. Naturally he asked which requirements or assumptions had changed that caused the tests to fail. The response: none. The tests were just breaking the build. This is certainly a problem in attitude (they were resistant to change their way of coding and fix their code to pass the tests) but might also include a problem in knowledge (did they even realize that the tests failed because their code was wrong?).

James Coplien wrote that most unit tests he sees are waste. Some people try to use his paper as an indictment of unit testing. However, my reading of the paper does not lead me to that conclusion. He writes that many programmers, for various reasons, do not write good tests.

Some people are apparently predisposed to dislike Agile, so they use Jim’s paper, apparently without reading and understanding it, to show that Agile is no good because one of its practices is no good. But as we see, Coplien is not saying that unit testing is no good; he is saying that it is often poorly used, leading to poor results. (As a note, I do not consider his paper a screed against Agile or unit testing.) Citing his paper that way is an attitude problem.

We must remember that unit testing is a tool, and therefore, no better than the programmer using it. As an example, one could use a hammer to drive in a screw. It would work; just not the way one would intend. However, that is neither the fault of the hammer nor the screw.

I encountered a unit test once where the programmer set up an object then called a Boolean method and tested that it returned true. There was no complimentary test that looked for false and nothing in the test showed why that particular configuration of the object should return true. I had to look at the method’s code itself to try to learn what it was supposed to do. There I learned that the method either returned true or threw an exception; it never returned false!

As we might guess, the method was used as a condition in an if-test in the client code (thankfully, it only occurred once in the system). I found a significant else-block to handle the false return, but nowhere did the system catch the exception. This clearly demonstrates incompetence on the programmer’s part but also a problem with knowledge and/or attitude. The programmer might not even have recognized the lack of skill, but clearly did nothing to address the shortcoming. This is a clear example of one thing Coplien was describing; a poorly written unit test. But it also shows a programmer who did not understand the exception mechanism or function returns.

A complaint against Agile that strikes me as interesting is that the various originators were all super-programmers who would succeed with whatever approach they used. A comment like that must have been made by someone who doesn’t know the history of software development. Super-programmers going all the way back to Winston Royce, and probably farther, have been trying to solve the problem of creating reliable software. We wouldn’t spend the time and effort if all it required were super-programmers.

The Commonality is Incompetence

Seeing that last complaint about super-programmers, then looking at the other observations that I have made, I notice among them a commonality. They all indicate a lack of competence at some level in the thee-legged stool. Notice that the three-legged stool is not unique to Agile; it applies to almost everything of consequence that we do. A team that is weak in any of the three areas will probably fail regardless of their approach. It is clearly not an indictment of Agile. Agile, as with any practice, process, or methodology, depends on competent people.

 

References

[1]

M. Fowler, "Writing the Agile Manifesto," [Online]. Available: http://martinfowler.com/articles/agileStory.html.

[2]

R. C. Martin, in The Clean Coder: A Code of Conduct for Professional Programmer, Prentice Hall, 2011.

[3]

A. Singleton, "Seven Things I Hate About Agile," 22 August 2012. [Online]. Available: http://blog.assembla.com/assemblablog/tabid/12618/bid/87899/Seven-Things-I-Hate-About-Agile.aspx.

[4]

S. Denning, "The Case Against Agile: Ten Perennial Management Objections," 17 April 2012. [Online]. Available: http://www.forbes.com/sites/stevedenning/2012/04/17/the-case-against-agile-ten-perennial-management-objections/.

[5]

D. H. Hansson, "TDD is dead. Long live testing.," 23 April 2014. [Online]. Available: http://david.heinemeierhansson.com/2014/tdd-is-dead-long-live-testing.html.

[6]

H. Makabee, "The End of Agile: Death by Over-Simplification," 17 March 2014. [Online]. Available: http://effectivesoftwaredesign.com/2014/03/17/the-end-of-agile-death-by-over-simplification/.

[7]

J. Coplien, "Why Most Unit Testing is Waste," [Online]. Available: http://www.rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf.

[8]

A. A. Manifesto, "Anti Agile Manifesto," February 2014. [Online]. Available: http://antiagilemanifesto.com/.

[9]

A. Cockburn, "Characterizing people as non-linear, first-order components in software development," 21 October 1999. [Online]. Available: http://alistair.cockburn.us/Characterizing+people+as+non-linear,+first-order+components+in+software+development.

 

 

 

Posted on Saturday, September 6, 2014 6:41 AM agile , TDD , Test Driven , Pair Programming | Back to top


Comments on this post: Why Agile Doesn’t Work

# re: Why Agile Doesn’t Work
Requesting Gravatar...
Seems to me like this is the same sort of reasoning that some people use to respond to failures of the "power of prayer," i.e. "you weren't doing it right."

If one has a group of people with the requisites you list: the knowledge, the skills, and the attitude, then, gosh, you're way ahead of the game already, Agile practices or not. If your group is lacking therein, they're probably doomed, Agile practices or not.

So, Agile seems beside the point in these cases. So, forget Agile, just give us a methodology for getting the people with the knowledge, skills, and attitude to get the job done, and filter out those that don't, and we'll go from there.

Well? We're waiting...

In the real world, you'll probably have a group somewhere between these extremes. If your methodology cannot cope with typical circumstances, then your methodology is not terribly useful.
Left by Dennis C Wilkinson on Mar 21, 2015 8:09 AM

# re: Why Agile Doesn’t Work
Requesting Gravatar...
I'm not sure if you're agreeing with me or not, but the point you made in the second paragraph is the point that I am trying to make. It isn't the fault of Agile that those teams fail, it's the team's incompetence, but Agile frequently gets the blame.

Agile does not claim to make up for incompetence, but neither does any other methodology. (Agile is not a methodology; it is a set of practices that enable a team to produce quality software while responding to changing requirements.)

As for finding the right people, that is a difficult task. However, if companies would let us fire people who are incompetent without wasting time justifying the decision, we would be a long way towards achieving that goal. Run the project like a sports team; if you aren't fulfilling your role, you're gone.
Left by Chuck Andersen on Mar 21, 2015 8:21 AM

# re: Why Agile Doesn’t Work
Requesting Gravatar...
Yes, I know that it is often said that "Agile is not a methodology but a set of practices". It seems a rather fine distinction to me, but feel free to substitute the phrase "methodology that adheres to Agile principles and practices" if that clarifies things.

And no, my point is not that teams do fail due to incompetence; that is pretty much a tautology. My primary point is that blaming the team is just as problematic as blaming a methodology, whether it adheres to Agile principles and practices or not. That is, it is essentially non-falsifiable in a general sense, just as is "the power of prayer." "I prayed to be healed of my illness, and I was." Prayer works. "I prayed to be healed of my illness, and I wasn't." You didn't have enough faith, or, God has another plan for you, or whatever. Prayer still works; see previous example. Etc.

My secondary point is that a methodology, or "set of practices," that only applies to the upper end of the Gaussian curve is rather limited (by definition) and should perhaps be modified, or even discarded.

As for being able to fire incompetents without all that time-wasting justification, well, like many things that sound good in theory, in practice, it's a bit more nuanced. Seems like one's workplace could easily devolve into a Stalinistic microcosm. I've worked at a place where "produce or hit the road" was the overt, rather than the normally more-or-less unstated threat, and it wasn't a pleasant, or particularly productive environment.
Left by Dennis C Wilkinson on Mar 26, 2015 1:19 PM

# re: Why Agile Doesn’t Work
Requesting Gravatar...
I think we agree that competence makes a big difference, but I know of no methodology, set of practices, lifecycle, whatever, that will make up for incompetence. If you know of one, please share it with us; we would all benefit from that.

However, neither have I heard of a methodology that claimed its purpose was managing incompetence. Agile advocates certainly have not made that claim.

My point is that Agile frequently gets blamed when projects fail, even though the failure has no relation to Agile. I tried to address some of that in my post.

Agile practices provide many benefits that are absent in a traditional waterfall approach, like the ability to show a working subset of the system at any moment, covered by an automated test suite. This provides the benefit of early feedback from the customer, which gives confidence to both the customer and the developers.

Making the code testable by creating the tests first keeps the architecture decoupled because tightly coupled code is difficult to test.

Having an automated test suite lets the developers make changes without fear of creating a regression error.

How can anybody complain about any of that?
Left by L Charles Andersen on Mar 26, 2015 6:42 PM

# re: Why Agile Doesn’t Work
Requesting Gravatar...
To reiterate my first point: blaming a team is just as problematic as blaming a methodology. By the Symmetric Property of Blame Assignment, blaming a methodology is just as problematic as blaming a team. Given a failure, proponents of one point of view or the other may pick through the ruins and cite examples that support their thesis.
Yes, of course competence makes a difference. But incompetence does exist, and it should be, and sometimes is, successfully managed. Even without the remorseless wielding of the axe. I believe the discipline is known as Personnel Management. While I do not claim to possess a Grand Unified Theory of Software Development, I humbly suggest that an approach that addresses human foibles might be more productive than simply decrying the idiots we must work with and for.
Left by Dennis C Wilkinson on Mar 26, 2015 7:44 PM

# re: Why Agile Doesn’t Work
Requesting Gravatar...
One thing that I have not so far seen mentioned in any articles on Agile is the effect of the quality of the codebase that is used. By this I mean it's design/engineering and implementation. It is rare now to start a project from scratch so most consist of modifications and/or enhancements to existing suites of software.
If the codebase is good (don't bother asking me to define good - just talk to the developers) then the average story estimation time will be accurate and everything should fall in to place.
However, if this is not the case then stories/sprints/deliveries will overrun, the product backlog will not come down at the desired rate, the problem list will grow and generally be ignored, developer morale will plummet and product quality will degrade even further. Engineering is now no longer important, as after all it will be delivered anyway as Managers are under pressure to get it out the door.

So I would suggest if you are thinking of adopting Agile as a solution to your problems first find out the true state of the codebase that will be used. If it is 'good' then Agile may help (but then why do you need it?) but it could result in the codebase degrading as functionality is flung in at an increased rate.
If however it is 'bad' then adopting Agile will not help, you could however implement it (including adding a few people that are 'passionate' about it) and adopt the delusion that it is working.



Left by DC. on May 07, 2015 4:44 AM

# re: Why Agile Doesn’t Work
Requesting Gravatar...
Agile software development is not just a function for software development teams and engineers. To successfully 'be' an agile development team, the team needs to be part of an agile business model echo system. A development team cannot be agile in vacuum.
The organization as a whole needs to adopt agile practices and mindset.
If you are trying to implement agile SDLC in a non agile business model, one of two outcomes are expected.
1) business model embraces agile and you successfully implement agile SDLC
2) business model does NOT embrace agile and you have been setup for failure.
Left by Henry K on May 27, 2015 11:38 AM

# re: Why Agile Doesn’t Work
Requesting Gravatar...
It can work if developers are at the top of their game having learned their trade in non-agile environments. However, it assumes the users know what they want and they rarely do. Indeed, the amount of involvement from users leads to a lack of initiative from developers. Another major issue is it can cause development to grind to a halt if testing and bug fixing take the time of development time. I prefer lean or waterfall.
Left by Zak on Jan 27, 2016 7:08 PM

# re: Why Agile Doesn’t Work
Requesting Gravatar...
Great article. I also don't think agile doesn't work, but in my office this is the opinion I keep hearing every single day - and with growing amount of hatred to boot.
I have a feeling it's become a fad now, that this is a bull**** method and only a fashion, that has no merit in productivity - and while partly this may be true, people are taking this on face value and spreading on without examination.
It's difficult to implement (!) yes, it takes work and time - yes, and it can fail - not all processes are meant to be agile.
But it really has been shown to work for tens of teams. There are some things to look out for ( just read a quick overview of them: http://kanbantool.com/blog/agile-implementation-problems ) - but with a little effort not to get an opinion before you get started - it can be a success. Try.
Left by Jason Rybihaijn on Jul 14, 2016 8:06 AM

Your comment:
 (will show your gravatar)


Copyright © L. Charles Andersen | Powered by: GeeksWithBlogs.net