Think of waterfall as being similar in concept to the old USSR central planning of the economy. Think of Scrum as similar to a market economy.

To date, the project has consumed 36,000 labor hours — 18 person-years — and $2.5 million.

At this morning’s enterprise-level status meeting, it was decided that Project X will be put on indefinite hold, as it is no longer a strategic priority.

This reminded me a lot of an article I read a few days ago:

What the waterfall does well is to keep useless projects from resulting in useless code that needs to be maintained. I’m not sure if that’s the real purpose, but it’s certainly a great side benefit. It may sound inefficient to pay a lot of engineers to get started on projects, do a bunch of analysis and design, and finally abandon the whole thing when something else becomes a higher priority, but every line of code they don’t write is another line that can’t break!

OK . . . you could make a case that waterfall “worked” here — clearly if, after 18 years of effort, people can’t even define the project, that sounds like a project that has no chance of success and shouldn’t be attempted — but it worked at a cost of $2.5 million.

That doesn’t seem very efficient.

What I find is that if you put the customer, the technical team and other appropriate representatives together for as little as four to eight hours, à la a Sprint Planning Meeting, it should be obvious whether or not anyone understands the problem well enough to go ahead and attempt a software solution.

Thus spoke The Programmer.

There have been a number of criticisms of the standard waterfall model, including

• Problems are not discovered until system testing.
• Requirements must be fixed before the system is designed — requirements evolution makes the development method unstable.
• Design and code work often turn up requirements inconsistencies, missing system components, and unexpected development needs.
• System performance cannot be tested until the system is almost coded; undercapacity may be difficult to correct.

The standard waterfall model is associated with the failure or cancellation of a number of large systems. It can also be very expensive.

We are so unprofessional it is incredible. The legacy of waterfall is so dominant it is scary.

And the Gileadites took the passages of Jordan before the Ephraimites: and it was so, that when those Ephraimites which were escaped said, Let me go over; that the men of Gilead said unto him, Art thou an Ephraimite? If he said, Nay;

Then said they unto him, Say now Shibboleth: and he said Sibboleth: for he could not frame to pronounce it right. Then they took him, and slew him at the passages of Jordan: and there fell at that time of the Ephraimites forty and two thousand.

Thus the original meaning of the word “shibboleth”: a password that people from one side can pronounce but their enemies can’t.

The word has since taken on a more general meaning as not necessarily a password, but a custom or practice that separates the good guys from the bad guys, the insiders from the outsiders.

Shibboleths serve an important role in the IT business. IT is a poorly educated profession. In some professions — medicine, law, nuclear physics — you can count on everyone having basically the same education and a common body of knowledge. In IT, you can’t. I’ve worked with English majors, French majors, people with no college education at all . . . it’s all over the map as far as what people know or don’t know.

And as for continuing education, according to DeMarco and Lister, the average software developer doesn’t own a single book on the subject of his or her work, and hasn’t ever read one.

It wasn’t always like this, but it is now.

So your ability to be happy in IT depends not so much on what you know, but on your ability to adopt the customs and practices — the shibboleths — of the profession.

That’s one reason why change is so painful and slow. A lot of our customs and practices — particularly with regard to the traditional waterfall life cycle — have been discredited about as thoroughly as possible, but we continue to use them.

Why? Because challenging the shibboleths marks you as an outsider. You lose the social and psychological benefits to be had by conforming to the group. Issues of technical merit become secondary to issues of personal identity and moral worth. Most people don’t want to deal with this.

As Nietzsche used to say, “If you want happiness and peace of mind, believe; if you want truth, seek.”

Thus spoke The Programmer.

The hardest single part of building a software system is deciding precisely what to build. . . .

Therefore, the most important function that the software builder performs for the client is the iterative extraction and refinement of the product requirements. For the truth is, the client does not know what he wants. The client usually does not know what questions must be answered, and he has almost never thought of the problem in the detail necessary for specification. . . . Complex software systems are, moreover, things that act, that move, that work. The dynamics of that action are hard to imagine. So in planning any software-design activity, it is necessary to allow for an extensive iteration between the client and the designer as part of the system definition.

I would go a step further and assert that it is really impossible for a client, even working with a software engineer, to specify completely, precisely, and correctly the exact requirements of a modern software product before trying some versions of the product.

 

Much of present-day software-acquisition procedure rests upon the assumption that one can specify a satisfactory system in advance, get bids for its construction, have it built, and install it. I think this assumption is fundamentally wrong, and that many software-acquisition problems spring from that fallacy. Hence, they cannot be fixed without fundamental revision—revision that provides for iterative development and specification of prototypes and products.

 

I have seen most dramatic results since I began urging [incremental development] on the project builders in my Software Engineering Laboratory class. Nothing in the past decade has so radically changed my own practice, or its effectiveness. . . .

The morale effects are startling. Enthusiasm jumps when there is a running system, even a simple one. Efforts redouble when the first picture from a new graphics software system appears on the screen, even if it is only a rectangle. One always has, at every stage in the process, a working system.

— “No Silver Bullet: Essence and Accidents of Software Engineering,” Computer, Vol. 20, No. 4 (April 1987) pp. 10-19.