Requirements Patterns and Antipatterns

Calculating the ROI for Requirements Analysis

Martin L. Shoemaker — Tue, 05 May 2009 00:53:37 GMT

My buddy Josh Holmes has written a very excellent post on the Return on Investment (ROI) for software. I recommend it to anyone who sees software as a business, not just a job or a hobby.

Last week, the always-worth-reading Patrick Greene made a comment that made me start thinking specifically about the ROI for Requirements Analysis. Most teams and most managers know they have a requirements problem; but too many of them say, “But we’re too busy to fix it, so we’ll just start coding.” Or “We’d like to fix it, but we can’t, so we’ll just start coding.” Or “Yeah, but our customers won’t answer our questions, so we’ll just start coding.” Or “Our executives want progress, they want us to just start coding.”

And that leads me to name a common, pernicious antipattern, JSC: Just Start Coding. This is not Agile Development, but it masquerades as Agile. It really means, “We’re gonna be late no matter what we do, and our executives are wondering why we’re not coding already. So we’re gonna Just Start Coding, and call it Agile.”

Which is exactly wrong. Good Agile, real Agile, is very requirements-centric. It’s about surfacing requirements and particularly requirements miscommunications quickly and frequently, so that we stay as close to the right path as possible, and waste as little time as possible on wrong paths. Agile makes short, quick moves, and short, quick corrections.

JSC, on the other hand, just makes quick movement, with no mechanism in place for planning and correction. “We’re on the wrong path? Who cares? At least we’re coding!”

And yes, let’s admit the dark side: we developers often contribute to this mess. Coding is comfortable. Requirements analysis (for many of us) is uncomfortable. It’s really easy to ignore nagging doubts and bury your concerns in lots of fun coding challenges. “Well, it doesn’t make sense to me; but this is what they asked for, so this is what I’ll build.” We abdicate responsibility for the wrong requirements.

But I want to follow in Josh’s footsteps, discussing software as a business, not just a job. And for a business, JSC has a cost. A huge cost. And Requirements Analysis can eliminate much or all of that cost. Maybe a little ROI calculation can help us to win executive support to do the job right.

Some Fundamental Assumptions

The ROI calculations that I’ll make are based on three fundamental assumptions: two from industry research, and one from faith.

Barry Boehm’s COCOMO II research tells us that poor requirements analysis can add 40% to as much as 100% to your schedule.
Many different sources (summarized by Steve McConnell) tell us that the average project is underestimated by roughly 100%.
Those were the research; here’s the faith. I contend that with a relatively minor investment of time compared to the overall schedule, your team can drastically improve your requirements, reducing that 40-100% schedule slip to nearly zero.

You can’t fix the tendency to underestimate (though you can make a good start by reading Steve McConnell); but I believe that most of what you need to fix requirements failures is simply paying attention to requirements early and refusing to ignore the obvious failures. I’ve worked on too many teams where everyone knew the requirements were fatally flawed, yet no one did anything about them. Everyone saw those flaws as “just the way things are around here”. They erected a giant SEP field around the requirements flaws, and they Just Started Coding.

I don’t think it has to be that way. I think that we as developers have the responsibility and the power to fix most of our requirements problems, if only we can persuade our managers to let us – and persuade ourselves to try. I’ve written a whole book (coming soon, I hope) on how we can perform better as Analyst-Developers; but to get management to back us, we have to show them a business reason. The assumptions above are part of that reason.

A Tale of Four Projects

To perform my ROI calculations, I’m going to envision four projects. Well, no, not really. Rather, I’m going to envision four different scenarios for the same project:

Just Start Coding (Typical). In this scenario, the team starts coding from the earliest possible moment. Management has a target date (they’ll call it an Estimate, but McConnell will call that a delusion). The more likely date is twice as far down the road, per Assumption 2 above; and then because the team fails to do adequate requirements analysis, the final schedule is 1.4 times that doubled schedule, for a total of 2.8 times the expected duration.
Just Start Coding (Out-of-Control). This scenario is similar to Just Start Coding (Typical); but the requirements analysis is even worse than usual, adding another 100% to the project. And these are factors, not sums; the final duration is 4 times the expected duration.
Analyzed (Time Bound). In this scenario, we assume enough Requirements Analysis at the start to recognize that the project cannot possibly meet the target date; and management tells us that the date is sacrosanct. We must meet that date, even if we have to cut features to do so. It’s always smarter and less error-prone to cut those features at the start than at the end, so we do so. Then we continue a small fraction of Requirements Analysis throughout the duration of the project, likely trimming more features as we go. We hit the target date with a lean but critical subset of the desired features, and a high-quality implementation of those features. We’ll add the other features next time.
Analyzed (Feature Bound). In this scenario, we assume enough Requirements Analysis at the start to recognize that the project cannot possibly meet the target date; and management tells us that the features are sacrosanct. We must supply those features, even if we have to slip the date to do so. Our final schedule will be 100% longer than the target; but when we finish, we’ll have the complete set of desired features, and a high-quality implementation of those features.

Obviously, some managers or customers will want Time Bound and Feature Bound. It can’t be done. Break the news to them gently.

Oh, all right, I know: some managers will demand Time Bound and Feature Bound. Let’s call this Scenario 5. It’s the Scenario where you do the analysis right up front, realize there’s no way to succeed, realize that management demands you violate the laws of space and time, and get your resume out on CareerBuilder.com so that you can be working somewhere else when the mandatory overtime and the punitive firings and the lawsuits begin. But since there’s no ROI for Scenario 5, we’ll ignore that scenario as we go forward.

More Assumptions

Next we need a few more assumptions (based on my personal experience, believe them as you will) and some parameters. First, the assumptions:

Up-front analysis requires only about 10% of the likely schedule. If your target is 6 months and so your likely schedule is 16.8 months (6 * 2 for estimation error * 1.4 for analysis error), then your up-front analysis time is about 1.68 months.
Because the analysis is work you would have to do to succeed anyway, just pushed to the front of the discussion, it involves the whole team, and yet doesn’t add to the final schedule.
Analysis is a skill most teams don’t have in abundance. You need a few skilled lead analysts to work with the team, roughly 1 lead analyst per 5 team members.
Once the initial analysis is done, you need little bits of analysis throughout the rest of the project. That will involve the lead analysts for about 20% of the total remaining schedule.

If you don’t like those assumptions, you can treat them as parameters and vary them; but these values fit with my experience.

As for the parameters, you really only need two:

How much does an hour of a developer’s time cost you (total)?
How much does an hour of a lead analyst’s time cost you (total)?

And that’s all you need to know. The rest of the possible factors – duration of the project, team size, etc. – all divide out, because everything is calculated as ratios of time to time, dollars to dollars, and so on. ROI is a relative measure.

And to be honest, you really don’t need to know the absolute costs for developers and lead analysts, just the ratio. As long as the ratio remains constant, you can plug in any values you like.

The Results

So given those assumptions and parameters, what’s the ROI?

Are you sitting down?

My parameters have a lead analyst costing 50% more than a developer. I don’t usually find that to be the case, but I wanted to make analyst costs relatively high to skew the ROI down.

And even with that skew, I ended up with these calculations:

An Analyzed (Feature Bound) project has an ROI of 652% when compared to a typical JSC project, and 2,081% when compared to an Out-of-Control JSC project.
An Analyzed (Time Bound) project has an ROI of 1,943% when compared to a typical JSC project, and 3,371% when compared to an Out-of-Control JSC project.

Those are astonishing ROI figures. Unheard of ROI figures.

And of course, those are ideals. Your Requirements Analysis won’t be 100% effective at cutting out lost time. Let’s assume you only cut out half the lost time that’s due to requirements failure. Then you get numbers like these:

An Analyzed (Feature Bound) project has an ROI of 24% when compared to a typical JSC project, and 776% when compared to an Out-of-Control JSC project.
An Analyzed (Time Bound) project has an ROI of 364% when compared to a typical JSC project, and 1,116% when compared to an Out-of-Control JSC project.

Still nothing to sneeze at. And I think you can do a lot better than just half. I know I can.

And although I've only calculated ROI based on labor savings, labor costs are not the only costs associated with requirements failures. Requirements failures increase costs across the board. Besides a longer schedule, you get more overtime, higher shipping costs, higher travel costs, missed commitments, lost opportunity costs, interest payments, performance penalties… even costly litigation.

And there's also a cost in quality. Reworked code has more bugs than original code. The rework is done under tighter deadlines, there’s more pressure, and there’s more overtime. Developers cut corners in response. This results in more bugs; and those bugs lead to more delays and even more schedule pressure. There's a nasty feedback effect here, and it can kill a project. Requirements Analysis can cut that feedback loop.

Conclusion

Again, this is not BDUF. I just spent a week at a client’s site, and spec’ed out their major requirements in that week in more useful detail than they had developed in months. Better Requirements Analysis does not mean bigger; it means more thorough. It means recognizing when you have questions, and making sure you answer them. It means identifying priorities and making choices.

And it means some resistance. It’s not by accident that we have a systemic problem with requirements in our industry. But it’s in our power to change that.

My spreadsheet for this ROI calculation is available if you want it. Leave a comment. If you disagree with any of my assumptions, leave a comment about that, too. I want to make the most clear, justifiable case possible to my stakeholders (and to yours!) to convince them that Defining the problem in a comprehensible, verifiable form is necessary to solving the problem.

Ulterior Motive Lounge Episode 32: Talking with Pilot

Martin L. Shoemaker — Wed, 04 Mar 2009 22:06:48 GMT

Continuing The Project that Time Forgot, a UML case study. (Click images for larger versions.)

On the surface, this Episode may seem almost social. Demented and sad, but social. But if you could join Hacker Girl, reading Geek Girl’s Tablet PC over her shoulder, you might see a different picture:

And you would also see this diagram:

You saw a conversation. Geek Girl hopes that Pilot saw a conversation. But she saw an interview, and a chance to capture and model requirements. Let’s review the notation here and discuss the diagram contents.

The ellipses are the use cases that a driver performs (indicated by the arrows from Driver to the use cases).
Dependencies (dashed arrows) with the <<include>> stereotype indicate that one use case includes another within at least one of its scenarios. So in order to Get Destination, the system must Scan GPS IDs.
The triangle-headed arrows (generalizations) indicate that one use case is a special case of another use case. So Contact Ops via GPS, Get Time from CIS, and Get Weather from CIS each has a special case that uses Voice Command; and each of those special cases is defective.

You might ask: “Why is Maintain Car in this diagram? Aren’t they modeling what the computer systems will do? Does the computer have any role in car maintenance?” And that’s a legitimate question. If the team were modeling the entire business, Maintain Car would certainly be a use case in the model; but why did Geek Girl include it in the system requirements model?

The answer is two-fold:

Pilot mentioned reflashing the engines, which is a computer-driven operation. As indicated in Geek Girl’s notes, it’s not clear at this time whether the Park’s computers are involved in reflashes or not. The reflashes may be done by dedicated tools that have no connection with Park systems; but it may be that the Park maintains records about the reflashes. Just in case, Geek Girl wanted to capture this.
And building on 1: it’s always best to capture whatever a user says, even if you’re not sure it’s relevant. You can always remove it later. The analyst’s first job is to listen and capture. Analysis and filtering comes later. Don’t filter while you’re listening, filter later!

But then you might ask: “Why did Geek Girl stop there? Why didn’t she include use cases like Fly Helicopter and Rebuild Diesel Ferry Engine?” The answer is that she ignored what I just said, and she filtered now. “Filter later” is a good guideline, but you do have to apply some measure of judgment. Certainly “Watch Your Language” doesn’t involve the computer system, so she knew she could ignore that. Fly Helicopter and Rebuild Diesel Ferry Engine are less clear; but there’s certainly a good reason for leaving them off this diagram: it’s a diagram of Driver use cases, and those aren’t Driver functions. Pilot does them, true; but when he does, he’s not acting in the Driver role. Because remember: actors are roles people or things can play, not the people themselves; and one person or thing may play multiple roles.

If you’re still uncomfortable with the distinction between Maintain Car and Rebuild Diesel Ferry Engine, good! Welcome to the world of requirements analysis! There are no hard and fast rules. There are no clear lines, only guidelines. You have to apply judgment; but as a general rule, err on the side of capturing more information, not less. Geek Girl captured a lot of notes on her Tablet PC, while converting the most clearly relevant ones into a diagram. After the interview, Geek Girl will find time to clean up and reorganize the diagrams, and perhaps rethink the contents.

In Episode 28, The UML Guy adopted a convention of indicating defects via notes on diagrams, as in this earlier diagram of Driver Use Cases:

Harlan Ellison once observed that a lot of dumb behavior can be traced back to the phrase, “It seemed like a good idea at the time.” Well, Geek Girl has decided that The UML Guy’s good idea at the time is a bad idea going forward. Not that the notes are a bad practice: they clearly indicate the defect, and diagram readers can easily see what the defect is. If your goal is printable diagrams which stand on their own, this might be a good practice.

But it has some weaknesses:

As we find a lot more defects, the diagrams will get cluttered, and it will get harder and harder to arrange the diagrams legibly.
We have to do extra work to show the defect on additional diagrams.
Printable diagrams aren’t your goal, or shouldn’t be, in my opinion. To quote myself:

The Model Rule

We saw The Model Rule in chapter 1; but it bears repeating. To use UML effectively, you should never be simply drawing pretty pictures; you should always be editing an underlying model, using the pretty pictures as your user interface. Thus, the model should contain more information than is displayed in any one diagram; and information in one diagram should not explicitly contradict information in another diagram. Information that is found in one diagram but not in another should not be considered a contradiction. Rather, this simply indicates that the former diagram displays more detail than the latter. Details may be omitted from a given diagram to make it more comprehensible.

But how do you keep these diagrams consistent with each other? How do you maintain the underlying model? This is where a good modeling tool proves its worth: a good modeling tool will maintain the model as you create and edit your diagrams. If you are not using some sort of modeling tool, this very mechanical burden will fall on you, rather than on the machine. That’s a poor division of labor: brains should do brain work and machines should do mechanical work. If you do the mechanical work, you will do it imprecisely and inefficiently, and you’ll have no time for brain work. I urge you to investigate and use one of the many UML tools available. A list of UML tools can be found in Appendix B.

And building on The Model Rule, defect notes on the diagram don’t let your model and your tool help you in managing the defects. As an example, Enterprise Architect from Sparx Systems allows us to add detailed requirements information to each element of the model. Here’s a sample screen shot:

And then you can create a requirements report that lists the defects, as in this snippet:

Launch Car Information System
Satus: Proposed	Priority:	Difficulty:
Phase: 1.0	Version: 1.0
Defect #1: Startup is delayed.

That’s a lot of power to give up just for the minor benefit of notes on the diagrams. (There’s a lot more power in Enterprise Architect reports than I’ve shown here; and if that’s not enough, EA is an incredibly powerful automation environment. You can build add-ins for it, and also build automation clients that control your EA models. I have one in the works that lets you directly create EA models from Microsoft Word 2007 and Microsoft Office 2007. When I first started using EA, I said, “This is a great UML tool for the price!” Today, I know better: Enterprise Architect is a great UML tool for any price!)

So in place of defect notes on the diagram, Geek Girl prefers to note the defects in the model, as above; and then use a stereotype to show that the use case has a defect. I think that’s a better practice, even if it’s a bit tool-specific. It makes it easier to add the additional use cases that Pilot described in this Episode (omitting the domain objects so we can focus on the use cases and defects). And for me, The Model Rule is more important than printable diagrams, by a long shot.

And as for the last item in Geek Girl’s notes: yes, why is H.G. so hostile? That must wait for an upcoming Episode…

Overlooking the Obvious

Martin L. Shoemaker — Wed, 25 Feb 2009 18:10:41 GMT

I just wanted to rename a Word document while I was working in Word.

You know what comes next. In practically every application out there, I have two choices:

Close the app. Navigate to the file in Windows Explorer and rename it. Double-click it to reopen the file.
Within the app, do a Save as…. Optionally delete the old file name.

In both cases, both file names now appear in the Most Recently Used list, even though the old file may no longer exist. Yawn. We know how this works.

But can you learned this? Didn’t it seem a little odd?

Can you remember the first time you tried to teach this to your non-geek relatives? Can you remember them saying, “That’s dumb, that’s confusing”? And then you said either, “But it’s easy enough” or “That’s just the way it works.”

Why is it just the way it works? Because we geeks have spent decades telling ourselves that this is normal. It’s a tradition going back at least as far as the Unix mv command, which can be used to move a file to a new folder but can also be used to rename a file. It’s the same way of thinking – a design way of thinking, not a requirements way of thinking – that says “If we have Create, and we have Delete, then we have Edit, because we can always just Delete and then Create again.” We geeks have just accepted “Save As and Delete” as normal.

Well, it’s not normal. We’re overlooking the obvious: if so many users need to be taught a complicated way to rename a file, it’s because users need a simple way to rename a file! Except in a few superior designed programs, we don’t have a way, we have a workaround. It’s just that we know this workaround so well, we think of it as normal.

Talk about not designing for the user experience! Somewhere right now, Alan Cooper is shaking his head in disgust…

Well, not for me, not any more! From now on, every file-based app I write will have a Rename command right up there with Save and Open; and when you rename, it will clean up the MRU list as well.

Ironically, you know what’s one of my favorite features in Visual Studio? It’s the way you can rename a file in the Solution Explorer, and then it asks if you want to rename the class to match the file, and it just correctly applies the rename everywhere (except in comments).

This is so easy, so convenient; yet somehow, I’ve been overlooking that convenience when it comes to my users. Have you?

You're Selling Software

Martin L. Shoemaker — Tue, 24 Feb 2009 21:33:23 GMT

Update: Fixed a typo and a calculation error.

Josh Holmes has a great post on Return on Investment (ROI). And by “great”, I mean great even by Josh’s usual standards. He worked hard on this one. I was privileged to review three drafts before he published it; and by draft two, I was saying, “Josh, this one’s a winner. I’m going to reference this one a lot.” So stop reading me, and go read what Josh has to say. I’ll be waiting here when you get back.

OK, you’ve read it. Pretty scary, huh? But the scariest part to me is that even though Josh is right, he shouldn’t be. To quote Paul Simon, these are the days of miracle and wonder. To be specific, these are the days of software. I’m not saying this is some grand new revelation; but businesses and software people forget just how much software changes the world. Today, whatever your business may be, you may find that deep under the hood, you’re selling software. Oh, not everybody, by any means. Pastry chefs are still selling pastries, day care workers are still selling child care, and airlines are still selling transportation. But a lot of people who once sold simple goods and services are now selling software, whether they realize it or not.

Let me draw some examples from projects where I’ve worked.

Material Handling

I’ve worked on a number of different material handling projects for a couple of leading companies in the field. This is a long-standing, well-established industry, concerned with moving packages and baggage and loads from one place to another with minimal human effort. If you’re not in the industry, you’ll likely think of it as “conveyor belts”; but there’s a lot more to it. There are conveyor belts, yes, but also…

Rollers
Sorters
Packers
Stackers
Scanners
Lifters
Diverters
Carts

And lots, lots more. But yes, it all starts with the simple conveyor belts and rollers that move a box from Point A to Point B.

Except nobody wants those any more. Oh, not nobody, but not enough customers to sustain the industry. Rollers and conveyors from Point A to Point B are a simple, long-solved problem.

No, what customers want today is a system to take a box from Point A to Point Wherever The Label (Or RFID Tag) On The Box Tells Us To Send It. Point A to Point B requires some human to look at the box, decide where it’s going, and put it on the right conveyor. What the customers want is for the computer to “just know” where this box is going.

And it’s more than that: they don’t want the box to have a label that says “Send this to Shipping Dock A”. No, they want it to have a label that says, “This box contains Order #1232. Send it to the right destination.” And sometimes they even want the computer to look at the box and say, “Well, this box contains 50 copies of The Dark Knight, and Tucson needs those; but the truck to Tucson is full, and Orlando needs 70 copies, so we’ll send them there instead, and then reduce Orlando’s outstanding count by 20. The truck for Orlando is at Shipping Dock C, so send this box there. Reschedule Tucson’s fulfillment for their next truck.” In other words, what customers want is not a material handling system, but rather a decision system that controls material handling.

And if your customers want a decision system, that means you’re selling software.

This is news to most of the material handling industry. After all, the hardware to move all those boxes around might range in the millions of dollars, often tens of millions or more. It may occupy literally acres of space. Parts of it may weigh in the tons. Compared to that, the software seems like such an insubstantial, insignificant thing. Even the cost is insignificant, though you can bet the project managers will complain that the cost is too much.

But that’s because – as Josh points out – they see the software as a cost center. It’s not. It’s a profit center. Without that software, those acres of steel and motors and conveyors won’t move those boxes anywhere. Worse, with bad software, they’ll move the boxes to the wrong places. And so without quality software, customers aren’t going to pay for the material handling systems.

Does that mean you’re only selling software? Nope. No line of code ever moved a single box by itself. The code has value because of the sorters and diverters, and the sorters and diverters have value because of the code.

But the difference is this: the material handling project managers know that the sorters and diverters and other equipment are profit centers. They know they’re selling those. But all too often, software is charged as an overhead or a cost center, not a profit center. So rather than try to sell more and better software, they try to skimp on the software costs, cutting schedules and budgets. And as McConnell has documented, the result is usually longer schedules, higher costs, and lower quality.

The software managers in the material handling world need to read Josh’s post, understand it, think about it, and think how they can express the value of their work. A representative material handling system might cost the customer $20 million. If the software to run that system costs $500,000 to build, then cost-center accounting sees that as $500,000 of lost profit. Management will wonder why it cost so much, and wonder how they can cut those costs. What may follow is pressure and effort to cut costs in ways that make software development take longer and cost more.

But if the software managers can see their group as a profit center and can explain it as such to the executives, then the whole picture changes. When they understand that better software leads to more sales or easier sales, executives will want to invest in better software development tools and practices. Josh gives some good starting points toward that culture change. I would add these observations:

The ROI of software starts with Requirements. Yes, that has become a recurring theme for me. There are good reasons for that. With a foundation of well-defined requirements and a process for accommodating change, software managers can show exactly how much of the customer benefit is directly dependent upon the software. If the system has 100 required functions and 90 of them require custom software implementation, then 90% of the profit of the system is directly dependent upon the software. That 90% is also dependent upon properly functioning hardware as well, so you can’t claim all of that profit as your accomplishment. In the material handling world, I saw a team of nearly 100 assemblers, machinists, construction workers, engineers, and other hardware folks install a system controlled by software written by 10 developers. I think that’s a good rough approximation: software provides 10% of the value of the systems with which it works. (In office systems and other systems not involving device controls, I would say that ratio is more like 50%, maybe higher. This Tablet PC would be useless to me without Windows XP, Journal, OneNote, MS Office, and Visual Studio, which collectively cost more than the Tablet PC.) So in our hypothetical example, the system has a value of $20 million; software is involved in 90% of that value, or $18 million; and software accounts for 10% of the value of those components, or $1.8 million. If the profit margin of the system overall is 50%, then you can argue that the software contributed to $900,000 in profit. These are back-of-the-envelope calculations. A more accurate estimate would require time spent looking at the value of each individual requirement and the percentage of that value contributed by software. But in our hypothetical example, $500,000 of software development costs yielded $900,000 in profit, for an ROI of 80%.
Hammer home the destructive effects of schedule compression, as described by McConnell. Schedule compression beyond about 20% leads to longer, more expensive schedules with more errors and more dissatisfied customers. If the potential ROI is 80%, the actual ROI will be a lot lower if the schedule compression increases the schedule by 50% and the costs by even more. McConnell explains how this happens, and why it’s almost inevitable.
There is a way to cut schedules effectively and productively: develop a true software engineering culture and a culture of quality. There is a better way to build software and systems, but “code faster!” isn’t it. Train your teams to be better developers. Improve your requirements process. (No, I’m not done with that theme yet.) Emphasize defect reduction, not schedule reduction, because defect reduction is one of the most effective ways to reduce the schedule.
Invest for the long haul. Near the end of my first material handling job, the team worked on building a good, well-engineered, reusable and configurable platform that could address a wide range of customer needs. This platform didn’t directly address the needs of any existing customer, so it didn’t receive a lot of management support. Again, this was cost-center thinking: “Why are those programmers wasting money on code we don’t even need?” My second material handling job, nearly a decade later, was with a company which had faced those same issues, asked those same questions, and decided to invest in the software platform anyway. It had some infamous rough spots in its earliest implementations. But by the time I worked there, that baggage handling software was nearly a decade old, and it just worked. Oh, there was always some amount of custom coding for every project; but for the most part, if they knew how many airport docks and how many baggage inducts and baggage claims they had to support, they knew to a high precision how much the software costs would be. And most of those costs were configuration costs. I often said they had Moore’s Law on their side. Moore’s Law says that computing power/speed roughly doubles roughly every 18 months. That means that after a decade of life for that software, server power had increased roughly 128 times; but while airports get larger over time, they don’t grow 128 times in 10 years! The software architecture that was a good solution in 1997 was a great solution in 2007, without any significant upgrade to the software.
Don’t cling to the long haul. As much as I think the baggage handling company was smart to reuse a successful architecture, you must always consider the competitive advantages offered by new technologies. When the baggage handling company did have to make code changes, their reliance on old tools and technologies made every change more expensive. Periodically reassess the value of an existing investment vs. the potential gains of a new investment.

Vehicle Diagnostic Systems

My most recent contract was a project for building vehicle diagnostic systems. One part of the team built the tool that connected to the vehicle bus and communicated with vehicle systems; the other part built the tools that talked to that tool via Bluetooth and USB, and then interacted with the user and with online systems.

Now with a description like that, you might guess that the company knew they were selling software. And you would be right. Oh, it was a mixed hardware-software product, but they were much more clear on the importance of good software than I found in the material handling world.

And yet – while I mean no offense to a great team and some great people that I’m already missing – the organization didn’t approach this project as a software project. They approached it as a manufacturing project.

A typical mass production manufacturing effort boils down to finding and implementing tools and processes to do the same thing over and over, so that you can sell and serve as many customers as possible for as little cost as possible. The rules are different for custom manufacturing; but for mass production, profit lies in reducing the number and costs of the repetitive parts of the production process.

Let’s say you’re building mailboxes. A traditional home mail box consists of a base, a shell, an end cap, a door, a door pull, a door catch, a flag, a flag stop, two swivels for the door, a swivel for the flag, and 21 rivets:

Every mailbox, according to this design, requires a person or machine to put the parts in place and insert the rivets and swivels. If every rivet and swivel can be inserted in one second and it takes ten seconds to align all the parts, then a single worker can assemble a single mailbox in roughly 34 seconds. If instead the shell, end cap, door catch, and base can be injection molded as a single piece complete with a flag stop and indents where the flag and door attach, and the door and pull can be injection molded as another single piece, then that same worker can assemble a single mailbox in roughly 3 seconds.

But that throughput assumes the worker has materials always ready to hand. Suddenly the process is limited not by the worker’s assembly rate, but by other factors: how quickly can you deliver parts to the worker, and how fast can the worker really work without slowing down due to fatigue? Those questions apply to the riveted design as well, but less so: no matter how fast you deliver the parts for the first design, the riveting is the bottleneck.

Once the parts availability becomes the bottleneck, the industrial process engineer changes focus: instead of trying to improve the design of the mailbox, the engineer must improve the design of the process that delivers parts to the worker. Or perhaps the engineer must design an improved assembly process, maybe improved machinery and tools, that will make less work per mailbox. The company makes money on each mailbox built, not on each step performed by the workers. Whatever your cost and effort is per mailbox, that’s a 1-for-1 cost: one unit of labor per one mailbox sold. If you can reduce that unit of labor, you can increase overall profitability and employ more mailbox workers.

So it’s common for industrial process engineers to invest a large amount of effort designing a more efficient manufacturing process. This effort can be seemingly large; but it’s is a one-time effort. Once it’s done well once, it’s done well for all of the mailboxes produced. So the cost of that process design is amortized over the tens or hundreds of thousands of mailboxes produced, and it reduces the overall cost of production for each mailbox.

This is Industrial Process 101; but here’s where the mistake comes in when you see software development as akin to manufacturing. Software development is like manufacturing, in a sense; but it’s like process design, not piece manufacturing!

The software equivalent of the worker assembling mailboxes over and over all day long is users applying the software over and over all day long. The value to the user (and the user’s organization) increases when the user can get more done with less work, and so get more tasks done in a single day. Just as the industrial process engineer creates value by improving the process and reducing the manufacturing cost, so too does the software team create value by improving the software and reducing the usage cost.

But this means that traditional manufacturing process improvement approaches can be aimed at the wrong targets when applied to software development. In manufacturing, it can be cost effective to spend more money on design process improvement to reduce the money spent on manufacturing; but when traditional manufacturing process improvement approaches are applied to software, often the goal is to spend less money on software development. Yet somehow, we expect those “improvements” to improve the quality of the finished product.

Now I’m not saying there’s no room to cut costs in software development. There are inefficient ways to design industrial processes: if your design team relearned basic facts every time they approached a new production line, or they ignored proven practices from other production lines, then they would waste a lot of time and money on lessons learned and relearned. The same is true for software engineering. Indeed, there’s an entire field of study, Lean Software Development, aimed at improving the development process in a similar fashion to the field of Lean Manufacturing.

But simply applying Lean Manufacturing practices to software development can be misleading – even counterproductive – because Lean Manufacturing aims (in part) to reduce the time required for repetitive processes; and in software development, there are no repetitive processes. Oh, that’s an exaggeration, of course, but it’s close enough to true. Industrial process engineers have repetitive processes, such as reading email, filing reports, printing blueprints, etc. In a similar fashion, software developers have repetitive processes, such as reading email, filing reports, compiling code, etc. But in both disciplines, the key work, the important and most time-consuming work, is brain work: thinking about the problems, proposing ideas, presenting ideas, reviewing ideas, revising ideas, etc. And this is work that virtually never repeats! It will repeat if you keep reinventing the wheel; but a good team should know standard patterns and solutions for common problems, so they can concentrate their efforts on the unique aspects of a new project. The truly repetitive processes – checking email, compiling code – are more amenable to fiscal solutions than process solutions. Buy your developers faster computers and better tools: it’s the closest equivalent to getting the mailbox worker a faster assembly tool.

And I’m going to repeat myself yet again: the most productive way to reduce the cost of software development is to improve your requirements process, followed closely by creating a true software engineering culture and a culture of quality. Those are goals of Lean Software Development and are consistent with Lean Manufacturing. Interestingly enough, they involve creating quasi-repetitive processes: standard processes for doing standard categories of work, while recognizing that ideally no work should ever be repeated. A standard way of testing code, a standard way of reviewing code, a standard way of communicating designs: these can add value if done right. However, they can also become straitjackets that prevent a team from solving a problem. I have a lot more to say on the value of quasi-standard quasi-repetitive processes; but that’s starting to veer pretty far from our topic today.

And that topic, you’ll recall, is ROI. It’s easy to figure the ROI of industrial process engineering:

ROI = (New net profit per part – Old net profit per part) * Number of parts produced / (Cost to design the new process + Initial cost to implement the new process) – 100%

This gets slightly more complex when you factor in time: Number of parts produced in how much time? In early usage, you’ll almost always see a negative return, because (as Josh mentions) there is a certain length of time it takes to recoup the initial investment. Businesses will usually set a time horizon in which to estimate ROI: 1 year, 3 years, maybe longer.

A similar calculation applies to the ROI for better software:

ROI = (New net value of work per user – Old net value of work per user) * Number of users / (Cost to develop the new software + Initial cost to deploy the new software) – 100%

But there’s a complication here. In the typical industrial situation, the same company is both designing the new process and manufacturing the parts. The ROI value is more clear-cut that way, because what matters most is the overall bottom line for the company. The same is true for in-house software development; but when you’re selling custom or shrinkwrap software to external customers, the value of the work and the deployment cost are measured by the customer, while the cost to develop the software are measured by your company. That’s a complication, but more of a deal negotiation concern than an ROI concern. The calculation in this case turns into two calculations:

Customer’s ROI = (New net value of work per user – Old net value of work per user) * Number of users / (Cost to purchase the new software + Initial cost to deploy the new software) – 100%

Your ROI = Cost paid by Customer / (Cost to develop the new software + Initial cost to deliver the new software) – 100%

The Customer’s ROI, along with many other factors, will determine what cost they’re willing to pay; and then that value will determine your ROI. You can make educated guesses of what they’re willing to pay, and then use those in estimating your ROI. And remember: just as with industrial process engineering, the initial return will almost always be negative. You have to prepare your arguments for that moment, when it seems like the system cost a lot and is too difficult to use. Is it really too difficult? Or is it just new and unfamiliar? Unless you’re very good at defining and communicating your requirements, users will always see “new” as “difficult”, and then cling to old and familiar; and in the process, they forget that the old was once new and unfamiliar, too. That’s why these ROI calculations include a deployment cost: that’s not just a cost for physically shipping and installing the code, but also for training the users and answering their questions and correcting their early mistakes. And lest we forget: finding and correcting our early bugs, and deploying corrections for those as well.

Conclusion (For Now)

Well, this has gone a lot further afield and gotten a lot wordier than I originally expected. (Regular readers are probably less surprised than I am.) I could go further, discussing ROI for training, ROI for technology upgrades, ROI for requirements analysis, ROI for architecture, ROI for community involvement, and ROI for employee retention. But each of those is a lengthy essay in its own right, and I’ll need some research before I’m ready to tackle some of those.

But I hope I’ve added some personal experience to Josh’s excellent post. Many projects can be technological successes, yet be seen as costly, not as a valuable investment. The difference: do we calculate and demonstrate an ROI? Or do we let stakeholders see only the cost of the software, without seeing the full return on the software?

The Analyst's Chorus

Martin L. Shoemaker — Fri, 20 Feb 2009 22:00:19 GMT

I am the mirror, reflecting what you say to me.

I am the mirror, I’ll show you where you want to be.

Talk to the mirror, I’ll tell you what you really said.

Look in the mirror, I’ll draw the picture in your head.

There in the mirror: it’s what you didn’t know you knew.

I am the mirror, and I’m listening… to you.

Requirements Patterns and Antipatterns: Checklists

Martin L. Shoemaker — Sun, 01 Feb 2009 19:43:50 GMT

There’s an old joke about an engineer called out of retirement by a company in trouble: their most vital machine has stopped working, and nothing they have tried will get it started again. The old engineer looks at the machine, studies it for a few minutes from different angles, and flips a switch. Presto! The machine starts to work, humming along like it never had a problem. The engineer hands the owner a bill for $10,000; and the owner says, “For flipping a switch? That’s outrageous! I’m not paying until I see an itemized bill.” So the engineer takes back the bill and writes on the back, “Flipping the switch: $1. Knowing which switch to flip: $9,999.”

Unfortunately, I find that a lot of patterns work is like that: if you know which pattern to apply, the work is (relatively) easy; but if you don’t know the patterns, you don’t know when to apply them. Like the company owner in the joke, there’s a switch out there, but you don’t know to flip it.

Well, the best way to learn patterns (like most things) is by experience. I can’t give you experience in a blog, but I can give you Diagnostic checklists that summarize experience. Patterns may also have Verification checklists, and Antipatterns may have Correction checklists. Some Patterns may have multiple Verification checklists. For example, Pattern 10. Brainstorming session has three checklists: one for preparing a Brainstorming session, a second for assessing the success of the session, and a third for summarizing the results of the session.

N/A: Your Defense Against Obsession

As I’ll discuss under Antipattern 26. Obsessive Compulsive Template Disorder (OCTD), it’s easy for bureaucracies to turn templates and checklists into must-do tools for “managing” a process, even when those templates or checklists don’t apply. This is a form of what Steve McConnell describes as “cargo cult software engineering”: going through the motions without understanding why, and without actually getting the benefits (Professional Software Development: Shorter Schedules, Higher Quality Products, More Successful Projects, Enhanced Careers, pp. 23-28.). These checklists aren’t goals in themselves; they’re tools to help you in your goal of more complete and useful requirements analysis.

The best defense against OCTD is N/A: Not Applicable. Whenever a line item in these checklists doesn’t apply to your project, mark it N/A. Then give your project 0 points or full points for that item, whichever you think is best.

Of course, it’s easy to take this too far, and mark everything as N/A. Perfection! You get 100%, right? Wrong. So when you do mark an item as N/A, also explain why it doesn’t apply to your project. If you can’t explain why, you can’t claim N/A.

Scoring the Checklists

Each checklist is scored on a 0 to 100 scale.

Diagnostic Checklists

Diagnostic Checklists are scored as follows:

0-20 Insignificant. This Pattern won’t help you much, or you have little risk of this Antipattern. Only worry about this Pattern or Antipattern if your project is mission-critical. Monetarily, this will cost you more than you’ll gain; but you could still mitigate non-monetary risks.

21-50 Minor. This Pattern may help you, or you have some risk of this Antipattern; but you probably have higher priority Patterns and Antipatterns to address. Worry about this Pattern or Antipattern if your organization is highly risk averse. Monetarily, this may cost you more than you’ll gain; but you could still minimize risks.

51-80 Significant. This Pattern will help you, or you have significant risk of this Antipattern. If you’re not addressing this Pattern or Antipattern in some fashion, that should be an item on your risk assessment.

81-90 Major. This Pattern should be a primary element of your requirements process, or this Antipattern is a primary risk for your project. If you’re not addressing this Pattern or Antipattern in some fashion, that should be a top item on your risk assessment.

91-100 Critical. This Pattern will be a critical element in the success or failure of your project; or this Antipattern is a critical risk for your project. If you’re not addressing this Pattern or Antipattern in some fashion, that should be an immediate item on your risk assessment.

Verification and Correction Checklists

Verification and Correction checklists are scored as follows:

0-20 Misleading. What you think you know regarding this Pattern or Antipattern is more wrong than right. If you proceed based on this score, expect to throw away and rework as much as 100% of your work.

21-50 High Risk. You’re starting to learn about this Pattern or Antipattern; but much of what you know is incomplete or incorrect. If you proceed based on this score, expect to throw away and rework as much as 50% of your work.

51-80 Agile. Agile processes are based on a conviction that many requirements can only be known through delivering and assessing code. Agile proponents point to examples and experience that show that requirements identified too completely in advance can be time-consuming and misleading. So an Agile team should aim for a middle-ground of knowledge: complete enough to lead to code, not so complete that they never get answers.

81-95 Optimal. Your knowledge regarding this Pattern or Antipattern is good enough for most teams to proceed. As Steve McConnell has documented (Professional Software Development: Shorter Schedules, Higher Quality Products, More Successful Projects, Enhanced Careers, pp. 15-16.), teams that focus on quality and defect reduction actually reduce costs and schedule by doing so, up to a point. After that point, even better knowledge costs more money and schedule, and is only appropriate for mission-critical projects. Most teams should proceed based on optimal knowledge.

96-100 Minimal Risk. Your knowledge regarding this Pattern or Antipattern is good enough for mission critical teams to proceed.

Point Lists

Point Lists apply to your entire requirements process. Each checked item scores a certain number of points. Check each item that applies, and sum the point points to get a score from 0 to 100.

Sample Point List: Feeding the Dog

□ 50 You have dog food; or you have table scraps.

□ 30 You have the dog.

□ 20 The dog is hungry.

____ Total

In some cases, the list potentially sums to greater than 100; but 100 is the maximum score in any case. For example, the Verification Checklist for Pattern 6, Trained Analysts, provides 100 points for dedicated analysts and 50 points for developers trained in analysis techniques; but a team with both still scores only 100 points.

Percentage Lists

Percentage Lists apply to your set of requirements, or perhaps to a set of one kind of requirement. Each requirement item is scored a 1 if it meets the criteria, or a 0 if it doesn’t (no partial credit allowed). Your score is the percentage of requirement items that scored 1 vs. the total number of items.

Sample Percentage List: Feeding the Dogs

%__ Percentage of all dogs which have been fed.

Partial Percentage Lists

Partial Percentage Lists are Percentage Lists that allow for partial points. Each requirement item receives some number of points based on various questions; and then you determine the percentages that meet different point scores. Full points count for full percentage; but the percentage of partial points are divided to yield a lower score. The final score is the sum of these adjusted percentages.

Sample Partial Percentage List: Caring for the Dogs

□ 1 The dog is fed.

□ 1 The dog is groomed.

□ 1 The dog is exercised.

%__ Percentage of dogs which score 3 out of 3.

%__/2 Percentage of dogs which score 2 out of 3.

%____ Total

Pattern 19. Requirements Archaeology

Martin L. Shoemaker — Sun, 01 Feb 2009 19:13:36 GMT

Diagnostic Checklist: Requirements Archaeology

□ 70 Your project is to supplement, upgrade, or replace an existing system.

□ 30 The existing system is poorly documented, or the documentation is out of date.

____ Total

You can learn how to score the checklists here.

Problem:

The project is to supplement, upgrade, or replace an existing system, which is the de facto primary source of your requirements.

Context:

In many organizations, new work has to be done in a context of legacy systems, which embody a wide range of explicit and implicit requirements that customers may overlook.

Forces:

Legacy systems represent an investment of capital and time that must be conserved.
Legacy systems may not be well documented or well understood by the current customer.
The team that built the legacy system may no longer work in this organization.

Legacy systems represent an investment of capital and time that must be conserved.

Solution:

Reverse engineer models (see the Modeling pattern) and specs for the current system, and review those to identify requirements. Then analyze and organize those “uncovered” requirements, and present your results back to your customer as a form of The Echo Effect. Be ready for your customer to be surprised, as you reveal features they’ve long forgotten, never known about, or just taken for granted. Listen for them to identify legacy features that they don’t need replicated or supported, and then explore those features to be sure there’s nothing there that other features depend upon.

UML tools that will reverse engineer UML structure include ArgoUML, Enterprise Architect, IBM Rational UML tools, and MacA&D, among others. In their preview of the 2010 version of Visual Studio Team System, Microsoft has demonstrated powerful UML tools, including the ability to reverse engineer sequence diagrams from source code. This is a major advance in general purpose reverse engineering, turning what used to be hours of manual work into minutes.

You may be able to rely on mechanical tools to do much of this reverse engineering. Many UML tools, for example, will reverse engineer models from source or from compiled code.[1] Other tools will reverse engineer and graphically depict database schemas, network layouts, and other elements of existing systems.

But you need to go beyond such mechanical reverse engineering for two reasons.

It’s far too coarse, as our archaeology metaphor shows. Imagine the look on an archaeologist’s face if a contractor showed up at a dig site and said, “Hey, I’ll dig up this whole excavation for you. Let me get my bulldozer.” While mechanical reverse engineering isn’t destructive like the bulldozer, it’s not much more discriminating. The bulldozer piles up a great big heap of earth and stone and rubble, while the mechanical tools pile up a great big heap of classes, relations, tables, columns, nodes, and other artifacts. If you don’t sift through the rubble, you really haven’t learned anything.
In my UML classes, I teach that UML isn’t about software design, but rather about system design, where a system is structure with behavior and goals. Well, I’m missing something in that definition: a system is structure with behavior that fulfills some intention. A good mechanical reverse engineering tool should produce a structural model of a legacy system. Modern tools shed a little light on the behavior model of a system. But only a human reader can produce “intentional” models. A mechanical model captures Functional Requirements at best; but an intentional model captures the User Requirements, Business Requirements, Non-Functional Requirements, Constraints, Actors, and Domain Objects that are represented by the Functional Requirements. (See the Categorization pattern for a discussion of these types of requirements.)

So certainly start with mechanical reverse engineering, as much as your tools will allow; but then dig into the resulting models and organize them (a form of The Outline Effect) to learn what’s in the models. From there, you can create behavioral and intentional models by looking at mechanical models, the user manuals, and the system in operation and then deducing how these fit together.

Resulting Context:

The reverse engineered requirements should serve as a baseline for or input into the requirements for the new system or extensions to the existing system. From there, turn to other patterns as a way to elicit and analyze new requirements.

Certainly start with mechanical reverse engineering, as much as your tools will allow; but then dig into the resulting models and organize them (a form of The Outline Effect) to learn what’s in the models.

There must be new requirements, right? You’re not just replacing the existing system for something to do, right? There should be some reason for replacing it: porting to a new platform, better maintainability, better extensibility, or something. If you’re just suffering from replacement-itis, you should read Joel Spolsky’s essay on rewriting.He makes a good argument for preserving the hard-earned knowledge embodied in legacy systems, and the consequences of replacing working code without a business case.

Discussion:

See also Modeling, The Outline Effect, and The Echo Effect for patterns that work well with Requirements Archaeology.

Verification Checklist: Requirements Archaeology

For each pre-existing system or component:

□ 1 You have a mechanically reverse-engineered structural model of the component.

□ 2 You have a manually reverse-engineered intentional model of the component.

□ 1 You have a presented one or both models to the Customer and received useful feedback.

%__ Percentage of requirements which score 4.

%__/2 Percentage of requirements which score 3.

%__/4 Percentage of requirements which score 2.

%____ Total

You can learn how to score the checklists here.

Ulterior Motive Lounge Episode 31: The Teams Split Up

Martin L. Shoemaker — Sat, 31 Jan 2009 06:59:22 GMT

Continuing The Project That Time Forgot, a UML case study in comic strip form... (Click pictures for larger images.)

After a long hiatus due to weather, illness, work, conferences, and more stuff than I can explain, the Lounge is back.

This Episode gets the ball rolling for Act II, so there’s not much new UML content here yet. But I can give you a few diagrams of the team’s review process. The process starts with some preliminaries, then splits into three threads of operations, each with a separate team:

Design Review. This team, led by The UML Guy, will create a Design Assessment, summarizing the state of the design and any clear risks and concerns. The details of the Design Review look like this:

Review the original design spec to determine the intent of the original team.
Review design documents and updates since the original spec.
Reverse engineer a design from the code. A good UML tool like Enterprise Architect makes this a lot easier, but not easy. VSTS 2010 will make it even easier, but still not easy. Incorporate this into a design summary.
Review the design with the dev team and revise until it’s ready for presentation.
Review the design with the full review team and revise until it’s ready for assessment.
Create the design assessment.

Functional Review. This team, led by Geek Girl, will create a Functional Assessment: a description of how well the system performs existing functions, with emphasis on usability and correctness. The details of the Functional Review process look like this:

Review the existing requirements docs.
Review outstanding defect reports, and make those the beginning of a functional assessment.
For each function identified in the requirements, identify actors and user who represent those actors. Interview those users to determine how well the system really works, and update the functional assessment.

Requirements Review. This team, led by The Reader, will create a Requirements Assessment: a description of what requirements the system covers; and more important, what it doesn’t cover. The details of the Requirements Review process look like this:

Review the existing requirements docs.
Assess the quality of the existing requirements. Are they quantified? Are they testable? Are they unambiguously stated in words, diagrams, and tests? Are they assigned business values and priorities? Do they have clearly identified actors and use cases and scenarios and goals for each feature? These and many more questions will help to create the requirements assessment.
For each actor represented in the requirements or discovered during interviews, identify and interview representative users. Determine whether they have missing requirements, as well as whether they know of or require previously unidentified actors.

For a really thorough review, you probably need at least three more threads of activity:

Code Review to inspect the code, either completely or through sampling, to identify how well it’s constructed and maintained.
Quality Review to identify Quality Assurance measures, Testing measures, and the status of each.
Process Review to analyze the processes followed by the dev team and related teams.

But for simplicity (and to keep most of my focus on my primary message), I’m going to restrict this case study to Design, Functionality, and Requirements. Code will be reviewed under Design, and Quality will be reviewed under Functionality. Process will be reviewed everywhere.

Is this a lot of work? Yep. But it’s a minimal necessary set of activities to review a project for status and risks after you’ve ignored them for a long time. It would be so much easier if you could do these sorts of assessments in real time right from the start as the project proceeds. It would be so much easier if you could report the status and health of the project at the push of a button, based upon the actual work the team does as they do it. Gee, wouldn’t Owner’s life be a lot easier if there were tools to do all that?

Oh, wait a minute… There are tools like that! In fact, there’s a whole category of such tools: Application Lifecycle Management (ALM). And my current favorite is the aforementioned Visual Studio Team System, perhaps the most poorly named product Microsoft produces today. Why poorly named? Because Visual Studio is a developer tool set, particularly a tool set for .NET developers; so when people hear a name like “Visual Studio Team System”, they assume it’s a developer tool set, only bigger and better. They assume it’s about .NET development.

VSTS is not about .NET development. Oh, VSTS includes Visual Studio, and then adds some powerful developer and architect and DBA and tester features; but those features, as powerful as they are, are just the icing on the cake.

The cake is ALM. VSTS is about your process and how you can define it and design it and manage it and track it, all through tools your team – your whole team, not just .NET developers – are already using. Your Project Managers can create task lists and project plans in Excel or Project. Your developers can work on those plans in Visual Studio – or not! If they’re working with Eclipse or other non-.NET tools, developers can use TeamPlain to access their work items. Testers can create and apply test plans. Executives can view summaries and reports and Key Productivity Indicators (KPIs) through a project Web portal.

And all of these tools are updated automatically as the team works, in real time, through the Team Foundation Server: a powerful set of databases, reports, services, and tools that integrate with Excel and Project and Visual Studio and Sharepoint and other tools to provide a seamless process involving all participants.

I didn’t intend this post to turn into an ad for VSTS, so I’ll leave you to explore further if you like. But I can tell you this: even if Owner had known about VSTS, even if he could’ve convinced Cowboy Consultant to look at it, they would’ve turned it down. Owner already balked at spending proper time and money on requirements and design. Cowboy Consultant already demonstrated that he has no patience for anything beyond the code itself. Well, as powerful as VSTS is, it’s not cheap (though I think it’s well cost justified), and it takes time and effort and training to use it properly. They both would’ve written it off as needless overhead, judging by the attitudes we’ve seen so far.

And those attitudes are going to lead to some mighty big problems pretty soon. Dinosaur sized, even.

One final note on today’s Episode. You may wonder why Dog is going out into the Park with The Reader and Stick Boy. Well, she’s a dog. She smells things, really interesting things. She wants to investigate, not hang around in some sterile, air conditioned computing center. Dogs gotta be walked, ya know?

An Argument for Requirements Analysts

Martin L. Shoemaker — Wed, 17 Dec 2008 00:11:14 GMT

An attempt to trade quality for cost or schedule actually results in increased cost and a longer schedule.

Steve McConnell,
Professional Software Development

What has long been known in other businesses is true for software development as well: if you cut corners for shorter schedules or lower costs, you will get longer schedules, higher costs, and higher defect rates; but if you take the right measures to lower defect rates, you can get lower defect rates and shorter schedules and lower costs. As Crosby wrote, “Quality is free.” But it’s free only in terms of ROI, meaning the investment must be made first; and it’s only free if you first define what you mean by “quality”.

Fortunately, Crosby provided the appropriate definition as well: quality is conformance to requirements. That can be a concrete, quantifiable definition; but in some way it just moves the problem down the road, leaving us to define requirements: not just the term, but the specific requirements of our projects. It leaves us with this inescapable truth:

If we don’t know our requirements,
we can’t have quality.

…we must define quality as “conformance to requirements” if we are to manage it.

Philip B. Crosby,
Quality is Free: The Art of Making Quality Certain

Data from Boehm et al., Software Cost Estimation with COCOMO II

Recent surveys have found that the most frequent causes of software project failure have to do with requirements problems – requirements that define the wrong system, that are too ambiguous to support detailed implementation, or that change frequently and wreak havoc on the system design.

Steve McConnell,
Professional Software Development

Poorly defined requirements are endemic to the software development industry. Boehm’s research on factors that affect development schedules and costs show that:

Excellent requirements analysts can reduce a project’s schedule by almost 30%, while inadequate analysis can increase the schedule by over 40%.

Again and again, schedule pressures lead teams to start developing before they have sufficient requirements definition.

Even though requirements analysis is a key skill, the topic isn’t as “hot” as new technologies and tools that are promoted by vendors and conferences and magazines. And many development teams feel swamped just trying to keep up with technologies and tools.

Martin L. Shoemaker

Data from Boehm et al., Software Cost Estimation with COCOMO II

Among all personnel factors, Analyst capability has the widest range of impact (multiplier range of 2, worst case divided by best case). Teams may tout their application experience as a strength; but application experience has an Influence Range of only 1.51. Application and platform experience combined have an Influence Range of 2.11. Teams would never throw out their domain knowledge and develop for an entirely new platform; but poor requirements practices have almost the same Impact Range.

These teams aren’t foolish, yet they foolishly let a critical aspect of their process get out of control on project after project. A look at their team rosters may give a clue as to why: while there are many roles on the rosters, there may be none with requirements as a primary responsibility. Marketing and sales have requirements responsibilities, but many conflicting responsibilities as well. Lead engineers are supposed to verify requirements; but they are also too busy, and are commonly focused on solutions, not requirements. Designers and developers also focus more on how than on what. Traditionally in software development, analysts have primary responsibility for and are evaluated on the correctness of requirements.

The role of requirements analysts is
to define the problem in a verifiable form,
so that teams may recognize a valid solution.

And next you must ask: who owns that responsibility in your organization? If the answer is "no one" or "I don't know", there's a ripe opportunity to cut your schedules by 30% to maybe even 50%, all while improving your quality.

Code is not enough. It's all about requirements; and that's all about communication.

Ulterior Motive Lounge Episode 27: Meet the Crew

Martin L. Shoemaker — Fri, 05 Dec 2008 23:34:39 GMT

Continuing The Project That Time Forgot, a UML case study in comic strip form... (Click pictures for larger images.)

There's the business, and then there's the role of the system within the business. If all you focus on is the role of the system, you can miss chances to see where the system's really needed.

So time permitting, I would rather start by understanding the whole business and then work inward. Oh, sometimes the division is very clear: if my client asks for a change to their payroll system, I probably don't need details about their waste management system.

Or do I? What if the waste management staff is made up of hourly employees? What if excess waste problems can result in overtime? What if that overtime gets charged to a different budget, such as the Environmental Quality Initiative?

The boundaries between "business" and "problem domain" can be fuzzy, and it's easy to cross the boundaries without realizing it. I find it's easier to start with a broad scope and narrow in than to start with a narrow scope and expand.

And if I had to pick the number one category of overlooked Actors in requirements analysis, it's the development and maintenance teams themselves. Customers think about what they need from your system; but how often do they think about what you need from the system? Do they think about running diagnostics to chase down problems? Do they think about maintenance and archiving of data? Do they think about staging and deploying patches and new releases? Do they think about detecting and tracking defects and problems and patterns of usage and percentage of down time? Do they think about support personnel and the tools they'll need to keep users running?

Maybe. Maybe, if they're a very experienced system buyer. But most often, the answer is no. If you don't think of the development and maintenance and support staffs as Actors, and you don't identify their Use Cases, no one will. Then you'll deploy, and the system will need maintenance, and your users will need support, and you'll have no tools to help them. And when you start trying to retrofit the tools after the fact, someone will say "You should've thought of that." And they're right. You should have. You're the professionals here.

So Owner has done a good job of identifying the "stars" of the story. These are Actors you will see again as the story plays out. (Trust me: if I didn't need them in the story, I wouldn't draw them. I'm just too busy to draw characters I don't need. (Somewhere Editor Bill and Curtis Gray are stunned into disbelief at that remark.)) But he has barely scratched the surface of the full Park staff. After much longer discussions, The UML Guy identified these basic classes of Actors:

This is a common pattern in many of my models:

Users are anyone who might be involved with the system in any way. They may be anonymous visitors to the Park's Web site; or they may be more specific Actors, as described below.
Authorized Users are Users who have permissions to do something on the system that Users can't do. This is represented in the diagram above as a set of Authorizations. Different Authorized Users will have different Authorizations. We'll get more specific about what that means when we get closer to code.
An Employee is an Authorized User that works within the organization and has additional Authorizations as a result.
A Supervisor is an Employee with a Staff of Employees, along with additional Authorizations and responsibilities.
A Vendor is an Authorized User who isn't employed by the business, but who provides services to the organization and may have additional Authorizations and responsibilities.

In addition, this model has another class of Authorized Users: Guests. We want each Guest to have Web access to Park content, so that we can sell additional services and souvenirs to them. And then Priority Guests are Guests who pay for full Internet access.

After identifying these major classes of Actors, the team identified more specific versions of each. The Vendor Actors diagram is the simplest:

The Supervisor Actors are more numerous:

The Employee Actors are where the "Actor Explosion" comes in. This is just the first diagram; more detailed Employee diagrams follow:

Most (but not all) of these Employee classes have more detailed Employees below.

Here are the Ops Staff Actors:

And Ops Staff is complex enough to require two even more detailed diagrams. Here are the Transport Staff Actors:

And here are the Facilities Staff Actors:

Here are the Laboratory Staff Actors:

Here are the Lodge Staff Actors

Here are Concession Staff Actors:

Here are the Marketing Staff Actors:

And finally, here are the Medical Staff Actors:

Whew! That's a total of 86 different classes and subclasses of Actors! And I'll bet if I spoke to people in the transport, hospitality, marketing, or amusement industries, I'd find more. Owner's 8 Actors may be the most important, but they're less than 10% of the total set of Actors.

And I haven't even started looking at system and device actors. We'll likely end up with around 100 candidate Actors. Will we need all of them? Probably not. But I'm more comfortable now that we haven't overlooked anything.

And yet categorizing isn't enough! We really need to understand relations between them as well. For instance, Employees report to Supervisors. Here is a rough Org Chart for the Park staff, with an arrow from each Employee to the Supervisor to which she or he reports:

There's one more lesson in this Episode: although I see this work as identifying Actors, I went a little farther than that, identifying their goals and motivations as well. That's a step down the road to identifying Personas, as described by Alan Cooper in About Face and The Inmates Are Running the Asylum. In fact, since these are fictional characters, it's a lot like defining Personas. Cooper argues that by creating fictional Personas that represent users with personalities and motivations and goals, you create a filter to differentiate "okay software that someone can use" from "great software that will really satisfy this person's goals and make this person happy". I like a lot of what Cooper has to say in these books. I'm not 100% persuaded yet, but he's persuasive. Of course, I've also gone way overboard by his standards. He recommends typically around 3 Personas, 5 for a big project. I have 8, plus the review team, plus almost 80 more Actors. But I have them in part for the demands of the story, not the system. If Alan Cooper were doing Interaction Design for this project, he would likely focus on just one or two of these Actors, and flesh them out to a handful of Personas, and work on designing great software for them. Then he would focus on other Personas over time.