Risk Management

Risks in E-Commerce

Siva — Thu, 11 Mar 2004 16:55:00 GMT

1) Authentication
2) Privacy and Protection of Information
3) Fraud and Misrepresentation
4) Reliability of trading partners
5) Technology Risk

Internet Project Risks and Mitigation Strategies

Siva — Wed, 03 Dec 2003 18:00:00 GMT

Risk: Personnel shortfalls

Mitigation:

Bring on a skilled core team.

Have the team mentor new people.

Make training and teamwork part of the culture.

Hire top-notch personnel while the market remains soft.

Risk: Misalignment with business goals

Mitigation:

Align developments with business goals and highlight importance of development.

Risk: Unrealistic customer and schedule expectations

Mitigation:

Make the customer part of the team.

Set schedule goals around frequent deliveries of varying functionality.

Risk: Volatile technology

Mitigation:

Introduce new technology slowly, according to a plan.

Use technology because it supports business goals, not because it is the latest and greatest thing to do.

Risk: Unstable software releases

Mitigation:

Stabilize requirements and designs as much as practical.

Plan to re-factor releases from start.

Don't deliver applications when quality is poor and systems crashes (say "no").

Risk: Constant changes in software functionality

Mitigation:

Managing functionality using releases.

Deliver working prototypes before you target new functionality.

Risk: Even newer methods and more unstable tools

Mitigation:

Introduce new methods and tools slowly, as justified by the business case, not merely because they are new and appealing.

Make sure methods and tools are of production quality.

Risk: High turnover

Mitigation:

Set clear expectations and measures of success.

Make staff feel they are learning, growing, and gaining valuable experience.

Risk: Friction within the team

Mitigation:

Staff the team carefully with compatible workforce.

Build team and provide it with leadership.

Manage conflicts to ease friction.

Risk: Unproductive office space

Mitigation:

Acquire dedicated workspace for the team.

Appropriate collaboration team.

Lot of space available for meetings and pizza.

Source: "Ten Deadly Risks in Internet and Intranet Software Development", Donald Reifer, IEEE Software, March/April 2002

Methods & Tools - News, Facts & Comments Edition - November 2003

As Internet-based applications are forming a major trend in software development these days, many of these risks are faced by new projects. Some of the risks and the mitigation's strategies have perhaps lost some of their pertinence (like the high turnover for instance), but most of them could still be valuable.

Risk Management

Siva — Tue, 04 Nov 2003 13:56:00 GMT

Risk avoidance: Risk is avoided by obviating the possibility that the undesirable event will happen. You refuse to commit to meeting milestone M by feature F - don't sign the contract until the software is done. This avoids the risk. As long as you enter into the contract to deliver specific scope by a specific date, the risk that it won't come about exists.

Risk reduction: this consists of minimizing the likelihood of the undesirable event. XP reduces the likelihood that you will lack some features at each milestone by reducing the amount of "extra" work to be done, such as paperwork or documentation, and improving overall quality so as to make development faster.

Risk mitigation: this consists of minimizing the impact of the undesirable event. XP has active mitigation for the "schedule risk", by insisting that the most valuable features be done first; this reduces the likelihood that important features will be left out of milestone M.

Risk acceptance: just grit your teeth and take your beating. So we're missing feature F by milestone M - we'll ship with what we have by that date. After reduction and mitigation, XP manages any residual risk this way.

Risk transfer: this consists of getting someone else to take the risk in your place. Insurance is a risk transfer tactic. You pay a definite, known-with-certainty amount of money; the insurer will reimburse you if the risk of not completing feature F by milestone M materializes. No provision in XP. Has anyone ever insured a software project against schedule/budget overrun ?

Contingency planning: substituting one risk for another, so that if the undesirable event occurs you have a "Plan B" which can compensate for the ill consequences. If we miss critical milestone M1 with feature set F1, we'll shelve the project and reassign all resources to our back-burner project which is currently being worked on by interns.

Key point from all the above: risk management starts with identifying specific risks. Also, I think you can perform conscious risk management using any process, method, technique or approach. It's important to recognize that any process, etc. simply changes the risk landscape; your project will always have one single biggest risk, then a second biggest risk, and so on.

Also: risks, like requirements, don't have the courtesy to stay put over the life of a project. They will change - old ones will bow out as risk tactics take effect, new ones will take their place.

Risk management is like feedback. If you're not going to pay attention to it, you're wasting your time. More than once I've tried to adopt a risk-oriented approach to projects, only to have management react something like, "Oh, you think that's a risk. Well, thank you for telling us. We're happy to have had that risk reduced. Now proceed as before."

One risk I often raise in projects is skills risk. Developers are supposed to crank out Java code who have only ever written Visual Basic, that sort of thing. Not once have I seen a response of risk avoidance (substituting other, trained team members for the unskilled ones), reduction (training the worker in Java), or mitigation (making provision for closer review of the person's code). It's always been acceptance - "We know it's less than ideal to have this guy working on that project, but he's what we've got at the moment. Can't hire anyone on short order, no time for training, no time for more reviews."

If you only ever have one tactic for dealing with risk, your risk "management" is a no-brainer.

---- From the Laurent Bossavit weblog

Investing in Software Testing

Siva — Tue, 28 Oct 2003 08:33:00 GMT

What Does Quality Cost?

The title of Phil Crosby book says it all: Quality Is Free. Why is quality free? Like Crosby and J.M. Juran, Jim Campenella also illustrates a technique for analyzing the costs of quality in Principles of Quality Costs. Campenella breaks down those costs as follows:

Cost of Quality = Cost of conformance + Cost of nonconformance

Conformance Costs include Prevention Costs and Appraisal Costs.
Prevention costs include money spent on quality assurance tasks like training, requirements and code reviews, and other activities that promote good software. Appraisal costs include money spent on planning test activities, developing test cases and data, and executing those test cases once.

Nonconformance costs come in two flavors: Internal Failures and External Failures. The costs of internal failure include all expenses that arise when test cases fail the first time they are run, as they often do. A programmer incurs a cost of internal failure while debugging problems found during her own unit and component testing.

Once we get into formal testing in an independent test team, the costs of internal failure increase. Think through the process: The tester researches and reports the failure, the programmer finds and fixes the fault, the release engineer produces a new release, the system administration team installs that release in the test environment, and the tester retests the new release to confirm the fix and to check for regression.

The costs of external failure are those incurred when, rather than a tester finding a bug, the customer does. These costs will be even higher than those associated with either kind of internal failure, programmer-found or tester-found. In these cases, not only does the same process described for tester-found bugs occur, but you also incur the technical support overhead and the more expensive process of releasing a fix to the field rather than to the test lab. In addition, consider the intangible costs: angry customers, damage to the company image, lost business, and maybe even lawsuits.

Two observations lay the foundation for the enlightened view of testing as an investment. First, like any cost equation in business, we will want to minimize the cost of quality. Second, while it is often cheaper to prevent problems than to repair them, if we must repair problems, internal failures cost less than external failures.

The Risks to System Quality

Myriad risks - i.e., factors possibly leading to loss or injury menace software development. When these risks become realities, some projects fail. Wise project managers plan for and manage risks. In any software development project, we can group risks into four categories.
Financial risks: How might the project overrun the budget?
Schedule risks: How might the project exceed the allotted time?
Feature risks: How might we build the wrong product?
Quality risks: How might the product lack customer-satisfying behaviors or possess customer-dissatisfying behaviors?

Testing allows us to assess the system against the various risks to system quality, which allows the project team to manage and balance quality risks against the other three areas.

Classes of Quality Risks
It's important for test professionals to remember that many kinds of quality risks exist. The most obvious is functionality: Does the software provide all the intended capabilities? For example, a word processing program that does not support adding new text in an existing document is worthless.
While functionality is important, remember my self-deprecating anecdote in the last article. In that example, my test team and I focused entirely on functionality to the exclusion of important items like installation. In general, it's easy to over-emphasize a single quality risk and misalign the testing effort with customer usage. Consider the following examples of other classes of quality risks.

Use cases: working features fail when used in realistic sequences.

Robustness: common errors are handled improperly.

Performance: the system functions properly, but too slowly.

Localization: problems with supported languages, time zones, currencies, etc.

Data quality: a database becomes corrupted or accepts improper data.

Usability: the software's interface is cumbersome or inexplicable.

Volume/capacity: at peak or sustained loads, the system fails.

Reliability: too often -- especially at peak loads -- the system crashes, hangs, kills sessions, and so forth.

Tailoring Testing to Quality Risk Priority

To provide maximum return on the testing investment, we have to adjust the amount of time, resources, and attention we pay to each risk based on its priority. The priority of a risk to system quality arises from the extent to which that risk can and might affect the customers’ and users’ experiences of quality. In other words, the more likely a problem or the more serious the impact of a problem, the more testing that problem area deserves.

You can prioritize in a number of ways. One approach I like is to use a descending scale from one (most risky) to five (least risky) along three dimensions.

Severity: How dangerous is a failure of the system in this area?
Priority: How much does a failure of the system in this area compromise the value of the product to customers and users?
Likelihood: What are the odds that a user will encounter a failure in this area, either due to usage profiles or the technical risk of the problem?

Many such scales exist and can be used to quantify levels of quality risk.

Analyzing Quality Risks

A slightly more formal approach is the one described in the International Standards Organization document ISO 9126. This standard proposes that the quality of a software system can be measured along six major characteristics:

Functionality: Does the system provide the required capabilities?
Reliability: Does the system work as needed when needed?
Usability: Is the system intuitive, comprehensible, and handy to the users?
Efficiency: Is the system sparing in its use of resources?
Maintainability: Can operators, programmers, and customers upgrade the system as needed?
Performance: Does the system fulfill the users’ requests speedily?

Not every quality risk can be a high priority. When discussing risks to system quality, I don’t ask people, "Do you want us to make sure this area works?" In the absence of tradeoffs, everyone wants better quality. Setting the standard for quality higher requires more money spent on testing, pushes out the release date, and can distract from more important priorities—like focusing the team on the next release. To determine the real priority of a potential problem, ask people, "How much money, time, and attention would you be willing to give to problems in this area? Would you pay for an extra tester to look for bugs in this area, and would you delay shipping the product if that tester succeeded in finding bugs?" While achieving better quality generates a positive return on investment in the long run, as with the stock market, you get a better return on investment where the risk is higher. Happily, unlike the stock market, the risk of your test effort failing does not increase when you take on the most important risks to system quality, but rather your chances of test success increase.