2) to involve passionate quality advocates (testers) in creating the highest quality requirements. However, there were several unexpected benefits that resulted from this initiative, as I will detail later in this article.
Initially, the business analysts exhibited some resistance to the participation of the testers in the requirements gathering process. It seems they perceived the testers as a threat—which perhaps is not all that surprising. Imagine if roles were reversed and the business analysts were suddenly added to the testing process in an effort to improve test plans and test execution. It is reasonable to assume that testers would feel threatened by this situation. In order to allay the concerns of the business analysts, we made it clear that this project was intended to improve the capabilities of both teams equally. The business analysts would learn more about quality and testing from the testers, and the testers would learn more about how new software is defined and documented from the business analysts. Both teams also were told that, from a career perspective, working in multiple areas of the SDLC would be beneficial. Although some residual skepticism appeared to remain in a few individuals, overall, both teams were comfortable with the new arrangement for this project.
Cross-Training
Semi-formal cross-training was provided to help the testers become capable business analysts and similarly to help the business analysts become capable testers. This training was in the form of a "kick-off meeting" held by the two groups together. At the meeting, one representative from each team made a twenty- to twenty-five-minute presentation about how his team (e.g., the business analyst team) executes its responsibilities and how the other team (e.g., the testing team) can integrate these points in adopting the new cross-functional responsibilities. The meeting was successful, but the primary education still took place "on the job," as the testers and the business analysts worked together and learned from each other.
Logistics
We knew that it would have been optimal for the new cross-functional team to sit together. However, the business analysts’ existing desks were in the corporate headquarters (because analysts generally need to be near the business users) andthe testers were in the IT development center (to be near the software developers). We did not relocate the teams because both teams had other responsibilities outside of this project that required them to remain close to their original locations and because this experimental initiative was viewed as a one-time trial not warranting such a major upheaval. If the trial succeeded, we would then consider merging the two teams into one location. Fortunately, the corporate head-quarters and IT facilities are only a short distance apart, and regular in-person meetings complemented the standard on-going exchange of ideas and documents via email and telephone.
In addition to ad hoc one-on-one meetings, recurring partial and whole team meetings (either in person, by conference call, or both) were held continuously during this project, usually several times per week. Many of the meetings were technical in nature: review deliverables, discuss strategies, and provide support for the team members who were new to the particular phase of the project (i.e., support for the testers during the business analysis phase). Other meetings were more organizational: ensure that the project was on schedule and allocate and balance resources to the tasks on the project plan. During the requirements gathering and definition phase of the project, the meetings were organized and led primarily by the business analysis manager. During the test planning phase, the meetings were led by the testing manager. By the time we reached test execution, both managers were sufficiently familiar with all aspects of the project, and both managers, depending primarily on availability, organized and led the meetings.
THE IMPORTANCE OF REQUIREMENTS TESTING
"Many potentially successful projects have failed as a result of poorly tested requirements ..."—Steven J. Offenbacher from Building Usable Software through Early Testing
I am a major proponent of requirements testing, where each requirement is analyzed ("tested ") by the testing team based on a set of established criteria to determine the quality and correctness of each requirement. Requirements testing performed by the testing staff can be a significant source of defect prevention —s ving the project valuable time and money. Unfortunately, formal testing of requirements prior to coding is all too uncommon. One might think that the presence of the testing team in the requirements definition process would eliminate the need for actual requirements testing. This is not true. Just as test plans created by tester benefit from peer review, so do requirements benefit from testing by peers who were not directly involved in writing those requirements. So, even when the testing team participates in creating high-quality requirements, be sure to test them. For more on this topic, see this issue ’s StickyNotes.
Requirements Gathering and Definition
The first phase of the project, to research and define major new system functionality, was quite a success. The addition of the testers to this effort introduced a fresh perspective to these tasks—tasks typically handled primarily by business analysts. And, because of extensive experience testing other programs in the application suite, the testers were able to clarify the requirements to provide a consistent look and feel for the new application (screen layout, data presentation, tab ordering, keyboard shortcuts, etc.). These nuances are important usability issues that are sometimes overlooked when business analysts without experience with the existing applications in the same suite are defining a new application.
The business users praised the participation of the testers in the requirements gathering process, claiming that their insightful research questions and knowledge of the existing application suite fleshed out scenarios and functionality that had been overlooked. Feedback from IT management and the software development team regarding the requirements was extremely positive. Although it took a bit longer to create the requirements because of the additional "hands-on" training for the testing staff, the requirements did not require any major revisions during code development—in-accurate or insufficient requirements had been a common problem in past projects. The development time saved in initial coding, coupled with time saved by a reduced number of bug fixes during test execution (as detailed below), allowed code development to be completed in 20 percent less time than was originally projected.
8 THINGS BUSINESS ANALYSTS SHOULD KNOW ABOUT TESTS AND TEST PLANNING
THINK OUT OF THE BOX. Create test cases that verify that the application does what it is intended to do and does not do anything that it should not do.
LEVERAGE QUALITY REQUIREMENTS. With high-quality requirements in hand, write test cases that map back to the requirements. If requirements change in the future, it will be easy to find the related test scenarios that need to be revisited.
TEST COMPLEX FUNCTIONALITY THOROUGHLY. Complex code is more prone to defects. The volume of test scenarios is roughly proportional to the complexity of the functionality
DON’T UNDERESTIMATE THE CHALLENGE OF TESTING. Software testing is a challenging intellectual effort and a detail-oriented craft. Don ’t underestimate the thought involved in test planning and execution. Business analysts tend to focus on the "big picture "business purpose of the application. Testing must also address the picayune details, including installation testing, keyboard shortcuts, and font sizes (to name a few).
KNOW YOUR ENVIRONMENT. Understanding the testing environment is key to successful testing. If the environment is dynamic, then testing must be similarly dynamic.
EXECUTE TESTS EXACTLY AS WRITTEN. When executing someone else ’s test plan, do not execute steps that diverge from the test plan —even if you feel that the test scenario is not correct. Instead, speak with the test plan author and update the test plan if needed. It is possible that the test planner knew exactly what she was doing.
USE DOCUMENTING SKILLS TO REPORT DEFECTS. Business analysts are very good at documenting new software. They should use the same skills to provide clear, detailed defect reports.
ASK FOR HELP. No one expects a business analyst to become a seasoned tester immediately Ask for help from teammates, especially when dealing with unfamiliar technology (data-base queries, performance monitoring, etc.).
8 THINGS TESTERS SHOULD KNOW ABOUT REQUIREMENTS GATHERING AND DEFINITION
IDENTIFY AND BEFRIEND THE SMES (SUBJECT MATTER EXPERTS) IN THE ORGANIZATION. They are the ones who really understand what is expected from the proposed application. Interviewing these individuals will be a major source of information.
A PICTURE IS WORTH MORE THAN A THOUSAND WORDS. Flow diagrams and GUI mock-ups can be a major benefit to successful documentation.
AMBIGUITY IS THE SCOURGE OF ALL REQUIREMENTS. Be as clear as possible. "Throughput should be at least 475 units " ((Per hour? Per day? Per second?).
DON ’T ADDRESS SOFTWARE DESIGN. The requirements should detail what the application should do —not how it should be implemented "under the hood."
TRACK CHANGES. Just like any body of work, track changes and use a central repository for storage.
SHARE AND REVIEW WITH TEAMMATES. If the application or module you are working on interfaces with other applications or modules, exchange and review requirements with the other author(s).
BE AVAILABLE TO THE DEVELOPERS DURING APPLICATION CODING. No matter how well you think you ’ve created the requirements, questions invariably will arise during application design and development. Be available to answer questions and/or perform further research. Be sure to update requirements documentation as needed.
ASK FOR HELP. No one expects a tester to become a seasoned business analyst immediately Ask for help from teammates.
Test Planning: Successes and Challenges
Test planning clearly improved as a result of this experimental approach. The team created detailed test plans based on the foundation of the high-quality requirements. Even more impressive, however, was the testing of the overall business intent of the new application. On previous projects, testers had sometimes focused on application details while overlooking the "big picture" intended functionality. This was not necessarily the fault of the testing team. Testing teams that lack in-formation about the ultimate business purpose of the application are likely to test only the specific items detailed in their inputs. Theoretically, all the individual functionality details should sum-up to represent the "big picture" intended functionality—but even when this assumption holds true, it is not always sufficiently clear to the testing team. By including the testing team in requirements gathering, however, the test plans reflected not only all the focused details of the application, but the "big picture" functionality as well.
Although using the cross-functional team for requirements gathering and the definition phases was a success, the team had some challenges in the testing phase. The first challenge was a general lack of interest among a few of the business analysts to perform testing tasks. Each individual had a different explanation of his dissatisfaction with testing, but there were two common threads: they preferred the "people-oriented" work of business analysis (meeting with clients and users, documenting requirements, reviewing with users and developers, etc.) rather than the more solitary "heads down" work of testing; and/or they were uncomfortable with the technical aspects of software testing tasks.
Another challenge was related to the test plans prepared by the business analysts. We had anticipated that high-quality requirements and a familiarity with the expected results would lead to a reduction in time necessary to complete test planning. This was true for the test plans created by the testers. However, the business analysts made mistakes in their test plans that negated the gains made by the testers. The primary mistakes in their test plans were that their test scenarios focused almost entirely on verifying that the application did what it was intended to do, and they lacked scenarios to check that it did not do any-thing that it was not intended to do. Also, the business analysts’ scenarios failed to take into account the dynamic nature of the test environment data. Due to the ever-changing nature of the test environment data, proper test cases for this application require either setup of data scenarios or identification of suitable existing data immediately prior to executing each test case. The business analysts’ test cases were static inasmuch as they specified the exact records to be used for test execution. The business analysts did not realize that, although the specified records may have been suitable at the time the test cases were defined, the records would likely have changed by the time they were executed. To give a very simple example, an application asking for someone’s birth date should not allow a value in the future. A static test case might read, "Enter September 20, 2005 in the Birthdate field and verify that the application warns the user that future values are not permitted." This static test case will no longer be valid come 21 September 2005. A superior dynamic test case might read, "Enter a date greater than the current system date in the Birthdate field . . .".
In total, test planning required roughly the same amount of time as previous test planning efforts—meaning, the expected time savings had been negated by the issues related to the participation of non-testers.
In retrospect, we learned two important lessons. First, while the testing team seemed motivated to participate in the requirements definition process, as the quality mandate mentality applies in many different venues, including requirements gathering, some business analysts weren’t interested in test planning. Second, those business analysts who did participate simply did not yet have the testing experience necessary to create accurate and comprehensive test plans. The business analysts’ back-grounds did not transfer well to test planning and execution.
OPPORTUNITIES AND CHALLENGES OF CREATING A CROSS-FUNCTIONAL TEAM
OPPORTUNITIES
1.Clearer and more complete requirements
2.Reduced overall development time
3.Faster test planning
4.More comprehensive test plans
5.Expanded testing focus on "big picture " business intent of the application
6.Reduced number of code defects
7.Reduced test execution time (due primarily to fewer bugfix builds requiring regression testing)
8.Reduced number of defects found in user acceptance tests
9.Reduced number of defects found in Production
CHALLENGES
1.Business analysts may feel threatened by testers participating in requirements process, or vice-versa
2.Need to cross-train testers and business analysts
3.Not all business analysts are interested in testing ,or vice-versa
4.Test execution sometimes requires technical skills not common to business analysts
Test Execution
Manual test execution was performed by all members of the team (a separate team would handle automation at a later date). Test execution had only one issue related to the new cross-functional testing team, which was primarily technical in nature: Only one of the business analysts was familiar with relational databases and SQL. This was a challenge when test scenarios required direct query of a database to verify data results. In situations such as this, the business analysts requested assistance from the testers on the team.
Experiment Results
The ultimate factor in determining whether this experiment was a success is the quality of the software that was deployed. Test execution identified far fewer bugs than prior testing of other applications in the same test suite. For those testing teams who incorrectly measure their success by the number of defects that they find, this would be perceived as bad news. However, quite the opposite was true in this case. The lack of defects identified in the testing phase was attributed directly to the quality of the requirements that were created and tested by the tester/analyst team. The reduced number of defects significantly lessened the number of bugfix builds (and the associated regression testing effort) during the test execution phase, thus decreasing the overall time to complete test execution.
The User Acceptance Testing (UAT), semi-scripted testing by the user community to confirm the general accuracy of the application implementation, found far fewer defects in their testing, which focuses on the "big picture" business functionality. This, too, was attributed to the clearer "big picture" understanding by the testing team and the related improvements to the test plans themselves. And, to date, the defects found in the Production system have been traced to a few incorrect requirements from the business users and environment-specific data issues that were not directly related to the application code. Even considering these defects, this is still a major improvement over the outcome of similar projects in the past.
From a total software development lifecycle perspective, the project was a huge success. Development was completed 20 percent ahead of schedule, testing was completed 15 percent ahead of schedule (mainly from time saved from having fewer bugfix builds during test execution), and the small number of bugs found in UAT and Production was a significant improvement over past results.
Moving Forward with Improvements Based on Lessons Learned
Following the overall success of our "experimental" approach to requirements gathering and testing, this method was expanded to other software projects. Based on what we had learned in the first attempt, we made some modifications to our approach. We no longer expected all business analysts to participate in test planning and execution. Those analysts who were not interested in testing received more traditional business analyst responsibilities and were not included on the cross-functional team.
Second, some of the most critical challenges we faced were the result of in-sufficient cross-training. We had incorrectly assumed that, as the teams were working together on this project and had substantial exposure to each other’s work on previous software projects, one semi-formal training session would be sufficient. In retrospect, I believe that more structured training would have added to the success of this project. To help give both testers and business analysts more exposure to each other’s skills, each business analyst who wanted to participate in the cross-functional team was partnered with a tester. By working together as a pair to perform requirements gathering, test planning, and execution, the tester and business analyst train each other. Eventually, we hope that all members of the team will have the same expertise in both business analysis and testing and will be capable of functioning independently in either realm. One challenge to this partnering approach is that, for organizational and logistical reasons, the testing team and the business analysis team remain in separate facilities. So, for the time being, it is necessary to rely on email, telephone calls, and frequent in-person meetings.
As a result of these improvements, subsequent projects using this new model succeeded in the test planning time savings that we had anticipated at the outset of this initiative: roughly a 25 percent savings in time needed to produce comprehensive test plans as compared to prior projects of similar scope.
How to Get Started
So, how do you convince the management in your company to consider implementing a similar project? The best business case for this type of change is based on the possible cost savings. The potential to reduce the time to market for a new application, coupled with reduction of Production defects (both of which are directly tied to project costs), should be of interest to upper management. Introduce the idea as a pilot project, with any commitment for long-term change de-pendent on the success of the pilot.
The first step beyond getting the OK from upper management is to work together with the managers of the business analysts and the testers to create an implementation plan that benefits both teams. For example, spend time preparing materials for the teams to train each other and allocate time to implement the training. The time spent by the teams training each other will form a bond between them that will surely pay off in enhanced cooperation down the road.
With a little bit of preparation and cooperation, there is a lot of potential for testers and business analysts to improve software quality—together. {end}
Gedaliah Friedenberg (gfriedenberg@gngroup.com) is senior partner at The GN Group, a New York-based consultancy focused on creative solutions for software testing and SDLC process improvement. Mr. Friedenberg has been in the software engineering field for fifteen years and holds a degree in mechanical engineering and an MBA.