VSTS 2010 (Codename Rosario)

All right, Mr. DeMille, I'm ready for my close-up

Martin L. Shoemaker — Tue, 26 May 2009 23:58:43 GMT

Back in January, Brian H. Prince from Microsoft interviewed me about the UML features in Visual Studio Team System 2010. Today, he informed me that the interview is finally live on Channel 9.

Pattern 19. Requirements Archaeology

Martin L. Shoemaker — Sun, 01 Feb 2009 19:13:36 GMT

Diagnostic Checklist: Requirements Archaeology

□ 70 Your project is to supplement, upgrade, or replace an existing system.

□ 30 The existing system is poorly documented, or the documentation is out of date.

____ Total

You can learn how to score the checklists here.

Problem:

The project is to supplement, upgrade, or replace an existing system, which is the de facto primary source of your requirements.

Context:

In many organizations, new work has to be done in a context of legacy systems, which embody a wide range of explicit and implicit requirements that customers may overlook.

Forces:

Legacy systems represent an investment of capital and time that must be conserved.
Legacy systems may not be well documented or well understood by the current customer.
The team that built the legacy system may no longer work in this organization.

Legacy systems represent an investment of capital and time that must be conserved.

Solution:

Reverse engineer models (see the Modeling pattern) and specs for the current system, and review those to identify requirements. Then analyze and organize those “uncovered” requirements, and present your results back to your customer as a form of The Echo Effect. Be ready for your customer to be surprised, as you reveal features they’ve long forgotten, never known about, or just taken for granted. Listen for them to identify legacy features that they don’t need replicated or supported, and then explore those features to be sure there’s nothing there that other features depend upon.

UML tools that will reverse engineer UML structure include ArgoUML, Enterprise Architect, IBM Rational UML tools, and MacA&D, among others. In their preview of the 2010 version of Visual Studio Team System, Microsoft has demonstrated powerful UML tools, including the ability to reverse engineer sequence diagrams from source code. This is a major advance in general purpose reverse engineering, turning what used to be hours of manual work into minutes.

You may be able to rely on mechanical tools to do much of this reverse engineering. Many UML tools, for example, will reverse engineer models from source or from compiled code.[1] Other tools will reverse engineer and graphically depict database schemas, network layouts, and other elements of existing systems.

But you need to go beyond such mechanical reverse engineering for two reasons.

It’s far too coarse, as our archaeology metaphor shows. Imagine the look on an archaeologist’s face if a contractor showed up at a dig site and said, “Hey, I’ll dig up this whole excavation for you. Let me get my bulldozer.” While mechanical reverse engineering isn’t destructive like the bulldozer, it’s not much more discriminating. The bulldozer piles up a great big heap of earth and stone and rubble, while the mechanical tools pile up a great big heap of classes, relations, tables, columns, nodes, and other artifacts. If you don’t sift through the rubble, you really haven’t learned anything.
In my UML classes, I teach that UML isn’t about software design, but rather about system design, where a system is structure with behavior and goals. Well, I’m missing something in that definition: a system is structure with behavior that fulfills some intention. A good mechanical reverse engineering tool should produce a structural model of a legacy system. Modern tools shed a little light on the behavior model of a system. But only a human reader can produce “intentional” models. A mechanical model captures Functional Requirements at best; but an intentional model captures the User Requirements, Business Requirements, Non-Functional Requirements, Constraints, Actors, and Domain Objects that are represented by the Functional Requirements. (See the Categorization pattern for a discussion of these types of requirements.)

So certainly start with mechanical reverse engineering, as much as your tools will allow; but then dig into the resulting models and organize them (a form of The Outline Effect) to learn what’s in the models. From there, you can create behavioral and intentional models by looking at mechanical models, the user manuals, and the system in operation and then deducing how these fit together.

Resulting Context:

The reverse engineered requirements should serve as a baseline for or input into the requirements for the new system or extensions to the existing system. From there, turn to other patterns as a way to elicit and analyze new requirements.

Certainly start with mechanical reverse engineering, as much as your tools will allow; but then dig into the resulting models and organize them (a form of The Outline Effect) to learn what’s in the models.

There must be new requirements, right? You’re not just replacing the existing system for something to do, right? There should be some reason for replacing it: porting to a new platform, better maintainability, better extensibility, or something. If you’re just suffering from replacement-itis, you should read Joel Spolsky’s essay on rewriting.He makes a good argument for preserving the hard-earned knowledge embodied in legacy systems, and the consequences of replacing working code without a business case.

Discussion:

See also Modeling, The Outline Effect, and The Echo Effect for patterns that work well with Requirements Archaeology.

Verification Checklist: Requirements Archaeology

For each pre-existing system or component:

□ 1 You have a mechanically reverse-engineered structural model of the component.

□ 2 You have a manually reverse-engineered intentional model of the component.

□ 1 You have a presented one or both models to the Customer and received useful feedback.

%__ Percentage of requirements which score 4.

%__/2 Percentage of requirements which score 3.

%__/4 Percentage of requirements which score 2.

%____ Total

You can learn how to score the checklists here.

Ulterior Motive Lounge Episode 31: The Teams Split Up

Martin L. Shoemaker — Sat, 31 Jan 2009 06:59:22 GMT

Continuing The Project That Time Forgot, a UML case study in comic strip form... (Click pictures for larger images.)

After a long hiatus due to weather, illness, work, conferences, and more stuff than I can explain, the Lounge is back.

This Episode gets the ball rolling for Act II, so there’s not much new UML content here yet. But I can give you a few diagrams of the team’s review process. The process starts with some preliminaries, then splits into three threads of operations, each with a separate team:

Design Review. This team, led by The UML Guy, will create a Design Assessment, summarizing the state of the design and any clear risks and concerns. The details of the Design Review look like this:

Review the original design spec to determine the intent of the original team.
Review design documents and updates since the original spec.
Reverse engineer a design from the code. A good UML tool like Enterprise Architect makes this a lot easier, but not easy. VSTS 2010 will make it even easier, but still not easy. Incorporate this into a design summary.
Review the design with the dev team and revise until it’s ready for presentation.
Review the design with the full review team and revise until it’s ready for assessment.
Create the design assessment.

Functional Review. This team, led by Geek Girl, will create a Functional Assessment: a description of how well the system performs existing functions, with emphasis on usability and correctness. The details of the Functional Review process look like this:

Review the existing requirements docs.
Review outstanding defect reports, and make those the beginning of a functional assessment.
For each function identified in the requirements, identify actors and user who represent those actors. Interview those users to determine how well the system really works, and update the functional assessment.

Requirements Review. This team, led by The Reader, will create a Requirements Assessment: a description of what requirements the system covers; and more important, what it doesn’t cover. The details of the Requirements Review process look like this:

Review the existing requirements docs.
Assess the quality of the existing requirements. Are they quantified? Are they testable? Are they unambiguously stated in words, diagrams, and tests? Are they assigned business values and priorities? Do they have clearly identified actors and use cases and scenarios and goals for each feature? These and many more questions will help to create the requirements assessment.
For each actor represented in the requirements or discovered during interviews, identify and interview representative users. Determine whether they have missing requirements, as well as whether they know of or require previously unidentified actors.

For a really thorough review, you probably need at least three more threads of activity:

Code Review to inspect the code, either completely or through sampling, to identify how well it’s constructed and maintained.
Quality Review to identify Quality Assurance measures, Testing measures, and the status of each.
Process Review to analyze the processes followed by the dev team and related teams.

But for simplicity (and to keep most of my focus on my primary message), I’m going to restrict this case study to Design, Functionality, and Requirements. Code will be reviewed under Design, and Quality will be reviewed under Functionality. Process will be reviewed everywhere.

Is this a lot of work? Yep. But it’s a minimal necessary set of activities to review a project for status and risks after you’ve ignored them for a long time. It would be so much easier if you could do these sorts of assessments in real time right from the start as the project proceeds. It would be so much easier if you could report the status and health of the project at the push of a button, based upon the actual work the team does as they do it. Gee, wouldn’t Owner’s life be a lot easier if there were tools to do all that?

Oh, wait a minute… There are tools like that! In fact, there’s a whole category of such tools: Application Lifecycle Management (ALM). And my current favorite is the aforementioned Visual Studio Team System, perhaps the most poorly named product Microsoft produces today. Why poorly named? Because Visual Studio is a developer tool set, particularly a tool set for .NET developers; so when people hear a name like “Visual Studio Team System”, they assume it’s a developer tool set, only bigger and better. They assume it’s about .NET development.

VSTS is not about .NET development. Oh, VSTS includes Visual Studio, and then adds some powerful developer and architect and DBA and tester features; but those features, as powerful as they are, are just the icing on the cake.

The cake is ALM. VSTS is about your process and how you can define it and design it and manage it and track it, all through tools your team – your whole team, not just .NET developers – are already using. Your Project Managers can create task lists and project plans in Excel or Project. Your developers can work on those plans in Visual Studio – or not! If they’re working with Eclipse or other non-.NET tools, developers can use TeamPlain to access their work items. Testers can create and apply test plans. Executives can view summaries and reports and Key Productivity Indicators (KPIs) through a project Web portal.

And all of these tools are updated automatically as the team works, in real time, through the Team Foundation Server: a powerful set of databases, reports, services, and tools that integrate with Excel and Project and Visual Studio and Sharepoint and other tools to provide a seamless process involving all participants.

I didn’t intend this post to turn into an ad for VSTS, so I’ll leave you to explore further if you like. But I can tell you this: even if Owner had known about VSTS, even if he could’ve convinced Cowboy Consultant to look at it, they would’ve turned it down. Owner already balked at spending proper time and money on requirements and design. Cowboy Consultant already demonstrated that he has no patience for anything beyond the code itself. Well, as powerful as VSTS is, it’s not cheap (though I think it’s well cost justified), and it takes time and effort and training to use it properly. They both would’ve written it off as needless overhead, judging by the attitudes we’ve seen so far.

And those attitudes are going to lead to some mighty big problems pretty soon. Dinosaur sized, even.

One final note on today’s Episode. You may wonder why Dog is going out into the Park with The Reader and Stick Boy. Well, she’s a dog. She smells things, really interesting things. She wants to investigate, not hang around in some sterile, air conditioned computing center. Dogs gotta be walked, ya know?

"Where do I start?"

Martin L. Shoemaker — Sat, 15 Nov 2008 21:10:19 GMT

Friends and family claim I have an annoying habit of answering a question with a question. They think I'm being difficult, and they think the answer I deserve is a kick in a tender spot. But I have a reason: I’m convinced that the answer to most useful questions is: It depends. The best answers depend not just on the question, but on the context.

So when people ask me “I want to learn UML, where do I start?”, my answer is, “Where are you stopped?” Well, OK, my first answer is, "My UML Applied Class." But after I make the obligatory shameless plug, I have to know where your current project is. UML is a communications tool, not a technology, so it's useful across the complete application lifecycle; but different diagrams are more generally useful at different stages of the lifecycle.

In the UML Applied classes, we'll start with use cases, which is a great place to start at the onset of a project. If you just can't get started writing code because no one is sure what to write, use case diagrams will help you to capture user requirements and communicate them to stakeholders for feedback. Draw an actor, add use cases for that actor, and then add collaborating actors or related information for each use case. Repeat for each actor and use case until everyone agrees that you've identified the significant tasks that users must perform.

If you're trying to learn and extend an existing, unfamiliar system, you might want to start by creating a component diagram that shows the existing components and the interfaces through which they communicate. From there, you can determine where your new code will fit within this architecture. You can also use these diagrams to plan out entirely new architectures.

Or you might start with a broader scope: a deployment diagram that shows where within a network the existing components are deployed. Again, you might also use these to plan a brand new deployment architecture.

Yet another way to learn existing code is to read the code and create sequence diagrams. A sequence diagram shows how parts of the code call other parts of the code to carry out user requirements. To learn how a particular user input -- say, a button click -- is processed, start with a lifeline that represents the form where the button click occurs. Read the button handling code, and add a self-message that describes any code within the handler. When the handler calls a method of another object, add a lifeline for that object, and then add a message from the form to the object, and label it with the method name. Repeat until you have diagrammed the entire handler. And then repeat for each call and method you have added, until you have a detailed view of how the handler works and how it collaborates with other objects.

I should add: the upcoming Visual Studio Team Systems 2010 (codename Rosario) will reverse engineer sequence diagrams straight from code!

Our final example today is when you're trying to understand or plan out the user interface flow of a system. For that, I like to use state diagrams, with each state representing one page within the user interface.

By no means are these the only ways to start using UML. You name a diagram type, and I'll tell you about a time where I started work with that diagram. But these are the ones I start with most often. I hope they'll help you get started, too.