A Technical Debtor

Mere minutes after I posted on “architectural refactoring,” Keith Nicholas has commented about the unsuitability of the term. He said:

you'd be better off calling it "evolutionary design"
The dilution of the word refactoring is disturbing, there's a certain rigor to "factorization" which shouldn't destroy the product.
refactoring != a cool way to say "I'm gonna change stuff"

There’s a fair bit to be said on this, so I thought I’d add another post rather than try to reply in an undersized comment box.

I agree that the use of "refactoring" in this context makes me a little squeamish. Keith is absolutely right that there's a lack of rigor here -- partly because the scope of the changes ensures that enough changes that this isn't really a refactoring in the Fowleresque sense.

However, as Fowler defines it, "evolutionary design means that the design of the system grows as the system is implemented". In the case I'm describing, the design isn't growing. Rather, suboptimal design decisions are being rethought, reworked, rebuilt. In an evolutionary sense, I suppose you could argue this is "growth," since the new architecture is "fitter" than the old.

However, to my mind, the word "evolutionary" brings with it a connotation that the improvement is a small forward step, building on what is already there -- not a massive reworking of the underpinnings of the architecture.

The scope of this effort is what's a little unusual. Kyle Baley, for example, talks here about architectural changes and how you know when it’s time to start over. Now, Kyle is a fellow Canuck and an all-around decent fellow. I even agree that, from what he describes, they made the right choice. However, he describes an effort that was scoped at a maximum of 2 developer-weeks. I had on the order of twice as much time personally into proof-of-concept for the changes that were made; all told, we invested something like 120 developer-weeks into the changes.

I have trouble calling a step of this magnitude “evolutionary.” Perhaps architectural renovation would be a more apt term.

(And, as a minor aside, when I said earlier that “I’m not even leaving the functionality intact,” I should have been clearer. The original architecture made heavy, brilliant and infuriating use of refactoring. That trait, among others, were ripped out. After the renovation, the code base had been reduced by close to 40%, and virtually every line of code had been touched, or at least inspected. However, the business functionality remained essentially unchanged – essentially, since we were able to identify and remove a fair number of bugs during the process.)

Thanks for keeping me honest, Keith – and for the great point you made.

• Share This Post:
• 
• 
• 
•   Short Url: http://wblo.gs/aed

I’m delighted to reveal that I’ve been accepted to speak at the International Association of Software Architects (IASA) IT Architecture Conference (ITARC) in New York City, September 22-24th. I’ll be speaking on effective architecture refactoring… or, in other words, telling people about the mistakes I made, in the hopes that they won’t have to repeat them.

For those in NYC, this is well worth attending. The keynotes will be delivered by some people you may have heard of: John Zachman, Scott Ambler, George Paras, Len Bass and Angela Yochem. Eric Evans has been relegated to only giving a talk this year, as is Paul Rayner. The inimitable Doc List will be moderating an Open Spaces session. All in all, the sort of conference that takes about a month to fully sink in. Hope to see you there!

• Share This Post:
• 
• 
• 
•   Short Url: http://wblo.gs/aea

Yet another in my frenzy of posts for tonight!

I’ve been accepted to speak at the Heartland Developer’s Conference in Omaha, NE (Sept 8-10). This is an amazing conference, put on by Joe Olsen. An absolutely star-studded cast of speakers – some of the names you might recognize include (in no particular order) Chris Williams, Amanda Laucher, Don Demsak, Jason Bock, and Jason Beres. (Yes, I’m still surprised Joe is going to let me speak.)

The price is very reasonable for a two-day conference. If you’re anywhere in the midwest, this is a great conference to attend. I went last year, and it was absolutely spectacular. Knowing Joe, he’s only going to outdo himself this year.

• Share This Post:
• 
• 
• 
•   Short Url: http://wblo.gs/aeX

You’ll notice for the next little while, I’m going to be talking about architectural refactoring. These posts will be a reflection on the past couple years as a software architect, specifically centered around what some people call “brownfield” development. That is, taking something that is in a less than ideal state, and improving it… not starting from scratch as in new (or “greenfield”) development.

Leaving aside philosophical discussions about whether all software development is “brownfield” or not (no plan survives contact with the enemy; no architecture survives contact with the writing of code), what I’m really interested in sharing is the experiences I’ve had with architectural refactoring. If Martin Fowler stumbles across this blog, I’m sure he’ll take exception to my use of the term “refactoring.” After all, I’m not making a series of small changes, validated through unit testing. I’m not even leaving the functionality intact. In at least one of the cases that I’ll describe, what I did was more akin to lifting the house off the foundation, blowing the old foundation into rubble, pouring a new basement, and then setting the house back down.

(I say “I” in this case… but really, I was the one that pushed the button that blew the foundation apart, then handed a new set of blueprints to the development team – they did the work rebuilding the business logic to deal with the changes I made.)

I had the privilege to give a talk about this, at Paul Rayner’s encouraging, for the IASA ITARC (I don’t know what it stands for either – it’s a regional conference) in Denver in May. The talk was incredibly well received, and George Fairbanks was gracious enough to encourage me to write about what I presented in that talk. So, here I am… writing about it, and also doing a fair bit more talking about it.

• Share This Post:
• 
• 
• 
•   Short Url: http://wblo.gs/aeW

I’m in a bit of an odd position, in some ways. CCI brought me on as a “software architect” a little over two years ago. That role wasn’t really defined – in fact, I got to spend several months doing what I felt would be a good use of my time, and then I wrote my own job description. (Perhaps that’s not exactly what other people remember, but it’s sure how things felt to me.)

Like anything, this approach has benefits and drawbacks. On the plus side, I was empowered to go where the smoke was – to find the areas where I could have a great deal of impact, quickly. (One of my favorite comments of this time comes from Kathleen Dollard, who recommended I apply at CCI. She told me once that “you’ve had more impact at CCI in the last three months than I thought anyone could in three years – and they haven’t fired you yet!”)

This was the time-frame in which I was modifying XSLT code generation templates to put repetitious code into a base class, rather than repeating it over and over and over and over and… well, you get the point. They were the days where “minor” changes like this resulted in dropping nearly a million lines of compiled code through the use of inheritance rather than code generation. (I tease one of our team leads that my job is to remove code faster than his team can write it… and I’m winning.) While this seems like a cool number to quote, the real business value actually didn’t have all that much to do with our code. True, there was a lot less code generated on a nearly-daily frequency. However, what was really valuable to CCI was the fact that the background compiler stopped crashing!

It turned out that our application was big enough that it taxed the background compiler in the VB IDE just a little bit. Just enough to find a pesky memory fragmentation issue in the background compiler. Just enough to ensure that Visual Studio would die around once per hour for every developer in the company. (For those of you that noticed this problem went away in VS2008 SP1…. you’re welcome! The leadership team at CCI was kind enough to let me ship our code base to Microsoft, where they easily reproduced the problem, and fixed it.)

These were the good parts of the early days. There were also some pretty rough parts. I mentioned earlier that I’m in an odd position. There are several different roles a software architect can play. One of them is the “gunslinger” or “hit man.” This is the outside consultant who comes in, declares that the architecture is a massive failure, and that everything has to be started anew. Often, this means firing the person that created the old architecture, having the consultant lay out a new one, and then walking away. A bad day for everyone. Lots of resentment, and lots of work scrapped.

So, what does CCI do? I mean, this is a company that has a 95%+ retention rate, and whose more senior developers have been there a decade or more.

They hired me. Not as a consultant, but on salary. They kept their corporate knowledge intact, added an opinionated young upstart to the mix, and put me at the desk beside the most senior developer in the company. Then, to heap craziness upon craziness, they invested heavily in “refactoring” the architecture. By heavily, I mean dedicating the development teams to making deep architectural changes for a few weeks.

And, you know, what? It’s been a great learning experience for everyone… and it’s working. More about that later.

• Share This Post:
• 
• 
• 
•   Short Url: http://wblo.gs/aeV