John Hines' Code Blog

There's a recent blog post that was sent to me in response to my insistence on Test-Driven Development:

"The Duct Tape Programmer" by Joel Spolsky.

The gist of the post is: Good programmers ship software.  I agree.  And my response is: Great programmers ship quality software.

When I wrote commercial software, there were no unit tests.  All testing in-house was done by some internal QA testers, and if the software passed smoke tests it was sent through a full test run.  This could take a week or more.  The schedule was King.

When I wrote code part-time and did full-time Active Directory management, there were no unit tests.  I didn't have time.  I had a full time job, a rotation on the help desk, new tools to write and old tools to fix.  My workload was my boss.

So if you're in the situation where just finishing is a victory, more power to you.  But don't settle there and take a fanboy post to mean you never need to bother with TDD or testing in general.  It could be that you constantly check in timely, buggy code that everyone else hates while simultaneously patting yourself on the back for your Duct Tapey-ness.

Today I'm developing tools full-time to be used inside of the company.  There are two huge differences between intranet software development and commercial programming: Deadlines are more flexible and there is no QA department.  You wrote it, you support it.  For life.

Adopting Test-Driven Development and Scrum has been a survival mechanism for us.  It's the reason my three person team (including myself) can support sixteen production apps.  We don't dare ship bugs, our entire roadmap crashes when we have to rework.

So all I'm trying to say is that you can have your cake and eat it too.  You can have good quality software that you ship quickly - just keep it as simple as possible, build quality into the process, and finish the thing.

In a big company there are often dozens, if not hundreds of projects going on at the same time.  Getting stuck on a bad one - one that's disorganized, frustrating, and sure to forget any contribution you make (no matter how major) - is a pretty common occurrence.  Disengaging yourself skillfully becomes a kind of art form.

My team reached its limit recently after being assigned to a poorly run project for three months.  As a parting gift, we gave lots of advice on how the project should be run - not 20% Scrum/80% Chaos, but 100% Scrum.  The accountable manager then had his team define project had goals, milestones, a short-term schedule, and yes, a daily stand-up meeting.

The result?  Just as my team is preparing to disengage, we find we're suddenly quite busy!  The daily scrum has done some impressive things.  It has:

1. Shown the real team.  The people who really aren't committed don't bother to come to a brief, daily meeting.  It's very clear who the people doing the work are.
2. Increased collaboration.  The area experts have become known, and people ask them questions.
3. Sped the project.  Issues that would have delayed people for days are now resolved much faster.

Not bad for a meeting that lasts no more than fifteen minutes.  You still need those other things - goals, milestones, and schedules.  But if you want an effective project team using Scrum, don't skip the daily stand up meeting.

Technorati tags: Scrum Scrum Process

Agile development isn’t without its traps and pitfalls.  What’s surprised me is how Scrum’s most common problem areas are the same as those I’ve seen in other development lifecycles.  Here’s a list of three problems I've seen or heard arise in multiple sprints and my advice on resolving them.  It’s my hope that this post can benefit anyone who’s seeing these issues – but I’m happy to learn better ways to do it!

1. "People aren't marking their tasks complete as they finish them."

Start by being informed.  A good Scrum Master will review the tasks assigned to each story at the start of each day, before the daily stand-up meeting.  This will clearly show if any tasks aren’t being updated as well as helping you get a handle on your team.  This isn’t meant to be a deep dive – just pattern recognition.

The daily stand-up meeting is your best tool to fix delayed task updates.  After you review team status, mention: “I’ve noticed that there has been no progress on the XYZ story.”  Those working on the story should remind you that they just told you of the progress they’re making.  Smile and respond, “Please update the tasks on the story.”  Repeat this process at every daily scrum until the habit is established.  Eventually (and probably quite quickly) the team will recognize that you’re pushing process.

It’s possible that the tasks on the story are just too big and people are bogged down.   That’s a different problem. J

2. “People aren't attending the meetings.”

This is the most common issue I see, and it’s far from being specific to Scrum.  Scrum has an erroneous reputation as having too many meetings.  At some companies it may also be acceptable to skip a meeting or two, but this really hampers Agile development.

Resolving attendance problems early is critical.  Meet privately with the person who's missing meetings and politely ask for a reason.  It could be that your daily stand-up meeting conflicts with that person’s schedule.  Or it could be that you were told this person was fully dedicated to the project when it’s really 50%.

The vast majority of what I’ve experienced has been people missing meetings without a compelling reason.  Worst is the engineer who happily decides to work independently, disappears for a few weeks, then reappears with work that was already done or has to be awkwardly merged with the whole.

This is where you need to be pragmatic.  If you’re losing a disruptive or unproductive person then feel glad and move on.  For potentially valuable resources, be direct; “I really need you to attend every meeting.  This project can’t be successful without your full involvement.”  The next step is to confirm that person’s expected time commitment from their manager.  The last step is to escalate the issue to their manager;; “I really need John to attend every meeting.  This project can’t be successful without his full involvement.”

If you are constantly fighting for people to show up or do work on your project, your peers are telling you something.  Your project may be conflicting with more important projects or it may be years ahead of its time.  A good Scrum Master doesn’t start a project without clear roles, clear commitment, and a clear definition of success.  Even if their boss tells them to.

3. “We’re all working hard, but we’re not making any progress.”

There’s generally a simple remedy for this – make sure your tasks are small enough to track the work that’s being done.  In other words, you’re probably making progress, but the tasks assigned to your stories are too big to show it.  This does assume that the members of your Scrum team are working on what they’ve been assigned.

Signs that tasks are too generic are when team members report the same status at consecutive daily stand-up meetings.  Remember that a task should be about one day’s work – if a person has been working on the “Test new functionality” task for a week, it’s a sure sign that the task should be broken up.  I’ve also learned that the level of detail in breakup of stories into tasks can depend on the person working on those tasks.  In these cases it’s always best to include as much detail as makes sense.  If a senior person takes ownership of the tasks, checking off a rapid rate of progress makes them look good, and you still have the flexibility of assigning a less experienced person if you need to.

Lastly, there are instances when individuals just aren’t productive.  A good Scrum Master first learns why.  In my experience it’s been due to (in order of frequency):

·         Time conflicts with other projects.

·         Mistaken assumptions of how much time can be given to the project.

·         Lack of training.

·         Lack of motivation.

The best way to fully utilize Scrum is to make sure the first three are taken care of before the project starts.  The last issue should be escalated to be handled outside of the Scrum process.

Technorati tags: Scrum Scrum Process

The real beauty of Scrum is that it gives you the tools needed to know very quickly when things aren't going well.  The most important part of Scrum is running successful sprints.   You can't expect to have a successful project with the majority of your sprints not meeting their goals.  The primary tool for measuring your sprint's progress is called a burndown chart.

Simply put, a burndown chart tracks how much work you have left to do in your sprint.  In its most basic form, an empty burndown chart looks like this:

The Y axis tracks the amount of work you've committed to finishing.  The X axis tracks the amount of time you've given yourself to finish the work.  The "Ideal Trend" line represents the best rate of progress - a steady, sustainable pace.

There are diffent styles of burndown charts, but if we add the number of tasks, progress, and work remaining for a perfect sprint, it would look something like this:

The number one job of a Scrum Master is to do everything possible to help the team finish the tasks by the end of the sprint. It's not to make every sprint look exactly like this. If you're just starting to use Scrum, you'll see a pattern similar to the next three burndown charts.

A typical first sprint vastly overestimates what the team can accomplish. I'll let the psychologists figure out the why, but it's a pattern I've seen again and again. I'll note that the places where the progress line is flat are either weekends or some technical issue. Here's a typical first sprint's burndown chart:

Beware! This is a pattern that can occur anytime the team has taken on too much work. Any Scrum Master who sees this happening should immediately focus on fixing this trend.

Depending on the Scrum Master, the second sprint tends to underestimate. There's a lot that goes into kicking off a project. It's typically safe to commit to a little more than what you got done for the first sprint, but no team wants to miss their goals for the first two sprints of the project. Typically the Scrum Master ratchets down the amount of work to ensure that positive status report:

In my experience it takes around three sprints to find the right balance of work to commit to. The amount of work is usually between that committed to for the first couple of sprints and the team is generally more productive.

You might be asking, if my project takes less than three sprints (six weeks in my world), is it worth using Scrum? If the small project involves less than five people I'd advise against it. Scrum is great for complex projects but it's not required for all of them.

I'll end with an example of one common mistake: The mid-sprint adjustment. Now, it's no problem to add stories to a sprint in progress. But it is a mistake to add work A) too soon or B) too often. The best sprint planning is to get the amount of work right in the planning meeting. But yes, I've done this:

Scrum Masters know that the burndown chart is their best friend. Review it at every stand-up meeting. Take it out to breakfast. But whatever you do, don't ignore it.

My last post discussed some basics for starting a sprint - in essence, how two meetings and a little process can dramatically clarify expectations and reduce workload.  Having said that, I see time and time again teams who adopt Scrum but build avoidable failures into their project before it even starts.

You can avoid pitfalls by following one simple rule: "When first using Scrum, start with the most formal process."  I recommend doing this for at least three sprints, but if you must do fewer, do it for at least one sprint..

Whatever you do, make sure that you:

• Don't skip the daily stand-up meeting.
  • It's hard to stress this enough.  In my career, this is the most common mistake people make from day zero.  Most projects I've worked on have had one weekly status/planning meeting.  An inexperienced lead will often say, "I like Scrum, but a daily meeting is too much to ask ."  One week later, half of the team is held up due to issues that could have been resolved with communication earlier in the week.
    
    Make no mistake: Scrum turns weekly progress into daily progress.  The daily scrum is the key to this rapid pace.
    
• Define your Scrum roles before the project starts.
  • Sometimes old school Project Leads make the mistake of carrying traditional project roles into Scrum.  They think that a Product Manager translates to Product Owner, and Project Lead is a Scrum Master.  I've found that Product Managers and Project Leads can be completely different from project to project.  Scrum roles are extremely well defined specifically to eliminate this variability.  It helps to start a project by introducing the person in each role, as well as what is expected of each role.
    
• Invest in tools.
  • Use the many free Scrum tools available.  The Scrum Team should be able to see all assigned tasks and their owners at any time.  The Scrum Master should produce a burndown chart and use it for trending.  One of the major benefits I've found in Scrum is the huge increase in visibility into the guts of the project.  Anyone can see the entire list of work to be done, the work we're doing for this sprint, how far we've gotten, and how much work each individual has produced.  They can do this because we use one tool instead of email, Word, and Excel.

Moving to Scrum from a traditional project lifecycle is a big change.  My advice is to make the biggest change you can, and only then remove the things that don't work for you.  Scrum is very flexible and accomodating, but, like medicine, it's most effective when taken all at once. 

Technorati tags: Scrum Scrum Process

The basic time interval for work in a Scrum is called a Sprint.  My team uses two-week sprints with formal meetings sprinkled throughout.  The formal Scrum meetings simply replace ad-hoc hallway conversations and those demos and design sessions that we "should have had".  They key is to increase transparency and avoid ad-hoc interruptions.

Once we have a product backlog defined we have a Sprint Planning meeting where the Product Owner identifies the most important stories to complete in the next two weeks.  The developers help by clarifying scope.  For example, if the customer wants a huge task done in the very first sprint the developers will flag it.  Then the entire Scrum team will break that task into smaller stories and those stories that can be finished in the first sprint get assigned.  During the sprint it's great to have the Product Owner come to our daily stand-up meeting to clarify any implementation questions.  But if the story is detailed and the task clear the interaction may be minimal.

Scenario: Customer wants full hologram support in first sprint.

Solution: Team will demo basic user interface to launch the hologram.

During the rest of the sprint we have two more regular meetings.  The Daily Scrum meeting is early in the morning, and yes, we do it every day.  It's no more than 15 minutes long and each developer discusses three things:

What was done yesterday.

What will be done today.

Any issues preventing work from being done.

It guarantees that everyone's on the same page.  It's a great place for raising issues and questions that allow people ot pair off after the meeting and resolve.  As a note, not every Scrum team in my org have a stand-up every day.  Beware of resistance to a daily Scrum, as it might points out a couple of problems.

Symptom 1: We don't have daily stand-up meetings because I'll just say the same thing I did yesterday.
Problem 1: The Sprint is too long, the current Story is too big, or the level of detail given isn't detailed enough.

Symptom 2: We don't have daily stand-up meetings because no one is dependent on another person on the team.
Problem 2: There is too much specialization or not enough backup on the team.

The bottom line here is to avoid meeting every week or two and realizing that days have been lost.  The daily Scrum serves as the heartbeat of your project.

The last meeting we have is a Sprint Review meeting on the last day of the sprint.  Ideally, all of the work is complete here as well.   During the review we demo the completed work to the customer, which is usually pretty fun.  I haven't experienced any shocked customers (yet) complaining that we've wasted two weeks - it's just too short a duration.

We hold a Sprint Retrospective and review the sprint itself: What went well, what we could have done better, and what we should change.  Everyone is asked, the Product Owner, Scrum Master, and Scrum team.  This is used to continually improve the scrum process.

Did you really make it this far?  Impressive!  Look, running a Sprint seems like a lot of work, coordination, and meetings.  But it isn't.  It's two meetings, one of which just happens every day.  Just follow this schedule:

1. Once, at the beginning of your project, hold the Product Planning meeting and define a backlog.
2. At the beginning of your Sprint and every two weeks after, hold your Sprint Planning meeting.
   1. Demo the work you've just completed.
   2. Hold a review of the last Sprint.
   3. Have the Product Owner define the work from the next Sprint.
3. Hold a Daily Scrum every day you don't have a Sprint Planning meeting.

Schedule other team functions, like design meetings, planning workshops, and company-sponsored lunches as needed.

I've been amazed how two recurring meetings have reduced my load as a technical project lead.  All of my emergency "how do I?" emails are usually addressed in the Daily Scrum.  The customer expectations are set every two weeks.  And when we demo, everyone knows where we are and how we got there.  So long, Waterfall!

Technorati tags: Scrum Scrum Process

So you're ready to kick off your first Scrum project and you ask the inevitable question - Where do I begin?

You start in the same place innumerable product teams have started before.  You gather requirements.  Lots of organizations gather all the requirements at once, building a pristine Product Requirements Document (PRD) that Can Never Change.  Scrum doesn't do this.  Instead, you can look at a requirements gathering as a good place to start, but it's not where you're going to finish.

The first thing you'll do is sit down with all of your well-defined Scrum roles: The Product Owner who is or is representing the customer.  The Scrum Master who will be guiding the project and shielding the team from the PO.  And the Scrum team itself - the developers and validators.  We call this the Product Planning meeting.  The expectation is that the PO is primarily responsible for the content of the meeting.  Either they've come with a list of requirements or are handing you a traditional PRD.  Each requirement gets discussed, prioritized, and a rough estimate on how long it will take to complete is attached.

In Scrum lingo, this turns out to mean we:

1. Create stories - Stories are individual requirements with well-defined acceptance criteria.  In other words, how can we demonstrate to the customer that we've met the requirement?  I also prefer tools that can strongly associate stories with the individual tasks needed to complete them.
   
   
   
   
2. Prioritize stories - We set an initial order of execution, subjective value, or Return on Investment calculation on the individual story.
   
   
   
   
3. Assign story points - We set a subjective "story size" on how long the work will take.  For my team this is a completely fictional number that evolved over time.  We tried to size stories by day: A five point story would take a work week to complete.  But somehow, given the people on the team, that turned into eight story points per week.  So these days we know we can finish sixteen story points in two weeks of time with the people we currently have.
   
   

If you're moving from Waterfall to Scrum and have a PRD with a hundred items in it, your Project Planning meeting will take a long time.  You'll probably have to break it out over a period of days.  In my experience translating a traditional PRD into Scrum is really interesting: You learn that most of the High priority items get done, but not so many of the Mediums and hardly any of the Lows.  It's another indication that static requirements up front don't work.

After your Project Planning is complete you'll have two things: A well-defined backlog and a team that understands all requirements for what they're coding.  I strive for common ownership on my teams, so the latter is really important.  The well-defined backlog lets you estimate how much time the project will take.  I've found that my initial estimates are pretty close to the real thing for projects under three months in duration.  Often I'll just say, "Okay, the project ends on [DATE], and we commit to completing all High priority items."  From there we can negotiate with the PO if there were Mediums that really should have been Highs, and vice-versa.  It's a liberating feeling to air all of that in the very first meeting: Clear requirements, realistic expectations, and a knowledgeable team.

What you're avoiding...

My last note is this: With lower priority projects, say for internal software, we've had a hard time getting a committed Product Owner.  Ask yourself - If my project doesn't have a single, dedicated owner, is it really worth doing?  I won't vent about invisible POs except to say that I think this is the most important role in Scrum.  You can't live without it.

Tasks accomplished: Roles defined, Backlog defined
Next up: Starting the Sprint

Technorati tags: Scrum Scrum Process

My last post was about deciding on what Scrum tools you'd be using before you jump into the process.  I imagine readers like myself who are new to Scrum or are constantly looking at ways to do it better.  So for those folks, I'll say that in the past three weeks I've learned, definitively, that the currently available Scrum tools won't force you into implementing Scrum more efficiently.  They're all good tools that require you, quite frankly, to know what you're doing.

So I'll try to break this down into three areas:

• Scrum Setup
  If you're just starting out with Scrum, by all means, use a whiteboard and Post-It notes.  Use whatever tools your most experienced Scrum team has been using.  Getting started and learning the process is the most important step - don't wait for that perfect tool.  I just wasted three weeks doing just that.
  
  Yes, there are tons of websites out there, but I have to say that the Scrum for Team System Process Guidance is my favourite place for information on actually implementing Scrum.  It's concise and covers everything.  Very nice.

• Enterprise Tools
  On a related note, my team has decided to go with Scrum for Team System as our enterprise-worthy 3rd party Scrum solution.  It's not as robust as our corporate internal tool and Visual Studio doesn't give a quick "at a glance" status of the guts of the project.  So we're waffling a little.  But the TFS templates are free and the web portal is beautiful.  We'll be going with it on a trial basis for a next-gen project.

• Informal Tools
  For sheer ease of use, I really like Scrum Edge.  It's super simple, quick, and extremely intuitive.  I can't recommend that anyone host information from their job on someone else's server.  But for projects that I'm personally implementing I'll be using it.

All of our tools investigations came from the posts of Boris Gloger, so I'll give a shout-out to him for his excellent tools reviews.  From here on out I'll detail how I run a Scrum project rather than talking about tools.  Tools are a bit of a black hole, the benefits of Scrum are totally independent of tools.  No matter what you're using, if your customers are happy, you're happy.

Next up: Product Planning Meeting

Technorati tags: Scrum Scrum Tools

if you're adopting Scrum you'll need to decide what level of tooling you'll use.  Are you going to use sticky notes on a whiteboard?  An expensive 3rd party solution?  My opinion is that If you're just learning Scrum it's worth the investment to learn the tools you're most likely to use long-term.

For general Agile software development I've been completely happy with Microsoft Team Foundation Server (TFS).  When you install TFS plus Visual Studio plus Team Explorer, you get the complete package of document repository, work item tracking, built-in unit testing, scheduled builds, email alerts, and more.  There's just one problem.  It doesn't natively support Scrum.

A hard-copy Scrum tracking process isn't going to do it for me.  In ten years at my current job I've never worked on a fully co-located team who's in the office every day.  Hard-copy planning and tracking absolutely works for some people, just see "Scrum and XP from the Trenches" by Henrik Kniberg [PDF].  But it's never going to work for me.

You'd think that Microsoft eScrum would come to the rescue.  From the site: "eScrum is a Web-based, end-to-end project management tool for Scrum built on the Microsoft Visual Studio Team Foundation Server platform."  Unfortunately it isn't working for us.  The UI is so unituitive - there's no way to view a simple hierarchy of Project\Product\Sprint\Stories\Tasks.  Just a couple levels of those would be nice, but I can only find queries against the entire backlog.  Our configurations has Areas and Iterations instead of Products and Sprints.  And since documentation on the web is quite sparse I'd need to sit down with an expert to flesh it all out.

Conchango is a company that offers a products called "Scrum for Team System".  It looks beautiful.  It's fully supported.  And it costs money.  Not an option for me at the moment (edit: see comments).

Many teams at work (including mine) are using an internally developed rich client application to implement Scrum.  But part of me wonders - are there really only four methods for implementing Scrum?  Hard-copy, Microsoft, Conchango, or roll your own?  It feels like I need to read up on the available Scrum tools.

Which leaves me to my recommendation.  If you have a small team that meets daily then implement Scrum the old-fashioned way with paper and marker.  If you have a team spread across multiple sites and time zones then it's your resources and budget that will decide what you do.  Honestly, I really don't want to use Sharepoint lists.

Technorati tags: Scrum Scrum Tools

So you've decided to try Scrum as your development methodology.  Good!  To start off on Scrum I relied primarily on these two documents:

1. "Scrum in Five Minutes" by Softhouse [PDF].  It really is what it claims: A simple, understandable guide to the roles, responsibilities, and processes defined by Scrum.  For my first few sprints and roadshows I would present this document first to get everyone speaking the same language.
   
2. "Scrum and XP from the Trenches" by Henrik Kniberg [PDF].  A 142-page ultra-detailed user guide on how Henrik implements Scrum.  A must-read for potential Scrum Masters or anyone wanting more detail on how Scrum works.  About six months after you've been implementing Scrum you'll want to read it again.

There are also key web sites to help you:

• The Scrum Alliance's overview called "What is Scrum?"
• Control Chaos is a site by Ken Schwaber, credited with inventing Scrum.
• As well, Mountain Goat Software is a site by Mike Cohn, a founding member of the Scrum Alliance.

Personally once I had read the documents, attended training, and implemented Scrum I found I didn't rely on the web much if at all.  It takes a lot of discipline just to keep the Scrum ceremonies going, and if you know what they are (such as the daily stand-up meeting) there's not a lot of mystery or refinement required.

My primary advice, though, is to make sure your team is adopting Scrum for the right reasons.

• Your requirements change often, usually near the end (or after the end) of the project.
• Your team has a hard time handling change at any stage of the project.
• All of the team's planning, scheduling, and velocity estimates are based on subjective data (like gut-feel).

And I do say "team" purposefully.  We found that a Scrum team under the recommended five person minimum is a lot of overhead per resource.  Due to attrition we're currently scraping by under the minimum, and as Scrum Master I can tell you it's more difficult to keep a few people at full capacity for a long duration than a larger team.  You tend to get tunnel vision and a bit burned out trying to maintain too high a velocity.

Set some goals for your Scrum adoption before you jump in.  Mine were to increase my team's agility (no more last-minute rewrites of fundamental components), predictability (knowing how much work we could do in a given time span), and visibility (where the customer always knew our status and plans).  Take some time to think about your problems.  If small iterations and a high-level of customer interaction won't solve them, don't adopt Scrum for that purpose.  I'm sorry, but no development process in the world can fix a development team whose primary problems are lack of skill, lack of motivation, or poor management.

Last but not least, when you first adopt Scrum my advice is to implement it "by the book".  Don't tweak it before you've tried it.  Do the daily stand-up meetings.  Slog it out through the initial, painful Product Backlog generation.  And do this for a few sprints.  If you don't like it after that by all means change it.  After all, Scrum is a flexible framework.  It's not like trying to swim up a waterfall.

Technorati tags: Scrum Agile