The Joel test is intended to gauge the quality of a software development organization. Ten years later, it is in need of an update. Below is a possible “New Joel Test”, updated for 2010.
1. Do you use source control?
1. Do you have a change management system?
There needs to be a configuration management plan, control of assets beyond “source code”, branching and merging strategy (even if it’s not to branch), release stabilization and deployment plan, security permissions and roles, quality gates for checking and/or merge, etc. Otherwise, “use source control” is just referring to a glorified backup system.
2. Can you make a build in one step?
2. Can everyone make a build in one step?
Individual developers need access to the automated build system. The official nightly build can “build everything”, but developers need to be able to build individual parts, pull together the required assets, and run from a representative staging area – maybe a virtual machine (or two), a network share, or just a folder on disk.
3. Do you make daily builds?
3. Do your daily build include automated tests?
It’s not enough to just build – we need an indication of build quality. “Break the build” should mean more than “does not compile”, it should also mean “does not pass the tests”.
4. Do you have a bug database?
4. Is work item tracking integrated with source control?
What artifacts were changed with a given bug fix? What bug or development task is associated with the most recent change-set? Which bugs are associated with a given merge?
Check-outs (or commits, depending on the source control system in use), must be associated with bug reports or development tasks. Each check-in should reflect changes for one task (unless they are coupled tasks). You should also be able to query, based on a bug report, which artifacts were changed (or ideally, currently in the process of being changed). In other words, change management must be integrated with bug tracking and project task tracking.
5. Do you fix bugs before writing new code?
5. Do you fix bugs and write new code?
A properly run project will have a branching / merging strategy (or tagging/labeling policy) that allows simultaneous bug fixes with new development. A quality organization will be able to correct bugs in previous releases, and ensure those corrections are in the current release. Without copying or duplicating the code base.
6. Do you have an up-to-date schedule?
6. Do you track progress and manage change?
Project tracking must include processes that allow for managing (planning / executing / tracking) change as well as tracking progress and periodic re-planning of the remaining work.
7. Do you have a spec?
7. Do you have a requirements management system?
The “spec” will change; that change needs to be managed. Too often I see organizations with an excellent process for development of the “spec” and a horrible process for managing change in the spec.
Note: In 2010, if you don’t have a “spec”, you automatically get a ZERO on the new Joel Test and there is no point in continuing on.
8. Do programmers have quiet working conditions?
8. Do programmers have quiet working conditions and teaming rooms?
The teaming rooms are more than a room with a white-board. There is a conference phone, computers (more than one), projector, and lots of white boards. Code reviews, code walk-through, mentoring, design, dispute resolution, meeting with customers – can all take place here.
9. Do you use the best tools money can buy?
No change needed. This one is timeless.
10. Do you have testers?
10. Are your testers involved in requirements management?
The test team needs to be involved in the requirements management system as well as the product validation process. Tests are executed against the requirements. The testers should have authority to approve/reject a deployment as well as approve or reject a requirement. Otherwise, they’re just testers.
11. Do new candidates write code during their interview?
11. Do new candidates review code during their interview?
“Writing code” is one thing – understanding code is another. Candidates should be able to answer questions like “is this code thread-safe?”, “is the code well-commented?”, or “Is the implementation appropriate” – and why. Printing fizz-buzz is one thing; being able to appropriately critique a page of code is another. As a bonus, you could always ask them to re-write it and get the best of both.
12. Do you do hallway usability testing?
Huh? I’ll have to go and read the original on this one, again. I will leave it alone here.
* * * * *
The Joel Test updated for the year 2010. And a few more months to get it right.
posted on Tuesday, June 16, 2009 4:23 AM
