There's a lot here in Scott Guthrie's blog about improving your build performance. He talks not only about how to improve your build times, but also on how to decide whether to use Web Application Projects or Web Site Projects and what the differences are between them. There are several useful nuggest and links here for any kind of ASP.NET Web site build issues.
As they say, read the whole thing.