So I've read "Singleton I love you, but you're bringing me down" at http://www.codingwithoutcomments.com/ and the articles it refers to. And it makes sense, basically.
In my latest code, I created one Singleton (see http://geekswithblogs.net/AngelEyes/archive/2011/07/21/code-worth-remembering-singleton-implemented-in-c-4.0.aspx ) and had it hold the reference to my global factory, which, I guess, makes it a service locator. The factory itself uses (and hides from the rest of the code) Ninject 2.0. As long as that will stay the only Singleton, and hold no state other than the Kernel, then I'm safe :)