Dependency Management

Remi Sabourin — Thu, 24 Jul 2008 07:09:32 GMT

The Challenge

Imagine a development shop that maintains 100 .Net solutions, each composed of multiple projects. Imagine roughly half of these being shared domain services or shared common libraries, the other half being end web applications servicing multiple large clients. Imagine any given project having the ability to reference classes/methods on any suitable re-usable service. Imagine trying to get a grip on it all - a great architectural paradise or a great architectural nightmare. Welcome to my world!

For a while I have been thinking about a way to manage dependencies. Dependencies hold us back, they make us scared to change things. But when we don't evolve, we don't improve, we spend time doing complete rewrites or adding layer over layer of code bloat and we don't refactor to achieve enough of my favorite thing..... (c'mon guys, you know it's coming).... RE-USE.

I am dreaming of a searchable database of dependencies, most it being automatically generated and updated, although some could be specified manually if indirect. Basically a master tool that would spider our build and production server(s) and document dependencies to any artifact - code, databases, folders, etc. If we want to remove/update/rename anything, we can be confident in our actions. It would point out orphans that we can clean up so as to keep our systems/code clean. It would plug into an automated build process and send dependency reports and alerts.

The Research

Voice inside my head says "Start with the code". I want to know for every method/property what method/property calls it in any project. Here's what has transpired so far:

First I figure a little app that calls the .Net framework could be in order, that spits the results to a database. I know System.Reflection has metadata on dlls built in. On further inspection, System.Reflection only exposes things in the manifest, which does include assembly references (a start), but does not provide an insight into method references. Next I look at the CodeDom namespace - if similar to XML DOM, I could create a solution to parse the source code and I could iterate the dom to find function calls. In fact, there is a CreateParser() method - too bad for me there's no implementation. Maybe in a future .Net version?

I took a look at NRefactory that is an API that comes with the SharpDevelop IDE but can't find much documentation on this to see if it could do the job.

Examined some out-of-the-box add-ins/tools, NDepends being a possibility. I figured I would have an automated build output an XML file from NDepends and then I could have code that parses it for the dependencies and puts them in a database.. NDepends looks like a good tool with some interesting features and a query language, but it does not output method dependencies (only type and assembly ones), although you can see them if you drill down using the application. There is also licensing per individual.

I thought dlls could maybe be reversed engineered; there is a tool that comes with .Net called ildasm.exe. Its output though is intermediate language code which I'm not familiar with and would be kindof brutal to parse. I found out Reflector has a "hidden" API if you reference the EXE. A nice MSDN article that has a promising name by James McCaffrey: http://msdn.microsoft.com/en-us/magazine/cc163641.aspx. May try this out, does not seem like the perfect high-level API but may not be too complicated knowing what exact statements to look for in intermediate language.

Then found out that FxCop also has a "hidden" API if you reference the EXE. Here's a nice doc Jason Kresowaty created to document this API: http://www.binarycoder.net/fxcop/pdf/fxcop.pdf. This looks like the abstract API I was looking for. This is my best bet right now.

The Execution

Concluding my research, my goal in the next few weeks is to:

- Use System.Reflection to log all the assembly references found in dlls

- Use FxCop or Reflector API to log all the method references found in dlls

- Log all the source code in the database (if this is impractical, use indexing service) so I can query it. What the hey, why not use SQL Server's scptxfr.exe utility and log all the stored procs/views/triggers too.

- Have a GUI for the development teams that displays dependencies between assemblies and allows you to drill down to methods. Have GUI that allows you to search for ad hoc strings in the source code..

I'll let you know how it turns out.

P.S. If you know of some wicked tool that can do this for me and allow me to RE-USE and conserve time, let me know!

Cool tools

Dependency Management