Systems Architecture and Design

Migrating SharePoint Content to Oracle UCM #2

Adam McKee — Tue, 20 Oct 2009 00:39:08 GMT

So we landed on a solution. It's not ideal by any stretch, but it does accomplish the purposes we set out to meet, and it does give us a (small) step in the right direction in terms of moving the client to using Oracle UCM as the sole company-wide content repository.

To recap, here's the skinny on what we're facing:

There is a mandate in the company to move all "content" to Oracle UCM (a content unification measure)
There is a significant amount of content in SharePoint (in the form of lists) that needs to make its way into Oracle UCM
Business users do not want to stop using the SharePoint interface for data entry (not to mention get all the "for-free" stuff that comes with SharePoint, such as alerts, content approval mechanisms, permissions, and workflow).
It is not feasible to move all of the lists over at once (we have a 3-week scope to move "something" over from SharePoint to UCM)

We came up with two approaches that had varying payoffs and risks.

The Simple Approach

Seeing how it was very important to keep all of the existing functionality in SharePoint in whatever solution we devised, we first thought of "replicating" list content to the UCM from SharePoint. This approach involved writing a custom workflow that would fire on list item update, and would intelligently call a checkin function in UCM.

This "simple" solution wouldn't come without risks - the first (and most obvious) one being that we'd have duplicated data that needed to stay in sync. After thinking about the implications, we found that this risk wasn't terribly bad. Since no content contribution happens on the UCM-side with this implementation, there's no harm if SharePoint goes down - there just wouldn't be any new data. If the UCM goes down, syncing the data would mean saving the items again (a solution acceptable by the client).
The UCM-Heavy Approach

This approach was quite a bit more aggressive in terms of planning, execution, and support, but yielded a much bigger step towards migrating the list content from SharePoint to UCM.

The plan was to "switch out" standard SharePoint CrUD (and other) functionality for custom functionality that manipulated UCM content directly. The idea was that all point-foward content would be manipulated directly in Oracle UCM through the SharePoint interface. We felt that if we could nail this approach that we'd set a good proof-of-concept for migrating other content to UCM from SharePoint.

This approach was monstrously invasive. It involved re-writing a lot of stuff that comes for free with SharePoint, and was therefore expensive and risky. However, the payoff of making such a big step towards UCM would be a huge win for the client.

The Accepted Solution

It will probably come as no surprise to many of you that we landed on solution #1. What killed solution #2 was that UCM doesn't have an OOB way to handle broadcasting alerts based on permissions (on the item level) when content was created or updated. Sure, we could write a workflow to sorta-kinda handle it in UCM, but we didn't have the time or budget for it.

Next time, I'll tell you a little about how we're going about implementing solution #1, and what sorts of challenges are cropping up.

wsdl.exe - Unable to import binding from namespace

Adam McKee — Fri, 16 Oct 2009 17:53:55 GMT

When trying to generate an ASP.NET proxy class from a physical wsdl on your local file system, you're probably doing something like this:

wsdl.exe MySevice.wsdl /o:Proxy.cs /l:CS

You'd expect a file named Proxy.cs to get created in C# for you. You'd be right, except if you're attempting to generate proxy for a service that uses complex types. If that is the case, you'll get an error that looks a lot like this:

Error: Unable to import binding 'MyBinding' from namespace 'http://www.example.org/MyNameSpace/'.
- Unable to import operation 'MyOperation'.
- The element 'http://mydomain.com:MyComplexType' is missing.

If you would like more help, please type "wsdl /?".

This error is basically saying that you your service is expecting one or more XSD file containing the definitions for those complex types. Here's what your command line should look like:

wsdl.exe MySevice.wsdl MyServiceDefinitions.xsd /o:Proxy.cs /l:CS

If you need to add additional XSD files for other complex type definitions, just add them to the command line like you did for the first one, separated by a space (order doesn't matter).

Migrating SharePoint Content to Oracle UCM

Adam McKee — Wed, 07 Oct 2009 03:21:56 GMT

I’ve got an interesting challenge a client has asked me help with. We are attempting to move items from multiple custom SharePoint lists to Oracle UCM and continue to manage/review/approve the data through the SharePoint interface.

Moving this list content to OUCM is part of a bigger strategy to centralize all of the “business truth” (content) to OUCM, while keeping SharePoint for collaboration. There are a lot of things to consider, and lots of risks to document.

I'm just now getting to documenting the state of the system, so I'll share some challenges and insights we've come across when we've landed on an approach - should be fun :)

Design Patterns That Get Implemented in Real Life

Adam McKee — Tue, 06 Oct 2009 01:40:21 GMT

I think that the Gang of Four should consider my design pattern recommendations for inclusion in their codified list. And while I have seen ALL of these patterns shipped to production in real life, it doesn't mean that these are by any means the only ones that scream for inclusion. Anyone got any others?

The Duct Tape Pattern

Its versatility is unmatched – any two bits of code from any two systems can co-operate with the use of this simple pattern. Need a piece of data from an XML document to display on your page? No sweat – the Duct Tape Pattern has an implementation that allows you to take data directly from your XML document and slap it straight on the page. Need to update meta data on a user profile? Easy as pie – the Duct Tape Pattern affords you the simplicity of running an in-line SQL update statement right in your button click event handler. Need I say more?

Arguments for Inclusion:

When the crap hits the fan, this pattern is used more often (and more consistently) than any other recognized design pattern.
If you’re using this pattern, your code isn’t going to be readable, much less maintainable – it’s guaranteed job security.

The Crayola Pattern

Many people confuse the Crayola Pattern for the Facade Pattern, yet they are very different. While the Facade Pattern attempts to provide a simplified interface to a larger body of code, the Crayola Pattern outright ignores complexity by making wide-sweeping assumptions about the intended usage of the system, thereby making programming against it as simple as coloring with crayons (but you only get to do as many things as you have methods). The Crayola Pattern is especially effective if you reduce the entire system to 8 method calls, but 64 is also an acceptable practice if you include a UseSharpenerOnBack(object) method in your pattern implementation.

Arguments for Inclusion:

Using this pattern will guarantee those who use it to get code comments such as:

/* we don’t know what this does, but the web server dies when we touch it */

Who doesn’t want to write code so sophisticated that nobody else can understand it? It's a badge of honor to be smarter than everyone else.

The Clipboard Pattern

This is the pattern by which a programmer takes an existing class, copies it to the clipboard, pastes it to a new code file, renames the class, adds one or more properties/methods, and seals the class. The intent is to make your classes utterly unusable for anything other than the use you originally intended, and utterly bewilder other developers as to why you did it. The maniacal laugh value never ends!

Arguments for inclusion:

Using this pattern adds a LOT of code overhead, and may very well lead to using the Crayola Pattern, which will inevitably lead to using the Duct Tape pattern, which we all know leads to job security.
Clipboard Pattern implenters will get to wear out your Ctrl, C, and V keys even faster than your W, S, A, and D keys!

The Shippit Pattern

This is the pattern whereby a programmer copies a block of code from the Internet, pastes it into his code, tweaks it a little, and ships it out to production in one fell swoop. This revolutionary design pattern has been taking the programming world by storm, slashing costs and boosting productivity across many industry verticals.

Arguments for Inclusion:

Saves literally thousands of dollars in development and testing time by simply not doing them.
Implementers get to blame someone else for the system not working because they didn't write the code - they simply implemented the Shippit Pattern, and thereby save face when the client can't use the system.

Developers and Designers – A House Divided

Adam McKee — Mon, 20 Jul 2009 04:20:34 GMT

As a web developer, I find myself having the same conversation over and over with HTML designers when it comes to agreeing on how to style common markup elements. The development cycle I experience usually ends up looking like this:

1. Requirements are agreed upon, and handed to the project team.

2. The PM tells the HTML designers make wire frames, screen comps, and they get approved.

3. Everyone codes frantically – HTML designers are making markup while coders are making functionality.

4. Coders finish a chunk of functionality, and match up the HTML they are rendering against the HTML that the designer has mocked up, only to find huge differences.

5. Frustration and refactoring on both sides of the fence ensue.

While the disconnection between steps 4 and 5 may seem pretty obvious (and may even be lessened by careful planning and communication), it happens frequently because of two immutable aspects of developers and designers:

1. Most HTML designers don’t know (or don’t care) about the hoops that developers have to jump through to get HTML to appear on the screen.

2. Unless you make styling a requirement for a deliverable, your developer is not going to even think about styling until the code at least performs the functions required.

I believe that the best way to address this problem is to cross-educate. Developers should spend some time becoming savvy on the practices and principles that their designers employ. Designers should become aware of how the developer’s programming platform generates common elements (such as how data repeaters will generate tables, how controls are dynamically named based on their containers, etc.).

Service Architecture Using Spring.NET

Adam McKee — Mon, 06 Jul 2009 03:14:36 GMT

If you’ve ever written a standard .NET ASMX web service before, you know the drill. You make your service, build out your web methods, publish your service, make your consuming web project, point your project to your service via a web reference, and voila! You get strongly-typed proxy classes auto-generated for your web service consumption pleasure… But this comes at a great cost. WSDL is slow, clunky, and bloated. All that proxy code that is generated results in serializing, packaging, unpacking, and de-serializing data each way (not to mention the ludicrous amount of code generated to perform relatively simple operations).

A better approach, I think, is to create a web service architecture that gleans the best aspects of REST, and combine it with Spring.NET on the back-end to allow simple functionality extension. And that is exactly what I intend to impart to you - a practical case study in doing exactly that.

Before I go any further, I want to talk briefly about the scope of what I'm trying to demonstrate. This post is not intended to show you how dependency injection works, or even how to roll your own service architecture. What I want to do is to walk you through a super-lightweight service implementation that we've created based on the dependency injection that comes with Spring.NET. Yeah, I know, there are a million things that Spring.NET does, but I’m mostly interested in talking about dependency injection and one really practical use for it.

I’m Too RESTY For My Shirt

So, what made us choose a REST-ful service implementation? Well, a few reasons:

We’re interested in REST because of how the service call allows us to very simply extract what action is to be performed or what data is to be returned by the very nature of the call, as opposed to unpacking a SOAP semi truck just to retrieve a toothpick of instruction.
REST service calls can very easily be constructed either from JavaScript or a server-side function, meaning that AJAX (or even server-side) calls can become very simple, and uniform.
You don't have to re-build a ginormous proxy class every time you change the functionality of your service - just change the query string on the service call, and away you go.

Before I dive in to how all of this marries up with Spring.NET, I feel that I should mention that I'm not using true REST – I'm only using the parts that make sense for the service implementation that I'm working with. In particular, I'm only interested in GET (for fetching stuff) and POST (for doing stuff) verbs. Conceivably, we could write actions for other common verbs, but you’ll see why we chose against it when we bring Spring.NET into the mix.

Is That a Spring in Your Code, or Are You Just Happy to See Me?

The purpose of using dependency injection in our service implementation serves two main purposes:

To segregate and VERY loosely couple the different functions (verbs, resources, and response types) of our REST-ful service
To make it easy to extend the functionality of the service to accommodate future requirements

Binding the Service Together

Before we dive into the code, there is a bit of terminology that I need to bring to your attention. In the code, you'll see frequent reference to an object called a "binder." This binder object is nothing more than a means of passing relevant data to and from various layers of the service. It has the following members:

string Format - how the output of the service call is to be formatted (JSON, XML, etc.)
string Resource - dictates what resource is to be queried (labor, employees, parts, etc.)
string Action - what is to be done to the resource (getAll, getById, getSomethingMeaningful, etc.)
Dictionary<string, object> LocalData - Holds any single values that need to passed from one layer to the next that doesn't fall into one of the above properties
DataSet ResultSets - any recordsets to be passed to the response handler to be translated to JSON, XML, or whatever is dictated by the Format property.
HttpContext Context - object reference to the current HttpContext other layers of the service can see it without having to trying to fetch the context themselves.

Now that we've cleared that up, let's explore some code to see what the service is really going to do when it gets a request.

The Service, Top-to-Bottom

Here's what a request to the service might look light:

http://svc.mycomany.com/servicehandler.ashx?resource=labor&action=get&id=1

What this does is tell the service everything it needs to know to fetch a results set. A results set might be a scalar value, or an entire recordset. In this case, the expected result will be a single labor record whose Id is We'll be following this example throughout the rest of this tutorial.

   public class ServiceHandler : IHttpHandler
   {
        public void ProcessRequest(HttpContext context)
        {
            // Create an instance of the Spring.NET application context
            IApplicationContext ctx = ContextRegistry.GetContext();

            // Create a binder object
            IBinder binder = (IBinder)ctx.GetObject("binder");
            binder.Context = context;

            // Default to JSON format (response)
            string format = "json";

            try
            {
                string resource = null;
                string action = null;

                HttpRequest request = context.Request;
                format = request["Format"];
                resource = request["Resource"];
                action = request["Action"];

                // An error should be thrown if no resource is provided in the service call
                if (Resource == null)
                {
                    throw new Exception("Resource was not found");
                }

                // An error should be thrown if no action is provided in the service call
                if (Action == null)
                {
                    throw new Exception("Action was not found");
                }

                binder.Format = format;
                binder.Resource = resource;
                binder.Action = action;

                // Call Spring.NET to get the appropriate request handler, based on the named passed in from the query string
                IServiceRequestHandler requestHander = (IServiceRequestHandler)ctx.GetObject(resource.ToLower() + "requesthandler");

                // Call the appropriate IServiceRequestHandler method, based on the RequestType of the call to the service
                switch (request.RequestType.ToLower())
                {
                    case "get":
                        requestHander.Get(binder);
                        break;
                    case "post":
                        requestHander.Post(binder);
                        break;
                }

            }
            catch (Exception e)
            {
                binder.SetLocal("error", e.Message);
            }

IServiceResponseHandler responseHandler = (IServiceResponseHandler)ctx.GetObject(Format.ToLower() + "responsehandler");

            responseHandler.binder = binder;
            responseHandler.write();
        }

        public bool IsReusable
        {
            get
            {
                return false;
            }
        }
    }

The service handler is designed to digest any call to servicehandler.ashx, break apart its request parameters, and determine what to do. After it breaks down the request and creates a binder, it calls Spring.NET to get an IRequestHandler based on the name of the resource.

Let's take a look at what the LaborRequestHandler class looks like:

public class LaborRequestHandler : IServiceRequestHandler
{

        public IBinder Get(IBinder binder)
        {
            IApplicationContext ctx = ContextRegistry.GetContext();
            IServiceAction action = (IServiceAction)ctx.GetObject(binder.Action.ToLower() + binder.Resource.ToLower() + "serviceaction");
            action.execute(binder);

return binder;
}

        public IBinder Post(IBinder binder)
        {
            IApplicationContext ctx = ContextRegistry.GetContext();
            IServiceAction action = (IServiceAction)ctx.GetObject(binder.Action.ToLower() + binder.Resource.ToLower() + "serviceaction");
            action.execute(binder);

            return binder;
        }
    }

By the time this class is instanced by the ServiceHandler, the ServiceHandler has determined the RequestType (GET or POST), and has determined that the Labor resource is necessary, and has instanced the proper IServiceRequestHandler (LaborRequestHandler, in this case). In this class, the action taken will be resolved down to an actual IServiceAction implementation, which then gets passed the binder instance, and executed.

You'll notice that both the Get and Post methods are identical - this is circumstantial. In this case, it happens that no further binder processing is necessary at this point, but this may not always be the case. This is why we've added this layer of abstraction into the mix, to allow for request type-specific processing at this level.

Note: The execute method returns a Binder - the implementation of the execute method on the IServiceAction will put any results into the binder, and pass it back up to the ServiceHandler for formatting into the output for the original service call from the client.

Here's what the GetLaborServiceAction looks like:

    internal class GetLaborServiceAction : IServiceAction
    {
        public GetLaborServiceAction()
        {
        }

        public bool execute(IBinder binder)
        {
            try
            {
                DataTable results = null;

// Your code here to fetch single labor record into the results DataTable

binder.addResultSet("Labor", results);

                return true;
            }
            catch (Exception e)
            {
                // Perhaps some logging here
                throw e;
            }
        }
    }

Once the database code has been executed and put back into the binder, the ServiceHandler will then instance the proper IServiceResponseHandler (JSONResponseHandler in this case), and call the write method. The write method will take the binder that was populated with data, and convert it into the appropriate format (JSON, in this circumstance).

The Beauty of It All

Now that you've seen how it all works, let's take a step back and think through some of the implications of using DI in this service.

Let's say that you have a requirement for a new, specialized get method - one that gets all Labor rates that are only available on Tuesdays. Here's all you'd have to do:

Write your stored procedure to actually fetch the data
Create a GetLaborRatesForTuesdays : IServiceAction class, and flesh out the execute method to call the stored procedure
Update your Spring.NET configuration to accommodate the new class

POOF! You're done! Your service has entirely new functionality, and you didn't have to touch any existing code. Using this methodology, you could also create IServiceRequestHandler implementations for other verbs, but that's up to you. We didn't use them here because we would end up with a lot of classes that had identical code, and it would mean more complex code on the client-side to know which verb to call - it was a lot easier just to use GET and POST on all fronts.

Speaking of client code, here's the best part: Your client code doesn't have to change one bit to accommodate this new functionality. In order to call this new method, all you have to do is slightly alter your query string to look like this:

http://svc.mycomany.com/servicehandler.ashx?resource=labor&action=getLaborForTuesdays

That's it! Purty, eh?

So, Adam, How Do We Wrap This One?

Without a shadow of a doubt, bust... oh wait, you mean me, not Adam Savage.

This is just one way to go about doing this. The implications go way deep and way wide - there are lots of ways to make this even better. For example, there's an entire DAO layer that I left out of this implementation - I felt it would have added a bit of complexity that would have detracted from the scope I was trying to maintain.

I'll be posting other thoughts I have on the matter as I come across them - there is so much here to learn.

Lessons Learned From Making SharePoint Recommendations

Adam McKee — Wed, 22 Apr 2009 10:24:05 GMT

I was recently asked what lessons I've learned when recommending SharePoint for for meeting a business need. While books are written on this topic, I've distilled the list down to what I think are the core of the decision-making process.

Know what your stakeholders require in objective, actionable terms

This is known as extracting core business drivers. Note that I didn't say requirements. Requirements are what you hand to your developers - core business drivers are what you use to make your recommendation.

Right questions to ask at this time:
- How does process x drive revenue?
- How much productivity loss constitutes a bottleneck in x process?
- How much must efficiency increase for x process to meet your defined goals?

Wrong questions to ask at this time:
- What problems do you have with your current system?
- What do you want your new system to look like?
- How do you want the system to function?

Note that the difference between the right and wrong questions is objectivity vs. subjectivity. If you can't get a quantifyable answer, the question should never be asked. And in reality, you should be answering those wrong questions yourself once you have digested the business drivers.
Map the core business drivers up to things that SharePoint does well

Another way to say this is, “Don't try to make SharePoint do something it's not designed to do.” This concept is simple, but it's very hard to establish effectively because SharePoint does so many blasted different things.

It's at this point where you'd decide whether using WSS or a full-blown MOSS implementation will fit the bill.

Be careful not to get into technical debt when mapping things up. If you've never used Form Services before, you'd better be darn sure it's the right fit for the job.

Every SharePoint feature you decide to implement should correlate to a core business driver. If you stick to this principle, you will avoid a lot of tehnical debt.

Just because SharePoint can do something doesn’t mean that you have to use it. Not every solution has a problem, so don't expect to use the entire of the SharePoint canon of functionality.
Decide whether the benefits outweigh the costs

Even if all of your quantified core business drivers are covered by your recommendation, there are a lot of unquantifiable things that will crop up. I'm talking about things like the personalities/predispositions of the stakeholders toward SharePoint, to the adoption propensity of the user base.

Rules are FAR more important than tools. Be smart on where governance is required to keep your SharePoint instance humming (even more books have been written on this matter, so I’m not going to get into it now), and build it into your proposal.

When not to use exception handling, or, "Don't try too hard"

Adam McKee — Sat, 28 Feb 2009 02:17:07 GMT

Okay, people. I officially deem it OK to remove someone's geek cred if they put a try block around the entire of a method body. Furthermore, you get to remove even more geek cred if they just re-throw the error that was caused in the try block. Here's an example of what I'm talking about:

private void foo()
{
   try
   {
      // do the foo.
      // do more foo.
      // do the rest of the foo.
   }
   catch (Exception ex)
   {
      throw ex;
   }
}

Contrary to popular perception, try/catch blocks are not for preventing errors - they are for directing logic flow. A try/catch block is sometimes treated with special reverence because it revolves around "errors that happen at runtime," and "errors" mean "bugs." It's an emotional trap everyone falls into - I've even seen project plans that have time allotted for "adding exception handling," as if exception handling were some magic bullet that somehow prevents bugs.

True, when properly implemented, exception handling does reduce the number of "bugs" in the system, but not due to their perceived inherent nature - they reduce bugs the same way adding an "if" construct or a "switch" construct in the correct place will reduce bugs.

It is with this understanding that I believe that try/catch should be treated no differently than any other programming/logic construct. Would you surround the entire body of a method with an if statement? If you answered yes, you need to re-think your code.

Here are some rules:

Catch only errors for which you plan to take additional action at that point in time.
Use a try block only around the code that will cause a fork in logic were an exception to occur.

DON'T

Instance the exception object unless you actually need it. If catching a type of error is good enough, then don't instance the exception object.
Catch errors because you think it will "help out somehow." Doing so runs the risk of swallowing errors (which make troubleshooting impossible), and adds unnecessary complexity.
Re-throw the error that just occurred unless the up-stream code requires it for logging, or any other specific actions. Re-throwing is expensive, and should be used sparingly.

97 Things Every Software Architect Should Know

Adam McKee — Fri, 27 Feb 2009 06:05:33 GMT

A co-worker shared this gem with me, and every bit of it is true. This is a must-read for aspiring architects.

http://97-things.near-time.net/wiki/97-things-every-software-architect-should-know-the-book

One thing that I really appreciate about the articles on this site is that they draw a clear distinction between an architect and a senior developer who calls himself an architect. While many people feel that a certain number of years of programming experience entitle you to architect status, programmers and archtitects speak different languages and use different tools to create their deliverables, and are responsible to different types of people.

There is a lot to be said here that the site says far better than I can. I highly recommend that you read it, absorb it, live it.

I'll also leave you with some food for thought: What would you call the key qualities that distinguish a true architect from a senior programmer who calls himself an architect? Feel free to drop comments if you like - I'd love to hear your thoughts.

Systems Design and Poker Odds

Adam McKee — Sun, 08 Feb 2009 09:15:03 GMT

Deciding whether to implement a new, unproven widget to save in development costs is a lot like deciding whether to call a player who goes all-in on the first hand of a poker game. You may land your cards, but is it really worth it?

The way to determine whether or not to take the risk with an unproven widget is a lot like how a professional would assess the scenario I just described.

An accomplished poker player knows what each hand is worth, not only in terms of the potential winnings, but in terms of how his invested chips will sway who has what equity in the game.

This same accomplished poker player will tell you that a big, early chip lead doesn't amount to anything worthwhile. In fact, if you were to double up on the first hand of the game (implying that you eliminated another player to do so), you'd end up with far less equity in the game than the face value of chips you have in front of you. This loss of equity occurs because someone being eliminated this early in the game is far more valuable to all players than the amount of chips you won by eliminating him. You won his chips, but everyone else won the benefit of having a player eliminated, and they didn't have to risk the farm to do it. Chip-for-dollar, your chips are now worth far less than everyone else's.

Now, let's apply this idea to systems design. When your tech lead promises that a widget will make the development life-cycle easier, it's always at the cost of configuration, maintenance, and/or other no-see-ums that add up very quickly come delivery time (and to top it off, they are never documented adequately, if at all). In other words, since he can't account for the the total cost of ownership (TCO) of the widget when he proposes it (hence it being unproven), he would accept the widget as a viable solution based solely on its perceived benefits without regard for the no-see-ums he'll have to wrangle.

"Saving" on development time with an unproven widget is a lot like winning that huge pot up-front. While you save in development, someone will pay for the ownership of something duct-taped together, and if you're a vendor, it will inevitably be you.

So if you are going to take a risk with an unproven widget, do yourself a favor and build an inordinate amount of time into your project plan. I mean on a factor of 4x of how fast you think you'd get it done if you were stoned, and had a mountain of cheesy poofs that demanded to be eaten.