It is quite easy to say that the IT world is a diverse world. For instance, it is quite rare that you would find an enterprise that is built upon a single vendor’s technology stack. Instead, you most likely find that a company is made up of a patchwork of systems. Some of the systems are based upon UNIX, some Microsoft, and some others (sometimes items that are considered legacy).
A company’s software and data repositories are something that grows quite organically over time. Usually you will find that a company’s systems not only grow organically, but they also start and finish at different times and under different editions of a company’s specifications. One year a company might want everything based upon Oracle, while the next (usually coming with a new wave of management) the company might want a company’s groups to build with a Microsoft SQL Server backend.
Another problem you will find in larger organizations is fiefdom. Groups are divided by divisions, vertical groups, horizontal groups, internal systems, or external systems. Your organization might be made up of numerous groups found in quite different parts of the world. If this is the case, you are generally going to find that many of these groups tend to work independently and under their own guidance.
If this sounds familiar, then you are in need of getting your firm to work towards an SOA, also known as a service-oriented architecture.
It is no doubt that you have seen the term SOA plastered almost everywhere these days. From magazines, to blogs, to books and more, you will find people are all buzzing about this new way of designing your solutions. With that said, what does it really mean?
You will find a lot of articles out there that define the reasoning for moving to a service-oriented architecture, but you will find very little out there that talks about how you should go about taking your first steps into this new world. You will also find many companies out there that are offering solutions that they say provide you with the SOA that you are looking for in your organization. What they are offering are software solutions whether that might be an enterprise service bus or something else. Let me just say, that there is no magic bullet to getting an SOA going. Instead, I will promote to you a combination of things that you can do to get you on your way to building an SOA that makes sense specifically for your situation.
Some of the steps might work for you and your organization, while others may not. It is not mandatory that all the items defined in this document are followed through to the letter, but from my years of experiences in getting large organizations with disparate systems to work together – these are the steps that helped me on my way.
Doing this is good for your business. You want the data and services that you firm offers internally or externally to make it more easily out of their silos so that they are more quickly and more intelligently accessed by the people, groups, or systems that need it.
Now, onto the steps you can take to bring your company to an SOA.
STEP 1 – Focus on Message-Based Activities
The important point of disparate systems is that they are required to communicate by requesting information as well as providing information or services themselves. These communications happen over protocols within a company or out there on public networks.
In building the processes of communicating messages, it is important to remember that your messages have to deal with the same issues as any program you develop. This means that you most likely have to deal with an authentication and authorization process, managing state, encryption, compression, and more. To help you in this process and make your messages as reachable as possible, a fundamental idea to understand is that messages and all the activities they perform need to occur and be message-based rather than protocol- or hardware- based.
This means instead of relying on HTTP to provide any steps related to the authentication and authorization process, routing, or encryptions – that you look to see how you can accomplish the same tasks through some type of encapsulation in the message itself. This gives you a lot of power if you choose to follow this important principle.
This means that instead of locking down your services by using SSL (thereby using HTTPS), you put the security aspects of your process in the message itself. This allows the security that needs to be applied to be contained within the message, thereby allowing the message to route and be placed on any protocol you want.
The first and most important gain in this is in the ability to transcend protocols as messages flow through the organization or even out to our customers and partners. This allows you to move messages from HTTP to TCP or even to proprietary back-end protocols as a message routes through a system. This also allows messages to be stored in some fashion – also storing a message’s metadata, security, encryptions, and more. There is too much power in this model. If you follow this one important step, this adopted model will cause many other questions to fall into place along specific paths.
STEP 2 – Dependency on SOAP
It should be stated that a fundamental structure of your messages you should agree upon is SOAP. There are many people in the world that say they building web services, but when you start talking with them, you find out that they are using proprietary XML and sending that over HTTP – nothing more. Instead, you will want to use a standard in how your messages look. This means using a defined schema and there is not a better schema in the market at this point in time than SOAP. However, it should be pointed out that there are other schemas out there that might suit your company’s needs. Currently the only one of any attraction in REST.
When using SOAP, you also need to commit to use the two defined areas of the SOAP envelope correctly. This means that the SOAP body can only contain items that directly relate to the message consumption – input parameters and output values. The other part of the SOAP message, the SOAP header, is the area of the SOAP message for any metadata pertaining to the message. This means that the SOAP header is the place to store security credentials, routing, encryptions, timestamps, digital signing management, and more.
The other item to make note of is the version of SOAP that is used. Presently, a majority of the world is working with SOAP 1.1, while some of the world's vendors have started offering SOAP 1.2 capabilities. All interfaces within your organization should (in the minimum) provide a SOAP 1.1 interface for quite some time to come. However, this requirement does not stop the ability for a development team to also provide a SOAP 1.2 interface. This only means that in addition to the SOAP 1.2 interface – a SOAP 1.1 interface will also need to be present. It should be noted that building web services in a Microsoft development environment would provide both a SOAP 1.1 and 1.2 interface automatically (unless it is turned off through configuration).
Another interesting note is that the future Windows Communication Foundation (WCF) does easily allow you to build a service once and make that service available via HTTP and TCP as SOAP through configuration.
STEP 3 – Security for Services
Security is always the top of everyone’s list as far as what should be dealt with in regards to a service-oriented architecture. Security in what I discuss here is to mean the ability to provide a user context (a set of credentials or token that represents a user for a set period), digital signatures, in addition to content encryption.
When looking at the security of web services within your organization – there should be two models – one for internal services and another for external services.
Internal services are services that are only offered within your organization and never make it out to the customer. These services are not offered over the public Internet and are always considered to be on a trusted network. This means that credentials are not in many cases required to be fully encrypted and therefore allow for the maximum in performance.
On the other hand, external services are services that are offered to clients over the public Internet and therefore the highest levels of security are offered to your customers. Though slower in process, security trumps performance in many cases
In focusing on internal services, the big question is whether to use WS-Security or not. There are good arguments for using WS-Security and, in my opinion, some arguments not to. If your organization is in control of both endpoints, there is little reason to use this security schema. Implementing and consuming services that are expanded using WS-Security are slightly tougher to build, consume, and simply understand when compared to building services where you control the structure of the security context representation. If your goal is to make your messages as consumable as possible for development staff that is at all levels in ability and on a multitude of different platforms, then using WS-Security is something that might be overkill. Especially for internal services that are working from database of records, you need to remember that there sometimes is value in having a certain level of trust in your organization. The more limitations you put on the movement of messages, the less they will flow.
External services should be treated differently. In addition, you should really try to avoid having external services work directly with your database of records in the backend of your environments. Instead, you should focus on passing external requests through intermediaries (other services or class libraries) thereby allowing you to move from more secure enforcements to less as the message moves up the chain.
STEP 4 – Compression
If you are sending SOAP documents over protocols such as HTTP, you will quickly realize that XML can get rather bloated. It will not take long to realize that you might need to do something in order to effectively move your messages around the enterprise.
To counter this problem you are going to want to compress SOAP messages in either the request or the response. Even better would be to compress both the response and deal with compressed requests that come to your services.
There are a couple of approaches, but it really depends on the vendor platform you are working with. In all cases, you are not going to want to have an absolute rule that messages will either need to be compressed in either direction as not all participants in your SOA are going to be able to comply to these kinds of absolute rules.
Microsoft’s WCF can deal with the compression of SOAP messages in either direction, while using IIS to compress SOAP responses from ASP.NET is also allowed – but you will find that it doesn’t deal with a compressed SOAP request from the client. gSOAP for C++ is also another web service platform that can deal with both compressed requests and responses.
STEP 5 – Binary Attachments
Moving your data and services around the enterprise is not only about moving ASCII text, but it is also about moving binary objects whether those objects are Excel documents, other Office documents, pictures, blobs, or more.
There are a couple of options in how to deal with sending binary objects. The first and easiest approach is to base64 encode your objects directly into the SOAP body and send that to the consumer. The consumer will have to do some work to take the value from the body and turn that back into the object they want to work with.
A better approach is to adopt a standard that is more tuned to dealing with binary attachments. A few years ago, I would have said to use DIME for your binary attachments, but today, you are going to want to turn to MTOM (http://www.w3.org/TR/soap12-mtom/).
There are many vendor technologies that support MTOM and it seems to have taken hold by all the major camps and will be something that you should make use of when moving your binary objects around the enterprise.
STEP 6 – SOAP Faults
Errors happen and an exception is represented as a SOAP fault. A SOAP fault is a specific format defined in the SOAP specification in how messages should appear if the message is an exception. Your organization should come together and define fault codes that are the same across the enterprise. You want a timeout exception to be the same in your organization no matter where in the organization this exception is thrown. This will make the lives of your consumers easier. Trust me.
STEP 7 – WSDL and XSDs
A SOAP interface is defined by another XML document – a WSDL document. You are going to define all your interfaces using a .wsdl file. In addition to the methods being defined within the SOAP document, you are also going to want to define the types that are passed in and out of the services through one or more schema documents (.xsd).
Working with schema-defined objects is the approach you should take rather than any type of string-based input. Do not fall into the trap of thinking that XML is nothing but a string. XML is a representation of objects being transmitted or stored. Your XSD documents define your objects and the WSDL documents define the interfaces that use these objects. This is an important rule to follow and one that many people that really don’t understand the value of XML abandon.
STEP 8 – Timestamps
In exposing sensitive data, you are going to want to include more than a username/password combination in accessing these services. You are also going to want to include a timestamp within the token or within the message in some format. This timestamp will allow you to invalidate messages that are considered expired.
STEP 9 – Encryptions
In moving messages from one location to another in your enterprise, there are portions of the message that might not be for every prying eye. For this reason, you are going to want to deal with partial message encryptions. You should not encrypt entire communication lines (like using SSL), but instead, you are going to only want to encrypt the portion of the message that is considered sensitive. That leaves the rest of the message to be worked with as a message might be routed around the enterprise.
Technologies like WCF and WSE (also from Microsoft) do allow for you to partially encrypt SOAP messages).
STEP 10 – Versioning
Usually, when you publish a SOAP interface, that interface will be around for some time to come. Other teams or products will build to that interface and will not be able to deal with you changing the interface or moving it to a new location.
For this reason, you are going to want to deal with versioning your web methods. One method of doing this is to name your service with a trailing _1_0 or _1_1 to define the version. This means you might have the following web methods:
- GetCustomers_1_0()
- GetCustomers_1_1()
This will allow other groups to easily understand what services they should use and how to migrate as well. All versioning within an organization should be done in the same manner. You are not going to want different teams defining different ways of applying versioning to services.
Working on these steps within your organization will allow you to start yourself on your path to an SOA. It is not too hard and only takes the core teams getting together to agree on the approach. Using these steps as your starting model will quickly get you on your way.