So far I saw only the waterfall methodology in the BizTalk project development. I worked for small and big companies and everywhere I saw only waterfall. Is there something special in the BTS project that Agile is never used with it?
So far we, BizTalk developers, have all disappointments of the waterfall development: the long stages of the project, huge and unusable documentation, disagreements between users, stakeholders and developers, bloated code, unsatisfactory quality of the code, scary deployments and modifications.
Recently our team decided to use Agile principles to address these issues. We had our victories and our defeats, but right now we feel better on our journey to the “Agile world”.
The business goal is simple, we desperately need faster development. When we deploy and test new code in hours not weeks, we make a lot more iterations, we make and find and fix a lot more errors. Which is just great. Now error is not a catastrophe, it is a small thing. That means the more reliable code. Now our applications are more reliable and we fix errors very fast.
The main reason to use Agile is because of economics. It is not only faster and more reliable, it is cheaper.
We decided to use Agile together with SOA and microservice architecture. First we thought, the BizTalk is too heavy tool set to be used with Agile. But it happens the BizTalk Server has very special set of attributes that suits microservice architecture very well out of the box. If you think about orchestrations and ports as the microservices, this part of BizTalk fits perfectly to SOA.
Three main things keep BizTalk developers from using Agile. It is the artifact dependencies, “niche, unnecessary tools”, and manual deployment.
Here I am talking only about technical side of problem. The management side will be touched later.
Dependency is the other side of the code reuse. And here lays one of the main differences between BizTalk applications and generic applications. The later are created as a set of dll-s on C#, Java or any other programming language. In many cases we prefer to simplify things with code reuse, which creates some dependency problems, but usually this is not a big problem. What about the BizTalk applications? BizTalk Server keeps strict control of the working artifacts. Reliability is the king. We cannot just replace one buggy artifact if there is a dependency on it. If hurts redeployment but, remember, reliability is the king. Anything else is not so important. So for us, as the BizTalk developers, the cost of dependency is really high.
Moreover, dependency is a stopper for the SOA application. Keep services independent, and it is easy to modify then, to add new one, to the service versioning.
Niche, Unnecessary Tools
The BizTalk Server is a big toolset. It is impossible to hire a full team of the expert BizTalk developers. Most of the folks are not specialized in BizTalk too much.
So if we keep the technology stack limited, the time to make a new developer productive would be short. Any tool could be replaced by C# code and the decision was to use a bare minimum of the BizTalk tools: Schemas, Maps, Orchestrations.
Completely prohibited are BRE, ESB Toolkit.
The custom pipelines, direct binding and xslt are limited to the very special cases.
The differentiator was a question “Is this tool 100% necessary and does it require a special skill set?”.
I am not going to start another holly war. Our decisions are based only on our use cases. In your company, in your zoo the decisions would be distinct.
How to make development iterations fast? One part of development cycle is deployment. It is not a problem at all if you write your program on Python or Go. It is not so big problem if you write your app on .NET. But in BizTalk development it is the Problem. So any methods to keep deployment quick are very important.
BTDF was a necessary tool in our case but the BizTalk PowerShell Provider is also used.
And deployment is always about dependencies. Penalize more dependencies and prize less dependencies, that is the idea.
We started with defining rules about technology side of the projects. We forced several SOA and microservice rules:
- Service size: Service encapsulates a single business function and exposes only a single interface. This rule effectively cuts a BizTalk application into a single orchestration (or couple ports) in most cases.
- Shared Contracts, API: The services must communicate only through contracts. The only permitted dependences between services are the contracts. We never share maps, orchestrations, ports, pipelines between applications. We only share schemas and API-s.
- Versioning: the service upgrade is published as a new service. Once published service is never changed and could be only removed, not changed.
- Tests: Tests are important part of application. Application without tests is not approved. We need only user acceptance tests. The minimum test set should cover: Successful tests and Failure tests. Performance tests and Unit tests are not mandatory. The special test data is part of design. We tried to design our application in such a way that test data can be used in production together with production data.
- Automation: Automated deployment is important part of application. Application without automated deployment is not approved.
As you can see this list has some specifics for the BizTalk projects. BizTalk is oriented to XML that’s why we tell an XML Schema when we mean a Contract. In BizTalk projects it is really hard to test endpoints, so we tried to be easy with testing. The deployment of the BizTalk application is the complex and long process, so we force the deployment automation.
These rules were not easy to push in development. There were many unanswered questions on the way: What size is “small” and what is “big”? What modification is considered as a new version? What test coverage is enough?
These rules are not ideal. We had discussions, we added and removed rules, we changed them. We are still in the process.
Those are the “technological” rules. But we need the management, team rules, because Agile is about the team structure and communications, right? The Conway’s law cannot be ignored if you think about Agile. Also we cannot avoid all this hype with DevOps movement. So we put in practice some additional “team” rules:
- One service implemented by one developer.
- One service implemented in one week sprint.
- Service developer is DevOp, he/she is responsible for deployment and all operations of the service in all environments, including production.
“One week” rule is just happened after couple months of experiments. We tried 2-3 days and 2 weeks. One week works because we want to keep all projects in one team. All developers work remotely, we never met in team together. And our team is not big, so “one week” was cause of those factors.
One of the key issues with Agile and DevOps is the service knowledge is tightly coupled with single developer. Enterprises cannot tolerate this issue because it drastically increases risks to lost service if developer is not accessible. So we added a new role (the senior developer) and two more “team” rules:
- A Senior developer approves the service design. This senior developer performs tests and deployment of the service in production and sign service to production.
- If service developer or service senior developer is not accessible, a new developer should take responsibility for this role.
The “Senior Developer” rule is tricky. With this rule a senior developer cannot just sign documents, approve something and voila. No way. This rule effectively forces senior developer to make good code review and monitor all development steps. Officially this rule is cover two short tasks, but they are short only if this team of two invests good amount of time to communicate all details of the service.
The true hart of the SOA and microservice is in limited dependencies. With BizTalk applications the dependency problem is even more important than in the standalone applications, because the BizTalk controls dependencies and prohibits many shortcuts permitted in the standalone applications. So eliminating dependencies simplifies development and makes possible to make simple services, the microservices, which is our goal.
But we cannot just remove all dependencies. The service dependencies are the necessary evils. We should share schemas, dll-s, services. So there are special “dependency” rules:
- Any changes in the shared resources, any changes that could go outside of the service boundaries must be approved by the whole service team. All team members should agree on this change. Any team member can veto the change.
For example, we implemented the shared infrastructure for logging. The first approach was to create the shared library (dll) and force everybody to use it. It was vetoed. The second approach was to create a special logging services and expose it as a single endpoint. It was also vetoed, because the code to use this service was too big. The next try was to use log4net or NLog and standardize the log format only. Then there was the next attempt and more. Now we discuss the InfluxDB, but what is matter, now we know much-much more what we need in reality and what we don’t need.
Complete Rule Set
Now we have this rule set:
- Single Interface
- Shared Contract
- Never Change [Published]
- Test inside Application
- Automated Deployment
- Senior Developer
- New Developer
- New Dependency
These rules are not ideal. For example, we still struggling with the Wrong Requirements problem. How to fix this problem?
Our rule set is too long. Now we consider to join the Test and Deployment rules.
This rule set works for our team. What is special about our team? A half of the team members are the full-time employee, which enables the DevOp and Senior Developer rules. Not sure these rules would work if all members are contractors. Our team has a big list of projects to develop and the big list of applications to support. If you have mostly applications to support, our approach is possibly not the best fit for you.
I did not describe the communications with our customers and stakeholders, which is important part of the whole picture.
I did not describe the Agile practices we use (but we definitely use the Kanban board), it is not the point of this topic.
We were happy with our management, which took a risk of changes in processes and the team. Now the management is happy (hmm… almost happy) with BizTalk development and operations. Now we are SUPERFAST team! We are not sharks, not yet, but we are not jellyfishes, not anymore.