Geeks With Blogs
A Curious Mind #tastic

If you consider that moving data around via messages is still ETL, then I want to talk a bit about why I think message based ETL or ETL:M is better than batch based ETL or ETL:B (batch based is typified by tools like SSIS).

I believe that one of the biggest reasons we have ETL:B is because Applications don't have a way to talk to each other. So the solution in the past has been to extract some logic from the source and destination applications and bury it in a ETL:B package that runs at regular intervals. Here I see two problems. One the regular interval could cause some very interesting consistency issues, two the extraction, duplication, and obsfucation of business logic in a tool that was most likely not built to provide the same level of explanatory power that well written code can.

ETL:B is like trying to transfer knowledge by brain transplant rather than the ETL:M approach which I believe is more akin to a conversation between two interested parties.

As we move to adopt a more ETL:M approach I think we will find that integrating our application becomes not only an easier process, but a more responsive process as well.

Posted on Wednesday, July 30, 2008 4:08 PM | Back to top

Comments on this post: Random Thought

# re: Random Thought
Requesting Gravatar...
Of course the main benefit is that if you go up front with ETL:M you can use in practice either ETL:B or ETL:M as you can batch your messages and push them at specified times etc in batches.

Left by Greg Young on Sep 22, 2008 4:18 PM

Your comment:
 (will show your gravatar)

Copyright © Dru Sellers | Powered by: