Farhan's Two Cents on Collaboration, Integration & Enterprise Tech.

Please expand on your thoughts in this post. especially where 10+ mb files would kill the app server.

Thanks!!!!!

The basic point I am trying to make here is that many a times workflow engines are used for extraction-transformation-loading (ETL) tasks and thats pretty much a misuse of the tool itself. I can understand a message oriented architecture where messages come and perform/trigger some work through orchestrations or workflow, however what does not strike me as natural is passing a substantially large message just for doing some sort of tedious mapping/transformation through the orchestration engine that will inevitably bring down the engine. Now appreciating your question, 10 MB may not be the best figure to quote, however the sizing limitations vary per scenario. I agree that MS does not put any upper limit on the Message size Biztalk can handle, the question then remains is that what is the optimal message size as well as the frequencey of message influx. Speaking through some of my experiences with high influx messaging environments, the size of the message inversely effects the BizTalk Server. In addition the adapter being used also plays a part in the bigger picture such as the SOAP adapter may take more time than FILE adapter to 'receive' the same message.
Lastly let me clarify that the intention was to differentiate between ETL data and 'messages' and the proper solution to handle each type. Therefore the nature of the message processing has to be understood before the tool can be applied to the envisioned solution; that is not to say that BizTalk would not be able to handle large messages but only to point out firstly that BizTalk is not an ETL tool and secondly as the message size gets larger, the performance decreases.