Richard Seroter talks here about feeding BizTalk events into StreamInsight or receiving events out as BizTalk messages. I spent a little time considering the possibility of using StreamInsight in conjunction with BTS messaging infrastructure by building custom BTS adapters or pipeline components that directly hook into StreamInsight. A preliminary look through the code samples (the best way, currently, of understanding this new engine, given that the CTP documentation is somewhat sparse) suggests to me that this is probably not the way to go. As I understand things, there seems to be an impedance mismatch between BTS messaging and the concept of an event stream. The closest thing BTS offers to an ‘event stream’ is its concept of a batch, used by adapters to submit multiple messages to BTS via the transport proxy as a single transactional interaction. I say ‘closest’, but in reality a BTS batch is quite different to an event stream, and consequently a BTS adapter is not at all similar to a StreamInsight adapter. Event streams may be continuous over extended periods of time, and don’t naturally fit a transactional model.
Of course, in constrained circumstances, it may be possible for the two models to co-exist. The sample TextFile input adapter, for example, reads a single CSV file and converts its contents into a stream of events. A second CSV files would be processed as a separate event stream. In this case, StreamInsight could co-exist nicely with a BTS pipeline that processes each CSV file as a separate message. However, in other scenarios, you would have to ensure that a StreamInsight adapter continues to run over an open-ended number of BTS batch submissions. Worse still, how would you cope with multiple instances of the same Receive Ports running in different host instances? Somehow you would have to share a single event stream between those multiple BTS host instances.
So, I think that as far as BTS is concerned, the obvious places you might use StreamInsight are:
a) outside of BTS, feeding events to a BTS Receive Port
b) from within a BTS orchestration, probably exploiting the convoying and long-running
transaction facilities of the orchestration to handle submission of event messages to
a single event stream.
c) outside of BTS, receiving events emitted by a BTS Send Port.
A special case of a) would be using StreamInsight in conjunction with RFID Server. Remember that RFID Server can be used for more than just RFID devices. c) would, I think, prove to be a rare scenario. b) and c) would both significantly compromise the raw performance of StreamInsight by inserting the relatively high latency BTS message box in front of a StreamInsight input adapter.
Having spent little time yet with the CTP, I reserve the right to make mistakes! Maybe I have not properly grasped the StreamInsight model, but this is how it looks to me at present.