I've been engaged full time with BizTalk Server as a consultant, architect and developer since late 2003, and work for a company with several hundred discrete BizTalk engagements under its belt. In our experience it is not uncommon for people to get the wrong idea about BTS. BTS remains a glorified developers' toolkit. The idea that you can just run through a couple of wizards, drop a couple of components into a designer and see a fully-featured enterprise-level integration or business process solution emerge before your very eyes is hopelessly naive. However, that is what some people expect. BTS provides a framework for building certain types of application, and the runtime environment needed to manage and scale robust message-based interchange and process automation. BTS can't make complexity go away - it can, however, give you a fighting chance of managing that complexity effectively.
A second observation is that, architecturally, BTS has been over-used in many scenarios, for the simple reason that no other tools were available at the time. This has changed dramatically, and for the better, with the advent of WCF and WF. Our own architectural thinking has evolved to concentrate on building service bus patterns that incorporate multiple technologies. We try to exploit the synergies that exist between BTS, WCF and WF. This is well aligned to the Microsoft's future roadmap for these technologies.
Lastly, with regard to ILOG Rules for .NET, you get what you pay for. If Microsoft were ever to 'retire' MS BRE, I doubt very much they would reduce the cost of a BTS licence accordingly. MS BRE is OK, but it lacks the kind of mature tooling that ILOG and other vendors can offer. The engine, itself, is missing a couple of features, and tends to rely more heavily than most on additional custom coding. The repository is basic. So, I don't see that ILOG's engine competes head-on with MS BRE. MS BRE provides a basic suite of tools, comes bundled with BTS, can be used to address a wide range of needs when building automated business processes and is very much aimed at developers (strangely, though, without the Visual Studio integration that Andrew describes for ILOG Rules). WF Rules, again, comes with virtually no tooling at all, is part of the .NET Framework, addresses sequential rule processing needs and is completely free of charge. By contrast, ILOG Rules for .NET is a serious enterprise-level piece of kit for companies who invest heavily in harvesting, development, deployment and management of business rules. This is reflected in the price tag. You simply wouldn’t use MS BRE at that level.
One thing. I have never understood why vendors like ILOG and Fair Isaac don't provide better integration between their technologies and Microsoft's platform and applications. For example, MS BRE is pluggable. It is technically possible to plug other engines into Microsoft’s Rules Framework. This could potentially allow their Rules Engine to be invoked by BizTalk's Call Rules shape, for example (something I’ve never bothered to prove). Now, there are some barriers to enabling this, but they can be overcome, especially if some pressure is applied to Microsoft to improve a couple of aspects of their rules framework. I talked to Chris Berg about this last year. It seems such an obvious way to open up an additional market for ILOG Rules for .NET.