Daniel Hester

Microsoft Integration Architecture, Frameworks and Technologies

  Home  |   Contact  |   Syndication    |   Login
  8 Posts | 0 Stories | 2 Comments | 0 Trackbacks

News



Archives

Post Categories

Thursday, April 4, 2013 #

Microsoft has added a BizTalk Server 2013 Beta virtual machine image to the gallery of available images on Windows Azure. In a few clicks you can create a new BizTalk Server that has all of its software pre-installed. The only thing you need to do is the final step of configuring the BizTalk Server using the BizTalk Configuration Utility. Note that the Virtual Machine functionality in Azure is currently in preview mode; you need to request that the Virtual Machine role be added to your Azure subscription before you can create Virtual Machines. If you have an MSDN Premium Subscription you’re entitled to run one (small) Virtual Machine free of charge. See here for details:

http://www.windowsazure.com/en-us/pricing/member-offers/msdn-benefits

Detailed Configuration

1. Logon to the Windows Azure Management Portal: https://manage.windowsazure.com

2. Click on the Virtual Machines tab on the sidebar navigation and click on the + NEW button at the bottom of the screen

clip_image002

3. Chose to create a new Virtual Machine from a Gallery Image:

clip_image004

4. Select the BizTalk virtual machine image from Gallery:

clip_image006

5. On the virtual machine configuration page choose the version of BizTalk (by release date), enter the virtual machine name, administrator password and select the virtual machine size (Medium is recommended):

clip_image008

When multiple versions of the image have been released it is recommended that you select the latest version.

You should consider using a name for the virtual machine that is descriptive enough to tell what it is used for, but short enough to make it easy to use. You have 3 to 15 characters for the virtual machine name.

6. On the virtual machine mode page either create a standalone virtual machine or connect to an existing virtual machine (for load balancing), select the DNS name, the storage account and the virtual machine location:

clip_image010

You can connect virtual machines to balance the load of traffic to your application or you can create a virtual machine that isn’t connected to any others.

The DNS name is the public name you will use to connect to your virtual machine within the CloudApp.net domain. You can use a friendly name or a more specific and longer name such as machinename-domainname e.g. dev-bts2013beta-danielhester. You can only use letters, numbers and hyphens in your DNS name.

Once you have created at least one virtual machine you can assign your new virtual machine to the already created Storage Account; other just select to use an automatically generated storage account.

Choose the region, affinity group, or virtual network in which you want to deploy the virtual machine.

7. On the virtual machine options page select or create an availability set if you want to replicate the Virtual Machine image to other Azure domains or data centers; otherwise just leave the availability set on . Click the check button to continue:

clip_image012

An Availability Set is a group of virtual machines that are deployed across fault domains and update domains. An Availability Set makes sure that your virtual machine is not affected by single points of failure, like the network switch or the power unit of a rack of servers. Microsoft charges a few extra cents per hour on the Virtual Machine when you configure an Availability Set.

8. The provisioning process begins…

clip_image014

9. The virtual machine is provisioned and started – click on the CONNECT button at the bottom of the page:

clip_image016

10. When you click on the CONNECT button at the bottom of the page you are given the option to Open or Save a Remote Desktop Connection file (*.rdp).

clip_image018

11. Personally, I save my remote desktop connections by clicking on the down-down on the Save button, choosing Save As, and saving the RDP file on my local file system:

clip_image020

12. Once I’ve saved the RDP file I can modify it (by right clicking on it and choosing Edit) so that I can save my credentials and connect my local drives to my remote server (click the More…button on the Local Resources tab and select Drives and other supported PnP devices:

clip_image022

clip_image024

clip_image026

 

Remember to save your changes to the RDP file by clicking back to the General tab and clicking the Save button.

13. Open (or run) the RDP file once you have downloaded it. Check the “Don’t ask me again for connections to this computer” checkbox and click Connect:

clip_image027

14. Enter your credentials (and check the “Remember my credentials” checkbox if you want):

clip_image028

15. Choose to not be bothered again by the Remote Computer’s certificate by checking the “Don’t ask me again for connections to this computer” checkbox and click the Yes button to continue:

clip_image029

16. Wait for the connection to complete:

clip_image030

17. The Server Manager Dashboard shows on your remote virtual machine:

clip_image032

18. Install your favorite utilities i.e. Classic Shell (http://www.classicshell.net) – to restore the classic Windows Start Button and Menu to Windows Server 2012 and begin the configuration of your new BizTalk server.

Note for guidance on configuring BizTalk Server please see the Microsoft Guides on MSDN:

Basic Configuration:

http://msdn.microsoft.com/en-us/library/jj248696(v=bts.80).aspx

Custom Configuration:

http://msdn.microsoft.com/en-us/library/aa561189(v=bts.80).aspx

Deleting and then Recreating your Azure Virtual Machine with the Same DNS Name

When you delete an Azure Virtual Machine a Cloud Service is retained for the Virtual Machine’s DNS name.  If effect, Azure is retaining a vestige of your Virtual Machine in case you want to recreate it with the same name.  So when you’re ready to recreate your VM, go to the Cloud Services tab within the Azure Management console, delete the Cloud Service for your old VM and then recreate your new VM with the same DNS name.

 Download this post as a Word Document

Updates to this Blog Post

Note that Microsoft have (as of August 2013) deployed the RTM image of BizTalk Server 2013 to Azure with three images to choose from: Enterprise, Standard, and Evaluation.  To configure BizTalk Server 2013 Standard or Enterprise Edition, an existing SQL Server 2012 or SQL Server 2008 R2 SP1 virtual machine is needed. Standard and Enterprise Editions cost additional dollars per hour.  The Evaluation image deploys the SQL Server on the same Virtual Machine as the BizTalk Server.  Please see the following two MSDN articles for reference:

Create a BizTalk Virtual Machine in Windows Azure:
http://msdn.microsoft.com/en-us/library/jj572843(v=bts.80).aspx

Configuring BizTalk Server 2013 on a Windows Azure VM:
http://msdn.microsoft.com/en-us/library/jj248689(v=bts.80).aspx


Friday, March 8, 2013 #

Magenic is sponsoring a FREE day of advanced education for senior technical people and technical decision makers on Friday April 19th, 2013 at the Microsoft Building (835 Market Street, Suite 700, San Francisco, CA 94103).

Rocky Lhotka will deliver the keynote speech -- “The Future of Microsoft Software Development”

The sessions will cover modern application development focusing on two tracks: modern web applications and the latest trends in business intelligence. After the sessions there will be a panel featuring local industry experts on end-to-end quality.

Register here:

http://codemastery.com/sanfrancisco/


Wednesday, September 5, 2012 #

 

Background

Designing Stored Procedures that are safe for multiple subscribers (to call simultaneously) can be challenging.  For example let’s say that you want multiple worker processes to poll a shared work queue that’s encapsulated as a SQL Table. This is a common scenario and through experience you’ll find that you want to use Table Hints to prevent unwanted locking when performing simultaneous queries on the same table. There are three table hints to consider: NOLOCK, READPAST and UPDLOCK. Both NOLOCK and READPAST table hints allow you to SELECT from a table without placing a LOCK on that table. However, SELECTs with the READPAST hint will ignore any records that are locked due to being updated/inserted (or otherwise “dirty”), whereas a SELECT with NOLOCK ignores all locks including dirty reads. For the initial update of the flag (that marks the record as available for subscription) I don’t use the NOLOCK Table Hint because I want to be sensitive to the “active” records in the table and I want to exclude them.  I use an Update Lock (UPDLOCK) in conjunction with a WHERE clause that uses a sub-select with a READPAST Table Hint in order to explicitly lock the records I’m updating (UPDLOCK) but not place a lock on the table when selecting the records that I’m going to update (READPAST).

UPDATES should be allowed to lock the rows affected because we’re probably changing a flag on a record so that it is not included in a SELECT from another subscriber. On the UPDATE statement we should explicitly use the UPDLOCK to guard against lock escalation. A SELECT to check for the next record(s) to process can result in a shared read lock being held by more than one subscriber polling the shared work queue (SQL table). It is expected that more than one worker process (or server) might try to process the same new record(s) at the same time. When each process then tries to obtain the update lock, none of them can because another process has a shared read lock in place. Thus without the UPDLOCK hint the result would be a lock escalation deadlock; however with the UPDLOCK hint this condition is mitigated against.

Note that using the READPAST table hint requires that you also set the ISOLATION LEVEL of the transaction to be READ COMMITTED (rather than the default of SERIALIZABLE).

Guidance

In the Stored Procedure that returns records to the multiple subscribers:

  • Perform the UPDATE first. Change the flag that makes the record available to subscribers.  Additionally, you may want to update a LastUpdated datetime field in order to be able to check for records that “got stuck” in an intermediate state or for other auditing purposes.
  • In the UPDATE statement use the (UPDLOCK) Table Hint on the UPDATE statement to prevent lock escalation.
  • In the UPDATE statement also use a WHERE Clause that uses a sub-select with a (READPAST) Table Hint to select the records that you’re going to update.
  • In the UPDATE statement use the OUTPUT clause in conjunction with a Temporary Table to isolate the record(s) that you’ve just updated and intend to return to the subscriber. This is the fastest way to update the record(s) and to get the records’ identifiers within the same operation.
  • Finally do a set-based SELECT on the main Table (using the Temporary Table to identify the records in the set) with either a READPAST or NOLOCK table hint.  Use NOLOCK if there are other processes (besides the multiple subscribers) that might be changing the data that you want to return to the multiple subscribers; or use READPAST if you're sure there are no other processes (besides the multiple subscribers) that might be updating column data in the table for other purposes (e.g. changes to a person’s last name).  NOLOCK is generally the better fit in this part of the scenario.

See the following as an example:

CREATE PROCEDURE [dbo].[usp_NewCustomersSelect]

AS

BEGIN

-- OVERRIDE THE DEFAULT ISOLATION LEVEL

SET TRANSACTION ISOLATION LEVEL READ COMMITTED

-- SET NOCOUNT ON

SET NOCOUNT ON

-- DECLARE TEMP TABLE

-- Note that this example uses CustomerId as an identifier;

-- you could just use the Identity column Id if that’s all you need.

DECLARE @CustomersTempTable TABLE

(

CustomerId NVARCHAR(255)

)

-- PERFORM UPDATE FIRST

-- [Customers] is the name of the table

-- [Id] is the Identity Column on the table

-- [CustomerId] is the business document key used to identify the

-- record globally, i.e. in other systems or across SQL tables

-- [Status] is INT or BIT field (if the status is a binary state)

-- [LastUpdated] is a datetime field used to record the time of the

-- last update

UPDATE [Customers] WITH (UPDLOCK)

SET

[Status] = 1,

[LastUpdated] = GETDATE()

OUTPUT

[INSERTED].[CustomerId]

INTO @CustomersTempTable

WHERE ([Id] = (SELECT TOP 100 [Id]

FROM [Customers] WITH (READPAST)

WHERE ([Status] = 0)

ORDER BY [Id] ASC))

-- PERFORM SELECT FROM ENTITY TABLE

SELECT

[C].[CustomerId],

[C].[FirstName],

[C].[LastName],

[C].[Address1],

[C].[Address2],

[C].[City],

[C].[State],

[C].[Zip],

[C].[ShippingMethod],

[C].[Id]

FROM [Customers] AS [C] WITH (NOLOCK), @CustomersTempTable AS [TEMP]

WHERE ([C].[CustomerId] = [TEMP].[CustomerId])

END

In a system that has been designed to have multiple status values for records that need to be processed in the Work Queue it is necessary to have a “Watch Dog” process by which “stale” records in intermediate states (such as “In Progress”) are detected, i.e. a [Status] of 0 = New or Unprocessed; a [Status] of 1 = In Progress; a [Status] of 2 = Processed; etc.. Thus, if you have a business rule that states that the application should only process new records if all of the old records have been processed successfully (or marked as an error), then it will be necessary to build a monitoring process to detect stalled or stale records in the Work Queue, hence the use of the LastUpdated column in the example above. The Status field along with the LastUpdated field can be used as the criteria to detect stalled / stale records. It is possible to put this watchdog logic into the stored procedure above, but I would recommend making it a separate monitoring function. In writing the stored procedure that checks for stale records I would recommend using the same kind of lock semantics as suggested above. The example below looks for records that have been in the “In Progress” state ([Status] = 1) for greater than 60 seconds:

CREATE PROCEDURE [dbo].[usp_NewCustomersWatchDog]

AS

BEGIN

-- TO OVERRIDE THE DEFAULT ISOLATION LEVEL

SET TRANSACTION ISOLATION LEVEL READ COMMITTED

-- SET NOCOUNT ON

SET NOCOUNT ON

DECLARE @MaxWait int; SET @MaxWait = 60

IF EXISTS (SELECT 1

FROM [dbo].[Customers] WITH (READPAST)

WHERE ([Status] = 1)

AND (DATEDIFF(s, [LastUpdated], GETDATE()) > @MaxWait))

BEGIN

SELECT 1 AS [IsWatchDogError]

END

ELSE

BEGIN

SELECT 0 AS [IsWatchDogError]

END

END

Downloads

The zip file below contains two SQL scripts: one to create a sample database with the above stored procedures and one to populate the sample database with 10,000 sample records. 

I am very grateful to Red-Gate software for their excellent SQL Data Generator tool which enabled me to create these sample records in no time at all.

References

http://msdn.microsoft.com/en-us/library/ms187373.aspx

http://www.techrepublic.com/article/using-nolock-and-readpast-table-hints-in-sql-server/6185492

http://geekswithblogs.net/gwiele/archive/2004/11/25/15974.aspx

http://grounding.co.za/blogs/romiko/archive/2009/03/09/biztalk-sql-receive-location-deadlocks-dirty-reads-and-isolation-levels.aspx


Wednesday, May 16, 2012 #

When installing SQL Server 2008 R2 (Developer, Standard, or Enterprise) within a Virtual PC you may suffer the following problem:

“The following error has occurred:  The installer has encountered an unexpected error installing this package. This may indicate a problem with this package. The error code is 2337.”

The error occurs in the following package: sql_engine_core_shared

It turns out that this problem is with the DVD/ISO Mounting Feature in Windows Virtual PC.  To workaround the problem you need to expand the ISO with WinRAR – not WinZip and not by copying the contents of the mounted ISO to the VPC’s hard drive.  You need to use WinRAR to extract the ISO; extracting the contents of the ISO with anything else just won’t solve the problem.

Microsoft have acknowledged the problem here:

https://connect.microsoft.com/SQLServer/feedback/details/550502/r2-rc0-install-problem-on-virtual-pc-2007

WinRAR can be found here:

http://www.win-rar.com/download.html


Wednesday, May 9, 2012 #

When you try to add a BTAHL7 Common Project, e.g. BTAHL7V2XCommon, to your Visual Studio 2010 solution you may encounter the following error:

“Object reference not set to an instance of an object”

image

This error occurs when you try to create the BTAHL7 Common Project within a Solution Folder that matches the location of a Physical Folder within your solution’s root folder.  That is, you might be following a best practice of storing your Visual Studio Projects within a "Projects" solution folder that matches a physical folder within your Solution's root folder. In this case when you try to create your BTAHL7 Common Project within C:\Projects\MyHL7Solution\Projects by right-clicking on your Projects solution folder and choosing New Project you'll experience this error. 

The workaround is to create the project (temporarily) in the solution’s root folder, save it, exclude it from the solution, copy the project folder over to where you want it (e.g. C:\Projects\MyHL7Solution\Projects) and then add it back as an existing project, picking it up from its new location.  If your solution is already source controlled then you might want to create the BTAHL7 Common Project in a dummy solution and then copy it over to the folder where you want it so as to avoid a fight with source control.

Here are the repro steps:

Error when adding a new BTAHL7 Project to a Solution Folder - Repro Steps.docx


Tuesday, May 8, 2012 #

Jon Flanders did some great work with SSO for storing BizTalk configuration data (used by .NET code) within the Enterprise SSO Database and the BizTalk team created a sample application for using SSO as a Configuration Store back in BizTalk Server 2006. Richard Seroter brought these things together in an SSO Configuration Utility that enables users to configure values that can then be read at runtime by an incredibly simple API. My BizTalk SSO Storage utility builds upon this work and I’ve updated the solution to Visual Studio 2010.  I’ve created a document that explains how to setup SSO Storage Applications and access their secret values using the Microsoft API.

The solution (in compressed form) can be found here:

BizTalkSSOStorage.zip

The document which describes the solution can be found here (and in the zip above):

BizTalk SSO Storage Application Documentation.docx


Microsoft have released a 2nd CTP of the Windows Azure ServiceBus EAI & EDI Labs SDK – this is the new Integration Services platform that will likely eclipse BizTalk Server at some point in the next three (3) to five (5) years:

Download:

http://www.microsoft.com/en-us/download/details.aspx?id=17691

Overview of the refreshed CTP:

http://blogs.msdn.com/b/windowsazure/archive/2012/04/09/announcing-the-refresh-of-service-bus-eai-amp-edi-labs.aspx

The CTP now supports the following features:

  • EDI
  • Flat File Processing
  • Transforms
  • One-Way Bridge
  • Request / Reply Bridge
  • Bridge Chaining
  • Protocol Bridging
  • Property And Data Lookup
  • Operational Tracking
  • External FTP connectivity

I’ve put together a detailed Installation Guide with screenshots.  The Installation Guide can be found here:

Windows Azure ServiceBus EAI & EDI SDK - April 2012 CTP - Installation Guide.docx


Wednesday, February 29, 2012 #

Microsoft Dynamics 4.0 has a 32-bit BizTalk Adapter which is not compatible with 64-bit versions of Windows or BizTalk.  Under Windows Server 2003, the AX 4.0 Adapter would not even install on the 64-bit version of the OS.  On Windows Server 2008 R2 (which is only 64-bit), the Adapter installs, but it won’t work.

However one of my colleagues, Justin, figured out what is wrong with the Adapter and a workaround to “fix it” so that it would work.  The AX 4.0 BizTalk Adapter is a remoting adapter and there’s a Windows Portable Executable (PE) -- axaifremsrv.exe -- that actually enables the communication between BizTalk and AX.  This EXE is NOT marked 32-bit only and that is why things fail.  Specifically, the EXE’s Header table doesn’t tell Windows and the CLR to load itself into a 32-bit only Process.

To test this assertion, use the CorFlags utility (which is one of the Visual Studio SDK Tools) to mark axaifremsrv.exe as a 32-bit only EXE, i.e.,

Open up a Visual Studio Command Prompt, then navigate to the AX 4.0 Bin folder (which is usually here on a 64-bit system: C:\Program Files (x86)\Microsoft Dynamics AX\4.0\Client\Bin) and then execute the following command on the axaifremsrv.exe:

corflags axaifremsrv.exe /32BIT+ /Force

The CorFlags utility will then change the EXE’s Header so that the EXE will now correctly identify itself as a 32-bit only Windows PE.

The BizTalk Host(s) assigned to run the send and receive functions of the Axapta Adapter must also be set to 32-bit only within the Host’s definition in the BizTalk Server Administration Console.

More information on the CorFlags Tool can be found here:

http://msdn.microsoft.com/en-us/library/ms164699.aspx

Lastly, I should say that this modification is not supported by anyone and may be a violation of some kind; I am just explaining what is wrong and how we discovered it.  The first supported version of the Microsoft Dynamics AX BizTalk Adapter for 64-bit systems is Microsoft Dynamics AX 2009 SP1:

http://blogs.technet.com/b/dynamicsaxse/archive/2011/09/22/microsoft-dynamics-ax-2009-sp1-is-compatible-with-biztalk-server-2010.aspx