.NET Framework

Windows Azure Tables notes

Kevin Shyr — Thu, 05 May 2011 14:03:06 GMT

Notes from camp: http://wabcdemos.cloudapp.net/

CreateCloudTableClient()
CreateIfNotExists
DeleteIfExists
CreateTable
Create Partition Key (don't use identity key as partition key)
Create Row Key, must be unique per PartitionKey
Use a lightweight copy to improve performance
Use "batching" to increase performance, and also save money by reducing the number of transaction
For example, ???Context.SaveChanges(SaveChangesOptions.Batch);
Queries
Over 30 second is cancelled
result > 1000, a continuation token is returned
Transaction
Limit 100 per batch
Column limit: 260 (URI length)
Data transfer: only counts the transfer in/out of data center (not within data center)

Code Certificate

Kevin Shyr — Thu, 31 Mar 2011 17:39:45 GMT

http://www.startssl.com/

Starting PowerShell learning process

Kevin Shyr — Thu, 31 Mar 2011 17:31:46 GMT

Today I start learning powershell, maybe 2 years too late!

http://powershell.com/cs/

http://www.powergui.org/index.jspa

http://poshcode.org/

Parallel programming in Visual Studio

Kevin Shyr — Thu, 31 Mar 2011 17:26:03 GMT

Not much experience with these products yet.

Free: http://developer.nvidia.com/object/nsight.html

Not free: http://software.intel.com/en-us/articles/intel-parallel-studio-purchase/

For SSIS, this looks very interesting:

http://extendedssispkgexec.codeplex.com/

Using BizTalk to bridge SQL Job and Human Intervention (Requesting Permission)

Kevin Shyr — Fri, 04 Feb 2011 21:29:09 GMT

I start off the process with either a BizTalk Scheduler (http://biztalkscheduledtask.codeplex.com/releases/view/50363) or a manual file drop of the XML message. The manual file drop is to allow the SQL

Job to call a "File Copy" SSIS step to copy the trigger file for the next process and allows SQL

Job to be linked back into BizTalk processing.

The Process Trigger XML looks like the following. It is basically the configuration hub of the business process

<ns0:MsgSchedulerTriggerSQLJobReceive xmlns:ns0="urn:com:something something">
<ns0:IsProcessAsync>YES</ns0:IsProcessAsync>
<ns0:IsPermissionRequired>YES</ns0:IsPermissionRequired>
<ns0:BusinessProcessName>Data Push</ns0:BusinessProcessName>
<ns0:EmailFrom>DoNotReply@company.com</ns0:EmailFrom>
<ns0:EmailRecipientToList>ProjectManagers@company.com</ns0:EmailRecipientToList>
<ns0:EmailRecipientCCList>Developers@company.com</ns0:EmailRecipientCCList>
<ns0:EmailMessageBodyForPermissionRequest>This message was sent to request permission to start the Data Push process. The SQL Job to be run is WeeklyProcessing_DataPush</ns0:EmailMessageBodyForPermissionRequest>
<ns0:SQLJobName>WeeklyProcessing_DataPush</ns0:SQLJobName>
<ns0:SQLJobStepName>Push_To_Production</ns0:SQLJobStepName>
<ns0:SQLJobMinToWait>1</ns0:SQLJobMinToWait>
<ns0:PermissionRequestTriggerPath>\\server\ETL-BizTalk\Automation\TriggerCreatedByBizTalk\</ns0:PermissionRequestTriggerPath>
<ns0:PermissionRequestApprovedPath>\\server\ETL-BizTalk\Automation\Approved\</ns0:PermissionRequestApprovedPath>
<ns0:PermissionRequestNotApprovedPath>\\server\ETL-BizTalk\Automation\NotApproved\</ns0:PermissionRequestNotApprovedPath>
</ns0:MsgSchedulerTriggerSQLJobReceive>

Every node of this schema was promoted to a distinguished field so that the values can be used for decision making in the orchestration. The first decision made is on the "IsPermissionRequired" field.

If permission is required (IsPermissionRequired=="YES"), BizTalk will use the configuration info in the XML trigger to format the email message. Here is the snippet of how the email message is constructed.

SQLJobEmailMessage.EmailBody
 = new Eai.OrchestrationHelpers.XlangCustomFormatters.RawString(
 MsgSchedulerTriggerSQLJobReceive.EmailMessageBodyForPermissionRequest +
 " " +
 "By moving the file, you are either giving permission to the process, or disapprove of the process." +
 " " +
 "This is the file to move: \"" + PermissionTriggerToBeGenereatedHere +
 "\" " +
 "(You may find it easier to open the destination folder first, then navigate to the sibling folder to get to this file)" +
 " " +
 "To approve, move(NOT copy) the file here: " + MsgSchedulerTriggerSQLJobReceive.PermissionRequestApprovedPath +
 " " +
 "To disapprove, move(NOT copy) the file here: " + MsgSchedulerTriggerSQLJobReceive.PermissionRequestNotApprovedPath +
 " " +
 "The file will be IMMEDIATELY picked up by the automated process. This is normal. You should receive a message soon that the file is processed." +
 " " +
 "Thank you!"
 );
SQLJobSendNotification(Microsoft.XLANGs.BaseTypes.Address) = "mailto:" + MsgSchedulerTriggerSQLJobReceive.EmailRecipientToList;
SQLJobEmailMessage.EmailBody(Microsoft.XLANGs.BaseTypes.ContentType) = "text/html";
SQLJobEmailMessage(SMTP.Subject) = "Requesting Permission to Start the " + MsgSchedulerTriggerSQLJobReceive.BusinessProcessName;
SQLJobEmailMessage(SMTP.From) = MsgSchedulerTriggerSQLJobReceive.EmailFrom;
SQLJobEmailMessage(SMTP.CC) = MsgSchedulerTriggerSQLJobReceive.EmailRecipientCCList;
SQLJobEmailMessage(SMTP.EmailBodyFileCharset) = "UTF-8";
SQLJobEmailMessage(SMTP.SMTPHost) = "localhost";
SQLJobEmailMessage(SMTP.MessagePartsAttachments) = 2;

After the Permission request email is sent, the next step is to generate the actual Permission Trigger file. A correlation set is used here on SQLJobName and a newly generated GUID field.

<?xml version="1.0" encoding="utf-8"?><ns0:SQLJobAuthorizationTrigger xmlns:ns0="somethingsomething"><SQLJobName>Data Push</SQLJobName><CorrelationGuid>9f7c6b46-0e62-46a7-b3a0-b5327ab03753</CorrelationGuid></ns0:SQLJobAuthorizationTrigger>

The end user (the human intervention piece) will either grant permission for this process, or deny it, by moving the Permission Trigger file to either the "Approved" folder or the "NotApproved" folder. A parallel Listen shape is waiting for either response.

The next set of steps decide how the SQL Job is to be called, or whether it is called at all. If permission denied, it simply sends out a notification. If permission is granted, then the flag (IsProcessAsync) in the original Process Trigger is used. The synchonous part is not really synchronous, but a loop timer to check the status within the calling stored procedure (for more information, check out my previous post: http://geekswithblogs.net/LifeLongTechie/archive/2010/11/01/execute-sql-job-synchronously-for-biztalk-via-a-stored-procedure.aspx) If it's async, then the sp starts the job and BizTalk sends out an email.

And of course, some error notification:

Footnote: The next version of this orchestration will have an additional parallel line near the Listen shape with a Delay built in and a Loop to send out a daily reminder if no response has been received from the end user. The synchronous part is used to gather results and execute a data clean up process so that the SQL Job can be re-tried. There are manu possibilities here.

Using SSIS to send a HTML E-Mail Message with built-in table of Counts.

Kevin Shyr — Wed, 02 Feb 2011 21:47:05 GMT

For the record, this can be just as easily done with a .NET class with a DLL call. The two major reasons for this ending up as a SSIS package are:

There are a lot of SQL resources in the organization for maintenance, but not as many .NET developers.
There is an existing automated process that links up SQL Jobs (more on that in the next post), and this is part of that process.

To start, this is what the SSIS looks like:

The first part of the control flow is just for the override scenario.

In the Execute SQL Task, it calls a stored procedure, which already formats the result into XML by using "FOR XML PATH('Row'), ROOT(N'FieldingCounts')". The result XML string looks like this:

<FieldingCounts>
<Row>
 <CellId>M COD</CellId>
 <Mailed>64</Mailed>
 <ReMailed>210</ReMailed>
 <TotalMail>274</TotalMail>
 <EMailed>233</EMailed>
 <TotalSent>297</TotalSent>
</Row>
<Row>
 <CellId>M National</CellId>
 <Mailed>11</Mailed>
 <ReMailed>59</ReMailed>
 <TotalMail>70</TotalMail>
 <EMailed>90</EMailed>
 <TotalSent>101</TotalSent>
</Row>
<Row>
 <CellId>U COD</CellId>
 <Mailed>91</Mailed>
 <ReMailed>238</ReMailed>
 <TotalMail>329</TotalMail>
 <EMailed>291</EMailed>
 <TotalSent>382</TotalSent>
</Row>
<Row>
 <CellId>U National</CellId>
 <Mailed>63</Mailed>
 <ReMailed>286</ReMailed>
 <TotalMail>349</TotalMail>
 <EMailed>374</EMailed>
 <TotalSent>437</TotalSent>
</Row>
</FieldingCounts>

This result is saved into an internal SSIS variable with the following settings on the General tab and the Result Set tab:

Now comes the trickier part. We need to use the XML Task to format the XML string result into an HTML table, and I used Direct input XSLT

And here is the code of XSLT:

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method="html" indent="yes"/>
<xsl:template match="/ROOT">
 <table border="1" cellpadding="6">
 <tr>
 <td></td>
 <td>Mailed</td>
 <td>Re-mailed</td>
 <td>Total Mail (Mailed, Re-mailed)</td>
 <td>E-mailed</td>
 <td>Total Sent (Mailed, E-mailed)</td>
 </tr>
 <xsl:for-each select="FieldingCounts/Row">
 <tr>
 <xsl:for-each select="./*">
 <td>
 <xsl:value-of select="." />
 </td>
 </xsl:for-each>
 </tr>
 </xsl:for-each>
 </table>
</xsl:template>
</xsl:stylesheet>

Then a script task is used to send out an HTML email (as we are all painfully aware that SSIS Send Mail Task only sends plain text)

using System;

using System.Data;

using Microsoft.SqlServer.Dts.Runtime;

using System.Windows.Forms;

using System.Net.Mail;

using System.Net;

namespace ST_b829a2615e714bcfb55db0ce97be3901.csproj

{

[System.AddIn.AddIn("ScriptMain", Version = "1.0", Publisher = "", Description = "")]

public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase

{

#region VSTA generated code

enum ScriptResults

{

Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,

Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure

};

#endregion

public void Main()

{

String EmailMsgBody = String.Format("<HTML><BODY>{0}{1}</BODY></HTML>"

, Dts.Variables["Config_SMTP_MessageSourceText"].Value.ToString()

, Dts.Variables["InternalStr_CountResultAfterXSLT"].Value.ToString());

MailMessage EmailCountMsg = new MailMessage(Dts.Variables["Config_SMTP_From"].Value.ToString().Replace(";", ",")

, Dts.Variables["Config_SMTP_Success_To"].Value.ToString().Replace(";", ",")

, Dts.Variables["Config_SMTP_SubjectLinePrefix"].Value.ToString() + " " + Dts.Variables["InternalStr_FieldingDate"].Value.ToString()

, EmailMsgBody);

//EmailCountMsg.From.

EmailCountMsg.CC.Add(Dts.Variables["Config_SMTP_Success_CC"].Value.ToString().Replace(";", ","));

EmailCountMsg.IsBodyHtml = true;

SmtpClient SMTPForCount = new SmtpClient(Dts.Variables["Config_SMTP_ServerAddress"].Value.ToString());

SMTPForCount.Credentials = CredentialCache.DefaultNetworkCredentials;

SMTPForCount.Send(EmailCountMsg);

Dts.TaskResult = (int)ScriptResults.Success;

}

Note on this code: notice the email list has Replace(";", ","). This is only here because the list is configurable in the SQL Job Step at Set Values, which does not react well with colons as email separator, but system.Net.Mail only handles comma as email separator, hence the extra replace in the string.

The result is a nicely formatted email message with count information:

What is the advantage of WCF

Kevin Shyr — Fri, 19 Feb 2010 20:07:12 GMT

I knew the basic idea of WCF and how to build one, but could never win an argument on why it is better than a web service.

Thanks for Christopher Roberts of Oakwood Systems Group Inc. sending me this link. I can answer that question much better now.

http://msdn.microsoft.com/en-us/library/dd943056.aspx

Convert GridViewRows into a DataTable to edit data in ASP .NET

Kevin Shyr — Fri, 19 Feb 2010 19:31:55 GMT

I recently ran into a business problem that I can't find any sample code online. I think I worked out a tolerable solution.

The business scenario is this:

All rows retrieve from the database must be in edit mode, but only some fields should be in edit mode, not all. Editing one row at a time is out of the question.
User might make changes and THEN want to sort the grid. Any user input needs to be preserved.
Have to use .NET control that come out of the box with Visual Studio due to maintenance concerns.

The solution i came up with basically get the GridViewRows on postback and put everything into a datatable, then I am free to do whatever I want with it. Also, there is no hard-coded control ID or column name or cell index, which makes it more reuseable.

Solution limitation:

Only 2 type of edit controls are considered: TextBox and DropDownList. Their ID have to be "txt/ddl" + database column name
Any additional field has to come before the first column or after the last column

Code (in VB.NET)

GridView:

<asp:GridView ID="grdClaims" runat="server" style="position: absolute; top: 224px; left:10px;"

AutoGenerateColumns="False" AllowPaging="False" EnableViewState="True"

AllowSorting="True" EmptyDataText="No claims found." EnableTheming="False" >

<asp:TemplateField HeaderText="Select" ItemStyle-CssClass="gridtext">

<asp:CheckBox ID="chkSelectAll" runat="server" onclick="fnSelectAll(this.checked)" />

</HeaderTemplate>

<asp:CheckBox ID="chkSelect" runat="server" />

</ItemTemplate>

</asp:TemplateField>

<asp:BoundField DataField="CLAIMNR" HeaderText="Claim#" SortExpression="CLAIMNR">

</asp:BoundField>

A Drop Down List in the GridView

<asp:TemplateField HeaderText="Delete" SortExpression="DELETECLMFLAG">

<asp:DropDownList ID="ddlDELETECLMFLAG" runat="server" Text='<%#DataBinder.Eval(Container.DataItem, "DELETECLMFLAG").ToString().Trim()%> '>

<asp:ListItem>Y</asp:ListItem>

<asp:ListItem>N</asp:ListItem>

</asp:DropDownList>

</ItemTemplate>

</asp:TemplateField>

Code behind:

[Updated on 2/22/2010 to fix a bug and for the code to be more efficient]

Dim dtClaims As System.Data.DataTable

Dim iColOffSet As Integer = 2 'This is because the display grid has 2 extra cells at the beginning

dtClaims = dtInsertClaimDetails.Clone() 'dtModifyClaimDetails was cloned the first time the dataset was populated

' Limitation for GridView:

' Data not from the datasource must be at the beginning or the end of the layout, and iColOffSet has to be set

' The code currently assumes that only TextBox or Drop Down List will be used and their ID must be "txt" + ColumnName for

' TextBox or "ddl" + ColumnName for Drop Down List

For Each drGridView As GridViewRow In gvInsertClaimDetails.Rows

Dim dr As System.Data.DataRow = dtClaims.NewRow()

For i As Integer = iColOffSet To dr.ItemArray.Count + iColOffSet - 1

Dim sColumnName As String = dtClaims.Columns(i - iColOffSet).ColumnName.ToString()

Dim sInputOrData As String

If Not drGridView.Cells(iColOffSet).FindControl("txt" + sColumnName) Is Nothing Then

'Found the textbox

sInputOrData = CType(drGridView.Cells(iColOffSet).FindControl("txt" + sColumnName), TextBox).Text

ElseIf Not drGridView.Cells(iColOffSet).FindControl("ddl" + sColumnName) Is Nothing Then

'Found the drop down list

sInputOrData = CType(drGridView.Cells(iColOffSet).FindControl("ddl" + sColumnName), DropDownList).SelectedValue

Else

sInputOrData = drGridView.Cells(i).Text

End If

' If the data is empty, do not insert back into datatable if the type is numbers, leave them NULL

If (dtClaims.Columns(i - iColOffSet).DataType.ToString() = "System.Int32" _

Or dtClaims.Columns(i - iColOffSet).DataType.ToString() = "System.Double") _

And (String.IsNullOrEmpty(sInputOrData) Or sInputOrData = " ") Then

' Do nothing

Else

' Set the column

dr.Item(i - iColOffSet) = System.Convert.ChangeType(sInputOrData, dtClaims.Columns(i - iColOffSet).DataType)

End If

dtClaims.Rows.Add(dr)

Return dtClaims

[2010-04-16] Update:

The performance of this is good for a form with 30 columns (20 editable fields) for about 20 rows (a couple of seconds), but pretty bad for the same form with about 300 rows (1 minute). This form was built with a couple of drop down list filters at the top of the page to filter result, so the general purpose is served. This benchmark was done on P4 single core server with 3 GB RAM with default performance setting for the web server (and SQL on the same server).