Anne Bougie

Sucky MSTest and the "WaitAll for multiple handles on a STA thread is not supported" Error

Anne Bougie — Wed, 07 Apr 2010 17:59:50 GMT

If you are doing any multi-threading and are using MSTest, you will probably run across this error. For some reason, MSTest by default runs in STA threading mode. WTF, Microsoft! Why so stuck in the old COM world? When I run the same test using NUnit, I don't have this problem. Unfortunately, my company has chosen MSTest, so I have a lot of testing problems. NUnit is so much better, IMO.

After determining that I wasn't referencing any unmanaged code that would flip the thread into STA, which can also cause this error, the only thing left was the testing suite I was using. I dug around a little and found this obscure setting for the Test Run Config settings file that you can't set using its interface. You have to open it up as a text file and add the following setting. This is for VS2008.

<ExecutionThread apartmentState="MTA" />

This didn't break any other tests, so I'm not sure why STA is the default, or why there is nothing in the test run configuration app to change this setting.

Here is the code I was testing:

public void ProcessTest(ProcessInfo[] infos)
{
   WaitHandle[] waits = new WaitHandle[infos.Length];
   int i = 0;
   foreach (ProcessInfo info in infos)
   {
      AutoResetEvent are = new AutoResetEvent(false);
      info.Are = are;
      waits[i++] = are;
 
      Processor pr = new Processor();
      WaitCallback callback = pr.Process;
      ThreadPool.QueueUserWorkItem(callback, info);
   }
 
   WaitHandle.WaitAll(waits);
}

Encrypt and Decrypt Strings

Anne Bougie — Tue, 02 Mar 2010 16:41:58 GMT

Here is a quick solution to encrypting and decrypting passwords, or any strings, easily. Write a test that runs the GetNewKey method to generate a unique key. I put the key right in the class here for demonstration purposes, but you can store it anyplace. Write another test to encrypt your password so you know the value to save in your configuration settings, and you're all set. Just remember not to save your test with the clear text password in it!

public static class Helper
{
   private static string _key = "DFQoC2KFQNdbs7fLUNjZQB9qDsbHLylC";
 
   public static string EncryptPassword(this string clearText)
   {
      byte[] originalAsBytes = Encoding.ASCII.GetBytes(clearText);
      byte[] result;
      byte[] keyArray = Convert.FromBase64String(_key);
      using (var tdes = new TripleDESCryptoServiceProvider())
      {
         tdes.Key = keyArray;
         tdes.Mode = CipherMode.ECB;
         tdes.Padding = PaddingMode.Zeros;
 
         ICryptoTransform transform = tdes.CreateEncryptor();
         result = transform.TransformFinalBlock(originalAsBytes, 0, originalAsBytes.Length);
      }
 
      string encrypted = Convert.ToBase64String(result);
      return encrypted;
   }
 
   public static string DecryptPassword(this string encrypted)
   {
      byte[] originalAsBytes = Convert.FromBase64String(encrypted);
      byte[] keyArray = Convert.FromBase64String(_key);
      var tdes = new TripleDESCryptoServiceProvider { Key = keyArray, Mode = CipherMode.ECB, Padding = PaddingMode.Zeros };
 
      ICryptoTransform transform = tdes.CreateDecryptor();
      byte[] result = transform.TransformFinalBlock(originalAsBytes, 0, originalAsBytes.Length);
      result = StripZeroPadding(result);
      string clearText = Encoding.ASCII.GetString(result);
      return clearText;
   }
 
   public static string GetNewKey()
   {
      var tdes = new TripleDESCryptoServiceProvider { KeySize = 192 };
      tdes.GenerateKey();
      byte[] keyArray = tdes.Key;
      string key = Convert.ToBase64String(keyArray);
      return key;
   }
 
   private static byte[] StripZeroPadding(byte[] array)
   {
      int i = array.Length - 1;
      int j = array.Length;
      while (array[i] == 0)
      {
         j = i;
         i--;
      }
 
      var result = new byte[j];
      for (i = 0; i < j; i++)
         result[i] = array[i];
      return result;
   }
}

Here are the NUnit tests:

[Test]
public void GetNewKeyTest()
{
   string key = Helper.GetNewKey();
   Console.WriteLine(key);
}
 
[Test]
public void EncryptDecryptPasswordTest()
{
   string password = "password";
   string encrypted = password.EncryptPassword();
   Console.WriteLine(encrypted);
   Assert.AreNotEqual(password, encrypted);
 
   string decrypted = encrypted.DecryptPassword();
   Assert.AreEqual(password, decrypted);
}

Symantec Customer Dis-Service Department

Anne Bougie — Mon, 01 Mar 2010 19:51:36 GMT

What's up with Symantec? My problems with them actually started a couple of months ago when I upgraded to Windows 7. I wanted to install the Norton anti-virus program that I have a subscription for. When I've needed it before, I could go to their site, log in, and re-download my product with no problem. This time, I couldn't find it. I kept being routed to an upgrade screen. I didn't want to upgrade since I was planning on dumping them and going to Kaspersky when my subscription expired anyway. I tried calling their customer support., but gave up after being on hold for about 30 minutes. I just went ahead and bought Kaspersky and figured I'd just cancel my subscription when they sent me a renewal. Well, it came two weeks ago, and I've been trying ever since then to contact them to cancel the subscription with no luck. When you call the customer service number at the bottom of the email, the <very long> message asks if you are calling about a recent charge on your credit card you don't recognize, and about how it's for their award-winning blah blah blah. OK, I'm a little annoyed by having to listen to this announcement which takes over a minute to find out how to reach someone I can talk to. So, I follow the menu commands to get in the right queue and wait..... and wait..... and wait. The recorded message from the overly-perky message-lady voice tells me, "We appreciate your patience, please don't hang up now. If you hang up and call back now, this will increase your wait time. And all calls are answered in the order they are received." After about 25 minutes, my cell phone dies. I tried over a period of days at several different times to get a hold of them to no avail. I was starting to look bad at work spending so much time with my phone stuck to my ear. I am a contractor, so I can't really spend hours on personal phone calls at work. Not that it looks any better for employees, but it's worse for contractors. I finally took off of work with the intention of getting this resolved. I made sure my phone had a good charge and called. And waited....and waited.....and waited..... I'm not sure how long I waited. I know it was well over an hour, and closer to two hours. I am wondering at this point if there really are any people there answering the phones. I am picturing this big automated center that holds all these calls with real people whose job is to occasionally interact with someone just to put them in another queue. Heck, they get their money by setting you up on their automatically renewing subscription "service". I think the real service is them serving themselves to your money. At this point, I've given up on trying to get my $45 back from them. I think I will tell my credit card company to cancel my card and issue me a new one so at least next year at this time I won't be stuck with another unwanted charge.

Using Delegates in Rhino Mocks With the .Do Statement

Anne Bougie — Fri, 21 Aug 2009 15:51:00 GMT

I use Rhino Mocks pretty much exclusively for mocking and testing. I find that there really isn't anything that I've wanted to do that I haven't been able to figure out how to accomplish with Rhino Mocks. What is everyone else using? I'd like to know so I can take a look.

Anyway, as you know, with Expect.Call we can return property values, return values from functions, throw an error, etc. Sometimes just returning a static value isn't going to work, or it's just easier to give it some simple code to run. When you have many components, all with with their mockable interfaces, sometimes when you need to write a test that is in the middle, you need to run some simple code to test certain scenarios. I think this feature can be used inappropriately if it is overused, but it definitely has it's place when there's just no other good way to set up the test scenario.

Here is a simple example of mocking out a billing component. I admit, it's a bit contrived, but demonstrates the use of delegates in Rhino Mocks. I have an interface named IBilling that I'm mocking. It's purpose is to bill several charges that fall within a billing period. The first four Expect.Call statements simply provide a date value. The .Do statement calls a delegate that passes in a function in my testing module. When your test code hits that function call, instead of just returning a predefined value, it will execute the function you passed in.

[TestMethod]
private void ChargePeriodEndsBeforeBillingPeriodDailyChargeTest()
{
   MockRepository mocks = new MockRepository();
   IBilling billing = mocks.DynamicMock<IBilling>();
   IRateCalculator rates = mocks.DynamicMock<IRateCalculator>();

   int customerId = 1;

   using (mocks.Record())
   {
Expect.Call(billing.GetPeriodBeginDate())
   .Return(DateTime.Parse("1/1/2010");
Expect.Call(billing.GetChargeBeginDate())
   .Return(DateTime.Parse("1/1/2010");
Expect.Call(billing.GetPeriodEndDate())
         .Return(DateTime.Parse("2/1/2010");
      Expect.Call(billing.GetChargeEndDate())
         .Return(DateTime.Parse("1/15/2010");
      Expect.Call(billing.CalculateChargeDays(
         new DateTime(), new DateTime()
         .IgnoreArguments()
         .Do(new CalcDaysDelegate(DaysDifference));
      Expect.Call(rates.GetRate(0)).Return(0.1);
      Expect.Call(() => billing.AddCharge(0, 0, new DateTime()))
         .IgnoreArguments()
         .Constraints(Is.Equal(customerId),
                      Is.Equal(1.6),
                      Is.Anything());
   }
   using (mocks.Playback())
   {
      ChargeBilling chargeBilling = new ChargeBilling(customerId, billing);
      chargeBilling.AddChargeToBill();
   }
}

private delegate int CalcDaysDelegate(DateTime start, DateTime end);
private int DaysDifference(DateTime start, DateTime end)
{
   TimeSpan ts = end - start;
   return ts.Days + 1;
}

In real life I would recommend just returning a "16" to the CalculateChargeDays function call, but a real-life scenario would be too long of a post ; -) I also want to point out the use of the lambda expression for calling a function call with a void return value, () => billing.AddCharge. You can also call it using the delegate keyword, but I find the lambda syntax to be cleaner and easier to read, much less easier to remember when you're coding.

Happy coding, and please let me know what you think about other mocking software you use that I might want to try.

Create an ADODB Recordset in code from XML

Anne Bougie — Fri, 03 Jul 2009 00:54:40 GMT

Recently I have been doing some work on an older system using ADODB recordsets. Every now and then we still have to support these old systems. I wanted to create an open recordset in code on the .NET side to pass to the older system, but ADODB really wants an open database connection in order to open a recordset. My first obvious attempt was to create the recordset object, then create the field objects and add them to the recordset. I could create the recordset this way, but I couldn't open it, and a recordset must be open in order to add data rows. What ADODB really wanted was a table that matched my schema so it could connect to it before adding rows. This was just not practicable.

I discovered that you can open an ADODB recordset from an XML data source, and with the ADODB Stream object, that source can be in memory. Since this was something that I needed to use a lot for this project, I wrote a helper class, named RecordSetHelper, that will create the xml with an easy-to-use interface. I used Linq XML objects to create the XML string, then some rather simple code to put the pieces together into the correct schema that ADODB was expecting. If anyone wants the complete code, I can email it to them. The high points are below.

Here is the xml for a simple ADODB recordset with a couple of data rows:

<xml xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882"
     xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882"
     xmlns:rs="urn:schemas-microsoft-com:rowset"
     xmlns:z="#RowsetSchema">
<s:Schema id="RowsetSchema">
    <s:ElementType name="row" content="eltOnly" rs:updatable="true">
      <s:AttributeType name="FirstName" rs:number="1" rs:write="true">
        <s:datatype
            dt:type="string"
            rs:dbtype="str"
            dt:maxLength="200"
            rs:precision="0"
            rs:fixedlength="true"
            rs:maybenull="false" />
      </s:AttributeType>
      <s:AttributeType name="LastName" rs:number="2" rs:write="true">
        <s:datatype
            dt:type="string"
            rs:dbtype="str"
            dt:maxLength="200"
            rs:precision="0"
            rs:fixedlength="true"
            rs:maybenull="false" />
      </s:AttributeType>
      <s:AttributeType name="DOB" rs:number="3" rs:write="true">
        <s:datatype
            dt:type="dateTime"
            rs:dbtype="variantdate"
            dt:maxLength="16"
            rs:precision="0"
            rs:fixedlength="true"
            rs:maybenull="false" />
      </s:AttributeType>
      <s:extends type="rs:rowbase" />
    </s:ElementType>
</s:Schema>
<rs:data>
    <rs:insert>
      <z:row
            FirstName="Hugh"
            LastName="Morris"
            DOB="1975-02-05T00:00:00" />
      <z:row
            FirstName="Anne"
            LastName="Teak"
            DOB="1932-06-07T00:00:00" />
      <z:row
            FirstName="Jean"
            LastName="Poole"
            DOB="1989-05-05T00:00:00" />
    </rs:insert>
</rs:data>
</xml>

Creating the XML in code each time seemed rather tedious, so I made an interface to create a recordset using the RecordSetHelper. Here is an NUnit test that uses the RecordSetHelper to create an open recordset that is identical to the xml listed above.

      [Test]
      public void RecordsetHelperTest()
      {
         RecordSetHelper helper = new RecordSetHelper();
         helper.AddField(new FieldItem
            {
               Name = "FirstName",
               FieldType = VariantType.String
            });
         helper.AddField(new FieldItem
            {
               Name = "LastName",
               FieldType = VariantType.String
            });
         helper.AddField(new FieldItem
            {
               Name = "DOB",
               FieldType = VariantType.Date
            });
         helper.AddDataRow(new object[]
            {
               "Hugh", "Morris", "2/5/1975"
            });
         helper.AddDataRow(new object[]
            {
               "Anne", "Teak", "6/7/1932"
            });
         helper.AddDataRow(new object[]
            {
               "Jean", "Poole", "5/5/1989"
            });
         XDocument xRS = helper.MakeXML();
         ADODB.Recordset rs = helper.GetRecordSet();
         Assert.IsInstanceOf(typeof(ADODB.Recordset), rs);
         Assert.AreEqual((int)ObjectStateEnum.adStateOpen, rs.State);
      }

Here is the code to take the XML (returned by the MakeXml function) and turn it into an open ADODB recordset that you can manipulate. The stream is the ADODB.Stream object, not a .NET Stream object.

      public Recordset GetRecordSet()
      {
         var doc = MakeXML();
         var rs = new Recordset();
         var stream = new Stream();
         stream.Open(
            Type.Missing,
            ConnectModeEnum.adModeUnknown,
            StreamOpenOptionsEnum.adOpenStreamUnspecified,
            null,
            null);
         stream.WriteText(doc.ToString(), StreamWriteEnum.adWriteChar);
         stream.Position = 0;
         rs.Open(
            stream,
            Type.Missing,
            CursorTypeEnum.adOpenUnspecified,
            LockTypeEnum.adLockUnspecified, 0);

         return rs;
      }

The MakeXML is the method that takes all the pieces needed to create the xml string and puts them all together.

      public XDocument MakeXML()
      {
         var root = MakeRoot();
         var fields = GetFieldObjects();
         var rowSetSchema = MakeFieldsXml(fields);

         root.Add(rowSetSchema);
         if (_rows.Count > 0)
         {
            var data = MakeDataXml();
            root.Add(data);
         }
         var xdoc = new XDocument(root);
         return xdoc;
      }

The first thing to do was to create the XML namespace objects to be used when attaching the field and data elements. I created these as fields at the class level.

      internal XNamespace xns
      {
         get { return "http://www.w3.org/2000/xmlns/"; }
      }
      internal XNamespace xnss
      {
         get { return "uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882"; }
      }
      internal XNamespace xnsdt
      {
         get { return "uuid:C2F41010-65B3-11d1-A29F-00AA00C14882"; }
      }
      internal XNamespace xnsrs
      {
         get { return "urn:schemas-microsoft-com:rowset"; }
      }
      internal XNamespace xnsz
      {
         get { return "#RowsetSchema"; }
      }

The "MakeRoot" function creates the root element, named "xml". The rest of the code just creates each piece and adds it to the root element. There is some funny kind of syntax needed to create the namespace attributes. You have to use the "+" operator with the XNamespace object and the prefix, then add the urn as the value. I struggled for a while with this and finally looked up the answer. It still seems like some strange idiomatic syntax, but it works.

      private XElement MakeRoot()
      {
         var root = new XElement("xml",
             new XAttribute(xns + "s", "uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882"),
             new XAttribute(xns + "dt", "uuid:C2F41010-65B3-11d1-A29F-00AA00C14882"),
             new XAttribute(xns + "rs", "urn:schemas-microsoft-com:rowset"),
             new XAttribute(xns + "z", "#RowsetSchema"));
         return root;
      }

Since each field is the same schema, but differs in its name, datatype, and a few other attributes, I created an abstract base class named RSField. The public GetFieldElement method calls the abstract method GetDataTypeElement, which is what creates the distinct xml for each data type correctly.

   public abstract class RSField
   {

      internal XNamespace xns
      {
         get { return "http://www.w3.org/2000/xmlns/"; }
      }
      internal XNamespace xnss
      {
         get { return "uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882"; }
      }
      internal XNamespace xnsdt
      {
         get { return "uuid:C2F41010-65B3-11d1-A29F-00AA00C14882"; }
      }
      internal XNamespace xnsrs
      {
         get { return "urn:schemas-microsoft-com:rowset"; }
      }
      protected XElement FieldElement;
      protected string FieldName { get; set; }
      protected int Ordinal { get; set; }

      protected RSField(string fieldName, int ordinal)
      {
         FieldName = fieldName;
         Ordinal = ordinal;
      }

      protected abstract XElement GetDataTypeElement();

      public XElement GetFieldElement()
      {
         var element = new XElement(xnss + "AttributeType",
             new XAttribute("name", FieldName),
             new XAttribute(xnsrs + "number", Ordinal.ToString()),
             new XAttribute(xnsrs + "write", "true"));
         element.Add(GetDataTypeElement());
         return element;
      }
   }

Here is an example of an overriding class:

   public class RSIntegerField : RSField
   {
      public RSIntegerField(string fieldName, int ordinal)
         : base(fieldName, ordinal)
      {
      }

      protected override XElement GetDataTypeElement()
      {
         var element = new XElement(xnss + "datatype",
             new XAttribute(xnsdt + "type", "i8"),
             new XAttribute(xnsdt + "maxLength", "8"),
             new XAttribute(xnsrs + "precision", "0"),
             new XAttribute(xnsrs + "fixedlength", "true"),
             new XAttribute(xnsrs + "maybenull", "false"));
         return element;
      }
   }

In the MakeXML method, after creating the root element, all the field objects are created. Each field object is created. The field objects are of type RSField, which contains the xml for each field.

      private RSField[] GetFieldObjects()
      {
         var fields = new RSField[_fields.Count];
         for (int i = 0; i < _fields.Count; i++)
         {
            FieldItem field = _fields.ElementAt(i);
            switch (field.FieldType)
            {
               case VariantType.Integer:
               case VariantType.Long:
               {
                  fields[i] = new RSIntegerField(field.Name, (i + 1));
                  break;
               }
               case VariantType.Single:
               {
                  fields[i] = new RSSingleField(field.Name, (i + 1));
                  break;
               }
               case VariantType.Double:
               {
                  fields[i] = new RSDoubleField(field.Name, (i + 1));
                  break;
               }
               case VariantType.Currency:
               {
                  fields[i] = new RSCurrencyField(field.Name, (i + 1));
                  break;
               }
               case VariantType.Date:
               {
                  fields[i] = new RSDateField(field.Name, (i + 1));
                  break;
               }
               case VariantType.String:
               {
                  fields[i] = new RSTextField(field.Name, (i + 1));
                  break;
               }
               case VariantType.Boolean:
               {
                  fields[i] = new RSBooleanField(field.Name, (i + 1));
                  break;
               }
               case VariantType.Byte:
               {
                  fields[i] = new RSByteField(field.Name, (i + 1));
                  break;
               }
               default:
               {
                  throw new ArgumentException("Field type is not valid");
               }
            }
         }
         return fields;
      }

The objects collection that this is looping through are the type of a FieldItem class used to hold the name and data type of the field. In the NUnit test above, I added several fields using FieldItem. I decided to use the Microsoft.VisualBasic namespace for the data type enum, but I could also have used the ADODB.DataTypeEnum, but the names seemed clunkier; "adBigInt" instead of just "Long".

   public class FieldItem
   {
      public string Name { get; set; }
      public VariantType FieldType { get; set; }
   }

The MakeFieldsXML, which is called from the MakeXML method, takes all the fields of RSField type and creates the Linq XElement tree for all the fields. This is then attached to the root element.

      private XElement MakeFieldsXml(IEnumerable<RSField> fields)
      {
         var schema = new XElement(xnss + "Schema",
               new XAttribute("id", "RowsetSchema"));
         var elementType = new XElement(xnss + "ElementType",
               new XAttribute("name", "row"),
               new XAttribute("content", "eltOnly"),
               new XAttribute(xnsrs + "updatable", "true"));
         foreach (var field in fields)
            elementType.Add(field.GetFieldElement());
         elementType.Add(new XElement(xnss + "extends",
            new XAttribute("type", "rs:rowbase")));
         schema.Add(elementType);
         return schema;
      }

The data is passed to the RecordSetHelper in the form of an object array. The array must have the same number of elements as the number of fields. The MakeDataXML method creates the xml that contains all the data rows.

      private XElement MakeDataXml()
      {
         var data = new XElement(xnsrs + "data");
         var insert = new XElement(xnsrs + "insert");
         foreach (var row in _rows)
         {
            var rowElement = new XElement(xnsz + "row");
            for (int i = 0; i < _fields.Count; i++ )
            {
               var field = _fields.ElementAt(i);
               object value = row[i];
               string s = value.ToString(field.FieldType);
               var val = new XAttribute(field.Name, s);
               rowElement.Add(val);
            }
            insert.Add(rowElement);
         }
         data.Add(insert);
         return data;
      }

The only thing special about this piece of code is the ToString method on the value object-type variable. Just using the base ToString didn't work since it was of type "object", and the value may have been entered as a string, it may not have internally converted it to the correct type. Also, the date fields had to be in a certain format or it would crash when you retrieved the value from the recordset. So I created an extension method overriding the base ToString on object. It just converts the object to the correct underlying data type before converting to a string.

      public static string ToString(this object me, VariantType type)
      {
         string s;
         switch (type)
         {
            case VariantType.Integer:
            {
               s = Convert.ToInt32(me).ToString();
               break;
            }
            case VariantType.Long:
            {
               s = Convert.ToInt64(me).ToString();
               break;
            }
            case VariantType.Single:
            {
               s = Convert.ToSingle(me).ToString();
               break;
            }
            case VariantType.Double:
            {
               s = Convert.ToDouble(me).ToString();
               break;
            }
            case VariantType.Currency:
            {
               s = Convert.ToDecimal(me).ToString();
               break;
            }
            case VariantType.Byte:
            {
               s = Convert.ToByte(me).ToString();
               break;
            }
            case VariantType.Date:
            {
               DateTime dtValue = DateTime.Parse(me.ToString());
               s = dtValue.ToString("s");
               break;
            }
            case VariantType.Boolean:
            {
               s = Convert.ToBoolean(me).ToString();
               break;
            }
            default:
            {
               s = Convert.ToString(me);
               break;
            }
         }
         return s;
      }

If you find this useful, and you would like the complete source code, let me know and I'll email you the project.

Technorati Tags:
ADODB
Linq
.NET
XML

Part 2: Copy a SharePoint List Programmatically

AnneBougie — Sun, 15 Mar 2009 20:14:38 GMT

I started this series to show how to copy different kinds of lists in SharePoint. Each type of list has its own challenges. In this article I will also demonstrate how to copy custom fields and keep the same internal name. It’s kind of annoying that you can’t create a field with the internal name you specify, but I found a pretty easy workaround. You create the field using AddFieldAsXml on the list Fields collection, and you use the internal name as the display name. This creates the field with the correct internal name. Then just change the Title property of the field to what you want displayed. Problem solved.

Copying a document library is similar to copying a generic list, except that you also need to copy the document file attached to the list item. You also need to determine the template that was used to create the original list, and use that template to create the new list. That way, when users add a new document, it is of the right document type. Another gotcha in copying document libraries, is that there is a subfolder structure that needs to be copied recursively to get everything in its correct place.

Here's the code. I copied it out of my project, which has everything in its own method, so I apologize if some of my capitalization or field names are not correct. Also, there is a function referenced, CopyFieldValues. If you need to know how to do this, refer to Part 1 of this series. Document libraries can have field values just like other lists, so you'll need to copy those too if you want a complete copy of the list.

// get the document template to use when creating the new list
SPDocTemplate docTemplate;
SPDocTemplateCollection templates = destinationWeb.DocTemplates;
XDocument templateSchema = XDocument.Parse(templates.SchemaXml);
IEnumerable<XElement> paths = templateSchema.Descendants("DocumentTemplateFile");

string sourceUrl = sourceDocumentTemplateUrl.Substring(sourceDocumentTemplateUrl.IndexOf("/") + 1);
XElement path = paths.Where(p => p.Attribute("TargetName").Value == sourceUrl).Single();
XElement docTemplate = path.Parent.Parent;
int type = Convert.ToInt32(docTemplate.Attribute("Type").Value);

SPDocTemplate destTemplate = null;
foreach (SPDocTemplate template in templates)
{
   if (template.Type == type)
      docTemplate = template;
}

Guid newListID = destinationWeb.Lists.Add(sourceList.Title,
                     sourceList.Description,
                     destinationWeb.ListTemplates["Document Library"], docTemplate);

SPList destinationList = destinationWeb.Lists[newListID];

// add any custom fields that were created
foreach (SPField field in SourceList.Fields)
{
   if (!DestinationList.Fields.ContainsField(field.InternalName))
   {
      // create field with correct internal name
      XDocument fieldXdoc = XDocument.Parse(field.SchemaXml);
      XElement fieldElement = fieldXdoc.Element("Field");
      XAttribute displayName = fieldElement.Attribute("DisplayName");
      displayName.Value = field.InternalName;
      string fieldName = DestinationList.Fields.AddFieldAsXml(fieldXdoc.ToString());
      SPField newfield = DestinationList.Fields.GetField(fieldName);
      newfield.Title = field.Title;
   }
}

Here is the code for CopyDocumentLibraryItems, which needs to be recursive to get all the items in all the subfolders:

      protected virtual void CopyDocumentLibraryItems(SPFolder sourceFolder, SPFolder destFolder)
      {
         // get a list of items in the source folder
         SPListItemCollection items = ((SPDocumentLibrary)SourceList).GetItemsInFolder(SourceList.DefaultView, sourceFolder);

         foreach (SPListItem item in items)
         {
            SPFileSystemObjectType myType = item.FileSystemObjectType;
            string relativeDestinationUrl = "{0}/{1}";
            if (myType == SPFileSystemObjectType.File)
            {
               SPFile fi = item.File;
               byte[] filebytes = fi.OpenBinary();
               relativeDestinationUrl = string.Format(relativeDestinationUrl, destFolder.Url, fi.Name);
               SPFile newFile = destFolder.Files.Add(relativeDestinationUrl, filebytes, true);
               SPListItem newItem = newFile.Item;
               CopyFieldValues(item, newItem);
               newItem.UpdateOverwriteVersion();
            }
            else
            {
               relativeDestinationUrl = string.Format(relativeDestinationUrl, destFolder.Url, item.Folder.Name);
               SPFolder destSubFolder = destFolder.SubFolders.Add(relativeDestinationUrl);
               SPFolder sourceSubFolder = item.Folder;
               // copy everything in sub folder
               CopyDocumentLibraryItems(sourceSubFolder, destSubFolder);
            }
         }
      }

Technorati Tags: SharePoint

SharePoint Content Deployment Issue

AnneBougie — Fri, 06 Mar 2009 15:01:18 GMT

The following error can occur during content deployment:

The changeToken refers to a time before the start of the current change log.

The changeToken refers to a time before the start of the current change log. at Microsoft.SharePoint.Deployment.SPExport.ThrowInvalidChangeTokenError(DateTime minChangeTime, Int32 minChangeNumber) at Microsoft.SharePoint.Deployment.SPExport.GetIncrementalChanges() at Microsoft.SharePoint.Deployment.SPExport.CalculateObjectsToExport() at Microsoft.SharePoint.Deployment.SPExport.Run()

Content deployment job 'Entire site' failed.The exception thrown was 'Microsoft.SharePoint.SPException' : 'The changeToken refers to a time before the start of the current change log.'

This error happens when more than 15 days have passed since the last deployment, and you have the job set to use incremental deployment. The fix is really easy. You just have to change the deployment options on the deployment job to deploy all content instead of deploying only new or changed content. Run the content deployment once, then you can change back to incremental deployment.

Technorati Tags: SharePoint

SharePoint Adventures - Reading an Excel Spreadsheet From A Stream

AnneBougie — Thu, 29 Jan 2009 22:40:12 GMT

When I needed to read an Excel spreadsheet from a SharePoint site, it seemed like a simple enough request. Previously, whenever I needed to open an Excel file, I used an OleDb connection with the following connection string:

string connectionString = @"Provider=Microsoft.ACE.OLEDB.12.0;
       Data Source={0};Persist Security Info=False;
       Extended Properties=""Excel 12.0;HDR=YES""";
connectionString = string.Format(connectionString, filePath);

Of course, you can't open the file from the SharePoint site this way. So, my first hurdle was to figure out how to download the file from the web site. The System.Net namespace provides a very easy way to do this:

WebClient client = new WebClient();
client.UseDefaultCredentials = true;
Stream stream = client.OpenRead(url);

Now I had my file as a stream, so I thought I'd just extract it, save it to a temporary file, then read it in in the usual way. Problem solved. This next bit of code should probably be optimized a little so I'm not resizing the array every single loop:

BinaryReader brdr = new BinaryReader(stream);
byte[] result = new byte[0];
int bufferSize = 32768; // 32k
byte[] buffer = new byte[bufferSize];
long pos = 0;

while (true)
{
      buffer = brdr.ReadBytes(bufferSize);
      if (pos > 0)
      {
            // copy old data to bigger result
            byte[] temp = new byte[result.LongLength];
            Array.Copy(result, temp, result.LongLength);
            result = new byte[temp.LongLength + buffer.Length];
            Array.Copy(temp, result, temp.LongLength);
            // add new data
            for (int i = 0; i < buffer.Length; i++)
            {
                result[pos + i] = buffer[i];
            }
            pos += buffer.Length;
      }
      else
      {
            result = new byte[buffer.Length];
            Array.Copy(buffer, result, buffer.Length);
            pos = buffer.Length;
      }
      if (buffer.Length < bufferSize)
            break;
}
string tempFile = Path.Combine(Environment.GetEnvironmentVariable("TMP"),
      "CopyList.xlsx");
using (var fs = new FileStream(tempFile, FileMode.OpenOrCreate))
{
      var writer = new BinaryWriter(fs);
      writer.Write(result, 0, result.Length);
      writer.Close();
      fs.Close();
}

This worked beautifully on my development machine, but this is part of a larger program that is manipulating a SharePoint site via the SharePoint API. Therefore, it has to run on the server, which is 64-bit. When I first ran it, I received the error "The 'Microsoft.ACE.OLEDB.12.0' provider is not registered on the local machine" . So, I did a little digging around, and found out that maybe I needed to install the driver. It is available from Microsoft as a separate download, so I thought I was good to go. I installed it, and was still receiving the same error. Huh??!! Then I did a little *more* digging, and found out that there is no OleDb driver for opening Excel files on a 64-bit machine. Fine. No problem. Just compile the project as x86, and I'm good to go. Right? Well, wrong, actually. When the project is compiled in 32-bit, I can't access the SharePoint sites. Aaaaarrrrrrrhhhh! What to do? Then, I'm looking all over, trying to find a way to read an Excel file directly from a stream. How hard can it be, right?

<SARCASM>
Actually, Microsoft has this really handy little guide that tells you just how to do it. It's only 349 pages. Turns out Excel's native format is something called a BIFF (Binary Interchange File Format). So, you just have to parse all the bytes from the stream into the right format. What a treat!
</SARCASM>

I know who I'd like to BIFF about now. Anyway, I did *even more* digging around, and I found some code from someone who has done just that. Excel Data Reader Thanks, iciobanu! However, it only works for Excel 2003 format. Now that is finally a problem I can easily solve! Save the spreadsheet to 2003 format, and read it in. Are we there yet? Almost! I was getting errors because the stream object returned from the web download didn't support seeking, which this code relies on. So, save the stream out to a temp file (back to that again), and read it back in as a FileStream object, which supports seeking, and my problems are, at long last, solved.

And that is how you open an Excel file from a SharePoint site into a spreadsheet without using a generic OleDb connection. Sucess at long last! Isn't there some famous quote somewhere about persistence paying off?

Technorati Tags: SharePoint

Part 1: Copy a SharePoint List Programmatically

AnneBougie — Fri, 23 Jan 2009 18:18:43 GMT

I've been working with the SharePoint API lately. One of my tasks is to copy SharePoint lists from one site to another site. This can be done in code as long as the two sites are on the same machine. It's OK if they are in different web applications, but it can't be from one server to another. If you need to copy from one server to another, you either need to do an export/import, or find a way to save the settings and import them on the new machine. I believe you can use the SharePoint Web Services to accomplish this, though. I haven't worked with that yet.

Anyway, it took a lot of digging around to figure out all the things I needed to copy lists, so I thought I'd post the code that is working for me. This post is only going to cover copying a basic list. I am planning on several more posts to handle document libraries, wikis, and other special cases. Each has their own challenges.

The requirements for me are to copy the bulk of the lists of a large SharePoint site to a new site with a different structure. Therefore, content and deploy is not going to work. Also, they are redesiging the pages, so they really only wanted the lists. What I'm doing is creating a new list on the new site and copying all the properties, so any metadata, such as who last modified the list, is unfortunately lost.

Here's the code:

// Instantiate web instances
SPSite sourceSite = new SPSite(@"http://SharePointServer:31001");
SPWeb sourceWeb = sourceSite.RootWeb;
SPSite destSite = new SPSite(@"http://SharePointServer:31002");
SPWeb destWeb = destSite.RootWeb;

// Get a reference to the source list
SPList sourceList = sourceWeb.GetList("/Lists/Announcements");

// if the list exists on the destination site, delete it
try
{
   SPList temp = destWeb.Lists[sourceList.Title];
   destWeb.Lists.Delete(temp.ID);
}
catch { }

// create new list on the destination web with same properties
Guid newListID = destWeb.Lists.Add(sourceList.Title, sourceList.Description,
                                   sourceList.BaseTemplate);
SPList destList = destWeb.Lists[newListID];

// copy items
foreach (SPListItem item in announcements.Items)
{
   SPListItem newDestItem = destList.Items.Add();
   foreach (SPField field in sourceList.Fields)
   {
      if (!field.ReadOnlyField)
      newDestItem[field.Id] = item[field.Id];
   }
   newDestItem.Update();
}

// set quicklaunch settings
destList.OnQuickLaunch = sourceList.OnQuickLaunch;
destList.Update();

Technorati Tags: SharePoint

InternalsVisibleTo with Strongly-Type Assemblies

AnneBougie — Wed, 01 Oct 2008 14:18:41 GMT

For testing, I often use the InternalsVisibleTo for my unit tests. It looks like this:

[assembly: InternalsVisibleTo("MyProject.Domain.Tests")]

When the assembly is strongly typed, though, this simple statement will not work, and we get the following message:

Error: "Strong-name signed assemblies must specify a public key in their InternalsVisibleTo declarations"

To fix this, you must do two things: You must sign the test assembly, and you must include the public key in the InternalsVisibleTo statement.

To get the public key value from the .snk file, use the sn.exe utility included in the .NET SDK. You must do two things, first, create a .snk file that contains only the public key:

sn -p MyKey.snk MyKey.PublicKeyOnly.snk

Then, you can extract the public key value with this command:

sn -tp MyKey.PublicKeyOnly.snk

This will produce a similar output:

Public key is
0024000004800000940000000602000000240000525341310004000001000100cfb8bc23b86a08
e70d021dd53d3b0293e716e71015870bdcc58a0231a4228618851a83e06077f5a44f42beb2baf3
56ad2d344521a96b0081ed0f25f9227523e3625eda524efe1cf2e1e5e41f3693a76ec52347684b
8129a4bb2d5fc49681adf33da0eecc4f81f011af4539d12abe1b4e760b5ce32d766db1012d4402
8381f0b4

Public key token is 2ff2b71993eeff95

Copy the public key value and update the InternalsVisibleTo:

[assembly: InternalsVisibleTo("MyProject.Domain.Tests, PublicKey=
0024000004800000940000000602000000240000525341310004000001000100cfb8bc23b86a08
e70d021dd53d3b0293e716e71015870bdcc58a0231a4228618851a83e06077f5a44f42beb2baf3
56ad2d344521a96b0081ed0f25f9227523e3625eda524efe1cf2e1e5e41f3693a76ec52347684b
8129a4bb2d5fc49681adf33da0eecc4f81f011af4539d12abe1b4e760b5ce32d766db1012d4402
8381f0b4")]

Voila! You can now access the internals from your test project for strong named assemblies.