Fervent Coder

Subversion (SVN) just released version 1.5.0 recently on June 18, 2008.  With it comes bug fixes and boosts in performance. It is highly recommended that you reference the release notes, and pay attention to merge tracking:

Merge tracking means Subversion keeps track of what changes have been merged where. This reduces the overhead involved in maintaining branches, and gives users a way to inquire what changes are merged — or are available to be merged — on different lines of development.

VisualSVN  responded by releasing VisualSVN Server version 1.5 on June 19, which adds easy configuration of Windows Authentication (and support for windows groups) in addition to some other great new features.  Download here.

TortoiseSVN, one of the most popular clients for Windows,  just released version 1.5 on June 21. Download 32 bit edition here and 64 bit here.

VisualSVN also released their popular Visual Studio Integrated Client VisualSVN on June 21. Visual SVN is not free (unless you qualify), but well worth the nominal fee of $49 per seat.  You must have TortoiseSVN installed (not sure if that means only the 32bit version or not). Download VisualSVN here.

New to SVN?

If you are new to SVN, you need both a server and a client tool installed.  Jeff Atwood has an excellent post on how to set up Subversion on Windows.  I prefer to use an easy to configure server that gives you the option to use the command line as well. Thus, I use VisualSVN Server. I use TortoiseSVN on the client side. I am trying out VisualSVN (VS integration) to see if I like it as well. 

With open source, you can easily get analysis paralysis because there are sometimes so many options.  So let me make it easy for you. Download VisualSVN Server and TortoiseSVN.  You can have VisualSVN Server installed and ready to go in less than ten minutes.  It has a great interface and can get you to DONE very fast. TortoiseSVN is a great tool for client side working with SVN.  If you need the Visual Studio integration, then get VisualSVN as well.

Calling a remote application, you want it to run almost always in its native directory and not against your calling End2End code's directory. Why? You don't want to have to verify whether the application uses or will use any kind of relative paths.  You want your End2End Tests to work in any respect.  So you can implement the basic code for starting a process using the System.IO and System.Diagnostics .Net libraries.

Public Sub RunApplication(ByVal filePath As String, ByVal args As String, ByVal waitForExit As Boolean, ByVal setWorkingDirectoryToPath As Boolean)

     Dim processStartInfo As New ProcessStartInfo(filePath, String.Format(" {0}", args))

     processStartInfo.UseShellExecute = False

     processStartInfo.RedirectStandardOutput = False

     processStartInfo.CreateNoWindow = False

     If (setWorkingDirectoryToPath) Then

         processStartInfo.WorkingDirectory = Path.GetDirectoryName(filePath)

     End If

     RunProcess(processStartInfo, waitForExit)

 End Sub

 Private Sub RunProcess(ByVal processInfo As ProcessStartInfo, ByVal waitForExit As Boolean)

     Using process As New Process()

         process.StartInfo = processInfo

         process.Start()

         If (waitForExit) Then

             process.WaitForExit()

         End If

     End Using

 End Sub

We have started an initiative this year. We call it End2End Automated Testing. The idea is based on a couple of factors. Many times in the past we have not had an easy way to verify our configuration, security and deployment to an environment in an automated way.  The second is that regression testing is a very arduous process for our QA staff when it must be completed.  How do we give time back to our testers and verify everything in a given environment is correct? With automation of course!

So what is End2End Testing? It is a way of taking what we expect the system to do based on our inputs and verifying that against the actual results in an automated fashion.  We influence the inputs into a system so we know what we expect the outputs are going to be.  Then we check those outputs, or actual results, against some expected outputs, or expected results files to see if we have matches. This helps free up the Quality Assurance staff to focus more on figuring out what should happen when a given scenario is passed into the system. 

We make it easy enough for them to focus on "If I pass this into the system, this is what I expect to get out of the system or to happen with the system."  The real benefit of this is an actual automated regression suite with a focus on what if for the QA staff! They get to concentrate more on actually breaking, err, testing the system to ensure it works correctly and meets a level of acceptable quality for the ultimate end users.

End2End testing is not a replacement for unit tests.  You need that quick feedback cycle that you get from those.  End2End is also not a replacement for your integration tests.  Although a little slower than unit (logical) tests, integration tests give you feedback much faster than End2End will be able to.  End2End is not a replacement for your testers. They must really focus on specifications to put good scenarios through the system to accurately tests all aspects of what might happen.

End2End is a strong collaboration between developers and testers.  In incorporating End2End, you will work very closely with each other. End2End is unforgiving. When it runs, it only passes if everything is 100% correct against specifications.  It takes awhile to get expected results built to verify actual system results against.  Once you do, the benefit is knowing the application you are creating is correct up to that point.

Principles of Automated End2End Testing

• End2End must be able to affect the inputs to the system.
• End2End must be able to verify results (possibly in a database).
• End2End will couple to those two points only, but will be tightly coupled to those points.
  • This is really no different than a QA (tester) testing a system under development would be coupled.
• End2End is not actually part of your project at all. It is in itself a mini-project.
  • That means it deserves it's own unit tests to ensure it's functionality.
• End2End should start from a known point for both inputs and outputs (preferably empty starting outputs).
• End2End should be able to be completely encapsulated on a LOCAL developer machine as well as on some remote environment.
• That means you may start some more processes or do a couple more setup options locally than you would in other environments.
• End2End should provide a Reset Test to clean up everything for your users or testers so they can start the process over.
• End2End should include a readme file for both your developers and testers.
• End2End should be helpful in identifying what exactly is wrong for every point that is wrong and possibly provide pointers on how to correct the issues.

How To Do the End2End Test

1. Setup Step1: Automatically set up the inputs.  This could be putting a file somewhere or setting up database tables with scripts.
2. Setup Step2: Optional - Set up network permissions on the remote directory.
3. Setup Step3: Call the remote application, have it execute in the remote directory. Optionally, wait for exit.
4. Setup Step4: Optional - Remove network permissions to the directory.
5. Setup Step5: Figure out the trigger to know when your application is ready. You could put a file watcher or a database table watcher to know when ready. Or possibly, waiting for the application to exit is good enough for your needs.
6. Setup Step6: Make a data table from your expected results.
7. Setup Step7: Make a data table from your actual results.
8. Setup Step8: Remove the columns that you have decided not to verify against.  We call these the ignored Fields.
9. Test1: Verify the total rows of expected results versus the actual results.
   1. Print the information. "There are _ rows in the expected results and _ rows in the actual results."
   2. Set the testSuccess variable to false if different.
   3. Continue the test suite.
10. Test2: Verify the column count of the expected results versus the actual results.
    1. Print the information. "There are _ columns in the expected results and _ columns in the actual results."
    2. Set the testSuccess variable to false if different.
    3. Continue the test suite.
11. Test3: Verify the columns are named the same in expected results versus actual results.
    1. Set the testSuccess variable to false if different.
    2. Print the information if fails. "I had column1, column2 in my expected results but could not find them in the actual results.  I had column3 in my actual results but could not find it in the expected results."
    3. Continue the test suite.
12. Test4: Verify the actual data in each row versus the expected result.
    1. Do not expect the results of both to be in order. Filter down and find the specific row.
    2. Print both the expected result row and the actual result row. Include headers if possible.
    3. Set the testSuccess variable to false if different.
    4. Print the information if it fails per column. "I expected a value of _ in column1 but actually had _."
    5. Continue the test suite for every row.
13. Print how many rows passed and how many failed.
14. Verify whether the End2End test failed or not. Assert.IsTrue(testSuccess)