Multithreading with PowerShell using RunspacePool

If you are working with PowerShell, you may want to start running certain scripts in parallel for improved performance. While doing some research on the topic, I found excellent articles discussing Runspaces and how to execute scripts dynamically. However I needed to run custom scripts already stored on disk and wanted a mechanism to specify the list of scripts to run in parallel as an input parameter. 

In short, my objectives were:

Provide a variable number of scripts, as input parameter, to execute in parallelExecute the scripts in parallelWait for completion of scripts, displaying their outputHave a mechanism to obtain success/failure from individual scripts

This article provides two scripts: a “main” script, which contains the logic for executing other scripts in parallel, and a test script that simply pauses for a few seconds.

Async Script

The following script is the main script that calls other scripts in separate processes running Runspace Pools in PowerShell.  This script accepts an input array of individual scripts, or script files; you might need to provide the complete path for script files. Then this script starts a new PowerShell process for each script and creates a reference object in the $jobs collection; the BeginInvoke operation starts the script execution immediately. Last but not least, this script waits for completion of all scripts before displaying their output and result.

It is important to note that this main script expects each script being called to return a specific property called Success to determine whether the script was successful or not. Depending on the script you call, this property may not be available and as a result this main script may report false negatives. There are many ways to detect script failures, so you can adapt the proper method according to your needs. A Test script is provided further down to show you how to return a custom property.

You can the below script like this, assuming you want to execute two scripts (script1.ps1 and script2.ps1):   ./main.ps1 @( {& “c:\myscripts\script1.ps1” }, {& “c:\myscripts\script2.ps1” })

Param([String[]] $toexecute = @())

        Write
    - Output("Received " + $toexecute.Count + " script(s) to execute")

          $rsp =
    [RunspaceFactory] ::CreateRunspacePool(1, $toexecute.Count) $rsp.Open()

        $jobs = @()
#Start all scripts
        Foreach($s in $toexecute) {
  $job = [Powershell] ::Create().AddScript($s) $job.RunspacePool =
      $rsp Write - Output $("Adding script to execute... " + $job.InstanceId)
                       $jobs += New - Object PSObject - Property @ {
        Job = $job Result = $job.BeginInvoke()
      }
}

#Wait for completion
do {
  Start - Sleep - seconds 1 $cnt =
      ($jobs | Where { $_.Result.IsCompleted - ne $true }).Count Write -
      Output("Scripts running: " + $cnt)
} while ($cnt - gt 0)

    Foreach($r in $jobs) {
  Write - Output("Result for Instance: " + $r.Job.InstanceId) $result =
      $r.Job.EndInvoke($r.Result)
#Display complete output of script
#Write - Output($result)

#We are assuming the scripts executed return an object
#with a property called Success
          if ($result.Success) {
    Write - Output " -> Script execution completed successfully"
#Use $result to print output of script
  }
  else {
    Write - Output " -> Script execution failed"
  }
}

Test Script

The script below is provided as a test script; this script waits for 5 seconds and returns an object as an output with a property called Success that the main script depends on.

Write-Output "Starting script..."

Start-Sleep -Seconds 5

$res = New-Object PSObject -Property @{
    Success = $true
}

return $res

Calling this test script twice, the output of the main script is as follows:

References

The above script was built by generalizing and slightly improving others’ excellent work:

Dynamic Code in Powershell, by Tome Tanasovski
https://powertoe.wordpress.com/2010/02/10/dynamic-code-in-powershell/

Example 1 of returning data back from a runspace, by Boe Prox
https://gist.github.com/proxb/803fee30f0df244fd850 

About Herve Roggero

Herve Roggero, Microsoft Azure MVP, @hroggero, is the founder of Enzo Unified (http://www.enzounified.com/). Herve’s experience includes software development, architecture, database administration and senior management with both global corporations and startup companies. Herve holds multiple certifications, including an MCDBA, MCSE, MCSD. He also holds a Master’s degree in Business Administration from Indiana University. Herve is the co-author of “PRO SQL Azure” and “PRO SQL Server 2012 Practices” from Apress, a PluralSight author, and runs the Azure Florida Association.