Search
Close this search box.

Multithreading with PowerShell using RunspacePool

If you are working with PowerShell, you may want to start running certain scripts in parallel for improved performance. While doing some research on the topic, I found excellent articles discussing Runspaces and how to execute scripts dynamically. However I needed to run custom scripts already stored on disk and wanted a mechanism to specify the list of scripts to run in parallel as an input parameter. 

In short, my objectives were:

  • Provide a variable number of scripts, as input parameter, to execute in parallelExecute the scripts in parallelWait for completion of scripts, displaying their outputHave a mechanism to obtain success/failure from individual scripts
  • This article provides two scripts: a “main” script, which contains the logic for executing other scripts in parallel, and a test script that simply pauses for a few seconds.

    Async Script

    The following script is the main script that calls other scripts in separate processes running Runspace Pools in PowerShell.  This script accepts an input array of individual scripts, or script files; you might need to provide the complete path for script files. Then this script starts a new PowerShell process for each script and creates a reference object in the $jobs collection; the BeginInvoke operation starts the script execution immediately. Last but not least, this script waits for completion of all scripts before displaying their output and result.

    It is important to note that this main script expects each script being called to return a specific property called Success to determine whether the script was successful or not. Depending on the script you call, this property may not be available and as a result this main script may report false negatives. There are many ways to detect script failures, so you can adapt the proper method according to your needs. A Test script is provided further down to show you how to return a custom property.

    You can the below script like this, assuming you want to execute two scripts (script1.ps1 and script2.ps1):   ./main.ps1 @( {& “c:\myscripts\script1.ps1” }, {& “c:\myscripts\script2.ps1” })

    Param([String[]] $toexecute = @())
    
            Write
        - Output("Received " + $toexecute.Count + " script(s) to execute")
    
              $rsp =
        [RunspaceFactory] ::CreateRunspacePool(1, $toexecute.Count) $rsp.Open()
    
            $jobs = @()
    #Start all scripts
            Foreach($s in $toexecute) {
      $job = [Powershell] ::Create().AddScript($s) $job.RunspacePool =
          $rsp Write - Output $("Adding script to execute... " + $job.InstanceId)
                           $jobs += New - Object PSObject - Property @ {
            Job = $job Result = $job.BeginInvoke()
          }
    }
    
    #Wait for completion
    do {
      Start - Sleep - seconds 1 $cnt =
          ($jobs | Where { $_.Result.IsCompleted - ne $true }).Count Write -
          Output("Scripts running: " + $cnt)
    } while ($cnt - gt 0)
    
        Foreach($r in $jobs) {
      Write - Output("Result for Instance: " + $r.Job.InstanceId) $result =
          $r.Job.EndInvoke($r.Result)
    #Display complete output of script
    #Write - Output($result)
    
    #We are assuming the scripts executed return an object
    #with a property called Success
              if ($result.Success) {
        Write - Output " -> Script execution completed successfully"
    #Use $result to print output of script
      }
      else {
        Write - Output " -> Script execution failed"
      }
    }

    Test Script

    The script below is provided as a test script; this script waits for 5 seconds and returns an object as an output with a property called Success that the main script depends on.

    Write-Output "Starting script..."
    
    Start-Sleep -Seconds 5
    
    $res = New-Object PSObject -Property @{
        Success = $true
    }
    
    return $res

    Calling this test script twice, the output of the main script is as follows:

    References

    The above script was built by generalizing and slightly improving others’ excellent work:

    Dynamic Code in Powershell, by Tome Tanasovski
    https://powertoe.wordpress.com/2010/02/10/dynamic-code-in-powershell/

    Example 1 of returning data back from a runspace, by Boe Prox
    https://gist.github.com/proxb/803fee30f0df244fd850 

    About Herve Roggero

    Herve Roggero, Microsoft Azure MVP, @hroggero, is the founder of Enzo Unified (http://www.enzounified.com/). Herve’s experience includes software development, architecture, database administration and senior management with both global corporations and startup companies. Herve holds multiple certifications, including an MCDBA, MCSE, MCSD. He also holds a Master’s degree in Business Administration from Indiana University. Herve is the co-author of “PRO SQL Azure” and “PRO SQL Server 2012 Practices” from Apress, a PluralSight author, and runs the Azure Florida Association.

    This article is part of the GWB Archives. Original Author: Herve Roggero

    Related Posts