Search results for 'app:weblogs' matching tags 'scripting techniques', 'arrays hashtables and dictionary objects', and 'Weekend Scripter'

Weekend Scripter: Improve Performance When Combining PowerShell Arrays

Anonymous — Sat, 27 Apr 2013 05:00:00 GMT

Summary: Microsoft premier field engineer, Chris Wu, talks about combining Windows PowerShell arrays.

Microsoft Scripting Guy, Ed Wilson, is here. Chris Wu, a Microsoft PFE, is back to share his knowledge. See previous Hey, Scripting Guy! Blog guest posts from Chris.

Here is contact information for Chris:

Twitter: https://twitter.com/chwu_ms
Facebook: https://www.facebook.com/mschwu
LinkedIn: http://ca.linkedin.com/in/mschwu

Take it away Chris…

While teaching a Windows PowerShell workshop, I was asked about how to combine two arrays of different objects (which share one key property) into one that contains objects with properties from both source objects. One real world scenario is to merge information retrieved from Active Directory (a list of Active Directory users and their properties) and Exchange Server (mailboxes).

The current approach by the student is to use two-level loops, which has seen performance issues when source arrays become huge in size (the following code snippet uses dummy data for demonstration purposes).

$ADList = @"

chwu,chwu@microsoft.com,Chris Wu

tst1,,Test User1

tst2,tst2@contoso.com,Test User2

"@ | ConvertFrom-Csv -Header Name,mail,CN

$EXList = @"

chwu,ex1.contoso.com

tst2,ex2.contoso.com

"@ | ConvertFrom-Csv -Header Name,MailServer

$Result = @()

foreach($ad in $ADList) {

$Match = $false

foreach($ex in $EXList) {

if ($ad.Name -eq $ex.Name) {

$Result += New-Object PSObject -Property @{Name=$ad.Name; mail=$ad.mail; CN=$ad.CN; MailServer=$ex.MailServer}

$Match = $true

}

if(-not $Match) {

$Result += New-Object PSObject -Property @{Name=$ad.Name; mail=$ad.mail; CN=$ad.CN}

}

In this post, I will explore several options to improve the performance and cleanness of this code snippet.

The first thing we can do is remove the use of the $Result array, which has two drawbacks:

Every assignment operation will create a new array in memory with data copied from the source, which is inefficient.
It defeats the streaming benefit of the WindowsPowerShell pipeline because it returns all objects as a whole at the end of the processing. A best practice in Windows PowerShell is to emit an individual object whenever it’s ready.

Another performance issue stems from the use of an inner loop to search for a matching record in the second array, which basically multiplies the total number of iterations. We can utilize a hash table for faster lookup. The Group-Object cmdlet offers a convenient AsHashTable parameter that can be used here.

Please be warned that the value portion of each entry is an array of matching records. If we are grouping records by using a property with unique values (such as SamAccountName), those arrays will apparently contains one single element each.

So an enhanced version of the code snippet is like this (with the constructing source arrays removed):

$h = $EXList | Group-Object -Property Name -AsHashTable

$ADList | %{

if($h[$_.Name]) {

New-Object PSObject -Property @{Name=$_.Name; mail=$_.mail; CN=$_.CN; MailServer=$h[$_.Name][0].MailServer}

} else {

New-Object PSObject -Property @{Name=$_.Name; mail=$_.mail; CN=$_.CN}

}

The last (but not the least) idea is to sort both arrays based on the key property (user name) beforehand, then we can pair records in a single iteration. Note that in this particular example, users found in Active Directory is a superset of users in Exchange, so we need a pointer variable to deal with this little quirk.

$ADList = @"

chwu,chwu@microsoft.com,Chris Wu

tst1,,Test User1

tst2,tst2@contoso.com,Test User2

"@ | ConvertFrom-Csv -Header Name,mail,CN | Sort-Object -Property Name

$EXList = @"

chwu,ex1.contoso.com

tst2,ex2.contoso.com

"@ | ConvertFrom-Csv -Header Name,MailServer | Sort-Object -Property Name

$ADList | % {$p = 0} {

if($_.Name -eq $EXList[$p].Name) {

New-Object PSObject -Property @{Name=$_.Name; mail=$_.mail; CN=$_.CN; MailServer=$EXList[$p].MailServer}

$p++

} else {

New-Object PSObject -Property @{Name=$_.Name; mail=$_.mail; CN=$_.CN}

}

The last two snippets perform much better than the original one, but which one is faster remains a question (I haven’t tested them against large arrays just yet). I would like to hear about your results, and I welcome your thoughts and ideas.

~Chris

Thanks Chris excellent blog post. Join me tomorrow for more cool Windows PowerShell stuff.

I invite you to follow me on Twitter and Facebook. If you have any questions, send email to me at scripter@microsoft.com, or post your questions on the Official Scripting Guys Forum. See you tomorrow. Until then, peace.

Ed Wilson, Microsoft Scripting Guy

Combine Arrays and Hash Tables in PowerShell for Fun and Profit

Anonymous — Sun, 11 Dec 2011 06:00:00 GMT

Summary: Take your Windows PowerShell scripting to the next level by combining hash tables and arrays.

Microsoft Scripting Guy, Ed Wilson, is here. The Scripting Wife and I are anxiously counting down the days until the first ever Pittsburgh PowerShell Users Group meeting. The event is quickly selling out; therefore, if you want to attend, you will need to sign up for the meeting. The reason for the reservation is that space at the Pittsburg Microsoft Office is limited. The meeting will be a lot of fun.

This past week, I talked about working with arrays. Although Windows PowerShell makes working arrays very easy, there are still lots of little things that need special attention. The five articles in the series devoted to working with Arrays in Windows PowerShell are listed here:

In the first post, Learn Simple Ways to Handle Windows PowerShell Arrays, I discussed creating arrays, indexing into arrays, and two techniques for walking through an array.
In the second post, Add, Modify, Verify, and Sort Your PowerShell Array, I discussed adding, modifying, or verifying values in a Windows PowerShell array, and two easy techniques for sorting the array.
In the third post, Find the Index Number of a Value in a PowerShell Array, I talked about working with arrays to determine the index number of a given value. In addition, I covered working with one half of the array at a time.
In the fourth post, I answered a question that was sent to scripter@microsoft.com about reading a CSV file and building distinguished names on the fly by using array notation. The Read a CSV File and Build Distinguished Names on the Fly by Using PowerShell post was really cool, and it presented a practical way to use arrays.
The last post was Easily Create and Manipulate an Array of Arrays in PowerShell, and I discussed working with an array of arrays.

Today, I want to look at creating an array that contains hash tables. Suppose I create a couple of hash tables, such as the ones that are shown here.

$a = @{"one"=1;"two"=2}

$b = @{"three"=3;"four"=4}

If I want to create an array with these two hash tables, it is as simple as using the equality operator, and assigning them to a variable by using standard array techniques. This command is shown here.

$c = $a,$b

The view of this array of hash tables appears to be a bit strange—it looks like I am looking at a hash table. This is shown here.

PS C:\> $c

Name Value

---- -----

two 2

one 1

three 3

four 4

The cool thing is that I can index into the elements of my array by using square brackets as shown here.

PS C:\> $c[0]

Name Value

---- -----

two 2

one 1

Because each element of the array contains a hash table, I can index into the array, and then use “hash table kinds of commands” such as querying the Keys or Values properties or using the Item method. These commands are shown here.

PS C:\> $c[0].values

PS C:\> $c[0].keys

two

one

PS C:\> $c[0].item("one")

The commands that create two hash tables, add the hash tables to an array, and then index into the array to work with the hash tables are shown in the following image.

If I am only interested in the keys that the hash tables contain, I can pipe the array to the Select-Object cmdlet. This technique is shown here.

PS C:\> $c | select keys

Keys

----

{two, one}

{three, four}

Although the output does contain the keys, it is not the best output. I can clean things up a bit by using the ExpandProperty parameter. This command is shown here.

PS C:\> $c | select -ExpandProperty keys

two

one

three

four

That is about all there is to working with an array of hash tables. Join me tomorrow as I begin a new week with a special guest blog post by Karl Mitschke.

Ed Wilson, Microsoft Scripting Guy

Create a Hash Table in PowerShell that Contains Hash Tables

Anonymous — Sat, 10 Dec 2011 06:00:00 GMT

Summary: Learn how to work with hash tables that contain other hash tables in Windows PowerShell.

Microsoft Scripting Guy, Ed Wilson, is here. Well, it is a typical winter day here in Charlotte, North Carolina in the United States. I am not talking about a nice, cool, sunny day with cobalt blue skies streaked by fluffy cotton candy clouds—nope, that is the “chamber of commerce” picture. A typical winter day around here seems to be cool, damp, and drizzled with rain—at least that is the way it has been for the past several days. That is OK; I figure we need all the rain we can get, as long as it does not come all at once.

I thought I would get up early and work on my presentation for the Pittsburgh PowerShell User Group meeting. The Scripting Wife and I will be there for the first meeting, and we are both really looking forward to it. Space is limited at the meeting; therefore, it is essential to sign up early to obtain a ticket (the tickets are free).

In the first post, Learn Simple Ways to Handle Windows PowerShell Arrays, I discussed creating arrays, indexing into arrays, and two techniques for walking through an array.
In the second post, Add, Modify, Verify, and Sort Your PowerShell Array, I discussed adding, modifying, or verifying values in a Windows PowerShell array, and two easy techniques for sorting the array.
In the third post, Find the Index Number of a Value in a PowerShell Array, I talked about working with arrays to determine the index number of a given value. In addition, I covered working with one half of the array at a time.
In the fourth post, I answered a question that was sent to scripter@microsoft.com about reading a CSV file and building distinguished names on the fly by using array notation. The Read a CSV File and Build Distinguished Names on the Fly by Using PowerShell post was really cool, and it presented a practical way to use arrays.
The last post was Easily Create and Manipulate an Array of Arrays in PowerShell, and I discussed working with an array of arrays.

One of the cool things to do with Windows PowerShell is to create a hash table that contains other hash tables. Creating a hash table that contains other hash tables offers a decent amount of flexibility, but it can become a bit confusing. Today, I thought I would spend a little time to help alleviate some of that confusion.

To create a hash table, I use the “hash table” operator that consists of an at sign (@), and a pair of curly brackets. Remember, that a hash table consists of one or more key/value pairings. For example, I create a hash table that consists of three items. This hash table is stored in the hashtable variable as shown here.

$hashtable = @{1="one";2="two";3="three"}

When I examine the hash table, I see that the numbers are the name (key) elements, and the words are the associated value. This is shown here.

PS C:\> $hashtable

Name Value

---- -----

3 three

2 two

1 one

If I create an additional hash table, and I attempt to add it to the hash table that is stored in the hashtable variable, an error occurs. The error is shown in the following image.

Receiving an error is certainly not my desired result. I then decide to use the += operator to add the two hash tables together. This technique is shown in the following image.

Unfortunately, merely adding the two hash tables together, does not allow me to have a hash table of hash tables. It creates a larger hash table and automatically incorporates the elements from the new hash table into the old hash table.

To create a hash table that contains other hash tables requires remembering that a hash table is comprised of key/value combinations. Therefore, I use the Add method, but I first add a key value, then I store the hash table in the value portion. The syntax is shown here.

$newhash.Add("hashtable",$hashtable)

In the following image, I create two hash tables. I then create an empty hash table and call the Add method to add the two hash tables to the newly created one.

When I have a hash table that contains other hash tables, I can use the Values property to view all the properties that are contained in the hash table. This command is shown here.

PS C:\> $newhash.Values

Name Value

---- -----

5 five

4 four

3 three

2 two

1 one

In addition to viewing the values, I can view the keys in the hash table by using the Keys property. This command is shown here.

PS C:\> $newhash.Keys

hashtable

I can also use the Item method to retrieve a specific hash table. To do this, I use the Key property, which displays in the output as a Name column as shown here.

PS C:\> $newhash.Item("h")

Name Value

---- -----

5 five

4 four

PS C:\> $newhash.Item("hashtable")

Name Value

---- -----

3 three

2 two

1 one

In addition to retrieving the hash tables that are stored in the newhash hash table via the Item method, I can also retrieve specific items from the stored hash tables. This technique is shown here.

PS C:\> $newhash.Item("hashtable").item(1)

one

PS C:\> $newhash.Item("h").item(4)

four

It is possible to work with the hash tables that are stored in the hash table in the same way that one works with other hash tables. For example, suppose I want to create two hash tables, and I store those two hash tables in a new hash table. The code to do this is shown here.

$a = @{"dog"="dog food";"cat"="cat food"}

$b = @{"people"="people food"}

$c = @{"one"=$a;"Two"=$b}

I can use the Keys property to retrieve the keys from one of the hash tables, and I can use the Values property to look at the values. This technique is shown here.

PS C:\> $c.Item("two").keys

people

PS C:\> $c.Item("two").values

people food

In addition, I can use the Add method to add key/value pairs to one of the hash tables. This appears here.

PS C:\> $c.Item("two").add("more people","more people food")

PS C:\> $c

Name Value

---- -----

Two {people, more people}

one {dog, cat}

Well, that is about all there is to working with hash tables that contain other hash tables.

Ed Wilson, Microsoft Scripting Guy