Dreaming in PowerShell

Regular Expressions Are Your Friend (Part 3)

2011-12-02T12:27:00Z

In Part 1, you lerned how Regular Expressions can extract useful information from noise text. In Part 2, we looked at using Regular Expressions to split text, yet another powerful technique to extract the pieces of information you may need. Today, we conclude our little excurse and use Regular Expressions to replace text. Fasten your seat belts, please!

Replacing Old Text With New Stuff

Replacing some information inside a text with some other information seems to be pretty straight-forward: simply use -replace and tell the operator which text you would like to replace with other text:

PS> 'Hello World!' -replace 'World', 'PowerShell'
Hello PowerShell!

It is not obvious at first that -replace in reality takes a Regular Expression. You may notice accidentally when you try and use characters that have a special meaning, like here:

PS> 'Hello.' -replace '.', '!'
!!!!!!

Huh? Well, "." is a special RegEx placeholder and represents any character, so the result is right. If you must use special characters, they need to be escaped:

PS> 'Hello.' -replace '\.', '!'
Hello!

Remember the trick from our previous parts? If you aren't sure which characters need to be escaped, ask PowerShell!

PS> [RegEx]::Escape('Hello.')
Hello\.

Eliminating Duplicates

Being able to use Regular Expressions does add the burden of escaping, but at the same time it adds incredible power. Let's assume you want to get rid of excessive whitespace and replace multiple whitespace with just one space. Here is the solution:

PS> $text = 'Too many blank characters'
PS> $text -replace '\s{2,}', ' '
Too many blank characters

Here, -replace was looking for any whitespace (placeholder: \s) that occurs at least 2 times (quantifier: {2,}). It then replaces these instances with a single space. Done.

Using The Match

Your new replacement text does not need to be static. It can reference the original match. Have a look:

PS> $text = 'The problem was described in KB123456. Look it up.'
PS> $pattern = 'KB\d{6,8}'
PS> $text -replace $pattern, 'a KB-Article ($0 to be precise)'
The problem was described in a KB-Article (KB123456 to be precise). Look it up.

Here, we define a pattern for KB article numbers. The replacement string contains a placeholder ($0). This placeholder seems to be a PowerShell variable, but it is not. It just looks like one, which is why you must use single quotes or else PowerShell would also interpret this as a variable and resolve it.

What exactly is $0? To explain this, take a look at the result returned by -match:

PS> $text = 'The problem was described in KB123456. Look it up.'
PS> $pattern = 'KB\d{6,8}'
PS> $text -match $pattern
True
PS> $matches

Name                           Value
----                           -----
0                              KB123456

As it turns out, $matches returns a match with the name "0", so this match is what $0 represents.

If you use grouping in your pattern (place parts of it in brackets), then you have even more choices:

PS> $text = 'The problem was described in KB123456. Look it up.'
PS> $pattern = 'KB(\d{6,8})'
PS> $text -match $pattern
True
PS> $matches

Name                           Value
----                           -----
1                              123456
0                              KB123456

Now, -replace would support both the placeholders $0 and $1:

PS> $text = 'The problem was described in KB123456. Look it up.'
PS> $pattern = 'KB(\d{6,8})'
PS> $text -replace $pattern, 'a KB-Article (with number $1 to be precise)'
The problem was described in a KB-Article (with number 123456 to be precise). Look it up.

Using Backreferences

What you have just seen is often called "backreference": your new replacement text is backreferencing the original text. That technique can be extremely versatile. Let's say you want to make sure each comma in a text is followed by a space.

Here is a solution that uses backreferences:

PS> $text = 'After each comma,I want a whitespace,but only if there was no whitespace in the first place, of course.'
PS> $text -replace ',(\S)', ', $1'
After each comma, I want a whitespace, but only if there was no whitespace in the first place, of course.

The pattern looks for a comma that is immediately followed by a non-whitespace (placeholder: /S - watch out, placeholders are case-sensitive!). If that was found, it is replaced by a comma, a space and then the backreference (which is the character that previously immediately followed the comma). Mission accomplished. Thank you, backreferences!

Finding Duplicates - Backreferences In Your Pattern

You can even find and eliminate unwanted duplicates. Here is some sample code for you:

PS> $text = 'this text text contains duplicate words words following each other'
PS> $text -replace '\b(\w+)(\s+\1){1,}\b', '$1'
this text contains duplicate words following each other

In this case, the original pattern itself uses backreferences, too. It looks for a word boundary (placeholder: \b) and then for a word (\w+). Note that the word pattern is in brackets, so it is the first group with the index 1.

Then, it looks for at least one space and the same word again (placeholder: \1). So it is backreferencing the first group match. The quantifier ({1,}) tells RegEx that we are looking for one or more duplicate words.

The -replace operator then replaces all of this with the backreference to the first word, thus eliminating all the duplicate words.

So if you want to backreference matches in your replacement text, you prefix the group number with a $. If you want to backreference matches in your pattern, prefix the index with \ instead.

Capitalizing First Letter in a Word

Every now and then, even Regular Expressions can't do the job, so PowerShell scriptblocks can assist. Let's assume you want to capitalize every first letter in every word. Identifying the first letter is easy, but Regular Expressions cannot capitalize characters.

That's why you can submit a PowerShell script block that tells RegEx what to do with the match. Here is a sample:

PS> $text = 'i want to capitalize every word in this sentence.'
PS> $pattern = '\b(\w)'
PS> [RegEx]::Replace($text, $pattern, { param($letter) $letter.Value.toUpper() })
I Want To Capitalize Every Word In This Sentence.

Since -replace cannot process PowerShell script blocks, you need to use .NET directly here. The pattern describes a word boundary (placeholder: \b) and then the immediately following first character (placeholder: \w). Note that this placeholder is placed in brackets, so it is the first group.

Next, this sample uses the .NET class [RegEx] and its Replace() method which is the same dog food -replace uses internally, too. This time, though, a script block is submitted as replacement text.

This script block is often called a "delegate" because it is called for each match and determines the replacement dynamically. It receives the match through a param() block and then can use whatever logic you want to return the replacement text. In our case, it simply converts the received letter to uppercase.

What’s Next?

We are done with our tour of PowerShells' RegEx support. Next time, we check out hashtables and the clever things you can do with them. Hashtables are really everywhere in PowerShell, but they are a little shy, so you won't see them at first. Knowing about hashtables, how they work and what can be done with them will get you a lot more control over PowerShell. I am sure you'll be surprised! Stay tuned...

Tobias

Microsoft MVP PowerShell Germany

P.S.
If you live in Germany or other parts of Europe and your company would like to set up a truly great PowerShell training, just contact me! I regularly train mid- to large-size companies. Trainings are always a blast with tons of real-world-examples and solutions. Here's how to get in touch with me: tobias.weltner@scriptinternals.de

Regular Expressions Are Your Friend (Part 2)

2011-11-14T10:37:00Z

In Part 1, we used Regular Expressions to extract useful information from noise text. That was pretty cool, and we used the operator -match, the automatic PowerShell variable $matches and the powerful Get-Matches function.

Today, we'll use Regular Expressions again, but this time they are used to split text.

Chopping Text Into Pieces

Regular Expressions can be your butchers' knife to split text into pieces. Why on earth would you like do that, though?

For example, to process a comma-separated list of values. Have a look:

'comma,separated,list' -split ','

Here, the operator -split takes the text on its left side and chops the text up at each comma it finds. Yes, Regular Expressions do not need to be monsters. They can be very small. In this case, our Regular Expression simply is the comma - and represents a comma. It is - uh - a placeholder for - a comma. Easy enough.

You now could pipe the results to Foreach-Object and do stuff for all the elements in your comma-separated list:

$computers = 'pc1,pc2,server12,serverAB'
$computers -split ',' | Foreach-Object { "I am doing something with $_" }
I am doing something with pc1
I am doing something with pc2
I am doing something with server12
I am doing something with serverAB

$computers -split ',' | Foreach-Object { Get-WMIObject Win32_BIOS -computername $_ }

You can also use splitting to take apart path names. Here, the pattern is the backslash (if you want the parts of the path) or a dot (if you are after the file extension). This time, however, the pattern to use is not as straight-forward. Both the backslash and the dot are special characters in Regular Expressions. So if you really mean a backslash or a dot, you need to escape them:

PS> 'c:\windows\file.txt' -split '\.'
c:\windows\file
txt
PS> ('c:\windows\file.txt' -split '\.')[-1]
txt
PS> 'c:\windows\file.txt' -split '\\'
c:
windows
file.txt
PS> ('c:\windows\file.txt' -split '\\')[-1]
file.txt

Escaping Regular Expressions

As you have just discovered, sometimes it is necessary to escape characters in Regular Expressions so they are taken literally.

Fortunately, there is a way to automatically escape characters. If you aren't sure if something you mean literally should indeed be escaped or not, try using this trick:

PS> [Regex]::Escape('c:\folder\file.txt')
c:\\folder\\file\.txt

It takes the text you provide and returns it in escaped form so Regular Expressions wouldn't treat anything in that text as special characters.

Splitting Without Consuming

By default, when you chop up text using -split, the split character(s) are "consumed", so they are gone after the split.

PS> $text = 'This text gets splitted, and the delimiter is by default consumed, as you can see'
PS> $text -split ','
This text gets splitted
and the delimiter is by default consumed
as you can see

With look-behind expressions, you can make sure that the splitting pattern is not consumed:

PS> $text -split '(?<=, )'
This text gets splitted,
and the delimiter is by default consumed,
as you can see

Basically, "?<=" tells the RegEx engine to split right after the splitting character. Splitting without consuming can become important as you'll see in a second.

Separating Hex-Pairs

Let's assume you have a long list of hexadecimal pairs and would like to convert that list back to decimals:

'00AB1CFFAB1034'

To do that, you would somehow have to split after every second character and without consuming because you do not want to loose any information. Here is how you solve this puzzle:

PS> '00AB1CFFAB1034' -split '(?<=\G[0-9a-f]{2})(?=.)'
00
AB
1C
FF
AB
10
34

Now you can pipe the pairs to Foreach-Object and convert them to decimals or bytes or whatever you may need:

PS> '00AB1CFFAB1034' -split '(?<=\G[0-9a-f]{2})(?=.)' |
Foreach-Object { [System.Convert]::ToInt32($_, 16) }
0
171
28
255
171
16
52

PS> '00AB1CFFAB1034' -split '(?<=\G[0-9a-f]{2})(?=.)' |
Foreach-Object { [System.Convert]::ToByte($_, 16) }
0
171
28
255
171
16
52

Whew, wait a minute! This time, the Regular Expression wasn't as simple anymore. Right. Regular Expressions aren't hated without a reason. Sometimes they become tricky little monsters with mind-blowing lookaheads, lookbehinds and lookarounds. But is that a reason not to use them? No! We all use microwaves to make popcorn, but do we care about the physics? I don't.

So always remember: you do not need to care about Regular Expression mechanics once you have one that does what you want it to do. So once you have a Regular Expression that splits Hex-Pairs, you are good. Simply use it.

Instead of spending frustrating hours of crafting complex RegEx patterns, think "code reuse" and visit a search engine like google.Look for "RegEx <Pattern>", for example "RegEx IPAddress". Chances are someone else has already done the dirty job for you, and all you need to do is copy and paste the RegEx pattern.

If you do want to digest patterns and really understand what is going on, there are plenty of great RegEx tutorial sites. This one explains lookbehinds, lookaheads and lookarounds and helps you understand the previous example: http://www.regular-expressions.info/lookaround.html.

What’s Next?

We covered -match and -split, but there's a third family member: -replace. So next time we'll continue talking about RegEx and look how you can replace things, update IP lists and more. Meanwhile, if you have spare time, go visit some of the plenty sites that focus on RegEx, and try and refine your understanding of RegEx placeholders, anchors and quantifiers. Stay tuned!

Tobias

Microsoft MVP PowerShell Germany

Regular Expressions Are Your Friend (Part 1)

2011-10-27T02:04:00Z

Yes, regular expressions can be complex but hey no, they do not need to be! You can do amazing things with small and relatively simple regular expressions. So let’s take a look at it.

Finding Information: -match

Often, you need some piece of information that is buried inside of noise text. Let’s assume you’d like to find the KB article number that is in this text:

PS> $text = 'The problem was discussed in KB552356 and is solved. KB652534 was not addressed.'

Regular expressions can do this for you. All you need to do is define the pattern you are looking for. Here is a sample pattern that identifies KB article numbers:

PS> $pattern = 'KB\d{4,6}'

Before we look at the pattern and how it is constructed, let’s see results. PowerShell has an operator called –match that takes regular expressions and returns $true when it found the pattern in the text:

PS> $text -match $pattern
True

If there was a match, you even get back the expression that matched your pattern in the automatic variable $matches:

PS> $matches

Name                           Value
----                           -----
0                              KB552356

PS> if ($text -match $pattern) { $matches[0] }
KB552356

How Patterns Work

Regular expression patterns basically contain three things: anchors, placeholders, and quantifiers. In the pattern we used, we had all three elements:

“KB” was an anchor. It expected the pattern to start at the static text “KB”.
\d is a placeholder. It represents a digit.
{4,6} is a quantifier. It expects the placeholder to occur four to six times, so it matches 4 to 6 digits.

That’s not too hard, eh?

Creating Groups

Let’s assume you are just interested in the number of the KB article. To get just the number, you can create groups by using parenthesis.

Here is the refined pattern:

PS> $pattern = 'KB(\d{4,6})'
PS> $text -match $pattern
True
PS> $matches

Name                           Value
----                           -----
1                              552356
0                              KB552356

Here, $matches returns two results. Index 0 holds the complete match, and index 1 holds the match of the first group. It contains just the number of the KB article.

PS> if ($text -match $pattern) { $matches[1] }
552356

Finding Multiple Matches

Unfortunately, -match only finds the first match. That’s why it is called –match and not –matches. And this is also why it only found the first KB article number in the sample text, not the second one.

There is a built-in way in PowerShell to find multiple matches, but it is somewhat hidden. Select-String can find multiple matches:

PS> $text | Select-String -AllMatches $pattern

The problem was discussed in KB552356 and is solved. KB652534 was not addressed.

However, the result is the entire text. Only when you poke in the results will you get the individual matches:

PS> $text | Select-String -AllMatches $pattern | Select-Object -ExpandProperty Matches | Select-Object -ExpandProperty Value
KB552356
KB652534

Since that’s not really beautiful, you can create your own filter and use it to extract multiple matches:

filter Get-Matches($Pattern) {
$_ | Select-String -AllMatches $pattern |
Select-Object -ExpandProperty Matches |
Select-Object -ExpandProperty Value
}

PS> $text | Get-Matches $pattern
KB552356
KB652534

Your filter is pretty flexible. You can use it for example to parse entire log files:

PS> Get-Content $env:windir\windowsupdate.log -Encoding UTF8 |
>> Get-Matches 'successfully installed.*?update:.*'
>>
successfully installed the following update: Microsoft Silverlight (KB979202)
successfully installed the following update: Security Update for Windows 7 for x64-based
Systems (KB2532531)

(...)

Check the pattern used here:

We have an anchor (‘successfully installed’), then a placeholder (‘.’=anything) with a quantifier (‘*?’ = any number, but as short as possible), then another anchor (‘update’), and then again a placeholder (‘.’ = anything) with a quantifier (‘*’ = any number).

Essentially, this extracts all lines from the log file where updates were installed.

Finding Multiple Matches Fast

Select-String is rather slow, though. To retrieve multiple matches, using .NET directly speeds up things considerably. Here’s the Get-Matches function as a .NET implementation:

function Get-Matches($Pattern) {
begin { $regex = New-Object Regex($pattern) }
process { foreach ($match in ($regex.Matches($_))) { ([Object[]]$match.Groups)[-1].Value } }
}

PS> Get-Content $env:windir\windowsupdate.log -Encoding UTF8 | Get-Matches 'successfully installed.*?update:.*'
successfully installed the following update: Microsoft Silverlight (KB979202)
successfully installed the following update: Security Update for Windows 7 for
x64-based Systems (KB2532531)
successfully installed the following update: Update for Windows 7 for x64-based
Systems (KB2533623)

(...)

This is not just a lot faster, it also supports grouping, so if you just wanted the product name that was installed, place the last placeholder-expression into parenthesis. Get-Matches always returns the last group:

PS> Get-Content $env:windir\windowsupdate.log -Encoding UTF8 | Get-Matches 'successfully installed.*?update: (.*)'
Microsoft Silverlight (KB979202)
Security Update for Windows 7 for x64-based Systems (KB2532531)
Update for Windows 7 for x64-based Systems (KB2533623)
Security Update for Windows 7 for x64-based Systems (KB2507938)
Security Update for Windows 7 for x64-based Systems (KB2555917)

Clever RegEx Matching

To make life much easier, here’s a function that returns regular expression matches as real PowerShell objects:

function Get-Matches {
param(
    [Parameter(Mandatory=$true)]
    $Pattern,

    [Parameter(ValueFromPipeline=$true)]
    $InputObject
)

begin {

    try {
   $regex = New-Object Regex($pattern)
  }
  catch {
   Throw "Get-Matches: Pattern not correct. '$Pattern' is no valid regular expression."
  }
  $groups = @($regex.GetGroupNames() |
  Where-Object { ($_ -as [Int32]) -eq $null } |
  ForEach-Object { $_.toString() })
}

process {
  foreach ($line in $InputObject) {
   foreach ($match in ($regex.Matches($line))) {
    if ($groups.Count -eq 0) {
     ([Object[]]$match.Groups)[-1].Value
    } else {
     $rv = 1 | Select-Object -Property $groups
     $groups | ForEach-Object {
      $rv.$_ = $match.Groups[$_].Value
     }
     $rv
    }
   }
  }
}
}

Now, if you’d like to extract information, you could use it like this:

PS> "KB2786873 good KB27897983 bad " | Get-Matches 'KB\d{4,8}'
KB2786873
KB27897983

You could also create named groups which you do by adding ?<Name> to your parenthesis. Then, Get-Matches automatically turns these groups into object properties:

PS> "KB2786873 good KB27897983 bad " | Get-Matches '.*?(?<KB>KB\d{4,8}).*?(?<
word>\w+)'

KB                                     word
--                                      ----
KB2786873                       good
KB27897983                     bad

Take a look how easily you now can parse a raw text file and turn it into real PowerShell objects! First read the log file:

$text = Get-Content $env:windir\windowsupdate.log -Encoding UTF8 -ReadCount 0

Next, try a couple of RegEx patterns:

PS> $text | Get-Matches 'successfully installed.*?update: (.*)'
Microsoft Silverlight (KB979202)
Security Update for Windows 7 for x64-based Systems (KB2532531)
Update for Windows 7 for x64-based Systems (KB2533623)
Security Update for Windows 7 for x64-based Systems (KB2507938)

(...)

PS> $text | Get-Matches 'successfully installed.*?update: (?<Product>.*)'

Product
-------
Microsoft Silverlight (KB979202)
Security Update for Windows 7 for x64-based Systems (KB2532531)
Update for Windows 7 for x64-based Systems (KB2533623)
Security Update for Windows 7 for x64-based Systems (KB2507938)
Security Update for Windows 7 for x64-based Systems (KB2555917)

(...)

PS> $text | Get-Matches '(?<Date>\d{4}-\d{2}-\d{2}).*?(?<Time>\d{2}:\d{2}:\d{2}).*?successfully installed.*?update: (?<Product>.*)'

Date                       Time                       Product
----                       ----                       -------
2011-10-17                 13:05:46                   Microsoft Silverlight ...
2011-10-17                 13:59:38                   Security Update for Wi...
2011-10-17                 13:59:38                   Update for Windows 7 f...
2011-10-17                 13:59:38                   Security Update for Wi...
2011-10-17                 13:59:38                   Security Update for Wi...

(...)

PS> $text | Get-Matches '(?<Date>\d{4}-\d{2}-\d{2}).*?(?<Time>\d{2}:\d{2}:\d{2}).*?successfully installed.*?update: (?<Product>.*?) \({0,1}KB(?<KB>\d{5,8})'

Date                Time                Product             KB
----                ----                -------             --
2011-10-17          13:05:46            Microsoft Silver... 979202
2011-10-17          13:59:38            Security Update ... 2532531
2011-10-17          13:59:38            Update for Windo... 2533623
2011-10-17          13:59:38            Security Update ... 2507938
2011-10-17          13:59:38            Security Update ... 2555917

(...)

PS> Get-Content $env:windir\windowsupdate.log -Encoding UTF8 -ReadCount 0 | Get-Matches '(?<Date>\d{4}-\d{2}-\d{2}).*?(?<Time>\d{2}:\d{2}:\d{2}).*?successfullyinstalled.*?update: (?<Product>.*?) \({0,1}KB(?<KB>\d{5,8})'

(...)

Using Special Options

There are two ways for you to find matches: use the built-in operator –match, or use the .NET Framework RegEx type. Generally, you want to use the .NET approach whenever you need to find more than just the first match.

Depending on which method you choose, there are different default settings, though, that you need to know about. The –match operator always works case-insensitive whereas .NET always works case-sensitive by default.

The easiest way to make an expression case-insensitive is to use control operators. This pattern for example works case-sensitive with –match and case-insensitive with .NET:

$pattern = ‘KB\d{6,8}’

This pattern always works case-insensitive, regardless of which method you use:

$pattern = ‘(?i)KB\d{6,8}’

Another important control operator turns on single-line-mode. Let’s assume you downloaded HTML code from a web page and want to find all instances of a pattern. To find all matches, you need to use .NET. To find the pattern in the entire HTML source code, you need single-line-mode.

To make this work, your pattern should start with (?si).

What’s Next?

Next week, I’ll keep talking about RegEx and look at splitting and replacing. Meanwhile, if you have spare time, go visit some of the plenty sites that focus on RegEx, and try and refine your understanding of RegEx placeholders, anchors and quantifiers. Stay tuned!

Tobias

Microsoft MVP PowerShell Germany

Reuse Your Code - Create Modules Automagically!

2011-10-14T16:34:00Z

When you write scripts, you want to automate things, fair enough. When it comes to writing scripts, a lot of people do not automate, though. They do the same coding over and over again and waste time and consistency.

So let’s look at some simple code-reusing techniques. It’s not that hard at all. In fact, after you read this post you can create your own PowerShell modules in seconds.

Code Reuse – Join The Game!

Actually, most PowerShell script writers reuse code all the time without realizing – they’re just not reusing their own code. When you run a cmdlet, you are reusing the code someone else put into that cmdlet. You know that’s cool because it makes PowerShell scripts much shorter and easier to write.

If you want to sort stuff, use Sort-Object. That's code reuse. In the old days of VBScript you had to re-invent and code bubble-sort-algorithms all the time. No more in PowerShell. If you want to manage Active Directory, import the AD module and use Get-ADUser to search for users. Again, no need for coding tricky AD connect and search stuff. It's done for you so you can focus on your tasks. Cmdlets are a big old code reuse party.

Most PowerShell scripters happily use cmdlets but eventually find that they need additional stuff because there aren’t predefined cmdlets for everything. So they start scripting day and night and produce long and convoluted scripts that are hard to manage and adjust.

Stop that. When you need to automate something, grab a coffee and think first. What is the “vocabulary” (cmdlets, functions) you need to address the problem? Do you have all the vocabulary you need for that? What is missing? Which PowerShell module might already provide the needed vocabulary? There are great modules for VMWare, HyperV, AD, Exchange, SQLServer, you name it.

Chances are, some vocabulary you need is missing. When that occurs, design the functions that provide the missing vocabulary first. Once they are done, continue with your initial job. Does that take more time? No, it saves time. Here's how.

Adding More Vocabulary - Outputting Data to Excel (for example)

Let’s say as part of your solution you need a way to output information to Microsoft Excel.

Instead of incorporating this functionality into your overall solution (and producing monster scripts), define a function that just focuses on outputting information to Microsoft Excel. Clean and sweet. It could be as simple as this one:

function Out-ExcelReport {
param(
$path = "$env:temp\report$(Get-Date -format yyyyMMddHHmmss).csv",
[switch]$open
)

$Input |
Export-Csv $path -NoTypeInformation -UseCulture -Encoding UTF8

if($open) { Invoke-Item $path }
}

Once your function is done, test it. Run it like this:

Get-Process | Out-ExcelReport –open

Or like this:

Get-Process |
Select-Object Name, Description, Company, MainWindowTitle |
Where-Object { $_.MainWindowTitle } |
Out-ExcelReport –open

Making Your Vocabulary Reusable, Too!

Once your new function is working like a charm, you could be tempted to copy and paste the code into your overall solution. Don’t do that! Don't assimilate your great new function into some script. Keep it separate! Make it reusable!

Any script that wants to export data to Microsoft Excel should import the necessary vocabulary from PowerShell modules. Which raises the question: how would you turn your function into a module? That’s done automagically when you download, unpack and import this helper module: http://powershell.com/cs/media/p/12823.aspx. Simply import it like this:

Import-Module c:\pathtomodulefolder\ModuleHelper

Make sure you specify the path to the unzipped folder named "ModuleHelper" inside the ZIP (due to zipping, there are two nested folders, take the one inside the other), and also note that as a general prerequisite, your PowerShell execution policy should of course allow scripts to run.

Now, to save your brand new function Out-ExcelReport to a new module of your choice, use the new command Out-Module:

Out-Module -Module ReportingStuff –Function Out-ExcelReport

That’s it! You may have noticed that there was no module “ReportingStuff” before, but there is now:

Get-Module –ListAvailable

So when you import this new module, you load your function Out-ExcelReport. Essentially, you created the same great code reuse mechanism that you enjoyed from other vendors to run AD or Exchange cmdlets. Open a new PowerShell console, and try this:

Import-Module ReportingStuff
Get-Service | Out-ExcelReport -open

Your new function Out-ExcelReport just became reusable, and you can import your module in every script that needs your new reporting capabilities now. Never do you have to waste time again to create that functionality again. Cool, eh?

Adding and Managing Scripts

So far, your new module “ReportingStuff” has only one function. You can add more, though. Your new module serves as a script repository for you, and you can save all functions into that module that deal with reporting. For example, maybe you’d also like to be able to easily create text files that are wide enough to show all results without truncation. Here’s the function:

<#
.SYNOPSIS
Saves results as text to a text file
.DESCRIPTION
Saves results to text file and automatically adjusts the file width so no columns and no data is truncated.
Truncation only occurs for object properties that contain multi-line text.
.PARAMETER Path
Optional. Name of text file to create. Existing files will be overwritten. By default, a temporary file in the temp folder is created.
.PARAMETER Open
Opens the text file after creation in the application that is associated with .txt-files.
.EXAMPLE
Get-Process | Where-Object { $_.MainWIndowTitle} | Select-Object Name, Description, Company, MainWindowTitle , CPU | Out-TextReport -Open
Lists various pieces of information about running processes and writes them to a text file. The file width is automatically adjusted to provide enough room to display all information. The text file then is opened by its default application.
.EXAMPLE
Get-ACL $env:windir | Out-TextReport -open
Reads the windows folder NTFS permissions and writes the results to a text file. The text file then is opened by its default application.

#>
function Out-TextReport {
param(
$Path = "$env:temp\report.txt",
[switch]$Open
)

$Input |
Format-Table -AutoSize |
Out-File $Path -Width 10000
if($open) { Invoke-Item $Path }
}

Run it, and test it thoroughly. Once done, again save the function to your module, directly from memory:

Out-Module ReportingTools Out-TextReport

Your new module now has two pieces of vocabulary. To update it, use –force while you import it:

Import-Module ReportingTools –Force -Verbose

Adding Help and Updating Modules

Note that with Out-TextReport, I added a comment-based help block to the code. That’s a really good idea because you probably remember how you discovered and learned 3rd party cmdlets – by using Get-Help and looking at the examples.

You should provide the same helpful information for the users of your own “cmdlets”. That’s just fair (and gives them an especially professional touch, too). So take a little time and create a great help block with practical examples.

If you save a function to your module that does not yet have a help block, Out-Module adds one. So when you load and edit functions from your module (hang in, we’ll cover that in a second), you just need to fill out the placeholders and save the script.

You can even update functions in your module(s). Just assume you wanted to polish a function. Simply open the module path, and load the function into your favorite script editor. Make the adjustments, then save it back:

Explorer (Get-ModulePath ReportingTools)

Likewise, you can send a function that you changed, back to the module by using Out-Module again. It will replace the old version.

More to come...

At our first European PowerShell Deep Dive yesterday, I also did a presentation about the power of regular expressions and how beautifully PowerShell embraces them. I added all the example functions to a sample module using the technique described above. So next week, I'll share some serious RegEx magic with you. Stay tuned!

Tobias

Microsoft MVP PowerShell Germany

Applying NTFS Permissions

2011-09-21T10:21:00Z

Recently, I needed to create a folder with NTFS permissions. PowerShell can do that for you, and when you look at your weapons, you'll find that sometimes it is best to mix command types and not just stick to cmdlets. At the end, I had a handy function that would take a path and a username and do all the tricky NTFS permissions stuff for me. Here is how.

Creating Folders

Let's start with the simple part and write a function that creates a folder if that folder does not yet exist:

function Create-Folder {
param($path)

if ( (Test-Path $path) -ne $true) {
New-Item -Path $path -ItemType Directory | Out-Null
}
}

Apply Permissions

Next, I want to add NTFS permissions to that folder. A specifc user should get change permission, and all Administrators should get full permission. There are cmdlets to do the job like Get/Set-ACL, but working with them can be hard because they really are just simple wrappers for low-level .NET methods.

PowerShell is not limited to cmdlets. You can happily use established console-based applications like cacls.exe. Here is an example:

function Create-Folder {
param($path, $user)

if ( (Test-Path $path) -ne $true) {
New-Item -Path $path -ItemType Directory | Out-Null
}

CACLS $path /G "Domain Admins":F
CACLS $path /E /G $user:C
}

As it turns out, this won't work yet. The first call to CACLS removes existing permissions which is what I want. However, this triggers a confirmation, so I would have to manually type "Y" to approve the operation. The second call to CACLS won't work at all. CACLS can't understand it, so it throws back its manual to the caller.

To correct these issues, a couple of tricks are needed. To automatically send a key to a confirmation, place it on the pipeline. And to resolve the syntax issue, submit arguments to CACLS as a string so PowerShell won't get confused:

function Create-Folder {
param($path, $user)

if ( (Test-Path $path) -ne $true) {
New-Item -Path $path -ItemType Directory | Out-Null
}

'y' | CACLS """$path"" /G ""Domain Admins"":R" | Out-Null
CACLS """$path"" /E /G ""$user"":F" | Out-Null
}

Now it is easy to create new folders and apply standard NTFS security:

Create-Folder -path c:\user1 -user mydomain\username

Careful!

Unfortunately, this approach is not culture-neutral. On non-US systems, you probably will want to change two things:

Make sure you use the correct name for the Admin group. On German systems, it is called "Domänen-Admins" instead of "Domain Admins"
Make sure you send the correct confirmation key. On German systems, it is "J" rather than "Y"

I know that sucks, but for a number of scenarios, incorporating tools like CACLS can simplify life tremendously. Learning points here are: console-based applications are (almost) equal PowerShell citizens. To submit arguments to them, you may have to turn them into a string in order to avoid parsing conflicts.

See you next time!

Tobias

Microsoft MVP PowerShell Germany

Bulk-Renaming Files (and other magic)

2011-09-12T07:49:00Z

Bulk-renaming files can cause some headache: maybe you'd like to nicely rename all your pictures with a keyword and an incrementing counter, or you'd want to clean up log files and assign them a standard name based on its creation date. It's easy to do that for one file, and with our latest tip, we illustrated how you can as just as easily do this for hundreds of files, too. Heck, did we get a lot of feedback, both additional questions and great suggestions, so let's check out what other options you have to bulk-rename files.

Creating Filenames with an Incrementing Counter

In our original tip, we wanted to rename the content of a folder filled with picture files. Each picture file was supposed to get a new file name like "pictureX", where "X" was an incrementing counter. Here is the logic:

$global:i = 1

dir c:\pictures\ -Filter *.jpg |
Rename-Item -NewName {
"picture_$i.jpg"; $global:i++
}

There are two things to note here: First, we need a global variable $i which holds the incrementing counter. It needs to be global because we want to remember its setting and increment it with each new file. By default, variables are valid only within their "territory" (scriptblock) and below, so if the counter variable was not global, the scriptblock would always start with a new variable.

Second, note how Rename-Item accepts a scriptblock. Instead of assigning a static file name, a scriptblock can dynamically create the filename for you. In our example code, the new filename consists of a fixed prefix "picture_" and the counter variable. Pretty nice! But it gets better.

Taking Advantage of $_

Jacques Barathon pointed out that the scriptblock that you submit to Rename-Item of course has a $_ automatic variable. This is true for most scriptblocks that you submit to cmdlets. $_ always represents the current object you are working with.

That's cool because now you can easily use the existing file name and manipulate it. Here is what Jacques did: he wanted to replace a single word in a filename. Let's check out how he did this:

dir *.docx |
Rename-Item -NewName {$_.Name.Replace(‘chapter’, ‘chapitre’)}

Whew! That was easy! Since $_ is referring to the current file, he accessed the original filename through $_.Name. The result is the filename, and it is a string data type. So he could then use the string method Replace() to replace a word in that filename. The result was the new filename where 'chapter' was replaced by 'chapitre'. This result was then used by Rename-Item to rename the file.

The Sky Is The Limit

Of course, $_ gives you complete access to all file properties, so you can basically access and use all file properties to construct the new filename. Let's say you wanted to replace a filename by a timestamp based on the creation date. Here is some code that does it:

$code = {
$timestamp = Get-Date $_.CreationTime -format 'yyyyMMddHHmmssff'
$extension = $_.Extension
$timestamp.$extension
}

dir c:\pictures | Rename-Item -NewName $code

Note how we placed the code (scriptblock) into an extra variable $code to keep the code clean.

While you are playing with this sample, you'll definitely run into issues. When doing bulk operations, you always have to take all kinds of edge cases into consideration. What if there are two files with the very same creation date? Since no two files can have the same name, Rename-Item would fail at the second file and not continue. Bummer.

Happily, you are not bound to any limits. Your scriptblock can contain as much logic as you like. So here is a more advanced approach that takes care of duplicates. It first creates the new filename, then checks if a file with that name already exists, and if so adds a counter to the filename.

$code = {
$timestamp = Get-Date $_.CreationTime -format 'yyyyMMddHHmmssff'
$extension = $_.Extension

$counter = 0
while ($true) {
$filename = '{0}{1}{2}' -f $timestamp, $( if ($counter) { "-$counter" } else { '' }) , $extension
$filepath = Join-Path (Split-Path $_.FullName) $filename
if (Test-Path $filepath) {
$counter++
} else {
$filename
break
}
}
}

dir c:\pictures | Rename-Item -NewName $code

You can reuse this logic for all events where you cannot be sure that the filename created by your scriptblock is unique.

That's the scoop for today. Hope to see you next week,

Tobias

Microsoft MVP PowerShell Germany

Clever Splatting to Pass Optional Parameters

2011-08-03T04:15:00Z

Getting system information from WMI is a pretty straight-forward thing with Get-WMIObject. You can even specify one or more computer names or IP addresses to retrieve the information remotely. That's all great, but how would you design a function that encapsulates Get-WMIObject (or any other cmdlet that can work both locally and remotely)? That's not so trivial.

Have a look. There's a really clever solution for that!

Prerequisites: Getting Information Locally or Remotely

Get-WMIObject is a great example of a cmdlet with built-in remoting. For example, to get information about your BIOS, use this call:

PS> Get-WMIObject -Class Win32_BIOS

SMBIOSBIOSVersion : 02LV.MP00.20081121.hkk
Manufacturer : Phoenix Technologies Ltd.
Name : Phoenix SecureCore(tm) NB Version 02LV.MP00.20081121.hkk
SerialNumber : ZAMA93HS600210
Version : SECCSD - 6040000

PS> Get-WMIObject -Class Win32_BIOS -ComputerName storage1

SMBIOSBIOSVersion : P03
Manufacturer : Phoenix Technologies LTD
Name : Ver 1.00PARTTBLw
SerialNumber : 98H340ED2H9300237A30A1
Version : PTLTD - 6040000

As you can see, the cmdlet gets information both from your local system and from one (or more) remote systems - provided they are correctly configured and you have appropriate permissions. It all depends on the parameters you submitted to Get-WMIObject.

We are not digging into the security details here because that's another topic, but if you can't get Get-WMIObject to work against remote systems, try enabling the "Remote Management Exception" in your firewall when you get "RPC"-exceptions, and use the parameter -credential to log on as a different user when you get "Access Denied"-exceptions.

Creating A (Stupid) Wrapper Function

The call to Get-WMIObject is really very trivial, but let's assume you still want to encapsulate it and use it as a new function. How would you design a function that can run both against the local machine and the remote machine(s)?

In traditional thinking, you would have to check whether the user has actually submitted the -computername parameter, and then call Get-WMIObject multiple times: if the parameter was submitted, you pass it on to the cmdlet, and if not, you omit it. The result usually is pretty awkward and becomes even more convoluted when you add additional parameters. Here is an ugly example:

function Get-BIOS {
param(
$computername = $null,
$credential = $null
)

if ($credential) {
Get-WmiObject Win32_BIOS -computername $computername -credential $credential
} elseif ($computername) {
Get-WmiObject Win32_BIOS -computername $computername
} else {
Get-WmiObject Win32_BIOS
}
}

At least, it does what it was supposed to do:

PS> Get-BIOS

SMBIOSBIOSVersion : 02LV.MP00.20081121.hkk
Manufacturer : Phoenix Technologies Ltd.
Name : Phoenix SecureCore(tm) NB Version 02LV.MP00.20081121.hkk
SerialNumber : ZAMA93HS600210
Version : SECCSD - 6040000

PS> Get-BIOS -computername storage1

SMBIOSBIOSVersion : P03
Manufacturer : Phoenix Technologies LTD
Name : Ver 1.00PARTTBLw
SerialNumber : 98H340ED2H9300237A30A1
Version : PTLTD - 6040000

PS> Get-BIOS -computername storage1 -credential (Get-Credential Administrator)

SMBIOSBIOSVersion : P03
Manufacturer : Phoenix Technologies LTD
Name : Ver 1.00PARTTBLw
SerialNumber : 98H340ED2H9300237A30A1
Version : PTLTD - 6040000

Create a (Clever) Wrapper Function

When you want to pass optional parameters to cmdlets, there is a much more clever and elegant way in PowerShell: Splatting! Instead of manually handing over parameters to a cmdlet, you create a hash table with the parameters you want to pass on. Then, you submit the hash table to the cmdlet.

The advantage of this not widely known technique is obvious once you see the improved version of the wrapper function: PowerShell already makes available an object that you can use for splatting! The system variable $PSBoundParameters contains all the parameters that were submitted to your function, so whatever someone submitted can be passed on to a cmdlet without the need to check for parameters and call different code:

function Get-BIOS {
param(
$computername,
$credential
)

Get-WmiObject -Class Win32_BIOS @psboundparameters

}

Isn't that a beauty? It works exactly the same, with much less code. The function calls Get-WMIObject to retrieve WMI BIOS information and optionally passes on the computername and/or credential parameter. No extra work for you anymore.

Thanks to Aleksandar and HAL (both PS MVPs) for their valuable input!

Hope to see you next week,

Tobias

Microsoft MVP PowerShell Germany

Dealing With File Locks

2011-06-23T10:40:00Z

Sometimes, when you try and access a file, it may refuse to open because it is locked by another user or process. Likewise, you may want to lock a file yourself to make sure the file is not accessed and read/changed while you are manipulating it. Which raises the question: how do you control file locks?

Locking a file

Get-Content can read text-based files. This cmdlet is not locking the file, so while you are reading it, others can still access or even change it.

To apply a file lock, you need to use low level .NET methods. The next example illustrates how to open a file with a read/write lock. While this script is running, no one can access the file:

$file = [System.io.File]::Open('c:\files\somefile.txt', 'Open', 'Read', 'None')
Read-Host 'Press ENTER to release file'
$file.Close()

The method Open() accepts four arguments:

The path to the file you want to access
The action you want to take, for example "Open"
The access you want to use, for example "Read"
The lock. "None" grants to access to others, so the file is locked. "Read" would allow others to read the file while you are using it, and "ReadWrite" would allow others to read and/or write to the file while you are using it

Reading File Content

When you lock a file, no one else can access it, so you cannot use Get-Content to read its content anymore, either. To read the file content, you would have to refer to .NET methods. Here is an example that reads file content of a file you applied a lock on:

$file = [System.io.File]::Open('c:\files\somefile.txt', 'Open', 'Read', 'None')
$reader = New-Object System.IO.StreamReader($file)
$text = $reader.ReadToEnd()
$reader.Close()
$file.Close()

Dealing With Existing File Locks

When you try and access a file, others may have already applied locks to it. For example, windows maintains a file called $env:windir\windowsupdate.log which carries a "Write" lock: you can read its contents, but you cannot write to it.

If you tried and accessed that file with a lock of type "None" (exclusive access), your code would fail because the system already implemented a lock which is incompatible to the lock you wanted to apply. You can only lock a file for exclusive access if noone else has applied a lock before.

To deal with this, you would have to make sure your lock is not conflicting with existing locks. When you specify a "ReadWrite" lock (essentially allowing others to still read and write the file), you can safely read the file despite other locks (except if someone had applied a "None" lock):

$file = [System.io.File]::Open("$env:windir\windowsupdate.log", 'Open', 'Read', 'ReadWrite')
$reader = New-Object System.IO.StreamReader($file)
$text = $reader.ReadToEnd()
$reader.Close()
$file.Close()

This is how Get-Content works. It uses a "ReadWrite" lock. The above low level .NET access is a lot faster though because it is not reading the file line by line. Instead, it returns the entire file content as one (large) string.

Hope to see you next week,

Tobias

Microsoft MVP PowerShell Germany

Renaming Files Puzzle - And Three Golden PowerShell Rules

2011-05-23T11:57:00Z

I am moderator for one of our Ask-the-Experts forums, and every once in a while, I get questions I would like to discuss. Like this one:

"I am tring to write a quick interactive command that will rename all the jpgs in the current directory and came up with the following command:

$i=1; gci | ? { ($_.Extension -eq ".JPG") -and !($_.Name.StartsWith("cover")) } | % { Rename-Item -Path ".\$_" -NewName [string]::Format("picture_$i.

jpg"); $i++ }

There appears to be a parameter binding issue. But the 2 parameters I have specified (-Path and -NewName) I have named so I assumed that position is not of relevance. What am I missing here?"

Let's see how we can solve this puzzle and come up with an easy solution using the "3 Golden PowerShell rules"!

Think PowerShell! Use Parameters!

Whenever you work with files, Get-Childitem (aka dir or ls) is your friend. And like most Get-* cmdlets, it has a lot of parameters to help you get what you want. Of course you can always use Where-Object (short: ?) to filter the results once you got them. That is only your last resort, though. Before you do that, always make sure the original cmdlet has no way of filtering. And Get-Childitem has. So the first part would be selecting the files you want to rename like this:

Get-Childitem c:\somefolder -Filter cover*.jpg

Another thing to watch out is this: never use Get-Childitem without specifying an absolute or relative path to start with. Else, it will depend on your current path, and that path may change. So the next time you are running your code, it may no longer work because your current path is no longer pointing to the files you want to rename.

Rule #1: Before you resort to Where-Object, make sure the upstream cmdlet has no built-in parameters to do the same! It is easier and much faster!

Think PowerShell! Use Pipeline Binding!

The next thing is finding the right cmdlet for the action you want to take. Rename-Item was the perfect choice. However, it is not necessary to use Foreach-Object (short: %) to iterate through all the files. Instead, Rename-Item is able to accept the files directly via pipeline. Get-Help can tell you that:

Get-Help -Name Rename-Item -Parameter Path

It returns: Accept pipeline input? true (ByValue, ByPropertyName)

So the second part of the solution looks like this:

Get-Childitem c:\somefolder -Filter cover*.jpg | Rename-Item -NewName "somenewname.jpg"

Rule #2: Before you resport to Foreach-Object, make sure the downstream cmdlet cannot receive your results via pipeline!

Think PowerShell! Use Scriptblocks!

Of course, renaming a bunch of files to the exact same name will fail, and thus you may be intrigued to use a foreach-object loop after all so you can compose individual file names. But that is not necessary. Many cmdlets accept scriptblocks which resemble executable code, so rather than submitting a fixed name, submit code that composes the name. The final solution looks like this:

$global:i = 1; dir c:\test1\ -filter cover*.bmp | rename-item -NewName { "picture_$i.jpg"; $global:i++}

This line will rename all jpg files that start with "cover" in folder c:\test1 to pictureX.jpg where X is an incrementing index number.

Note the use of global: (you could also use script:). With this prefix, you tell PowerShell to use the same variable. If you omitted that, incrementing $i inside the scriptblock would not have an effect on the global variable $i, because inside a script block, the variable is local and would be discarded each time after the script block was done.

Rule #3: Before you design complex code to calculate results for parameters, make sure the parameters do not accept scriptblocks in the first place.

Admittedly, this third rule is not that easy to follow because there is no easy way to know whether a parameter can actually receive a script block.

Hope to see you next week,

Tobias

Microsoft MVP PowerShell Germany

Troubleshooting Windows (and Dimming Your Display, too)

2011-05-17T07:55:00Z

You can't get a device working you just hooked up to your computer? Your computer is running slow? Or you just can't get these fancy Aero Glass features working? When bad things happen, it's usually time to get some coffee, lock the door and browse zillions of Internet forums that provide more or less useful suggestions. Or is there a smarter way?

With Windows Vista and Server 2008, Microsoft introduced "TroubleshootingPacks" which can analyze and then solve specific problems. They are really a clever idea: if your printer is not working, the printer troubleshooter "knows" all the common pitfalls and can check them for you. Troubleshooters can't solve everything, so when a troubleshooter can't come up with an automated solution, you at least know there is nothing stupid you overlooked before you invest hours to investigate yourself.

Why am I telling you this in my PowerShell blog? Because these troubleshooters run on PowerShell! That's correct: the entire problem analysis and repair process is plain PowerShell code. So the troubleshooters really are just another type of PowerShell host with a graphical user interface, and you could create your own set of troubleshooters to solve entirely different things.

Today, I am showing you how you can invoke troubleshooters from PowerShell, run them unattended to get problems solved fast, and visit the secret core of those troubleshooters where you find tons of useful internal code that you can reuse for your own things - like dimming the display after 5pm. Ready? Let's rock!

Note: Remember that the troubleshooting technology is available only on Windows Vista, Windows 7 and Server 2008/2008 R2 or better. On Windows XP, you can't use it.

Where are my Troubleshooters?

The built-in troubleshooters live in the diagnostics subfolder inside of your Windows folder. There, you find a folder called index which acts like a table of contents, and other folders like System and Scheduled which contain the actual troubleshooters. If you know the ID to a particular troubleshooter, you can invoke it easily from PowerShell. For example, to run the Printer troubleshooter, its ID is PrinterDiagnostic, and this is how you'd launch it from PowerShell:

Msdt.exe /ID PrinterDiagnostic

This runs the Printer troubleshooter. Watch out: if you happily run the wizard, it will automatically "repair" any problems it detected. Click Advanced and uncheck Apply repairs automatically if you would just like to see a list of detected problems. If you'd like to see other troubleshooters, open control panel:

control.exe /name Microsoft.Troubleshooting

This opens a dialog with all the locally installed troubleshooters, and in the left hand column you can click View All to access additional troubleshooters online.

That's all cool, but how do you know the troubleshooter ID you need to invoke from PowerShell? And is there also a way to call troubleshooters silently so they solve the problem without extensive user interaction?

Discover the TroubleshootingPack Module

Turns out there is. Actually, there is a dedicated PowerShell module for troubleshooters called TroubleShootingPack:

PS > Get-Module -ListAvailable Tro*

ModuleType Name ExportedCommands
---------- ---- ----------------
Manifest TroubleshootingPack
Let's import it and discover the new cmdlets:
PS > Import-Module trouble* -verbose
VERBOSE: Importing cmdlet 'Get-TroubleshootingPack'.
VERBOSE: Importing cmdlet 'Invoke-TroubleshootingPack'.

So there are just two cmdlets in it: Get-TroubleshootingPack and Invoke-TroubleshootingPack. Get-TroubleshootingPack gets you all the detail information about a specific troubleshooting pack including its ID. Just specify the path to the folder that contains the troubleshooter:

PS > Get-TroubleshootingPack -Path $env:windir\diagnostics\system\printer | ft -AutoSize

Id Name Publisher Version
-- ---- --------- -------
PrinterDiagnostic Printer Microsoft Windows 1.0

As always with objects, piping it to Select-Object * reveals a ton of additional information:

PS > Get-TroubleshootingPack -Path $env:windir\diagnostics\system\printer |
Select-Object *

Path : C:\Windows\diagnostics\system\printer
Id : PrinterDiagnostic
Version : 1.0
Publisher : Microsoft Windows
Name : Printer
Description : Troubleshoot problems preventing printing from completing.
MinimumVersion : 6.1
PrivacyLink : http://go.microsoft.com/fwlink/?LinkId=104288
SupportsClient : True
SupportsServer : True
SupportsX86 : True
SupportsAmd64 : True
SupportsIA64 : True
RequiresElevation : True
Interactive : True
RootCauses : {Print Spooler service isn't running, The Print Spooler ser
vice is experiencing problems, No physical printer is insta
lled, %PRINTERNAME% is not the default printer...}
FoundRootCauses :
Interactions : {Which printer would you like to troubleshoot?, Which print
er would you like to troubleshoot?, Add a printer, Update t
he printer driver}
ExtensionPoint : #document

Running Troubleshooters Unattended

With Invoke-TroubleshootingPack, you can run the troubleshooter. This is not like using msdt.exe, and the troubleshooter does not open a dialog window. Instead, it runs inside of your PowerShell window. Solving the problem is still an interactive process, though:

PS > Get-TroubleshootingPack -Path $env:windir\diagnostics\system\printer | Invo
ke-TroubleshootingPack
Checking the Spooler service...
Checking for Spooler service errors...

Which printer would you like to troubleshoot?
Select the printer that isn't working correctly.

[1] Fax
[2] HP LaserJet 4200/4300 PCL6
[3] Microsoft XPS Document Writer
[4] DYMO LabelWriter 400 Turbo

[?] Help
[x] Exit
:

To run troubleshooters unattended, you create an answer file with Get-TroubleshootingPack -answer and submit these answers to Invoke-TroubleshootingPack. You then can run troubleshooters even as a scheduled task unattended.

Here is a great detailed step-by-step introduction on how to create answer files and run troubleshooters unattended: http://blogs.technet.com/b/heyscriptingguy/archive/2011/02/10/use-powershell-and-scheduled-tasks-to-automate-troubleshooting.aspx

ExploitingTroubleshootingPacks

Each troubleshooting pack consists of a list of root causes (which describe why a specific problem may have occured), and each root cause has its own PowerShell scripts that identify problems and repair settings. For example, these are the root causes covered by the Power troubleshooter that aims to maximize your battery life:

PS > (Get-TroubleshootingPack $env:windir\diagnostics\system\Power).RootCauses

Name
----
Minimum processor state is set too high
Time before display dims is too long
Power plan set to High Performance
Time before display goes to sleep is too long
Time before computer goes to sleep is too long
Time before hard disk goes to sleep is too long
USB Selective Suspend is disabled
Wireless adapter is not optimized for power saving
Screen saver is enabled
Display brightness setting is too high

To check what a troubleshooter would do if a root cause was detected, take a look at the resolutions:

PS > ( Get-TroubleshootingPack $env:windir\diagnostics\system\power).RootCauses[
-1]

Name
----
Display brightness setting is too high

PS > ( Get-TroubleshootingPack $env:windir\diagnostics\system\power).RootCauses[
-1].Resolutions | ft -AutoSize

Name Manual Elevation Interactive
---- ------ --------- -----------
Decrease the display brightness False True True

Reusing TroubleShootingPack Cleverness

As you have seen, the Power troubleshootingpack does check for excessive display brightness and also contains a resolution for it. So it must contain some PowerShell code that is actually able to change the display brightness. But how?
Actually, the answer to this is the ID tagged to the resolution:

PS > ( Get-TroubleshootingPack $env:windir\diagnostics\system\power).RootCauses[
-1] | select Name, ID

Name Id
---- --
Display brightness setting is too high RC_ScreenBrightness

RC_ScreenBrightness stands for "Root Cause ScreenBrightness", and in the troubleshooter folder, you find two corresponding PowerShell scripts: TS_ScreenBrightness.ps1 and RS_ScreenBrightness.ps1. TS is short for "TroubleShooter", and RS contains the resolution code that fixes the problem. Actually, this is a fishy rule of thumb, though. When you look inside the folder, the resolver script is really called RS_AdjustScreenBrightness.ps1. Well, we never said digging into PowerShellPacks was that easy!

This code must contain a way to adjust the display brightness, so let's open that file:

Notepad $env:windir\diagnostics\system\power\RS_AdjustScreenBrightness.ps1

Whew. It is full of code - and the code isn't looking too friendly. Before you dig into it, have a look at the beginning of that file: it actually imports a library file called PowerConfig.ps1. That's where the gems are because this library makes all the energy savings settings accessible from PowerShell.

Notepad $env:windir\diagnostics\system\power\powerconfig.ps1

As you'll see, the library encapsulates Windows API functions so they can be called from PowerShell.

Dimming Display Brightness for Beginners

After three liters of coffee and nothing else to do in my hotel room, I created a little script that is much easier to use. It provides you with simple functions to get and set your notebook display brightness.

Get-VideoBrightness reads the current brightness percentage, and Set-VideoBrightness sets it. Get-DefaultVideoBrightness returns the recommended brightness level. All functions have a parameter -isAC because there are actually two schemes, one for battery power and one for AC power. This parameter by default is set by isAC, a tiny helper function that finds out whether you are running on battery or not. You can override this parameter, though, if you want to explicitly change the AC or the DC power scheme. You can DOWNLOAD THE SCRIPT here.

Note: this script depends on the Power TroubleshootingPack library so it runs only on systems that have this TroubleshootingPack (like Windows 7). Of course it only works for displays that can be dimmed, so use it on notebooks and not on your servers.

# by Tobias Weltner, PowerShell MVP

. $env:windir\diagnostics\system\power\powerconfig.ps1

function isAC {
(gwmi BatteryStatus -Namespace root\wmi).PowerOnline
}

function GetActiveSchemeGuid()
{
$activeSchemeGuid = [PowerConfig]::ActiveSchemeGuid()
return $activeSchemeGuid
}

function Get-VideoBrightness([bool]$isAC=$(isAC))
{
$activeSchemeGuid = $powerconfig::ActiveSchemeGuid()
$subGroupGuid = new-object system.Guid("7516b95f-f776-4464-8c53-06167f40cc99")
$settingGuid = new-object system.Guid("aded5e82-b909-4619-9949-f5d71dac0bcb")
$settingvalue = 0
$res = [PowerConfig]::ReadPowerSetting($isAC,[ref]$activeSchemeGuid,[ref]$subGroupGuid,[ref]$settingGuid,[ref]$settingvalue)
if($res -eq 0)
{
$settingvalue
}
}

function Get-DefaultVideoBrightness([bool]$isAC=$(isAC))
{
$BalancedPowerPlan = $powerconfig::BalancedPowerPlan()
$subGroupGuid = new-object system.Guid("7516b95f-f776-4464-8c53-06167f40cc99")
$settingGuid = new-object system.Guid("aded5e82-b909-4619-9949-f5d71dac0bcb")
$settingvalue = 0
$res = [PowerConfig]::ReadDefaultSetting($isAC,[ref]$BalancedPowerPlan,[ref]$subGroupGuid,[ref]$settingGuid,[ref]$settingvalue)
if($res -eq 0)
{
return $settingvalue
}
}

function Set-VideoBrightness([int]$settingvalue = 100, [bool]$isAC=$(isAC))
{
$activeSchemeGuid = $powerconfig::ActiveSchemeGuid()
$subGroupGuid = new-object system.Guid("7516b95f-f776-4464-8c53-06167f40cc99")
$settingGuid = new-object system.Guid("aded5e82-b909-4619-9949-f5d71dac0bcb")
[void][PowerConfig]::WritePowerSetting($isAC,[ref]$activeSchemeGuid,[ref]$subGroupGuid,[ref]$settingGuid,$settingvalue)
}

# get current video brightness (0-100%)
Get-VideoBrightness

# get recommended brightness
Get-DefaultVideoBrightness

# set video brightness (0-100)%
Set-VideoBrightness 100

What You Can Do Next

As you have seen, TroubleshootingPacks are really rich PowerShell code repositories that contain a lot of clever code to analyze and fix common computer problems.

Experienced PowerShell scripters can analyze the code and extract the parts they need to control windows. There are a lot more gems to find, especially when it comes to tricky device management.

Not so experienced PowerShell scripters can take advantage of scripts like the one I provided to dim display brightness, encapsulating all the complex stuff.

And hey, if you don't want to rip a TroubleshootingPack apart, you now also know how to run it altogether, interactively or unattended with an answer file. Pretty cool, huh?

Hope to see you next week,

Tobias

Microsoft MVP PowerShell Germany

Multiple Text Replacement Challenge

2011-04-28T02:45:00Z

Ready for a challenge? Here it is: take a text, and replace a number of different characters. How would you do that?

I wanted PowerShell to replace all German special characters with alternative text. Boy, was I in for a surprise!

If you'd like to take on the challenge, pause reading now and go find a good solution! Then come back and compare it with the solutions that came to my mind.

Ok, if you do take on the challenge, this is the text to work with:

$text = 'Österreich ärgert süße klötenhafte Ärgonauten' * 1000

The job is to do the following replacements:

ä = ae, ö = oe, ü = ue, Ä = Ae, Ö = Oe, Ü = Ue, ß = ss

Note: You can download all code samples on this page right here: http://powershell.com/cs/media/p/10171.aspx.

Creative Solutions using Switch

You can use Switch to do multiple string replacements. For it to work, you would need to split the text in characters first, do the replacements, then -join it together.

The code is clean, and it's easy to edit the replacement table:

$text = 'Österreich ärgert süße klötenhafte Ärgonauten' * 1000

-join $(switch -case ( [Char[]]$text ) {
ä { 'ae' }
ö { 'oe' }
ü { 'ue' }
Ä { 'Ae' }
Ö { 'Oe' }
Ü { 'Ue' }
ß { 'ss' }
default { $_ }
})

With 883 milliseconds, this approach takes a long time to run, though. Bummer.

Creative Solutions Using Hashtables

Hashtables are another good way of providing replacement tables in a way that makes it easy to edit and adjust them.

However, since PowerShell creates hashtables with case-insensitive keys, it is not possible to differentiate between lower case and upper case keys.

The workaround here is to get the hashtable from .NET directly. This way, keys are case-sensitive. Here is the solution:

$text = 'Österreich ärgert süße klötenhafte Ärgonauten' * 1000
$ht = new-object System.Collections.Hashtable
$ht.ä='ae'
$ht.ö='oe'
$ht.ü='ue'
$ht.Ä='Ae'
$ht.Ö='Oe'
$ht.Ü='Ue'
$ht.ß='ss'

[Regex]::Replace($text, "[$(-join $ht.keys)]", {$ht[$Args[0].Value]})

With 405 milliseconds, this approach is a lot faster. However, it is still relatively slow, and the way the text is actually replaced is a bit complex, using a regex replacement with a dynamic function.

Using a Replacement Table - Much Faster

I did like the idea of using a hash table as a replacement table, though. I did not like the complex regex clause, and of course I wanted it faster.

So here is the next version. It uses a foreach loop and the -creplace operator:

$text = 'Österreich ärgert süße klötenhafte Ärgonauten' * 1000
$ht = new-object System.Collections.Hashtable
$ht.ä='ae'
$ht.ö='oe'
$ht.ü='ue'
$ht.Ä='Ae'
$ht.Ö='Oe'
$ht.Ü='Ue'
$ht.ß='ss'
foreach($key in $ht.Keys) { $text = $text -creplace $key, $ht.$key }
$text

This code just takes 7,3 milliseconds, so it runs 55x faster - wow. Can it run even faster?

Hard Coding - and a Big Surprise

Inspired by the top performance -creplace delivered, I next tried a hard-coding approach:

$text = 'Österreich ärgert süße klötenhafte Ärgonauten' * 1000
$text -creplace 'ä', 'ae' -creplace 'ö', 'oe' -creplace 'ü',
'ue' -creplace 'Ä', 'Äe' -creplace 'Ö', 'Oe' -creplace 'Ü',
'Ue' -creplace 'ß', 'ss'

Surprisingly, this code was not faster than the previous approach. Considering how ugly and unflexible this code was, I moved on immediately in disgust.

Hard Coding again - and another Surprise

Next, I tried another hard-coded approach, this time using string methods over regular expressions:

$text = 'Österreich ärgert süße klötenhafte Ärgonauten' * 1000
$text.Replace('ä', 'ae').Replace('ö', 'oe').Replace('ü',
'ue').Replace('Ä', 'Ae').Replace('Ö', 'Oe').Replace('Ü',
'Ue').Replace('ß', 'ss')

While this code was ugly, too, with just 2 milliseconds it ran lightning fast. There is no faster way for multiple string replacements.

Dynamic Code Combines Best of Both Worlds

So I finally was looking for a way to take advantage of the speed yet create a code design that would allow it to edit and adjust the replacement strings dynamically. Here it is:

$text = 'Österreich ärgert süße klötenhafte Ärgonauten' * 1000
$t = 'ä,ae,ö,oe,ü,ue,Ä,Ae,Ö,Oe,Ü,Ue,ß,ss'
Invoke-Expression ('$text' + -join $(
foreach($e in $t.Split(',')) {
'.Replace("{0}","{1}")' -f $e,
$([void]$foreach.MoveNext();$foreach.Current)
}
))

With still less than 3 milliseconds, this approach was super fast, 338x faster than the original approach. Note how it uses Invoke-Expression to actually generate dynamic code on-the-fly, so it is no longer hard-coding the replacement strings. With the list of replacement texts in $t, it is very easy to adjust. So I decided to stick with this one and make it a function:

function Update-Text {
param(
[Parameter(Mandatory=$true)]
$text,

$replacementlist = 'ä,ae,ö,oe,ü,ue,Ä,Ae,Ö,Oe,Ü,Ue,ß,ss'
)
Invoke-Expression ('$text' + -join $(
foreach($e in $replacementlist.Split(',')) {
'.Replace("{0}","{1}")' -f $e, $(
[void]$foreach.MoveNext()
$foreach.Current)
}
)
)
}

PS > Update-Text "This is a test" 'T,t,i,I,te,TE'
thIs Is a TEst

Hope to see you next week,

Tobias

Microsoft MVP PowerShell Germany

One PowerShell Tip A Day Keeps The Doctor Away

2011-04-12T15:19:00Z

You probably know I have been writing one PowerShell tip every day - since 2008. This piled up to a tremendous gold mine of little tips and tricks you can use to improve your code.

Sometimes, the truly amazing thing is to see what you do with these bits and pieces, and today I'd like to thank Robbie Wiley for sharing his personal use case and how he redesigned it using one of our last weeks' tips.

" I have a function called Get-Scrapnotes that I use to find a series of text files that I create each month. The file is always called scrapnotes.txt and it has been my practice for several years to archive that file. WHERE it has been archived has changed a little bit over the years. I have to look for it in my archive directory at 1 or 2 levels deep beneath the year/month directory."

Turning Slow Code into Fast Code

One of the first solutions Robbie came up with was to use Get-Childitem with -recurse like this:

Get-ChildItem e:\archives -include 'scrapnotes.txt' -recurse

This worked but took minutes to complete because it had to visit any single folder below the archives directory, and some of which were extensive. It also was using slow 'client side' filtering: Get-ChildItem first gets every single file, and then -include picks the one you want.

After some investigation, Robbie improved the solution:

Get-ChildItem e:\archives -filter 'scrapnotes.txt' -recurse

This cut the time almost in half because -filter is using "server side" filtering: Get-Childitem gets the files you want in the first place and skips all others. Still, it took minutes to run.

He then created a sophisticated script to optimize the other source of performance penalty: to limit the number of subfolders the script needed to visit. His script did improve execution time considerably but was too large, complex and specific to share here.

Searching Just Two Levels Deep

Last week we published the tip below that inspired Robbie to revisit his solution and improve it a bit more:

"Did you know that you can use multiple wildcards in paths? This will give you a lot of control. Check this out: This line will find all DLL-files in all sub-folders up to two levels below your Windows folder:

Resolve-Path $env:windir\*\*\*.dll -ea 0

And this line will lists the Desktop folder content for all user accounts on your computer-provided that you have sufficient privileges:

dir c:\users\*\desktop\*"

Take a look what exactly Robbie did with this piece of information. Since he knew the files he was looking for were located anywhere within the first two subdirectory levels, he managed to search just these levels within a matter of milliseconds, using the wildcard trick:

$scrapnotes = @()
$scrapnotes += resolve-path E:\Archives\20*\*\*\scrapnotes.txt -ea 0 | % {Get-Item $_.path}
$scrapnotes += resolve-path E:\Archives\20*\*\scrapnotes.txt -ea 0 | %{Get-Item $_.path}
$scrapnotes += resolve-path E:\Archives\20*\scrapnotes.txt -ea 0 | %{Get-Item $_.path}
$scrapnotes = $scrapnotes | sort creationtime

That is pretty excellent. Thanks Robbie for sharing this!

Hope to see you next week,

Tobias

Microsoft MVP PowerShell Germany

Executing Code On Behalf Of Someone Else

2011-03-13T15:19:00Z

In almost all of my trainings I get this question: "Can I launch a program remotely so another user can actually see it?" To answer that question, I'd like to take you on a quick security tour and provide you with a test tool that may or may not produce surprising results in your environment.

Why Launching Things For Other Users Can Be Bad

Back in the old days, you were able to use WMI to launch a process visibly in another users context. So if you were working at some help desk, and a user called in, you could simply help them and launch things for them. However, anything you launched continued to run under your account, so if you launched a notepad.exe for someone else, that person was able to use notepad with your credentials. Of course, Microsoft disabled that for security reasons.

Today, when the target is Windows Vista/Server 2008 or better, you can again launch processes visibly in another users context. This time, however, the task scheduler is used. It can be instructed to launch whatever you want in the interactive session of someone else. All you need is this:

Local Administrator privileges on the target machine
Windows Vista/W7 or Server 2008/R2 on all machines

Because the task scheduler runs the task on behalf of the target person, you are not jeopardizing your own identity anymore. The user can use whatever you launched, but he will be using it with his own privileges, not yours.

That sounds great until you start thinking backwards. What if the person I am launching the program for has actually more privileges that I do? Huh?

Houston - We (Might) Have A Problem!

Which brings us to the second objective of this post: Security. In not so few companies, local admin accounts are below security radar. Sometimes, the password for the local "Administrator" account is even common knowledge because it is used whenever someone needs to install something special or change settings. That's convenient until people start to realize that a local administrator is potentially just as powerful as the most powerful domain account that exists in your entire company. Let's check out why.

With the task scheduler, any local admin can launch programs on behalf of the logged on user. So if that user happens to be Enterprise Admin, and you wouldn't launch something as cute as notepad.exe but instead a PowerShell script that adds your account (or all accounts in your company) to the Domain Admin group, you probably can imagine what this means to corporate security. Since the script runs on behalf of the other person, if someone analyzes the change to find out the bad guy, logs won't show the true script initiator but instead the person that it was scheduled for.

So is this an exploit? Something to get really excited about? No - except if you did not realize the importance of local admin accounts in overall corporate security. A local admin owns a local machine. So a local admin could have installed a key logger, replaced explorer.exe with some malicious clone, or simply put script code into the logon script of someone else. The script I am going to present is special only in that it is (a) pretty easy to use and (b) a great security intrusion detector because if you really can do something bad with it, then it's time to seriously review your security settings.

Word Of Caution

Use the script to test-drive your security only.

If you are a domain admin, then you know you have plentyful privileges but still might find the script useful to launch software visibly for other people. Since you are domain admin, you are authorized to do this. If you abuse your privileges, then that's like stealing pencils or selling company secrets, and you get into trouble.

If you are however a regular user and you still can launch things on behalf of others, or if you know local admin credentials because they are not kept secret, and you can use them to launch things in other users sessions, then you know something is wrong. Don't continue to use this script to launch "special" websites on behalf of your boss just to see what happens next, or fire up 1000 instances of excel on behalf of that buddy of yours down the isle. Instead, report it to your security department to help create more awareness.

Launching Commands On Behalf of Someone Else

The following script provides a function called Start-ProcessInteractive. You can specify a target computer, a path to an executable on that computer and also some arguments. If you do not specify a path and arguments, then the defaults will run a powershell command and write a file into the target windows folder. That's a proof-of-concept, because if that command works it illustrates that the PowerShell command was launched with full privileges.

The function schedules the task for the currently logged on user. If no currently logged on user could be identified, then either no user was physically logged on, or the target machine is running virtual machines in which case you can manually submit the user account you want to schedule the command for.

Note that this type of task scheduling requires Windows Vista/Server 2008 or higher. It will not work on XP.

You can download the script here.

function Start-ProcessInteractive {
param(
$filepath = 'powershell.exe',
$arguments = '-noprofile -command Get-Date | Out-File $env:windir\testfile.txt',
[Parameter(Mandatory=$true)]
$computername
)

function Execute-Tool($path) {
$r = (Invoke-Expression $path) 2>&1
if ($LASTEXITCODE -ne 0) { Throw $r[0].Exception.Message }
}

$computername | ForEach-Object {
try {
$username = Get-WmiObject Win32_ComputerSystem -ComputerName $_ |
Select-Object -ExpandProperty UserName
} catch {}
$computer = $_

if ($username -eq $null) {
Write-Warning "On $computername no user is currently physically logged on."
$username = Read-Host "Enter username of logged on user at the remote system"
}
if ($username -ne '') {

$xml = @"
<?xml version="1.0" encoding="UTF-16"?>
<Task version="1.2" xmlns="http://schemas.microsoft.com/windows/2004/02/mit/task">
<RegistrationInfo />
<Triggers />
<Settings>
<MultipleInstancesPolicy>IgnoreNew</MultipleInstancesPolicy>
<DisallowStartIfOnBatteries>false</DisallowStartIfOnBatteries>
<StopIfGoingOnBatteries>false</StopIfGoingOnBatteries>
<AllowHardTerminate>true</AllowHardTerminate>
<StartWhenAvailable>false</StartWhenAvailable>
<RunOnlyIfNetworkAvailable>false</RunOnlyIfNetworkAvailable>
<IdleSettings />
<AllowStartOnDemand>true</AllowStartOnDemand>
<Enabled>true</Enabled>
<Hidden>false</Hidden>
<RunOnlyIfIdle>false</RunOnlyIfIdle>
<WakeToRun>false</WakeToRun>
<ExecutionTimeLimit>PT72H</ExecutionTimeLimit>
<Priority>7</Priority>
</Settings>
<Actions Context="Author">
<Exec>
<Command>$filepath</Command>
<Arguments>$arguments</Arguments>
</Exec>
</Actions>
<Principals>
<Principal id="Author">
<UserId>$username</UserId>
<LogonType>InteractiveToken</LogonType>
<RunLevel>HighestAvailable</RunLevel>
</Principal>
</Principals>
</Task>
"@

$jobname = 'remotejob{0}' -f (Get-Random)

try {
$xml | Out-File "$env:temp\tj1.xml"
Execute-Tool "schtasks /CREATE /TN $jobname /XML $env:temp\tj1.xml /S $computer"
Start-Sleep -Seconds 1
Execute-Tool "schtasks /RUN /TN $jobname /S $computer"
Execute-Tool "schtasks /DELETE /TN $jobname /s $computer /F"
}
catch {
Write-Warning "$_ (trying to access user '$username' on system '$computer')"
}
}
}
}

I have used this script successfully in a number of environments but I cannot guarantee anything. It is just a proof-of-concept. I am looking forward to get your feedback.

Security is important, and I sincerely hope I was able to illustrate why protecting even local admin accounts is important in any good security concept. If you feel that this script could do harm in your environment, then don't blame the script for it. Instead, make sure it can't do harm by reviewing your corporate security settings. This script is just one example of a multitude of other approaches that take advantage of the power of local admins. So if this script can do something it shouldn't, then a heck of a number of other techniques could, too. Close that security hole.

If security is not important to you (or your security is in great shape), then you still might find that script useful. It demonstrates how to schedule tasks using XML. The problem with the regular command line switches is that they are (a) localized (different for every language/culture) and (b) incomplete. You cannot control all the things you can set up when scheduling a task using the UI. With XML, you can define the task culture-neutral and you have access to all settings.

Hope to see you next week,

Tobias

Microsoft MVP PowerShell Germany

Analyzing Networks With PowerShell

2011-03-09T02:16:00Z

I often need to analyze networks and find machines and IP addresses. So today, I am going to share a set of four (4) very useful PowerShell functions that make it easy to analyze network segments - and they are pretty fast, too! This is the second part of a 2-part-series. Here is the first part btw.

Check this out: this simple line just scanned my hotel wlan and returned my neighbors computers IP addresses (and some host names) in about 5 seconds:

PS > New-IPRange | Test-Online | Get-HostName

HostName Aliases AddressList
-------- ------- -----------
server.mshome.net {} {192.168.0.1}
SCHUPIAP1 {} {192.168.0.50}
WARNING: IP 192.168.0.51 did not return DNS information
WARNING: IP 192.168.0.52 did not return DNS information
WARNING: IP 192.168.0.53 did not return DNS information
demo5 {} {192.168.0.158}

Of course the real value is not sniffing through public networks. That's what you do when you are grounded at some hotel and bored. In real life, you would use these functions to find network devices, printers, or just identify online systems that are ready and available for remote access.

Network Analyzing Tools Made by PowerShell

I decided to create these four PowerShell functions:

Get-IPAddress [-first] [-IPv4] [-IPv6]
New-IPRange [-first]
New-IPRange [-start <String>] [-end <String>]
Test-Online [-computername] <String> [[-throttleLimit] <Int32>]
Get-HostName [-IPAddress] <String[]>

Get-IPAddress returns your current IP address(es). You can select whether you want IPv4, IPv6 or both, and whether you want just the first available IP address or all of them.
New-IPRange creates an IPv4 address range. If you don't specify anything, you get the range your network adapter is in. You can also use -start and -end to create other IP address ranges
Test-Online uses background jobs and parallel processing to ping a range of ip addresses. It is very fast. -throttleLimit controls how many simultaneous pings are used. The default is 300.
Get-HostName resolves a list of ip addresses and returns host name and alias.

You can download the functions here.

Some Background

PowerShell comes with a cmdlet called Test-Connection which can “ping” ip addresses and host names. Like most cmdlets, it works sequentially by default. Pinging hundreds of computers can become a lengthy task. Fortunately, PowerShell V2 supports background jobs. That way, tasks can be executed in parallel rather than sequentially. Pinging an entire network segment now takes only seconds.

The most efficient way to use PowerShells new background job capabilities is to use cmdlets that support background jobs natively. These cmdlets all support the –asJob switch parameter.

To make it simple to ping your current network segment, Get-IPAddress can find out your own IP address, and New-IPRange uses this to return all IP addresses within that segment.

You can then pipe the IP address list to Test-Online, and after just a couple of seconds, you receive a filtered IP address list with only online addresses.

Of course, the cmdlet Test-Connection used by Test-Online has limited ressources, and once you process more than 400 or so addresses at once, you may get exceptions. That's why it makes sense to implement a throttle limit. This works by splitting arrays of IP addresses into chunks and processing these chunks sequentially. Throttling needs to be implemented manually by Test-Online because the -throttleLimit parameter supported natively by Test-Connection is broken and does not do anything.

What's Next?

More fun is coming up. I am currently teaching PowerShell at a number of companies and universities, and we always find interesting things to play with. For example, we just created a script to launch commands on behalf of someone else. It was considerable fun to open web pages in the session of our enterprise admin (although his amusement was somewhat limited). It was not so much fun anymore when we discovered that we could launch arbitary PowerShell code with full privileges on his behalf. All we needed were local admin rights on his machine, and it is common practice to grant this to a lot of folks in many companies (but obviously not a good idea as you'll see).

So before I share with you excatly how that worked and what we did, I'd like to check back with a couple of folks and also want to point out that this whole thing really primarily emphasizes how important even local admin accounts are for corporate security.

Hope you had fun, see you around next week ...

Tobias

Microsoft MVP PowerShell Germany

Accessing Databases from PowerShell

2011-03-01T16:54:00Z

Recently, I needed to access an SQL database from PowerShell. There are no cmdlets to help you, but with a bunch of helper functions, it is easy and straightforward to access databases (including Microsoft Access files), retrieve data using SQL and even inserting and updating information (I know I promised to talk about speeding up range pings in your network but this database issue got into my way so I decided to cover this first ).

Connection String: Where It All Begins

The first thing you need to access a database is a "connection string". It is a cryptic piece of text telling the computer where the database is located and what database driver you want to use. Constructing such a connection string can be a mess. Fortunately, there are some secret tricks you can use to create the connection string automatically. Have a look at the function New-ConnectionString (which you can download here):

function New-ConnectionString {
$path = "$env:temp\connection.udl"
New-Item $path -ItemType File -Force | Out-Null
Start-Process $path -Wait
try {
(Get-Content $path)[2..10] | Out-String
} catch { }
Remove-Item $path
}

When you run New-ConnectionString, it invokes a system dialog that helps you choose the correct database driver, pick the database you want to connect to, configure the connection using logon credentials etc. Simply choose the option to use a connection string and then click the button to create one.

Now, the wizard walks you through all the necessary configuration steps. Once done, you can even test the connection to see if you did everything right, and when you close the wizard, New-ConnectionString spits out the magic connection string you just created! Cool, eh?

NOTE: Many database drivers still only exist in 32bit versions, so if you are using a 64bit machine, you may want to try this in a 32bit PowerShell console. You will notice that New-ConnectionString will offer a lot more database drivers when launched from within a 32bit PowerShell than a 64bit PowerShell. To run PowerShell in 32bit mode on 64bit, launch PowerShell (x86) instead of PowerShell.

Here is how you would create a connection string to a plain old Microsoft Access mdb database:

Launch the 32bit Version of PowerShell when you work on a 64bit machine (most Access database drivers only exist in 32bit versions)
Call New-ConnectionString to invoke the wizard
Choose the option to use a connection string (second option in section #1)
Click the button to create a new connection string
A new window opens. Click the "computer data source" tab, and double-click the database driver you want to use, for example "MS Access Database"
Yet another window opens. Click the button "Database", and select the mdb database file you want to use
Leave the fields for username and password blank if the database does not require authentication, and click OK
Optionally, you can now configure details. For example, check "No password required", or click the "Advanced" tab to make additional adjustments
Click the button "Test Connection" at the lower end of the dialog window to test your configuration and check whether you can access the database.
Finally, click "OK". New-ConnectionString returns the connection string to you. It looks similar to this:

PS > New-ConnectionString
Provider=MSDASQL.1;Persist Security Info=False;Extended Properties="DSN=MS Access Database;
DBQ=c:\skripts\nordwind.mdb;DefaultDir=c:\skripts;DriverId=25;
FIL=MS Access;MaxBufferSize=2048;PageTimeout=5;UID=admin;"

Accessing a Database

Now that you have a valid connection string to your database, you can send plain SQL to it and get back results. I created a small function called Invoke-SQL. It takes a valid SQL command that you want to send to your database, and the connection string you just created so that it knows where to retrieve the data. Here's the code in action (apologies for using a German sample database, but I am sure you get the idea):

PS > $c = New-ConnectionString
PS > Invoke-sql 'select Firma, Land, Telefon from kunden' $c

Land Firma Telefon
---- ----- -------
Deutschland Alfreds Futterkiste 030-0074321
Mexiko Ana Trujillo Emparedados y helados (5) 555-4729
Mexiko Antonio Moreno Taquería (5) 555-3932
Großbritannien Around the Horn (71) 555-7788
Schweden Berglunds snabbköp 0921-12 34 65
Deutschland Blauer See Delikatessen 0621-08460
Frankreich Blondel père et fils 88.60.15.31
Spanien Bólido Comidas preparadas (91) 555 22 82
(...)

NOTE: In this example, I am creating the connection string from scratch using New-ConnectionString and going through the wizard. In real-world scenaros, of course, you'd run New-ConnectionString only once to get the string and then hard-code that into your scripts.

Here is Invoke-SQL (you can download it here):

function Invoke-SQL {
param(
[Parameter(Mandatory=$true)]
$sql = "select * from Kunden",
[Parameter(Mandatory=$true)]
$connectionstring
)

$db = New-Object -comObject ADODB.Connection
$db.Open($connectionstring)
$rs = $db.Execute($sql)
while (!$rs.EOF) {
$hash = @{}
foreach ($field in $rs.Fields) {
$hash.$($field.Name) = $field.Value
}
$rs.MoveNext()
New-Object PSObject -property $hash
}
$rs.Close()
$db.Close()
}

Note that Invoke-SQL returns rich PowerShell objects so once you get the database results, you can easily pipe it on to other cmdlets to export it to a csv-file or do whatever else you want. Note also that I am using old COM objects to do the database access. I compared performance with native .NET methods, and there is really no speed difference. The COM approach is reliable, fast and needs much less code than .NET.

Important: When you run Invoke-SQL on a 64bit machine, the same restrictions apply that I mentioned for New-ConnectionString: if you picked a database driver that only exists in a 32bit environment, then you can only use this driver (and connection string) from within a 32bit PowerShell. So again, on 64bit machines you may want to run this code in a 32bit PowerShell console.

Hope you had fun, see you next week around...

Tobias

Microsoft MVP PowerShell Germany