Regular Expressions Are Your Friend (Part 2)

Tobias — Mon, 14 Nov 2011 10:37:00 GMT

In Part 1, we used Regular Expressions to extract useful information from noise text. That was pretty cool, and we used the operator -match, the automatic PowerShell variable $matches and the powerful Get-Matches function.

Today, we'll use Regular Expressions again, but this time they are used to split text.

Chopping Text Into Pieces

Regular Expressions can be your butchers' knife to split text into pieces. Why on earth would you like do that, though?

For example, to process a comma-separated list of values. Have a look:

'comma,separated,list' -split ','

Here, the operator -split takes the text on its left side and chops the text up at each comma it finds. Yes, Regular Expressions do not need to be monsters. They can be very small. In this case, our Regular Expression simply is the comma - and represents a comma. It is - uh - a placeholder for - a comma. Easy enough.

You now could pipe the results to Foreach-Object and do stuff for all the elements in your comma-separated list:

$computers = 'pc1,pc2,server12,serverAB'
$computers -split ',' | Foreach-Object { "I am doing something with $_" }
I am doing something with pc1
I am doing something with pc2
I am doing something with server12
I am doing something with serverAB

$computers -split ',' | Foreach-Object { Get-WMIObject Win32_BIOS -computername $_ }

You can also use splitting to take apart path names. Here, the pattern is the backslash (if you want the parts of the path) or a dot (if you are after the file extension). This time, however, the pattern to use is not as straight-forward. Both the backslash and the dot are special characters in Regular Expressions. So if you really mean a backslash or a dot, you need to escape them:

PS> 'c:\windows\file.txt' -split '\.'
c:\windows\file
txt
PS> ('c:\windows\file.txt' -split '\.')[-1]
txt
PS> 'c:\windows\file.txt' -split '\\'
c:
windows
file.txt
PS> ('c:\windows\file.txt' -split '\\')[-1]
file.txt

Escaping Regular Expressions

As you have just discovered, sometimes it is necessary to escape characters in Regular Expressions so they are taken literally.

Fortunately, there is a way to automatically escape characters. If you aren't sure if something you mean literally should indeed be escaped or not, try using this trick:

PS> [Regex]::Escape('c:\folder\file.txt')
c:\\folder\\file\.txt

It takes the text you provide and returns it in escaped form so Regular Expressions wouldn't treat anything in that text as special characters.

Splitting Without Consuming

By default, when you chop up text using -split, the split character(s) are "consumed", so they are gone after the split.

PS> $text = 'This text gets splitted, and the delimiter is by default consumed, as you can see'
PS> $text -split ','
This text gets splitted
and the delimiter is by default consumed
as you can see

With look-behind expressions, you can make sure that the splitting pattern is not consumed:

PS> $text -split '(?<=, )'
This text gets splitted,
and the delimiter is by default consumed,
as you can see

Basically, "?<=" tells the RegEx engine to split right after the splitting character. Splitting without consuming can become important as you'll see in a second.

Separating Hex-Pairs

Let's assume you have a long list of hexadecimal pairs and would like to convert that list back to decimals:

'00AB1CFFAB1034'

To do that, you would somehow have to split after every second character and without consuming because you do not want to loose any information. Here is how you solve this puzzle:

PS> '00AB1CFFAB1034' -split '(?<=\G[0-9a-f]{2})(?=.)'
00
AB
1C
FF
AB
10
34

Now you can pipe the pairs to Foreach-Object and convert them to decimals or bytes or whatever you may need:

PS> '00AB1CFFAB1034' -split '(?<=\G[0-9a-f]{2})(?=.)' |
Foreach-Object { [System.Convert]::ToInt32($_, 16) }
0
171
28
255
171
16
52

PS> '00AB1CFFAB1034' -split '(?<=\G[0-9a-f]{2})(?=.)' |
Foreach-Object { [System.Convert]::ToByte($_, 16) }
0
171
28
255
171
16
52

Whew, wait a minute! This time, the Regular Expression wasn't as simple anymore. Right. Regular Expressions aren't hated without a reason. Sometimes they become tricky little monsters with mind-blowing lookaheads, lookbehinds and lookarounds. But is that a reason not to use them? No! We all use microwaves to make popcorn, but do we care about the physics? I don't.

So always remember: you do not need to care about Regular Expression mechanics once you have one that does what you want it to do. So once you have a Regular Expression that splits Hex-Pairs, you are good. Simply use it.

Instead of spending frustrating hours of crafting complex RegEx patterns, think "code reuse" and visit a search engine like google.Look for "RegEx <Pattern>", for example "RegEx IPAddress". Chances are someone else has already done the dirty job for you, and all you need to do is copy and paste the RegEx pattern.

If you do want to digest patterns and really understand what is going on, there are plenty of great RegEx tutorial sites. This one explains lookbehinds, lookaheads and lookarounds and helps you understand the previous example: http://www.regular-expressions.info/lookaround.html.

What’s Next?

We covered -match and -split, but there's a third family member: -replace. So next time we'll continue talking about RegEx and look how you can replace things, update IP lists and more. Meanwhile, if you have spare time, go visit some of the plenty sites that focus on RegEx, and try and refine your understanding of RegEx placeholders, anchors and quantifiers. Stay tuned!

Tobias

Microsoft MVP PowerShell Germany