Multiple Text Replacement Challenge

Tobias — Thu, 28 Apr 2011 02:45:00 GMT

Ready for a challenge? Here it is: take a text, and replace a number of different characters. How would you do that?

I wanted PowerShell to replace all German special characters with alternative text. Boy, was I in for a surprise!

If you'd like to take on the challenge, pause reading now and go find a good solution! Then come back and compare it with the solutions that came to my mind.

Ok, if you do take on the challenge, this is the text to work with:

$text = 'Österreich ärgert süße klötenhafte Ärgonauten' * 1000

The job is to do the following replacements:

ä = ae, ö = oe, ü = ue, Ä = Ae, Ö = Oe, Ü = Ue, ß = ss

Note: You can download all code samples on this page right here: http://powershell.com/cs/media/p/10171.aspx.

Creative Solutions using Switch

You can use Switch to do multiple string replacements. For it to work, you would need to split the text in characters first, do the replacements, then -join it together.

The code is clean, and it's easy to edit the replacement table:

$text = 'Österreich ärgert süße klötenhafte Ärgonauten' * 1000

-join $(switch -case ( [Char[]]$text ) {
ä { 'ae' }
ö { 'oe' }
ü { 'ue' }
Ä { 'Ae' }
Ö { 'Oe' }
Ü { 'Ue' }
ß { 'ss' }
default { $_ }
})

With 883 milliseconds, this approach takes a long time to run, though. Bummer.

Creative Solutions Using Hashtables

Hashtables are another good way of providing replacement tables in a way that makes it easy to edit and adjust them.

However, since PowerShell creates hashtables with case-insensitive keys, it is not possible to differentiate between lower case and upper case keys.

The workaround here is to get the hashtable from .NET directly. This way, keys are case-sensitive. Here is the solution:

$text = 'Österreich ärgert süße klötenhafte Ärgonauten' * 1000
$ht = new-object System.Collections.Hashtable
$ht.ä='ae'
$ht.ö='oe'
$ht.ü='ue'
$ht.Ä='Ae'
$ht.Ö='Oe'
$ht.Ü='Ue'
$ht.ß='ss'

[Regex]::Replace($text, "[$(-join $ht.keys)]", {$ht[$Args[0].Value]})

With 405 milliseconds, this approach is a lot faster. However, it is still relatively slow, and the way the text is actually replaced is a bit complex, using a regex replacement with a dynamic function.

Using a Replacement Table - Much Faster

I did like the idea of using a hash table as a replacement table, though. I did not like the complex regex clause, and of course I wanted it faster.

So here is the next version. It uses a foreach loop and the -creplace operator:

$text = 'Österreich ärgert süße klötenhafte Ärgonauten' * 1000
$ht = new-object System.Collections.Hashtable
$ht.ä='ae'
$ht.ö='oe'
$ht.ü='ue'
$ht.Ä='Ae'
$ht.Ö='Oe'
$ht.Ü='Ue'
$ht.ß='ss'
foreach($key in $ht.Keys) { $text = $text -creplace $key, $ht.$key }
$text

This code just takes 7,3 milliseconds, so it runs 55x faster - wow. Can it run even faster?

Hard Coding - and a Big Surprise

Inspired by the top performance -creplace delivered, I next tried a hard-coding approach:

$text = 'Österreich ärgert süße klötenhafte Ärgonauten' * 1000
$text -creplace 'ä', 'ae' -creplace 'ö', 'oe' -creplace 'ü',
'ue' -creplace 'Ä', 'Äe' -creplace 'Ö', 'Oe' -creplace 'Ü',
'Ue' -creplace 'ß', 'ss'

Surprisingly, this code was not faster than the previous approach. Considering how ugly and unflexible this code was, I moved on immediately in disgust.

Hard Coding again - and another Surprise

Next, I tried another hard-coded approach, this time using string methods over regular expressions:

$text = 'Österreich ärgert süße klötenhafte Ärgonauten' * 1000
$text.Replace('ä', 'ae').Replace('ö', 'oe').Replace('ü',
'ue').Replace('Ä', 'Ae').Replace('Ö', 'Oe').Replace('Ü',
'Ue').Replace('ß', 'ss')

While this code was ugly, too, with just 2 milliseconds it ran lightning fast. There is no faster way for multiple string replacements.

Dynamic Code Combines Best of Both Worlds

So I finally was looking for a way to take advantage of the speed yet create a code design that would allow it to edit and adjust the replacement strings dynamically. Here it is:

$text = 'Österreich ärgert süße klötenhafte Ärgonauten' * 1000
$t = 'ä,ae,ö,oe,ü,ue,Ä,Ae,Ö,Oe,Ü,Ue,ß,ss'
Invoke-Expression ('$text' + -join $(
foreach($e in $t.Split(',')) {
'.Replace("{0}","{1}")' -f $e,
$([void]$foreach.MoveNext();$foreach.Current)
}
))

With still less than 3 milliseconds, this approach was super fast, 338x faster than the original approach. Note how it uses Invoke-Expression to actually generate dynamic code on-the-fly, so it is no longer hard-coding the replacement strings. With the list of replacement texts in $t, it is very easy to adjust. So I decided to stick with this one and make it a function:

function Update-Text {
param(
[Parameter(Mandatory=$true)]
$text,

$replacementlist = 'ä,ae,ö,oe,ü,ue,Ä,Ae,Ö,Oe,Ü,Ue,ß,ss'
)
Invoke-Expression ('$text' + -join $(
foreach($e in $replacementlist.Split(',')) {
'.Replace("{0}","{1}")' -f $e, $(
[void]$foreach.MoveNext()
$foreach.Current)
}
)
)
}

PS > Update-Text "This is a test" 'T,t,i,I,te,TE'
thIs Is a TEst

Hope to see you next week,

Tobias

Microsoft MVP PowerShell Germany

P.S.
If you live in Germany or other parts of Europe and your company would like to set up a truly great PowerShell training, just contact me! I regularly train mid- to large-size companies. Trainings are always a blast with tons of real-world-examples and solutions. Here's how to get in touch with me: tobias.weltner@scriptinternals.de