To remove duplicates using PowerShell is exceedingly straightfroward. However, I have a habit of forgetting the cmdlet so parking this here for future reference.
For example, we can use the Get-Unique cmdlet or Select-Object -Unique parameter.
Get-Unique
As a simple example:
Get-Content .\File.txt | Sort-Object | Get-Unique
The Get-Unique cmdlet compares each item in a sorted list to the next item, eliminates duplicates, and returns only one instance of each item. The list must be sorted for the cmdlet to work properly.
Get-Unique is case-sensitive. As a result, strings that differ only in character casing are considered to be unique.
Select-Object-Unique
Get-Process | Select-Object -Unique
This parameter is case-sensitive. As a result, strings that differ only in character casing are considered to be unique
VB Sorting and Get Unique
Trying to do these activities back in the 90s using VBScript was not as simple. To do this with large volumes of data I have bad memories of using client side disconnected record sets.
This could would be something similar to this.
Function SortOutput(strTemp As String, _ rstTemp As Recordset) With rstTemp Debug.Print strTemp Debug.Print " Sort = " & _ IIf(.Sort <> "", .Sort, "[Empty]") .MoveFirst ' Enumerate Recordset. Do While Not .EOF Debug.Print " " & !LastName & _ ", " & !FirstName .MoveNext Loop End With End Function
Other Legacy Commands You Can Now Chuckle At...
Some other perennial favourites include the FOR command to parse out only certain elements in an input.txt file
FOR /F "Tokens=1,4 Delims=+ " %a IN (C:\Temp\Input.txt) DO @ECHO %a %b >> Output.txt
This can now be done so much easier in PowerShell, and so much more...
Get-ChildItem -Recurse -Filter *.log | Get-Content | Where-Object {$_ -Match "Microsoft+Outlook+15.0" -And $_ -Match "Contoso" -And $_ -Match "POST /Autodiscover/autodiscover.xml"} | Out-File C:\Autodiscover.txt
Cheers,
Rhoderick
