Category Archives: Reference

Powershell – ForEach-Parallel

Posted by on 2019/11/13 No comments

Background

In PowerShell I often find the need to speed things up when having to do querying lots of similar things – usually getting the same stuff from multiple machines. So I ventured out onto the Interwebs and over time slap together a PowerShell module that does parallel processing. I’m not claiming this to be purely my work – I only added and refined it a bit for my needs.

This example works with ‘older’ versions of PowerShell (think version 4/5 and perhaps 3). The latest versions of PowerShell (afaik version 7) now has something like this built in.

Implementation

Using any sort of parallel coding requires a different approach than normal procedural (Start at the top and run code line by line until you get to the end). Basically it is multi-threading but PowerShell ‘out of the box’ has nothing like it. To implement multi threading you need to use ‘runspaces’ which effectively means you are running multiple instances of PowerShell.

The ForEach-Parallel function takes a scriptblock as input. Optionally you can also specify the number of threads used to do the parallel processing.

#ForEach-Parallel.psm1
function ForEach-Parallel {
     param(
         [Parameter(Mandatory=$true,position=0)]
         [System.Management.Automation.ScriptBlock] $ScriptBlock,
         [Parameter(Mandatory=$true,ValueFromPipeline=$true)]
         [PSObject]$InputObject,
         [Parameter(Mandatory=$false)]
         [int]$MaxThreads=5
     )
     BEGIN {
         $iss = [system.management.automation.runspaces.initialsessionstate]::CreateDefault()
         $pool = [Runspacefactory]::CreateRunspacePool(1, $maxthreads, $iss, $host)
         $pool.open()
         $threads = @()
         $ScriptBlock = $ExecutionContext.InvokeCommand.NewScriptBlock("param($_)r`n" + $Scriptblock.ToString())
     }
     PROCESS {
         $powershell = [powershell]::Create().addscript($scriptblock).addargument($InputObject)
         $powershell.runspacepool=$pool
         $threads+= @{
             instance = $powershell
             handle = $powershell.begininvoke()
         }
     }
     END {
         $notdone = $true
         while ($notdone) {
             $notdone = $false
             for ($i=0; $i -lt $threads.count; $i++) {
                 $thread = $threads[$i]
                 if ($thread) {
                     if ($thread.handle.iscompleted) {
                         $thread.instance.endinvoke($thread.handle)
                         $thread.instance.dispose()
                         $threads[$i] = $null
                     }
                     else {
                         $notdone = $true
                     }
                 }
             }
         }
     }
 }
 export-modulemember -function ForEach-Parallel

To make use of this ‘module’ save it to a file named ForEach-Parallel.psm1 and place it in C:\Users\{your user account}\Documents\WindowsPowerShell\modules\ForEach-Parallel

Example

#Example
$computers = 'computer1','computer2','computer3'
 $computers | ForEach-Parallel -MaxThreads 20 -ScriptBlock {
     $computerToPing = $_
     [int]$ResponseTime = 0
     [string]$ResolvedName = ''
     $StepPing = (Test-Connection -ComputerName $computerToPing -Count 1 -ErrorAction SilentlyContinue)
   if ($StepPing -ne $null){
      $ResponseTime = $StepPing.ResponseTime 
   } 
   New-Object psobject -Property @{     
    Name               = $computerToPing     
    ResponseTime       = $ResponseTime                  
   }
 }

Summary

There are a few things to watch out for – like any multi-threading coding. While the code is running there is no indication how far each ‘thread’ is – you have to wait until all of them are done. Threads are independent so they cannot reference each other or even variables in the calling thread.

So.. happy multi-threading.

SSH on Windows 10

Posted by on 2019/08/07 No comments

I remember that they (Microsoft) mentioned the bash shell on Windows 10 a while ago and other Linux/Ubuntu integration stuff but at the time it seemed to be just a gimmick.

But since I’m looking into Nagios and some Windows monitoring I now the need to daily connect to my Nagios server making configuration changes. So far I’ve been using good old Putty/WinSCP and it works well. Then I started looking at a way to generate configurations files for some Windows machines and upload them automatically via PowerShell (or it could end up being another tool created in C# eventually). This is how I stumble across the Windows 10 built in SSH functionality – it’s been there for at least a year! (since build 1803).

That means you can simply use the ssh command in a Windows command shell (or PowerShell) and send commands to your Linux server. To start it simple open a command prompt (cmd.exe) and run ssh:

As can be seen it supports all the basics of any ssh client.

So that cover the basics. The real power comes when you start mixing it with PowerShell and proper scripting and/or batch files.

One very interesting (and incredibly useful) example is something Scott Hanselman posted – how to automatically log into a remote linux machine – see this.

My quick Nagios client install guide

Posted by on 2019/04/28 No comments

Due to changing winds at my day job I’ve been tasked to start looking into Nagios for monitoring a Windows environment. I’ve taken this as a learning opportunity and also to compare monitoring systems (aka SCOM). This blog post is not about comparing Nagios with SCOM or any other monitoring system but rather a summary of the process of installing Nagios client (NSClient++ on Windows and nrpe on Linux). Installing the server portion of Nagios is an affair for another day – as it is a rather hairy story only for the brave…

Installing a Linux client

Since I have a mixed test environment (at home) I also made use of Nagios to monitor them too. I mostly use Ubuntu/Debian based distros (like Ubuntu and Mint) so this guide will only cover them.
1. First make sure no other updates are outstanding:
sudo apt-get update
2. Then download/install the nrpe package (which includes both server and plug-in parts).
sudo apt-get install nagios-nrpe-server nagios-plugins
3. Once installed you need to configure the client so it can act as a monitoring end-point for the Nagios server. The bare minimum you have to specify is to allow the server to connect to the client. To do this you need to edit the nrpe.cfg file. By default it should be installed at /etc/nagios
sudo gedit /etc/nagios/nrpe.cfg
then search for the line that starts with allowed_hosts= and set it to the ip address for the Nagios server.
allowed_hosts=<ip address of nagios server>
4. Lastly you need to restart the agent:
sudo /etc/init.d/nagios-nrpe-server restart
Once all is done you (or the Nagios admin) need to configure the server portion. This will be covered later as it is basically the same for Linux and/or Windows clients.

Installing a Windows client

For Windows monitoring I’m using the NSClient++ agent. It is available for most Windows versions – from Windows XP to Windows 10 as well as Windows server.
1. Download the appropriate MSI version for your version of Windows from
https://nsclient.org/download/
2. Install the MSI. You can do it manually – double-click the MSI or using the command line
3. I tend to choose the the Complete option to choose
Enable common checkplugins
Enable nsclient server (check_nt)
Enable NRPE server (check_nrpe)’ with Safe mode (default)
Enable NSCA client (optional)
4. If for some reason you want to use the command line you can use something like this (which is for a silent install) – this is for the current latest x64 version 0.5.2.35-x64.
msiexec /qn /l* NSCPInstall.log /i NSCP-0.5.2.35-x64.msi /norestart CONF_CAN_CHANGE=1 CONF_NSCLIENT=1 CONF_NRPE=1 CONF_NSCA=1 ADDLOCAL=ALL

Configuring the server

This quick guide is not intended as complete reference for configuring Nagios so I’ll only be giving a summary here. By default Nagios config is found at /usr/local/nagios/etc (for version 5.x afaik).
I tend to put host configs (aka clients) in a separate directory ‘/usr/local/nagios/etc/servers’ – it just makes it easier to manage. For this quick guide I simply show how to add the ‘host’ entry and not how to configure services and any other things.
1. Create a new file for the the client/agent. I use the name ‘hostname.cfg’
2. Edit the file
example config here for a linux host
define host {
use linux-server
host_name <hostname>
alias <hostname>
address <x.x.x.x or dns name>
}
Windows host
define host {
use windows-server
host_name <hostname>
alias <hostname>
address <x.x.x.x or dns name>
}
3. After all config editing has been done I always check that changed config is okay (for typos and so on…) using the following command on the Nagios server
/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg
4. If all checks out you can restart the Nagios server
systemctl restart nagios.service

Summary
For now this is the quick guide. I’ll update it from time to time if I find more things that can be included.

SSH from PowerShell

Posted by on 2016/11/14 No comments

I’ve been playing with more non Microsoft technologies recently and even have a few Linux VMs running here and there. In the process I’ve learned a lot about interacting between the two worlds/environments using technologies like SSH and even running the Powershell Alpha previews on my Linux VMs. Then I got ‘bored’ and looked for a way to programmatically interact with my VMs (for monitoring) and discovered the open source project SSH.Net which I now have incorporated into even QuickMon. Then I got even more bored and started looking for a way to use SSH from Powershell and came across the SSH-Sessions module which incidentally also use SSH.Net to give Powershell a way to call/connect using SSH.

No PassPhrase

This all works nicely except… SSH-Sessions had one shortcoming.. which I just had to fix. The creator(s) of this module implements a way to specify a key file when connecting to an SSH server but does not provide a way to specify the ‘PassPhrase’ at all! WHY?

Any way, I knew the SSH.Net library does support it since I actually make use of this functionality in QuickMon. Then I started digging inside the SSH-Session script files and made a few adjustments…

Fix

To enable PassPhrase functionality I simply had to add the following code in the ‘New-SshSession’ function:

Change the function header to:

function New-SshSession {
    param([Parameter(Mandatory=$true)][string[]] $ComputerName,
          [Parameter(Mandatory=$true)][string]   $Username,
          [string] $KeyFile = '',
          [string] $PassPhrase = 'blankPassphrase',
          [string] $Password = 'SvendsenTechDefault', # I guess allowing for a blank password is "wise"...          
          [int] $Port = 22,
          [switch] $Quiet
    )

and then inside the function:

if ($KeyFile -ne '') {
        if (-not $Quiet) {
            "Key file specified. Will override password. Trying to read key file..."
        }
        if ($PassPhrase -eq 'blankPassphrase') {
            $SecurePassPhrase = Read-Host -AsSecureString "key provided. Please enter pass phrase for $KeyFile"
            $PassPhrase = ConvertFrom-SecureToPlain $SecurePassPhrase
        }
        if (Test-Path -PathType Leaf -Path $Keyfile) {
            $Key = New-Object Renci.SshNet.PrivateKeyFile( $Keyfile, $PassPhrase ) -ErrorAction Stop
        }
        else {
            "Specified keyfile does not exist: '$KeyFile'."
            return
        }   
    }

Example

And with the you can now use the module to connect to SSH using a Key file that has a PassPhrase. If you don’t specify the PassPhrase the script will prompt you for one. If you actually don’t have a PassPhrase at all then pass an empty string as the value of PassPhrase (not tested but it should work).

Import-Module SSH-Sessions
New-SshSession -Computer mySSHServer -Username me  -KeyFile 'c:\mykeys\mykey.key' -PassPhrase 'somePhrase'
Invoke-SshCommand  -ComputerName mySSHServer -Command "cat /proc/cpuinfo | grep processor" -Quiet
Remove-SshSession -RemoveAll -Quiet

How to install Powershell on Ubuntu 16.10

Posted by on 2016/11/05 1 comment

I recently tried installing the latest Alpha of Powershell (Alpha 12) on the newly released Ubuntu 16.10 and discovered that the install failed because of a dependency on the library libicu55 which has been replaced in the new version of Ubuntu (libicu57). Actually, the whole .Net Core install fails because of this.

Fortunately this is easy to fix by manually installing the old library from http://cz.archive.ubuntu.com/ubuntu/pool/main/i/icu/libicu55_55.1-7_amd64.deb

Then you can download the latest Powershell version (Alpha 12 as of this writing) from https://github.com/PowerShell/PowerShell/releases

Hopefully the creators of the Powershell packages will release an updated version that is natively compatible with Ubuntu16.10.

 

SCOM agent Info script

Posted by on 2016/09/07 No comments

When you have a case where you start integrating your in house SCOM 2012 R2 environment with VMs that are hosted in Azure and ‘other’ people keep on screwing up your SCOM agents with the OMS version you need a way to track what version is actually installed on a machine – plus whether the original SCOM 2012 R2 config is still in tact – which is  not the case since the upgrade process from SCOM 2012 R2 agent to the OMS version totally screws up the SCOM 2012 R2 config (like in F*ing deletes it !!!)

Therefore I had to create a little Powershell script to help check suspected machines if they have been affected by this issue. From the SCOM console side it just appears the Agent is ‘dead’ even though if you check manually it is still running (but of course the config is gone).

if ($args.count -ne 0) {
    $ComputerName = $args[0]    
}
else {
    $ComputerName = Read-Host -Prompt 'Computer Name'
}

$pingTest = (Test-Connection $ComputerName -Count 1 -TTL 255 -ErrorAction SilentlyContinue) 
if ($pingTest -ne $null -and $pingTest.IPV4Address -ne $null){
    write-host "IP Address : " $pingTest.IPV4Address.IPAddressToString -foregroundcolor "green"
    $serviceDetails = Get-Service -ComputerName $ComputerName | where DisplayName -Like 'Microsoft Monitoring Agent'
    if ($serviceDetails -ne $null ) {
        write-host "Health Service state: " $serviceDetails.Status.ToString() -foregroundcolor "green"

        $Reg = [Microsoft.Win32.RegistryKey]::OpenRemoteBaseKey('LocalMachine', $ComputerName)
        $AgentGroupKeys = $Reg.OpenSubKey("SYSTEM\CurrentControlSet\Services\HealthService\Parameters\Management Groups")
        write-host "Groups: " $AgentGroupKeys.SubKeyCount.ToString() -foregroundcolor "green"
        $AgentGroupKeys.GetSubKeyNames() | % {
            if ($_ -like "AOI*") {
                write-host " OMS" -foregroundcolor "yellow"
            } else {
                write-host " $_" -foregroundcolor "green"
            }            
        }        
        
        $VersionNo = "NOT found!"
        $RegKeyProducts = $Reg.OpenSubKey("SOFTWARE\Classes\Installer\Products")
        $RegKeyProducts.GetSubKeyNames() | % {
            $RegKeyProduct = $Reg.OpenSubKey("SOFTWARE\Classes\Installer\Products\" + $_)
            if ($RegKeyProduct.GetValue('ProductName') -eq "Microsoft Monitoring Agent") {
                $VersionNo = $RegKeyProduct.GetValue('Version').ToString("X")
            }
        }

        if ($VersionNo -ne "NOT found!") {
            write-host "Agent version" $VersionNo -foregroundcolor "green"
        }
        else {
            write-host "Agent version not found!" -foregroundcolor "red"
        }

    }
    else {
        write-host "Health service NOT found!" -foregroundcolor "red"
    }
}
else {
    write-host "$ComputerName is not pingable!" -foregroundcolor "red"
}

This script first checks whether the machine (VM) is pingable, if it is then whether the HealthService service is installed, if so what state it is in and what version it is. It also lists the management groups the agent reports to including OMS.

Have fun kidz…

Hiding passwords in (C# App) console

Posted by on 2016/04/13 No comments

I’m actually a bit surprised that despite .Net/C# being more than 10 years old there is still no native Console.ReadLine() overload that hides the characters you are tying – for when you want to prompt for a password or so. Of course writing your own version to do this is not really difficult but still it would have been nice…

Here is one quick version… in case someone else wants to have one (and is too lazy to create it from scratch) hehe.

 

        private static string GetPwd()
        {
            string pass = "";
            Console.Write("Enter your password: ");
            ConsoleKeyInfo key;

            do
            {
                key = Console.ReadKey(true);

                // Backspace Should Not Work
                if (key.Key != ConsoleKey.Backspace && key.Key != ConsoleKey.Enter)
                {
                    pass += key.KeyChar;
                    Console.Write("*");
                }
                else if (key.Key == ConsoleKey.Backspace) 
                {
                    if (pass.Length > 0)
                        pass = pass.Substring(0, pass.Length - 1);
                    Console.Write("\b \b");
                }
            }            
            // Stops Receving Keys Once Enter is Pressed
            while (key.Key != ConsoleKey.Enter); 
            Console.WriteLine();
            return pass.TrimEnd('\r', '\n'); 
        }

The case of the missing ‘Process’ Performance counter

Posted by on 2016/02/23 1 comment

I recently had the need to do some monitoring of the Explorer.exe process (like one of the Windows 10 Insider Preview bugs) that every so often went bananas (Minions would like it…) for no reason at all. This is a pain if you have other processes on the same machine that is sensitive to things like running in ‘low’ priority (Like Boinc workloads). This effectively block other processes to run or just plain make the whole machine feel slow and sluggish.

Then I discovered the problem that perfmon could not ‘see’ the ‘Process’ category. This is weird since I  thought this would be one of the most basic performance counters that Windows always had and probably still have. Somehow the ‘Process’ category disappeared or became corrupted or something – I thought.

Anyway, after some digging around (aka Google) I found an old article about similar issues people had with older versions of Windows (even server versions). The culprit (at least in this case) was that for some reason the registry key to enable the ‘Process’ category was disabled. I have no idea how that happen (no I deny any involvement whatsoever… 🙂 ).

To fix do this: (standard registry editing disclaimer: do it at your own risk).

  1. Open Regedit
  2. Find the key HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\PerfProc\Performance
  3. Check the value of ‘Disable Performance Counters’. If it is anything but 0 it means it is disabled.
  4. Change the value to 0 and save.
  5. Restart whatever performance counter monitoring tool you are using since it will not be aware of the change until that process restart.
  6. Jump up and down for joy! (just because you can).

 

Visual Studio 2013 and 2015 icons

Posted by on 2015/08/03 No comments

Thanks to VS2015 that happen to have the same icon as VS2013 it can be confusing if you have both installed (side-by-side). So as a simple solution I created my own customized icons based on the standard VS icon that also display the version (shortened). I’m simply sharing it here for other if they like to re-use it.

VS201x

Simply extract somewhere on your local HDD and then customize (create shortcut/pin to taskbar and then properties, Change Icon, browse and select the icon) and whoala!.

Sharing is caring… so they say.

Windows 10 upgrade survival guide

Posted by on 2015/08/03 No comments

So I’ve been through the process to upgrade 3 machines (2 laptops and one desktop so far) to Windows 10 with a 33% success rate… yeah, I know that does not sound that promising. The other 66% I managed to fix with ‘resets’ which is basically a fresh install. In the end that comes to (technically) 99%. There are a few things that isn’t 100%… (small) things that are still causing some headaches – this guide is not about them.

So I decided to create this post to help others with the process. This will not be a all-out guide to upgrading because I’m sure I didn’t experience ALL the troubles you can have while upgrading to Windows 10.

Before the install

First tip I have is to rather use the Windows 10 Media creation tool to download the full ISO ( https://www.microsoft.com/en-us/software-download/windows10 ). The reasons for this are simply because then you have control over the download and if that fails the actual upgrade process does not fail at the same time. Secondly, you can then use the same ISO image to upgrade multiple machines without having to download the same ‘stuff’ multiple times.

Second tip is to prepare your machine for the upgrade by backing up (always a good idea) and cleaning any unused stuff – like uninstall applications which you really don’t use anymore as all applications whether you use it or not has to go through the upgrade process (be part of it) and can increase the duration time of the upgrade. The same applies to any unused hardware and drivers. Just get rid of them before you start. Unplug any external USB flash/hard drives. The Windows 10 upgrade process also checks for compatibility in case you missed something that is not compatible. Also, save of make a copy (or even a screenshot) of any saved passwords (e.g. those that are stored in your browser).

Thirdly it is useful to have a backup of all your drivers (yes, even those that you don’t even have the original disk for anymore!). I use a simple too like DriverBackup! ( http://sourceforge.net/projects/drvback/ ) but you can use any similar tool. In some cases Windows 8.x drivers should be compatible with Windows 10 (but don’t quote me on that). If for some reason Windows 10 did not recognize your hardware after the upgrade you can try to point device manager’s check for updated drivers to this driver backup directory.

4th’ly check for any outstanding Windows updates (yes the upgrade tool can also do this but it is better that you do this manually and you get to choose what/when/how this gets installed).

Lastly uninstall your anti-virus software (yes, that thing that hogs your machine worse than a virus itself). Some (possibly all) anti-virus software have been known to cause problems while doing Windows upgrades. You can re-install your anti-virus software after the upgrade if it is supported (might need to be upgraded to a Windows 10 compatible version in any case).

The Install

All of this and you haven’t even started the upgrade yet! phew… hehe

To start the upgrade process you can either create a DVD from the ISO using you favorite DVD burning tool (even Windows explorer in Windows 7 have an option to burn to DVD) or you can use a tool to open the ISO like a zip file (e.g. 7-zip which is free – http://www.7-zip.org/ ) and extract all the files somewhere on your hard drive. Once this is done simply run the Setup.exe in the root directory of this DVD/extracted directory. At this stage choose the option NOT to ‘also download updates’ (since you already did that during tip number 4).

The rest of the install process is mostly automated – with multiple reboots… crashes… bluescreens… just joking. If it does crash it should roll back and restore the previous version of Windows (this I’ve experienced a few times with the Insider previews so it does work). Just sit back and relax… right…

Post installation

Post the installation you are given some options to choose privacy options. For my taste I choose to disable most of them (like the typing and advertising settings which are enabled by default). Also, the Wireless Sense stuff I also disable (old habits of not trusting others…). When prompted for default apps you can choose to customize and then un-tick the ones presented as replacements. In my opinion the default should be to keep you existing default apps (like browser, music player etc) and have the new Microsoft alternatives as suggestions – not forced down you computing throat.

Wrap up

Ok, so once the setup/installation part is done and you are back at your desktop there are a couple of things to check and make sure all is happy in Windows 10 land. One of the first things I always check after such an upgrade is that all hardware (drivers) are properly installed. You can check this (in Windows 10) by right clicking on the new Start button (yes right-click on the start button actually does something useful – if you come from Windows 7 world) and choose ‘Device Manager’. Check for any yellow/red warning triangles (usually it should be very obvious as the tree nodes will be expanded by default showing the problem components). At this stage you can right click of the problematic component/tree node and choose ‘Update Driver Software’.

Try the online search option first as that ‘should’ get the latest available driver if any exists (and the device has been recognized). If it doesn’t work to ‘fix’ the device then try the same procedure again but this time choose the ‘Browse my computer…’ option and specify the driver backup directory as created in tip no. 3. (or if you happen to have the original disk that came with the device… what?? anyone keep those disks?? hehe). If this does not work try searching the manufacturer’s web site for updated drivers. Chances are that they won’t have Windows 10 drivers yet as it takes time to create and certify those drivers. In some cases Windows 8.1 drives ‘may’ work but… don’t quote me on that either. If all of this does not work then.. it just wasn’t meant to be… for now 🙂

Troubleshooting (Update)

If for some reason after this upgrade you experience problems like hardware devices bot working (even if drivers are installed and seems to be ok) or you get Blue screens of death (yes, they’re still blue) you can try to do a Windows ‘Reset’. This is rather destructive as far as your applications goes but at least your ‘Data’ files stay in tack.

If you can get into Windows (in case you don’t have Blue screen of death on login) you can go to Settings -> Recovery and Choose the ‘Reset this PC’ option. Choose to keep your files… unless you don’t want them anymore…

Alternatively if for some reason can can’t do this from within Windows itself make/obtain a bootable image (DVD or of your machine supports it – USB) and boot up from it. After selecting the languages and stuff on the first screen you will see a ‘Repair’ option on the second screen you get. From there you can select ‘Troubleshooting’ (or something like that since I don’t have that screen open now) and then ‘Reset this PC’. Remember to choose ‘Keep my files’. After that it will start (re)installing just like a full installation, multiple reboots etc. and eventually back at the login screen. At that point you’ll have a fairly ‘blank’ Windows installation with nothing but the default Windows applications installed. Your files will still be there but any applications you use that didn’t came with Windows itself will need to be reinstalled. Sometimes this actually might be a good thing to get rid of all the crap one has installed over the years…

Conclusion

All-in-all I think Windows 10 does have more positives than (drumroll) negatives. There are things I really don’t like about it but then there are some I do (like). Once you get it up and running it does seem to be worth the trouble. Happy Windows 10ing… (but I will miss my Windows 7)