Problem:

Today at work I had a QA guy complain about general performance on some of the Linux boxes I manage.  Specifically, he said that querying the RPM database was slow.  He said it tended to be very slow on the first query, but usually got faster on subsequent attempts.  That was causing our software to crap out (that’s a technical term) because it did not handle the long delays very well.

Solution:

Very understandable problem.  But it’s not that precise.  I needed to quantify it, and I didn’t want to pull out a stopwatch, I wanted to really time it.  I also wanted to do this across many systems (we have a bunch), and well–I’m a Windows guy at heart, so I want to do it from my XP laptop–not from a Linux box.  Some of my co-workers are more on the Unix/Linux side and they would do this differently I’m sure.

Anybody who works with Unix/Linux systems in a cross-platform environment probably knows about the Putty utility and its related tools.  Putty, or more accurately, PuTTY, has been around for quite a while.  It’s a graphical SSH client for Windows, and a very capable one at that.  One of the perhaps lesser-known of its related tools is plink.  Plink is a command-line and quite scriptable version of Putty.  I use it a lot, and I’ve mentioned it on my blog before; most recently in the Get-hVm.ps1 script.

My solution for this morning’s problem involved two functions.  The first is named Invoke-QaSshCmd, and it’s a very simple wrapper around plink.  It takes a computer name and a command to execute as input, and the output is whatever plink decides to output.  As an added tweak, I plugged my Select-Alive function into it so that we don’t waste time trying to work on down systems (or invalid hostnames/IPs).  Also, since this is intended to be used for my strictly controlled lab environment which has a shared user account across all test/dev systems, I embedded those credentials in the script to save typing.  Obviously I would not do the same for production systems for security reasons.  Here’s the function:

   1: function Invoke-QASshCmd {

   2:     Param (

   3:         $Computer,

   4:         $Command

   5:     )

   6:     if ( $Computer | Select-Alive ) {

   7:         plink <my username>@$Computer -pw <my password> $Command

   8:     }

   9:     else { Write-Error "Host not pingable." }

  10: }

With that out of the way, we move on to the meat of the problem–how do I measure the amount of time a command takes?  Luckily, that’s really easy with PowerShell.  There is a cmdlet made just for the task, and it’s called Measure-Command.  Let’s break down the whole thing following the code listing.

   1: filter Measure-RpmStats {

   2:     $out = "" | Select-Object Computer, TimeSec, RpmCount

   3:     $out.Computer = $_

   4:     $out.TimeSec = ( $_ | Measure-Command { $rpms = Invoke-QASshCmd $_ "rpm -qa" } ).TotalSeconds

   5:     $out.RpmCount = ( $rpms | Measure-Object ).Count

   6:     if ( $out.RpmCount -eq 0 ) { $out.rpmCount = "QUERY FAILED" }

   7:     write-output $out

   8: }

First note on line 1 that I’ve made this a filter.  A filter, for those that don’t know, is a shortcut for a certain type of function.  Specifically, it’s just the PROCESS {} code block of a function.  This code block, which is optional to use in functions, is special in that you have really easy access to the PowerShell pipeline.  Functions which use a PROCESS code block, and filters (just a shortcut to the same thing), are able to access the pipeline using the $_ automatic variable.  Also, when used this way, you do not block the pipeline.  In other words, items in the pipeline will travel through it individually, get processed by each statement in turn, and then this loop control moves on to the next item, processes it to completion, and so on.  Other types of loops often block the pipeline, which means you have to sort of stack your objects into a pile, do one thing on every item in the stack–and then pass those results in a lump on to the next thing in the pipeline.

Anyway, on to line 2.  Here I use the Select-Object shortcut (thanks as always to BSonPosh for it) to create a new custom object with three properties.  Then I start to assign stuff to those properties.

Line 4 is where I get the measuring done.  What I do here is take whatever is in the pipeline (which is supposed to be a hostname or IP), and pipe it to Measure-Command.  That cmdlet takes as its parameter a codeblock object which can be any bit of code you want.  I’m using my Invoke-QaSshCmd function to have plink run the “rpm -qa” command on the computer in question.  This will query the software database on the remote box, and it’ll spit out a list of all the RPM packages installed on it.  Once the execution of the scriptblock is complete, Measure-Command returns an object with several properties.  I chose to work with the TotalSeconds one for my needs, and it gets assigned to my custom object.

You may have noticed that in line 4 I assigned the results of the Invoke-QaSshCmd function to the variable $rpms.  In line 5, I use the Measure-Object cmdlet to give me the linecount, which here corresponds to the number of RPMs installed on the destination system, also a handy item to measure.  If that number is zero, I assume something went wrong, and line 6 writes that to the Count property.  Lastly, we write the custom object to “stdout”, in Linux parlance.  Here’s what it looks like:

Computer                      TimeSec     RpmCount
--------                      -------     --------
anaconda.mydomain.blah    11.7584265          601
blackwidow.mydomain.blah    3.487206          516
bigfury.mydomain.blah      3.6997333          559
boss.mydomain.blah        15.7640088          570
coaster.mydomain.blah      4.9751851          787
dodonpa.mydomain.blah      7.6276085          789
hydra.mydomain.blah        3.9643745         1085
jaguar.mydomain.blah        84.32717          939
panther.mydomain.blah    127.9514488         1012
cougar.mydomain.blah      86.3153758         1030
bearcat.mydomain.blah    193.9079667         1176
tigercat.mydomain.blah   102.8501097          765
tigereye.mydomain.blah    510.413652          769
snarf.mydomain.blah        1.1505414          516

Long delays, indeed. 

I hope the analysis of my problem and the solution using PowerShell will help you with your own scripting.  Good luck!

(Updated @ 12:29 to include the output.)