Friday, September 18, 2015

Great balls o' fire...

For anyone reading this that's looking for updated information on our situation, everything is good.

For those curious, I live in the Sierra foothills in California.  The Butte fire which started a week ago just ravaged my area.  70,000 acres burned, 360 homes destroyed, 2 fatalities.

The fire stopped about 500 yards from my home.  We were EXTREMELY lucky.  Seems to be a freak effect of the area we live in that caused the fire to shift and detour around our street.  All the houses on our street are fine, no damage, and just across the creek are foundations with no houses.  Damn scary.

The roads are still closed due to downed power poles and lines.  I understand the need to make things safe, and I am amazed at the herculean effect PG&E is taking to get power back in place,  but I am really getting tired of people telling that "the roads are still closed" just to watch the "special" people drive right down them.  Those would include anyone with any sort of "official" looking vehicle, regardless of who they may be.  And no, that doesn't include us.

Many thanks to the hundreds of fire fighters who focused on saving homes.  They did a phenomenal job and many homes were saved, mine among them.  Tragically two hold-outs lost their lives when they refused to leave when evacuations were called.  I do promise to not complain for the next year about the "special fire tax" being imposed on us due to gross mismanagement at CalFire, or state government, or whoever screwed that up, but after that I reserve the right to complain again.

Here some pics I took to give you an idea.  I left them full size so they are pretty big.

These are views from the house we stayed at while evacuated.  We got evacuated from there twice as well.






Yesterday I was able to get back home for the first time via some nasty logging roads to get the fridge cleaned out and disinfected.  Smelled pretty nasty.  The main road is still closed to it will be hit and miss on getting normal access.  For now looks like the 1.5 hour trip down the logging road.  That road is actually pretty scary.  There are some compounds back there miles from anything with high chain link fences with razor wire.  Every time I drive there I hear the banjos from Deliverance playing in my head...

I took this one from my porch.  You can see how close the fire got through the trees.  The hillside is all burned as well as the trees across the creek.



Tuesday, September 1, 2015

VMware Datastore Utilization Tracker

I like to keep an arsenal of automated scripts that run as scheduled tasks to report on the status of various things I manage.  Here is one of those.

Our virtualization environment uses a SAN backend as most do.  We run exclusively in NFS (NAS) mode.  This has numerous advantages (an explanation of which is outside the scope of this post).  I like to keep a rough idea of the gains and losses of each NFS share on a daily basis.

I run the following script automatically each day and it emails me the details of the gains or losses due to systems being added or removed.  I included a "noupdate" option that allows testing without processing the files.  That way you can rerun using the current file again and again for testing.

Results are very basic and color coded for gain/loss.  The raw dump files are stored in the script folder.  I have another process I use to import them into Excel and graph them over time.  If I get a chance I'll post that as well.

NOTE: This NOT the current version.  I am keeping current versions on the MS PowerShell Gallery at: https://www.powershellgallery.com/profiles/Kcmjr/.

<#======================================================================================
         File Name : Datastore-Tracker.ps1
   Original Author : Kenneth C. Mazie (kcmjr AT kcmjr DOT com)
                   :
       Description : Tracks SAN datastores over time.  Emails a daily report on changes.
                   :
             Notes : Normal operation is with no command line options.  Basic logs are written to C:\Scripts
                   : Optional argument: -Console $true (defaults to false)
                   :                    -NoUpdate $true (runs with current files and doesnt replace them for debugging)
                   :
          Warnings : The script is coded to switch colors between 2 vCenters.  Adjust as
                   : needed below where commented if you use more or less.
                   :  
             Legal : Public Domain. Modify and redistribute freely. No rights reserved.
                   : SCRIPT PROVIDED "AS IS" WITHOUT WARRANTIES OR GUARANTEES OF
                   : ANY KIND. USE AT YOUR OWN RISK. NO TECHNICAL SUPPORT PROVIDED.
                   :
           Credits : Code snippets and/or ideas came from many sources including but
                   :   not limited to the following:
                   : Based on "Track Datastore Space script" Created by Hugo Peeters of www.peetersonline.nl
                   :
    Last Update by : Kenneth C. Mazie
   Version History : v1.0 - 09-16-14 - Original
    Change History : v1.1 - 08-28-15 - Edited to allow color coding of HTML output    
                   :
#=======================================================================================#>
#requires -version 3.0

Param(
     [bool]$Console = $False,
     [bool]$NoUpdate = $False
     )
If ($Console){$Script:Console = $true}

#--[ Uncomment to programatically hide the console ]--
#Add-Type -Name Window -Namespace Console -MemberDefinition '
#[DllImport("Kernel32.dll")]
#public static extern IntPtr GetConsoleWindow();
#
#[DllImport("user32.dll")]
#public static extern bool ShowWindow(IntPtr hWnd, Int32 nCmdShow);
#'
#$Script:consolePtr = [Console.Window]::GetConsoleWindow()
#[Console.Window]::ShowWindow($Script:consolePtr, 0)

Clear-host
$Script:Body = ""  
$ErrorActionPreference = "silentlycontinue"

#--[ Excel Non-Interactive Fix ]------------------------------------------------
If (!(Test-path -Path "C:\Windows\System32\config\systemprofile\Desktop")){New-Item -Type Directory -Name "C:\Windows\System32\config\systemprofile\Desktop"}
If (!(Test-path -Path "C:\Windows\SysWOW64\config\systemprofile\Desktop")){New-Item -Type Directory -Name "C:\Windows\SysWOW64\config\systemprofile\Desktop"}
#--[ Excel will crash when run non-interactively via a scheduled task if these folders don't exist ]--

#-------------------------------------------------------------------------------
#--[ NOTE: The script is set up for two vCenters.  Adjust the coloring below if you have more or less ]--
$Script:vCenters = @("vCenter1","vCenter2")  
#-------------------------------------------------------------------------------
If ($Script:NoUpdate){
       $Script:CurrentFile = $Script:PSScriptRoot+'\Temp_Current.xml'
       $Script:PreviousFile = $Script:PSScriptRoot+'\Datastores_Previous.xml'
       $Script:DifferenceFile = $Script:PSScriptRoot+'\Temp_Difference.txt'
}Else{
       $Script:CurrentFile = $Script:PSScriptRoot+'\Datastores_Current.xml'
       $Script:PreviousFile = $Script:PSScriptRoot+'\Datastores_Previous.xml'
       $Script:DifferenceFile = $Script:PSScriptRoot+'\Datastores_Difference.txt'
}
If (Test-Path $Script:DifferenceFile) {rm $Script:DifferenceFile -Confirm:$false -Force}
If (!(Get-PSSnapin | ? {$_.name -like “vmware.vimautomation.core”})) {Add-PSSnapin vmware.vimautomation.core}
$Script:Digits = 2
$Script:From = "DailyRepoprts@mydomain.com"                                     #--[ Who the email is being sent by ]--
$Script:UserName = "mydomain\username"                                          #--[ The user to connect to vCenter with ]--
$Script:Password = "userpassword"                                               #--[ That users password ]--
$Script:EmailTargets = "user1.domain.com, user2.domain.com"                     #--[ a comma seperated list of email recipients ]--
$Script:SMTPServer = "HostName.domain.int"                                      #--[ what SMTP server to use ]--

Function SendEmail ($Script:OutBody){                                           #--[ Email settings ]--
       $Script:HTML = $true
       $Script:Email = new-object System.Net.Mail.MailMessage
       $Script:Email.From = $Script:From
       $Script:Email.To.Add($Script:EmailTargets)
       $Script:Email.Subject = "SAN Daily Utilization Status"
       $Script:Email.IsBodyHtml = $true
       $Script:Email.Body = $Script:ReportBody
       $Script:smtp = new-object System.Net.Mail.SmtpClient($Script:SMTPServer)
       $Script:smtp.Send($Script:Email)
}

#--[ Add header to html log file ]----------------------------------------------
$Script:ReportBody = @()
$Script:ReportBody += '
<style type="text/css">
       table.myTable { border:5px solid black;border-collapse:collapse; }
    table.myTable td { border:2px solid black;padding:5px}
       table.myTable th { border:2px solid black;padding:5px;background: #949494 }
       table.bottomBorder { border-collapse:collapse; }
       table.bottomBorder td, table.bottomBorder th { border-bottom:1px dotted black;padding:5px; }
</style>
<left><h1>&nbsp;&nbsp;&nbsp;- SAN Volume Utilization Report -</h1></left>
The following report displays the current SAN volume usage and the percent of<br>
change from yesterdays usage.  The raw data files are retained with the script<br>
for use in long term utilization tracking purposes.<br><br>'
$Script:ReportBody += '<table class="myTable"><tr><th>vCenter</th><th>Volume Name</th><th>Percent Free</th><th>Difference</th></tr>'

If (!(Test-Path "$PSScriptRoot\creds.txt")){
  $Script:Credential = Get-Credential
  $Script:Credential.Password | ConvertFrom-SecureString | Set-Content "$PSScriptRoot\creds.txt"
}

#--[ Use if you need to use a credential file, otherwise current creds are used ]--
#$Encrypted = Get-Content "$Folder\creds.txt" | ConvertTo-SecureString
#$Credential = New-Object System.Management.Automation.PsCredential($Username, $Encrypted)
#$LogFileName = "SAN-Vol_{0:MM-dd-yyyy_mm}_Stats.log" -f (Get-Date)
$Script:myColCurrent = @()                                                       #--[ Create an array to hold the output ]--

ForEach ($Script:VIServer in $Script:vCenters){
       If ($Script:Console){Write-Host "`r`nProcessing: $Script:VIServer" -ForegroundColor Cyan}
       $Script:VC = Connect-VIServer -Server $Script:VIServer -user $Script:UserName -pass $Script:Password #-Credential $Script:Credential #--[ Connect to Virtual Center ]--
       $Script:datastores = Get-Datastore | Sort-Object Name | Select -Unique            #--[ Get all datastores and put them in alphabetical order & remove accidental duplicates ]--
       ForEach ($Script:store in $Script:datastores)                                     #--[ Loop through datastores ]--
              if ($Script:store -notlike "*Local*"){
                     $Script:myObj = "" | Select-Object vCenter, Name, CapacityGB, UsedGB, PercFree              #--[ Create a custom object and define its properties ]--
                     #--[ Set the values of each property ]--
                     $Script:myObj.vCenter = $Script:VIServer
                     $Script:myObj.Name = $Script:store.name
                     $Script:myObj.CapacityGB = [math]::Round($Script:store.capacityMB/1024,$Script:digits)
                     $Script:myObj.UsedGB = [math]::Round(($Script:store.CapacityMB - $Script:store.FreeSpaceMB)/1024,$Script:digits)
                     $Script:myObj.PercFree = [math]::Round(100*$Script:store.FreeSpaceMB/$Script:store.CapacityMB,$Script:digits)
                     $Script:myColCurrent += $Script:myObj   
                     #--[ Add the object to the output array ]--    
              }
       }
       Disconnect-VIServer -Confirm:$False                                      #--[ Disconnect from Virtual Center ]--
}

$Script:myColCurrent | Export-Clixml -Path $Script:CurrentFile                  #--[ Export the output to an xml file; the new Current file ]--

$Script:CurrentDate = (Get-Item $Script:CurrentFile).LastWriteTime              #--[ Get file dates for new file names ]--
$Script:PreviousDate = (Get-Item $Script:PreviousFile).LastWriteTime

#--[Compare the Current information to that in the Previous file ]--------------
If (Test-Path $Script:PreviousFile){                                            #--[ Check if a Previous file exists ]--
       $Script:myColPrevious = Import-Clixml $Script:PreviousFile               #--[ Import the Previous file ]--
       $Script:myColCurrent= Import-Clixml $Script:CurrentFile                  #--[ Import the Previous file ]--

       $Script:myColDiff = @()                                                                                   #--[ Create an array to hold the differences ]--
       ForEach ($Script:myObjCurrent in $Script:myColCurrent){                  #--[ Loop through the current datastores ]--
           $Script:VCCurrent = $Script:myObjCurrent.vCenter
              $Script:RowData = ""
              $Script:diff = Compare-Object ($Script:myColPrevious | Where { $_.Name -eq $Script:myObjCurrent.Name }) $Script:myObjCurrent -Property PercFree  # The actual compare command
              $Script:myObjDiff = "" | Select-Object vCenter, VolName, PercentFree, Diff #--[ Create a custom object and properties for outputting results ]--
              $Script:myObjDiff.vCenter = $Script:myObjCurrent.vCenter                          #--[ Setting the values of each property ]--
              $Script:myObjDiff.VolName = $Script:myObjCurrent.Name                            
              $Script:myObjDiff.PercentFree = $Script:myObjCurrent.PercFree
                    
              #--[ The most important property is the calculated difference between the current and previous values of PercFree. You can substitute it for UsedGB if you like. ]--
              $Script:myObjDiff.Diff = ($Script:diff | Where { $_.SideIndicator -eq '=>' }).PercFree - ($Script:diff | Where { $_.SideIndicator -eq '<=' }).PercFree
                    
              If (($Script:myObjDiff.Diff -eq "") -or ($Script:myObjDiff.Diff -eq $null)){$Script:myObjDiff.Diff = "0.00"}
              $Script:myColDiff += $Script:myObjDiff                                                   #--[ Adding it to the output array ]--

              $Script:BGColor = "#dfdfdf"                                         #--[ Grey cell background ]--
              $Script:RowData += '<tr>'
                          
              if($Script:VCCurrent -eq "vcenter1"){                               #--[ NOTE: Tweak this name to keep colors rotating ]--
                     $Script:FGColor = "#408080"                                  #--[ Color to distinguish 1st vCenter ]--
              }ElseIf($Script:VCCurrent -eq "vcenter2"){                          #--[ NOTE: Tweak this name to keep colors rotating ]--
                     $Script:FGColor = "#808000"                                  #--[ Color to distinguish 2nd vCenter ]--
              }Else{
                     $Script:FGColor = "#000000"                                  #--[ Default color, black ]--
                     }
             
              $Script:RowData += '<td bgcolor=' + $Script:BGColor + '><font color=' + $Script:FGColor + '>' + $Script:myObjDiff.vCenter + '</td>'           

              $Script:FGColor = "#000000"
              $Script:RowData += '<td bgcolor=' + $Script:BGColor + '><font color=' + $Script:FGColor + '>' + $Script:myObjDiff.VolName + '</td>'           
              $Script:RowData += '<td bgcolor=' + $Script:BGColor + '><font color=' + $Script:FGColor + '>' + $Script:myObjDiff.PercentFree + '</td>'       
              If ($Script:myObjDiff.Diff -eq "0.00"){
                     $Script:FGColor = "#000000"                                   #--[ Black foreground for no change ]--
                     $Script:RowData += '<td bgcolor=' + $Script:BGColor + '><font color=' + $Script:FGColor + '>' + $Script:myObjDiff.Diff
              }ElseIf ($Script:myObjDiff.Diff -lt 0){
                     $Script:FGColor = "#700000"                                   #--[ Red foreground for storage loss]--
                     $Script:RowData += '<td bgcolor=' + $Script:BGColor + '><font color=' + $Script:FGColor + '><strong>' + $Script:myObjDiff.Diff + '</strong>'
              }Else
                     $Script:FGColor = "#007000"                                   #--[ Green foreground for storage gain ]--
                     $Script:RowData += '<td bgcolor=' + $Script:BGColor + '><font color=' + $Script:FGColor + '><strong>' + $Script:myObjDiff.Diff + '</strong>'
              }
                          
              $Script:RowData += '</td></tr>'
              $Script:ReportBody += $Script:RowData
              Clear-Variable diff -ErrorAction "SilentlyContinue"                                      #--[ Clearing the variable used inside the loop to prevent incorrect output in case of problems setting the variable! ]--
       }
       If ($Script:myColDiff.Length -eq 0){$Script:myColDiff = "No changes since last check."#--[ If nothing changed, we don't want an empty file. ]--
       $Script:myColDiff | Format-Table -AutoSize | Out-File $Script:DifferenceFile -Force -Append         #--[ And we conclude by outputting the results to a text file, which can be emailed or printed. ]--
}

If ($Script:myColDiff.Length -ne 0){
       (Get-Content $Script:DifferenceFile) | Where { $_ } | Set-Content $Script:DifferenceFile
       (Get-Content $Script:DifferenceFile) | Where {$_ -notmatch '----'} | Set-Content $Script:DifferenceFile
       Add-Content $Script:DifferenceFile –value "`nCurrent-Report $Script:CurrentDate &nbsp; &nbsp;`nPrevious-Report $Script:PreviousDate &nbsp; &nbsp;"
       $Script:FGColor = "#000000"
       $Script:BGColor = "#bbbbbb"
       $Script:RowData += '<tr>'
       $Script:RowData += '<td bgcolor=' + $Script:BGColor + '><font color=' + $Script:FGColor + '>Current Report</td>'    
       $Script:RowData += '<td bgcolor=' + $Script:BGColor + '><font color=' + $Script:FGColor + '>' + $Script:CurrentDate + '</td>'  
       $Script:RowData += '<td bgcolor=' + $Script:BGColor + '><font color=' + $Script:FGColor + '>Previous Report</td>'   
       $Script:RowData += '<td bgcolor=' + $Script:BGColor + '><font color=' + $Script:FGColor + '>' + $Script:PreviousDate + '</td>' 
       $Script:RowData += '</tr>'
       $Script:ReportBody += $Script:RowData
}

$Script:ReportBody += '</table><br><br>'
<#--[ Enable this section to add notes to the bottom of the report. ]--
$Script:ReportBody += '
<ul>
    <li>Only powered-on Vms are shown since the script relies on WMI to get the block size.</li>
       <li>"Vol", or "Volume" refers to a Windows disk partition.</li>
       <li>The hidden system partition, which is typically Vol 0 on Disk 0, is not shown.</li>
       <li>Disk numbers are assigned beginning with 0 (zero).</li>
       <li>Partition numbers are assigned beginning with 1 (not 0).</li>
</ul>'
#>
$Script:OutBody = Get-Content $Script:DifferenceFile #| Out-String
SendEmail $Script:OutBody

If ($Script:Console){Get-Content $Script:DifferenceFile | Out-String}

#--[ Cleanup ]--
$Script:attachment.Dispose()
$Script:msg.Attachments.Dispose()
$Script:msg.Dispose()
$Script:smtp.Dispose()
If ($Script:NoUpdate){
       If (Test-Path $Script:DifferenceFile){Remove-Item -Path $Script:DifferenceFile -force}
       If (Test-Path $Script:CurrentFile){Remove-Item -Path $Script:CurrentFile -force}
}Else{
       If (Test-Path $Script:DifferenceFile){rename-Item -Path $Script:DifferenceFile -newname ("$PSScriptRoot\SAN-Vol_{0:MM-dd-yyyy}_Diff.log" -f (Get-Date))}
       If (Test-Path $Script:PreviousFile){Remove-Item -Path $Script:PreviousFile -Force} #If a Previous file exists remove it
       If (Test-Path $Script:CurrentFile){Copy-Item -Path $Script:CurrentFile -Destination ("$PSScriptRoot\SAN-Vol_{0:MM-dd-yyyy}_Stats.xml" -f (Get-Date))}
       If (Test-Path $Script:CurrentFile){Rename-Item -Path $Script:CurrentFile -NewName $Script:PreviousFile}                    # If a Current file exists, rename this Current file to Previous
}
$Script:Credential = ""
$Script:CurrentFile = ""
$Script:PreviousFile = ""
$Script:DifferenceFile = ""
$Script:OutBody = ""

If ($Script:Console){Write-Host '--- Completed ---'}

#iex '$PSScriptRoot\reload.bat'  #--[ Used for input file refresh while debugging ]--