The Gears Inside My Head: February 2016

Those of you who are working with Tegile storage systems are very aware that reporting is one of the things that is still lacking. There are a number of canned reports you can get via the GUI, but as far as custom reports or getting data out programatically, good luck.

I'm told by folks in Tegiles' management that they are developing PowerShell commandlets for use with their storage appliances. That's great news. When we get to see them I don't know.

In the interim I needed to get data out of the arrays. I came up with a PowerShell script that will connect to a list of controllers and extract replication data into an Excel spreadsheet. I only care about completed verses failed so the output is limited to that. Actually as it sits it only reports good runs, I'm working on failed runs.

This script will create a spreadsheet with the last 2 months of successful replication events. It can be modified for other logs but this is specifically designed to parse the replication logs. Each target is a new worksheet, and each tab is labeled as such. It's still a work in progress so some variables are incomplete (I like to have them all at the "script:" context...). I'll update this as I get farther along, but it does work as is.

UPDATE: 09-02-16
It's been noted that this script doesn't catch failed jobs. It's very targeted towards completed jobs. I have retooled it to (hopefully) catch the failed jobs. Seems to be working on my end. The parsing of the ingested logs is tricky since the formats change depending on what's being reported.

I've overwritten the code below with the v2 version. I shrunk the font down to try and avoid wrapping. Tegile has released their PowerShell commandlets. They do a lot but still don't allow for this type of reporting.

Updated to correct a few typos... sorry :-)

UPDATE: 11-07-16
Tegile recently released the 3.5.x firmware update. The initial release had a bug and a I sincerely hope none of you experienced the same issues that we did with it.

Another issue is that they have changed the formatting of the replication log messages. I had to completely rewrite the script below to allow the results to be captured properly. In addition I added detection of aborted jobs.

Note the config file has changed. We now specify DAYS instead of WEEKS to report.

UPDATE: 08-21-18
Note that this script is now located on the MS PowerShell Gallery. It can be found on my library main page at https://www.powershellgallery.com/profiles/Kcmjr/. This way I don't need to constantly update this post when versions change.

The script noted below is an OLD VERSION. See the link for updates...

<#==============================================================================

File Name : ReplicationReport.ps1

Original Author : Kenneth C. Mazie (kcmjr @ kcmjr.com)

Description : This script will query multiple Tegile Zebi Controllers,

: parse the replication logs, and populate an Excel spreadsheet.

: The results are pulled from back as far as a preset number of

: days. It then emails the resulting spreadsheet. X number of

: prior runs are retained to avoid filling the disk.

Arguments : Named commandline parameters: (all are optional)

: "-console" - Displays console output during run.

: "-debug" - Switches email recipient and some criteria.

Notes : Settings are loaded from an XML file located in the script folder.

: See the end of the script for config file example.

Requirements : Requires PS v5. Requires Posh-SSH module.

Warnings : None

Legal : Public Domain. Modify and redistribute freely. No rights reserved.

: SCRIPT PROVIDED "AS IS" WITHOUT WARRANTIES OR GUARANTEES OF

: ANY KIND. USE AT YOUR OWN RISK. NO TECHNICAL SUPPORT PROVIDED.

Credits : Code snippets and/or ideas came from many sources around the web.

Last Update by : Kenneth C. Mazie (email kcmjr AT kcmjr.com for comments or to report bugs)

Version History : v1.0 - 02-03-16 - Original

Change History : v1.1 - 03-09-16 - Added XML config file

: v2.0 - 09-02-16 - Added detection of failed jobs. Retooled parser.

: v3.0 - 09-12-16 - Retooled for Pos-SSH v1.7.6. Eliminated PSEXEC

: requirement. Eliminated work file requirement. Added save only

: the last X copies. Added debug name change. Added note on email

: that refers to failures found.

: v4.0 - 11-04-16 - Complete redesign of time calculation. Now

: uses epoch starting at 01-01-1970 as reference. This avoids issues

: with using the week of the year. Also corrected for up to 4 different

: variations is log formats seen in v3.5 Zebi firmware.

#===============================================================================#>

#requires -version 5.0

Param (

[bool]$Debug = $false,

[bool]$Console = $false

)

clear-host

#--[ Store all the start up variables so you can clean up when the script finishes. ]--

if ($startupvariables) { try {Remove-Variable -Name startupvariables -Scope Global -ErrorAction SilentlyContinue } catch { } }

New-Variable -force -name startupVariables -value ( Get-Variable | ForEach-Object { $_.Name } )

If (!(Get-Module Posh-SSH)){Import-Module "Posh-SSH" -ErrorAction SilentlyContinue}

If ($Debug){$Script:Debug = $true}

If ($Console){$Script:Console = $true}

$ErrorActionPreference = "silentlycontinue"

$Script:Attach = $true

$Script:Datetime = Get-Date -Format "MM-dd-yyyy_HH:mm" #--[ Current date $ time ]--

$Script:Today = Get-Date -Format "MM-dd-yyyy" #--[ Current date ]--

$Script:ThisYear = Get-Date -Format "yyyy" #--[ Current year ]--

$EpochDiff = New-TimeSpan '01 January 1970 00:00:00' $(Get-Date) #--[ Seconds since 01-01-1970 ]--

$EpochSecs = [INT] $EpochDiff.TotalSeconds #--[ Rounded ]--

$EpochDays = [INT] (($EpochDiff.TotalSeconds)/86400) #--[ Converted to days ]--

$Script:StatusFail = $false

#--[ Excel Non-Interactive Fix ]------------------------------------------------

If (!(Test-path -Path "C:\Windows\System32\config\systemprofile\Desktop")){New-Item -Type Directory -Name "C:\Windows\System32\config\systemprofile\Desktop"}

If (!(Test-path -Path "C:\Windows\SysWOW64\config\systemprofile\Desktop")){New-Item -Type Directory -Name "C:\Windows\SysWOW64\config\systemprofile\Desktop"}

#--[ Excel will crash when run non-interactively via a scheduled task if these folders don't exist ]--

#--[ Functions ]----------------------------------------------------------------

Function ResetVariables {

Get-Variable | Where-Object { $startupVariables -notcontains $_.Name } | ForEach-Object {

try { Remove-Variable -Name "$($_.Name)" -Force -Scope "global" -ErrorAction SilentlyContinue -WarningAction SilentlyContinue}

catch { }

}

Function SendEmail {

If ($Script:Debug){$ErrorActionPreference = "stop"}

$email = New-Object System.Net.Mail.MailMessage

$email.From = $Script:EmailFrom

$email.IsBodyHtml = $Script:EmailHTML

$email.To.Add($Script:EmailTo)

$email.Subject = $Script:EmailSubject

$email.Body = $Script:ReportBody

If ($Script:Attach -eq $true){

$email.Attachments.Add("$PSScriptRoot\$Script:SaveFile.xlsx")

}

$smtp = new-object Net.Mail.SmtpClient($Script:SmtpServer)

$smtp.Send($email)

If ($Script:Log){Add-content -Path "$PSScriptRoot\debug.txt" -Value "-------------[ Sending Email ]--------------"}

If ($Script:Console){Write-Host "`nEmail sent...`n"}

}

#--[ Read and load configuration file ]-----------------------------------------

If (!(Test-Path "$PSScriptRoot\Configuration.xml")){ #--[ Error out if configuration file doesn't exist ]--

Write-host "MISSING CONFIG FILE. Script aborted." -forgroundcolor red

break

}Else{

[xml]$Script:Configuration = Get-Content "$PSScriptRoot\Configuration.xml" #--[ Load configuration ]--

$Script:DebugEmail = $Script:Configuration.Settings.Email.Debug

$Script:EmailTo = $Script:Configuration.Settings.Email.To

$Script:EmailHTML = $Script:Configuration.Settings.Email.HTML

$Script:EmailSubject = $Script:Configuration.Settings.Email.Subject

$Script:EmailFrom = $Script:Configuration.Settings.Email.From

$Script:SmtpServer = $Script:Configuration.Settings.Email.SmtpServer

$Script:UserName = $Script:Configuration.Settings.Credentials.Username

$Script:Password = $Script:Configuration.Settings.Credentials.Password

[array]$Script:Targets = $Script:Configuration.Settings.General.Targets

$Script:DaysBack = $Script:Configuration.Settings.General.Days

$Script:ReportsToKeep = $Script:Configuration.Settings.General.Reports #--[ How many reports to retain ]--

$Script:FileName = $Script:Configuration.Settings.General.FileName

}

#-------------------------------------------------------------------------------

#$Credential = New-Object PSCredential($UserName, (ConvertTo-SecureString $Password.SubString(64) -k ($Password.SubString(0,64) -split "(?<=\G[0-9a-f]{2})(?=.)" | % { [Convert]::ToByte($_,16) })))

#$Credential = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $UserName, ($Password | ConvertTo-SecureString)

Get-ChildItem -Path "$PSScriptRoot\" | Where-Object {-not $_.PsIsContainer -and $_.Name -like "*.xlsx"} | Sort-Object -Descending -Property LastTimeWrite | Select-Object -Skip $Script:ReportsToKeep | Remove-Item

If ($Script:Debug){

[string]$Script:TodayDate = Get-Date -Format MM-dd-yyyy_hhmm #--[ Adds hour and min to filename for debugging ]--

}Else{

[string]$Script:TodayDate = Get-Date -Format MM-dd-yyyy

}

[string]$Script:SaveFile = $Script:FileName+"-"+$Script:TodayDate

If (Test-Path "$PSScriptRoot\$Script:SaveFile.xlsx"){ #--[ If not in debug mode will rename exisiting XLS to allow it to overwritten ]--

Rename-Item -Path "$PSScriptRoot\$Script:SaveFile.xlsx" -NewName "$PSScriptRoot\$Script:SaveFile-old.xlsx"

}

$Script:ReportBody = "Attached are the replication results from the prior "+$Script:DaysBack+" days."

$Script:ReportBody += " - All Zebi controllers are included regardless of whether they actively participate in replication or not."

$Script:ReportBody += " - Each controller has it's own tab on the spreadsheet."

$Script:ReportBody += " - Only outgoing replication jobs are tracked."

$Color = @{

"1" = "green"

"2" = "red"

"3" = "yellow"

"4" = "cyan"

"5" = "magenta"

}

$MissingType = [System.Type]::Missing

$Excel = ""

$Excel = New-Object -ComObject Excel.Application

If ($Script:Console -or $Script:Debug){

$Excel.visible = $True

$Excel.DisplayAlerts = $true

$Excel.ScreenUpdating = $true

$Excel.UserControl = $true

$Excel.Interactive = $true

}Else{

$Excel.visible = $False

$Excel.DisplayAlerts = $false

$Excel.ScreenUpdating = $false

$Excel.UserControl = $false

$Excel.Interactive = $false

}

$TargetCount = $Script:Targets.Target.count

$Workbook = $Excel.Workbooks.Add()

$Workbook.Worksheets.Add($MissingType, $Workbook.Worksheets.Item($Workbook.Worksheets.Count), ($TargetCount) - $Workbook.Worksheets.Count, $Workbook.Worksheets.Item(1).Type) | Out-Null

#--[ Note that new sheets are always adding in reverse ]--

$SheetCount = 1

$Target = @()

Foreach ($Target in $Script:Targets.Target){

if ($Console){Write-Host "`n--[ Processing Target: $Target ]-----------------------------------" -ForegroundColor Yellow }

$Count = 1

$Worksheet = $workbook.Sheets.Item($SheetCount)

$Worksheet.Activate()

$Worksheet.name = $Target

$Row = 1

$Col = 1

$WorkSheet.Cells.Item($Row,$Col) = "Date:"

$WorkSheet.Cells.Item($Row,$Col).font.bold = $true