#NetApp DataOnTap and Invoke-NaSysstat

So I have been play with the NetApp DataOnTap add in for a while now want to share this.  Essentially all it does is run Invoke-NaSysstat at timed intervals and saves it to an hourly csv file

[void][System.Reflection.Assembly]::LoadWithPartialName('Microsoft.VisualBasic')
$filer   = [Microsoft.VisualBasic.Interaction]::InputBox("Enter the comma seperated filer name", "Enter Filer Name", "")

#Change to match you path
Import-Module E:\PsMON\netapp\DataONTAP.psd1
Connect-NaController $Filer -Credential root

$matrix = @()
$doit = $True

$timer = 10

$LastHour = (Get-Date -format "HH")

while($doit){
  $ThisHour = (Get-Date -format "HH")
  $tSysStat = Invoke-NaSysstat -Count 1 | Select
  $tSysStat
  $matrix += $tSysStat
  If($ThisHour -ne $LastHour){
    $outFile = $filer + "_" + (Get-Date -format "yyyy-MM-dd_HHmm") + ".csv";
    $Matrix | Export-Csv $outFile -NoTypeInformation;$matrix = @();
    "-$outFile-"
  }
  $LastHour = $ThisHour

  For($i=0;$i-le $timer;$i++){Sleep 1 }
}

$outFile = $filer + "_" + (Get-Date -format "yyyy-MM-dd_HHmm") + ".csv";$Matrix | Export-Csv $outFile -NoTypeInformation

NetApp SnapManager for Exchange and The communication object, System.ServiceModel.Channels.ServiceChannel, cannot be used for communication because it is in the Faulted state.

#NetApp #MsExchange

Okay so nailed this one eventually.

[SERVER1] Initializing SnapManager server on remote machine [SERVER2]…
[SERVER1] Connecting to remote server [SERVER2]…
[SERVER1] Remote server [SERVER2] is connected successfully.
[SERVER1] Verify Sme Launch Exception, Error code: 0x80131501, Error description: The communication object, System.ServiceModel.Channels.ServiceChannel, cannot be used for communication because it is in the Faulted state., stack trace:
Server stack trace:
at System.ServiceModel.Channels.CommunicationObject.ThrowIfFaulted()
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMes
[21:02:47.831]  [SERVER1] Verify SME Launch on SERVER2 returned error 0x80131501
[SERVER1] Verify Sme Launch2 Exception, Error code: 0x80131501 [The communication object, System.ServiceModel.Channels.ServiceChannel, cannot be used for communication because it is in the Faulted state.], stack trace:
Server stack trace:
at System.ServiceModel.Channels.CommunicationObject.ThrowIfDisposedOrNotOpen()
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IM
[SERVER1] CCR/DAG Remote Rename Backup Exception, Error code: 0x80131501 [The communication object, System.ServiceModel.Channels.ServiceChannel, cannot be used for communication because it is in the Faulted state.], stack trace:
Server stack trace:
at System.ServiceModel.Channels.CommunicationObject.ThrowIfDisposedOrNotOpen()
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturn

[SERVER1] Error Code: 0x80131501
Please check the SnapManager-SnapDrive Reports and Windows Event Logs for more details.

It was caused by the Cisco ASA firewall on our Inter DC link timing out idle connections after one hour.

When the SME service starts, it establishes a connection over TCP port 810 to other servers in it’s DAG. SME expects the link to be available all the time and not time out.

SME is a but stupid, in that as it EXPECTS the link to be open, and s.

Workaround: Restart the SnapManagerService before the Backups Starts.
Solution#1: Increase the timeout value on the ASA firewalls for port 808 and 810 from 1 hour to8 hours  (we do backups every 6 hours).
Solution#2: Product change to SME, so before it initiates a backup, it actually re-establishes the connection before it starts. BUG/RFE 828920 has been raised to address this.

 

 

NetApp SnapManager for Exchange and Asynchronous call DoSnapshotSet timed out! errors

#NetApp #MsExchange

Have some issues with SME backing up Exchange and it throwing “Asynchronous call DoSnapshotSet timed out!” all over the backup log.

Well it appears that having free_space_realloc=on set on the Aggregates caused this.  As soon as it was set to off, this issue went away.

That just leaves this error to be fixed

Verify Sme Launch Exception, Error code: 0x80131501, Error description: The communication object, System.ServiceModel.Channels.ServiceChannel, cannot be used for communication because it is in the Faulted state.

 

#Powershell and #NetApp SnapDrive

I blogged this script earlier, but discovered an undocumented feature where the last entry in the output was not added to the array object.  Here is the updated version

Write-Host "Running sdcli disk list"
$diskInfo = Invoke-Expression "sdcli disk list"
$sdclidisks  = @()

$fields  = "UNCPath","LUNPath","StorageSystem","StorageSystemPath","Type","Diskserialnumber","BackedbySnapshotCopy"
$fields += "Shared","BootOrSystemDisk","SCSIport","Bus","Target","LUN","Readonly","Size","SnapmirrorSource","SnapvaultPrimary"
$fields += "DiskPartitionStyle","CloneSplitRestorestatus","DiskID","VolumeName","Mountpoints","IPAddresses","FCinitiatorWWPN"
ForEach($Item in $DiskInfo){
  $tmpItem = $Item.Trim()
  $tmpItemSplit = $tmpItem.Split(":")
  Switch -Wildcard ($tmpItem){
    "The operation completed successfully.*" {$sdclidisks  += $sdcliDiskList}
    "UNC Path:*"  {$sdclidisks  += $sdcliDiskList
                   $sdcliDiskList = "" | Select $fields
                   $sdcliDiskList.UNCPath = $tmpItemSplit[-1]}

    "LUN Path:*"                   {$sdcliDiskList.LUNPath                 = $tmpItemSplit[-1]}
    "Storage System:*"             {$sdcliDiskList.StorageSystem           = $tmpItemSplit[-1].trim()}
    "Storage System Path:*"        {$sdcliDiskList.StorageSystemPath       = $tmpItemSplit[-1].trim()}
    "Type:*"                       {$sdcliDiskList.Type                    = $tmpItemSplit[-1].trim()}
    "Disk serial number:*"         {$sdcliDiskList.Diskserialnumber        = $tmpItemSplit[-1].trim()}
    "Backed by Snapshot Copy:*"    {$sdcliDiskList.BackedbySnapshotCopy    = $tmpItemSplit[-1].trim()}
    "Shared:*"                     {$sdcliDiskList.Shared                  = $tmpItemSplit[-1].trim()}
    "BootOrSystem Disk:*"          {$sdcliDiskList.BootOrSystemDisk        = $tmpItemSplit[-1].trim()}
    "SCSI port:*"                  {$sdcliDiskList.SCSIport                = $tmpItemSplit[-1].trim()}
    "Bus:*"                        {$sdcliDiskList.Bus                     = $tmpItemSplit[-1].trim()}
    "Target:*"                     {$sdcliDiskList.Target                  = $tmpItemSplit[-1].trim()}
    "LUN:*"                        {$sdcliDiskList.Lun                     = $tmpItemSplit[-1].trim()}
    "Readonly:*"                   {$sdcliDiskList.Readonly                = $tmpItemSplit[-1].trim()}
    "Size:*"                       {$sdcliDiskList.Size                    = $tmpItemSplit[-1].trim()}
    "Snapmirror Source:*"          {$sdcliDiskList.SnapmirrorSource        = $tmpItemSplit[-1].trim()}
    "Snapvault Primary:*"          {$sdcliDiskList.SnapvaultPrimary        = $tmpItemSplit[-1].trim()}
    "Disk Partition Style:*"       {$sdcliDiskList.DiskPartitionStyle      = $tmpItemSplit[-1].trim()}
    "Clone Split Restore status:*" {$sdcliDiskList.CloneSplitRestorestatus = $tmpItemSplit[-1].trim()}
    "DiskID:*"                     {$sdcliDiskList.DiskID                  = $tmpItemSplit[-1].trim()}
    "Volume Name:*"                {$sdcliDiskList.VolumeName              = $tmpItemSplit[-1].trim()}
    "*Mount points:*"              {$sdcliDiskList.Mountpoints             = $tmpItem.Split("`t")[-1].trim()}
    "IP Addresses:*"               {$sdcliDiskList.IPAddresses             = $tmpItemSplit[-1].trim()}
   "FC initiator WWPN:*"           {$sdcliDiskList.FCinitiatorWWPN         = $tmpItem.Split("`t")[-1].trim()}
  }
 }

$sdclidisks = $sdclidisks | where {$_.DiskID -ne $Null}

Now you have $sdclidisks you can say export to csv.

Download

Enjoy

NetApp SnapManager for Exchange Reports and Powershell Brackets

#NetApp #Powershell

SME is a great tool, but it has an undocumented feature, where it doesn’t flush it’s report files.  Now these files get big and I wanted a way to zip or delete the files.

The issue is that the folder name has brackets in it [servername] and powershell treats brackets as “special”.
Try it! Open powershell, create a folder and put a file in the folder

md C:\ps\[test]

now dir it:

dir C:\ps\[test]

Nothing right, but it works with the command prompt.  It took figging ages and a fair amount of Bing time to work a way around it, it’s dead simple:

dir -LiteralPath C:\ps\[test]

Yeah baby! using -LiteralPath does the trick

Long boot times may be observed with Data ONTAP DSM 3.5

NetApp

Interesting bug! that is supposed to be fixed in DSM 4.0 – http://support.netapp.com/NOW/cgi-bin/bol?Type=Detail&Display=591952

Bug ID 591952
Title Long boot times may be observed with Data ONTAP DSM 3.5
Duplicate of
Bug Severity 2 – System barely usable
Bug Status Fixed
Product Unknown
Bug Type ONTAP DSM
Description
 If a Windows Server host is running Data ONTAP DSM 3.5, long boot times may be observed.
 The boot time will vary based on the number of mapped LUNs and paths per mapped LUN.
 Boot times will be approximately 3 seconds per path, per LUN.
Workaround
 Reduce the number of mapped devices during Windows Server OS boot. You can do this
 several ways, which includes disabling the initiator ports before reboot or taking the
 mapped LUNs offline before the system startup.

#NetApp Data ONTAP DSM for Windows MPIO 4.0

Found version 4.0 on the NetApp Support website post 14 Feb 2013

The Data ONTAP DSM for Windows MPIO enables you to manage multiple Fibre Channel (FC) and iSCSI paths between a NetApp or IBM N series storage system and a Windows host computer. You can have one or more active paths and zero or more passive paths, depending on the load balance policy. The DSM automatically selects paths for I/O as needed, maximizing throughput and maintaining the host’s access to its storage. The DSM has both graphical and command line management interfaces. You can install the DSM interactively or from a Windows command prompt.

New features: The Data ONTAP DSM 4.0 for Windows MPIO includes several new features and support for additional configurations:

  • Support for Windows Server 2012.
  • An enhanced GUI, offering a better organized and more detailed view of the virtual disks and paths managed by the DSM:
      • Extended virtual disk and path information.
      • Bulk operations on virtual disks and paths.
      • Visual indication of path status.
      • Extended path, LUN, and I/O information.
      • Persistent reservation status and reservation key information.
      • Persistent reservation configuration information.
      • Extended MPIO configuration information.
      • ALUA feature status.
      • PowerShell 2.0 or later is now required
  • Enhanced performance.
  • The mbralign.exe program is no longer supported. Use the Data ONTAP PowerShell Toolkit to detect and correct VHD partition alignment.
  • The legacy dsmcli is no longer supported. Use the PowerShell cmdlets shipped with the DSM to manage the DSM programmatically.

NetApp sdcli disk list to CSV with Powershell

#NetApp #sdcli #Powershell

So I have been breaking playing around with Jetstress, NetApp filers and Cisco UCS.  Not have much fun or luck, as I just cant get the damn thing to pass even a simple test.  But that is another story!

I have been looking at sdcli and getting some powershell automation going, and I wanted to find out details of all the disks connected to my severs.  Now sdcli is the worst thing in the word for getting anything decent out, so I wrote the following.  Basically run it on a server with NetApp SnapDrive and it will give you a CSV file with the output of sdcli disk list.  Shweet ;-)   It crude but effective!

Enjoy

Get-SdcliDiskList.ps1

$diskInfo = Invoke-Expression “sdcli disk list”$tmpsdcliDiskListMatrix = @()
ForEach($Item in $DiskInfo){
$tmpItem = $Item.Trim()
$tmpItemSplit = $tmpItem.Split(“:”)
Switch -Wildcard ($tmpItem){
“UNC Path:*”  {$tmpsdcliDiskListMatrix  += $sdcliDiskList
$sdcliDiskList = “” | Select UNCPath, LUNPath, StorageSystem, StorageSystemPath, Type, Diskserialnumber, BackedbySnapshotCopy, Shared, BootOrSystemDisk, SCSIport, Bus, Target, LUN, Readonly, Size, SnapmirrorSource, SnapvaultPrimary, DiskPartitionStyle, CloneSplitRestorestatus, DiskID, VolumeName, Mountpoints, IPAddresses, FCinitiatorWWPN
$sdcliDiskList.UNCPath = $tmpItemSplit[-1]}
“LUN Path:*”                  {$sdcliDiskList.LUNPath = $tmpItemSplit[-1]}
“Storage System:*”            {$sdcliDiskList.StorageSystem = $tmpItemSplit[-1].trim()}
“Storage System Path:*”       {$sdcliDiskList.StorageSystemPath = $tmpItemSplit[-1].trim()}
“Type:*”                      {$sdcliDiskList.Type = $tmpItemSplit[-1].trim()}
“Disk serial number:*”        {$sdcliDiskList.Diskserialnumber = $tmpItemSplit[-1].trim()}
“Backed by Snapshot Copy:*”   {$sdcliDiskList.BackedbySnapshotCopy = $tmpItemSplit[-1].trim()}
“Shared:*”                    {$sdcliDiskList.Shared = $tmpItemSplit[-1].trim()}
“BootOrSystem Disk:*”          {$sdcliDiskList.BootOrSystemDisk = $tmpItemSplit[-1].trim()}
“SCSI port:*”                  {$sdcliDiskList.SCSIport = $tmpItemSplit[-1].trim()}
“Bus:*”                        {$sdcliDiskList.Bus = $tmpItemSplit[-1].trim()}
“Target:*”                     {$sdcliDiskList.Target = $tmpItemSplit[-1].trim()}
“LUN:*”                        {$sdcliDiskList.Lun = $tmpItemSplit[-1].trim()}
“Readonly:*”                   {$sdcliDiskList.Readonly = $tmpItemSplit[-1].trim()}
“Size:*”                       {$sdcliDiskList.Size = $tmpItemSplit[-1].trim()}
“Snapmirror Source:*”          {$sdcliDiskList.SnapmirrorSource = $tmpItemSplit[-1].trim()}
“Snapvault Primary:*”          {$sdcliDiskList.SnapvaultPrimary = $tmpItemSplit[-1].trim()}
“Disk Partition Style:*”       {$sdcliDiskList.DiskPartitionStyle = $tmpItemSplit[-1].trim()}
“Clone Split Restore status:*” {$sdcliDiskList.CloneSplitRestorestatus = $tmpItemSplit[-1].trim()}
“DiskID:*”                     {$sdcliDiskList.DiskID = $tmpItemSplit[-1].trim()}
“Volume Name:*”                {$sdcliDiskList.VolumeName = $tmpItemSplit[-1].trim()}
“*Mount points:*”              {$sdcliDiskList.Mountpoints = $tmpItem.Split(“`t”)[-1].trim()}
“IP Addresses:*”               {$sdcliDiskList.IPAddresses = $tmpItemSplit[-1].trim()}
“FC initiator WWPN:*”          {$sdcliDiskList.FCinitiatorWWPN = $tmpItem.Split(“`t”)[-1].trim()}
}
}

$tmpsdcliDiskListMatrix  | Export-Csv SDCLIDisks.csv -NoTypeInformation -Delimiter “|”