Dell EMC PowerFlex MP for vROps 8.x - Part6 - Create custom alerts

In this post, we will take a look at creating custom alerts for PowerFlex by adding symptom definitions and alert definitions. Refer to my previous blog post to understand more about the alerting aspects in vROps. Here we will take an example scenario and see how we can create custom symptom definitions and alert definitions.

Scenario

The user is running some latency-sensitive business-critical applications using PowerFlex storage. Below are the symptoms that he would like to define and alerts should be produced for the same and these should affect the "Health" badge of the PowerFlex volume object.

Step1: Add Symptom Definitions

Go to Alerts - Symptom Definitions - Click Add.

Select base object type: Expand PowerFlex Adapter - Select Volume.

• Select the metric User Data SDC Read Latency (ms): double click on it twice so that you can define both warning and critical symptoms.
• Select the metric User Data SDC Write Latency (ms): double click on it twice so that you can define both warning and critical symptoms.

Now, fill all the required fields as per the conditions we defined earlier.

Click Save. Now as you can see below the 4 symptom definitions are created.

Step2: Add Alert Definitions

Go to Alerts - Alert Definitions - Click Add.

• Provide alert name, select the base object type and advanced settings and click Next.

• Filter and search the symptoms that we created earlier. Drag and drop the two volume read latency related symptoms and select Any. Click Next.

• If you want to provide any recommendations you can add it in this step and click Next.
• Select vSphere Solution's Default Policy and click Next and click Create.

Similarly, you can create an alert definition for PowerFlex Volume Write Latency too.

Now, we are all done. Let's test the alerts! I am using FIO to generate IO load on one of the PowerFlex volume.

You can see the Read Latency for this volume is grater than 1 ms, and so a warning alert should be produced for this specific volume.

Hope it was useful. Cheers!

Related posts

Part1: Install
Part2: Configure
Part3: Dashboards
Part4: Resource kinds and relationships
Part5: Collection interval 

References

Product guide: https://infohub.delltechnologies.com/section-assets/powerflexadapter-for-vrops-product-guide

PowerFlex website: https://www.delltechnologies.com/PowerFlex

PowerFlex white papers and blog: https://infohub.delltechnologies.com/t/powerflex-14/

Dell EMC PowerFlex MP for vROps 8.x - Part5 - Collection interval

In this post, we will take a look at modifying the collection interval of  PowerFlex Adapter instances. The PowerFlex Management Pack for vROps supports 4 instance types.

• PowerFlex Gateway
• PowerFlex Networking
• PowerFlex Manager
• PowerFlex Nodes

The default collection interval for all these adapter instances is set to 5 minutes. In most cases, you don't need to modify this. But, say you want to get PowerFlex storage performance metrics more frequently, then you have to change the collection interval of the PowerFlex Gateway instance. You can set it to as low as 1 minute. As per the testing that I have done in the lab, a PowerFlex Gateway adapter instance is able to complete the collection process of a PowerFlex storage cluster in less than a minute.

Note: If you are modifying the collection interval from the default value, make sure to verify that the collection process is able to complete successfully within the new time interval.

Administration - Inventory - Adapter Instances - PowerFlex Adapter Instance

Note: In the product guide it is recommended to configure not more than 40 Cisco switches in one PowerFlex Networking instance. So, if you have 80 switches in your PowerFlex system, you will need to configure 2 PowerFlex Networking instances where each instance will connect/ query/ collect details from 40 switches. This is based on the default collection interval of 5 minutes.

This simply means, in 5 minutes one PowerFlex Networking adapter instance can complete the collection from a max of 40 switches only. So, in 1 minute, it can complete the collection of a maximum of 8 switches. This is a rough calculation and it depends on factors like REST API response, switch firmware/ OS version, etc. So if you change the default interval, always make sure to monitor it (the collection cycle) for some time and verify whether the collection process is able to complete successfully within the new time interval. 

Hope it was useful. Cheers!

Related posts

Part1 - Install
Part2 - Configure
Part3 - Dashboards
Part4 - Resource kinds and relationships

References

Product guide: https://infohub.delltechnologies.com/section-assets/powerflexadapter-for-vrops-product-guide
PowerFlex website: https://www.delltechnologies.com/PowerFlex
PowerFlex white papers and blog: https://infohub.delltechnologies.com/t/powerflex-14/

Working with REST API using PowerShell

In this blog, I am consolidating all the articles I published regarding REST APIs and PowerShell. This will be useful for all those who like to kick-start working with various REST APIs using PowerShell.

iDRAC

Working with iDRAC9 Redfish API using PowerShell - Part 1

Working with iDRAC9 Redfish API using PowerShell - Part 2

Working with iDRAC9 Redfish API using PowerShell - Part 3

Working with iDRAC9 Redfish API using PowerShell - Part 4

VxFlex OS (ScaleIO)

Working with ScaleIO REST API using PowerShell - Part 1

Working with ScaleIO REST API using PowerShell - Part 2

Working with ScaleIO REST API using PowerShell - Part 3

Creating HTML report of ScaleIO cluster using PowerShell

Cisco Nexus 9K switch

Working with Cisco Nexus 9K switches using PowerShell

Hope it was useful. Cheers!

References

https://virtuallysober.com/2018/01/04/introduction-to-powershell-rest-api-authentication/

Creating HTML report of ScaleIO cluster using PowerShell

This post is a reference to a small reporting script for ScaleIO environments. The project will generate a brief HTML report of your ScaleIO Ready Node SDS infrastructure (with AMS - Automated Management Services) by making use of ScaleIO Ready Node AMS REST APIs and PowerShell. The report provides information about MDM cluster state, overall cluster capacity, system objects, alerts, and health state of all disks in the cluster. Here the API is available as part of ScaleIO Ready Node AMS. These AMS REST API allows you to query information and perform actions related to ScaleIO software and ScaleIO Ready Node hardware components. To access the API you need to provide AMS username and password. Responses returned by AMS server are formatted in JSON format.

Project reference: https://github.com/vineethac/sio_report

Use case: This script can be used/ leveraged as part of daily cluster health/ stats reporting process, or something similar; so that monitoring Engineers or whoever responsible can have a look at it on a daily basis to make sure everything is healthy and working normal. 

Related references:

Working with ScaleIO REST API using PowerShell - Part 1

Working with ScaleIO REST API using PowerShell - Part 2

Working with ScaleIO REST API using PowerShell - Part 3

Hope this was helpful. Cheers!

Working with ScaleIO REST API using PowerShell - Part 3

As you all know disk drives are the most important and basic building blocks of a storage system. In order to ensure data availability, the disk drives in your storage system should always stay healthy. Unhealthy drives not only cause risk to your data but also contribute to degraded performance. In this article, I will show how to query and get disk health-related information in a ScaleIO cluster using REST API and PowerShell. 

Please refer my previous posts for API authentication, token generation and 'Invoke-RestMethod'.

Working with ScaleIO REST API using PowerShell - Part 1

Working with ScaleIO REST API using PowerShell - Part 2

#Query all devices in the ScaleIO cluster

$all_devices = (Invoke-RestMethod -Uri "https://192.168.11.15:443/api/types/Device/instances " -Method Get -Headers $ScaleIOAuthHeaders)  

#Select necessary properties and sort

$all_devices = (Invoke-RestMethod -Uri "https://192.168.11.15:443/api/types/Device/instances " -Method Get -Headers $ScaleIOAuthHeaders) | select sdsId, storagePoolId, name, deviceCurrentPathName, errorState, deviceState, ssdEndOfLifeState, temperatureState, aggregatedState  | sort sdsId | ft -AutoSize

#Sample output

Hope this was helpful. Please refer to ScaleIO 2.5 API reference guide for more details. Cheers!

Working with ScaleIO REST API using PowerShell - Part 2

In my previous post, I explained how to connect and authenticate with ScaleIO REST API, and use Invoke-RestMethod. In this article, I will explain how to query selected statistics/ properties with POST method. 

First, you have to define the set of properties that you would like to query, and then pass it to "invoke-restmethod". You can get the list of available properties of the respective object from the ScaleIO REST API Reference Guide. 

Example 1: Query and collect MDM cluster information
Here I am collecting the below-mentioned properties of "MDMCluster" object. 

#selected properties to query

$param1 = @'

{

    "properties":["name", "clusterMode", "master", "clusterState", "virtualIps"]

}

'@

#invoke-restmethod with respective uri

$MDM_cluster_stats = (Invoke-RestMethod -Uri "https://192.168.11.15:443/api/instances/System/queryMdmCluster " -Body $param1 -ContentType "application/json" -Headers $ScaleIOAuthHeaders -Method Post)

#organize output in a hash table

$mdm_props = @{

    'Cluster name'      = ($MDM_cluster_stats).name

    'Mode'              = ($MDM_cluster_stats).clusterMode

    'Cluster state'     = ($MDM_cluster_stats).clusterState

    'Master MDM IP'     = ($MDM_cluster_stats.master).managementIPs[0]

    'Cluster VIP01'     = ($MDM_cluster_stats).virtualIps[0]

    'Cluster VIP02'     = ($MDM_cluster_stats).virtualIps[1]

}

Write-Output $mdm_props

#sample output screenshot given below

Example 2: Query and collect overall ScaleIO system capacity details

Here I am collecting some properties of "System Statistics" object. 

#selected properties to query

$param2 = @'

{

    "properties":["maxCapacityInKb", "capacityInUseInKb", "spareCapacityInKb", "failedCapacityInKb", "degradedFailedCapacityInKb"]

}

'@

#invoke-restmethod with respective uri

$system_overall_stats = (Invoke-RestMethod -uri "https://192.168.11.15:443/api/types/System/instances/action/querySelectedStatistics " -Body $param2 -ContentType "application/json" -Headers $ScaleIOAuthHeaders -Method Post)

#organize output in a hash table

$system_capacity_props = @{

    "System max capacity (TB)"                = (($system_overall_stats.maxCapacityInKb)/1024/1024/1024)

    "System capacity in use (TB)"             = (($system_overall_stats.capacityInUseInKb)/1024/1024/1024)

    "System spare capacity (TB)"              = (($system_overall_stats.spareCapacityInKb)/1024/1024/1024)

    "System failed capacity (TB)"             = (($system_overall_stats.failedCapacityInKb)/1024/1024/1024)

    "System degraded failed capacity (TB)"    = (($system_overall_stats.degradedFailedCapacityInKb)/1024/1024/1024)

}

Write-Output $system_capacity_props

#sample output screenshot given below

Hope this was useful. Please refer Dell EMC ScaleIO Ready Node Version 2.5 API Reference Guide for more details.

Working with Dell EMC PowerFlex REST API using PowerShell - Part 1

In this article, I will explain briefly how to work with REST API exposed by PowerFlex (formerly known as ScaleIO and VxFlex OS Software) using PowerShell to manage/ monitor your PowerFlex cluster. The PowerFlex REST API is served from the PowerFlex Gateway which allows querying information and performing actions related to PowerFlex software components. PowerFlex Gateway connects and queries master MDM, reformats the results in a RESTful manner, and passes it to a REST client.

Access to the API requires a login using Gateway username and password. A successful login returns a token, and you can use this token for later authentications for other requests. The responses or results returned are in JSON format. In the following sections, I will explain how to use PowerShell to query PowerFlex REST API and obtain relevant results.  

PowerShell uses Invoke-RestMethod cmdlet which sends HTTP/ HTTPS requests to REST web services that return rich structured data. PowerShell will format the response based on the data type. For JSON PowerShell converts the response content to objects. 

Step 1: Authentication and token generation

#Note: 192.168.11.15 is the IP of PowerFlex Gateway

$User = Read-Host -Prompt "Please enter username"

$SecurePassword = Read-Host -Prompt "Enter Password for user $user" –AsSecureString

$Credentials = New-Object System.Management.Automation.PSCredential ($user,$Securepassword)

#HTTPS Get request is invoked with credentials for authentication

$Token = Invoke-RestMethod -Uri "https://192.168.11.15:443/api/login" -Method Get -Credential $Credentials

Step 2: Create header with the token

$auth = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes(':'+$Token))

$global:PowerFlexAuthHeaders = @{'Authorization' = "Basic $auth"

'Content-Type' = "application/json" }

Step 3: Invoke-RestMethod

#Query all alerts in the system

$alerts = (Invoke-RestMethod -Uri "https://192.168.11.15:443/api/types/Alert/instances/" -Method Get -Headers $PowerFlexAuthHeaders)

#Query all SDSs in a PD

$all_SDS = (Invoke-RestMethod -Uri "https://192.168.11.15:443/api/instances/ProtectionDomain::3ce4af5f00000000/relationships/Sds " -Method Get -Headers $PowerFlexAuthHeaders)

#Query a specified PD

$pd01 = (Invoke-RestMethod -Uri "https://192.168.11.15:443/api/instances/ProtectionDomain::3ce4af5f00000000 " -Method Get -Headers $PowerFlexAuthHeaders)

#Query statistics of a specified PD

$pd01_stats = (Invoke-RestMethod -Uri "https://192.168.11.15:443/api/instances/ProtectionDomain::3ce4af5f00000000/relationships/Statistics " -Method Get -Headers $PowerFlexAuthHeaders)

Hope this was useful. Refer to the PowerFlex REST API Reference Guide for more details.

In the next article, I will explain how to query selected statistics/ parameters with POST method which will help you to

create custom monitoring scripts for your PowerFlex clusters.

Note:

If you are getting the below error while trying to create the token during step 1, please add the below lines of code to the

start of your script.

"Invoke-RestMethod : The underlying connection was closed: An unexpected error occurred on a send."

#Solution to above error!

add-type @"

    using System.Net;

    using System.Security.Cryptography.X509Certificates;

    public class TrustAllCertsPolicy : ICertificatePolicy {

        public bool CheckValidationResult(

            ServicePoint srvPoint, X509Certificate certificate,

            WebRequest request, int certificateProblem) {

            return true;

        }

    }

"@

[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy

[System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]::Tls12 -bor [System.Net.SecurityProtocolType]::Tls11