Using the Pure Storage PowerShellSDK2 - Part 5 - Checking Replication Status

2024-06-12

FlashArray, Pure Storage, Powershell, Pure Storage PowerShell SDK 2 Series

Page content

Introduction

Welcome back to the fifth installment of our blog series on using the Pure Storage PowerShell SDK2. In this post, we’re diving into a hands-on demonstration of using the Pure Storage FlashArray API to track replication performance between two arrays. This is especially useful for DBAs and storage admins, who must ensure their data replication processes run smoothly and efficiently. A typical scenario here is ensuring a snapshot is entirely replicated between sites before kicking off some other process. Following this demo, you’ll learn how to monitor replication status and verify the completion of snapshot transfers between arrays.

Setting Up the Environment

First, let’s define the source and target arrays. For this demo, we’ll be using two arrays identified by their respective endpoints:

$SourceArrayName = 'flasharray1.example.com'
$TargetArrayName = 'flasharray2.example.com'

Next, we must specify the Protection Group on the Source Array, $SourcePGroupName, and construct the Target Protection Group name, $TargetPGroupName. For the Target Protection Group name, we concatenate the Source Array name and the Source Protection Group name. In this example, the Source Protection Group is aen-sql-22-a-pg, and our Target Protection Group Name is flasharray2:aen-sql-22-a-pg. We will use this to track the replication status on the Target FlashArray.

$SourcePGroupName = 'aen-sql-22-a-pg'
$TargetPGroupName = $SourceArrayName.split('.')[0] + ":" + $SourcePGroupName

Connecting to the Arrays

To interact with the arrays, we’ll use the Connect-Pfa2Array cmdlet, which establishes a connection to each array using the provided credentials:

$SourceFlashArray = Connect-Pfa2Array -EndPoint $SourceArrayName -Credential $Credential -IgnoreCertificateError 
$TargetFlashArray = Connect-Pfa2Array -EndPoint $TargetArrayName -Credential $Credential -IgnoreCertificateError

Initiating the Replication Process

First, we create a snapshot of the Protection Group on the Source Array using the New-Pfa2ProtectionGroupSnapshot cmdlet. We set the parameters -ForReplication $true and -ReplicateNow $true to replicate it immediately to the Target Array, storing the cmdlet output in the $snapshot variable. Using these parameters ensures the latest data is transferred to the Target Array.

Next, we define the variable $TargetSnapshotName; this is how we will find the snapshot on the Target Array. In the example below, the snapshot name on the Source Array is aen-sql-22-a-pg.1628; when it’s on the Target Array, we need to address it as flasharray1:aen-sql-22-a-pg.1628.

$Snapshot = New-Pfa2ProtectionGroupSnapshot -Array $SourceFlashArray -SourceName $SourcePGroupName -ForReplication $true -ReplicateNow $true 
$Snapshot

Name              : aen-sql-22-a-pg.1629
Created           : 6/14/2024 12:37:09 PM
Destroyed         : False
Pod               : 
Source            : @{Id='53340721-d581-366f-078d-6d6be001fab3'; Name='aen-sql-22-a-pg'}
Space             : 
Suffix            : 1629
TimeRemaining     : 
EradicationConfig : @{ManualEradication='enabled'}
Id                : 2ac96111-3354-03b5-7eb7-553c6537d463

# Build the name of the snapshot on the Target Array
$TargetSnapshotName = $TargetPGroupName + "." + $Snapshot.Suffix

$TargetSnapshotName
flasharray1:aen-sql-22-a-pg.1628

Monitoring Replication Status

To check the status of snapshot transfers that are not yet completed on the Target Array, we use the Get-Pfa2ProtectionGroupSnapshotTransfer cmdlet specifying our Target Array with the parameter -Array $TargetFlashArray, then the Target Protection Group name with the parameter -Name $TargetPGroupName. We can then add a -Filter "name='$($TargetSnapshotName)'" to ask for exact Target Snapshot Name on the Target Array. We learned about filtering earlier in our blog series in the post Using the Pure Storage PowerShellSDK2 - Part 2 - Working With Data.

In the output below, you can see the status of the snapshot replication for our Protection Group snapshots that still need to complete replication. The replication has a Started date and time of 6/14/2024 12:37:09 PM. The Completed field is empty, which means the snapshot replication is still in progress, and that Progress is 0, which means we haven’t started replication yet.

Get-Pfa2ProtectionGroupSnapshotTransfer -Array $TargetFlashArray -Name $TargetPGroupName -Filter "name='$($TargetSnapshotName)'"

Id                   : 67c86d9e-d312-e238-cd39-82395d4eb752
Name                 : flasharray1:aen-sql-22-a-pg.1629
Completed            : 
DataTransferred      : 343446
Destroyed            : False
PhysicalBytesWritten : 0
Progress             : 0
Started              : 6/14/2024 12:37:09 PM

After some time, you check the status again to see if there are any ongoing transfers by using the same cmdlet and parameters. You can now see that Progress is around 54%, DataTranferred and PhysicalBytesWritten has increased, and Completed is still empty. You can continue to execute this command or wrap it in a loop in code to monitor the status of the snapshot replication.

Get-Pfa2ProtectionGroupSnapshotTransfer -Array $TargetFlashArray -Name $TargetPGroupName -Filter "name='$($TargetSnapshotName)'"
Id                   : 67c86d9e-d312-e238-cd39-82395d4eb752
Name                 : flasharray1:aen-sql-22-a-pg.1629
Completed            : 
DataTransferred      : 1349177
Destroyed            : False
PhysicalBytesWritten : 39810
Progress             : 0.5471904
Started              : 6/14/2024 12:37:09 PM

Verifying Completed Transfers

Finally, when our snapshot’s replication is completed between the two arrays, we now want to retrieve the most recent completed snapshot transfer to ensure the replication process has been completed. This helps verify that our data is safely replicated to the Target Array. This is also done with the Get-Pfa2ProtectionGroupSnapshotTransfer cmdlet, but this time, we’re changing -Filter "completed and name='$($TargetSnapshotName)'" which will return the Target Snapshot we are looking for on the Target Array since the field Completed is populated with a value. So now we know our snapshot is entirely on the Target Array.

The output below shows that Completed is now populated with the completion date and time, and Progress is now 1. Together, these indicate the completion of the snapshot replication.

Get-Pfa2ProtectionGroupSnapshotTransfer -Array $TargetFlashArray -Name $TargetPGroupName -Filter "completed and name='$($TargetSnapshotName)'" 

Id                   : 67c86d9e-d312-e238-cd39-82395d4eb752
Name                 : flasharray1:aen-sql-22-a-pg.1629
Completed            : 6/14/2024 12:37:21 PM
DataTransferred      : 1349177
Destroyed            : False
PhysicalBytesWritten : 1288915
Progress             : 1
Started              : 6/14/2024 12:37:09 PM

Do I have to do this in PowerShell?

So you might be wondering if I have to do this in PowerShell. Absolutely not. You can do this using the CLI, Python, or even REST. For any of the examples above, you can easily find the REST endpoint by adding the —Verbose parameter to the cmdlet. Here’s how to check if the replication is completed in REST. I’ll follow up with a post with an example in Python.

VERBOSE: PureStorage.Rest Verbose: 10 : 2024-06-14T15:12:32.1878550Z GET https://flasharray1.example.com/api/2.33/protection-group-snapshots/transfer?filter=completed and name%3d%27flasharray1%3aaen-sql-22-a-pg.1629%27&names=27flasharray1%3aaen-sql-22-a-pg <no body>

Wrapping Up

Using the FlashArray API and PowerShell, we can effectively monitor and manage replication between Pure Storage arrays. This script provides a straightforward way to ensure that your replication processes function correctly, giving you peace of mind that your data is consistently and reliably duplicated across arrays.

Following these steps, you can quickly adapt and extend this demo to suit your specific replication monitoring needs, ensuring data in your storage environment is always in sync and up to date.

Pure Storage PowerShell SDK2 Blog Series

This article is part of a blog series covering the Pure Storage PowerShell SDK2. Check out the other posts in this series:

You can find the supporting code for this blog series at this GitHub Repo, and you can watch a webinar Unlocking the Full Potential of Pure Storage with APIs that walks through all of these demos.