I recently came across a need to review the Storage Policies in use within a vCenter environment and how many objects or virtual machines where using each policy.
I saw this as an excuse to refresh my PowerShell skills and wrote a quick function.
Source code can be found on my GitHub, here. Check there for any updates but below is the code at the time of writing.
function Get-vSANSPSummary {
<#
.SYNOPSIS
Export vSAN Storage Policy Information.
.DESCRIPTION
Export vSAN Storage Policies from vCenter showing FTT & Stripe information and amount of amount of VM's using each.
.PARAMETER ExportPath
Path the export the output HTML file.
.NOTES
Tags: VMware, vCenter, SPBM, PowerCLI, API
Author: Stephan McTighe
Website: stephanmctighe.com
.EXAMPLE
PS C:\> Get-vSANSPSummary -ExportPath "C:\report\vSAN-Storage-Policy-Summary.html"
Outputs a HTML file containing the Storage Policy Information for vSAN Storage Policies to a specified location.
#>
#Requires -Modules VMware.VimAutomation.Storage
[CmdletBinding()]
param (
[Parameter(Mandatory)]
[string] $ExportFilePath)
Begin {}
Process {
try {
$Output = @()
$vSANstoragepolicies = Get-SpbmStoragePolicy -Namespace "VSAN"
$SPBM = $vSANstoragepolicies | Select-Object Name, AnyOfRuleSets
ForEach ($SP in $SPBM) {
$Attributes = @( $SP | ForEach-Object { $_.AnyOfRuleSets } | Select-Object -ExpandProperty AllofRules)
$object = [PSCustomObject]@{
SPName = $SP.Name
ObjectCount = $ObjectCount = (Get-SpbmEntityConfiguration -StoragePolicy "$($SP.name)").count
VMCount = $VMCount = (Get-SpbmEntityConfiguration -StoragePolicy "$($SP.Name)" | Where-Object {$_.Entity -notlike "hard*"}).count
RAID = $attributes | Where-Object { $_.Capability -like "*VSAN.replicaPreference*" } | Select-Object -ExpandProperty Value
FTT = $attributes | Where-Object { $_.Capability -like "*VSAN.hostFailuresToTolerate*" } | Select-Object -ExpandProperty Value
SubFTT = $attributes | Where-Object { $_.Capability -like "*VSAN.subFailuresToTolerate*" } | Select-Object -ExpandProperty Value
Stripes = $attributes | Where-Object { $_.Capability -like "*VSAN.stripeWidth*" } | Select-Object -ExpandProperty Value
ForceProvision = $attributes | Where-Object { $_.Capability -like "*VSAN.forceProvisioning*" } | Select-Object -ExpandProperty Value
StorageType = $attributes | Where-Object { $_.Capability -like "*VSAN.storageType*" } | Select-Object -ExpandProperty Value
IOPSLimit = $attributes | Where-Object { $_.Capability -like "*VSAN.iopsLimit*" } | Select-Object -ExpandProperty Value
}
$Output += $object
}
$Output | ConvertTo-Html -Property SPName, VMCount, ObjectCount, RAID, FTT, SubFTT, Stripes, ForceProvision, StorageType, IOPSLimit | Out-File $ExportFilePath
}
catch {
Write-Host "An error occurred!" -ForegroundColor Red
Write-Host $_ -ForegroundColor Red
}
}
}
Output currently as a basic HTML table but you could change this to add some ‘HTMLness’ or output to CSV.
As always, thanks for reading and I hope this has been useful to someone.
If you like my content, consider following me on Twitter so you don’t miss out!
As technology moves forward, more and more ways to achieve your goal become available. Many people still rely on the good old trusty GUI to achieve their goal, I know I do at times. Is this because it’s quicker, more comfortable or familiar? Or perhaps because they don’t realise there are other options out there!?
This blog post will be one of many, where I highlight some of the options available for completing various technical tasks or configurations, in the hope it can provide additional options or tools for consideration.
To kick off, let’s take a look at a common example for a vSphere Administrator, creating Port Groups on a Distributed Switch.
vSphere Client
So let’s first look at the process via the GUI, in this case, the vSphere Client. I wont go into too much detail on the steps involved, as it is a well documented process, but the screenshots are below:
Repeat for the remaining Port Groups and you will be left with the finished article.
And there we have it, three Port Groups on a distributed Switch. Now, imagine doing this for 10’s or 100’s of Port Groups? It’s going to be slow and painful, so let’s look at some other options.
PowerShell
Firstly, PowerShell, specifically the VMware PowerCLI PowerShell module. Here is an example script that will create the same three Port Groups that we did using the GUI:
So lets break down this code. Firstly we are defining some variables;
$vDSName – This is the name of an existing virtual distributed switch in which you will be creating your Port Groups.
$Ports – This defines the number of ports the Port Group will be initially configured with. (By default 128 ports are created, there is nothing wrong with using the default, see the note further down as to why I have specified 8.)
$LoadBalancing – This is the load balancing policy I wish to set for the Port Group. Available options are:LoadBalanceLoadBased, LoadBalanceIP, LoadBalanceSrcMac, LoadBalanceSrcId, ExplicitFailover. This can be adjusted as required.
$ActiveUP – This variable defines the uplinks you wish to set as active for the Port Group. (If you want to add standby uplinks, you could add this parameter in too)
$VDPGS – Finally, this is an array containing both the name and VLAN ID for each Port Group.
Now we have our input information in variables, we move onto the next two lines of code. These are within a ‘ForEach Loop’. This will take each entry within an array and run a block of code against it. In this case, each Port Group we wish to create.
So for each entry in the array, ‘Get-VDswitch -Name $vDSName‘ gets the existing Virtual Distributed Switch based on the variable and then pipes (‘|’) this into the command (New-VDPortGroup -Name $VDPG.PG -VLanId $VDPG.VLANID -NumPorts $Ports) to create the Port Group on the Distributed Switch, using the properties set for each line of the array.
Secondly, we get the Port Group we just created (Get-VDswitch -Name $vDSName | Get-VDPortgroup $VDPG.PG) and then ‘Get & Set’ the Teaming and Loadbalancing options (Get-VDUplinkTeamingPolicy | Set-VDUplinkTeamingPolicy -LoadBalancingPolicy $LoadBalancing -ActiveUplinkPort $ActiveUP), again ‘piping’ the results into the next command.
Below is the output from PowerShell after running the script above:
Now let’s take a look at using Terraform to achieve the same result. Terraform is an infrastructure and code tool used to manage infrastructure in the form of configuration files and state:
provider "vsphere" {
vsphere_server = "vCenter Server FQDN"
user = "Domain\\Username"
password = "Password"
}
data "vsphere_datacenter" "datacenter" {
name = "dc-smt-01"
}
data "vsphere_distributed_virtual_switch" "vds" {
name = "vDS-Workload-Networks"
datacenter_id = data.vsphere_datacenter.datacenter.id
}
resource "vsphere_distributed_port_group" "pg20" {
name = "dvPG-Guest-VM-1"
distributed_virtual_switch_uuid = data.vsphere_distributed_virtual_switch.vds.id
number_of_ports = 8
vlan_id = 20
}
resource "vsphere_distributed_port_group" "pg21" {
name = "dvPG-Guest-VM-2"
distributed_virtual_switch_uuid = data.vsphere_distributed_virtual_switch.vds.id
number_of_ports = 8
vlan_id = 21
}
resource "vsphere_distributed_port_group" "pg25" {
name = "dvPG-Secure-VM-1"
distributed_virtual_switch_uuid = data.vsphere_distributed_virtual_switch.vds.id
number_of_ports = 8
vlan_id = 25
}
Lets break this down.
First we are specifying which terraform provider we want to use, this will be the vSphere provider in this case. We are then providing some parameters for Terraform to connect to your vCenter instance; VCSA FQDN and credentials.
We then have two ‘data’ blocks. These are used to get information about an existing resource, such as the Distributed Switch and the Datacenter it resides in. You could loosely consider this similar to populating variables in the PowerShell example.
Next we have three ‘resource’ blocks. Each block represents one of the three Port Groups we want to configure. It provides parameters for Name, number of ports and vlan ID for each, along with a reference to the Distributed Switch from the ‘data’ block.
Now when you run ‘terraform apply’ to apply for code, here is the output:
terraform apply
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# vsphere_distributed_port_group.pg20 will be created
+ resource "vsphere_distributed_port_group" "pg20" {
+ active_uplinks = (known after apply)
+ allow_forged_transmits = (known after apply)
+ allow_mac_changes = (known after apply)
+ allow_promiscuous = (known after apply)
+ auto_expand = true
+ block_all_ports = (known after apply)
+ check_beacon = (known after apply)
+ config_version = (known after apply)
+ directpath_gen2_allowed = (known after apply)
+ distributed_virtual_switch_uuid = "50 33 5e 01 05 1e 32 66-ea f7 7c 42 ce fa f1 96"
+ egress_shaping_average_bandwidth = (known after apply)
+ egress_shaping_burst_size = (known after apply)
+ egress_shaping_enabled = (known after apply)
+ egress_shaping_peak_bandwidth = (known after apply)
+ failback = (known after apply)
+ id = (known after apply)
+ ingress_shaping_average_bandwidth = (known after apply)
+ ingress_shaping_burst_size = (known after apply)
+ ingress_shaping_enabled = (known after apply)
+ ingress_shaping_peak_bandwidth = (known after apply)
+ key = (known after apply)
+ lacp_enabled = (known after apply)
+ lacp_mode = (known after apply)
+ name = "dvPG-Guest-VM-1"
+ netflow_enabled = (known after apply)
+ network_resource_pool_key = "-1"
+ notify_switches = (known after apply)
+ number_of_ports = 8
+ port_private_secondary_vlan_id = (known after apply)
+ standby_uplinks = (known after apply)
+ teaming_policy = (known after apply)
+ tx_uplink = (known after apply)
+ type = "earlyBinding"
+ vlan_id = 20
+ vlan_range {
+ max_vlan = (known after apply)
+ min_vlan = (known after apply)
}
}
# vsphere_distributed_port_group.pg21 will be created
+ resource "vsphere_distributed_port_group" "pg21" {
+ active_uplinks = (known after apply)
+ allow_forged_transmits = (known after apply)
+ allow_mac_changes = (known after apply)
+ allow_promiscuous = (known after apply)
+ auto_expand = true
+ block_all_ports = (known after apply)
+ check_beacon = (known after apply)
+ config_version = (known after apply)
+ directpath_gen2_allowed = (known after apply)
+ distributed_virtual_switch_uuid = "50 33 5e 01 05 1e 32 66-ea f7 7c 42 ce fa f1 96"
+ egress_shaping_average_bandwidth = (known after apply)
+ egress_shaping_burst_size = (known after apply)
+ egress_shaping_enabled = (known after apply)
+ egress_shaping_peak_bandwidth = (known after apply)
+ failback = (known after apply)
+ id = (known after apply)
+ ingress_shaping_average_bandwidth = (known after apply)
+ ingress_shaping_burst_size = (known after apply)
+ ingress_shaping_enabled = (known after apply)
+ ingress_shaping_peak_bandwidth = (known after apply)
+ key = (known after apply)
+ lacp_enabled = (known after apply)
+ lacp_mode = (known after apply)
+ name = "dvPG-Guest-VM-2"
+ netflow_enabled = (known after apply)
+ network_resource_pool_key = "-1"
+ notify_switches = (known after apply)
+ number_of_ports = 8
+ port_private_secondary_vlan_id = (known after apply)
+ standby_uplinks = (known after apply)
+ teaming_policy = (known after apply)
+ tx_uplink = (known after apply)
+ type = "earlyBinding"
+ vlan_id = 21
+ vlan_range {
+ max_vlan = (known after apply)
+ min_vlan = (known after apply)
}
}
# vsphere_distributed_port_group.pg25 will be created
+ resource "vsphere_distributed_port_group" "pg25" {
+ active_uplinks = (known after apply)
+ allow_forged_transmits = (known after apply)
+ allow_mac_changes = (known after apply)
+ allow_promiscuous = (known after apply)
+ auto_expand = true
+ block_all_ports = (known after apply)
+ check_beacon = (known after apply)
+ config_version = (known after apply)
+ directpath_gen2_allowed = (known after apply)
+ distributed_virtual_switch_uuid = "50 33 5e 01 05 1e 32 66-ea f7 7c 42 ce fa f1 96"
+ egress_shaping_average_bandwidth = (known after apply)
+ egress_shaping_burst_size = (known after apply)
+ egress_shaping_enabled = (known after apply)
+ egress_shaping_peak_bandwidth = (known after apply)
+ failback = (known after apply)
+ id = (known after apply)
+ ingress_shaping_average_bandwidth = (known after apply)
+ ingress_shaping_burst_size = (known after apply)
+ ingress_shaping_enabled = (known after apply)
+ ingress_shaping_peak_bandwidth = (known after apply)
+ key = (known after apply)
+ lacp_enabled = (known after apply)
+ lacp_mode = (known after apply)
+ name = "dvPG-Secure-VM-1"
+ netflow_enabled = (known after apply)
+ network_resource_pool_key = "-1"
+ notify_switches = (known after apply)
+ number_of_ports = 8
+ port_private_secondary_vlan_id = (known after apply)
+ standby_uplinks = (known after apply)
+ teaming_policy = (known after apply)
+ tx_uplink = (known after apply)
+ type = "earlyBinding"
+ vlan_id = 25
+ vlan_range {
+ max_vlan = (known after apply)
+ min_vlan = (known after apply)
}
}
Plan: 3 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
vsphere_distributed_port_group.pg20: Creating...
vsphere_distributed_port_group.pg21: Creating...
vsphere_distributed_port_group.pg25: Creating...
vsphere_distributed_port_group.pg25: Creation complete after 0s [id=dvportgroup-2669728]
vsphere_distributed_port_group.pg21: Creation complete after 0s [id=dvportgroup-2669730]
vsphere_distributed_port_group.pg20: Creation complete after 0s [id=dvportgroup-2669729]
Apply complete! Resources: 3 added, 0 changed, 0 destroyed.
For more information on the vSphere provider from Terraform, check out this link.
You will have noticed that I have explicitly defined the number of ports in both the PowerShell and Terraform examples. This is purely to match up with the default value that is set when using the vSphere Client; 8. By default the port allocation automatically expands as required, so this is for consistency rather than anything else.
If you are someone who relies heavily on a GUI as part of your work, I hope this have given you some idea’s on how you can perhaps leverage other options, especially when looking to build or configure in bulk.
Having recently had to do some work with RDM perennial reservations I looked into ways to make this less of a manual headache. There are plenty of examples out there for doing this, which I took as a basis to make a PowerShell function. If anything it was a great way to refresh my PowerShell skills and an opportunity to learn some new skills.
Note: Although this has been tested in my environment, please make sure you test it appropriately before running against a production environment!
Lets take a look…
Get-PerennialReservation
This function targets a vSphere cluster, gets all RDM disks that are connected to VM’s and then queries each host in the cluster to check if the disk/storage device is perennially reserved or not.
There are multiple ways to use it, whether that is by specifying the target cluster using the -Cluster parameter or by piping it from Get-Cluster. You can also specify a specific canonical name or a comma separated string of them, if you just want the status of a single/select disk(s) using the -CanonicalName parameter. There is also an Export flag to export the results to CSV, if you wish to make use of the data outside of PowerShell. You can get the full usage information by running the following command once you have loaded the function into your PowerShell session:
This function again targets a vSphere cluster, gets all RDM disks that are connected to VM’s and sets the IsPerenniallyReserved flag too ‘True’ on all hosts.
There are multiple ways to use it like the Get function; specifying the target cluster using the -Cluster paramater or by piping it from Get-Cluster. You can still specify a specific canonical name or a comma separated string of them, if you just want to set the flag of a single/select disk(s) using the -CanonicalName parameter. There is still an Export function that will provide you an output to CSV. You can get the full usage information by running the following command once you have loaded the function into your PowerShell session:
To complete the set there is a Remove function. This function again targets a vSphere cluster, but this time you need to pass in the canonical name you wish to set the IsPerenniallyReserved flag too ‘False’ for.
To use this one, you need to specify the target cluster using the -Cluster paramater and specify a specific canonical name or a comma separated string of them, using the -CanonicalName parameter. There is still an Export function that will provide you an output to CSV. You can get the full usage information by running the following command once you have loaded the function into your PowerShell session:
Recently I needed to build out some test Active Directory Forests that resemble production in order to complete some testing. One of the forests contained a significant amount of OU’s that I had no intention of manually recreating.
To run the New-ADOrganizationalUnit cmdlet, you need to provide the OU name and the Path where you want to create it. However, Get-ADOrganizationalUnit doesn’t provide the path, so you need to determine it from the DistinguishedName.
After a number of google searches, I couldn’t find exactly what I needed, so I began piecing together various bits of Powershell that I found. I ended up learning a bit of Regex in the process! Powerful tool if you know how to use it.
I came up with two versions in the end, you can see both below with the differences highlighted.
The first one effectively takes everything up to the first ‘,’ and replaces it with nothing, effectively removing the OU Name. The second one captures everything after the first ‘,’ and replaces the whole string with what was captured. Both have produced the same result in my scenario, but it was useful to understand both methods for future use of Regex.
Both also have a property called ‘OUNum’, this property counts how many time ‘OU=’ appears in the original DistigushedName string. OU’s need to be created in order, so that the parent OU exists before the child. This orders the OU’s in ‘tiers’ before exporting them to CSV. All OU’s in the root of the directory will get a value of 1, OU’s within these will get a value of 2 and so on. Credit to ‘David Z’ for this bit!
Once you have your data, you may or may not need to modify the domain. If you are importing it into a different domain, you’ll need to. In my case it was simple enough to do a find and replace in a text editor (eg. DC=lab,DC=local to DC=lab2,DC=local). You could look at using concepts from above to achieve this before exporting the data if you so wish.
Now you have your data, you need to import it. You can run the following in the target domain.
Thanks for coming back! If you missed the first post in my Home Lab series you can find it here.
In this post I will begin drilling into the equipment and software that makes up my Home Lab and my reasoning for these choices.
I’m going to skip the original Raspberry Pi, there are enough blogs covering the use cases for them and begin at the first significant device; my MacBook Pro (late 2013). I wanted something mobile to start with so I could take it to work, use it on commutes to other offices etc. The MacBook came with an Intel i7 2.3Ghz Quad core chip, 16GB of memory and a 512GB SSD. This wasn’t going to be able to run everything, but its enough to run what I need when I’m away from home.
Before I dive into the VM’s and nested hosts, lets look at the networking configuration I used in VMware Fusion. I created four custom networks in total. One being a Management network for my ESXi Hosts and my VCSA. The second for vMotion and the other two as guest VM networks. None of these networks are NAT’d or have DHCP enabled, however I have selected the ‘Connect the host Mac to this network option in the VMware Fusion Preferences for the management network.
There are two ways to set these custom networks up. The first being the UBER Network Fuser and the second, editing the VMWare Fusion network file. In ‘/Library/Preferences/VMWare Fusion’, you will find the file called ‘networking’.
There are guides already available if you search google for either option so I won’t go into this further. This is the one I used – https://tinyurl.com/y7cjkhky.
Now onto the virtual machines. Running directly on VMware Fusion, I have a Windows Domain Controller / DNS Server, a PFSense firewall and two 6.7 ESXi hosts. They all use local storage, including the ESXi Datastores. My PFSense virtual firewall provides my layer 3 routing and eventually interfaces with my physical firewall. The Domain Controller/DNS Server is a ‘standard’ deployment, nothing special. The two ESXi hosts are the standard 6.7 image available from the VMUG Advantage subscription. Check out the last post for more on VMUG.
Then within ESXi, there’s my vCenter Appliance and the DR node of my vSphere Replication Appliances. At this point you might be wondering how I have fit this into 16GB of memory…
To start with, I built the first ESXi host with 12GB of memory and deployed my vCenter appliance on this (the tiny appliance requires 10GB). Once I had successfully deployed the appliance, I reduced the vCenter memory to 6GB and then followed this by reducing the ESXi host to 7GB.
I then created the second ESXi host, which also has has 7GB allocated. Its a tight squeeze but it allows me the basics I need when I’m not at home and there’s still enough room for some small VM’s with the nested ESXi hosts if needed.
One final thing… To ease the starting and suspension of this lab, I use the following script that I run from PowerShell on the Mac.
I was asked recently ‘Do we have PowerCLI downloaded?’. Yes, we may, but it could be anywhere and it is likely an outdated version.
There is no need to download the installer! You can install PowerCLI using the Install-Module cmdlet in Windows PowerShell. (Providing you have an internet connection!) Below we will look at the steps required to install the latest version of PowerCLI on your system.
From an elevated PowerShell prompt run the following –
Install-Module VMware.PowerCLI.
If you don’t already have it installed, you will be prompted to install the NuGet Provider. Type ‘y’ and enter to continue.
You will get a further prompt to confirm you are happy to install a module from the ‘PSGallery’. Again, ‘y’ and enter to continue.
The PowerCLI Module will then begin to install. It will cycle through installing multiple dependent packages which will take a few minutes. Sit back and wait…
Once returned to the prompt, you can confirm the installation by running –
Get-Module VMware.PowerCLI -List Available | FL
You have now installed PowerCLI version 12.0.0.15947286. You will likely end up installing a later version.
Last step, load the module for use –
Import-Module VMware.PowerCLI
You’re ready to go! But…
Not every system you need to use this module on will have internet access. In this, case the ‘Save-Module’ cmdlet is your friend.
Save-Module -Name VMware.PowerCLI -Path <Path to directory>
The module will then proceed to be downloaded into the directory you have specified and will look like this –
On your target server, you will need to confirm your module paths. You can do this by using the following command. You may have more than one path.
$env:PSModulePath
Now copy the directory that contains the module you have saved, to a module path on the target server. Likely ‘C:\Programfiles\WindowsPowerShell\Modules’ on a Windows System.
Now the Module is on your system, all that’s left is to import the module as above –
Import-Module VMware.PowerCLI
Thanks for reading! Hope this has been of use and catch you in the next post.
Stephan McTighe 