Monday 2 July 2018

Enhanced Linked Mode with Embedded PSC

Enhanced Linked Mode with Embedded PSC


The latest 6.5 U2 and 6.7 release support enhanced linked mode when used in a greenfield site. I was a bit confused over how this is done so decided to lab it just to see the steps required.

"With vCenter Server 6.5 Update 2, you can deploy by using the GUI or CLI installer up to 15 vCenter Server Appliance instances in Embedded Linked Mode, and manage these instances with the vSphere Web Client or vSphere Client from any of the instances."

So start with I'm using 6.5 U2 VCSA ISO and the deployment is in two stages as per normal. I choose embedded PSC:
 I wait for the deployment to process:
 Now for Stage two:
 Standard Stuff so far:
This is the key screen. You can setup the first VCSA with a new SSO. Later I'll join an existing SSO domain on my second VCSA deployment and see what happens.
Here is my vCenter up with a test host added:
Now I'll deploy the second vCenter and same story for part 1. Now I've just spotted an issue from the release notes:

Second node in Embedded Linked Mode might be deployed in a new site with the default site name regardless of the first node configuration
If you configure a vCenter Server instance in Embedded Linked Mode on a site with a name of your choice and then you add another node, the second node might be deployed in a new site and get the a  default name, regardless of the first node configuration. This issue is specific for the GUI installer of vCenter Server Appliance and not for the CLI installer.
Workaround: None. Site names in vCenter Server Appliance configured in Embedded Linked Mode by using the GUI installer must default to Default-First-Site and not be modified. 
Oh, great, stick with the site name defaults then. Too late for me, let's see what happens!

I've grabbed the following screenshot from the second part of the deployment phase which shows me joining the existing SSO:
Hmmm...no sign of being able to choose a different site name so I'll need to check both appliances after the second is finished to see what the two SSO site names end up as!!! Will ELM work or not, that's the question!!
So, here we are - looks ok - ELM see's both vCenters and VMs.
This is labvc01:
 It has the custom site name

This is labvc02:
This one defaulted to the default site name as per release notes.

Now, in vSphere 6.7 Sites are deprecated, i.e. no more sites so no worries. For now I would leave the Site to default and leave it well alone! The issue with the settings above is if I wanted to point vCenter at the other PSC in a DR scenario, as it's in a different site, that's not possible!!
https://communities.vmware.com/thread/587203

Now, let's try again and this time stick to the default site name:
 I tried leaving the two fields blank but you're forced to write something into them. Get this - the default site name shown is WRONG! Don't type in "default-site", type in "Default-First-Site" as shown below:
Validate with these commands (enable SSH during install!):
https://kenumemoto.blogspot.com/2017/06/vcsa-65-how-to-find-which-psc-your-vcsa.html

To find your SSO Domain Name:
/usr/lib/vmware-vmafd/bin/vmafd-cli get-domain-name --server-name localhost

To find your SSO Site Name:
/usr/lib/vmware-vmafd/bin/vmafd-cli get-site-name --server-name localhost

To find you which PSC your vCSA is pointing to:
/usr/lib/vmware-vmafd/bin/vmafd-cli get-ls-location --server-name localhost
So far, so good. Now let's look at the second one:
So, same settings as before and after it's finished we can check the site info using putty:
All looks fine now:
Once those Site Names match you're good to go.....Hope this saves someone a few minutes out there!

Tuesday 27 February 2018

vRealize Operations Musings

vRealize Operations Musings


This post is a quick delve into the world of vRealize Operations. Let me state up front I'm not a fan but I thought I'd try and trace why and have a fresh look at the product to find some good points that would help balance my perspective a bit!

I was responsible for the delivery of Microsoft System Center Operations Manager back in the day for about a year on and off. I thought then and still do that it's over-engineered. You can tell from some products if they were designed well and intuitive to use, and others where nothing is how you'd expect it.

I read a book a few months ago about design theory, related to doors and things like that. A glass door with no obvious handle will confuse people as to which way it opens and via which side, cue broken nose or broken glass everywhere. My opinion is SCOM is like that. VMware vRealize Operations echoes that feeling to me but is no way as bad, but still it's not in a good place compared to other software products I work with which is a shame.

To give you an example - cue how to add a vCenter into vRealize Operations:

When you first log into vRealize and the dust is settling you see this:
Ok, I thought, where do I add my vCenter. I click on the plus and get this:

What's a PAK file?!! Do I need one? That's what I mean. You have to highlight the VMware vSphere solution on the previous screen, then click on the configure cogs button to get this:
Intuitive? Hardly! 

I've already added mine here, seems ok? Wait until you get to the credentials section, type in administrator@vsphere.local and have a laugh to yourself! You have to add a credential - see the plus beside the credential area to get this:
Ok, so it's not a big thing but these two items are enough to trip me up for 5 minutes, figuring out where and why. I know there's documentation but if you've worked in IT for a few years is it too much to expect that you will get a more intuitive start to the product. This is key as this is the impression that will stay with you. I know, it's stayed with me...!

It's that kind of structure that determines how much you're going to like playing with a product long after you've gotten it working the way you want. Take reports as an other example. You can output basic inventory reports, I did so and got blank, zip, nada detail in them:
So I chose to get a Hardware Summary. You need to highlight the report, THEN click the play button on the taskbar (no right click here) - again why is it designed this way, the second from last icon?!!!!! WHY????!! Then you get to choose from the following:
So, the defaults look good, let's go with vSphere World, right? This is the result:
There are two problems here, the lack of data and the formatting. Let's fix the data first, you need to run the report but choose THIS fecker from the non default drop down list:
So instead of the default "custom groups" at the top, I chose "all objects" and then pointed the dozy product at my only connected resource, my vCenter Datacenter. Not exactly hard but completely irrational for a VMware Product. This is the result:
I love the thought that went into the header, footer, 1st page VMware logos and index but there's one small problem, the 40 pages of content looks like the above. Now, you do have a CSV option but we all know that requires a little massaging. If I wanted to schedule a report to email itself weekly, this one isn't going to do it for me. I'd have to edit the template and only add sufficient fields to fill the width. There's no obvious formatting option to correct this so it's pretty much useless. One of the most basic reports I could think of too. Get RVTOOLS here by the way, much better:
Now, about scheduling useful reports.....you can email them and save them to a network share, so that's something. I setup saving them to a share and you can choose weekly or monthly but that's it, no choice over the format either?! You can choose the start hour but not the minute, try testing your settings once per hour that way, see where I'm going?! No way to manually test the schedule either. Still think an Engineer designed  this?!! No idea if it works or what the format is - I gave up here.....!!

I did spot one nice thing when I was configuring the vCenter connection, that is to check compliance against the vSphere Hardening Guide. Brilliant! 
Define Monitoring Goals is one of the optional sections when setting up the vCenter connect - it defaults to No, even after you save it (!) but I am getting alerts for my Hosts and VMs against the hardening guide settings which is very useful. Thought I'd need configuration manager for that but at least it's a bonus here. 
So, that's one thing I'd definitely find useful beyond Host Profiles and the battle I have with them regularly! There's an associated report, great, let's run it!
The format is better at least but look at the number of pages - 236!!! Lol. So not that useful then. Like host profiles I'd be better off using scripts etc to do the hardening (unless you've bought vRealize Configuration Manager) and using this report to audit compliance........ 

There are good growth forecast reports which are useful, I'm not going to be running vRealize in my lab long enough to see the benefit but capacity planning is a strength and good to have. 

One other thing, when you run a report, once it finishes the screen refreshes so you've to scroll down AGAIN to find the damn thing unless you use a filter first. Annoying, simple for someone to fix but I doubt it ever will be.....

Here's a report on reclaimable space:
No percents listed, I've several templates but it's not clear if this means it didn't detect them or as they are thin provisioned there's nothing to reclaim off them. 

I love this report:


Very Graphic, clear and precise and tells us absolutely nothing....!

Now I think you can blame the user on this one, I'm sure after some additional knowledge & upskilling, training and time the product would deliver everything VMware raves about but a door is a door, put a handle on it and don't waste my time searching for things that should be obvious. Defaults should work for the most common use cases, like vCenter objects in reports. Maybe they intended to use this with a much wider palette than vCenter but if that's your core base, play to them first. This doesn't feel like a product designed by VMware to work with vCenter, doesn't that sound wierd to you? Like they wanted to connect this to all your physical servers too (there are agents for that) and suck in everything here. 

I finally decided to find the best view to put on a TV in an Ops room and this was it:
Not too bad. Given time it should provide more detail than my labs shows here and you can customize these (Advanced or Enterprise only) to your needs a bit more. There's a good blog post here on reports via this newer HTML 5 interface in 6.6:

I've not yet found a good resource about creating custom dashboards in 6.6 as all the articles are on the previous version and no longer valid. 

So, it's all down to if you're already bought into vROps via a licensing deal or do you have flexibility to look at other solutions. I'm very fond of Veeam One but whatever you choose / end up using you need to ensure it delivers sufficient quality information and no more, otherwise a spammed inbox isn't going to get any attention. 

Use the VMware Hands On Labs to look at this product or download a trial of the OVF like I did. I hope you find this useful and get the right solution for your Organisation. Best of Luck! 















Tuesday 20 February 2018

Bring out the VVOLS...!!

Bring out the VVOLS...!!


I had a chance to play around with VVOLS today and to see how they've improved since my last encounter. Now, I don't have a physical SAN to play with so how do you get VVOLs in a Lab?! I was hoping StoreVirtual would have advanced by now to support VVOLs but that hasn't happened so I've been scouting around for a replacement that would offer VVOLs but not require dedicated hardware, i.e. a virtual appliance. There are a few around but they can be hard to get. My solution was to use Nimble. Now, the appliance isn't any easier to get hold of so good luck there! I'd recommend trying the NexentaStor with a trial license if you're stuck.

Between shutting down my Lab one day and starting it up the next I ran into an issue where the previous VVOL I'd created wasn't accessible. While I got a REALLY good look at the CLI and deep dived to troubleshoot I found two things:

There is no ESXi CLI command to remove a troubled VVOL Storage Container:

esxcli storage vvol storagecontainer
Usage: esxcli storage vvol storagecontainer {cmd} [cmd options]

Available Namespaces:
  abandonedvvol         Operations on Abandoned Virtual Volumes.

Available Commands:
  list                  List the VVol StorageContainers currently known to the ESX host.

Yep, just list, darn it! I was able to list the fe*ked container but could do nothing with it! 

Second, sometimes it's better to start all over again. I deleted what I could in VMware, removed the iSCSI Target to the Nimble appliance and unregistered the Nimble management interface with vCenter which also pulls out the VASA provider with it. Then rebooted the ESXi host as it was still showing up the Protocol Endpoint. If in doubt, start over! Then I got somewhere!

The Nimble appliance has an interface that sets up the VASA Provider and Web Plugin here:

The Thick client isn't going to help you with vCenter 6.5 but it's there anyway for older versions. I'm on Nimble OS 4.5 and they've since released 5.0 just to note. 

So in vCenter you can check your VASA Provider by bringing up the vCenter configure tab as shown here:
 This is with the Lab Group selected:
This area is very important as you may need to kick, sorry, refresh the VASA provider when you're pulling you hair out. You can choose a rescan or sync option. The Nimble logs as in /var/logs/nimble if you're bored and want to tail something. 

Here is the iSCSI view of the PE (Protocol Endpoint) after I'd created a VVOL folder in Nimble:

You can see the 512 Byte volume on LUN 0 which is the PE. The other Targets are my offline StoreVirtual, ignore those! 

Now you can create a VVOL and hope everything works as expected. If not rescan Storage, rescan Adapter etc etc. I created a small VVOL and Storage vMotioned a CentOS VM onto it:

I took a snapshot and then viewed the VM through the Nimble interface. The snap took 10 minutes to complete but that's most likely due to my Lab. But the Veeam backup snapshot only took a few seconds! Here's the interesting pictures:

This is the contents of the VVOLS1/CentOS folder:

Here is the view via Nimble:
What's surprising to me is that there's only 5 files.....even after a snapshot is taken. The memory snapshot is listed but not the disk snapshot. And what about all the other files?!!

Here is Veeam backup job:

Here are the available storage policies that Nimble makes available:








So you can choose to expose encryption and deduplication options to vCenter Admins. 

By the way Google "vSphere Virtual Volumes Technical Overview" - there a paper dated January 2018 worth reading. Or try this link if it works:

Next have a look at what happens after I take a second snapshot that includes memory (and it still took 10 minutes):


There's the two snaps, 1.6GB each: 
As third snap, taken without the VMs memory makes NO change to the Nimble view. 

So, according to the PDF there are meant to be 5 VVol objects:

(Taken from the VMware PDF referenced earlier)

So what CAN we see? The Config VVol is the first one listed in Nimble, then you have the Memory-VVol, the two Snapshot VVols are only present when you choose to snap the memory and the Data-VVol is the vmdk as listed. The last VVol is the Cluster HA folder so it's nice to know it can use VVols too! 

Now, about those Snapshots. Why don't we see these in Nimble? Well, we do, kinda - watch the last column on the right:

I can also create a local snapshot directly on the Nimble. You can view all the snapshots here. Note it doesn't expose the Nimble snapshot up to vCenter:
Now, let's delete it and restore using Veeam! 
And we're back up and running:
Snapshot free of course! Thanks Veeam!

Now to create a Clone - I can power on the CentOS2 clone and view the space usage. Nimble appears to take care of the DeDup and uses a small snap:
 There 7.4MB snap is shown below:
CentOS2 otherwise uses NO space! Nice!

So, I've had a little play and done the basics. Replication / DR would be the next level but I'm not going there yet. It's more seamless than I thought and maybe Storage Policies are the way to go? With everything moving to flash, with dedup and encryption as standard is there much left to choose apart from the protection level / replication? 

Hope this gave you an insight into VVols and you found it useful.