Saturday 5 April 2014

vCenter 5.5 U1 vs Server 2012 Upgrade to R2 MU1

Thought I'd snapshot my Lab's vCenter running 5.5 U1 and upgrade the O/S from 2012 to 2012 R2 MU1 which was just released. Don't try this on Production folks!

I took a snapshot and kicked off the upgrade. After the final reboot, all the services started up except the vSphere Web Client. I tried the C# Client but that wouldn't connect either. I turned off the Firewall to see if that helped as the main vCenter service had started and I still got a connection failure. Then I noticed that the IP Address had changed, it was using DHCP. I changed back to the original static address, enabled the Firewall again and rebooted. Now the C# Client worked but the web client service was still not responding.

I did a repair install (remove/install) of the Web Client to see if that would help. This got the Web Client Service into a running state finally and after a few minutes I could connect via the Web Client Interface.

I had the same trouble as this with 5.5.0b and was hoping 5.5 U1 would get around it but obviously not. For Production Servers you have the choice of trying the steps above with good backups in place (!) or build a fresh vCenter on 2012 R2 MU1 and migrate the Database, SSL keys etc.

Note: By MU1 I mean the maintenance update released by Microsoft for Server 2012 R2 in April 2014. There is a set of 6 patches for existing 2012 R2 systems or you can give it the integrated ISO which is the method I used above to upgrade from 2012.

Thursday 3 April 2014

vSphere 5.5 observations

Well, as I mentioned in another post, I've finished a vSphere 5.5 deployment. Just thought I'd jot down a few observations. Disclaimer: these are based on my own opinion and do not represent the views of any Company I've worked for past, present or future.

I found the new web interface a little daunting at first, I tried to do everything possible with the it rather than drop back to the C# client.So in no particular order:

The refresh rate is a little slow which is annoying, when you power off a VM it doesn't instantly update and I find myself forcing a refresh several times a day.

Drag and drop works nicely but you have to wait until it's finished as it removes the object temporarily and then puts it back which can screw up your next selection!

Great working with the new SSO, no Database required, didn't give any trouble at all.

Tried using the VCSA but it gave a LOT of trouble so after two rebuilds I called time and reverted to the Windows version. It's just not ready for the big time outside of Labs / Test / Development, again, my opinion only. The web client was more unstable with the VCSA than with the windows one.

The Notes field doesn't show custom attribute data so it's either input what the VM does or have Veeam input the latest backup job information. Tags don't cut it for dynamic data like that. You can use Custom Attributes but the web client won't display them. You can get a community written plugin that does however.

Use the latest vCenter build you can get provided your plugins work with it. There's many, many bugfixes in the newer versions, especially over the last year and multiple vCenter releases. Bit concerning but at least they are trying to address it. One outstanding issue is you can't deploy OVFs with the web client, you have to drop back to the C# Client.

Tip: Deploy vCenter and SQL VMs with Hardware version 9 and keep a standard switch handy with no physical uplinks to allow you to restore vCenter back onto the network where you are using Distributed switches exclusively. Remember the C# Client can't edit VMs with a hardware version of 10, hope your PowerCLI skills are good in that case when vCenter is down and you've to restore from backup!

The console plugin doesn't launch great in Firefox, it stalls the browser completely for about 10 seconds. When doing testing using multiple VMs, drop back to the C# Client, console is a little better there. Or try a different browser?!

All the third party plugins are pretty Cool, I've used HP Insight Control v 7.3 (it Rocks! Especially for Storage visibility), Veeam is good also - last 7 days of backup history on tap.

Sometimes you right click on a VM and the options you expect to be available aren't. Try the Action menu at the top instead. Annoying especially when it's due to a refresh lag.

Filtering Advanced ESXi Host settings is very handy, looking for words like "Scratch", "BPDU" or "PDL" makes it very easy to configure or check settings.

Not using Tags to date but found Datastore Clusters interesting. When they overlap with Storage features however they are less useful.

Still not getting LLDP exposed through HP Virtual Connect but one day.....

Don't forget to backup your DVS configuration! Handy feature but could be placed better or allow automation?





Veeam & 3PAR Integration

I've recently finished a VMware 5.5 Deployment that included HP Blades, 3PAR 7400 and Veeam Backup & Replication 7.0. I was curious how this would work from a SAN backup point of view and I got good experience configuring the integration. One thing I found lacking was a good guide from either Veeam or HP to tie these together. I knew in Theory what needed to happen but it's not listed anywhere I could find so here's some pointers and how I got it to work.

I'm using Datastore based backups, i.e. 1 Veeam Backup job per Datastore so I'm not taking multiple 3PAR snapshots of each Datastore as it sifts through the grouping of VMs. The solution has two 3PAR 7400's, a virtual Veeam Enterprise Manager, two physical Veeam Proxy Servers all running 7.0 with Patch 3. The Veeam Proxy Servers have 10Gb Nics and 8Gb Fibre. The backup storage is iSCSI, the Fibre is used to connect to the 3PAR Storage where the VMs reside and a Tape Library for good measure. So far, so fun, let's see where configuration is needed:
  • SAN Switch Zoning - I examined one of the existing Blade Server Zones and used that as a template. There are two SAN fabrics, each needed two Zones set up for each Proxy Server. Proxy Server HBA #1 is zoned with one SAN Controller and then separately to the second SAN's Controller. Then HBA #2 then zoned to the same way to the other controller on each SAN. 
  • 3PAR - The Proxy Server can now log in to the SAN Fabric and you should see the WWIDs appear in 3PAR. Define a Host using both sets of WWIDs. You need to do this on each 3PAR.
  • Double check the Proxy O/S is not set to automount. Veeam disables this by default but recheck.
  • Now you can configure a test backup job, you need Veeam B&R Enterprise Plus to do this remember, check your license. Enable SAN Based Backups on the test job.
During the Backup window you'll see temporary snapshot volumes being presented to the Proxy Server in Disk Manager, these disappear after a time. At no stage do you need to export the live Production LUNs used for VMware to the proxy server. 3PAR snapshots the LUN and presents this so it's a safe configuration and fast.

Restores have to be carried out over the network. I've seen backups processing 100's of MB/s but restores typically are in the region of 50 MB/s even with 10Gb network speeds. So a 200GB VM could take 2 hours to restore but that's typical of most backup / restore comparisons (it takes twice as long to restore than to backup).

I had one issue where some Datastores would use SAN snapshots and others wouldn't. When I spent some time looking at it I found it was working for SAN #1 but not SAN #2. Turned out I had defined the Host on SAN #1 but forgotten to do it also on SAN #2. Easy fix!

Get your SAN administrator to do the Zoning & 3PAR piece if you're not comfortable as it can be tricky. The Server zoning was already done by an expert so I just copied their naming convention and config to extend it to the Proxy Servers which made it handy!

Tape Integration is next, so far so good, basic but it works nicely. I tested instant restore also which worked like a charm and used Storage vMotion to relocate the VM afterwards.

I really like Veeam, they deliver great software and it's very usable from an Engineer's point of view. Probably one of the last product out there I can say that about!