Wednesday 27 July 2016

vSphere Upgrade 5.0 to 6.0 - Part 3

vSphere Upgrade 5.0 to 6.0 - Part 3


This is the list of services I found on 5.0 from vCenter (top) and SRM (below):


Check you've stopped them all - then we can proceed to SQL. Start by backing up the three SQL Databases to a handy location (I've used C:\Temp):
Next, configure the backup for each as follows, giving each a different name:
Then you should end up with the following files:
Copy them to the new SQL 2014 server:
Restore the Databases one at a time by choosing the "Device" option and browsing to the file:
Do this operation separately for each Database:
Then when they are restored you can edit each to change the compatibility from 100 to 120:
Finally recreate the SQL logins if you are using those or grant permissions to the domain service accoutns if not. Assign DB ownership etc as before. We have to install vCenter before we can recreate the rollup jobs for SQL Agent. 

Next go to the new vCenter and configure the DSN's for vCenter and Update Manager and test the connection for each of them:
Both components are still not 64-bit in this edition of vCenter, see below. Note that the Native Driver is used NOT the ODBC one for SQL 2014:

Now, let's try installing vCenter 6.0U2 and pointing it at an existing database and see where we get! 

So - this is new VM, Server 2012R2, DSN pointing at SQL 2014 SP1 but with restored copy of vCenter 5.0 Database, just to be clear!! 

Note: the vCenter 6.0 installer checks there are TWO vCPUs for vCenter, make any necessary adjustments to your lab. 

Here we go, install External vCenter:
Next confirm the vCenter system name: 
 Give it the Platform Services Controller FQDN:
 Click OK: 
 If you're using the right native driver you'll be able to pick the DSN and enter the credentials below:
 Now see what mess I've gotten myself into...!!!!
SO......this approach will not work. I'll have to perform a vCenter upgrade in place on Server 2008R2 and SQL 2008R2 and THEN perform the replatform and resql tasks afterwards.....darn it! Time to startup all those services again...! I'll cheat and just reboot the 4 legacy VMs..! Don't forget to clean up the databases on SQL 2014. 

Don't forget vCenter 6.0 installer checks for TWO vCPU so if you're using a lab increase them now. I've also increased the RAM to 8GB to ensure smooth sailing....

So here we are again....this time performing an in place vCenter upgrade on the old 2008R2 Server VMs: 

Once you run the installer it immediately recognises it's performing an upgrade:
 You select the external model and we've already our PSC's ready to rock and roll:
Enter the Administrator password and Click OK regarding the linked mode warning shown below:
 Enter the PSC FQDN and password:
 Click OK on the certificate validation warning:
 Make any changes you need to the destination paths, I'm in a Lab so I don't care...!!
 Tick the box and sign your life away......
Click Upgrade to process the VM and transform it to vCenter 6.0. This is a clean and pristine install of vCenter 5.0 so I'm not expecting issues but I've read about a few production systems that ran into difficulty so ensure you have good backups of EVERYTHING if you're doing this live. 
So we have a web interface we can log into and let's compare views before we upgrade our second vCenter 5.0:
 There is no sign of the other vCenter anymore in either view so linked mode is down until I upgrade the second vCenter.

So no sign of SRM plugin in the web interface of course, I checked the C# client after running the 6.0 client installer, it's still there but doesn't work. One thing I found was trying to login with "lab\administrator" fails but just using "administrator" worked fine. This was in both the Web and C# client. Probably something I need to fix in the SSO. Log in as administrator@vsphere.local to resolve. As you see there is no lab.local domain listed:
We need to define our AD domain here:
And here we add in the Domain Administrator account as a vsphere.local Administrator:
now, log out and test!

We can also upgrade the Update Manager at this point. 
 It knows this is an upgrade
 Use the FQDN always for SSL certificate use
 Enter the DSN password
 The default here is to NOT upgrade so change it
Ensure the FQDN is selected. The Download Service is an optional component for DMZ or air gapped situations, you're not missing anything with just the core Update Manager install here. 

If we take a look now in the Web and C# client we can see if both Sites are back again. This is using the PSC enhanced linked mode feature:
 The Web Client is ok but the C# client no longer shows the other Site as legacy ADAM linked mode is gone in 6.0 and you have to use the lovely, lovely web client (!) to manage both sides from here on in!!

Note: The configuration change to add in the domain admin user to SSO is only required once as we are using a unified SSO domain (!), so when you check it on Site B you'll see it already is there! 
























Thursday 21 July 2016

vSphere Upgrade 5.0 to 6.0 - Part 2

vSphere Upgrade 5.0 to 6.0 - Part 2


This post takes up where we left off and we're ready to install a pair of Platform Service Controllers, we could do this anytime of course as 5.0 has no clue that SSO exists so we'll just spin these up and do the install ready for the vCenter upgrade itself.

Now is a good time to start using snapshots as I'm trying to figure out how to install the PSCs to get Enhanced Linked Mode working but use separate sites. They say in the KBs this is for load balancers which I'm not after so I'm wondering how this works out. My two installs were configured as follows:

This is the first PSC:
This is the second PSC: 
Now, you only get to name a site in the first PSC yet one architecture mentioned is as follows:


This is a bit annoying, I want the picture above, but how do I get it...?! Ahaa....when you run the install on the second one and click next, and move onto the NEXT SCREEN, you get this:

Now, we're talkin'! I've now got two defined Sites but a single SSO Domain for Enhanced Link Mode - brilliant! They are using self signed cert but we'll get that....!

So, now we're ready to proceed. I've found KBs on moving vCenter to a new Server, relocating the vCenter Database and the usual upgrade approaches but I've not come across anything that combines all of these while preserving settings, changing OS and dealing with security settings. So the question really is is it possible - yes, but how to do it reliably in the least number of steps. 

Note: I was using Remote Desktop Connection Manager 2.2 on windows 10 and after opening more than 5 or 6 connections my desktop would hang. I've upgrade to version 2.7 but lost all my configuration, so back yours up if you're facing a similar scenario! Mine works fine now....I am also getting huge peaks due to .NET security updates doing a optimization sweep after their reboot, so it's a matter of waiting patiently until that settles down.....I only gave my VMs a single vCPU so that's not helping! I'll have to save up for that second physical CPU!!

Now, back to those KBs:
Moving the VMware vCenter Server 4.x/5.x/6.0.x SQL database (7960893)

Updating rollup jobs after the error: Performance data is currently not available for this entity (1004382)

Migrating vCenter Server to a different host machine (5850444) 
(Note: This is for vCenter versions up to 5.5, it doesn't mention 6.0!!)

Manually backing up and restoring the VMware vCenter Server 4.x and 5.x ADAM instance data (1029864)

The first two deal with SQL, the third and fourth deal with vCenter and the linked mode ADAM database (although this is not used in vCenter 6.0). One advantage is we're using vCenter 5.0 which has no SSO to worry about, we've deployed clean and prestine PSC's so the next step really is to deal with vCenter itself. We need do to the following:
  • Upgrade from 5.0 to 6.0
  • Point upgrade at external PSC
  • Preserve all existing settings and Hosts, VMs etc
  • Replatform to Server 2012R2 - new host so new certificates will be required....!
  • Switch Database to new SQL Server - new DSN required here and recreate rollup jobs
  • Upgrade any components such as distributed vswitches, etc to 6.0 mode
There are some restrictions due to compatibility. For instance we know vCenter 5.0 doesn't support SQL 2014 so we can't do that bit until AFTER we upgrade to vCenter 6.0. 

One scenario I wondered about was if we move the database anyway and installed vCenter 6.0 would pointing it at that moved vCenter 5.0 Database trigger an upgrade? What else would we have to pre-copy over to do the install on the 2012R2 vm rather than install on 2008R2 on top of vCenter 5.0 and THEN move it?! 


So, to prep, we install the SQL Native Client from SQL 2012 SP3 Feature Pack, as the "Microsoft ODBC Driver 11 for SQL Server - Windows" doesn't work with VMware!!!:

This goes on both new vCenter Servers running Server 2012 R2. 

The last thing I will do with the old environment is to load up vCenter and check everything is working ok. In my case I'd forgotten to power up the Storevirtual lab so all my VMs appeared disconnected!! Dooh. I just want to make sure I'm coming from a stable environment before shutting things down and performing partial upgrades. 

So, I've created an Update Manager baseline, added a 5.0.0 distributed vSwitch attached to a spare virtual nic on my nested esxi hosts and checked SRM is ok and now we're good to go! 

So, to begin with I'll stop all the VMware services on the legacy vCenter and SRM VMs. then move the vCenter and Update Manager databases to SQL 2014. Stay tuned for my next post and I'll let you know how I got on!



Wednesday 20 July 2016

vSphere Upgrade 5.0 to 6.0 - Part 1

vSphere Upgrade 5.0 to 6.0 - Part 1


So, these posts are around upgrading from vSphere 5.0 to 6.0U2, quite a big jump! I'm testing this is a Lab to see if I hit any issues or gotchas. The steps also involve changing from Server 2008R2, dealing with certificates and keeping vCenter historical data and SRM intact along the way. So, reasonably complex and I've heard of others experiencing difficulties so even if this works in a Lab, it's not a given it will work in the real world. For that I'd recommend cloning the vCenter and a Domain Controller into a test network bubble and injecting the vCenter 6.0 ISO and trying it on the actual vCenter install (Include SQL of course!).

First things first I need a Lab and I've decided on 16VMs for this lab as follows:

lab50sqla 2008R2 SQL 2008R2
lab50vca 2008R2 vCenter 5.0
lab50srma 2008R2 SRM 5.0
lab50esxia ESXi 5.0
lab50sqlb 2008R2 SQL 2008R2
lab50vcb 2008R2 vCenter 5.0
lab50srmb 2008R2 SRM 5.0
lab50esxib ESXi 5.0
lab50pscc 2012R2 PSC 6.0
lab50sqlc 2012R2 SQL 2014/2016
lab50vcc 2012R2 vCenter 6.0U2
lab50srmc 2012R2 SRM 6.1.1
lab50pscd 2012R2 PSC 6.0
lab50sqld 2012R2 SQL 2014/2016
lab50vcd 2012R2 vCenter 6.0U2
lab50srmd 2012R2 SRM 6.1.1



I've used a powershell script to deploy these:

new-vm -vmhost bdragon.lab.local -template "2008R2 Std Template" -Datastore Datastore1 -diskstorageformat thin -oscustomizationspec "server 2012" -name lab50sqla
new-vm -vmhost bdragon.lab.local -template "2008R2 Std Template" -Datastore Datastore1 -diskstorageformat thin -oscustomizationspec "server 2012" -name lab50vca
new-vm -vmhost bdragon.lab.local -template "2008R2 Std Template" -Datastore Datastore1 -diskstorageformat thin -oscustomizationspec "server 2012" -name lab50srma
new-vm -vmhost bdragon.lab.local -template "2008R2 Std Template" -Datastore Datastore1 -diskstorageformat thin -oscustomizationspec "server 2012" -name lab50sqlb
new-vm -vmhost bdragon.lab.local -template "2008R2 Std Template" -Datastore Datastore1 -diskstorageformat thin -oscustomizationspec "server 2012" -name lab50vcb
new-vm -vmhost bdragon.lab.local -template "2008R2 Std Template" -Datastore Datastore1 -diskstorageformat thin -oscustomizationspec "server 2012" -name lab50srmb
new-vm -vmhost bdragon.lab.local -template "2012R2 Std Template" -Datastore Datastore0 -diskstorageformat thin -oscustomizationspec "server 2012" -name lab50pscc
new-vm -vmhost bdragon.lab.local -template "2012R2 Std Template" -Datastore Datastore0 -diskstorageformat thin -oscustomizationspec "server 2012" -name lab50sqlc
new-vm -vmhost bdragon.lab.local -template "2012R2 Std Template" -Datastore Datastore0 -diskstorageformat thin -oscustomizationspec "server 2012" -name lab50vcc
new-vm -vmhost bdragon.lab.local -template "2012R2 Std Template" -Datastore Datastore0 -diskstorageformat thin -oscustomizationspec "server 2012" -name lab50srmc
new-vm -vmhost bdragon.lab.local -template "2012R2 Std Template" -Datastore Datastore0 -diskstorageformat thin -oscustomizationspec "server 2012" -name lab50pscd
new-vm -vmhost bdragon.lab.local -template "2012R2 Std Template" -Datastore Datastore0 -diskstorageformat thin -oscustomizationspec "server 2012" -name lab50sqld
new-vm -vmhost bdragon.lab.local -template "2012R2 Std Template" -Datastore Datastore0 -diskstorageformat thin -oscustomizationspec "server 2012" -name lab50vcd
new-vm -vmhost bdragon.lab.local -template "2012R2 Std Template" -Datastore Datastore0 -diskstorageformat thin -oscustomizationspec "server 2012" -name lab50srmd

Now, I've just to sit back and hope I don't run out of disk space!! I'll get around to using VMFork at some stage to make this faster!

The overall plan is to upgrade vCenter in place and point at the new PSC's (Same SSO Domain, Separate Sites). I'm using HPE Storevirtual iSCSI SANs to replicate real ones so I can implement SRM recovery plans and ensure they survive the transition. Once the upgrades are done I'll replatform onto the newer OS and see how I go. Snapshots and backups are going to be important as I'm expecting to hit snags and issues as I've found parts of this plan but not a complete end to end write up anywhere. 

Most of the virtual hardware will remain low but I'll beef up the vCenter and SQL a bit to 8GB Ram. 

One of the first things to do is to check the VMware Interoperability Guide. What we're initially looking for is if we can do an in place upgrade or if the Operating Systems and Database versions preclude this.
https://www.vmware.com/resources/compatibility/sim/interop_matrix.php#db
Start with the Database, we can see that vCenter 5.0 is compatible with SQL 2008, 2008 R2 and SQL 2012 but not any later SQL. We want to go to SQL 2014 or later so this is an issue. Now add in another version of vCenter, 6.0U2. This IS compatible with SQL 2008R2 once a service pack is applied (SP1/SP2). So this gives us a path forward, we can upgrade in place and THEN move the databases to SQL 2014 and we should be ok. Update Manager is much the same story.
Note: SQL 2016 is not mentioned at all.

So, we can also check interoperability between vCenter 6.0U2 and SRM 5.0 - not match of course! So, SRM will not function until it's also been upgraded and we should go to the latest version and SRA. Now, this is not as simple as it looks, the third section in the interoperability guide allows us to plug in SRM. As you can see from the chart below we can't go straight to 6.1, we have to go to 5.5.1, then 6.0 and THEN 6.1.1 !!! Now we should check database compatibility for all of these versions. So we're fine using SQL 2008 R2 (SP1 or SP2 ONLY) across all of these versions. SRM 6.0 & 6.1.1 support SQL 2014 too.


What about the Operating Systems? Well, Server 2008 R2 is 64-bit so that helps.

Build Notes: 
Open TCP Port 1433 in Windows Firewall on SQL Servers
You only need to install the x64 SQL Native Client on the vCenter Server VMs
Use C:\Windows\Syswow64\odbcad32.exe to create Update Manager 32-bit DSN (Ah, the good 'ol days!)
When installing Update Manager use the FQDN of vCenter to comply with CA Certificate requirements later (both during the connection page and port settings page)
In order to get the web interface to work use the following when replacing certificates to get the website unregistered/registered:
C:\Program Files\VMware\Infrastructure\vSphere Web Client\scripts
admin-cmd unregister https://lab50vca.lab.local:9443/vsphere-client lab50vca.lab.local lab\administrator
admin-cmd register https://lab50vca.lab.local:9443/vsphere-client lab50vca.lab.local lab\administrator

If you get Error #2046 see here and upgrade to vCenter 5.0 U3e:
There is a workaround but it needs to be reapplied after each reboot! 

Install SRM before the SRA
SRM uses a 32-bit ODBC DSN on this version (5.0)
.NET required for Storevirtual SRA on each SRM server to work at all
rename rui.pfx to rui.p12