vSphere Upgrade 5.0 to 6.0 - Part 5
This post deals with deploying signed SSL certificates to vSphere 6.0 using a subordinate CA and the VMware VMCA. Ideally I'd like to use the new PSC UI in 6.0 U1 but there's scant resources on the exact steps and when I tried importing the generated Root Certificate it did nothing!
I've been trying various approaches for a while to get this to work and not having much luck! I've a dual datacenter setup simulated in my lab and an external PSC. The first two VMs I need to get this to work on are:
lab50psc.lab.local - this is the external Platform Services Controller
lab50vcc.lab.local - this is the VMware vCenter Server
The Certificate Tool VMware provides the following 8 options:
1. Replace Machine SSL certificate with Custom Certificate
2. Replace VMCA Root certificate with Custom Signing Certificate and replace all Certificates
3. Replace Machine SSL certificate with VMCA Certificate
4. Regenerate a new VMCA Root Certificate and replace all certificates
5. Replace Solution user certificates with Custom Certificate
6. Replace Solution user certificates with VMCA certificates
7. Revert last performed operation by re-publishing old certificates
8. Reset all Certificates
The best article to read first is this one that explains the 3 approaches:
Understanding and using vSphere 6.0 Certificate Manager (2097936)
https://kb.vmware.com/selfservice/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=2097936
So we have the following approaches:
- Regenerate all the default certificates - option 4 (Maybe they are about to expire etc)
- Use Delegated Certificates with the VMCA - option 2 (This one is for me which uses subordinate certificates and saves you a lot of trouble later!)
- Replace all Certificates with custom signed ones - option 5 (This is where you generate multiple custom certificates yourself using your Microsoft CA)
(Note: Update the path to your VMware installation to find the command below)
C:\Windows\system32>"E:\Program Files\VMware\vCenter Server\vmcad\certificate-manager.bat"
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
| |
| *** Welcome to the vSphere 6.0 Certificate Manager *** |
| |
| -- Select Operation -- |
| |
| 1. Replace Machine SSL certificate with Custom Certificate |
| |
| 2. Replace VMCA Root certificate with Custom Signing |
| Certificate and replace all Certificates |
| |
| 3. Replace Machine SSL certificate with VMCA Certificate |
| |
| 4. Regenerate a new VMCA Root Certificate and |
| replace all certificates |
| |
| 5. Replace Solution user certificates with |
| Custom Certificate |
| |
| 6. Replace Solution user certificates with VMCA certificates |
| |
| 7. Revert last performed operation by re-publishing old |
| certificates |
| |
| 8. Reset all Certificates |
|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|
Note : Use Ctrl-Z and hit Enter to exit.
Option[1 to 8]: 2
Do you wish to generate all certificates using configuration file : Option[Y/N] ? : y
Please provide valid SSO and VC priviledged user credential to perform certificate operations.
Enter username [Administrator@vsphere.local]:
Enter password:
Please configure MACHINE_SSL_CERT.cfg with proper values before proceeding to next step.
Press Enter key to skip optional parameters or use Default value.
Enter proper value for 'Country' [Default value : IE] :
Enter proper value for 'Name' [Default value : vmca-] : vmca-machine-ssl
Enter proper value for 'Organization' [Default value : Lab] :
Enter proper value for 'OrgUnit' [Default value : IT] :
Enter proper value for 'State' [Default value : Leinster] :
Enter proper value for 'Locality' [Default value : Dublin] :
Enter proper value for 'IPAddress' [optional] :
Enter proper value for 'Email' [Default value : administrator@lab.local] :
Enter proper value for 'Hostname' [Enter valid Fully Qualified Domain Name(FQDN), For Example : example.domain.com] : testvc.lab.local
Please configure machine.cfg with proper values before proceeding to next step.
Press Enter key to skip optional parameters or use Default value.
Enter proper value for 'Country' [Default value : IE] :
Enter proper value for 'Name' [Default value : vmca-] : vmca-machine
Enter proper value for 'Organization' [Default value : Lab] :
Enter proper value for 'OrgUnit' [Default value : IT] :
Enter proper value for 'State' [Default value : Leinster] :
Enter proper value for 'Locality' [Default value : Dublin] :
Enter proper value for 'IPAddress' [optional] :
Enter proper value for 'Email' [Default value : administrator@lab.local] :
Enter proper value for 'Hostname' [Enter valid Fully Qualified Domain Name(FQDN), For Example : example.domain.com] : testvc.lab.local
Please configure vsphere-webclient.cfg with proper values before proceeding to next step.
Press Enter key to skip optional parameters or use Default value.
Enter proper value for 'Country' [Default value : IE] :
Enter proper value for 'Name' [Default value : vmca-] : vmca-webclient
Enter proper value for 'Organization' [Default value : Lab] :
Enter proper value for 'OrgUnit' [Default value : IT] :
Enter proper value for 'State' [Default value : Leinster] :
Enter proper value for 'Locality' [Default value : Dublin] :
Enter proper value for 'IPAddress' [optional] :
Enter proper value for 'Email' [Default value : administrator@lab.local] :
Enter proper value for 'Hostname' [Enter valid Fully Qualified Domain Name(FQDN), For Example : example.domain.com] : testvc.lab.local
Please configure vpxd.cfg with proper values before proceeding to next step.
Press Enter key to skip optional parameters or use Default value.
Enter proper value for 'Country' [Default value : IE] :
Enter proper value for 'Name' [Default value : vmca-] : vmca-vpxd
Enter proper value for 'Organization' [Default value : Lab] :
Enter proper value for 'OrgUnit' [Default value : IT] :
Enter proper value for 'State' [Default value : Leinster] :
Enter proper value for 'Locality' [Default value : Dublin] :
Enter proper value for 'IPAddress' [optional] :
Enter proper value for 'Email' [Default value : administrator@lab.local] :
Enter proper value for 'Hostname' [Enter valid Fully Qualified Domain Name(FQDN), For Example : example.domain.com] : testvc.lab.local
Please configure vpxd-extension.cfg with proper values before proceeding to next step.
Press Enter key to skip optional parameters or use Default value.
Enter proper value for 'Country' [Default value : IE] :
Enter proper value for 'Name' [Default value : vmca-] : vmca-vpxd-ext
Enter proper value for 'Organization' [Default value : Lab] :
Enter proper value for 'OrgUnit' [Default value : IT] :
Enter proper value for 'State' [Default value : Leinster] :
Enter proper value for 'Locality' [Default value : Dublin] :
Enter proper value for 'IPAddress' [optional] :
Enter proper value for 'Email' [Default value : administrator@lab.local] :
Enter proper value for 'Hostname' [Enter valid Fully Qualified Domain Name(FQDN), For Example : example.domain.com] : testvc.lab.local
1. Generate Certificate Signing Request(s) and Key(s) for VMCA Root Signing certificate
2. Import custom certificate(s) and key(s) to replace existing VMCA Root Signing certificate
Option [1 or 2]: 1
Please provide a directory location to write the CSR(s) and PrivateKey(s) to:
Output directory path: c:\temp
Please configure certool.cfg with proper values before proceeding to next step.
Press Enter key to skip optional parameters or use Default value.
Enter proper value for 'Country' [Default value : IE] :
Enter proper value for 'Name' [Default value : vmca-] : vmca-certool
Enter proper value for 'Organization' [Default value : Lab] :
Enter proper value for 'OrgUnit' [Default value : IT] :
Enter proper value for 'State' [Default value : Leinster] :
Enter proper value for 'Locality' [Default value : Dublin] :
Enter proper value for 'IPAddress' [optional] :
Enter proper value for 'Email' [Default value : administrator@lab.local] :
Enter proper value for 'Hostname' [Enter valid Fully Qualified Domain Name(FQDN), For Example : example.domain.com] : testvc.lab.local
2016-07-29T14:25:45.410Z Running command: ['E:\\Program Files\\VMware\\vCenter Server\\vmcad\\certool.exe', '--genkey', '--privkey', 'c:\\temp\\vmca_issued_ke
y.key', '--pubkey', 'c:\\users\\admini~1.lab\\appdata\\local\\temp\\2\\pubkey.pub']
2016-07-29T14:25:45.712Z Done running command
2016-07-29T14:25:45.713Z Running command: ['E:\\Program Files\\VMware\\vCenter Server\\vmcad\\certool.exe', '--gencsr', '--privkey', 'c:\\temp\\vmca_issued_ke
y.key', '--pubkey', 'c:\\users\\admini~1.lab\\appdata\\local\\temp\\2\\pubkey.pub', '--config', 'E:\\ProgramData\\VMware\\vCenterServer\\runtime\\tmp\\vmware\\c
ertool.cfg', '--csrfile', 'c:\\temp\\vmca_issued_csr.csr']
2016-07-29T14:25:46.146Z Done running command
CSR generated at: c:\temp\vmca_issued_csr.csr
1. Continue to importing Custom certificate(s) and key(s) for VMCA Root Signing certificate
2. Exit certificate-manager
Now at this stage you use the CSR and the "vSphere 6.0 VMCA" CA Template (covered in later section below) to generate the required certificate. Download the cert and chain in base 64, open the chain and export the root ca cert in base 64. Next create a chain.cer file with the cert contents at the top and the root ca cert content below it with no spaces:
Chain.cer
-----BEGIN CERTIFICATE-----
MIIFnDCCBISgAwIBAgITKQAAAFhlR+fgdiITFAABAAAAWDANBgkqhkiG9w0BAQsF
-------------
blah.blah - this is the vmca-certool certificate using the vSphere 6.0 VMCA Template
-------------
W3a6NgBlkZpBWULSItO0SvSfzyZAKKPOfaYBQxf6APIUCxYGyGf0IFNxVjsuDfwk
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIDojCCAoqgAwIBAgIQFwovcSg587lHjAqGSoM7KTANBgkqhkiG9w0BAQsFADBD
------------
blah,blah - this is the root CA certificate exported from the p7b chain certificate
------------
T/EU8xHmMqi5bXmQvte3y0Z/joIjAg==
-----END CERTIFICATE-----
Then proceed. You will find these steps described in many of the KBs and other Blog articles and after doing them 30 times I got bored so I'm not covering the steps here!
Option [1 or 2]: 1
Please provide valid custom certificate for Root.
File : c:\temp\chain.cer
Please provide valid custom key for Root.
File : c:\temp\vmca_issued_key.key
You are going to replace Root Certificate with custom certificate and regenerate all other certificates
Continue operation : Option[Y/N] ? : y
Get site nameCompleted [Replacing Machine SSL Cert...]
default-first-site
Lookup all services
Get service default-first-site:f2adacfb-7533-4370-ac88-9189c89acdda
Update service default-first-site:f2adacfb-7533-4370-ac88-9189c89acdda; spec: c:\users\admini~1.lab\appdata\local\temp\2\svcspec_slmoug
---------------
Ditto
---------------
Updated 24 service(s)
Status : 100% Completed [All tasks completed successfully]
The steps above show a successful SSL cert replacement on an embedded vCenter installation, i.e. the Platform Services Controller and all vCenter services on a single VM and I used the postgres database to make life easier as this was just a lab.
VMware Update Manager
Update Manager is an extra step. You use Openssl to generate a CSR and the vSphere 6.0 template to issue it. Then create a PFX file and copy the crt/key/pfx file to the update manager ssl folder, stop the service, run the update manager utility and start the service again and that's it! The key thnig when installing Update Manager is to ALWAYS select the FQDN on the following 2 screens if you intend to use SSL:
Change the IP Address to a FQDN here:
E:\Program Files (x86)\VMware\Infrastructure\Update Manager\VMwareUpdateManagerUtility.exe
It outlines the steps and I used the following commands on my Openssl config to generate the CSR and PFX:
openssl req -new -nodes -out rui.csr -keyout rui.key -config openssl.cfg
openssl pkcs12 -export -in rui.crt -inkey rui.key -name rui -passout pass:testpassword -out rui.pfx
I used both domain names in the openssl.cfg file:
[alt_names]
DNS.1 = testvc.lab.local
DNS.2 = testvc
In the next section I'll describe the process of creating the two CA Templates.
CA Template
The first step is to configure two new Certificate Templates on your Microsoft CA server
Creating a Microsoft Certificate Authority Template for SSL certificate creation in vSphere 6.0 (2112009)
https://kb.vmware.com/selfservice/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=2112009
Firstly, I'm keen to ensure the certificates that will withstand scrutiny and avoid currently broken cyphers (one's now considered too weak for production). Ideally I'd like RSA with 2048 bits and SHA512 for good measure. Note - ideally you select this when setting up your CA so I checked mine, just view any of the certificates you've already issued and you can see the Signature Algorithm and Public Key fields:
There is a good blog post on changing the signing algorithm here:
https://blogs.technet.microsoft.com/askpfeplat/2015/03/15/sha-1-deprecation-and-changing-the-root-cas-hash-algorithm/
The two CA interfaces used are as follows:
Click Start > Run, type certtmpl.msc, and click OK
Click Start > Run, type certsrv.msc, and click OK
Here are the properties of the vSphere 6.0 Template used for Update Manager, Machine SSL and Solution User certificates (Duplicated from the default Web Server template):
Check the Extensions Tab and Application Policies should be blank, remove any listed inside it
Check the Key Usage matches the following:
The vSphere 6.0 VMCA Template for the subordinate CA should be copied from the Subordinate Certificate Authority Template and match the following:
Ensure it is published to AD:
Note: The Certificate issued later to the PSC will take 24 hours before ESXi Hosts can be joined successfully so time this carefully! You can generate the request and issue the Certificate but hold off applying it until 24 hours have passed if you want to reconnect your Hosts immediately.
Now we can publish these templates:
So we'll leave this post as is. The next step is to apply SSL certs to a distributed setup / external PSC which is a LOT of fun I can tell you! As they say, easy when you know how!!
