Help. I'm out of ideas and have tried a lot.
Our custom certificates expired a couple of days ago. Attempted to use the VMware certificate apply tool to update these and have received failures on the first step of applying the SSO certificate.
I then went through the steps manually to identify where the issue was.
Essentially it has successfully applied the certificate to the SSO service, the root-trust.jks also applies successfully.
if I navigate to the browser location of https://<SSO.FQDN>:7444/lookupservice/sdk the certificate appears fine and is all "green in IE", correct dates and all happy.
The problem is stage 2.
Applying the cert to the various services. its failing at the first hurdle. the listServices command gives this.
C:\Program Files\VMware\Infrastructure\SSOServer\ssolscli>ssolscli.cmd listServices https://<SSO.FQDN>:7444/lookupservice/sdk
Intializing registration provider...
Getting SSL certificates for https://<SSO.FQDN>:7444/lookupservice/sdk
Return code is: OperationFailed
100
And that's all I get.
The C:\Program Files\VMware\Infrastructure\SSOServer\logs\lookuplog.log also indicates some nastiness.
[2014-09-30 12:59:11,137 DEBUG opID=c4dfd934-534e-41af-98c7-bda6af847146 pool-15-thread-1 com.vmware.vim.vmomi.server.impl.InvocationTask] Invoking com.vmware.vim.binding.lookup.ServiceInstance.retrieveServiceContent
[2014-09-30 12:59:11,137 DEBUG opID=c4dfd934-534e-41af-98c7-bda6af847146 pool-15-thread-1 com.vmware.vim.vmomi.server.impl.ActivationQueueCompletion] Result for com.vmware.vim.binding.lookup.ServiceInstance.retrieveServiceContent is success
[2014-09-30 12:59:11,141 DEBUG opID=e4708bb3-6b74-4be5-b89e-252ecaa553f4 pool-15-thread-1 com.vmware.vim.vmomi.server.impl.InvocationTask] Invoking com.vmware.vim.binding.lookup.LookupService.getViSite
[2014-09-30 12:59:11,141 DEBUG opID=e4708bb3-6b74-4be5-b89e-252ecaa553f4 pool-15-thread-1 com.vmware.vim.lookup.vlsi.util.VmodlEnhancer] Executing getViSite
[2014-09-30 12:59:11,141 DEBUG opID=e4708bb3-6b74-4be5-b89e-252ecaa553f4 pool-15-thread-1 com.vmware.vim.lookup.vlsi.util.VmodlEnhancer] Vmodl method getViSite return value is '{DE8E2DD0-C594-41A7-B3AE-0799B53016BA}'
[2014-09-30 12:59:11,141 DEBUG opID=e4708bb3-6b74-4be5-b89e-252ecaa553f4 pool-15-thread-1 com.vmware.vim.vmomi.server.impl.ActivationQueueCompletion] Result for com.vmware.vim.binding.lookup.LookupService.getViSite is success
[2014-09-30 12:59:11,144 DEBUG opID=d6fc41d5-fcb4-4c4e-a3c1-5c40184cf544 pool-15-thread-1 com.vmware.vim.vmomi.server.impl.InvocationTask] Invoking com.vmware.vim.binding.lookup.LookupService.find
[2014-09-30 12:59:11,144 DEBUG opID=d6fc41d5-fcb4-4c4e-a3c1-5c40184cf544 pool-15-thread-1 com.vmware.vim.lookup.vlsi.util.VmodlEnhancer] Executing find services(com.vmware.vim.binding.lookup.SearchCriteria:
serviceType = urn:sso:sts,
viSite = {DE8E2DD0-C594-41A7-B3AE-0799B53016BA},
endpointProtocol = null
inherited from com.vmware.vim.binding.lookup.SearchCriteria@7932e6fd)
[2014-09-30 12:59:11,145 DEBUG opID=d6fc41d5-fcb4-4c4e-a3c1-5c40184cf544 pool-15-thread-1 com.vmware.vim.lookup.impl.DbStorage] Executing SELECT LS_SERVICE.ID, LS_SERVICE.OWNER_ID, LS_SERVICE.VERSION, LS_SERVICE.DESCRIPTION, LS_SERVICE.FRIENDLY_NAME, LS_SERVICE.SERVICE_TYPE, LS_SERVICE.PRODUCT_ID, LS_SERVICE_ENDPOINT.URI, LS_SERVICE_ENDPOINT.SSL_TRUST_ANCHOR, LS_SERVICE_ENDPOINT.PROTOCOL, LS_SERVICE_ENDPOINT.SERVICE_ID FROM LS_SERVICE LS_SERVICE LEFT JOIN LS_SERVICE_ENDPOINT LS_SERVICE_ENDPOINT ON LS_SERVICE.ID = LS_SERVICE_ENDPOINT.SERVICE_ID WHERE 1=1 AND LS_SERVICE.SERVICE_TYPE = ?
[2014-09-30 12:59:11,145 ERROR opID=d6fc41d5-fcb4-4c4e-a3c1-5c40184cf544 pool-15-thread-1 com.vmware.vim.lookup.util.ValidateUtil] Invalid certificate
[2014-09-30 12:59:11,146 ERROR opID=d6fc41d5-fcb4-4c4e-a3c1-5c40184cf544 pool-15-thread-1 com.vmware.vim.lookup.vlsi.util.VmodlEnhancer] Failed to find services(com.vmware.vim.binding.lookup.SearchCriteria:
serviceType = urn:sso:sts,
viSite = {DE8E2DD0-C594-41A7-B3AE-0799B53016BA},
endpointProtocol = null
inherited from com.vmware.vim.binding.lookup.SearchCriteria@7932e6fd) because of Invalid certificate
java.lang.IllegalArgumentException: Invalid certificate
Anyone any idea what's going on here and why my lookupservice seems to have lost the plot? Obviously its detecting an invalid certificate, my guess is that its still using the old expired one. but how to update it as its basically just giving error 100 for anything that attempted on it.
I would much rather not go down the route of re-creating the vcenter. It utilises distributed switching and Nexus 1000v's so re-config would be a giant nightmare. The Nexus is already complaining it cant connect to the vcenter.