I didn't try this myself yet, so this is more a pointer. For the vCenter Server appliance it seems that adding an entry for vCenter Server itself to the /etc/hosts file allows to install SSO properly. Maybe this works for the Windows version too, (see vCenter Server Appliance 5.5 in a home lab, without SSO errors)
André