Just to clarify...
You are trying to install Windows as a nested guest under ESXi as a guest under Windows, right?
At some point, when you started the ESXi guest, you should have seen the following message:
Nested virtual machines are not supported with binary translation for the outer virtual machine. If you want to run nested virtual machines under an ESX virtual machine, you need a host on which VMware Workstation supports Intel VT-x or AMD-V. Continue without nested virtual machine support?
You are no longer seeing this message, since you've checked the box indicating that you don't want to see it.
You should enable AMD-V in your physical machine's BIOS. Then, you will be able to run the ESXi guest with hardware-assisted virtualization, and that will allow you to run nested guests.