Thank you for the reply. This is the KB article I followed to get past the "directory not empty message". To clarify my issue:
After performing the steps in the kb article, I am no longer presented with the option to update my esxi host. I can only choose "Install". The custom iso is no longer seeing my existing installation and allowing me to choose update.