Ok to install the .vib instead of a depot file it is a -v. Using your method jpd as far as removing the driver, good call!
esxcli software vib install -v http://10.x.x.x.x/ks/esx/VMware_bootbank_scsi-lpfc820_8.2.3.1-127vmw.510.0.0.799733.vib
Installation Result
Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
Reboot Required: true
VIBs Installed: VMware_bootbank_scsi-lpfc820_8.2.3.1-127vmw.510.0.0.799733
VIBs Removed:
VIBs Skipped: