I would do it through the web client.
The syntax of the command you are proposing seems to be incorrect.
I was able to figure out what it should actually be like:
# esxcli system settings kernel set -s maxVMs -v 100
After reboot:
~ # uptime 08:51:59 up 00:02:48, load average: 0.01, 0.00, 0.00 ~ # esxcli system settings kernel list -o maxVMs Name Type Description Configured Runtime Default ------ ------ -------------------------------------------------------------------- ---------- ------- ------- maxVMs uint32 Max number of VMs VMKernel should support. 0 == determine at runtime 100 100 0