Does anyone know if the two editions (Enterprise Plus/Enterprise) can co-exist somehow?
Yes, they can co-exist without problems, but some features that need Enterprise Plus edition like Distributed Switch need to be removed from other hosts if you want add this new host on the same cluster and make your environment consistent.
And take in mind that to add this new vSphere ESX 5.x to same vCenter that already manages the vSphere ESXi 4.x, you will need first upgrade your vCenter server to version 5.x.