I came across this at a clients site the other day,
2 MSCS clusters which were connected to SYMMETRIC storage were seeing 2000ms latency on the RDMs attached to the servers.
After looking at the issue it appeared to be set up correctly, looking at ESXTOP stats I was seeing all the time was being spent in the Kernel I was noticing upwards of 2000ms in KAVG, and with that there was queuing, then it would go and return.
This made me look at the PSP configuration for the RDMs, This was set to Round Robin which is not support for use in MSCS I am guessing for this very reason and makes sense when you think about it, as the virtual machine has a lock on the LUN when the path would switch the the queue would build up and this would be the time its waiting in the kernel for the path to switch back.
Please see THIS article from VMware for MSCS setup.
This had not been a problem prior to upgrading to ESXi 5.1, this is because any new devices added in 5.1 that are in an active/active configuration it automatically assigned to round robin by default. prior to 5.1 it would assign a fixed PSP for active/active which is what it should be in a MSCS setup.