Unlock the VMware VM vmdk file
Kill -9 PID
- 1. Logon to the ESX host where the VM was last known to be running.
- 2. vmkfstools -D /vmfs/volumes/path/to/file to dump information on the file into /var/log/vmkernel
- 3. less /var/log/vmkernel and scroll to the bottom, you will see output like below:
- a. Nov 29 15:49:17 vm22 vmkernel: 2:00:15:18.435 cpu6:1038)FS3: 130: <START vmware-16.log>
- b. Nov 29 15:49:17 vm22 vmkernel: 2:00:15:18.435 cpu6:1038)Lock [type 10c00001 offset 30439424 v 21, hb offset 4154368
- c. Nov 29 15:49:17 vm22 vmkernel: gen 66493, mode 1, owner 46c60a7c-94813bcf-4273-0017a44c7727 mtime 8781867] ك Bold type added to number for emphasis.
- d. Nov 29 15:49:17 vm22 vmkernel: 2:00:15:18.435 cpu6:1038)Addr <4, 588, 7>, gen 20, links 1, type reg, flags 0x0, uid 0, gid 0, mode 644
- e. Nov 29 15:49:17 vm22 vmkernel: 2:00:15:18.435 cpu6:1038)len 23973, nb 1 tbz 0, zla 2, bs 65536
- f. Nov 29 15:49:17 vm22 vmkernel: 2:00:15:18.435 cpu6:1038)FS3: 132: <END vmware-16.log>
- 4. The owner of the lock is on line 3c, the last part is all you need, in this case 0017a44c7727
- 5. esxcfg-info | grep -i 'system uuid' | awk -F '-' '{print $NF}' will display the system uuid of the esx server. You need to run the esxcfg-info command on each esx server in the cluster to discover the owner.
- 6. When you find the ESX server that matches the uuid owner, logon to that ESX server and run the command: ps -elf|grep vmname where vmname is the problem vm. Example output below:
- a. 4 S root 7570 1 0 65 -10 - 435 schedu Nov27 ? 00:00:02 /usr/lib/vmware/bin/vmkload_app /usr/lib/vmware/bin/vmware-vmx -ssched.group=host/user/pool2 -@ pipe=/tmp/vmhsdaemon-0/vmxf7fb85ef5d8b3522;vm=f7fb85ef5d8b3522 /vmfs/volumes/470e25b6-37016b37-a2b3-001b78bedd4c/iu-lsps-vstest/iu-lsps-vstest.vmx0
- 7. Since there is a process running, pid 7570 in the example, you need to kill it by following steps 5-12 on stopping a VM above
- 8. Once the kill is complete the files should be released.
Thank you for your article.
ReplyDeleteCan you explain detailed process 5? How must run esxcfg-info command on each server? For example, in my esxi, esxdfg-info shows 2 servers: 9440c9ed81ca and 9110ba24c577. What is the command to run on 9440c9ed81ca and what info is the key.
Once I know what server matches uuid owner, how logon on ESX server?
Finally, where can find steps 5-12 to stop VM?