Unlocking Locked VM's in ESXi

So one of your virtual machines either won't lock or got locked in ESXi and you need to fix it quickly? Well VMware's KB can be pretty useful when it's direct and to the point. However I found it difficult to figure out after a bit of digging around in my ESXi system. So I thought I would clarify this for everyone here. Often times you'll probably see one of these errors:

  • A virtual machine cannot power on.
  • Powering on a virtual machine fails.
  • Unable to power on a virtual machine.
  • Adding an existing disk (VMDK) to a virtual machine that is already powered on fails with the error:
Failed to add disk scsi0:1. Failed to power on scsi0:1  
  • When Powering on the virtual machine you see one of these errors:

Unable to open Swap File
Unable to access a file since it is locked
Unable to access a file since it is locked
Unable to access Virtual machine configuration

  • In the /var/log/vmkernel log file, you see entries similar to:
WARNING: World: VM xxxx: xxx: Failed to open swap file <path>: Lock was not free  

WARNING: World: VM xxxx: xxx: Failed to initialize swap file <path>  
  • When opening a console to the virtual machine, you receive this error:
Error connecting to <path><virtual machine>, vmx because the VMX is not started  
  • Powering on the virtual machine results in the power on task remaining at 95% indefinitely.
  • Cannot power on the virtual machine after deploying it from a template
  • The virtual machine reports conflicting power states between VMware vCenter Server and the ESXi host console.
  • Attempting to view or open the .vmx file using the cat or vi command reports the error:
cat: can't open '[name of vm].vmx': Invalid argument  

Any one of the above error statements that is output by your ESXi server there is an easy fix for it: The virtual machine files are commonly locked for run time use (i.e. when the host is running the machine) are usually any of the files in the directory. The easiest way to identify the locked file is to attempt to power on the VM and you will see an error message that will give you the full path to the VM's directory plus the locked file. So the work around is as follows:

  1. Login to the ESXi host with SSH client (I use PuTTY).
  2. Confirm that the virtual machine is registered on the server and obtain the full path to the VM. Run the command:
# vmware-cmd -l
  1. The out put returns a list of VM's registered to the host server. Should look something like this:
[<datastore>] <VMDIR>/<VMNAME>.vmx
  1. You need to move to the VM's working directory:
# cd /vmfs/volumes/<datastore>/<VMDIR>
  1. You can use any linux text editor to view the vmware.log file, near the end it will have more verbose output to your error.
  2. The command is really easy to fix the whole unlock/lock issue for the whole directory:
touch *  

You should now be able to start up your VM without issues.

ESXi VM need to manually change MAC Address

Today, I had to deal with a FlexLM Licensing server issue which resulted from a VM which had it's NIC configured for the SAN's VLAN rather than for the production VLAN which caused the server to have some considerable amounts of downtime. I will talk about the setting up my SAN on a different VLAN from the rest of my production network in another post. In VMWare ESXi vSphere console for the VM host server which my guest VM was hosted on I attempted to change the MAC address of a specific NIC so the license server would work correctly. Well I was able to force this in ESXi in a previous version of ESXi 3.5 which allowed me to use any MAC address that I wanted to use. Well in ESXi 4.1 you can't do that anymore as it forces you to only modify the last 6 digits of the MAC address rather than the whole set. My work around evolved below:

As you notice in the above figure, the MAC Address is a random address given by my ESXi server. I need to bypass this and force a MAC Address from a NIC that I am not using anymore on an old server that I needed to move my FlexLM Licensing from. To accomplish this I went into the NIC's Advanced configuration in the driver settings:

This allowed me to set the MAC Address for the NIC and forces this address over the one set by ESXi.

NOTE: I have confirmed that this setup works with FlexLM on both Windows Server 2003 and Server 2008. I have not yet tested on Windows Server 2012.

Also make note that: You can never fire up that NIC ever again on your old server. Even if you plan to re-purpose it unless you are able to change it's MAC Address manually or install a new NIC as firing up that NIC will cause the above setup to fail miserably as you will have tons of trouble connecting to the new server from the old one.

P2V Conversion Failure Fixed

  We’ve been trying to convert a few Windows 2000 SP4 server to a VMWare machine for a while and running into the same issue.  I would always get a MODEEXCEPTIONNOT_HANDLED BSOD right after the login screen would appear.  I tried the normal fixes listed on the VMWare website including the right SP4 rollup 1 v2 and checking the versions of scsiport.sys.  Fortunately, I stumbled across this wonderful article: I set the virtual machine to a kernel mode dump and let it BSOD as recommended by the article.  I then copied to memory.dmp file to my Windows 7 workstation where I had installed the debugging tools referenced in the article.  I quickly identified usbsp.sys as the offending driver.  I renamed it on the virtual and rebooted.  Result?  Perfect!  I will definitely make sure I use this in the future to get a handle on blue screens when they pop up. Here is the relevant data from the process with the info highlighted in red.   How cool is that?

EXCEPTION_PARAMETER1:  00000000EXCEPTION_PARAMETER2:  00000000 ERROR_CODE: (NTSTATUS) 0 - STATUS_WAIT_0 BUGCHECK_STR:  0x1E_0 DEFAULT_BUCKET_ID:  DRIVER_FAULT PROCESS_NAME:  System LAST_CONTROL_TRANSFER:  from f2695af5 to 8042be0b STACK_TEXT: f245fc78 f2695af5 0000001e 00000001 804b1cd8 nt!KeBugCheck+0xf WARNING: Stack unwind information not available. Following frames may be wrong. f245fc90 804b1d5e 828696f0 82614000 f219fd08 usbsp+0xaf5 f245fd58 804b1f9f 0000008c 82614000 f219fd08 nt!IopLoadDriver+0x672 f245fd78 80417b47 f219fd08 00000000 00000000 nt!IopLoadUnloadDriver+0x3f f245fda8 80457838 f219fd08 00000000 00000000 nt!ExpWorkerThread+0xaf f245fddc 8046c8e6 80417a98 00000001 00000000 nt!PspSystemThreadStartup+0x54 00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16 STACK_COMMAND:  kb FOLLOWUP_IP: usbsp+af5 f2695af5 8d45f4          lea     eax,[ebp-0Ch] SYMBOL_STACK_INDEX:  1 SYMBOL_NAME:  usbsp+af5 FOLLOWUP_NAME:  MachineOwner MODULE_NAME: usbspIMAGE_NAME:  usbsp.sys DEBUG_FLR_IMAGE_TIMESTAMP:  3cc859dd FAILURE_BUCKET_ID:  0x1E_0_usbsp+af5 BUCKET_ID:  0x1E_0_usbsp+af5 Followup: MachineOwner