Thursday, January 15, 2015

vCAC Server Appliance 6.x upgrade fails, the Update / Install runs for long time and fails

In my previous post, I discussed how to come out of the checking for updates loop. Sometimes after 3-4 hours the appliance was able to detect the updates and again the installation of updates would take a very long time and fail after couple of hour. Good amount of space was available on all VM partitions.

Here is what I did to fix this issue,

 * Before jumping on to the solution, make sure you backup below files
  1.  /etc/vcac/
  2. /etc/vco/
  3. /etc/apache2/
  4. /etc/rabitmq/

 Important: Backup "ovf-env.xml" located at "/opt/vmware/etc/vami", in my case this file was empty after couple of reboots. This file is required while the upgrade process runs post-install scripts.

Don’t forget to take snapshot of VM before starting the upgrade!!!

Time(NTP) is synchronized on all machines.

Solution:

Uninstall the Vmware tools. As its an Appliance the procedure is different.

1. Take ssh connection to vCAC Server Appliance.

2. Make note of the vmware tools RPMs installed by running the below command

lab-vcac-server:~ # rpm -qa | grep -i vmware-tools

vmware-tools-libraries-nox-9.4.6-1.sles11
vmware-tools-core-9.4.6-1.sles11
vmware-tools-vmxnet-common-9.4.6-5.sles11
vmware-tools-vsock-kmp-default-9.5.6.0_3.0.76_0.11-5.sles11
vmware-tools-plugins-deployPkg-9.4.6-1.sles11
vmware-tools-plugins-powerOps-9.4.6-1.sles11
vmware-tools-vmxnet-kmp-trace-2.0.15.0_3.0.76_0.11-5.sles11
vmware-tools-guestlib-9.4.6-1.sles11
vmware-tools-services-9.4.6-1.sles11
vmware-tools-vmci-kmp-default-9.5.13.0_3.0.76_0.11-5.sles11
vmware-tools-vsock-common-9.4.6-5.sles11
vmware-tools-vmxnet-kmp-default-2.0.15.0_3.0.76_0.11-5.sles11
vmware-tools-plugins-autoUpgrade-9.4.6-1.sles11
vmware-tools-plugins-grabbitmqProxy-9.4.6-1.sles11
vmware-tools-plugins-hgfsServer-9.4.6-1.sles11
vmware-tools-plugins-timeSync-9.4.6-1.sles11
vmware-tools-plugins-vmbackup-9.4.6-1.sles11
vmware-tools-esx-nox-9.4.6-1.sles11
vmware-tools-esx-kmods-default-9.4.6-1.sles11
vmware-tools-foundation-9.4.6-1.sles11
vmware-tools-vmci-common-9.4.6-5.sles11
vmware-tools-vmci-kmp-trace-9.5.13.0_3.0.76_0.11-5.sles11
vmware-tools-plugins-guestInfo-9.4.6-1.sles11
vmware-tools-plugins-vix-9.4.6-1.sles11
vmware-tools-vsock-kmp-trace-9.5.6.0_3.0.76_0.11-5.sles11

3. Now run the below command to uninstall all vmware-tools RPMs
# zypper remove 'vmware-tools-*'

This command works for all SLES 11 editions.

4. Confirm all the RPM are gone by running the command rpm -qa | grep -i vmware-tools

This will uninstall the vmware tools.

5. Now go ahead and Run the "Check Updates" it would take 20-30 seconds and click on "Install Updates" which would take couple of minutes.

At the final stage of upgrade, the appliance runs the post-upgrade scripts which would look for "ovf-env.xml" which has vCAC Server Appliance's network information, vm description, SSH settings and vsphere port group information. Sometimes after reboot this file goes blank, if that happens it would again fail the upgrade. Make sure you backup this file.


After the successful upgrade, the vmware tools are installed automatically. I hope this post was informative.