Recently after a network outage in my lab, all my machines were restarted. I started getting the service not available messages everywhere under the Infrastructure tab, when I brought my vCAC infrastructure online. I am documenting the steps I followed to fix this issue, you may jump to the final resolution part. I hope these troubleshooting steps would certainly help for other similar issues.
Issue: Any selection
under Infrastructure tab show error- HTTP Error 503. The service is unavailable.
Catalina.out log @
/var/log/vcac/ on vCAC Server shows below errors
2014-12-28
01:36:00,036 vcac: [component="cafe:component-registry"
priority="WARN" thread="mainTaskExecutor-48"
tenant=""]
com.vmware.vcac.platform.rest.RestTemplateIpv6.handleResponseError:581 - GET
request for "https://lab-vcac-iaas.ps.local/WAPI/api/status"
resulted in 503 (Service Unavailable); invoking error handler
2014-12-28
01:38:00,044 vcac: [component="cafe:component-registry"
priority="WARN" thread="mainTaskExecutor-44"
tenant=""]
com.vmware.vcac.platform.rest.RestTemplateIpv6.handleResponseError:581 - GET
request for "https://lab-vcac-iaas.ps.local/WAPI/api/status"
resulted in 503 (Service Unavailable); invoking error handler
2014-12-28
01:40:00,046 vcac: [component="cafe:component-registry"
priority="WARN" thread="mainTaskExecutor-35"
tenant=""]
com.vmware.vcac.platform.rest.RestTemplateIpv6.handleResponseError:581 - GET
request for "https://lab-vcac-iaas.ps.local/WAPI/api/status"
resulted in 503 (Service Unavailable); invoking error handler
2014-12-28
01:42:00,045 vcac: [component="cafe:component-registry"
priority="WARN" thread="mainTaskExecutor-31"
tenant=""]
com.vmware.vcac.platform.rest.RestTemplateIpv6.handleResponseError:581 - GET
request for "https://lab-vcac-iaas.ps.local/WAPI/api/status"
resulted in 503 (Service Unavailable); invoking error handler
2014-12-28
01:44:00,109 vcac: [component="cafe:component-registry"
priority="WARN" thread="mainTaskExecutor-50"
tenant=""]
com.vmware.vcac.platform.rest.RestTemplateIpv6.handleResponseError:581 - GET
request for "https://lab-vcac-iaas.ps.local/WAPI/api/status"
resulted in 503 (Service Unavailable); invoking error handler
2014-12-28
01:46:00,029 vcac: [component="cafe:component-registry"
priority="WARN" thread="mainTaskExecutor-38"
tenant=""]
com.vmware.vcac.platform.rest.RestTemplateIpv6.handleResponseError:581 - GET
request for "https://lab-vcac-iaas.ps.local/WAPI/api/status"
resulted in 503 (Service Unavailable); invoking error handler
2014-12-28
01:48:00,044 vcac: [component="cafe:component-registry"
priority="WARN" thread="mainTaskExecutor-47"
tenant=""]
com.vmware.vcac.platform.rest.RestTemplateIpv6.handleResponseError:581 - GET
request for "https://lab-vcac-iaas.ps.local/WAPI/api/status"
resulted in 503 (Service Unavailable); invoking error handler
Possible
Resolutions:
I am listing out the
steps I followed to fix this issue, maybe any of the below step might your
issue as well.
- Make sure all the vCAC services are up on IaaS Server by going to services.msc.
- Make sure the vcac server service is running on vCAC appliance. Confirm it by running
service
vcac-server status
If
it is stopped, then start or restart it by below command
service
vcac-server start
- Microsoft Hotfix for Server 2008 R2 which could help in case if vCenter server is throwing the above error (refer Vmware KB - kb.vmware.com/kb/2033822)
- Make sure all the services in IaaS are running with the same account or maybe local system account if appropriate.
Below
is the step which worked in my case
- Open - Internet Information Services (IIS) Manager and Select the Application Pool under the root node.
- Make sure the DefaultAppPool service is running. Try restarting if it is still running.
- If this doesn’t resolve the
issue then make sure these services associated with vCAC are running with
the same account as in services.msc
- RepositoryAppPool
- vCACAppPool
- WapiAppPool
In my case the above
3 services were stopped. After starting these services, everything was back to
normal.
