The task Upgrading resources of application 'UX1 Marketplace' fails with the error:
Chart with name 'ux1-marketplace' is in 'FAILED' state and cannot be updated automatically
The UX1 pod stuck in the Pending state:
[root@linmn01 ~]# kubectl get pod NAME READY STATUS RESTARTS AGE gdpr-backend-749f99895-fzqth 1/1 Running 1 176d gdpr-backend-859d498b5b-wr9fl 0/1 Pending 0 18d ux1-marketplace-connector-58964fdb66-nvwlf 0/1 Pending 0 1h ux1-marketplace-connector-7c847644b4-7t4kh 1/1 Running 0 18d ux1-marketplace-elastic-97cb69689-r72gh 1/1 Running 0 18d
The upgrade failed due to the lack of resources for the pod:
[root@linmn01 ~]# kubectl describe pod ux1-marketplace-connector-58964fdb66-nvwlf ... Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning FailedScheduling 2m (x398 over 1h) default-scheduler 0/1 nodes are available: 1 Insufficient cpu.
In order to fix the issue and complete the task the below steps have to be applied:
Add the required system resources to the Kubernetes node. Note that, during helm chart upgrade, additional resources are required, since Helm upgrades the chart by creating new pod (which requires resources), and only when a new pod is created, the old one is removed.
Get a list of the UX1 revisions:
[root@linmn01 ~]# helm history ux1-marketplace REVISION UPDATED STATUS CHART DESCRIPTION 1 Mon Nov 12 04:57:30 2018 SUPERSEDED ux1-marketplace-1.2.0-502 Install complete 2 Mon Nov 26 08:58:15 2018 SUPERSEDED ux1-marketplace-1.4.1-662 Upgrade complete 3 Fri Nov 30 01:04:32 2018 FAILED ux1-marketplace-1.4.1-662 Upgrade "ux1-marketplace" failed: timed out waiting for t...
Rollback the UX1 chart to previous revision as follows:
[root@linmn01 ~]# helm rollback ux1-marketplace 2
Restart the kubelet service on the kubernetes node:
[root@k8s ~]# systemctl restart kubelet
- Resubmit the task.