Symptoms
It is not possible to delete a node from CP even though this node is physically destroyed.
An error shown on the attempt to delete the node:
Host #101 is in deleting state.
There are other errors while navigating the tabs of the node:
- There is no such account #0.
- java.lang.RuntimeException: Unexpected query result: 0
- An unforeseen error occurred. Please contact no-reply@provider.com
The following can be observed in /var/log/poa.log on the Management Node:
Jul 15 10:20:30 server : DBG [UI:b911d62d:1792298:1468580620165 1:22497:a8dc0b70 chief 305920937]: [ Host_impl::getHost] ===> ENTRY
Jul 15 10:20:30 server : DBG [UI:b911d62d:1792298:1468580620165 RequestProcessor-35740 pau 305920937]: com.parallels.pa.service.db.ejb.DBServiceBean Executing non-locking query: ' SELECT h.host_id AS host_id, h.primary_name AS hostname, h.state AS host_state, CASE WHEN uh.host_id IS NULL THEN 'a' ELSE 'i' END AS sc_state, (SELECT COUNT(cs.host_id) FROM( SELECT s.host_id AS host_id FROM services s JOIN domain_services ds ON (s.service_id = ds.service_id) WHERE s.host_id = h.host_id UNION ALL SELECT lh.host_id AS host_id FROM hosts lh JOIN hosts hw ON (hw.hw_host_id = lh.host_id) WHERE lh.host_id = h.host_id ) AS cs GROUP BY cs.host_id ) AS cs_number, h.htype AS host_type, h.destination AS host_kind, h.provisioning AS provisioning_state, h.rt_instance_id AS resource_id, h.owner_id AS owner_id FROM hosts h LEFT JOIN
Jul 15 10:20:30 server : unreachable_hosts uh ON (h.host_id = uh.host_id) WHERE h.host_id = ?'. Parameters: [101] [sqlpid: 2611]
Jul 15 10:20:30 server : DBG [UI:b911d62d:1792298:1468580620165 RequestProcessor-35740 pau 305920937]: com.parallels.pa.corba.ServiceInvoker Service invocation for service DBService$$$view10 took 5 ms
Jul 15 10:20:30 server : DBG [UI:b911d62d:1792298:1468580620165 1:22497:a8dc0b70 chief 305920937]: [ Host_impl::getHost] <=== EXIT [0.006377]
Jul 15 10:20:30 server : DBG [UI:b911d62d:1792298:1468580620165 1:22497:a8dc0b70 lib 305920937]: [ HostManager_impl::onPrepareRemoveHost] {module_id="HostManager"; code="520"} Host #101 is in deleting state.
Jul 15 10:20:30 server : DBG [UI:b911d62d:1792298:1468580620165 1:22497:a8dc0b70 chief 305920937]: [ HostManager_impl::onPrepareRemoveHost] <=== EXIT (by exception) [0.010736]
Jul 15 10:20:30 server : ERR [UI:b911d62d:1792298:1468580620165 1:22497:a8dc0b70 lib 305920937]: [ SDK::PrepareRemoveHostObserver::onPrepareRemoveHost] ExSystem: module_id:'HostManager', ex_type_id:'520',Message:'Host #101 is in deleting state.', deprecated_codes = (0, 0), properties = { host_id: '101', }
Jul 15 10:20:30 server : ERR [UI:b911d62d:1792298:1468580620165 1:22497:a8dc0b70 lib 305920937]: [ SDK::PrepareRemoveHostObserver::onPrepareRemoveHost] failed to notify observer sc_id 10 [SDK::PrepareRemoveHostObserver] (CORBA::Exception)
Cause
Partially, the issue is caused by software related issue #POA-95947. There are two records in "hosts_to_cleanup" table which block each other.
Additionally, a task to delete the node could not be seen in CP because of "deleting", "is_physically_destroyed", and "probably_unreachable" statuses of the node in Operations Automation Database.
Resolution
Please contact Odin Technical Support in order to proceed with the node deletion.