Symptoms
Disable subscription task gets constantly rescheduled and then fails for an OACI subscription.
OACI IM log /var/log/pa/vps.log shows repeating attempts to stop the VE, every 2 seconds:
2017-08-31 00:00:40,107 (9ba1de69-9509-4051-a40a-faafd560c01f) DEBUG Pipeline [Shared executor thread #16 @1 @INTERACTIVE] - Submitting pipeline #23614 [(VE_STOP for: VeId [customerId=1010101, name=TEST]), created at node [im1], step 1 ([Ve Traffic Request]), mode: EXEC, state:SHARED_RUN, reqIs: null]
2017-08-31 00:00:42,131 (f2cadf37-56f1-4234-b1be-6417710a0a36) DEBUG Pipeline [Shared executor thread #17 @1 @INTERACTIVE] - Submitting pipeline #23615 [(VE_STOP for: VeId [customerId=1010101, name=TEST]), created at node [im1], step 1 ([Ve Traffic Request]), mode: EXEC, state:SHARED_RUN, reqIs: null]
2017-08-31 00:00:44,153 (48e1191b-b1bc-4e5c-bdc6-c9ec4b9254a4) DEBUG Pipeline [Shared executor thread #11 @1 @INTERACTIVE] - Submitting pipeline #23616 [(VE_STOP for: VeId [customerId=1010101, name=TEST]), created at node [im1], step 1 ([Ve Traffic Request]), mode: EXEC, state:SHARED_RUN, reqIs: null]
OA core log /var/log/pa/core.log shows the task as the origin of these requests:
2017-08-31T03:26:02.899280-05:00 core.pa.telmex.com : DBG [task:60109266:136 1:22497:f53feb70 PACI 1231622572]: [ PACI::PACI_impl::saveCallbackUrlToDB] serial 'method=PUT
uri_tail=/stop
ve_uri=http://192.00.00.1:4465/paci/v1.0/of/1010101/ve/TEST
'
2017-08-31T03:26:04.921009-05:00 core.pa.telmex.com : DBG [task:60109266:136 1:22497:f53feb70 PACI 1231622578]: [ PACI::PACI_impl::saveCallbackUrlToDB] serial 'method=PUT
uri_tail=/stop
ve_uri=http://192.00.00.1:4465/paci/v1.0/of/1010101/ve/TEST
'
The VE status history shows that it was in "duplicate detected" state at some point:
Aug-21-2017 13:08:21 Stopped
Aug-21-2017 13:08:02 Running
Aug-17-2017 19:34:20 Duplicate detected
Aug-17-2017 19:34:20 Unknown
Aug-17-2017 19:34:20 Running
Cause
VE got stuck in DUPLICATE_DETECTED state in the OACI backend, while the actual status in CP is valid. VE in DUPLICATE_DETECTED state cannot be stopped.
This is addressed as 2 product issues:
CCU-17100 - proper clean-up for the DUPLICATE_DETECTED state added
CCU-17096 - to rework the "Disable subscription" task and avoid sending repetitive STOP signals in order not to overload OACI IM service
Resolution
#CCU-17100 has been fixed in OA v 7.3. Please consider to upgrade your installation.
Contact your Technical Account Manager or Pooled Technical Associates Team (pta@odin.com) to trace the status of the CCU-17096 and discuss possible workaround.