Symptoms

Office 365 "Provisioning Tenant for APS application" task is being rescheduled with the following output message:

Operation pending. Provisioning: resource xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx of type 'tenants' (http://www.parallels.com/Office365/Tenant/1.1) for APS application 'Microsoft Office 365-6.1-110': Provisioning is being executed by the application
Additional info: Waiting for subscriptions provisioning: SubscriptionId yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy (OfferId xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx)

Office 365 "Provisioning "AsyncOperation" for APS application Office 365" task is being rescheduled with the following output message:

Operation pending. Provisioning: resource xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx of type 'asyncOperations' (http://www.parallels.com/Office365/AsyncOperation/1.0) for APS application 'Microsoft Office 365-16.6-43': Provisioning is being executed by the application
Additional info: Configuring subscriptions is in progress. Provisioning the subscription in the Microsoft cloud  (Subscription Id is yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy, Offer Id is xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx).

Cause

  1. Caused by the software-related issue APSA-12923: "[O365] CSP: Subscription events are removed from the event queue prematurely".

O365 license stuck with In Provisioning state due to the fact, that subscription provisioned event was removed from the queue, but the corresponding subscription entity was not properly removed from the WaitSubscription table of MOSI DB.

  1. Caused by delay in provisioning. Lengthy waiting for callback from Microsoft Billing system. Normally it takes no longer than 48 hours, but exceptions are possible sometimes.

Resolution

If task has been running for more than 48 hours:

- for Syndication agreement:

For already existing tasks analyze the XMLPayload logs to check if subscription is provisioned on the MOP side and remove the corresponding record from the WaitSubscription table of the MOSI db.

- for CSP agreement:

Check the following entries in Wildfly server.log:

  1. Initial POST request and corresponding response (trace it by task-56 identifier)

    POST https://api.cp.microsoft.com/[Reseller Tenant Id]/orders
    
  2. GET request for subscription status and its response:

    GET https://api.cp.microsoft.com/[Reseller Tenant Id]/subscriptions/[MS subscription id]
    
  3. And GET request for subscription events:

    GET https://api.cp.microsoft.com/[Reseller Tenant Id]/subscription-streams/[Reseller Tenant Id]/Queue_[Queue Id]/pages
    
  4. In case items set is empty in the response below, there is no event from MS side:

    {"items":[],"object_type":"EventPage","contract_version":"1.1","links":{"completion":{"href":"/[Reseller Tenant Id]/subscription-streams/[Reseller Tenant Id]/Queue_[Queue Id]/pages?token=","method":"GET"}}}
    

If the task is running for more than 48 hours then please contact your Technical Account Manager or a member from Pooled Technical Associates pta@cloudblue.com to apply the internal workaround.

Internal content

Link on internal Article