Search Engine: Elastic

Article ID: 133076, created on Aug 23, 2018, last review on Dec 24, 2018

  • Applies to:
  • Operations Automation 7.4

Introduction

This instruction describes how to migrate subscriptions from the CI Hosting model (service controller based) to the CS Hosting model (APS-based).

Pre-requisites for CI Hosting migration

Make sure that the following requirements are met before the migration:

  1. Cloud Infrastructure Automation module, Cloud Infrastructure APS package and endpoint are installed. The minimum required Cloud Infrastructure package version is oaci-17.13-396.
  2. CS Hosting resource types are present in PCP. Create two resource types – CS Instance- AUTOPROVISION and Virtual Server- NO AUTOPROVISION – as described below. They will be used in CI service template for migration.
  3. The following services are up and running:
    • pa-service
    • pau
    • pba
    • PACI-im
  4. The Management Node can communicate with the Cloud Infrastructure Instance Manager and Billing Application Server.

Creating 'CS Instance- AUTOPROVISION' resource for migration

  1. Go to Application > Cloud Infrastructure > Resource Types > Create.
  2. Choose the Application Service type.
  3. Specify the name CS Instance- AUTOPROVISION (use this naming).
  4. Select service of application with the name Cloud Servers.
  5. Make sure that the auto-provisioning option is on.

Creating 'Virtual Server- NO AUTOPROVISION' resource for migration

  1. Go to Application > Cloud Infrastructure > Resource Types > Create.
  2. Choose the Application Service type.
  3. Specify the name Virtual Server- NO AUTOPROVISION (use this naming).
  4. Select service of application with the name Virtual Server.
  5. Make sure that the auto-provisioning option is off.

Overview of migration process

The migration process consists of the following steps:

  1. Uploading the package ci2cs (md5sum: ba064559356d772a9ff02b75d0d17c24 ci2cs-1.9.zip) with migration tools on the Management Node and edit the config.py file.
  2. Preparing the subs.example.json file with the list of subscriptions to be migrated.
  3. Preparing the resources.py file.
  4. Evaluation of CI subscriptions.
  5. Insertion of CS Hosting resources in the CI Hosting service template and subscriptions.
  6. Provisioning of CS Hosting resources.
  7. Insertion of CS Hosting resource rates in CI Hosting service plan.
  8. Final stage (synchronizing CI subscriptions with changes in the CI Hosting service plan).

Let's consider each step in details.

Step 1. Uploading the migration tools.

  1. Unpack the archive ci2cs-1.9.zip on the Management Node. This archive has scripts and files that are needed for migration:

    • config.py - contains configuration settings for connection to Billing Application server, Billing DB, and performance settings.
    • main.py - main script that performs evaluation, insertion and provisioning of CS resources, finalizes the migration process.
    • resources.py - this file is used by the main script on the stage when insertion of CS resources in subscriptions is performed. The list of resources in the file should be adjusted according to the list of CS resources in PCP. You should list only those resources that you want to be added to one CI Hosting service template. If you have several CI Hosting service templates, do not use the resources.py file for inserting CS resources in both of them simultaneously.
    • subs.example.json - this file contains the list of IDs of CI subscriptions that should be migrated to the CS Hosting model. Note that all the subscriptions should be based on the same CI service template.
  2. Open the config.py file and replace all the placeholders (inside the angle brackets) with your system data.

    #!/usr/bin/python
    # -*- coding: utf-8 -*-
    
    # Billing APP IP and Pass could be taken from /usr/local/bm/conf/_amt_service_.res 'RootPassword'
    billing_app_ip = '<IP Address of Billing Application Server>'
    billing_app_pass = '<Password>'
    
    # Billing DB IP and Pass could be taken from /usr/local/bm/etc/ssm.conf.d/global.conf
    billing_db_user = 'pba'
    billing_db_name = 'pba'
    billing_db_ip = '<Billing DB IP Address>'
    billing_db_pass = '<Billing DB password>'
    
    # Performance configuration
    openapi_threads = 5
    aps_api_threads = 5
    pba_api_threads = 5
    
    # Migration Database setting
    database = "./pypaci.db"
    
    # Days to consider subscription inactive
    days_of_inactivity = 90
    
    # Default batch size of subscriptions to be processed
    default_batch_size = 10
    

Note: If you plan to migrate a short list of subscriptions (< 100), it is recommended to set the performance parameters to 1 thread.

Step 2. Preparing the `subs.example.json` file.

The subs.example.json file has the list of subscription ids to be migrated. Preparing this file is not required, you can list all the subscriptions in the migration script explicitly. You can use this file if the list of subscriptions is long.

Important: Make sure that all the subscriptions listed in the file subs.example.json are based on the same CI service template. Do not mix subscriptions based on different CI service templates.

    {
        "subscriptions": [1000100, 1000101, 1000102, 1000103, 1000104, 1000105, 1000106, 1000107, 1000108]
    }

The subs.example.json file can be created automatically. In the folder with the migration tools, you can find the propose.py file that can generate the suggested subs.json files based on your CI subscriptions and CI Service Templates. The script searches for subscriptions in the paci_usage_timestamps table, which includes all subscriptions based on the service template with CI resources, and group them by the service template ID.

To generate the subs.example.json file, run the following command:

# python propose.py --prepare

As a result, you will get the files such as st_5.subs.json, st_6.subs.json. For example, the st_6.subs.json will look like the following:

# cat st_6.subs.json 
{
  "subscriptions": [
    4,
    5,
    6
  ]

Where the st_6 means that the file contains all CI subscriptions based on the Service Template #6.

Step 3. Preparing the `resources.py` file.

The resources.py file has a set of resources, their names and limits. Note that the list of CS resources in the file is dedicated for insertion in no more than one CI Hosting service template.

You can use the following example for reference:

    cs_resources = [
        {'name': u'DNS Hosting', 'resource_class_name': 'DNS Hosting', 'resource_id': 0, 'limit': -1},
        {'name': u'DNS Management', 'resource_class_name': 'DNS Management', 'resource_id': 0, 'limit': -1},
        {'name': u'OACI Location DEFAULT', 'resource_class_name': 'rc.saas.service.link', 'resource_id': 0, 'limit': 1},
        {'name': u'OACI Backup Schedule - Daily', 'resource_class_name': 'rc.saas.service.link', 'resource_id': 0, 'limit': -1},
        {'name': u'OACI Backup Schedule - Weekly', 'resource_class_name': 'rc.saas.service.link', 'resource_id': 0, 'limit': -1},
        {'name': u'OACI Backup Schedule - Monthly', 'resource_class_name': 'rc.saas.service.link', 'resource_id': 0, 'limit': -1},
        {'name': u'OACI Backup Schedule - Custom', 'resource_class_name': 'rc.saas.service', 'resource_id': 0, 'limit': -1},
        {'name': u'CS Application', 'resource_class_name': 'rc.saas.service.link', 'resource_id': 0, 'limit': 1},
        {'name': u'CS Virtual Server Template', 'resource_class_name': 'rc.saas.service.link', 'resource_id': 0, 'limit': -1},
        {'name': u'CS UI Policy', 'resource_class_name': 'rc.saas.service.link', 'resource_id': 0, 'limit': 1},
        {'name': u'CS Load Balancer', 'resource_class_name': 'rc.saas.service', 'resource_id': 0, 'limit': 10},
        {'name': u'CS Virtual Environment Image Limit', 'resource_class_name': 'rc.saas.service', 'resource_id': 0, 'limit': -1},
        {'name': u'CS Virtual Environment Backup Limit', 'resource_class_name': 'rc.saas.service', 'resource_id': 0, 'limit': -1},
        {'name': u'CS Public IPv4 Address', 'resource_class_name': 'rc.saas.service', 'resource_id': 0, 'limit': 10},
        {'name': u'CS Public IPv6 Address', 'resource_class_name': 'rc.saas.service', 'resource_id': 0, 'limit': 100},
        {'name': u'CS CPU Power Usage Running Container', 'resource_class_name': 'rc.saas.resource.mhzh', 'resource_id': 0, 'limit': -1},
        {'name': u'CS CPU Power Usage Stopped Container', 'resource_class_name': 'rc.saas.resource.mhzh', 'resource_id': 0, 'limit': -1},
        {'name': u'CS CPU Power Usage Running Virtual Machine', 'resource_class_name': 'rc.saas.resource.mhzh', 'resource_id': 0, 'limit': -1},
        {'name': u'CS CPU Power Usage Stopped Virtual Machine', 'resource_class_name': 'rc.saas.resource.mhzh', 'resource_id': 0, 'limit': -1},
        {'name': u'CS Inbound traffic', 'resource_class_name': 'rc.saas.resource.kba', 'resource_id': 0, 'limit': -1},
        {'name': u'CS Outbound traffic', 'resource_class_name': 'rc.saas.resource.kba', 'resource_id': 0, 'limit': -1},
        {'name': u'CS RAM Usage Running', 'resource_class_name': 'rc.saas.resource.mbh', 'resource_id': 0, 'limit': -1},
        {'name': u'CS RAM Usage Stopped', 'resource_class_name': 'rc.saas.resource.mbh', 'resource_id': 0, 'limit': -1},
        {'name': u'CS Local Diskspace Usage Running', 'resource_class_name': 'rc.saas.resource.mbh', 'resource_id': 0, 'limit': -1},
        {'name': u'CS Local Diskspace Usage Stopped', 'resource_class_name': 'rc.saas.resource.mbh', 'resource_id': 0, 'limit': -1},
        {'name': u'CS PCS Diskspace Running', 'resource_class_name': 'rc.saas.resource.mbh', 'resource_id': 0, 'limit': -1},
        {'name': u'CS PCS Diskspace Stopped', 'resource_class_name': 'rc.saas.resource.mbh', 'resource_id': 0, 'limit': -1},
        {'name': u'CS Bandwidth Running', 'resource_class_name': 'rc.saas.resource.kbpsh', 'resource_id': 0, 'limit': -1},
        {'name': u'CS Bandwidth Stopped', 'resource_class_name': 'rc.saas.resource.kbpsh', 'resource_id': 0, 'limit': -1},
        {'name': u'CS Image Space Usage', 'resource_class_name': 'rc.saas.resource.mbh', 'resource_id': 0, 'limit': -1},
        {'name': u'CS Backup Space Usage', 'resource_class_name': 'rc.saas.resource.mbh', 'resource_id': 0, 'limit': -1},
        {'name': u'Virtual Server- NO AUTOPROVISION', 'resource_class_name': 'rc.saas.service', 'resource_id': 0, 'limit': -1},
        {'name': u'CS Instance- AUTOPROVISION', 'resource_class_name': 'rc.saas.service', 'resource_id': 0, 'limit': 1},
    ]

Your list of resources can be different (you may have different 'Virtual Server Template' or you may not have 'DNS Management'), therefore adjust the list of resources in the file according to the set of CS resources in your PCP.

Important: Make sure that the resource 'CS Virtual Environment Limit' which will be used for migration (in our example it is called 'Virtual Server- NO AUTOPROVISION') has the activation parameter 'Automatically provision service' turned off.

Step 4. Evaluation of CI subscriptions.

During this step, you should run the script which performs the following:

  • Collects information about all CI subscriptions.
  • Prepares a virtual server template with defined limits and step. This template has the same parameters and presets as in the original CI Service Templates.
  • Prepares a set of CS resources to be inserted into live subscriptions (w\o provisioning).

Run the main.py script for evaluation. In the command parameters you can specify either the file with the list of subscriptions to be migrated or define explicitly the subscription ids.

    # python main.py --subscriptions_file subs.example.json --stage evaluation

OR

    # python main.py --subscriptions 1000005 1000006 1000007 --stage evaluation

    2018-07-11 10:27:28.129 [DEBUG] collecting information about CI subscriptions [1000005, 1000006, 1000007] and it's Service Template
    2018-07-11 10:27:30.010 [DEBUG] [evaluation] parsed service templates and virtual server templates
    2018-07-11 10:27:30.338 [DEBUG] [evaluation] parsed subscriptions

Then, check the status of subscriptions:

    # python main.py --status
    2018-07-11 10:28:17.589 [DEBUG] getting status of subscriptions and migration itself
    1000005 --> injected --> active
    1000006 --> evaluated --> was_active
    1000007 --> evaluated --> inactive

After evaluation is done, you can see the status of each subscription:

  • active status – currently have VEs in OACI
  • was_active status – currently no VEs in OACI but the subscription has usage for the past 3 months. The period during which the script checks usage is configurable in the config.py file by means of the days_of_inactivity option.
  • inactive status – no VEs in OACI at all.

Now you can choose which type of subscriptions to process by means of the --oaci option of the main.py script. By default, active subscriptions will be selected for processing.

See the example, how to use the script with the --oaci option:

    usage: main.py
                   [--oaci {inactive,was_active,active}]


    # python main.py  --subscriptions 1000005 1000006 1000007 --stage injection --oaci active
    2018-09-11 09:04:10.242 [DEBUG] Call openapi.OpenAPI method to get PACI settings
    2018-09-11 09:04:10.850 [DEBUG] injecting CS resources into subscriptions [1000005 1000006 1000007]
    active
    2018-09-11 09:04:10.850 [DEBUG] [injection] these of specified subscriptions are in 'evaluated' state and could be injected [1000005]

In the example, only the 100005 subscription has been chosen as appropriate for insertion of CS resources because only it has active OACI VEs.

After performing the evaluation stage, a new virtual server template is created. This template has the same parameters and presets as in the original CI Service Templates. It will have a name similar to 'Virtual Server Template For 5 Service Template'. In this name, 5 means the ID of CI Hosting service template for which a corresponding virtual server template was created. You can review it, adjust as needed and add to the resource.py file instead of the virtual server template defined previously.

Step 5. Insertion of CS Hosting resources in CI service template and subscriptions.

  1. In Operations PCP, deactivate the CI Hosting service template.
  2. Add CS Hosting resources to it:

    • DNS Hosting; limit = -1
    • DNS Management; limit = -1
    • OACI Location DEFAULT; limit = 1
    • OACI Backup Schedule - Daily; limit = -1
    • OACI Backup Schedule - Weekly; limit = -1
    • OACI Backup Schedule - Monthly; limit = -1
    • OACI Backup Schedule - Custom; limit = -1
    • CS Virtual Environment Limit; limit = -1
    • CS Load Balancer; limit = 10
    • CS Virtual Environment Image Limit; limit = -1
    • CS Virtual Environment Backup Limit; limit = -1
    • CS Public IPv4 Address; limit = 10
    • CS Public IPv6 Address; limit = 100
    • CS CPU Power Usage Running Container; limit = -1
    • CS CPU Power Usage Stopped Container; limit = -1
    • CS Inbound traffic; limit = -1
    • CS Application; limit = 1
    • CS Virtual Server Template; limit = -1
    • CS UI Policy; limit = 1
    • CS CPU Power Usage Running Virtual Machine; limit = -1
    • CS CPU Power Usage Stopped Virtual Machine; limit = -1
    • CS Outbound traffic; limit = -1
    • CS RAM Usage Running; limit = -1
    • CS RAM Usage Stopped limit; limit = -1
    • CS Local Diskspace Usage Running; limit = -1
    • CS Local Diskspace Usage Stopped; limit = -1
    • CS PCS Diskspace Running; limit = -1
    • CS PCS Diskspace Stopped; limit = -1
    • CS Bandwidth Running; limit = -1
    • CS Bandwidth Stopped; limit = -1
    • CS Image Space Usage; limit = -1
    • CS Backup Space Usage; limit = -1
    • CS Instance- AUTOPROVISION; limit = 1
    • Virtual Server- NO AUTOPROVISION; limit = -1
  3. Activate the CI Hosting service template.
  4. Insert CS resources in the specified CI subscriptions by running the main.py with the option --stage injection.

    # python main.py --subscriptions 1000053 1000054 --stage injection
    2018-07-11 10:50:22.676 [DEBUG] injecting CS resources into subscriptions [1000053, 1000054]
    2018-07-11 10:50:39.913 [DEBUG] [injection] injected resources into specified subscriptions
    
    # python main.py --status
    2018-07-11 10:50:42.027 [DEBUG] getting status of subscriptions and migration itself
    1000053 --> injected
    1000054 --> injected
    

As a result, you have CS resources in the specified CI subscriptions now. They are not active but can be provisioned now.

For more information about configuration of the CS Hosting resource model, refer to the Cloud Infrastructure Provider's Guide.

Step 6. Provisioning of CS Hosting resources.

Run the main.py with the option --stage provisioning.

    # python main.py --subscriptions 1000053 1000054 --stage provisioning
    2018-07-11 10:46:31.837 [DEBUG] provisioning CS APS resources for subscriptions [1000053, 1000054]
    2018-07-11 10:50:50.009 [DEBUG] [linking] legacy IM subscription 1000053 has been attached to CS model
    2018-07-11 10:50:51.969 [DEBUG] [linking] legacy IM subscription 1000054 has been attached to CS model

    # python main.py --status
    2018-07-11 10:59:31.119 [DEBUG] getting status of subscriptions and migration itself
    1000053 --> provisioned
    1000054 --> provisioned

As a result, you have provisioned CS resources for CI Hosting subscriptions. Your customer can manage the server in both the UX1 panel and CCP v1 (migration is not completed yet).

You can also revert the provisioning stage in case you need it.

    # python main.py --subscriptions 1000010 --stage rollback
    2018-08-22 13:30:16.590 [DEBUG] Call openapi.OpenAPI method to get PACI settings
    2018-08-22 13:30:16.827 [DEBUG] rolling back migrated subscriptions [1000010]
    2018-08-22 13:30:16.828 [DEBUG] [rollback] these of specified subscriptions are in 'provisioned' state and could be reverted back [1000010]
    2018-08-22 13:30:17.942 [DEBUG] [rollback] subscription 1000010 has been rolled back to CI model

After reverting, your subscriptions will return to the injected stage with no provisioned CS resources, no UX1 panel. The server will be active and manageable from old UI.

Step 7. Insertion of CS Hosting resource rates in service plan.

  1. In Billing PCP, find the CI Hosting service plan.
  2. Add CS Hosting resource rates to it and set prices. As a result, you get a service plan with both CI and CS Hosting resource rates while the respective existing subscriptions do not have CS resources.

If you have non-zero price for the 'CI Virtual environment limit' resource, you should set it to zero otherwise you will charge a customer twice for the virtual environment (price for 'CI Virtual environment limit' and price for 'CS Virtual environment limit'). Note that setting zero price for the 'CI Virtual environment limit' resource will cause no charge for it at the end of billing period because billing doesn't save price changes in the subscription. For example, setting price to 0 on the 10th day of a month will cause money loss for the resource for 10 days.

Step 8. Final Stage.

Run the main.py with the option --stage finalize. This action will turn off the old usage reporting, old UI, and the CI Hosting subscriptions with changes in the CI Hosting service plan.

    # python main.py --subscriptions 1000053 1000054 --stage finalize

    2018-07-11 11:01:43.956 [DEBUG] getting status of subscriptions and migration itself
    1000053 --> migrated
    1000054 --> migrated

Now your CI Hosting subscriptions are switched to the CS Hosting model (APS-based). Customers can now manage these subscriptions in UX1 panel.

Migration in One Run

Please use this migration approach only for staging testing.

If you want to migrate one or few subscriptions at a time, and you don't want to run every stage, there is a stage called 'fast'.

  1. Perform the manual steps (add CS resources in the service template, add and set up resource rates in the service plan).
  2. Run the main.py with the option --stage fast:

    # python main.py  --subscriptions 1002000 --stage fast
    2018-08-24 15:43:31.078 [DEBUG] [finalize] all old UI has been turned off for subscription 1002000
    2018-08-24 15:43:31.079 [DEBUG] [finalize] finished migration for subscriptions
    

Partial Switch to CS Hosting (no provisioning of CS resources)

You might have non-active CI subscriptions that do not need the CS resources and UX1 panel now. They also don't need CI Hosting UI (CCP v1) to be available. These subscriptions will need a possibility to be switched to the CS Hosting model in future.

In that case, there is a method of CI > CS Hosting migration during which the CS resources will be inserted into CI subscriptions, but they will not be provisioned, and CI usage reporting and CI UI will be turned off.

A provider will have a possibility to provide the CS Hosting to the customer at any time.

  1. Perform the manual steps (add CS resources in the service template, add and set up resource rates in the service plan).
  2. Run the main.py with the option --stage evaluation.

    # python main.py  --subscriptions 1002001 --stage evaluation
    2018-08-24 15:51:30.671 [DEBUG] [evaluation] parsed subscriptions [1002001]
    
  3. Run the main.py with the option --stage only-ui.

    # python main.py  --subscriptions 1002001 --stage only-ui
    2018-08-24 15:51:40.727 [DEBUG] [fast_turnoff_ui] finished migration for subscriptions
    

Migration in Batches

By default, migration will be performed with the batch size 100 because the parameter default_batch_size = 100 in the config.py file. But you can adjust the size of a batch by specifying the --batch option.

In the following example, four subscriptions are passed for processing to turn off the UI only, but the batch size is defined as 1. As a result, the list was reduced to only 1 subscription.

# python main.py  --subscriptions 1002001 1002002 1002003 1002004 --stage only-ui --oaci inactive --batch 1
2018-09-26 11:09:21.266 [DEBUG] Call openapi.OpenAPI method to get PACI settings
2018-09-26 11:09:21.432 [DEBUG] batch size of current migration is: 1, reducing list of subscriptions to that size
2018-09-26 11:09:22.648 [DEBUG] [evaluation] parsed service templates and virtual server templates
2018-09-26 11:09:22.839 [DEBUG] [evaluation] parsed subscriptions [1002001]

1002001 --> evaluated --> inactive --> only-ui

5356b422f65bdad1c3e9edca5d74a1ae caea8340e2d186a540518d08602aa065 55fe109b4b4fe3fbb893f22dbb85a41a 1941880841f714e458ae4dc3d9f3062d

Email subscription for changes to this article
Save as PDF